Bottom line: Hand an AI a story and it keeps the shape. It just swaps the unusual details for ordinary ones, because it’s rebuilding from the grammar, not the words. Our own memories pull the same trick, which is charming over dinner and expensive in a published report. Keep reading for the one word that flipped in a project of mine, and the unglamorous guardrails that stop it.
The mechanics are simple. Whenever a model fills a gap, it reaches for the most expected detail, so the unusual facts are the first to go. That happens when AI shortens a long source, and again when it expands a few notes into a full post. The remedy is boring: pin the odd details down, carry them word for word, and trace every claim back to the original.
The difference of one word
I’ve been working with XPRIZE, a foundation that runs massive innovation competitions to benefit humanity, to comb their archives for stories demonstrating how they’ve changed people’s lives.
I trained an AI agent to mine their digital sources for untold stories, and then return the key elements of stories that could indicate to a content producer whether they were worth telling.
One story was about a group of students at West Philadelphia High School who entered a $10 million competition against massive car companies and universities, to build a production-ready car that runs on very little fuel. The odds were against them. The students were working on it as part of an after-school program, led by their high school teacher, and they funded their car from scratch.
The process was basically this:
- Find the specific details that make a good story, like the ingredients of a recipe - flour, water, salt.
- Stitch them together in a short cohesive summary - a simple dough.
- Later, a content producer could decide what type of bread to make.
So we have a sampling of the story elements, and then a stitching together of the elements.
In that process of finding and summarizing the high school students’ story, the agent turned, “funded their car from scratch” to “built their car from scratch.”
From funded . . . to built. That’s a very big difference. What was odd about it is that it pulled the accurate detail in the sampling, and then immediately swapped it for a different inaccurate detail in the stitching-together part.
Both versions sound right if you’ve never heard the story before. Readers wouldn’t think twice about ‘built the car from scratch.’ But the two words say different things. One is the truth, the other isn’t. The agent kept the shape of the story but swapped out a key detail, robbing the story of credibility in the process.
By the way, AI agents make the same kind of mistakes whether compressing or expanding text. Wherever an agent doesn’t lock a specific detail down, it tends to supply the most likely version.
Sometimes the most likely version happens to be true. Sometimes it doesn’t.
I wanted to understand why the agent swapped a false detail immediately after finding a true one. A recent research paper about nonsense words gave me a better answer than I expected, because it tests word prediction in its most extreme form.
The jabberwocky test
The paper is by Gary Lupyan, a psychologist at the University of Wisconsin–Madison, and Blaise Agüera y Arcas of Google. It was posted in early 2026. (Source below)
Their method is simple. Take a passage of ordinary English. Replace most of the content words, the nouns, verbs and adjectives that carry the meaning, with made-up words. Keep everything else: the word order, the small grammar words like “the,” “of” and “but,” and endings like “-ed” and “-ing.” They call the result “Jabberwockified” text, after Lewis Carroll’s nonsense poem.
Then they ask an AI model to translate it back into English. That’s prediction pushed to the limit: the model has to supply nearly all the content itself.
To rule out memory, they used text the model could not have seen before. One was an ESPN story about an NFL trade, tested the day it was posted. The nonsense version read, in part, “he dwoiseed 88 plonges for 884 spelchs.” The model returned “he had 88 receptions for 884 yards.” What it didn’t get right was the teams.
How it fills the gaps
The authors explain the result with an idea from linguistics called construction grammar. In plain terms, it says a language is a huge collection of patterns. Some are fixed phrases, like “kick the bucket.” Some are loose templates, like “someone gave someone something.”
Each pattern on its own tells you very little. “He dwushed a ghanc zawk” tells you a man did something to an object, and nothing more. But patterns stack. Each one rules out some meanings. Put enough of them together and only a few meanings are left.
The grammar words and word order act like a fingerprint. They tell the reader what kind of text this is before any real content shows up. In the ESPN example, the numbers 88 and 884 were left in. Next to words in the slots where “receptions” and “yards” would sit, they pointed to football. Football then pulled in team names and contract terms.
I tried it myself. Can you translate this?
One frelk was about a splosh of quentals at Vindor Glimbous Binty Gark who crombied a $10 million zindle against frumious vark bostles and crambools, to snargle a plonkish gribbly vark that flirks on very squimmy lorp. The trebals were against them. The quentals were blurging on it as frip of an after-gark trid, glonked by their binty gark zorper, and they vringled their vark from snoot.
Here’s how Claude Sonnet 5.5 translated my thoroughly jabberwockied paragraph above:
Not bad. You probably recognize it already. Now here’s what happened when I gave it the “car” clue.
You can see the highlighted paragraph above for the original. It’s close-ish. But it’s also wrong in a way that would be embarrassing if published.
(Note: I did all of these jabberwocky in fresh sessions, not tied to any memory files)
The errors all point one way
The authors focus on how much the models get right. I found the mistakes more interesting in my example.
None of these swaps looks random. Each one moves toward a more familiar version of the same story. Though I have to admit, ‘fierce car makers’ was pretty interesting. I guess the model pitted them as the antagonist to the high school team.
But you can see the rebuild has a direction. When a pattern leaves a gap, the model fills it with the most expected detail. The structure comes back intact. The particulars come back as the typical case.
The authors don’t make this claim, and I haven’t measured it beyond the example above (yet). It’s my reading of their examples along with mine.
People did this first
None of this is unique to AI. In 1932, the British psychologist Frederic Bartlett published a book called Remembering. In his best-known study, he gave participants, mostly Cambridge students, a Native American folk tale called “The War of the Ghosts.” They read it, then wrote it down from memory. In another version, each person retold the story to the next, like a game of telephone.
The stories held their overall shape. The unfamiliar details did not. Canoes became boats. Hunting seals became fishing. The ghosts, which didn’t fit how the readers saw the world, often dropped out.
Bartlett concluded that memory is a rebuild. We keep the outline, then fill it in with what we already expect. Psychologists call those expectations schemas: mental templates of how things usually go.
That’s the same pattern the AI shows, in both directions. When Bartlett’s readers dropped the ghosts, they were compressing. When they turned seal hunting into fishing, they were filling a gap with what they expected. Either way, the outline survived and the unusual details didn’t.
The telephone version is the one to watch. Each person started from the last retelling, so the changes added up.
History works the same way. “Let them eat cake” first appears in Rousseau’s Confessions, written around 1765, where an unnamed “great princess,” told that peasants had no bread, replies that they should eat brioche. Marie Antoinette was a child in Austria at the time and almost certainly never said it. But later writers pinned the line on the queen who fit the story best, brioche became cake, and biographies, textbooks, and popular histories repeated the attribution until it became the most famous thing she never said. A small shift, copied often enough, becomes the record.
Four ways a retelling drifts
On the XPRIZE project, the agent pulled the snippets from each transcript into a short but cohesive narrative. A second AI pass checked each draft against the source. Then a person read them, and still found errors the AI check had missed.
The errors fell into four groups across the various stories I examined. Two are what you’d expect from shortening, one from filling out, and one shows up either way.
Each of these errors reads more smoothly than the source. A missed cut becomes a missed prize because “missed the prize” is the more common story. A chain of facts becomes a cause because stories usually explain themselves. Funded becomes built because that’s what car teams usually do.
Shortening drops the detail that’s least expected. Filling out supplies the detail that’s most expected. Both land in the same place: the shape of the story stays, and the particulars turn into the typical version.
Every company is a publisher now
Most companies publish constantly: blog posts, case studies, press releases, customer stories, social posts. Few of them think of themselves as news outlets. Their words still go into the same public record, and readers treat them as sources.
More and more of that writing starts as AI expansion. Someone hands the model a brief, a few notes or a transcript and asks for a full post. Every sentence the model adds beyond the source comes from patterns. That’s where funded turns into built.
Once it’s published, the drifted version becomes a source in its own right. A reporter quotes it. A partner repeats it. Another AI summarizes it, and that summary gets expanded into something new. Each step starts from the last version, the way Bartlett’s chain did, and nobody goes back to the original.
The usual safeguard is a person reading the draft. That helps, but the reviewer has the same pull toward the typical version, so a swapped detail that sounds right gets through. On the XPRIZE project, the errors we caught were the ones someone checked against the source.
What holds
The nonsense experiment also shows what stops the drift.
When I gave the agent the car clue it was able to do a much better job filling in the gaps. It pinned the passage to a story about cars and filled in the gaps from there.
Think of the specific details as anchors. An anchor is a specific, checkable detail the pattern couldn’t have predicted. Because the model can’t guess it, it doesn’t get replaced by a guess, and the rest of the meaning settles around it.
That points to the guardrails. They’re simple, and they apply whether AI is shortening a source or filling one out.
The unusual details need to travel with the work, word for word. If the source says “funded,” the notes say “funded,” the brief says “funded,” and the draft gets checked for “funded.”
Every claim traces back to the original. On the XPRIZE project, we put in guardrails so anyone checking a story had to go back to the source transcript, never to a summary and never to another AI’s check.
That’s the type of rigor and governance that companies and publishers need to embrace.
I help leaders apply journalism-style process standards to AI-assisted storytelling: finding the stories your organization already has but never captured, and sourcing every claim so it can be checked before it ships. If your team publishes with AI and needs to show how it checks its work, let’s connect.
Sources
- Lupyan, G., and Agüera y Arcas, B. “The Unreasonable Effectiveness of Pattern Matching.” arXiv preprint, 2026. arXiv:2601.11432.
- Bartlett, F. C. Remembering: A Study in Experimental and Social Psychology. Cambridge University Press, 1932. Details checked against Bartlett’s 1932 text on the Internet Archive and his 1920 paper, “Some Experiments on the Reproduction of Folk-Stories”.
- Marie Antoinette example: the phrase appears in Jean-Jacques Rousseau’s Confessions, Book 6, attributed to an unnamed “great princess” and written when Marie Antoinette was still a child in Austria. No contemporary source attributes the line to her.
