AI Ran Out of Internet
The clean human data is gone. Now the models are drinking their own exhaust, and you can taste it.
Here is the sentence nobody puts on the keynote slide. The good data is almost gone. Every leap of the last five years came from one boring fact: there was an enormous pile of human writing sitting on the open web, free for the scraping, and the labs shoveled all of it into the furnace. Books, forums, code, decades of Wikipedia edits, every blog anyone ever bothered to write. That pile was the fuel. And a finite pile burns.
Epoch AI, a research group that actually counts this stuff, projected that the stock of high-quality public text could be effectively used up somewhere in the late 2020s at current training appetites. Read that again. Not a hundred years out. Not some hazy singularity. A couple of model generations from now, on a timeline you can circle on a calendar. The entire strategy was "make it bigger." Bigger needs more. More is what's running out.
So what do you feed the machine when you've already fed it the internet? The industry's answer is synthetic data. You have the model write the training material for the next model. On paper it sounds elegant, an infinite fountain, data that never runs dry. In practice it's a photocopier making copies of copies of copies. Each pass loses a little. The rare stuff at the edges, the weird human turns of phrase, the minority opinions, the facts that only show up once, they thin out first and then they vanish.
Feed a model its own output long enough and it forgets the parts of the world it saw least. That is not a bug you patch. That is the physics of the loop.
This has a name now. Researchers at Rice called it Model Autophagy Disorder, MAD, which is about the most honest acronym the field has produced. A group publishing in Nature last year showed it plainly: train models on the output of earlier models across generations and the whole thing degrades into sludge, converging on a bland average and dropping the tails of the distribution entirely. The variety collapses. The model gets confidently, smoothly wrong, and it stops even knowing what it lost.
And here's the part that turns a research curiosity into a real problem. The open web is now full of AI text. Every content farm, every SEO mill, every LinkedIn thought-leader running a prompt on autopilot is dumping machine writing into the exact pool the next model will scrape. The labs didn't choose to train on their own exhaust. They're going to do it anyway, because the exhaust is now indistinguishable from the water. They poisoned the well and then got thirsty.
You already taste it, even if you can't name it. That flat, agreeable, weirdly interchangeable tone that a lot of AI writing has drifted into? That's the average winning. When the training signal gets diluted with a million polite, hedged, competent-sounding paragraphs that all say roughly the same thing, the model learns to be the middle of that. The edges are where the interesting was. The edges are what's disappearing.
Now the objection. "Bigger, cleaner curation will fix it. Better filters, licensed data, humans in the loop." Maybe, at the margins. But notice what that admits. The free lunch is over. The next round of gains doesn't come from a bigger scrape, it comes from paying for scarce, verified, genuinely human data, which is slow and expensive and does not compound the way "download everything" did. The whole miracle economics of the last five years assumed the fuel was free and infinite. Neither was true. It just took a while for the tank to read empty.
Who benefits from you not knowing this? The same people who need the growth story to hold for one more funding round. "We're a few scaling steps from everything" is a much better pitch than "our core input is a depleting resource and our own product is contaminating the supply." Both can't be true. Watch which one they say on stage and which one shows up quietly in the research papers.
I'm not saying the models stop working tomorrow. They're useful today and they'll be useful next year. I'm saying the specific trick that made them feel like magic, pour in more data and watch it get smarter almost for free, is hitting a wall made of arithmetic. There was only ever one internet. We already fed it to the thing. What comes next has to be earned, not scraped, and earning is a different business than the one everybody bought stock in.
The machine ate the whole library. Now it's chewing on its own reflection and calling it a meal.