Hallucinations aren't bugs but price of intelligence
The first time we catch a model lying, it feels like a defect. A crack in the machine.
That's actually comforting.
It's also false/
There is a darker colder truth, if you build a system to generalize from finite data into an unbounded world, perfect truthfulness is not a feature you can ship later, but its a property you cannot guarantee at all unless you accept a system that sometimes refuses to speak
not because engineers are lazy, but because math has teeth
and if you want to win everywhere, you will lose everywhere
There is no universal learner that dominates across all possible worlds, this paper makes the point bluntly. Without assumptions about the data generating process, performance guarantees evaporate.
A model might have bias, preferences about what kinds of patterns are likely, and bias means in some sense, it will be wrong.
so when people say "hallucinations will disappear at scale", what they mean is "we'll cover more of the world with better priors and better data
Training pipeline rewards guessing and not humility. Even when a question is answerable theres another structural pressure:we grade models like test takers
researchers at openai (allegedly so closed lol), describes hallucinations are something far less mystical than many assume in this paper
They are simply a errors in binary classification
and they make a key point many ignore
some real world questions are inherently unanswerable
so we come up with our own cap theorem for models
- do you want the models to be helpful (answer often)
- do you want the models to be fast (no accur verification)
- do you want the models to be perfect (never fabricate)
pick 2
so
0nly way models can avoid hallucination is by refusing to answer when uncertain
so the dark truth is a perfectly truthful model is often just a model that declines to speak outside verified ground
perfection is possible ... in silence