Why the next real breakthrough in AI isn’t a bigger brain — it’s a machine that can admit ignorance.
A reader caught me out.
Last column I argued that the great AI buildout — the hundreds of billions pouring into data centers and the GPUs that fill them — is aimed at the wrong layer. We are spending as if the bottleneck were the size of the model’s brain, when the real bottleneck is getting the right information in front of it. Cheap retrieval, I said, not expensive cognition.
A reader replied, pointing out the name Jevons.
In 1865, a young English economist named William Stanley Jevons noticed something strange about coal. As steam engines got more efficient — as they wrung more work out of every lump — Britain did not burn less coal. It burned more. Efficiency made steam power cheaper, cheaper made it worth using everywhere, and “everywhere” swamped the savings many times over. The better we got at not wasting the stuff, the more of it we wanted.
The reader’s point was simple and, annoyingly, correct. Even if I’m right that retrieval is cheap and the brains are overbuilt, that won’t shrink the GPU bill. Make AI cheaper to run and we will simply run more of it. Demand eats the savings. The buildout survives. Jevons always wins.
He’s right. I concede the whole thing.
And I want to thank him, because in correcting me he handed me a better column.
Here is what I should have said the first time. The case for what comes next in AI was never really about cost. Cost is a weak argument; cost gets competed away, and Jevons makes sure of it. The argument that does not get competed away — the one still standing after the dust settles — is honesty.
There is exactly one problem in artificial intelligence that no amount of cheaper compute, and no amount of bigger compute, has ever solved or can solve by getting cheaper or bigger: the machine does not know what it does not know.
Ask today’s best models a question they cannot answer, and they will not pause. They will not hedge. They will hand you a fluent, confident, beautifully formatted answer that happens to be wrong, and they will deliver it with precisely the same swagger they bring to the answers that are right. We have taught them to sound certain. We have not taught them to be calibrated. And you cannot Jevons your way out of that. Make a confident liar a thousand times cheaper and you have fixed nothing — you have a thousand times more confident lying.
In a consumer toy, this is a parlor trick gone wrong. The chatbot invents a court case, the lawyer who trusted it gets sanctioned, everyone has a good laugh, life goes on. In an enterprise, it is the whole reason the technology keeps stalling at the door.
I have watched this movie up close. A bank, a hospital, an insurer, a law firm — they do not deploy a system that is confidently wrong five percent of the time. They can’t. Five percent confidently wrong, in a contract or a diagnosis or a compliance filing, is not a rounding error. It is a lawsuit, a recall, a fine, a firing. So the pilot dazzles everyone in the demo and then dies quietly in procurement, and the executives go back to muttering that AI “isn’t ready” — and they are right, but not for the reason they think.
The thing standing between AI and the enterprise was never speed and was never price. It is trust. And trust is not a mood; it is a property. It requires the machine to know the boundary of its own knowledge and to tell you, out loud, when you have walked past it.
Twenty-four hundred years ago the smartest man in Athens built a whole philosophy on four words: I know that I know nothing. Socrates’ entire edge was that he knew the edge — he could feel where his competence ran out. That, not raw recall, is what we actually mean when we call someone an expert. The junior analyst answers every question. The senior one says, “I’d have to check.” We trust the second one more, and we are right to.
We have built, at staggering expense, the AI junior analyst. Confident everywhere. Calibrated nowhere. The breakthrough that matters — the one I would put real money on — is not a model that knows more. It is a model that knows when it doesn’t, and has the nerve to say so.
And here is why this argument, unlike my last one, is bulletproof. Efficiency is a commodity; it falls in price until it is nearly free, and Jevons drags the demand along behind it. But knowing what you don’t know is not an efficiency. It is a capability. It either lives in the system or it doesn’t. You cannot out-cheap your way to it, which means no one can Jevons their way past it. The moment a buyer can choose between an AI that fabricates and one that flags its own ignorance, there is no contest — and no price war that changes the outcome. Honesty does not get absorbed by demand. It gets demanded.
Can such a thing actually be built — a system that checks itself against what it genuinely knows and raises its hand when it has wandered outside that — or is “I don’t know” forever beyond a machine that is, at bottom, an engine for guessing the next plausible word? I think it can be built. I think the architecture for it looks nothing like the brain-in-a-bigger-jar we have been financing. (Full disclosure: I have co-founded a company, 2Brains, built around exactly this problem, so discount my optimism by whatever margin you judge fair.) But the how is a column for next time.
For now I will leave you with the reader who set me straight. He read my argument, found the spot where it didn’t hold, and said so plainly. He knew the edge of what I had proven, and he had the nerve to name it.
That is the whole trick.
The machines should be so lucky.
Indeed, Bob. Humans have a real knack for finding ways to exploit resources. Is there a traffic problem in your city? Simple, build a bigger/wider freeway. That will solve it. Except—since there’s more road, now more people will just use it. They’ll use it until the growth in new traffic takes it to the same nuisance that existed to begin with.
I had the same gut feeling when reading about Shimizu Corporation’s Luna Ring project, a ring of solar panels around the moon designed to supply all the electricity the earth needs. Oh, I feel we’ll find a way to use it. Use it and then some.
So good to see you back in the saddle! I’ve been reading you since you were driving Pammy around in that Studebaker from the old InfoWorld days.
I have questions: Twenty-four hundred years ago the smartest man in Athens built a whole philosophy on four words: I know that I know nothing.
The Dunning-Kruger effect (the less you know the more confident you are). Funny the silicon has the same problems as the synapse version.
Hi Bob,
On 29 May your wrote about “…a man in Manassas who had lived in the same house for nearly forty years and opened an electricity bill for $281 — roughly triple what he’d paid the month before.”
What you didn’t mention — because is wasn’t germane to the point you were making — is that the man’s price increase was entirely due to the failure of governments to properly regulate “public utilities”. In many parts of the USA, governments have “de-regulated public utilities”. This means that some essentials of life are now subject to so-called “market forces”.
Where a commodity is essential to ordinary life, it is an “essential service” and by definition cannot be left to market forces. The air we breathe (and other essentials) cannot be subject to capital market pricing. Such essential elements must be regulated to ensure that everyone has access.
God damn welcome back.
You wrote, “…built a whole philosophy on four words: I know that I know nothing.”
AIs are known to be bad at counting, did you use a chatbot that couldn’t count those six(6) words?
Second comment, does Jevons apply when there is venture capital in the game, making the first hit free? The drug dealer analogy makes sense–the AI companies have hooked some users into switching off their mind. Now that it’s gone they they charge you for their second-rate replacement-mind.
I can’t speak to other uses of AI, but I’ve been using it for software development for a couple years now. When it comes to hallucination, you’re talking about a 2025 problem in 2026.
A year or two ago, AI had very annoying tendencies towards making up subroutine or variable names that were plausible but non-existent. With the latest models (e.g. Claude’s recent Opus), that simply hasn’t been an issue. The code I get back from it is generally correct. Sometimes, with experience on my part I can see things that need refinement or architectural considerations for future work. These are easily corrected. But at least in my field, the “confidently creating garbage” bug has been fixed.
For those wondering (like me) about the word count in “I know that I know nothing”, in the original language this is often quoted as “scio me nihil scire”.
100% agree with your point about “trust” but I’d say there are more than “exactly one problem in artificial intelligence” (only for the frontier models) … great lack of transparency about training data and post-training RLHF etc. not to mention TESCREAL.
If interested, see sandboxing article at https://designingsecuresoftware.com/writings/2026may/
Great to see you writing again, always thought provoking!
While the conversation here has understandably centered on hallucinations in desk jobs like coding, analysis, legal work, etc., I think we’re overlooking a bigger distinction Bob’s piece hints at but doesn’t quite spell out. The hallucination problem that plagues LLMs is fundamentally a language-model issue. It shows up in knowledge work precisely because the machine is guessing the next plausible token rather than grounding itself in reality. A robot folding clothes, by contrast, is never going to “hallucinate” that the garments are already perfectly folded when they’re not. Its world is sensors, actuators, and physics – no probabilistic storytelling, no invented facts, just direct interaction with the physical state of the laundry. So, while “knowing what you don’t know” is critical for enterprise cognitive tasks, it may prove far less relevant in embodied robotics. The real AI revolution could split. The desk jobs will still be wrestling with hallucinations, while the robot in the laundry room simply gets on with the job reliably, quietly, and without the need to confess ignorance it never had in the first place. I believe this is the true value people want from AI anyway. Fold my laundry, wash the dishes, clean the house. I can then create my own art, music, and do my own thinking without your AI thank you very much.
Bob,
There’s a big mistake buried in here.
“…A bank, a hospital, an insurer, a law firm — they do not deploy a system that is confidently wrong five percent of the time. They can’t. Five percent confidently wrong, in a contract or a diagnosis or a compliance filing, is not a rounding error. It is a lawsuit, a recall, a fine, a firing….”
Business have spent the last few decades working on one thing above all, which is neutralizing the sources of accountability for things like this. They have been very successful in that. I’ll give you hospitals, and you didn’t mention aviation, which is the other big example of an industry that still has some incentives to care about quality. But I can’t give you banks; they know they will be bailed out, and so will the insurance industry when it comes to it. For the rest, enforcement has been effectively abolished.
You will find this out when your new firm goes to pitch its service based on arguments from quality, or even from cost-benefit. All that matters today is looking good. At the moment, wasting $$$B looks good, but when it stops looking good, the reaction will not settle on your value proposition; it will sweep you away with everything else, like the dam broke.
How many unique words in “I know that I know nothing”? 4
Great article and series. Thank you Bob.