Why doesn't AI just say "I don't know"?
Training rewards a confident guess over an honest blank.
Ask a chatbot for the birthday of someone who's only slightly well known. Maybe a researcher, or a local business owner.
You'll often get a date, stated calmly.
Ask again in a new chat and you might get a different date, stated just as calmly.
Neither one is "I don't know".
A multiple-choice test, four options per question, one point for a right answer. A blank gets zero, and so does a wrong answer.
You don't know question 12. What do you do?
You guess. A guess gets the point one time in four. A blank never gets it.
Anyone who's sat that kind of test learns the lesson: never leave it blank.
Before release, models are measured on huge sets of test questions. The scores go on leaderboards that companies compete on.
Most of those tests mark answers as either right or wrong. "I don't know" counts as wrong.
So a model that guesses scores higher than a model that admits it isn't sure. Training pushes towards whatever scores higher.
Benchmark: a standard set of test questions used to compare AI models, like a league table for exams.
Over thousands of questions, guessing wins. Nobody chose to train a bluffer. The scoring did it.
In September 2025, researchers at OpenAI published a paper making exactly this argument.
Their words: models are "optimized to be good test-takers, and guessing when uncertain improves test performance."
They also point out that some facts are close to impossible to learn from patterns. If a person's birthday appears once in all the training text, there's nothing to generalise from. The model has to have memorised it or guess, and the scoring rewards the guess.
You don't need a new test. You need to change how the existing ones are marked: take more points off for a confident wrong answer than for "I'm not sure".
Some exams for people already work this way, with marks deducted for wrong answers so that guessing stops paying.
Until the leaderboards change, assume the habit of guessing is still there.
Those are the places where it has the least to go on, so a guess is most likely.
Give it permission not to know. Add: "If you're not sure, say so. A wrong answer is worse than no answer." It won't fix the habit, but it helps.
Ask which parts are shaky. "Which parts of this answer are you least sure about?" Then check those parts first.
For specific facts, make it look them up and show you the source (see "AI's sources need checking too").