Hallucination Scoring & Old AP Test Scoring
Lack of Guessing Penalties: The Source and Solution to Hallucination?
Language models like GPT-5 “are optimized to be good test-takers, and guessing when uncertain improves test performance” Why Language Models Hallucinate This is the key to AI hallucinations, according to a new research paper from OpenAI, the maker of ChatGPT, published on September 4, 2025. I think this explanation has merit, although it doesn't seem to explain when large language models (LLMs) have access to sources with the correct answers and incorrectly summarize them.
The most interesting point to me in the paper is their call for changing how AI benchmarks score different AI models to penalize wrong guesses. This reminded of how for most multiple-choice tests in school, you should choose any random answer rather than leave the answer blank. If the answers are ABCD, you have a 25% chance of getting the answer right and you always have a positive expected value, because you either get one point or zero. Zero for a wrong answer is the same as zero for no answer. However, Advanced Placement (AP) tests used to give negative points for wrong answers. When I went to find a source for my recollection about AP test scoring, I learned that this policy had changed shortly after I graduated high school. (“AP creates penalties for not guessing,” July 2010). So it appears that penalizing guessing is just as unpopular with human benchmarks as AI benchmarks. I, for one, am in favor of wrong-guess penalties for both.
AI hallucinations include not only completely made-up references (which are often easy to spot if you put in any effort to check), but also subtly incorrect summaries or correct-yet-irrelevant citations that do not support the point being made (which may require close reading of the cited sources).