Kruse v. Karlen Fictitious Cases Study

Google AI Hallucination Outcomes

SEARCHES AI OVERVIEW? AI RESPONSE OUTCOME 22 SEARCHES 10 AI OVERVIEW GENERATED 12 NO AI OVERVIEW → AI MODE 5 CORRECT 5 HALLUCINATED 4 CORRECT 1 MISATTRIBUTED 7 HALLUCINATED 9 CORRECT (41%) 12 HALLUCINATED (55%) 1 MISATTRIBUTED ~45% ~55% 50% error 58% error
Correct identification
Hallucinated (claimed real)
Misattributed source
AI Overview generated
AI Mode (manual)

Key Findings

100%
Google Search alone found correct links to Kruse v. Karlen
~23%
Error rate from AI Overview without opting into AI features
~55%
Error rate if user opted for AI Mode when Overview absent
50%
AI Overview hallucination rate (5 of 10)
58%
AI Mode hallucination rate (7 of 12)