Ask AI Why It Sucks at Sudoku. You’ll Find Out Something Troubling About Chatbots
- How much can you trust a generative AI tool if it can’t explain itself honestly or accurately?
Jon Reed — Aug. 8, 2025 4:00 a.m. PT
*** begin quote ***
That’s what researchers at the University of Colorado at Boulder found when they challenged large language models to solve sudoku. And not even the standard 9×9 puzzles. An easier 6×6 puzzle was often beyond the capabilities of an LLM without outside help (in this case, specific puzzle-solving tools).
*** and ***
You might expect LLMs to be able to solve sudoku because they’re computers and the puzzle consists of numbers, but the puzzles themselves are not really mathematical; they’re symbolic. “Sudoku is famous for being a puzzle with numbers that could be done with anything that is not numbers,” said Fabio Somenzi, a professor at CU and one of the research paper’s authors.
*** end quote ***
That’s some “wonderkid” you’ve been celebrating in the press. “AI will revolutionize … <insert favorite profession>”!
So we have kids using it to cheat in school; NBD — no big deal.
Legal opinions cite incorrectly or cites non-existent court cases — BIG deal!
And, now it can solve a 6×6 sudoku. Or gives the wrong answer?
With the final straw, it can’t “show it’s work” (i.e., the steps it used to reach the “solution”).
Pardon me if I declare AI “not ready for prime time”!
The human equivalent is the Titan submersible suffering a catastrophic implosion and you want me to let is drive a truck? We don’t need AI to make disasters. Humanity can do it quite well without help.
— 30 —
Reinke Faces Life 