Large language models are, at their core, extraordinarily sophisticated autocomplete engines. They predict the next word — or, more precisely, the next token — based on patterns absorbed from billions of text examples. This mechanism produces uncanny fluency in language, passable legal briefs, and occasionally moving verse. It also produces confident assertions that 7 × 8 equals 54.
The innumeracy of LLMs is not a bug awaiting a patch. It is an inevitable consequence of how these systems process information. Understanding why reveals something fundamental about what artificial intelligence is — and what it is not.
The token problem
When you type "347 + 589" into a language model, you imagine the system seeing two numbers and an operator. The model sees something quite different: a sequence of tokens that might split those digits in arbitrary ways. Depending on the tokeniser, "347" could be one token or three. The number "589" might share a token boundary with the plus sign. The model has no internal representation of numerical magnitude — no sense that 347 is closer to 350 than to 3.
This is not how humans do arithmetic. We learn that numbers exist on a continuous line, that 8 is one more than 7, that multiplication is repeated addition. We develop mental algorithms. Language models develop statistical associations: they have seen "2 + 2 = 4" so many times that they reproduce it reliably, but they have seen "347 + 589 = 936" far less often, if ever. They are pattern-matching, not calculating.
Why more data does not help
The intuitive response is that models simply need more mathematical training data. But this misunderstands the architecture. A transformer's attention mechanism excels at capturing relationships between tokens across long contexts — perfect for understanding that a pronoun refers to a noun mentioned paragraphs earlier. It is poorly suited to the rigid, position-dependent operations that arithmetic requires.
Consider carrying in addition. When you add 7 and 9 in the ones column, you must carry 1 to the tens column. This requires tracking state across positions in a way that transformers do not naturally support. Researchers have demonstrated that even models trained exclusively on arithmetic problems plateau at modest accuracy for multi-digit operations. The ceiling is architectural, not informational.
The workaround era
The industry's solution has been to route around the problem. Modern AI assistants detect mathematical queries and dispatch them to external calculators or code interpreters. When ChatGPT correctly computes your tax liability, it is likely executing Python behind the scenes, not reasoning numerically. This is elegant engineering but also a quiet admission: for tasks requiring symbolic manipulation, the language model is the interface, not the engine.
Similar limitations appear in counting (how many r's in "strawberry"?), in logical puzzles with strict constraints, and in any domain where statistical approximation is insufficient and precision is mandatory. The model's strength — generalising from patterns — becomes a weakness when the task demands exact symbolic processing.
Our take
The innumeracy of language models is clarifying. It reminds us that fluency is not intelligence, that pattern recognition is not reasoning, and that the most impressive AI systems are still tools with sharply defined boundaries. The hype cycle wants us to believe that scale solves everything — more parameters, more data, more compute. Arithmetic suggests otherwise. Some capabilities require different architectures, different approaches, perhaps different paradigms entirely. The AI that writes your emails and the AI that balances your books may never be the same machine, and that is not a failure. It is simply the shape of the technology we have built.
