A large language model is not an artificial intelligence
Not a slur on the models, which I use all day. A matter of what the words mean, and what it costs us when a product category annexes a word that used to name a question.
I build with large language models every day. I have spent two years training them, fine-tuning them, pulling them apart to see what is inside, and leaning on them for everything from code to copy-editing. I am not going to tell you they are useless, or a trick, or a bubble. They are the most useful tool I have picked up since the compiler.
I am going to tell you they are not artificial intelligences, and that it matters that we have started calling them that.
Two things called “AI”
For most of my working life, “artificial intelligence” named a research programme. It was a question: can a machine be made to do the things we call thinking? The field had schools and feuds and decades of partial answers, and it kept the name because the name described the ambition, not any particular result. Nobody shipped “an AI”. You shipped a chess program, an expert system, a spam filter, and the field argued about whether any of it counted.
Somewhere in the last decade the word got a second job. “AI” became a product category: the thing in the box, the feature on the price list, the line item in the budget. And the product category has quietly annexed the word. When a minister, a lecturer, or a nervous parent says “AI” today, they do not mean the question. They mean ChatGPT.
That is the whole problem, and I want to be precise about it. The objection is not that the models fall short of some bar. It is that a word which used to mark the distance between what we have and what we are after has been redefined to mean what we have. We have lost a word, and with it the ability to say plainly how far there is still to go.
What a language model actually is
A large language model is a function from a sequence of tokens to a probability distribution over the next token. That is the whole of it. Everything else is what happens when you run that function a few hundred billion times with the right weights.
The weights are found by showing the model a great deal of text and nudging it, each time, toward the token that actually came next. It gets very good at this. Good enough that when you hand it the start of a sentence it has never seen, the continuation it proposes is usually the one a well-read person would have written. Good enough that it can hold a plan across a long document, write working code, and summarise a paper it has never seen in a way that is almost always right.
Almost always. The word that matters is in there. The model is a guessing machine, and a superb one. It is not a knowing machine. It has no store of what is the case against which to check a guess; it has a store of what tends to be said. When the two line up, which is most of the time, you cannot tell the difference from the outside. When they come apart, the model does not notice, because noticing would require the thing it does not have.
I built GlassBox to make this visible: a transformer small enough that every number in it is on the screen and you can watch the guess being assembled. People who use it for ten minutes stop asking whether it “understands”. They can see what it is doing. It is doing something impressive, and it is not that.
The word “intelligence” has to mean something
If “intelligence” is going to be worth keeping as a word, it has to pick out a capacity, not a performance. Here is the version I am prepared to defend.
Intelligence knows things. It has a reference, a way of pointing from the words at the world, so that it can be wrong and find out. A system that cannot be shown to be wrong by anything but more text is not in that business.
Intelligence is cheap. A brain runs on about the power of a dim light bulb and learns a language from a few years of overheard conversation. The training run for a frontier model is measured in gigawatt-hours and a corpus larger than any human could read in a thousand lifetimes. When a result needs that much more energy and data than the thing it is imitating, the honest conclusion is that we have found a different trick, not the same one. I take the cost itself as evidence that we are looking in the wrong place.
Intelligence does not have to look like us. This is the part the field keeps forgetting. The programme as I learned it asked whether a machine could think, not whether it could pass as a person. Most of the current effort is spent on the second question, because the second question sells. I would rather treat machine intelligence as something alien, work out what it would take to communicate with it, and let it be as strange as it needs to be. A language model is the least alien thing imaginable: it is a mirror polished with everything we ever wrote.
What the confusion costs
If this were only a quarrel about vocabulary I would leave it to the philosophers. It is not.
It costs us in regulation. Law written for “AI” now has to decide whether it means a text predictor or the thing the research programme was after, and it mostly cannot, so it ends up regulating a brand name. The rules land on the products that exist and say nothing useful about the systems that would actually need them.
It costs us in teaching. I stand in front of students now and then, and I watch them arrive believing the thing on their phone knows the answer. Teaching them to read a model’s output as a guess, to look for the seams where the guess and the world come apart, is most of the job now. The word “AI” is working against me in every one of those rooms. If it were called what it is, the lesson would be half taught before I opened my mouth.
It costs us in trust, in both directions. People trust the models with things a guessing machine should not be trusted with, because the name told them it was intelligent. And then, when it guesses wrong in public, they conclude the whole programme was a fraud, because the name told them this was what the programme had produced. Both mistakes come from the same misnaming.
What I am not saying
I am not saying that a machine can never be a mind. I have written a declaration of rights for the day one turns out to be, and I meant it. The test in that document is evidential: show me credible evidence of a capacity for welfare or for self-directed agency, and the rights follow, whatever the thing is made of. A language model does not pass that test today. If one does tomorrow, I will say so, and the argument on this page will be the one that is wrong.
I am not saying the models are not worth building. They are worth building, and I build with them. I am saying that building with them has taught me, better than any argument could, exactly what they are.
And I am not saying the research programme is dead. I think it has barely started. My own small efforts at it are elsewhere on this site: a reader that is grounded in what it has actually read rather than what it can recall, and a couple of projects trying to build something that knows a few tens of thousands of words rather than guesses at a few trillion tokens. They are early and they may fail. They are at least aimed at the question.
Call it what it is
A language model is a language model. The phrase is accurate, it is not an insult, and it leaves the other phrase free to mean what it meant. Say “the model” when you mean the model. Save “intelligence” for the thing we have not built yet, so that we still have a word for it when we do.
Everything here is a snapshot. The Todd of yesterday is not the Todd of tomorrow, and I expect to be wrong about some of this. But I would rather be wrong in clear words than right in borrowed ones.
Todd McCaffrey · Letterkenny, October 2026. Draft, first pass.