What is an AI model? A beginner’s guide to the numbers inside

Phone keyboards suggest the next word by guessing what is likely to follow. An AI chatbot does the same kind of guessing on a far bigger scale. The thing doing the guessing is called a model: a large set of numbers plus a fixed procedure that turns an input into an output. Here the input is the text typed in, and the output is a best guess at what should come next. The process is a bit like following a recipe. The ingredients go in, the same steps run every time, and a dish comes out.

Those numbers can make a guess because each one acts like a dial. Picture a sound desk in a music studio, covered in sliding dials. Each dial shifts the sound a little, and together their positions decide how the whole song comes out. A model works the same way. Each dial is a parameter, also called a weight, which is one adjustable number inside the model. GPT-2 small, a model OpenAI released in 2019 and the running example for this series, has about 124 million of them.

That many dials would be useless in one jumbled heap, so they are arranged in stages. Text enters the first stage, which does some simple calculations with its dials and hands the result to the next. The arrangement is like a factory assembly line, where each station does one small job and passes the item along until a finished car rolls off. Each stage is called a layer, and the whole stack is a neural network: a model built from layers of simple calculations, loosely inspired by brain cells. GPT-2 small has 12 layers, one after another.

As text passes through those layers, the model also needs a way to tell which words matter to each other. In "The trophy didn't fit in the case because it was too big," the word "it" only makes sense once "trophy" is weighed as the likely reference. The step that gives a model that glance back is called attention: it lets each piece of text weigh how relevant every other piece is.

Attention turned out to be so useful that, in 2017, researchers built a whole design around it. They called it the transformer, and it is the design behind today's language models, GPT-2 included. It works like a blueprint style for buildings, where the same basic plan can serve a small house or a huge tower. Inside each of GPT-2 small's 12 layers, 12 attention steps run side by side and track several kinds of links between words at once.

Take a transformer, make it very large, and train it on huge amounts of text to guess what comes next, and the result is a large language model, or LLM. That is what OpenAI built GPT-2 to do. To write a paragraph it guesses one chunk at a time, adding each guess to the text before the next. Part 3 explains what those chunks are.

When a finished model produces an answer, that process is called inference: using the model without any learning taking place. It is like using a calculator. The buttons can be pressed all day, and the wiring never changes, so all 124 million dials stay where training left them and a chat reply does not quietly retrain the model.

An AI model, then, is a giant set of numbers arranged so that text goes in and a best guess comes out. How those 124 million dials got set in the first place is the story of Part 2, "How is AI trained?"