Class is in!
When you use ChatGPT or DeepSeek, the words pop out one at a time. That’s because, at heart, it does just one thing: guess the next word. Here are five steps to understanding large language models.
Step 1: Cut the text into tokens. The model doesn’t know words, only numbers. A sentence is first cut into small pieces, and each piece is called a token. In Chinese a single character is often one token; long English words get split into several pieces. Every token has an ID number, so what the model actually sees is just a string of numbers.
Step 2: Guess the next one. Hand the tokens so far to the model, and it gives every token in its vocabulary a probability, picks one, adds it to the end of the sentence, then feeds the whole thing back in and guesses again. There’s also a setting called temperature: turn it down and it always picks the safest choice; turn it up and it’s more willing to pick unusual words. That’s why asking the same question twice often gets different answers.
Step 3: How it learned. First it reads an enormous amount of text: Meta’s Llama 3 read more than 15 trillion tokens. Training is a gigantic fill-in-the-blank game: hide the next word and have it guess, and when it’s wrong, nudge its internal parameters a tiny bit. The largest Llama 3.1 has 405 billion parameters, and everything it learned ends up stored in those numbers.
Step 4: Understanding context. In 2017 Google introduced the Transformer, whose core idea is called attention: every word looks at the other words in the sentence to decide which ones relate to it. The same word “apple” next to “eat” is a fruit; next to “launch” and “phone” it’s the company. This attention is stacked dozens to over a hundred layers deep.
Step 5: Teaching it to chat. A freshly trained model can only continue text; ask it a question and it might just make up more questions. So it’s taught with lots of example Q&A, and people score and rank its answers, and that’s how it becomes the AI assistant you use every day.
One last thing to watch out for: hallucinations. It’s writing a fluent next word, not looking things up. Ask it “Did Lu Xun ever get into a fight with Zhou Shuren?” and it may gravely invent a whole feud. But Lu Xun was Zhou Shuren: it was his pen name. For important facts, remember to double-check them yourself.
Cut into tokens, guess the next one, fill in blanks at massive scale, attention, then teach it to chat: that’s a large language model. Do you think it truly understands?
