How large language models actually work

Toolkit Blog · July 11, 2026 · Explainers · 10 min read

Ask a chatbot to plan a week in Lisbon and it produces something fluent, organized and mostly sensible in seconds. Ask it how many r's are in "strawberry" and it may confidently get it wrong. Both behaviors fall out of the same machinery. This piece walks through that machinery — tokens, next-word prediction, attention, training, temperature — in plain language, and explains why the design that makes the text so fluent also makes some of it false.

It starts with tokens, not words

A neural network is, at bottom, a very large pile of arithmetic. It cannot operate on letters or words directly — everything must become numbers first. So before your message reaches the model, a component called a tokenizer chops the text into tokens: chunks that can be whole words, pieces of words, or single characters. For Claude, one token works out to roughly 3.5 English characters on average^[1]. Common words like "the" typically get a token of their own; rarer words get split — "unbelievably" might arrive as something like un + believ + ably. Each token is then mapped to a long list of numbers, and those numbers are what the model actually processes.

This single design choice explains a whole family of famous failures. The model never sees the letters inside a word — "strawberry" reaches it as one or two opaque chunks, not as eleven characters. Asking it to count the r's is a bit like asking you to count the letters in a word you've only ever heard spoken. The same goes for reversing strings, precise rhyme schemes and arithmetic on long numbers, which get sliced into tokens at boundaries that have nothing to do with place value. The model isn't being lazy; the information genuinely isn't in front of it in that form.

The whole trick: predict the next token

Here is the core of it, and it's almost disappointingly simple. Given all the text so far, the model produces a probability for every token in its vocabulary being the next one. One token gets picked, appended to the text, and the process repeats — one token at a time, until the answer is done^[1]. A thousand-word chatbot reply is roughly a thousand of these predictions in a row. There is no separate planning module, no fact-retrieval step, no grammar engine. Prediction is the entire mechanism.

Why does such a modest trick produce coherent essays? Because predicting the next word well, across trillions of words of human writing, is a brutally demanding task. To beat the competition at it, a model is forced to internalize spelling patterns, then grammar, then style, then facts about the world, then something that often functions like reasoning — because all of those things reduce prediction error. The task acts as a compression engine for the patterns in everything people have written. What it never requires, though, is a way to check whether a prediction is true. Hold that thought.

Training: two very different phases

Where do the model's abilities come from? Training happens in two broad stages, and the distinction matters.

Pretraining is the famous, expensive part. The model reads an enormous corpus of text and plays the prediction game billions upon billions of times: guess the next token, compare against what actually came next, nudge the internal parameters — the billions of numbers that define the model — a tiny step toward doing better, repeat. No human labels anything; the text itself is the answer key. The result is a raw model that is an astonishing autocomplete engine but not an assistant — it completes text rather than following instructions, and it will happily continue your question with more questions^[1].

Post-training turns that raw engine into something usable. The model is first fine-tuned on examples of the desired behavior (question in, helpful answer out), and then refined with reinforcement learning from human feedback (RLHF): humans compare alternative outputs and rank them, and the model is pushed toward producing the kind of answer people prefer^[1][2]. This step is remarkably powerful — OpenAI's InstructGPT work found that a model with 1.3 billion parameters, tuned this way, produced outputs human evaluators preferred over a raw model 100× its size^[2]. One honest caveat: RLHF optimizes for what human raters score highly, which overlaps with "true and useful" but is not identical to it. A confident, polished wrong answer can rate well. That tension never fully goes away.

Attention: every word gets to look at every other word

The architecture underneath all modern LLMs is the transformer, introduced in a 2017 Google paper with the now-famous title "Attention Is All You Need"^[3]. Earlier language models read text the way a person reads a ticker tape: one word at a time, carrying along a running summary. By the end of a long sentence, information from the beginning had faded — like a game of telephone played with the model's own memory.

The transformer's answer is a mechanism called attention, and the intuition is exactly what the name suggests: when processing each token, the model gets to look at every other token in the text and decide, with learned weights, which ones matter right now. Take "The trophy didn't fit in the suitcase because it was too big." To handle "it," the model needs to weigh "trophy" against "suitcase" — attention is the machinery that lets it do that directly, no matter how far apart the words sit. Stack this dozens of layers deep and each token's internal representation gets progressively refined by everything around it: "bank" near "river" ends up represented differently than "bank" near "loan."

The paper's authors showed that attention alone — "dispensing with recurrence and convolutions entirely" — matched or beat the best translation systems of the day^[3]. Just as important, attention processes all positions in parallel rather than one at a time, which is what made it practical to train on previously absurd amounts of text and hardware. That parallelism, as much as any cleverness, is why models could scale to where they are now.

Context windows, and why models "forget"

Everything above operates on one bounded stretch of text: the context window — all the text a model can reference while generating, including its own reply as it writes it. Think of it as working memory, entirely separate from whatever the model absorbed in training^[4]. Your messages, the model's answers, any documents you paste — all of it has to fit inside.

This is why long conversations get strange. When a chat outgrows the window, the software around the model quietly drops or summarizes the oldest turns. The model hasn't "forgotten" your instructions the way a distracted person would — that text is simply no longer part of its input. It also can't remember you between conversations: a new chat is a blank slate unless the product deliberately re-injects notes about you. As of mid-2026, frontier models advertise windows from a couple hundred thousand tokens up to a million^[4] — hundreds of pages — but bigger isn't automatically better: as the window fills, accuracy and recall measurably degrade, a phenomenon practitioners call context rot^[4].

Why models make things up

Now the heart of it. The single most useful thing to understand about an LLM is that there is no database inside. What the model "knows" is smeared across billions of numeric parameters as statistical tendencies — there is no row it can look up, no record it can double-check, no flag that distinguishes a memorized fact from a plausible guess. When you ask a question, the model does not retrieve an answer. It does the only thing it ever does: continue the text in the most plausible way.

For well-worn facts — capital cities, famous dates — the most plausible continuation and the correct one are the same, because the truth appeared thousands of times in training. For obscure facts, they come apart, and the machinery produces something with the exact shape of a right answer: a confident citation of a paper that doesn't exist, in a journal that does; a court case with a plausible name and a fabricated docket number. This is why fluency is no signal of reliability. Fluency is the one thing the model is good at unconditionally; it writes falsehoods in precisely the same assured register as facts.

Anthropic's interpretability research has actually traced part of this mechanism. Inside Claude, refusal appears to be the default: a circuit that is on by default pushes the model to say it has insufficient information, and a separate "known entities" feature suppresses that refusal when the model recognizes the subject. Hallucinations happen when this misfires — the model recognizes a name, the "I know this" signal activates, but the underlying facts aren't there, so the machinery fills the gap with something plausible^[5]. That's also why models are poor judges of their own uncertainty: the "do I know this?" check is itself just another learned circuit that can be wrong.

Products bolt on mitigations — web search, retrieval from documents, citations — which genuinely help by putting real text into the context window for the model to work from. But the underlying tendency is architectural. The practical rule: treat an LLM like a brilliant, widely read colleague who never checks sources and never says "I'm not sure" unless trained to. Verify anything load-bearing.

Temperature: the randomness dial

One loose end from earlier: the model produces probabilities for the next token — something still has to pick one. That choice is called sampling, and temperature is its main control. Low temperature makes the model stick closely to its most probable choices — conservative, repeatable-ish output, good for code or extraction. Higher temperature flattens the odds so less-likely tokens get picked more often — more varied and surprising phrasing, better for brainstorming^[1]. This is why the same question can get a differently worded answer on different days: you're watching sampling, not indecision. Two fine print items: even at temperature zero, outputs aren't perfectly deterministic in practice^[1], and turning temperature down does not turn hallucination off — a fabrication can be the most probable continuation.

What nobody fully understands yet

A closing dose of honesty: we built these systems, and we still can't fully read them. Training doesn't write legible code — it grows billions of parameters whose collective behavior nobody designed line by line. The field trying to reverse-engineer them, interpretability, is young and openly incomplete. Anthropic's team has extracted millions of interpretable "features" from a production model — including one for the Golden Gate Bridge which, when artificially amplified, made the model obsess over the bridge in unrelated answers^[6] — and traced circuits showing that a model asked to write rhyming poetry plans candidate rhymes ahead before writing the line, despite being trained only to predict one token at a time^[5]. Yet the same researchers estimate their methods capture only a fraction of the computation happening inside even short prompts^[5].

So the picture in this article — tokens in, attention over context, next-token prediction out, shaped by pretraining and RLHF — is solid. The fine-grained why behind any particular answer often remains an open research question. That's the right frame to carry out of this piece: genuinely impressive, genuinely useful, not magic, and not fully understood — including by the people who make them. Verify accordingly.

Sources

1. Anthropic — Claude Docs: Glossary (tokens, pretraining, RLHF, temperature), accessed July 2026
2. arXiv — Ouyang et al., "Training language models to follow instructions with human feedback" (2022), accessed July 2026
3. arXiv — Vaswani et al., "Attention Is All You Need" (2017), accessed July 2026
4. Anthropic — Claude Docs: Context windows, accessed July 2026
5. Anthropic — "Tracing the Thoughts of a Large Language Model" (March 2025), accessed July 2026
6. Anthropic — "Mapping the Mind of a Large Language Model" (May 2024), accessed July 2026

Related: How AI vocal removal works · Why Toolkit runs entirely in your browser · Browse all tools