What Is an LLM, Really?

Large language models are not search engines or databases - they learn patterns from text and predict what comes next. Here is how that simple loop produces something that genuinely reasons.

TL;DR: A large language model is trained to predict the next piece of text given everything before it. Do that billions of times on enormous amounts of human writing, and the model is forced to learn grammar, facts, reasoning, and tone - not because anyone programmed those in, but because predicting text well requires them. That is the whole trick.

Start here: what a "token" actually is

Before anything else, you need to know how an LLM reads text. It does not see letters, and it does not see whole words. It sees tokens - chunks that usually correspond to common word fragments, short words, or punctuation.

The word "tokenization" typically splits into two tokens: token and ization. The word "the" is one token. A space, a comma, a line break - each can be its own token. In English prose, a rough rule of thumb (from OpenAI's documentation) is that one token is about four characters, or around 0.75 words.

Modern tokenizers use a method called Byte-Pair Encoding (BPE), which builds a vocabulary of the most common subword fragments found in training data. Most large models end up with vocabularies of roughly 32,000 to 128,000 tokens or more. Every piece of text you feed a model - and every word it writes back - gets converted to and from this token vocabulary.

Why does this matter? Because the model's job, at the most literal level, is to predict which token comes next. Everything else follows from that.

The training loop: next-token prediction at scale

Here is how an LLM is built.

1. Gather an enormous amount of text. Books, websites, code, scientific papers, forum posts - text at a scale that is hard to visualize. The LLaMA training set, for reference, contains roughly 1.4 trillion tokens.
2. Show the model a sequence of tokens. Ask it: what token comes next?
3. Check if it was right. Adjust the model's internal numbers to make the correct answer slightly more likely.
4. Repeat, billions of times.

That is pretraining - and it is self-supervised, meaning no human has to label anything. The text itself is the teacher. The "right answer" is just whatever token actually appeared next in the original document.

As Anthropic describes it: "Language models like Claude aren't programmed directly by humans - instead, they're trained on large amounts of data. During that training process, they learn their own strategies to solve problems."

Those strategies get encoded in billions of numerical parameters - weights inside the model's layers that get nudged slightly with every training step.

Why predicting text forces a model to learn real things

Here is the key insight that surprises most people.

To accurately predict the next token in a wide range of human writing, a model cannot just memorize. There is too much text and too much variety. Instead, it has to learn why things are written the way they are - which means learning grammar, logic, world facts, social conventions, and cause and effect.

A model that has read millions of documents about chemistry, and been trained to predict what comes after "the boiling point of water is," has to have compressed something real about chemistry into its weights. Not a lookup table of facts - something more like an implicit understanding of how facts relate.

Anthropic's interpretability research makes this concrete. When the model computes "36 + 59", its internal circuitry runs multiple parallel paths - one approximating the magnitude, another pinpointing the last digit - rather than looking up a stored answer. More strikingly, the same arithmetic features activate in entirely different contexts: a journal volume number and a founding year that happen to share the same digit pattern will trigger the same addition circuit the model uses for explicit math problems.

Similarly, when asked "what is the capital of the state containing Dallas?", the model represents "Texas" as an intermediate step - it genuinely chains reasoning together rather than jumping straight to "Austin."

Inside the model: attention and features

The architecture that makes this work is called a transformer, introduced in the 2017 paper "Attention Is All You Need." The key mechanism is attention: the model can look at any part of the input sequence when predicting the next token, and it learns which parts are relevant to attend to.

In a sentence like "The trophy didn't fit in the suitcase because it was too big," figuring out what "it" refers to requires attention - the model has to relate "it" to "trophy" or "suitcase" based on context. Attention is how transformers handle that.

Internally, the model represents concepts as features - patterns of neural activations spread across many neurons simultaneously. Anthropic's research has extracted millions of interpretable features from Claude's middle layers, finding features for cities, people, abstract concepts like "code bugs," and emotional states. These features are not stored in individual neurons - each concept activates across many neurons, and each neuron participates in many concepts.

One striking finding: the model operates in "a conceptual space that is shared between languages." The same internal features activate whether you write "Paris" in English, French, or Japanese - suggesting the model builds a language-independent map of meaning, not a per-language translation dictionary.

After pretraining: turning a text predictor into an assistant

A pretrained model is good at continuing text in any style - including harmful styles. The second training phase, called post-training, is what turns it into something useful and safe to deploy.

As Anthropic describes it: "Later, during 'post-training,' the model is taught to play the role of a character, typically an 'AI assistant.'" Model developers specify how this character should behave - be helpful, be honest, don't cause harm.

This typically involves two techniques:

• RLHF (Reinforcement Learning from Human Feedback): Human raters compare pairs of model responses and say which is better. A reward model learns to predict human preference, and the LLM is fine-tuned to score higher on that reward model.
• Constitutional AI (Anthropic's approach): The model critiques and revises its own responses based on a set of principles, then gets fine-tuned on AI-generated feedback derived from those principles - reducing the need for humans to label every edge case.

The result is a model that retains the broad knowledge from pretraining but behaves in a way aligned with the guidelines its developers specified.

What LLMs still get wrong (and why)

Understanding the mechanism explains the failure modes.

• Hallucinations: The training objective is to predict plausible next tokens - not necessarily true ones. Anthropic's research found a specific hallucination circuit: the model maintains default "can't answer" signals that are normally suppressed by "known entity" features when the model genuinely recognises a topic. When those known-entity features misfire on an unfamiliar subject, the refusal signal gets suppressed anyway and the model produces confident-sounding false information.
• Inconsistency: There is no persistent memory across conversations by default. Each context window starts fresh. The model does not "remember" what it told you last week.
• Math errors on novel problems: The model can chain real arithmetic steps, but it operates token by token and can lose track on long calculations. It is not running a calculator; it is predicting what a calculation would look like.
• Introspection limits: Anthropic's research found that models can sometimes recognise their own internal states - but only about 20% of the time in controlled tests, even with the best injection protocol tested on Claude Opus 4.1. When an LLM explains its reasoning, that explanation may not match what actually happened internally.

Key takeaways

• LLMs are trained to predict the next token - that single objective, applied at enormous scale, forces the model to learn grammar, facts, reasoning, and more.
• Text is broken into tokens (roughly 0.75 words each in English) before the model ever sees it.
• The transformer's attention mechanism lets the model relate any token to any other in the context window - that is how it handles complex references and long-range dependencies.
• Internally, concepts are represented as features spread across millions of neurons, not stored in individual locations - and those features are largely language-independent.
• Post-training (RLHF / Constitutional AI) shapes the pretrained model into an assistant that behaves helpfully and safely.
• Hallucinations are a direct consequence of the training objective: the model is optimised to produce plausible text, not guaranteed-true text.
• The model does not "look things up" - it runs computation over compressed patterns from training data. That distinction matters when you decide what to trust it with.

Try this next: now that you understand how an LLM works at the model level, the next question is how to talk to one effectively. Prompt Engineering Basics walks through how to structure your inputs so you get the outputs you actually want.