How do LLMs work? A plain-English guide to large language models

A large language model (LLM) is a program that writes text by guessing the next word, again and again. It learned how to make good guesses by reading a huge amount of text. That simple idea sits behind ChatGPT, Claude, Gemini and every other AI chatbot. This guide explains, in plain English, how an LLM works: what happens to your prompt, how the model was trained, and why it sometimes gets things wrong.

What is a large language model?

An LLM is a neural network trained on text. It is "large" because it has billions of parameters, the numbers it adjusts while learning. It is a "language model" because its one job is to predict which piece of text comes next.

You type a question. The model does not look up an answer in a database. It writes a reply one small piece at a time, each time choosing a likely next piece based on everything before it.

How does an LLM work, step by step?

Here is what happens between pressing Enter and seeing the reply.

1. Your text is split into tokens. "How do LLMs work?" becomes a short list of tokens, and each token becomes a number.
2. Each token becomes a vector. The model turns every token into a long list of numbers called an embedding. Words with similar meanings end up with similar embeddings.
3. Attention mixes in context. In each layer, every token looks back at the tokens before it and decides which ones matter. This is how "it" in "the cat sat on the mat because it was tired" gets linked to "cat".
4. Dozens of layers refine the meaning. The model repeats attention plus a small calculation step many times. Each layer builds a richer picture of what the text means.
5. The model scores every possible next token. The last layer gives a probability to each token in its vocabulary.
6. One token is picked. The model chooses a likely token, adds it to the text and starts again from step 3. It stops when it picks an "end" token.

This is why answers appear word by word on your screen: the model really does write them one token at a time.

The transformer: the design behind modern LLMs

Almost every modern LLM is built on the transformer, a design published by Google researchers in 2017 in the paper "Attention Is All You Need".

Older models read text one word at a time and slowly forgot the start of long sentences. The transformer's attention lets every token look at all earlier tokens directly. It also lets a computer process a whole passage at once, which made training on huge amounts of text possible on GPUs.

A transformer is a stack of identical blocks. Each block has two parts:

• Attention, where tokens share information with each other.
• A feed-forward network, where each token is processed on its own.

Chatbots like ChatGPT and Claude use the "decoder-only" form, which only ever looks at earlier text. That is exactly what you need to predict the next token.

How are LLMs trained?

Training happens in stages. Each stage teaches the model something different.

Pretraining

The model starts with random numbers. It reads text, guesses the next token, checks the real one and adjusts its parameters slightly to do better. Repeated across trillions of tokens, this guessing game teaches it grammar, facts, style and even some reasoning.

DeepMind's Chinchilla research found that, for a fixed compute budget, a model should see about 20 training tokens for every parameter. Pretraining a frontier model takes thousands of GPUs running for weeks or months.

Fine-tuning and RLHF

A pretrained model only continues text. Ask it a question and it might write more questions. Fine-tuning on example conversations turns it into an assistant. OpenAI's InstructGPT work showed the recipe: a few thousand written answers, then reinforcement learning from human feedback (RLHF), where people compare answers and the model is nudged toward the ones they prefer.

What is a context window?

The context window is everything the model can see while it writes: your messages, its own replies, any documents you attached and the hidden instructions from the app. It is measured in tokens.

When a chat grows past the window, the oldest text has to go. That is why a long chat can "forget" something you said at the start. Research also shows models use information at the start and end of a long context better than information in the middle.

Tip: start a new chat for a new topic. It keeps the window clean and the answers sharper.

Why do LLMs make things up?

LLMs "hallucinate" because they are built to produce likely text, not checked facts. If the model never saw a fact, or saw it rarely, it still produces a fluent guess. Training that rewards an answer over "I don't know" makes this worse.

What helps:

• RAG (retrieval-augmented generation): the app finds relevant documents and puts them in the prompt, so the model answers from real sources.
• Asking for sources and checking them.
• Clear prompts that give context, a goal and the format you want.

Can you run an LLM on your own computer?

Yes. Smaller open models run on a laptop or even a phone. The trick is quantization: storing each parameter in 4 bits instead of 16, which cuts memory about four times.

A rough rule: at 4 bits a model needs about half a gigabyte per billion parameters, plus extra memory for the conversation. A 7-billion-parameter model fits in a laptop with 8 GB of free memory. Tools like Ollama and LM Studio make it a one-command install.

Frequently asked questions

Is an LLM the same as AI? No. An LLM is one kind of AI, specialised in text. AI also includes image generators, speech recognition, recommendation systems and more.

Does an LLM understand what it says? It builds detailed internal representations of meaning, which is why it can answer new questions. But it has no direct experience of the world and can be confidently wrong, so check important answers.

Does ChatGPT search the internet? The model itself only knows its training data, up to a cutoff date. Many apps add a web search tool that fetches fresh pages and puts them in the context window.

How big are LLMs? Small open models have 1 to 10 billion parameters. Frontier models are far larger. Most companies no longer publish exact sizes.

What is the difference between GPT, Claude and Gemini? They are LLM families from different companies (OpenAI, Anthropic and Google). All are transformers trained in similar stages; they differ in training data, size, tuning and the tools around them.

Go deeper: our How LLMs work series

• Part 1: From the perceptron to reasoning models
• Part 2: The math of neural networks
• Part 3: Attention
• Part 4: The transformer, block by block
• Part 5: Pretraining and scaling laws
• Part 6: Fine-tuning and LoRA
• Part 7: RLHF and reasoning

Learn how LLMs work, lesson by lesson

This guide covers the basics. Our video course How LLMs work goes all the way: GPUs and data centers, the context window, prompts, RAG, prompt security, evaluation, quantization, attention and the transformer, pretraining, fine-tuning, RLHF, and finally building your own small LLM. It has 23 animated lessons with Anna, Max and Akami, and it comes out in November.

Read the course announcement

Sources

• Vaswani et al.: Attention Is All You Need (2017)
• Hoffmann et al.: Training Compute-Optimal Large Language Models (Chinchilla, 2022)
• Ouyang et al.: Training language models to follow instructions with human feedback (InstructGPT, 2022)
• Liu et al.: Lost in the Middle: How Language Models Use Long Contexts (2023)
• Lewis et al.: Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks (2020)
• OpenAI: What are tokens and how to count them?