How Large Language Models Actually Work

An inside look at the architecture, training, and magic behind conversational AI

"A large language model is just a complex mathematical function that predicts the next word for any text."

— Core mechanism behind every LLM

Core Idea

Imagine you have a text describing a conversation between a person and an AI assistant. The person's question is there, but the assistant's answer is missing. Now imagine you have a magic machine that takes any text and makes a logical guess for the next word.

You can complete the scene by using the machine to predict the first word of the response, then repeat with the updated text until the dialogue is finished.

When you interact with a chatbot, this is exactly what happens.

How Prediction Works

Instead of outputting just one word, the model outputs all possible choices along with their probabilities.

To build a chatbot, you provide a short prompt that sets up a conversation between a person and an intelligent assistant. You add the user's question at the start, then let the model predict the next word of the assistant's response. This process repeats, and the complete answer is delivered to the user.

To make responses feel more natural, the model can randomly select words with lower probabilities, introducing variation. Even though the underlying calculations are deterministic, the same question can lead to different answers.

Training at Scale

Models learn to make predictions by being trained on massive amounts of text, usually downloaded from the internet.

2,600+ years of continuous reading — that's how long it would take a human to read the text used to train GPT-3 alone.

Newer models are trained on even larger datasets, growing with each generation.

Parameters: The Dial Board

Think of training like adjusting knobs on a giant machine. The model's entire behavior is determined by these settings — called parameters or weights. Changing these weights shifts the probabilities the model assigns to each possible next word.

They're called "large" language models because they can contain hundreds of billions of these parameters.

Weights

Hundreds of billions

Initial state

Random → meaningless

Weights start random and are gradually tuned using countless text examples.

The Training Loop

Take a text sample — could be a short phrase or a very long document.

Remove the last word and have the model predict what it should be.

Compare the predicted probabilities with the actual word.

Use backpropagation to update all weights, nudging probabilities toward the correct word.

Repeat across billions or trillions of examples — the model gradually produces more accurate predictions, even on text it has never seen.

The Scale of Computation

Imagine you could perform 1 billion mathematical operations per second.

How long do you think it would take to complete all the calculations required to train the largest language models?

100 million years

That's the actual answer — far beyond a human lifetime, or even 10,000 years. This massive computation is only possible thanks to specialised chips (GPUs) that perform many calculations in parallel.

Beyond Raw Prediction

A model's ability to complete random internet text is very different from being a helpful, smart assistant. That's where Reinforcement Learning from Human Feedback (RLHF) comes in.

Human reviewers flag unhelpful or problematic outputs. Fixing these issues adjusts the model's weights, making predictions more suitable and aligned for users.

The Transformer Breakthrough

Before 2017, language models processed words one at a time. That changed when a team of Google researchers introduced the Transformer architecture. Instead of reading text from start to finish, it processes the entire text in parallel.

Old way

Word by word

Transformer

All at once

The first step inside a Transformer (and most language models) is to assign a long list of numbers to each word. Because training works with continuous numerical values, not words, we encode language through numbers.

Attention Mechanism

What makes Transformers unique is a special process called attention. It allows the numerical lists representing words to interact with each other. Word meanings are updated based on surrounding context — all in parallel.

Example: The number list for the word "bank" might shift based on context — adding a second meaning like "river bank" when the surrounding text calls for it.

Transformers also include a second type of computation called a feed-forward neural network, enabling the model to store more linguistic patterns learned during training.

How Information Flows

All of this information flows through multiple repetitions of these two core computations (attention + feed-forward). Each round enriches the number lists, allowing them to encode new information the model may need to predict the next word accurately.

Finally, a last mathematical function is applied to the final list in the word sequence. This list, shaped by the input context and everything the model learned during training, produces a prediction — all possible next words with their probabilities.

Emergent Behavior

"Although researchers design the framework for each step, the model's overall behavior is an emergent phenomenon — arising from tuning hundreds of billions of weights during training."

This makes it extremely difficult to pinpoint exactly why a model makes any particular choice. What we do know is that when large language models are used to complete text, the words they produce are surprisingly fluent, impressive, and even useful.

For the Curious

If you're new to this and want to learn more about how Transformers and attention work, here are some resources:

→ A series on deep learning that explains attention and every other step in the Transformer in detail.

→ A recorded lecture delivered at TNG in Munich, available on a second channel — presented in a casual, conversational style.

Choose the format that suits you best.