Explainer

What is an LLM? Large language models explained simply

ChatGPT, Claude and Gemini are all built on large language models. Here’s what an LLM actually is, how it learns, and why that explains both its strengths and its mistakes.

Key points

  • A large language model (LLM) is a neural network trained on huge amounts of text to predict the next piece of text, called a token.
  • Most modern LLMs use the transformer architecture, introduced by Google researchers in 2017.
  • Chatbots like ChatGPT, Claude and Gemini wrap an LLM with extra training to follow instructions, plus tools like web search.
  • Because LLMs generate plausible text rather than look up facts, they can be confidently wrong.

The short answer

A large language model is a computer program that has learned patterns in language by processing an enormous amount of text. Give it some text, and it predicts what text is likely to come next. Repeat that prediction over and over — one small chunk at a time — and you get paragraphs, emails, code and conversations.

“Large” refers to two things: the size of the training data (a large share of the public text available, plus licensed and other data, depending on the company) and the number of internal settings, called parameters, which can run into the billions or more. Those parameters are adjusted during training so the model gets better at prediction.

How an LLM is built, in three stages

1. Pre-training: learning from text

The model reads vast amounts of text, and for each piece it tries to predict the next token. When it’s wrong, its parameters are nudged slightly to make the right answer more likely next time. After trillions of these tiny adjustments, the model has absorbed grammar, facts, styles of writing, programming languages and patterns of reasoning — all as statistical relationships rather than a database of facts. The 2020 GPT-3 paper showed that models trained this way at large scale could perform many tasks from just a few examples in the prompt, without task-specific training.

2. Instruction tuning and feedback

A pre-trained model is good at continuing text but not at being a helpful assistant. Developers therefore train it further on examples of instructions and good responses, and use human feedback to reward helpful, honest and harmless answers. OpenAI’s 2022 “InstructGPT” paper described this approach, often called reinforcement learning from human feedback (RLHF), and found that people preferred the outputs of the tuned model over a much larger untuned one.

3. The product layer

What you use in an app is more than the model. Chatbot products add a system prompt with instructions, safety filters, memory features, the ability to search the web or run code, file upload and so on. That’s why the same underlying model can behave differently in different apps.

Tokens: how an LLM “sees” text

LLMs don’t read letters or whole words. They split text into tokens — common words, parts of words, punctuation and spaces. OpenAI’s documentation gives a rule of thumb that one token is roughly four characters or three-quarters of a word in English. Everything is measured in tokens: how much you can paste in, how long an answer can be and how much API usage costs. Our explainer What are tokens? goes deeper, and the token counter lets you estimate your own text.

The transformer and “attention”

Most modern LLMs are based on the transformer, described in the 2017 paper “Attention Is All You Need” by Vaswani and colleagues. Its key idea is attention: when processing each token, the model weighs how relevant every other token in the input is. In the sentence “The trophy didn’t fit in the suitcase because it was too big”, attention helps the model connect “it” with “trophy”. Transformers can process all tokens in parallel during training, which made it practical to train on far more data than earlier designs.

How a chatbot answer is generated

  1. Your message, the conversation so far and the app’s hidden instructions are turned into tokens.
  2. The model calculates a probability for every possible next token.
  3. One token is chosen — usually one of the most likely, with a little randomness so answers aren’t identical every time.
  4. That token is added to the input, and the process repeats until the answer is complete.

This loop explains the typewriter effect you see in chat apps: the answer really is being produced one token at a time.

What LLMs are good at

  • Language tasks: drafting, rewriting, summarising, translating and adjusting tone.
  • Explaining: breaking a concept down at different levels, with examples.
  • Structure: turning messy notes into tables, outlines or checklists.
  • Code: writing, explaining and reviewing code in many languages.
  • Brainstorming: generating options quickly for you to judge.

What LLMs are bad at (or need help with)

  • Guaranteed facts. An LLM produces plausible text, not verified truth. It can invent details, citations and numbers. See Why AI hallucinates.
  • Recent events. Knowledge stops at the training cutoff unless the app adds web search.
  • Your private context. It knows nothing about your company, your class or your life unless you tell it — which is why good prompts include context.
  • Exact calculation. Many chatbots now run code to calculate, but a model reasoning “in its head” can still slip on arithmetic.
  • Very long inputs. Even with large context windows, research has found that models can use information in the middle of long inputs less reliably than information at the start or end.

Today’s major model families

As of November 2026, the best-known LLM families include OpenAI’s GPT models (used in ChatGPT), Anthropic’s Claude models, Google’s Gemini models, and open-weight families that anyone can download and run, such as DeepSeek’s models. Companies typically offer several sizes: large flagship models for the hardest tasks and smaller, faster, cheaper models for simple ones. Anthropic’s documentation, for example, lists its current models with context windows of one million tokens. Because model names and versions change every few months, we keep current model and price details in our API cost calculator with dated sources rather than in this explainer.

Some models are “closed” — available only through the company’s app or API — while others are “open-weight”, meaning the trained parameters are published so you can run them yourself. Our guide to open-source AI models explains the difference and why it matters for privacy and cost.

Why this matters for everyday users

Understanding that an LLM predicts plausible text — rather than retrieving verified facts — changes how you use it. It’s a superb drafting partner, tutor and brainstormer. It’s an unreliable encyclopedia unless it can search and cite sources, and even then you should check. Give it context, ask for its reasoning, verify what matters, and you’ll get the benefits while avoiding the traps.

Editor’s note — add your own experience: Add a short personal paragraph: the moment LLMs “clicked” for you, or a mistake an LLM made that taught you something about how they work.
Remove this box by adding your content here, or hide all notes with "showEditorNotes": false in site.config.json.

Frequently asked questions

Is ChatGPT an LLM?

ChatGPT is a chatbot app built on top of OpenAI’s large language models. The app adds a chat interface, memory, web search, file handling and safety systems around the underlying model.

Does an LLM understand what it says?

That’s debated. LLMs clearly capture a lot of structure about language and the world, but they work by predicting likely next tokens, not by checking facts against reality. That’s why they can be fluent and wrong at the same time.

Does an LLM learn from my conversations?

Not in real time. A model’s knowledge is fixed when training ends. Some providers may use conversations to train future models depending on your privacy settings — see our AI privacy guide.

Why do LLMs have a knowledge cutoff?

Training uses a snapshot of data up to a certain date. Anything after that isn’t in the model unless the chatbot looks it up, for example with web search.

What’s the difference between an LLM and AI in general?

AI is the broad field. LLMs are one type of AI model specialised in language. Image generators, recommendation systems and self-driving software are other kinds of AI.

Sources

  1. Vaswani et al. (2017), Attention Is All You Need (opens in a new tab)
  2. Brown et al. (2020), Language Models are Few-Shot Learners (opens in a new tab)
  3. Ouyang et al. (2022), Training language models to follow instructions with human feedback (opens in a new tab)
  4. Liu et al. (2023), Lost in the Middle: How Language Models Use Long Contexts (opens in a new tab)
  5. Wikipedia — Large language model (opens in a new tab)
  6. OpenAI — Key concepts (opens in a new tab)
  7. Anthropic — Models overview (opens in a new tab)

Facts, prices and features were checked against these sources on November 19, 2026. AI products change quickly; if you spot something out of date, please tell us.