Explainer

Context windows explained: how much can an AI chatbot remember?

Why does a chatbot forget what you said an hour ago, or miss a detail in a long PDF? The answer is the context window — and how models use it.

Key points

  • The context window is the maximum amount of text — measured in tokens — that a model can consider at once.
  • It includes everything: hidden instructions, your messages, the AI’s replies and any files.
  • Many current flagship API models list one-million-token windows; consumer apps may use different limits.
  • Bigger isn’t automatically better: models can overlook details buried in very long inputs.

What is a context window?

Think of the context window as the model’s working desk. Everything the model uses to write its next reply has to fit on that desk: the app’s system instructions, the whole conversation so far, any documents you’ve uploaded, and the reply it’s writing. The size of the desk is measured in tokens, not words or pages. Anything that doesn’t fit isn’t seen.

The model doesn’t “remember” earlier conversations by itself. Each time you send a message, the app sends the relevant conversation history back to the model as part of the context. That’s why long conversations eventually run into limits.

How big are context windows today?

Context windows have grown dramatically. As of our check on November 19, 2026:

  • Anthropic lists a context window of 1M tokens for its current Claude models, including Claude Opus 5.5, Sonnet 5.5 and Haiku 5.5, with maximum outputs of 128K tokens.
  • Google lists 1M-token context for its Gemini 4 models, and an input token limit of 1,048,576 for models such as Gemini 3.8 Flash.
  • OpenAI’s pricing page distinguishes “short context” (up to 272K input tokens) from “long context” for several models, with higher prices for long context.

Using OpenAI’s rule of thumb (about 0.75 words per token), a million tokens is roughly 750,000 English words. That’s enough for several books, a large codebase or hundreds of reports. These are API figures; free and paid chat apps may apply their own limits that providers don’t always publish.

What happens when the window fills up

When a conversation exceeds what the app allows, something has to give. Depending on the app, older messages may be dropped, automatically summarised, or the app may ask you to start a new chat. Either way, details from early in the conversation can be lost. Symptoms include the chatbot contradicting an earlier decision, forgetting a constraint you set at the start, or asking for information you already gave.

Bigger isn’t automatically better

A large window means the model can see a lot of text, not that it uses all of it equally well. The 2023 study “Lost in the Middle” by Nelson Liu and colleagues found that language models performed best when relevant information was at the beginning or end of a long input, and noticeably worse when it was in the middle. Models have improved since, but the practical lesson holds: the more you put in, the more likely a detail gets underweighted.

Long inputs are also slower and, through APIs, more expensive — every token you send is billed, and some providers charge more beyond a length threshold.

Practical tips

For long documents

  • Put your question after the document and repeat the key instruction at the end; providers’ prompting guides often suggest placing long documents first and the query last.
  • Ask for quotes. “Quote the passage that supports each point” makes the model locate evidence and lets you check it.
  • Split big jobs. Summarise each chapter separately, then combine the summaries.
  • Paste only what’s relevant when you know which section matters.

For long conversations

  • Start fresh for new topics. A clean context gives better answers than one cluttered with an unrelated discussion.
  • Carry over a summary. Ask: “Summarise the key decisions and constraints from this chat in 10 bullet points,” then paste that into a new chat.
  • Restate important constraints when you notice drift.

For developers

  • Track token usage per request; long histories quietly multiply costs.
  • Use caching for repeated prefixes such as long system prompts, where your provider supports it.
  • Consider retrieval (RAG) instead of sending whole collections every time.

Context window vs memory features

Many chatbots now have “memory” features that save facts about you between conversations — your name, your preferences, your projects. That’s different from the context window: memories are stored separately and selectively inserted into new conversations. You can usually view and delete them in settings. If privacy matters, check what’s stored; see our AI privacy guide.

Worked example

You upload a 120-page contract (roughly 60,000 words, or about 80,000 tokens by OpenAI’s rule of thumb) and ask about termination clauses. That fits comfortably within a one-million-token window. But a clause on page 64 that modifies the termination rules might still be overlooked if you ask a broad question. A better prompt: “List every clause that affects termination, including clauses elsewhere that modify it. Quote each one with its section number.” Then read those sections yourself — and for legal decisions, have a lawyer review them.

Common misconceptions

  • “A bigger window means the AI knows more.” No — the context window is about how much text it can consider right now, not what it learned in training.
  • “If it fits, the AI read every word carefully.” Not necessarily. Models can underweight details in very long inputs, so ask targeted questions and request quotes.
  • “The chatbot remembers everything I’ve ever told it.” Only what’s in the current conversation, plus anything saved by a separate memory feature.
  • “Context limits are the same in every app.” Providers publish API limits; apps may differ by plan.

The practical rule is simple: give the model what it needs, no more, and put the most important instruction where it can’t be missed — at the start, and again at the end of a long prompt.

Editor’s note — add your own experience: Test one long PDF (50+ pages) in two chatbots with the same detailed question about something in the middle of the document. Report whether each found it and quoted it correctly.
Remove this box by adding your content here, or hide all notes with "showEditorNotes": false in site.config.json.

Estimate before you paste

Not sure how much of a window your text uses? Paste it into our token counter to see an estimate and the share of a one-million-token window it would occupy.

Frequently asked questions

How many pages is one million tokens?

Using OpenAI’s rule of thumb of about 0.75 words per token, one million tokens is roughly 750,000 English words — several long novels. Real capacity depends on the language and formatting.

Is the context window the same as chatbot “memory”?

Not exactly. The context window is what the model can see in the current conversation. Many apps also have a separate “memory” feature that saves facts about you across conversations.

Why did the chatbot forget something I said earlier?

The conversation may have exceeded the app’s limit, so older parts were dropped or summarised, or the model simply didn’t give enough weight to an early detail in a long chat. Restate key facts or start a new chat with a summary.

Do bigger context windows cost more?

Through APIs, you pay for every input token, so filling a large window costs more per request. Some providers also charge higher rates above a length threshold — OpenAI lists separate long-context prices above 272K input tokens for some models.

Is the app’s context window the same as the API’s?

Not necessarily. Providers publish API context windows; consumer apps may apply different limits by plan, and these aren’t always published.

How do I continue a long conversation in a new chat?

Ask the chatbot to summarise the key facts, decisions and open questions in a short list, then paste that summary into a new chat as your starting context.

Sources

  1. Anthropic — Models overview (context windows) (opens in a new tab)
  2. Google — Gemini models (input token limits) (opens in a new tab)
  3. OpenAI API pricing (short vs long context pricing) (opens in a new tab)
  4. Liu et al. (2023), Lost in the Middle: How Language Models Use Long Contexts (opens in a new tab)
  5. Google — Understand and count tokens (opens in a new tab)

Facts, prices and features were checked against these sources on November 19, 2026. AI products change quickly; if you spot something out of date, please tell us.