A context window is the maximum amount of text, measured in tokens, that an AI model can work with in a single request: your instructions, the conversation so far, any attached documents and the model’s own reply all have to fit inside it. Anything outside the window does not exist for the model. As of October 2026 the flagship models from Anthropic, OpenAI and Google all advertise windows of roughly one million tokens, but a bigger window does not guarantee better answers, and filling it costs money and time.
This guide explains what fills a context window, how the current models compare according to their providers’ documentation, how to budget a window with a worked example, and what to do when a model starts “forgetting” things.
Tokens: the unit the window is measured in
A token is a chunk of text that a model reads and writes as one unit: often a whole short word, sometimes part of a longer word, a punctuation mark or a space. Models never see characters or words directly; a tokenizer converts text into token IDs first. The full mechanics are covered in this guide to what a token is and how tokenization works.
For planning, the providers publish rough conversions for English text:
- OpenAI’s help center puts one token at about four characters, or about three quarters of a word, so 100 tokens is roughly 75 words.
- Google’s Gemini documentation says 100 tokens is about 60 to 80 English words.
- Other languages usually need more tokens for the same meaning. OpenAI notes that non-English text often has a higher token-to-character ratio, which also makes it more expensive.
- Images, audio and video are tokenized too. Gemini’s docs, for example, count 258 tokens for a small image (each 768×768 tile of a larger one costs the same) and 32 tokens per second of audio.
These are averages. Code, tables, URLs and unusual vocabulary tokenize less efficiently than plain prose, so use each provider’s token-counting tool when the number matters.
What actually fills the window
Everything sent with a request counts against the context window, not just the message you typed. Anthropic’s documentation lists the system prompt, every message in the conversation (including tool results, images and documents) and the tool definitions. The model’s output counts too, and so do reasoning tokens: OpenAI’s conversation-state guide names input, output and reasoning tokens as the three components of the total.
In a chat app this happens invisibly. Every time you send a new message, the app re-sends the whole conversation so far, because the model itself keeps no state between requests. A long chat therefore grows its token usage with every turn, and so does an agent that reads files and calls tools, since each tool result is appended to the history.
Two details trip people up:
- Cached tokens still count. Prompt caching lowers the price of re-sending the same prefix, but Anthropic’s docs are explicit that cached tokens still occupy the window. Caching changes cost, not capacity.
- Output has its own cap. A model with a 1M-token window cannot write a 1M-token answer. Each provider sets a separate maximum output, typically tens of thousands of tokens.
Context windows of current models (October 2026)
The table below is taken from each provider’s model documentation as checked on 24 October 2026. Providers change these figures and their pricing rules often, so treat it as a snapshot and check the linked docs before you build on it.
| Model (API ID) | Context window | Max output | Long-prompt pricing rule |
|---|---|---|---|
| Claude Opus 5.5 (claude-opus-5-5) | 1M tokens | 128K | Same per-token rate across the full window |
| Claude Sonnet 5.5 (claude-sonnet-5-5) | 1M tokens | 128K | Same per-token rate across the full window |
| Claude Haiku 5.5 (claude-haiku-5-5) | 1M tokens | 128K | Higher rates for prompts over 100,000 tokens |
| GPT-6 Astra (gpt-6-astra) | 1,050,000 tokens | 128K | Over 272K input tokens: 2x input, 1.5x output |
| GPT-6.1 Sol (gpt-6.1-sol) | 1,050,000 tokens | 128K | Over 272K input tokens: 2x input, 1.5x output |
| GPT-6 Luna (gpt-6-luna) | 1,050,000 tokens | 128K | Over 272K input tokens: 2x input, 1.5x output |
| Gemini 4 Argon (gemini-4-argon) | 1,048,576 tokens, shared by input, thinking and output | 65,536 by default; more with long decode | No prompt-size tier listed |
| Gemini 3.8 Flash (gemini-3.8-flash) | 1,048,576 input tokens | 65,536 | No prompt-size tier listed |
The pricing column matters as much as the size column. On models with a long-prompt tier, crossing the threshold changes the price of the whole request, not just the tokens above it. For per-token rates and caching discounts side by side, see the context caching pricing comparison for Claude, GPT and Gemini.
Consumer chat apps may not give you the full API window. Plan limits, file-upload handling and automatic summarization in apps like ChatGPT, Claude and Gemini are set separately from the API figures above.
Worked example: budgeting a context window
Budgeting a window means adding up everything a request will carry before you send it. The scenario below is illustrative, with round numbers estimated from the rules of thumb above, not measured token counts.
A procurement analyst wants a model to compare a 90,000-word supplier contract against a 6,000-word internal policy, then answer follow-up questions in the same chat.
| Item | Estimate | How it was estimated |
|---|---|---|
| System prompt and instructions | about 1,500 tokens | roughly 1,100 words |
| Contract | about 120,000 tokens | 90,000 words at about 0.75 words per token |
| Internal policy | about 8,000 tokens | 6,000 words at the same ratio |
| First answer | up to 4,000 tokens | the max output you allow |
| Ten follow-up turns | about 20,000 tokens | questions plus answers of around 2,000 tokens each |
| Total by the last turn | about 153,500 tokens |
Three conclusions follow from that arithmetic:
- It fits comfortably in any of the 1M-token models above, so size is not the constraint.
- Cost is. Because the whole history is re-sent each turn, the contract is billed about eleven times across the session (once per request). Prompt caching on the contract prefix is the obvious fix.
- The pricing tier matters for some models. On Claude Haiku 5.5, every one of these requests is above the 100,000-token threshold and pays the higher rate. On the GPT-6 models the requests stay below 272K, so the standard rate applies. Sending only the relevant contract sections would change both answers.
Why a bigger window does not mean better answers
Models do not use every part of a long context equally well, which is why the size of the window is a ceiling, not a promise. Anthropic’s context-window documentation names the effect: as token count grows, accuracy and recall degrade, which it calls “context rot”. Its conclusion is that curating what goes into the context matters as much as how much space is available.
Independent research points the same way. The 2023 “Lost in the Middle” paper by Nelson Liu and colleagues found that models performed best when the relevant information sat near the beginning or end of a long input and noticeably worse when it was in the middle, even for models built for long contexts. Google’s long-context guide adds a different caveat: Gemini’s strong results on single “needle in a haystack” searches do not carry over equally when you need to pull out many separate facts at once.
The practical consequences:
- Put the question last. Google’s guide recommends placing your query at the end of the prompt, after the long material.
- Put critical instructions where they will be seen. A rule buried halfway through a long system prompt can be skipped. Restating a key rule near the end of a long prompt is cheap insurance.
- Send excerpts, not archives. A focused 20-page extract usually beats a 500-page dump. When the material is too large to curate by hand, retrieval is the tool: retrieval-augmented generation finds the relevant passages and sends only those.
- Expect slower responses. Google notes that longer inputs generally increase the time to the first token.
What happens when you hit the limit
Exceeding a context window produces an error or a cut-off answer at the API level, and silent forgetting in many chat apps. The exact behavior depends on the provider:
- Anthropic: if the input alone is larger than the window, the API rejects the request with a “prompt is too long” error. On Claude 4.5 and newer models, if the input fits but the answer runs out of room, generation stops with the stop reason
model_context_window_exceeded. - OpenAI: the conversation-state guide warns that tokens generated beyond the limit may be truncated in the response.
- Google: for Gemini 4, where input and output share one budget, the output is capped to whatever space the input leaves.
A truncated answer can look complete at a glance, so production code should always check the stop or finish reason. This guide to stop_reason and finish_reason across Claude, OpenAI and Gemini lists the values to watch for.
Chat apps usually avoid hard errors by trimming or summarizing older turns behind the scenes. That is why a long conversation can “forget” a constraint you set at the start: the original message may no longer be in the window, or may survive only as a short summary.
Keeping long sessions inside the window
Managing a context window means deciding what stays in, what gets summarized and what gets dropped, rather than letting the app decide for you. These techniques work in both chat and API settings:
- Start fresh with a summary. When a chat gets long, ask the model to summarize decisions, constraints and open questions, then paste that summary into a new conversation.
- Restate key constraints. If a rule matters for the whole session, repeat it near the end of your latest message instead of trusting the first turn to survive.
- Clear what is finished. In agent workflows, old tool results are often the largest and least useful part of the history. Anthropic offers context editing to clear old tool results, and both Anthropic and OpenAI document server-side compaction that summarizes earlier turns automatically.
- Count before you send. Use the token-counting endpoint or tokenizer for your provider in any pipeline that handles user-supplied documents.
Here is a summary prompt you can paste at the end of a long chat before moving to a new one:
We are about to continue this work in a new conversation.
Write a handover note I can paste there. Include:
1. The goal of this work in one sentence.
2. Every decision we made, with the reason in a few words.
3. Constraints and preferences I stated (format, tone,
things to avoid), quoted exactly where possible.
4. Facts and figures we established, with where they came from.
5. Open questions and the next step.
Do not include anything we discussed and then rejected,
unless the rejection itself is a decision worth keeping.
Keep it under 400 words.Items 2 and 3 carry most of the value: decisions and stated constraints are exactly what tends to get lost when a chat app trims old turns. The word limit stops the handover note from becoming another long context. For more on running multi-turn sessions, see how to keep long AI conversations on track.
Is the context window the same as memory?
No. Anthropic’s docs describe the context window as the model’s working memory for a request, separate from the knowledge in its training data. “Memory” features in chat apps are something else again: the app stores notes about you and inserts the relevant ones into the context window of future conversations. Those notes still consume tokens when they are loaded, and they only help if the app chooses to include them.
