Context Windows
The context window is the model's working memory — every token it can see at once. Longer windows enable longer conversations and documents, but memory cost grows quadratically and inference cost grows linearly. KV cache makes multi-turn conversation practical. RAG makes unlimited document length practical.
The forgotten dog
In a very long chat, an AI can lose what you told it at the start.
It didn’t forget the way you forget. Something more mechanical happened.
It rereads everything
Every time it replies, the AI reads the whole conversation again from the top.
Your messages, its replies, anything you pasted. It all goes back in, every turn. It has no other memory of the chat.
A fixed window
There is a limit. It can only read so much at once, and that limit is its window.
The size depends on the model. When a chat outgrows it, the oldest parts are dropped or shortened.
Measured in tokens
Windows are measured in tokens, not words. Tap a text to see what it costs.
Every piece shown here takes one slot in the window.
In the window?
Which of these does the AI actually see while it answers?
Tap “In the window” or “Not in it”.
Longer isn’t free
A bigger input isn’t free. The more tokens it reads, the slower and costlier the answer.
Choose what goes in
So with long material, you choose what goes in: a short summary, or only the relevant parts.
Chat apps often summarise old messages for you. Picking the relevant parts automatically is the next lesson.
Only the window
The AI only sees what fits in its window, and the window is counted in tokens.
This is the context window. Anything outside it simply isn’t there for the model.