Context Window

This unlocks the rest of your AI journey. Every large language model (LLM) you will ever use — ChatGPT, Claude, Gemini, or the open-weight models you might run yourself — has a context window. Understanding it tells you why a chatbot forgets what you said ten messages ago, why a long PDF might get summarised badly even though the model 'read' it, why costs rise with every word you paste in, and why small models on your phone behave differently from cloud giants. This single concept powers prompt engineering, retrieval-augmented generation (RAG), agentic systems that plan and call tools, and model selection itself. Master it now, and everything downstream becomes clearer.

Think of the context window as the model's entire working memory — but a very strange one. When you send a prompt to a model, the model does not remember anything else. It has no file cabinet, no notes from yesterday, no access to the internet. The only thing it can see is the text you give it in this one call. The context window is the maximum number of tokens (roughly, pieces of words) that the model can attend to in that single call. Nothing outside that window exists to the model, full stop. The model does not know about your earlier conversations, your other documents, or the fact that you fed it a 200-page book — unless the relevant portions fit within the window. Imagine reading a book but being allowed to hold only a page at a time in your hands. You can read that page perfectly, but you cannot flip back to check a name on page 5. The page in your hands is your context window. If the book is longer than the window, you can only work with the last portion you were given. This is why long conversations seem to make models 'forget' early details — those early tokens have fallen outside the window. It is also why a model can produce a confident answer based on only the 2,000 tokens you pasted, when the full document was 50,000 tokens. The window is a budget: more tokens in the window means more compute time (latency) and higher cost. But a bigger window does not mean the model uses it well. The model sees all of it, but it may not pay attention to the middle of a very long input — a known weakness. So the context window is both a limit and a management problem.

Imagine you are a busy chef in a tiny kitchen. Your countertop is the context window. You have one counter, and it can hold only so many ingredients and tools at once — say, enough for one dish. The recipe book (your whole knowledge) lives in the pantry, but you cannot bring the whole book to the counter. You bring out the recipe page you are using, the vegetables, the spices, the pan. Everything else stays in storage, out of reach. If a customer asks for a dish that requires an ingredient you stored in the back, you cannot cook it unless you have room on the counter to fetch it. And if you try to work with a pile of thirty ingredients, you will spend so long finding the right one that the dish slows down (latency) and you may knock things over (errors). The counter is your budget. Now, where does the analogy break? First, a chef can remember a few things without looking at the counter — a recipe she has cooked a hundred times. A model has no such memory outside the window; everything is gone unless it is literally in the input. Second, a chef can glance at the counter and immediately see everything laid out. A model does not 'see' its entire window with equal ease — it struggles with the middle, especially when the window is very long. Third, the counter can be made bigger (models with larger context windows exist), but a bigger counter does not make the chef smarter; it just lets her spread out more ingredients, which can be overwhelming. So the analogy holds for the core idea — a fixed working space that limits what you can work with — but the model's lack of any memory beyond the counter is the critical difference.

The context window is the maximum number of tokens a model can attend to in a single call — its entire working memory for that call.

You have not yet built anything in this course, so this is your first building block. Context window is a property of every large language model (LLM), and it builds on the transformer architecture and the attention mechanism — topics you will meet soon. It also connects directly to topics you will see in your library: tokenization (because the window is measured in tokens, not words), sliding window sampling (a training technique that shapes how models learn from long sequences), and the quality-cost-latency tradeoff (because a bigger window costs more and runs slower).