Context Window

The context window is the information available to a model for a particular response. It can include instructions and conversation history, plus retrieved files and tool results. It may also hold generated reasoning or output, depending on the model and application.

Capacity And Usage Measure Different Things

A token represents a unit of text or other model input. It is not necessarily a word. Token counts vary by model and language, as well as content. The advertised context limit describes capacity; it does not promise the model will use every detail within that limit correctly.

Total usage accumulates across calls. A workflow can repeatedly send a 20,000-token input and generate 2,000 output tokens per call. Over ten calls that is roughly 220,000 input-plus-output tokens, while each call stays much smaller than that total. Provider accounting may also include cached input and reasoning tokens.

Longer Context Can Reduce Reliability

Context degradation describes declining task performance as a conversation grows long, noisy, or contradictory. Common symptoms include missed constraints and repeated investigations, plus continued reliance on an abandoned approach. There is no universal token count at which this begins.

The course demonstrations show why irrelevant material deserves attention. They do not establish a fixed threshold for every model. Measure the behavior that matters on the actual task and keep the required evidence available.

How Context Degradation Shows Up

This decline is called context degradation. A simple test makes it visible. Put one rule in the project instruction file: never use em dashes. The model follows it in a short session. Load the full text of a novel into the context first and the rule starts to slip. The rule stayed the same while its share of the model's attention shrank.

A style rule fails where you can see it. An architecture rule or a security constraint fails the same way out of sight. Task prompts and implementation plans are instructions too and decay along the same curve. Treat a missed simple rule as a warning about the complex ones.

Four Ways Context Goes Bad

Long sessions in an LLM context window go wrong in four distinct ways:

• Volume: attention spreads thin as the token count rises.
• Pollution: plausible but irrelevant material crowds out what matters, and near-duplicate functions confuse the model's choice.
• Poisoning: an error enters the context and later steps build on it as fact.
• Erosion: a lossy summary drops details and no one checks what went missing.

Each problem has its own response. Volume and pollution call for trimming. Poisoning calls for removing the bad turn or starting fresh, because patching around the error keeps it in view. Erosion calls for checking a summary against its source before relying on it.

Record Cost Without Inventing It

The control panel should distinguish current context usage from cumulative input and output tokens. It should also show monetary cost separately. Cost depends on the provider and model. Cache treatment and current pricing also affect it. Missing usage data should appear as unavailable rather than zero.

Identify the decision that more context supports before adding it. A failing assertion and its relevant function often help more than the complete test log.

Exercise 4: Watch A Rule Slip

Add one line to the instruction file of a working project: never use em dashes. In a fresh session, ask the agent to explain part of the codebase in a few paragraphs, then count the em dashes in its answer and note the token count.

Repeat the request in another fresh session, but first have the agent read a long document that has nothing to do with the task, such as the full text of a public domain novel. Count the em dashes again and record the token count. Run each version three times, because a single run can land either way by chance. The instruction file is identical in both versions, so any difference comes from what else filled the context.

September, 2026