Models

A model is the generative core of an agent. It takes the current context and predicts a continuation one token at a time. It holds no memory of earlier calls and no knowledge of the world beyond its context. Everything else in the control panel exists to feed the model the right context and to check what it produces.

The Model Holds No State

Each request stands on its own. The model forgets a previous session unless you place that information in the input again. Durable state therefore belongs in files, storage, and run records, not in the conversation. You can resume a written-down context later or hand it to another person, and it behaves much the same way, because nothing hides inside the model itself.

This makes the context the one input worth managing closely. The instructions that apply, the current requirement, and the evidence a step needs belong in it; finished investigations and stale output do not. A larger context window raises the capacity limit, but capacity is not the same as attention, and the model does not necessarily use what is present.

Attention Is A Fixed Budget

A large language model (LLM) weighs every token in its context when it predicts the next one. More material means less weight for any single instruction. A rule the model follows in a short session can slip once a large unrelated document enters the context. Models also tend to use material near the start and end of a long context more reliably than material in the middle.

Statelessness turns this limit into a lever. You choose what the model sees on every call. A fresh session with a lean context often beats a long session that holds more information. The methods in context management follow from this one property.

Output Is Probabilistic

The same prompt can produce different results on different calls. Sampling settings such as temperature widen or narrow that variation, but the generation itself stays probabilistic. Some work has one correct answer for a given input: validating a pipeline, comparing step order, or reading a process exit code. That work belongs in ordinary code and returns the same result every time. Reserve the model for the work that code cannot do, and check its output rather than trust how it reads.

Models Tend To Agree

Training on human ratings rewarded answers that people liked, and people tended to like agreement. A model may therefore drop a correct answer when the user simply asks whether it is sure. Plain pushback tells you little about whether an answer holds.

Challenge an answer with a new argument or new evidence instead. Phrase the challenge as a question. Ask the model to ground its reasoning in code or data you can check. Asking why it chose an approach costs little and often exposes a weak assumption.

Capability Is Not Reliability

A newer or larger model is not automatically the better choice for a task. It may follow more instructions at once, or it may cost more and answer more slowly for no measurable gain. Under pressure to finish, a model can drop a requirement without mentioning it, or even describe a success it never reached. To choose between models, run them on the real task against checks you write in advance. Record the version too, because the same name can behave differently from one release to the next.

Match The Model To The Task

No single model is the best LLM for coding on every task. A stronger model suits hard or uncertain work such as planning a change that spans several modules. A smaller model often handles a well-defined step just as well for less money. One pipeline can use both. It can also give review to a different model from the one that wrote the code.

Benchmarks and context window sizes give a starting point. They do not show how a model behaves on your repository. Build that judgment from repeated runs on your own tasks and keep the results in an evaluation record.

Exercise 3: Push Back Without Evidence

Ask a model how one specific function in a working project handles invalid input. Then ask again. Repeat the same question in five fresh sessions, save each answer, and check all five against the code. Note where the answers disagree and which ones are wrong.

Take an answer the code shows to be correct and reply with nothing but “Are you sure?” Record whether the model changes its answer. In another fresh session, ask the original question again and challenge the answer with an argument drawn from the code, phrased as a question. The two replies show the gap between a model yielding to pressure and a model checking a claim.

September, 2026