The model is stateless

The model remembers nothing. Not between conversations, not between turns, not between the tool call it made two seconds ago and the response you are about to send it. Every API call is the first call it has ever received.

Everything that looks like memory — the model referring to what you said earlier, following up on its own reasoning, remembering a file it read — is your code re-sending the entire history, every time. There is no session on the other end. There is a function, and you call it with a big array.

What actually happens on each call

Turn three of a conversation does not send turn three. It sends turns one, two, and three, concatenated, as one flat input:

Call 1  ->  [system, user]
Call 2  ->  [system, user, assistant, tool_result]
Call 3  ->  [system, user, assistant, tool_result, assistant, tool_result]

The array grows monotonically. The model reads the whole thing from the top, forms an opinion, emits tokens, and forgets. Continuity is a property of your array, not of the model.

The bugs that dissolve once you accept this

"It forgot what I told it." It did not forget. Either that message was never in the array you sent, or it was trimmed by a history-truncation step you may not have written yourself — many frameworks silently drop old turns to fit the window. Print the array. The message is missing.

"It keeps redoing work it already did." The result of the earlier work is not in the context, or it is in there in a form the model cannot recognize as a result. A tool that returns None on success and appends an empty string teaches the model nothing happened.

"It contradicts itself between turns." Nothing enforces consistency. The model is not consulting its earlier position; it is re-reading text and re-deciding. If consistency matters, the constraint has to be stated in the context on every call, not once at the start and then assumed.

"It worked in the playground but not in my app." The playground sent a different array than your app does. This is nearly always true and nearly always the answer.

What this costs you

Because the whole history is re-sent every turn, input tokens do not grow linearly with conversation length — they grow quadratically. A ten-turn conversation does not cost ten units. Turn one sends 1 unit, turn two sends 2, and so on, so ten turns costs roughly 55.

This is why a long agent run gets expensive in a way that surprises people, and why the last few turns of a long run cost far more than the first few. It is also why prompt caching exists, and why caching only helps if the beginning of your array stays byte-identical between calls — a timestamp injected at the top of your system prompt will quietly destroy your cache hit rate on every single call.

The useful consequence

Statelessness is not only a constraint. It means the context is entirely under your control, which makes agents far more debuggable than they first appear:

• Runs are reproducible. Save the array, replay it, get substantially the same behavior. You can put a failing run in a test.
• You can edit history. Nothing stops you rewriting an earlier turn before re-sending — summarizing a huge tool result, or removing a wrong turn so it stops influencing the model.
• You can fabricate it. A "conversation" can be assembled from scratch to set up exactly the state you want to test.
• There is no hidden state to chase. Whatever the model knows, you can print.

Once this clicks, agent debugging becomes an ordinary data problem: the input is a list of strings you constructed, and you can look at it.