Anatomy of the agent loop

One turn of an agent loop has five phases. Almost every agent bug you will chase lives in a specific one of them, and being able to name the phase is most of the diagnosis.

  1. ASSEMBLE   build the messages array
  2. CALL       send it, get a response
  3. DECIDE     tool calls, or done?
  4. EXECUTE    run the requested tools
  5. APPEND     put results back in the array
       |
       +------- back to 1

1. Assemble

You construct the input: system prompt, conversation so far, tool schemas, and whatever context you are injecting this turn. This phase is where the agent is actually designed. Everything downstream is mechanical.

Order matters more than people expect. Stable content belongs at the top — system prompt, tool definitions, unchanging reference material — because prompt caching keys on an exact prefix, and anything volatile near the top invalidates the cache for everything after it. Volatile content belongs at the bottom, where the model also weights it most heavily.

2. Call

The network call. The only phase that is not your logic, and the one that fails in ordinary ways: rate limits, timeouts, truncated output because you hit the max-tokens ceiling mid-response.

That last one deserves attention. A response cut off at the token limit can end mid-tool-call, producing a malformed request that looks like a model failure but is a configuration failure. Check the finish reason on every response. If it is length rather than a natural stop, you have a truncation problem, not a reasoning problem.

3. Decide

Branch on the response. Tool calls present means continue; none means the model considers itself finished. This is the phase where you enforce your own limits — step count, token budget, wall-clock — rather than trusting the model to wind down on its own.

It is also where repetition should be caught. An agent calling the same tool with identical arguments three times running is not making progress, and it will happily do it forty more times. Compare the current call against recent history and break the loop yourself.

4. Execute

Run what was asked. Two rules matter here, and both get learned the hard way.

Never let a tool raise into the loop. An unhandled exception kills a run that the model could have recovered from. Catch everything and return the error as a string result — the model reads it, understands it failed, and tries something else. A tool that returns "FileNotFoundError: config.yaml" is more useful than a stack trace that ends the process.

Validate arguments before executing. The model produces arguments that satisfy your schema but not your intent — a path outside the working directory, a limit of 1,000,000, an ID that does not exist. The schema is a shape check, not a safety check.

5. Append

Results go back into the array, and the format of what you append is a prompt-engineering decision disguised as plumbing.

Raw output is often the wrong thing to append. A tool returning 200KB of JSON burns your window and buries the signal. A tool returning nothing on success teaches the model that nothing happened. What you want is the smallest representation that lets the model take the next step correctly — often a summary, a count plus the first few rows, or a plain confirmation string.

Reading a real trace

Log every turn as a four-line record and most debugging becomes reading:

turn 3  in:4,812 tok  out:96 tok  stop:tool_use
  -> read_file(path="src/config.py")
  <- 1,204 chars
  elapsed 2.1s

From that you can see cost accumulating, whether the model is making progress or thrashing, which tool is slow, and where the run went sideways. Build this before you build anything clever — it pays for itself on the first real bug.