Tool calling: schemas and structured output

Tool calling is the mechanism that lets a text model act on the world. The model cannot run anything — it emits a structured request naming a tool and its arguments, and your code decides what to do with that. Understanding the round trip precisely is worth more than any amount of prompt tuning.

The round trip

Four steps, every time:

1. You send tool schemas alongside your messages — name, description, and a JSON Schema for the arguments.
2. The model returns a tool call: a tool name, an arguments object, and a call ID.
3. Your code executes it and produces a result.
4. You append the result, tagged with that call ID, and call again.

The call ID is the part people skip and then debug for an hour. With parallel tool calls, the model may request three tools at once, and the results can come back in any order. The ID is what pairs each result to its request. Mismatch them and the model silently reasons over the wrong data — no error, just wrong answers.

Your schema is a prompt

This is the idea worth taking away. The schema does not merely validate arguments after the fact — it is read by the model beforehand and shapes what it decides to do. Every field name, every description, every enum is prompt real estate.

{
  "name": "search_orders",
  "description": "Find orders by customer email. Returns at most
                  20 matches, newest first. Use when the user
                  refers to a past purchase. Does NOT search
                  by order ID - use get_order for that.",
  "parameters": {
    "type": "object",
    "properties": {
      "email": {
        "type": "string",
        "description": "Exact customer email. Not a partial match."
      },
      "status": {
        "type": "string",
        "enum": ["pending", "shipped", "delivered", "cancelled"],
        "description": "Optional filter. Omit to search all."
      }
    },
    "required": ["email"]
  }
}

Note what the description is doing. It states what the tool returns, when to reach for it, and — critically — when not to, with a pointer to the right alternative. That last clause prevents more misfires than any system-prompt instruction, because it sits directly where the model is making the choice.

Four schema decisions that change behavior

Enums over free strings. An enum constrains the output at the decoding level on most providers. "status": "string" gets you "Shipped", "in transit", and "SHIPPED". An enum gets you one of four values, every time.

Required fields force a decision. Marking a field required means the model must produce it — which is right when it is genuinely necessary, and wrong when it is not, because the model will invent a plausible value rather than omit it. Optional-with-a-good-description beats required-and-guessed.

Flat beats nested. Deeply nested argument objects produce more malformed calls than flat ones. If you find yourself three levels deep, that is usually a sign the tool is doing too many jobs.

Name for the model, not your codebase. search_orders is a better tool name than OrderRepositoryQueryHandler. The model is pattern-matching on names; make them describe the action in plain words.

Parallel calls

Most current models can request several tools in one response. This is a large latency win — three independent lookups in one round trip instead of three — but it introduces two requirements.

First, every requested call must get a result appended before the next model call. Skip one and the conversation is malformed; most providers will reject it outright.

Second, the tools must actually be independent. The model cannot know that create_user has to finish before assign_role, and it will cheerfully request both at once. If ordering matters, either enforce it in your executor or design the tools so it cannot come up — one tool that does both is often the honest answer.

When the call comes back wrong

It will. Arguments that miss the schema, hallucinated tool names, valid-but-nonsensical values. The reflex is to catch it and retry silently; the better move is usually to append the validation error as the tool result and let the model correct itself. It reads the error, sees what it did, and fixes it — typically on the first attempt, and at a fraction of the cost of a full retry.

That repair loop has limits and failure modes of its own, which is the next page.