A tool is code you already wrote, plus a declaration describing it. The model only ever sees the declaration.
Almost everything that goes wrong with tool calling follows from that one asymmetry. This page walks the mechanism end to end on a real recorded transcript. What a tool is. What actually crosses the wire. Who runs what. The live demo then lets you watch what happens when the sentence is a bad one.
Nothing about a tool is an AI concept
A tool can wrap anything you already have: an API call, a database query, an MCP operation, a step in a workflow. The six in this lab happen to be ordinary Python functions that read flight, passenger, maintenance, gate, and weather records for a fictional US airline. They worked before any model was involved and they would keep working if you deleted the model tomorrow.
What makes that code a tool is the declaration you write alongside it: a name, a description, and a schema for the parameters. That declaration is not documentation. Nobody on your team will ever read it. It is an interface, and it is the only one the model has.
Turn on Show the model's view below. What remains is genuinely everything it gets. Not the function body, not the docstring, not the data. It cannot check whether your sentence is true, and it will not notice when it is wrong.
- tools
- 6tools
- lines of Python
- 114lines of Python
- lines the model sees
- 0lines the model sees
- characters of prose it does see
- 2826characters of prose it does see
tools.py · 19 lines
def get_flight_status(flight_number: str) -> dict:
"""Return status, gate, departure time, delay, and tail number for a flight."""
flight = FLIGHTS.get(flight_number.upper())
if flight is None:
return {"status": "error",
"message": f"No flight found with number '{flight_number}'"}
return {
"status": "ok",
"flight_number": flight_number.upper(),
"flight_status": flight["status"],
"gate": flight["gate"],
"departure_time": flight["departure_time"],
"delay_minutes": flight["delay_minutes"],
"tail_number": flight["tail_number"],
"aircraft_type": flight["aircraft_type"],
"origin": flight["origin"],
"destination": flight["destination"],
}
tool declaration · 548 characters
{
"name": "get_flight_status",
"description": "Returns the current status (scheduled/delayed/departed/cancelled), the ASSIGNED gate, scheduled departure time, delay in minutes, and the tail number of the specific aircraft operating a flight, identified by its flight number (e.g. 'AA118'). Use this to answer questions about a flight's current or scheduled state, and to find which gate a flight is already departing from. This returns the gate the flight is assigned to - it does NOT find a new or empty gate. It is also the only way to get the tail number for a flight, which other tools need.",
"input_schema": {
"type": "object",
"properties": {
"flight_number": {
"type": "string",
"description": "Airline flight number, carrier code plus digits, e.g. 'AA118', 'UA455', 'DL290'."
}
},
"required": [
"flight_number"
]
}
}The model never runs your code
This is the part most explanations skip, and it is the part that matters for anyone responsible for what an agent is allowed to do. When a model calls a tool, it does not execute anything. It returns a block of JSON containing a name, some arguments, and an id.
That is a request. Your own loop reads it and decides what to do. Call the function. Refuse. Log it. Rate limit it. Stop and ask a human. Every side effect an agentic system has ever produced was a line of your code choosing to honour a request.
Step through a real exchange below. It is run 1 of the wrong-airport scenario, recorded against claude-sonnet-4-5-20250929. Watch for the gold step. That one is yours, not the model's.
- Your user
Should I be worried about James Okafor's flight?
Four questions, or the model guesses
Here is a useful test. Hand your tool declaration to a competent stranger with no access to the codebase and ask them to use it correctly on the first try. Whatever they have to ask you is what is missing. Four questions come up every time.
- 01
What does it return?
So the model can tell whether it now has the answer, or still needs another call.
…returns the status, the assigned gate, the departure time, and the tail number…
- 02
What does it identify things by?
This is the single highest value sentence you can write. A name can express what a tool does. It can never express what its argument should look like.
…identified by its 3-letter IATA code (e.g. 'SFO')…
- 03
When should it be used?
Relevance is not obvious. A flight can look perfectly healthy and still be at risk for a reason that lives in a different tool.
…use this whenever a question depends on conditions at the ORIGIN airport…
- 04
When should it NOT be used?
Tools sit next to near neighbours. Saying which one this is not is often faster than describing what it is.
…this returns the gate a flight is assigned to. It does NOT find an empty gate…
Question 02 turned out to carry almost all the weight. Every failure this lab was able to reproduce came from an argument, not from picking the wrong tool. The name and the schema both reach the model, but an argument format is precisely the thing a well chosen function name cannot tell you. The full experiment behind that claim, all 36 recorded runs of it, is in the repository. Watch it happen in the demo.
Retrieval fails soft. Tools fail hard.
Grounding a model in retrieved documents is read only and fails gently. Fetch the wrong passage and you get a worse answer, but nothing in the world changes. Grounding it in functions is different in kind. The wrong tool, or the right tool with the wrong argument, does something.
In this lab the worst case is a confusing answer, because every tool here only reads. In a system with write tools it is a cancelled booking or a reassigned gate. The error surface also compounds across steps, because each call's output becomes the next call's input.
Retrieval risk is about information quality. Tool calling risk is about information quality, action correctness, and sequencing.
Which is why the interface between your code and the model deserves more care than a rushed sentence, and why the rest of this lab is spent measuring exactly what that sentence is worth.
Run the demo