---
source: 'https://howaiworks.ai/glossary/function-calling'
section: glossary
title: Function Calling (Tool Calling)
description: >-
  The model never runs your function. It emits a structured request; your code
  decides whether to honour it, executes it, and hands the result back.
tags:
  - Tool Calling
  - AI Agents
  - LLM
  - API
  - Model Context Protocol
  - Developer Tools
category: Software Development
datePublished: '2025-08-29'
lastUpdated: '2026-07-24'
---

# Function Calling (Tool Calling)

> The model never runs your function. It emits a structured request; your code decides whether to honour it, executes it, and hands the result back.

## Definition

**The model never runs your function.** Function calling is the mechanism by which a
[language model](https://howaiworks.ai/glossary/large-language-model) *asks* your program to do something. You send the
model a list of tools — each one a name, a sentence of description, and a JSON Schema for its
arguments — and instead of replying in prose, the model can reply with a structured request:
`get_weather({"city": "Lisbon"})`. That request is text. The model has no network socket, no file
handle, no database connection and no shell. Your code receives the request, decides whether to
honour it, runs the actual function, and sends the return value back as another message — and only
then does the model write an answer.

Almost every confusing thing about the feature dissolves once that is clear. The model cannot
"accidentally delete the database", because it cannot delete anything; a program you wrote can,
after you read a JSON object and chose to pass it to your delete function. There is no sandbox to
configure, because the sandbox is your dispatch code, and its strength is whatever you put in it.
Equally, no amount of writing "only call this for internal users" in a tool description enforces
anything — descriptions are prompt text, and prompt text is a suggestion. The check has to live in
the function.

The naming is worth getting right because half the documentation you will read predates the other
half. OpenAI shipped the feature on **13 June 2023** for `gpt-4-0613` and `gpt-3.5-turbo-0613`, as
a `functions` parameter with a matching `function_call`. At DevDay on **6 November 2023** those
were superseded by `tools` and `tool_choice`, partly so that a single response could carry more
than one call. "Function calling" and "tool calling" now name the same mechanism, providers use
both, and a code sample using `functions=` is simply from 2023.

## How It Works

### Four messages, and only one of them does anything

A tool-calling turn is a conversation with a specific shape. Nothing about it is magic; it is four
messages and one `if` statement.

**First request.** You send the usual system and user messages *plus* an array of tool
definitions. A minimal one is a name, a description, and a JSON Schema:
`{"name": "get_weather", "description": "Current conditions for a city",
"parameters": {"type": "object", "properties": {"city": {"type": "string"}}, "required": ["city"]}}`.
The description matters more than the schema does — it is the only thing telling the model *when*
this tool is the right one, and it is read as prompt text, not as documentation.

**The model's reply.** If the model decides a tool is needed, the response comes back with a
`tool_calls` array instead of (or alongside) content, each entry carrying an id, a name, and an
arguments string that your code must parse. The response's finish reason says `tool_calls`. This
message is the entire output of the model for this turn. It has changed nothing in the world.

**Your executor.** Your code parses the arguments, looks the name up in a dispatch table, checks
that this caller is allowed to invoke that function with those arguments, and runs it. This step
is ordinary software. It is also the only place any policy exists.

**The result goes back as a message.** You append the model's own tool-call message to the
transcript, then a message with role `tool`, carrying the same `tool_call_id` and the return value
serialised as a string, and you send the whole transcript again. The API is stateless: the second
request repeats everything the first one contained, plus two more messages. The model reads the
result and writes the final answer — or requests another tool, and the loop goes round again.

**Termination is a model decision.** The loop ends when a reply arrives with no tool calls in it.
That is the only signal you get, which is why every production loop also carries a hard iteration
cap: a model that keeps re-requesting a tool that keeps failing will otherwise spin until it fills
the [context window](https://howaiworks.ai/glossary/context-window). What to do when the tool itself fails — a timeout,
a 500, or arguments that are perfectly valid JSON and still name a customer who does not exist —
is the subject of [error handling in AI systems](https://howaiworks.ai/glossary/error-handling), and the short version
is that the failure goes back to the model as a tool message like any other result.

### The arithmetic of a round trip

One tool call costs a **minimum of two model invocations**: one to emit the call, one to interpret
the result. A chain of *n* dependent calls costs *n* + 1. That is the single most useful number on
this page, because it converts directly into latency and cost.

| Task | Model invocations | Tool executions |
|---|---|---|
| Plain completion | 1 | 0 |
| One tool call | 2 | 1 |
| Three independent tools, called in parallel | 2 | 3 (concurrent) |
| Three dependent tools, chained | 4 | 3 (sequential) |

Put illustrative timings on it. If a model turn takes about 2 seconds and a tool call about 200 ms,
a plain completion answers in ~2 s, a single tool call in ~4.2 s, and a three-step chain in
~8.6 s — more than four times the latency of the answer the user thinks they asked for. Those
timings are an illustration rather than a measurement, but the multiplier is structural: you cannot
have fewer than two model turns, and every additional dependent step buys another full one.

### You re-send every tool definition on every request

Because the API is stateless, the whole tool array is part of each request in the loop. That makes
the schema block a fixed tax paid repeatedly, and it is larger than most teams estimate. A
measurement filed against the [Model Context Protocol](https://howaiworks.ai/glossary/model-context-protocol)
specification in 2026 counted the [tokens](https://howaiworks.ai/glossary/token) in eleven production tools: **6,807 in
total**, from 103 for a trivial `stats` call to 1,024 for a batch executor — a mean of about 620
tokens per tool. The same report puts a typical session of 20–30 registered tools at 15–30 KB of
context before the user has typed anything.

Run that through the round-trip table. A three-step chained task issues four requests, so those
eleven tools cost `4 × 6,807 ≈ 27,000` tokens of schema for one task — none of it the user's
problem, the model's reasoning, or the answer. This is a different cost from the one on the
[AI agent](https://howaiworks.ai/glossary/ai-agent) page, and the two add up: there, the growing transcript makes token
use rise super-linearly with step count; here, a constant block is re-billed once per step.

### The model is choosing from a menu it has to re-read

The context cost is the visible half. The invisible half is that a longer menu makes the choice
worse — and tool-use benchmarks exist to catch exactly this, because the failure hides. What they
score is not whether your code ran but whether the model picked the *right* tool, filled its
arguments with usable values, and invented nothing: the three errors that recur are the wrong tool,
malformed arguments, and hallucinated ones — argument values the model made up that are
syntactically fine and factually baseless. A JSON Schema on your side rejects the malformed case
and is silent on the other two.

Anthropic's November 2025 report on deferred tool loading is the cleanest evidence that menu length
is itself one of those error sources, because it holds the model and the available tools fixed and
varies only whether all the schemas sit in context. Loading definitions on demand cut a library of
fifty-plus tools from ~77K tokens to ~8.7K — about an 85% reduction — and lifted the model's
measured tool-selection accuracy: on the weaker of the two configurations Anthropic tested, by
roughly half in relative terms, bought by showing the model fewer options at once rather than by
changing the model. The absolute scores are model-specific and move as models do; the direction and
the size of the effect are the durable part.

The mechanism is attention dilution, and it is why the reflex to "just expose everything" backfires.
A model asked to pick one of fifty tools is reading fifty descriptions written by fifty different
people, several of which overlap. It is also why the
[Berkeley Function-Calling Leaderboard](https://gorilla.cs.berkeley.edu/leaderboard.html) scores
**irrelevance detection** as its own category — the ability to answer without calling anything when
no tool applies. Calling the wrong tool and calling a tool at all are two separate failures, and
the second one is the one people forget to measure.

## Types

There is a real taxonomy here, and it is defined by round trips rather than by what the functions
do. The Berkeley leaderboard's categories use these names, and so do most provider APIs.

**Single calling** is one tool per assistant turn: two model invocations, one result. This is the
case that is now largely solved — near-saturated on the single-call benchmark categories, which is
why BFCL v4 (released April 2026) reweighted most of its score toward multi-turn agentic tasks. On
that harder mix nobody is close to solved: across the thirteen models a third-party tracker listed
in July 2026, the leader scored 0.750 and the mean was 0.611. Treat the figures as a snapshot —
leaderboards move, and the point is the gap between single-call and multi-turn, not the ranking.

**Parallel calling** is several tool calls in one assistant message, dispatched concurrently by
your executor. It is only correct when the calls are independent — three city lookups, not "get
the user id, then fetch their orders" — and when it applies, it collapses *n* round trips into one
and makes wall-clock tool time the *maximum* rather than the *sum*. The LLMCompiler paper
(Kim et al., December 2023) measured up to **3.7× lower latency and 6.7× lower cost** than a
sequential ReAct loop on the same tasks, with about 9% better accuracy — the accuracy gain being
the surprise, and attributable to the model planning the whole set at once instead of drifting
across turns. This is the capability the `functions` → `tools` rename existed to enable.

**Sequential or chained calling** is the dependent case, where each call needs the previous result.
It cannot be parallelised, it costs *n* + 1 model invocations, and it is where an
[agentic workflow](https://howaiworks.ai/glossary/agentic-workflow) starts and this page hands off.

Cutting across all three is **who decides**. `tool_choice` is typically `auto` (the model may call
zero, one or many), `required` (it must call something), a named tool (it must call *that* one), or
`none` (it may not). Forcing a specific tool turns the model into a parameter extractor and removes
the decision that makes function calling interesting — which is exactly what you want when you
already know the operation and only need the arguments filled in.

## Real-World Applications

**Coding agents are function calling with a very small tool list.** When
[Claude Code](https://howaiworks.ai/ai-tools/claude-code) or [Cursor](https://howaiworks.ai/ai-tools/cursor) reads a file, greps a
repository, edits a line or runs a test, each of those is a tool call: the model emits
`edit_file({...})`, the harness validates and applies it, and the diff or the test output comes
back as a tool message. Nothing in the model touched the disk. This also explains why file
permissions and command allow-lists live in the harness rather than in the prompt — the harness is
the only component that executes anything.

**MCP is a distribution mechanism for tool schemas, not a replacement for this.** A
[Model Context Protocol](https://howaiworks.ai/glossary/model-context-protocol) server's job is to hand a client a list
of tools and to execute the ones it is asked to run. What reaches the model is still a schema in
the context window and what comes back is still a `tool_calls` array. MCP changes where the
definitions come from — one protocol instead of bespoke glue per integration — and it is why the
token arithmetic above suddenly mattered to everyone: connecting five servers can add dozens of
tools you never chose individually.

**Provider-hosted tools are the one real exception to "the model does not execute".** When you
enable a server-side code interpreter or web search, the provider runs the loop inside its own
infrastructure and returns the result to you already resolved. The model still only emits a
request; it is just that somebody else's executor honours it. Knowing which side of that line a
given tool sits on determines who is liable for what it does.

**Retrieval becomes a decision instead of a step.** Classic
[RAG](https://howaiworks.ai/glossary/retrieval-augmented-generation) retrieves before generating, every time. Exposed as
a tool, search becomes something the model may invoke zero times for "hello", once for a factual
question, or three times with refined queries when the first result was thin — which is a better
system when the retrieval is expensive and a worse one when the model declines to search and
answers from memory instead.

**Structured extraction with a forced tool.** A large amount of production "function calling" never
calls a function at all: you define a tool whose schema is the record you want, set `tool_choice`
to that tool, and use the argument object as the output. It is the most reliable way to get
schema-conformant JSON out of a model, and the function on the other end is often just a database
insert.

## Key Concepts

- **A tool definition is prompt text with a validator attached.** The JSON Schema constrains the
  *shape* of the arguments; the description string is what actually decides *whether* the tool gets
  picked. Rewording a description changes behaviour, and renaming a parameter from `q` to
  `search_query` measurably changes how often it is filled in correctly. This is
  [prompt engineering](https://howaiworks.ai/glossary/prompt-engineering) wearing an API's clothes.
- **The `tool_call_id` is what makes parallel calls tractable.** Each result message quotes the id
  of the request it answers, so three concurrent calls can return out of order and the model still
  knows which weather belongs to which city. Dropping or reusing an id is a common source of
  confusing behaviour with no error attached.
- **Your executor is a policy enforcement point, whether or not you designed it as one.** The model
  may have read an untrusted web page one message earlier, so a tool call is an *untrusted*
  request. Authority has to come from the session — who is logged in, what they may touch — not
  from the arguments the model supplied.
- **Tool definitions are the one part of the context you control completely.** Prompts drift and
  transcripts grow, but the tool array is a list you wrote, and pruning it is the cheapest
  available intervention on both cost and accuracy.

## Challenges

**The context tax is charged before anyone says anything.** Twenty tools at ~620 tokens each is
about 12,000 tokens of overhead on every single request in the loop — and unlike a long
conversation, you cannot summarise it away. Prompt caching reduces what you *pay* for those tokens
without reducing the context window they occupy or the attention they dilute, which is why
pagination and on-demand loading appeared as soon as tool libraries got past a couple of dozen
entries.

**The failure that leaves no trace is a tool that was never called.** A model that answers a
question about live inventory from its training data returns a fluent, well-formed paragraph, logs
nothing, and raises nothing. It is why irrelevance and relevance detection are scored separately on
BFCL, and why a tool-call rate is worth putting on a dashboard: a sudden drop is often the first
sign that a description was reworded or a schema grew a required field.

**Arguments can be perfectly valid and completely wrong.** The schema will confirm that
`order_id` is a string and `quantity` is an integer, and it has no opinion about whether that order
exists. Catching that class needs something to check *against* — a lookup, a constraint, a second
call — and it is covered in full on the [error handling](https://howaiworks.ai/glossary/error-handling) page rather than
duplicated here.

**A tool call is a confused deputy waiting to happen, and [prompt injection](https://howaiworks.ai/glossary/prompt-injection) is how that deputy gets its orders.** The model is a text generator that may have
just read an attacker's document, and the tool call it emits is byte-identical whether it came from
the user's intent or from an instruction hidden in that document. Since the model has no notion of
who is asking, the blast radius equals whatever your executor is willing to do — which is the
argument for read-only tools by default, per-tool authorisation, and never letting the model choose
the credentials a call runs under.

**Schemas rot in a way prompts do not.** Change a function's signature and the schema, the dispatch
code and the model's learned habits fall out of step at different rates. Because the description is
also the selection signal, tool definitions need versioning and review like an external
[API](https://howaiworks.ai/glossary/api) contract, not like an internal helper.

## Future Trends

The clearest direction is **not putting the tools in the context at all**. Deferred loading — the
model is given a search tool over the tool catalogue and pulls in definitions when it needs them —
went from a workaround to a shipped provider feature in November 2025, and the large tool-selection
accuracy jump above explains why it spread quickly. It also inverts the design advice of the previous two
years: a large, well-described tool library is now an asset again, provided the model sees only the
slice it needs.

A more radical version is **replacing the calls with code**. Anthropic's November 2025 write-up on
code execution with MCP presents tools as importable functions in a sandboxed runtime and has the
model write a short program instead of emitting one call per step. Loops, filtering and
intermediate results then stay inside the sandbox instead of crossing the context window twice, and
their reported example fell from 150,000 tokens to 2,000 — a 98.7% reduction. The trade is that you
now need a real code sandbox, which is a much larger security surface than a dispatch table.

The third direction is **standardisation above the call**. MCP standardised where tool definitions
come from; the [Agent2Agent Protocol](https://howaiworks.ai/glossary/agent2agent-protocol) is doing the same for
delegation between agents, having absorbed IBM's
[Agent Communication Protocol](https://howaiworks.ai/glossary/agent-communication-protocol) in August 2025. Both sit on top of this mechanism rather than replacing it — one system
asking another to do something is still a structured request that somebody else's code decides
whether to honour.

## Code Example

The whole loop, with no provider SDK, so that the shape is visible. `call_model` stands in for
whichever chat endpoint you use; every provider's version of this is the same five steps.

```python
import json

TOOLS = [
    {
        "name": "get_weather",
        "description": "Current conditions for a city. Use for questions about "
                       "weather right now, not forecasts.",
        "parameters": {
            "type": "object",
            "properties": {"city": {"type": "string", "description": "City name"}},
            "required": ["city"],
        },
    }
]

def get_weather(city: str) -> dict:
    """The real function. Note that the model never reaches this line itself."""
    return {"city": city, "temp_c": 19, "condition": "cloudy"}

# The dispatch table is the security boundary. A name the model invents
# that is not a key here simply does not run.
HANDLERS = {"get_weather": get_weather}

def run(user_message: str, max_steps: int = 5) -> str:
    messages = [{"role": "user", "content": user_message}]

    for step in range(max_steps):
        reply = call_model(messages=messages, tools=TOOLS, tool_choice="auto")
        messages.append(reply)

        # Termination: an ordinary message with no tool calls means we are done.
        if not reply.get("tool_calls"):
            return reply["content"]

        # One message can carry several calls; independent ones can run concurrently.
        for call in reply["tool_calls"]:
            name = call["function"]["name"]
            args = json.loads(call["function"]["arguments"])

            handler = HANDLERS.get(name)
            if handler is None:
                result = {"error": f"unknown tool {name!r}"}
            else:
                try:
                    result = handler(**args)          # <- the only execution anywhere
                except Exception as exc:
                    # A failure is just another result. The model reads it and retries
                    # or gives up; it does not see an exception.
                    result = {"error": str(exc)}

            messages.append({
                "role": "tool",
                "tool_call_id": call["id"],           # binds this result to that request
                "content": json.dumps(result),
            })

    return "stopped: hit the step limit without a final answer"
```

Three things in that listing are the whole subject. `handler(**args)` is the only line that does
anything to the world, and it is in your file, not the model's. `HANDLERS.get(name)` means an
invented tool name is a dead end rather than an incident. And the `for` loop's bound is there
because nothing else guarantees the model will ever stop asking.

## Frequently Asked Questions

### Does the model actually run my function?

No. The model emits a structured request — a tool name and a JSON object of arguments — and nothing else. It has no network socket, no file handle and no database connection. Your code receives that request, decides whether to honour it, runs the function itself, and sends the return value back as another message. The model only ever reads and writes text.

### What is the difference between function calling and tool calling?

Nothing, except the year. OpenAI shipped the feature as `functions` on 13 June 2023 and superseded it with `tools`/`tool_choice` at DevDay on 6 November 2023, partly so that one request could carry several calls. Providers, SDKs and blog posts still use both names for the same mechanism.

### How does the tool-calling loop know when to stop?

It stops when the model returns an ordinary message with no tool calls in it. That is the only termination signal, and it is a model decision rather than a guarantee — so every production loop also carries a hard cap on iterations, because a model that keeps re-requesting the same failing tool will otherwise spin until it exhausts the context window.

### Why does adding more tools make the model worse?

Every tool schema sits in the context on every request, so the model chooses from a menu it must re-read each time. When Anthropic let models load definitions on demand instead of all at once (November 2025), the measured tool-selection accuracy jumped — roughly half again in relative terms on the weaker of the two configurations it tested — with the same model and the same tools available, changing only which schemas sat in context.

### How many tokens do tool definitions cost?

More than people expect, and you pay it on every request. A measurement filed against the Model Context Protocol specification in 2026 put eleven production tools at 6,807 tokens in total, ranging from 103 tokens for a trivial one to 1,024 for a complex one. A three-step task means four requests, so that same block is billed four times.

### Is function calling secure?

The mechanism is neutral; your executor is the security boundary. A tool call is a request from a text generator that may have read an attacker's web page moments earlier, so the function name, the arguments and the caller's authority all have to be checked in your code. A JSON Schema validates shape, never permission.

## Related

### Related terms

- [AI Agent](https://howaiworks.ai/glossary/ai-agent)
- [Model Context Protocol (MCP)](https://howaiworks.ai/glossary/model-context-protocol)
- [Error Handling in AI Systems](https://howaiworks.ai/glossary/error-handling)
- [Agentic Workflow](https://howaiworks.ai/glossary/agentic-workflow)
- [API](https://howaiworks.ai/glossary/api)
- [Large Language Model (LLM)](https://howaiworks.ai/glossary/large-language-model)

---

Source: https://howaiworks.ai/glossary/function-calling — HowAIWorks.ai