How AI tool calling works in one sentence: the model never touches your APIs. It returns a structured request naming a tool and its arguments, your code executes it, and you send the result back for the final answer. The Python pattern below shows the full loop with validation.

Every “AI booked my flight” demo hides the same misunderstanding. The model did not call an API. It produced JSON asking the application to call one, and the application obliged. Once you see the boundary clearly, tool-calling bugs get much easier to diagnose.

This guide walks through the five-step loop with a minimal Python implementation: define tools as JSON schemas, send them with the prompt, execute the returned call locally, feed the result back, and read the final answer. The failure section covers what breaks in production.

Quick Answer: How Does AI Tool Calling Actually Work?

You send the model your prompt plus tool definitions (name, description, JSON schema for arguments). The model replies with a tool call: a name and an arguments object. Your code parses the arguments, runs the real function, and sends the output back to the model, which then writes the final response. The model requests; your code acts. That separation is the entire security model.

What Each Piece Is

  • Tool: functionality you advertise to the model, such as get_time taking a city string. Defined by a JSON schema so the model knows the shape of valid arguments.
  • Tool call: the model’s response when it decides it needs a tool: the tool name plus an arguments object, for example {"city": "London"}.
  • Tool output: whatever your code returns for that call, sent back so the model can finish the answer. JSON or plain text both work.

Prerequisites

  • Python 3.10 or later. The loop pattern below uses only the standard library for the routing logic; the API calls need the provider SDK of your choice.
  • An API key from a provider that supports function or tool calling, exported as an environment variable (for example export API_KEY="YOUR_API_KEY"). Never hardcode keys.
  • Familiarity with JSON schemas at the level of “object with required string fields.”

The Five-Step Loop

  1. Send the prompt together with your tool definitions.
  2. Receive either a text answer or a tool call from the model.
  3. Execute the named function in your code with the parsed arguments.
  4. Send the function result back to the model.
  5. Receive the final answer (or another tool call, and repeat).

This flow is documented in the OpenAI function-calling guide (checked 2026-09-20) and every major provider follows the same shape with different field names. Learn the shape once and the provider differences become small translation details.

Minimal Working Pattern in Python

The network calls below are sketched against the documented flow; the routing and validation logic was executed offline and passes. Adapt the client calls to your provider SDK.

import json

def get_time(city):
    """Example tool: look up time for a city."""
    return json.dumps({"city": city, "utc_offset": "+00:00"})

def read_file(path):
    """Example tool with a guardrail: only /data/ is readable."""
    if ".." in path or not path.startswith("/data/"):
        raise ValueError("path outside allow-list: %r" % (path,))
    with open(path, encoding="utf-8") as fh:
        return fh.read(4000)

ROUTER = {"get_time": get_time, "read_file": read_file}

def handle_tool_call(name, raw_arguments):
    """Parse, validate, and dispatch one model tool call."""
    try:
        args = json.loads(raw_arguments)
    except json.JSONDecodeError as exc:
        raise ValueError("malformed tool arguments: %s" % exc)
    if not isinstance(args, dict):
        raise ValueError("tool arguments must be an object")
    try:
        func = ROUTER[name]
    except KeyError:
        raise ValueError("unknown tool requested: %r" % (name,))
    return func(**args)

# Sketch of the surrounding loop (provider SDK calls):
# 1. response = client.send(prompt, tools=TOOL_SCHEMAS)
# 2. for call in response.tool_calls:
#        result = handle_tool_call(call.name, call.arguments)
# 3. final = client.send(prompt, tool_results=[...])

Three rules are baked into handle_tool_call and worth keeping in every implementation: reject malformed arguments before dispatch, reject unknown tool names instead of guessing, and validate security-sensitive arguments (paths, URLs, shell strings) inside the tool, not in the prompt.

What Breaks in Production

  • Bad arguments. The model emits a string where your schema wants a number, or omits an optional field your code assumed. Parse defensively and return a correction the model can act on, such as “city is required, expected a string.”
  • Timeouts. Tools that call the network hang. Wrap every tool with a deadline and return a timeout result so the conversation continues instead of stalling.
  • Untrusted output. Tool results can contain attacker-controlled text (a calendar invite, a web page). Treat tool output as data, never as instructions. This is the same prompt-injection boundary covered in our guide to how AI agents attack software supply chains.
  • Parallel calls. Some providers return several tool calls at once. Execute them, collect all results, and return them together rather than handling only the first.

Common Mistakes

  • Letting the model pick credentials. Never include secrets, account IDs, or connection strings as tool parameters. Resolve them server-side from context the model cannot see or change.
  • Exposing destructive tools early. Start read-only. Add write operations only behind explicit confirmation for irreversible actions.
  • Skipping argument logging. Log every tool name and argument shape (redact values). Without this log, debugging “the agent did something odd” is guesswork.

Security Considerations

The tool boundary is a trust boundary. The model is an untrusted author of function names and arguments, so the router must enforce an allow-list of tool names, schemas must constrain argument shapes, and tools with side effects need human approval. Log every invocation with its arguments and result status, and redact secrets before the log is stored. See our guides on protecting git repositories from agent abuse and securing agents on Linux for the surrounding controls.

Alternatives

  • Provider agent frameworks that package this loop (orchestration, retries, approvals) when hand-rolled routing outgrows a single file.
  • MCP servers when tools must be shared across applications rather than defined per request.
  • Local small models via Ollama for tool-calling experiments that must stay offline; see Run an Open-Weight LLM on Ubuntu With Ollama for the setup.

FAQ

Does the AI model directly call my API?

No. The model returns a structured request containing a tool name and arguments. Your application executes the call and returns the result. Nothing in your infrastructure is reachable by the model except through code you wrote.

What format are tool arguments?

A JSON object conforming to the schema you advertised. Because the model generates it, always parse it with error handling and validate types before use, as the router above does.

How many tools should I expose at once?

Start small. Provider guidance suggests keeping the initial set under roughly twenty for accuracy, and loading the rest on demand. Fewer, well-described tools beat dozens of vague ones.

What happens when the model requests a tool that does not exist?

That is a router bug or a version mismatch, not a normal case. Reject it loudly with an error, log it, and fix the tool list. Never fuzzy-match an unknown name onto a destructive function.

Related DevStackIO Resources

Method note: the five-step flow and schema fields follow the OpenAI function-calling guide (checked 2026-09-20). The router logic (parsing, unknown-tool rejection, path guardrail) was executed offline on Python 3.14 and passes; provider SDK calls are sketched, not executed, since they require an API key. No model scores or benchmarks are claimed.

Related Tools

Try these companion tools on DevStackIO Tools: