Prompt Engineering Tool Calling

Tool calling, also called function calling, lets an LLM ask your application to run a function, such as searching a database or checking the weather, and then use the result in its answer. The model only produces a structured request; your code executes it.

The Tool Calling Loop

• 1. You send the conversation plus a list of tools (name, description, JSON schema of parameters).
• 2. The model decides whether a tool is needed and returns a tool call with arguments.
• 3. Your code runs the function and returns the result as a tool message.
• 4. The model reads the result and writes the final answer (or calls another tool).

A Tool Definition

Most APIs describe tools in a similar JSON shape:

{
  "name": "get_order_status",
  "description": "Look up the shipping status of a customer order. Use when the user asks where their order is.",
  "parameters": {
    "type": "object",
    "properties": {
      "order_id": { "type": "string", "description": "Order number, e.g. 'A-10293'" }
    },
    "required": ["order_id"]
  }
}

User: Where is my order A-10293?

Sample model tool call:
{"name": "get_order_status", "arguments": {"order_id": "A-10293"}}

Tool result: {"status": "out_for_delivery", "eta": "today"}

Sample final response:
Good news! Order A-10293 is out for delivery and should arrive today.

Writing Good Tool Descriptions

The description is a prompt. It decides when the model will use the tool.

• Say what the tool does and when to use it (and when not to).
• Describe each parameter with format and an example.
• Use clear, distinct names; avoid two tools that sound alike.
• Return compact, meaningful results, including clear error messages.
• Keep the tool list short; too many tools confuse the model.

Safety

Treat tool arguments like untrusted user input. Validate them, limit permissions, and require human confirmation for actions with real-world effects such as payments, deletions or sending messages.

Frequently asked questions

Does the LLM run the function itself?

No. It returns the function name and arguments; your code decides whether and how to run it.

What is the difference between tool calling and structured output?

Structured output shapes the final answer. Tool calling lets the model request actions mid-conversation and continue after getting results.

Do all models support tool calling?

Most modern hosted and many open-weight models do, but quality varies. Check the model documentation.