The difference between a chatbot and an AI agent comes down to one thing: tools. A chatbot can talk. An agent can act. And the mechanism that bridges the gap is function calling -- the ability of an AI model to decide when and how to use external tools.
In this tutorial, we'll build a working AI agent from scratch using function calling. By the end, you'll have an agent that can check the weather, search a database, perform calculations, and chain multiple actions together to accomplish complex goals.
What is Function Calling?
Function calling (also called tool use) lets an AI model invoke external functions or APIs. Instead of just generating text, the model can say "I need to call the get_weather function with these parameters," and your code executes that function and returns the result.
Here's the basic flow:
- You define the available tools (functions) and their schemas
- The user asks a question
- The model decides which tools to call (if any) and with what arguments
- Your code executes the tool and returns the result
- The model uses the result to generate the final answer
This loop -- think, call tool, get result, think again -- is the foundation of every AI agent.
Prerequisites
- An AI API key (we'll use OpenAI-compatible format, which works with Qubax AI, OpenRouter, OpenAI, and many others)
- Python 3.10+ with the
openailibrary - Basic understanding of Python and REST APIs
pip install openaiStep 1: Define Your Tools
First, let's create the tools our agent can use. Each tool is a regular Python function with a corresponding JSON schema that tells the model what it does and what parameters it accepts.
import json
import requests
# --- Tool implementations ---
def get_weather(location: str) -> str:
"""Get current weather for a location."""
# Using a free weather API
try:
resp = requests.get(
f"https://wttr.in/{location}?format=j1",
timeout=10
)
data = resp.json()
current = data["current_condition"][0]
return json.dumps({
"location": location,
"temperature_c": current["temp_C"],
"temperature_f": current["temp_F"],
"condition": current["weatherDesc"][0]["value"],
"humidity": current["humidity"]
})
except Exception as e:
return json.dumps({"error": str(e)})
def calculate(expression: str) -> str:
"""Safely evaluate a math expression."""
try:
# Only allow basic math operations
allowed = set("0123456789+-*/.() ")
if not all(c in allowed for c in expression):
return json.dumps({"error": "Invalid characters in expression"})
result = eval(expression) # Safe due to character filtering
return json.dumps({"expression": expression, "result": result})
except Exception as e:
return json.dumps({"error": str(e)})
def search_knowledge_base(query: str) -> str:
"""Search a knowledge base for relevant information."""
# In production, this would search your vector database
mock_results = [
{"title": "Company Refund Policy", "content": "Full refunds within 30 days..."},
{"title": "Shipping Information", "content": "Free shipping on orders over $50..."}
]
return json.dumps({"query": query, "results": mock_results})Step 2: Define Tool Schemas
The model needs to know what tools are available. We define these as JSON schemas:
TOOL_DEFINITIONS = [
{
"type": "function",
"function": {
"name": "get_weather",
"description": "Get the current weather for a given city or location name.",
"parameters": {
"type": "object",
"properties": {
"location": {
"type": "string",
"description": "City name, e.g. 'San Francisco' or 'London'"
}
},
"required": ["location"]
}
}
},
{
"type": "function",
"function": {
"name": "calculate",
"description": "Evaluate a mathematical expression.",
"parameters": {
"type": "object",
"properties": {
"expression": {
"type": "string",
"description": "Math expression, e.g. '2 + 2'"
}
},
"required": ["expression"]
}
}
},
{
"type": "function",
"function": {
"name": "search_knowledge_base",
"description": "Search the company knowledge base.",
"parameters": {
"type": "object",
"properties": {
"query": {
"type": "string",
"description": "Search query"
}
},
"required": ["query"]
}
}
}
]
# Map tool names to functions
TOOL_MAP = {
"get_weather": get_weather,
"calculate": calculate,
"search_knowledge_base": search_knowledge_base
}Step 3: Build the Agent Loop
Now for the core of the agent -- the loop that handles tool calls:
from openai import OpenAI
# Initialize client (works with Qubax AI, OpenAI, OpenRouter, etc.)
client = OpenAI(
api_key="your-api-key",
base_url="https://api.qubax.ai/v1" # Or your provider's base URL
)
def run_agent(user_message: str, max_iterations: int = 10):
"""Run the agent loop: think -> call tools -> respond."""
messages = [
{
"role": "system",
"content": (
"You are a helpful AI assistant with access to tools. "
"Use tools when needed to provide accurate answers."
)
},
{"role": "user", "content": user_message}
]
for i in range(max_iterations):
print(f"--- Iteration {i + 1} ---")
# Call the model
response = client.chat.completions.create(
model="gpt-4o", # Or any model that supports function calling
messages=messages,
tools=TOOL_DEFINITIONS,
tool_choice="auto" # Let the model decide
)
message = response.choices[0].message
# If no tool calls, we're done
if not message.tool_calls:
print(f"Agent: {message.content}")
return message.content
# Add the assistant's message (with tool calls) to history
messages.append(message)
# Execute each tool call
for tool_call in message.tool_calls:
function_name = tool_call.function.name
arguments = json.loads(tool_call.function.arguments)
print(f" Calling: {function_name}({arguments})")
# Execute the tool
if function_name in TOOL_MAP:
result = TOOL_MAP[function_name](**arguments)
else:
result = json.dumps({"error": f"Unknown function: {function_name}"})
print(f" Result: {result[:200]}")
# Add the tool result to messages
messages.append({
"role": "tool",
"tool_call_id": tool_call.id,
"content": result
})
return "Max iterations reached without a final answer."Step 4: Add Error Handling and Retry Logic
Production agents need robust error handling. Here's an enhanced version:
import time
def execute_tool_safely(tool_name: str, arguments: dict, max_retries: int = 2):
"""Execute a tool with error handling and retries."""
for attempt in range(max_retries + 1):
try:
if tool_name not in TOOL_MAP:
return json.dumps({"error": f"Unknown tool: {tool_name}"})
result = TOOL_MAP[tool_name](**arguments)
# Validate the result is not an error
parsed = json.loads(result)
if "error" in parsed and attempt < max_retries:
print(f" Retry {attempt + 1} for {tool_name}")
time.sleep(1)
continue
return result
except json.JSONDecodeError:
return json.dumps({"error": "Tool returned invalid JSON"})
except TypeError as e:
return json.dumps({"error": f"Invalid arguments: {str(e)}"})
except Exception as e:
if attempt < max_retries:
print(f" Error, retrying: {e}")
time.sleep(2 ** attempt) # Exponential backoff
continue
return json.dumps({"error": f"Tool execution failed: {str(e)}"})
return json.dumps({"error": "Max retries exceeded"})Step 5: Multi-Step Reasoning Example
Here's how the agent handles a complex, multi-step request:
# Complex query requiring multiple tool calls
response = run_agent(
"I'm planning a trip to Tokyo. What's the weather like there? "
"Also, if I budget $150 per day and stay for 12 days, "
"how much will the trip cost? And what is our company's refund policy "
"for cancelled trips?"
)The agent will:
- Call
get_weather("Tokyo")to check the weather - Call
calculate("150 * 12")to compute the total cost - Call
search_knowledge_base("refund policy cancelled trips")to find the policy - Combine all results into a comprehensive answer
Step 6: Adding Memory (Conversation Context)
For multi-turn conversations, maintain message history:
class AgentSession:
def __init__(self, system_prompt: str = None):
self.messages = [{
"role": "system",
"content": system_prompt or "You are a helpful AI assistant."
}]
def chat(self, user_message: str) -> str:
self.messages.append({"role": "user", "content": user_message})
result = run_agent_with_messages(self.messages)
return result
# Usage
session = AgentSession("You are a helpful travel planning assistant.")
session.chat("What's the weather in Paris?")
session.chat("Now check London weather and compare.") # Has contextBest Practices
1. Write Clear Tool Descriptions
The model's ability to choose the right tool depends entirely on your descriptions. Be specific about what each tool does, when to use it, and what parameters mean.
2. Use Parallel Tool Calls
Many modern models support calling multiple tools simultaneously. If a user asks about weather in three cities, the model can call get_weather three times in parallel instead of sequentially.
3. Validate Tool Arguments
Always validate the arguments the model provides before executing a tool. Models can produce unexpected parameter values.
4. Implement Rate Limiting
If your tools call external APIs, implement rate limiting to avoid hitting quotas and incurring unexpected costs.
5. Log Everything
Keep detailed logs of tool calls, arguments, and results. This is invaluable for debugging agent behavior.
Cost Optimization
Function calling adds tokens to your context (tool definitions, call/response pairs). To minimize costs:
- Use a cheaper model for simple tool-selection tasks and a powerful model for complex reasoning
- Limit the number of tools you expose at once
- Use concise tool descriptions
- Consider using a platform like Qubax AI that offers competitive pricing across many models
Conclusion
Function calling transforms AI from a conversational interface into an autonomous agent that can interact with the world. With just a few dozen lines of Python, you can build agents that check weather, perform calculations, search databases, and chain multiple actions together.
The key insight is that the agent loop -- think, act, observe, think -- is universal. Whether you're building a simple weather bot or a complex multi-tool system, the same pattern applies.
For more tutorials and API documentation, visit Qubax AI Docs. To explore available models that support function calling, check out Qubax AI Models.
FAQ
What models support function calling?
Most modern models support function calling, including GPT-4o, Claude 3.5+, Gemini 1.5+, and many open-source models. Check your provider's documentation for specifics.
Can an agent call tools in parallel?
Yes. Modern models can request multiple tool calls in a single response, and you can execute them concurrently to speed up your agent.
How many tools can an agent have?
There's no hard limit, but practically, having more than 20-30 tools can confuse the model. Group related tools and only expose the relevant ones for each context.
Is function calling the same as MCP?
No. MCP (Model Context Protocol) is a standardized way for AI applications to connect to external tools and data sources. Function calling is the mechanism models use to invoke tools. MCP uses function calling under the hood.
How do I handle tool failures gracefully?
Implement retry logic, timeouts, and fallback responses. Always return structured JSON from your tools so the model can understand when something went wrong.
Can I use function calling with streaming responses?
Yes, but it's more complex. You need to accumulate streamed content and check for tool call markers. Most API clients provide helper methods for this.
What is the difference between an AI agent and a chatbot?
A chatbot only generates text responses. An agent can use tools, make decisions, take actions, and work toward multi-step goals. Function calling is what makes this possible.
