Introduction
Tool Calling
Enable LLMs to call external functions and tools.
Tool Calling
Enable LLMs to call external functions and tools. The gateway supports both OpenAI-style function calling and Model Context Protocol (MCP).
Function Calling
Define tools in your request and let the model decide when to call them:
curl https://viprouter.com/v1/chat/completions \
-H "Content-Type: application/json" \
-d '{
"model": "openai/gpt-4o-mini",
"messages": [{"role": "user", "content": "What is the weather in Tokyo?"}],
"tools": [{
"type": "function",
"function": {
"name": "get_weather",
"description": "Get current weather",
"parameters": {
"type": "object",
"properties": {
"location": {"type": "string", "description": "City name"}
},
"required": ["location"]
}
}
}]
}'Response:
{
"choices": [{
"message": {
"role": "assistant",
"tool_calls": [{
"id": "call_abc123",
"type": "function",
"function": {
"name": "get_weather",
"arguments": "{\"location\": \"Tokyo\"}"
}
}]
}
}]
}MCP Tools Integration
The gateway integrates with Model Context Protocol (MCP) to provide tool-calling capabilities:
curl https://viprouter.com/v1/chat/completions \
-H "Content-Type: application/json" \
-H "X-BF-MCP: true" \
-d '{
"model": "openai/gpt-4o-mini",
"messages": [{"role": "user", "content": "Search for recent AI news"}],
"tools": [{"type": "mcp", "server": "search-server"}]
}'Multi-Turn Tool Calling
The gateway handles the full tool-calling loop:
- Model requests tool call
- Gateway returns tool call to your application
- Your application executes the function
- You send the result back
- Model continues with the result
# Multi-turn tool calling example
messages = [{"role": "user", "content": "What's the weather?"}]
# First call - model requests tool
response = client.chat.completions.create(
model="openai/gpt-4o-mini",
messages=messages,
tools=tools
)
# Add assistant message with tool call
messages.append(response.choices[0].message)
# Execute tool and add result
messages.append({
"role": "tool",
"tool_call_id": response.choices[0].message.tool_calls[0].id,
"content": '{"temperature": 22, "condition": "sunny"}'
})
# Second call - model uses tool result
response = client.chat.completions.create(
model="openai/gpt-4o-mini",
messages=messages,
tools=tools
)Parallel Tool Calls
Models can request multiple tool calls simultaneously:
{
"tool_calls": [
{"id": "call_1", "function": {"name": "get_weather", "arguments": "{\"location\": \"Tokyo\"}"}},
{"id": "call_2", "function": {"name": "get_weather", "arguments": "{\"location\": \"London\"}"}}
]
}Next Steps
- MCP API Reference - Full MCP documentation
- Streaming - Stream tool-calling responses