← Back to Home

Building Model-Agnostic AI Agents

Most AI Agent frameworks are born tied to a specific model vendor. LangChain was originally built around OpenAI. Claude Code is naturally Anthropic-exclusive. But in practice, you often need to switch between models — for cost, latency, capability matching, or simply to avoid lock-in.

This article shows you how to build an Agent that works with any model — Claude, GPT, DeepSeek, Llama, or locally deployed open models — by swapping a single line of configuration.

Why Model-Agnostic Matters

What is Model-Agnostic Architecture

Model-agnostic means your Agent's core logic doesn't depend on any specific model's API format. The Agent loop — observe → think → act → observe — stays identical regardless of which model powers the "think" step.

The architecture has three layers:

1. Agent Core — the ReAct loop, tool execution, memory management (model-independent)
2. Adapter Layer — translates between the core's unified format and each model's specific API
3. Model Providers — Claude, GPT, DeepSeek, Llama, etc. (swappable)

┌─────────────────────────────────┐
│         Agent Core Loop          │  ← Never changes
│  observe → think → act → observe │
└──────────────┬──────────────────┘
               │ Unified interface
┌──────────────▼──────────────────┐
│        Adapter Layer             │  ← Swap per model
│  ┌────────┐ ┌──────┐ ┌───────┐  │
│  │Claude  │ │ GPT  │ │DeepSk │  │
│  │Adapter │ │Adapter│ │Adapter│  │
│  └───┬────┘ └──┬───┘ └───┬───┘  │
└──────┼─────────┼─────────┼──────┘
       │         │         │
   Anthropic  OpenAI  DeepSeek APIs

The Adapter Interface

Every adapter implements the same interface. Here's the contract:

from abc import ABC, abstractmethod
from dataclasses import dataclass
from typing import Any

@dataclass
class AgentResponse:
    """Unified response from any model."""
    content: str | None        # Final text answer (if is_final=True)
    tool_call: dict | None     # Tool call request (if is_final=False)
    is_final: bool             # True = done, False = tool call needed
    usage: dict                # Token usage: {"input": N, "output": M}

class ModelAdapter(ABC):
    """Every model adapter must implement this interface."""

    @abstractmethod
    def chat(self, messages: list[dict],
             tools: list[dict] | None = None,
             temperature: float = 0.7,
             max_tokens: int = 1000) -> AgentResponse:
        """Send messages + tools → receive response or tool call."""
        ...

    @abstractmethod
    def normalize_tools(self, tools: list[dict]) -> list[dict]:
        """Convert unified tool schema to model-specific format."""
        ...

    @abstractmethod
    def normalize_messages(self, messages: list[dict]) -> list[dict]:
        """Convert unified messages to model-specific format."""
        ...

Implementing Real Adapters

Here are concrete implementations for the three most common model families. Notice how each handles tool calling differently.

OpenAI Adapter (GPT-4o, GPT-4, GPT-3.5)

from openai import OpenAI

class OpenAIAdapter(ModelAdapter):
    def __init__(self, model="gpt-4o", api_key=None, base_url=None):
        self.client = OpenAI(api_key=api_key, base_url=base_url)
        self.model = model

    def normalize_tools(self, tools):
        # OpenAI uses the standard function-calling format — minimal changes
        return [{"type": "function", "function": t} for t in tools]

    def normalize_messages(self, messages):
        # OpenAI format is the baseline; tool results use role "tool"
        return messages  # Already in correct format

    def chat(self, messages, tools=None, temperature=0.7, max_tokens=1000):
        kwargs = dict(
            model=self.model,
            messages=self.normalize_messages(messages),
            temperature=temperature,
            max_tokens=max_tokens
        )
        if tools:
            kwargs["tools"] = self.normalize_tools(tools)

        resp = self.client.chat.completions.create(**kwargs)
        msg = resp.choices[0].message

        return AgentResponse(
            content=msg.content,
            tool_call={
                "name": msg.tool_calls[0].function.name,
                "arguments": msg.tool_calls[0].function.arguments
            } if msg.tool_calls else None,
            is_final=msg.tool_calls is None,
            usage={
                "input": resp.usage.prompt_tokens,
                "output": resp.usage.completion_tokens
            }
        )

Anthropic Adapter (Claude Sonnet, Claude Opus)

import anthropic

class AnthropicAdapter(ModelAdapter):
    def __init__(self, model="claude-sonnet-4-20250514", api_key=None):
        self.client = anthropic.Anthropic(api_key=api_key)
        self.model = model

    def normalize_tools(self, tools):
        # Anthropic uses a different tool format — no "type": "function" wrapper
        normalized = []
        for tool in tools:
            inner = tool.get("function", tool)  # Unwrap if nested
            normalized.append({
                "name": inner["name"],
                "description": inner.get("description", ""),
                "input_schema": inner.get("parameters",
                    {"type": "object", "properties": {}})
            })
        return normalized

    def normalize_messages(self, messages):
        # Anthropic needs system prompt extracted to separate parameter
        normalized = []
        for msg in messages:
            if msg["role"] == "system":
                continue  # Handled separately
            if msg["role"] == "tool":
                # Anthropic uses "tool_result" blocks inside user messages
                normalized.append({
                    "role": "user",
                    "content": [{
                        "type": "tool_result",
                        "tool_use_id": msg.get("tool_call_id", "unknown"),
                        "content": msg["content"]
                    }]
                })
            else:
                normalized.append({"role": msg["role"],
                                   "content": msg["content"]})
        return normalized

    def chat(self, messages, tools=None, temperature=0.7, max_tokens=1000):
        system = next((m["content"] for m in messages
                       if m["role"] == "system"), None)
        normalized_msgs = self.normalize_messages(messages)

        kwargs = dict(
            model=self.model,
            messages=normalized_msgs,
            max_tokens=max_tokens,
            temperature=temperature
        )
        if system:
            kwargs["system"] = system
        if tools:
            kwargs["tools"] = self.normalize_tools(tools)

        resp = self.client.messages.create(**kwargs)

        # Extract tool use blocks from response
        tool_calls = [
            block for block in resp.content
            if block.type == "tool_use"
        ]

        return AgentResponse(
            content=resp.content[0].text if resp.content[0].type == "text"
                    else None,
            tool_call={
                "name": tool_calls[0].name,
                "arguments": tool_calls[0].input,
                "id": tool_calls[0].id
            } if tool_calls else None,
            is_final=len(tool_calls) == 0,
            usage={
                "input": resp.usage.input_tokens,
                "output": resp.usage.output_tokens
            }
        )

OpenAI-Compatible Adapter (DeepSeek, vLLM, Ollama, local models)

Many models (DeepSeek, Llama via vLLM/Ollama, Groq) use the OpenAI-compatible API. One adapter covers them all — just change the base_url:

# DeepSeek — 10x cheaper for simple tasks, great for Chinese
agent = ModelAgnosticAgent(
    OpenAIAdapter(
        model="deepseek-chat",
        api_key="sk-xxx",
        base_url="https://api.deepseek.com/v1"
    ),
    tools, prompt
)

# Local Llama 3 via Ollama — zero cost, full privacy
agent = ModelAgnosticAgent(
    OpenAIAdapter(
        model="llama3:70b",
        api_key="ollama",  # Ollama ignores the key
        base_url="http://localhost:11434/v1"
    ),
    tools, prompt
)

# Groq — fastest inference for real-time use cases
agent = ModelAgnosticAgent(
    OpenAIAdapter(
        model="llama-3.1-70b-versatile",
        api_key="gsk_xxx",
        base_url="https://api.groq.com/openai/v1"
    ),
    tools, prompt
)

Tool Format Normalization

Different providers have slightly different tool schemas. The key insight: standardize on OpenAI's function-calling format as the internal representation, and let each adapter convert to its native format.

Model Selection Strategy

With a model-agnostic architecture, you can route tasks to the optimal model based on characteristics:

def smart_route(task: str) -> ModelAdapter:
    """Route task to the best model based on heuristics."""
    if any(kw in task.lower() for kw in ["code", "debug", "math", "logic"]):
        return AnthropicAdapter(model="claude-opus-4-20250514")
    if any(kw in task.lower() for kw in ["中文", "chinese", "翻译"]):
        return OpenAIAdapter(model="deepseek-chat",
                            base_url="https://api.deepseek.com/v1")
    if len(task) < 100:  # Simple, short query
        return OpenAIAdapter(model="gpt-4o-mini")
    return AnthropicAdapter(model="claude-sonnet-4-20250514")  # Default

The Complete Agent

Here's the full model-agnostic Agent. The core loop never changes — only the adapter does:

class ModelAgnosticAgent:
    def __init__(self, model: ModelAdapter, tools: list[dict],
                 system_prompt: str):
        self.model = model
        self.tools = tools
        self.messages = [{"role": "system", "content": system_prompt}]
        self.total_cost = 0.0

    def run(self, user_input: str, max_turns: int = 20) -> str:
        self.messages.append({"role": "user", "content": user_input})
        turns = 0

        while turns < max_turns:
            response = self.model.chat(
                self.messages, self.tools,
                temperature=0.7 if turns == 0 else 0.4  # Cool down
            )
            turns += 1

            if response.is_final:
                return response.content

            # Execute tool and feed result back
            tool_name = response.tool_call["name"]
            tool_args = response.tool_call.get("arguments", {})
            result = self._execute_tool(tool_name, tool_args)

            self.messages.append({
                "role": "assistant",
                "content": None,
                "tool_calls": [{
                    "id": f"call_{turns}",
                    "type": "function",
                    "function": {
                        "name": tool_name,
                        "arguments": json.dumps(tool_args)
                            if isinstance(tool_args, dict)
                            else tool_args
                    }
                }]
            })
            self.messages.append({
                "role": "tool",
                "tool_call_id": f"call_{turns}",
                "content": json.dumps(result)
            })

        return "Max turns reached without completion."

Testing Your Model-Agnostic Layer

How do you know the adapter is working correctly? Test with a simple tool-calling task across all models:

def test_adapter(adapter: ModelAdapter):
    """Verify an adapter handles tool calling correctly."""
    tools = [{
        "name": "get_weather",
        "description": "Get current weather for a city.",
        "parameters": {
            "type": "object",
            "properties": {
                "city": {
                    "type": "string",
                    "description": "City name"
                }
            },
            "required": ["city"]
        }
    }]

    messages = [
        {"role": "system", "content": "You are a helpful assistant."},
        {"role": "user", "content": "What's the weather in Tokyo?"}
    ]

    response = adapter.chat(messages, tools)

    assert not response.is_final, "Should request a tool call"
    assert response.tool_call["name"] == "get_weather", \
        f"Wrong tool: {response.tool_call['name']}"
    assert "Tokyo" in str(response.tool_call.get("arguments", "")), \
        "Missing city argument"

# Run against all your adapters
for name, adapter in [
    ("OpenAI", OpenAIAdapter()),
    ("Anthropic", AnthropicAdapter()),
    ("DeepSeek", OpenAIAdapter(base_url="https://api.deepseek.com/v1"))
]:
    try:
        test_adapter(adapter)
        print(f"✅ {name}: PASS")
    except Exception as e:
        print(f"❌ {name}: FAIL — {e}")

Existing Frameworks (and When to Roll Your Own)

Key Takeaways

1. Define a unified adapter interface — one chat() method, one normalize_tools(), one normalize_messages(). Everything else is implementation detail.
2. Standardize on OpenAI's tool format internally — it's the de facto standard that most providers implement or can convert to.
3. Test every adapter with the same tool-calling scenario — tool format translation bugs are silent and hard to debug in production.
4. Smart routing saves money — simple queries on cheap models (DeepSeek at $0.14/M tokens), complex reasoning on premium models (Claude at $3/M tokens). The difference can be 20x in cost.
5. Don't over-engineer — if you only use one model, skip this. But if you're building something that lasts, the adapter pattern is a cheap insurance policy against vendor lock-in.