multi-model-mcp

An MCP server that exposes tools for sub-agent style reasoning across multiple LLM providers. From Claude Code (or any MCP client), you can delegate prompts to OpenAI, Anthropic, Gemini, Groq, Ollama, OpenRouter, and any LiteLLM-supported provider — then run critique loops, debates, red-teaming, and answer ranking without leaving your conversation.

Tools

reason_together strategies

• `independent_then_critique` (default): All models answer independently → critic synthesizes
• `debate`: Models see each other's answers and refine over N rounds → critic synthesizes
• `red_team`: Proposer answers → red teamers attack → proposer revises (N rounds) → critic finalizes

Setup

1. Install

Requires Python ≥ 3.11 and uv.

git clone https://github.com/YOUR_USERNAME/multi-model-mcp
cd multi-model-mcp
uv sync

2. Configure models

Copy and edit models.yaml — it ships with common models pre-configured. Each entry is a model alias pointing to a LiteLLM model string:

models:
  gpt:
    litellm_model: gpt-4.1
    api_key_env: OPENAI_API_KEY

  claude:
    litellm_model: claude-sonnet-4-5
    api_key_env: ANTHROPIC_API_KEY

  local:
    litellm_model: ollama/qwen3:latest
    api_base: http://localhost:11434   # no key needed

Add any provider LiteLLM supports: Groq (groq/llama-3.3-70b-versatile), Mistral, Together AI, DeepSeek, OpenRouter (openrouter/...), etc.

3. Set API keys

cp .env.example .env
# edit .env with your keys

Only keys for providers you actually use are required.

4. Register with Claude Code

Add to your project's .mcp.json (or ~/.claude.json for global):

{
  "mcpServers": {
    "multi-model": {
      "command": "uv",
      "args": [
        "run",
        "--project", "/path/to/multi-model-mcp",
        "multi-model-mcp"
      ],
      "env": {
        "OPENAI_API_KEY": "sk-...",
        "ANTHROPIC_API_KEY": "sk-ant-...",
        "GEMINI_API_KEY": "...",
        "MODELS_CONFIG_PATH": "/path/to/multi-model-mcp/models.yaml"
      }
    }
  }
}

Or if you install it:

uv tool install .

Then use "command": "multi-model-mcp" without args.

Example Claude Code usage

# Simple query
Use ask_model with alias "gpt" to explain backpressure in streaming systems.

# Parallel comparison
Use ask_many with aliases ["gpt", "claude", "gemini"] to explain the CAP theorem.
Compare their answers.

# Multi-model reasoning
Use reason_together with task "Should we use event sourcing for this service?"
model_aliases ["gpt", "gemini"], critic_model_alias "claude", strategy "independent_then_critique"

# Debate
Use reason_together with task "Is GraphQL worth the complexity over REST?"
model_aliases ["gpt", "claude"], critic_model_alias "gemini", strategy "debate", rounds 2

# Red-team a decision
Use reason_together with task "Our plan is to use a single Postgres instance for all tenants"
model_aliases ["gpt", "gemini", "groq"], strategy "red_team", rounds 2

# Critique a draft
Use critique_answer with question "What is eventual consistency?"
draft_answer "It means data will eventually be the same across nodes."
model_aliases ["claude", "gpt"]

# Pick the best
Use pick_best_answer with question "What is the best way to handle auth tokens?"
candidate_answers ["Store in localStorage", "Store in httpOnly cookies", "Store in memory only"]
judge_model_alias "claude"

Configuration reference

models.yaml fields

LiteLLM model strings by provider

See LiteLLM providers docs for the full list.

Design notes

• No key leakage: API keys are never logged; errors are sanitized before returning.
• Failure isolation: one model failing in ask_many / reason_together does not crash the call.
• Synthesis ≠ truth: reason_together presents the critic's output as a synthesized answer, not ground truth.
• No hidden reasoning exposed: traces summarize what happened (which model, which step) without exposing chain-of-thought internals.
• Easy to extend: add any LiteLLM-supported model in models.yaml with no code changes.