mcpgen

mcpgen is a command-line tool that takes a URL — either an API documentation page or a regular website — and automatically generates a working MCP server for it. You run one command, point it at a URL, and get back a folder with a Python server you can plug into Claude.

The problem this solves

MCP (Model Context Protocol) is how AI assistants connect to external tools and services. If you want Claude to be able to call a specific API or interact with a website, you need an MCP server that wraps it. Writing those servers by hand is repetitive: you read the docs, figure out which endpoints matter, write boilerplate FastMCP code, set up authentication, and write tests. That takes hours per service.

mcpgen automates that whole process. You give it the URL you would have read manually, and it does the reading, the design decisions, and the code generation for you. It handles two distinct cases:

• API documentation (like https://pokeapi.co/docs/v2) — it reads the docs and generates an MCP server that makes HTTP calls using httpx
• Regular websites with no API (like https://twitter.com) — it generates an MCP server that uses Playwright browser automation to interact with the UI directly

The second case is where it becomes genuinely useful. Most services either have no public API, have a paid API, or have an API that lags behind what you can do through the actual website. Browser automation fills that gap.

How it works

The pipeline has four stages:

1. Crawl — Firecrawl fetches the page and converts it to clean Markdown, stripping ads, navigation, and HTML noise
2. Analyze — an LLM reads the Markdown and produces a structured tool spec: what tools to create, what parameters they take, what authentication they need
3. Generate — a second LLM call takes the tool spec and writes the actual Python server code following specific patterns (structured return dicts, output compression, startup auth checks)
4. Write — the output folder is assembled with server.py, requirements.txt, .env.example, a README.md, and smoke tests

Results from steps 1 and 2 are cached locally so you can tweak generation or try a different model without re-crawling and re-paying for LLM tokens.

Prerequisites

You need accounts and API keys for:

• Firecrawl — for the crawling step. Sign up at firecrawl.dev. The free tier is enough for testing.
• An LLM provider — at least one of: Anthropic (Claude), OpenAI (GPT-4o or GPT-5.5), or Google (Gemini). mcpgen uses LiteLLM under the hood so it supports all of them with the same interface.

You need Python 3.11 or higher. The project uses uv for package management but pip works fine too.

Installation

Clone the repo and install in a virtual environment:

git clone https://github.com/your-repo/mcpgen
cd mcpgen
python -m venv .venv
source .venv/bin/activate      # on Windows: .venv\Scripts\activate
pip install -e .

For UI mode (browser automation), you also need the Playwright browser binaries:

pip install playwright
playwright install chromium

Configuration

Create a .env file in the project root. At minimum you need Firecrawl and one LLM provider:

# Required for crawling
FIRECRAWL_API_KEY=fc-your_key_here

# Pick at least one LLM provider
ANTHROPIC_API_KEY=sk-ant-your_key_here
OPENAI_API_KEY=sk-your_key_here
GOOGLE_API_KEY=your_key_here

You do not need all three LLM providers. If you only have an Anthropic key, just set that one and pick Claude from the model menu when you run mcpgen.

Basic usage

mcpgen https://pokeapi.co/docs/v2

This will:

1. Ask you which model to use (interactive menu)
2. Crawl the PokeAPI documentation
3. Analyze it and discover tools
4. Generate a Python MCP server
5. Write everything to ./pokeapi-mcp/

The output folder looks like this:

pokeapi-mcp/
  server.py           # the MCP server — this is what you run
  requirements.txt    # fastmcp, httpx, python-dotenv
  .env.example        # shows which env vars the server needs
  README.md           # setup instructions specific to this server
  tests/
    __init__.py
    test_tools.py     # smoke tests with httpx mocked out

CLI flags

mcpgen [URL] [OPTIONS]

Arguments:
  URL                  The page to generate an MCP server for

Options:
  --model, -m TEXT     LiteLLM model string (skips the interactive menu)
                       Examples: claude-sonnet-4-6, claude-opus-4-7, openai/gpt-5.5
  --mode TEXT          auto, api_docs, or ui  [default: auto]
  --out, -o PATH       Output directory  [default: {site_name}-mcp]
  --dry-run            Print the discovered tools without generating code
  --no-cache           Ignore cached crawl and tool spec, start fresh
  --cache-only         Skip crawl and analysis, regenerate code from cache

The --dry-run flag is useful for checking what tools mcpgen would create before spending tokens on code generation. The --cache-only flag is useful when you are happy with the tool spec but want to try a different model for code generation.

Model selection

When you run mcpgen without --model, it shows a menu:

Which AI model to use?
  1. claude-sonnet-4-6  (Recommended)
  2. claude-opus-4-7    (Most capable)
  3. openai/gpt-5.5
  4. gemini/gemini-3.1-pro-preview
  5. Enter model name manually

You can also pass any LiteLLM-supported model string directly:

mcpgen https://example.com/api/docs --model claude-opus-4-7
mcpgen https://example.com/api/docs --model openai/gpt-5.5

Note: GPT-5.5 is a reasoning model and does not accept a temperature parameter. mcpgen handles this automatically.

Mode detection

By default mcpgen looks at the URL to decide whether to generate an API client or a browser automation server. URLs containing segments like /docs, /api, /reference, /developers are treated as API documentation. Everything else is treated as a UI target.

You can override this:

# Force API mode even if the URL looks like a regular site
mcpgen https://some-site.com/endpoints --mode api_docs

# Force UI/browser mode
mcpgen https://some-site.com --mode ui

What the generated code looks like

For API mode, the generated server.py follows these patterns:

• Uses httpx for HTTP calls, not requests
• Calls resp.raise_for_status() before processing any response
• Wraps all return values with a _compact() helper that truncates lists to 10 items and strings to 300 characters, so the LLM context does not get flooded
• Includes a startup check that raises a RuntimeError immediately on python server.py if required environment variables are missing, instead of failing silently on the first tool call
• List endpoints include a limit: int = 10 parameter

For UI mode, the generated server.py:

• Uses playwright.sync_api.sync_playwright for browser automation
• Launches Chromium with headless=True by default (set MCPGEN_HEADLESS=false to see the browser)
• Returns structured dicts: {"status": "success", "data": ..., "url": page.url} — never raw page content
• Wraps all browser operations in try/except with finally: browser.close(), so a missing selector does not crash the server

Tool descriptions follow a specific formula designed to help the LLM pick the right tool:

"[ACTION] [OBJECT] — use this when [TRIGGER]. Do not use this to [DISAMBIGUATION]."

For example: "Retrieves a Pokemon's data — use this when you know the Pokemon's name or ID. Do not use this to list all available Pokemon."

Adding to Claude Code

After generating a server, you can add it to Claude Code with one command (shown at the end of the generation output):

claude mcp add pokeapi -- python /absolute/path/to/pokeapi-mcp/server.py

Adding to Claude Desktop or Cursor

Find your MCP config JSON file (claude_desktop_config.json on Desktop, or your Cursor settings) and add an entry:

{
  "mcpServers": {
    "pokeapi": {
      "command": "python",
      "args": ["/absolute/path/to/pokeapi-mcp/server.py"]
    }
  }
}

The server reads its environment variables from the .env file in its own directory, so credentials stay with the server rather than in the MCP config.

Running the tests

pytest

The test suite covers the analyzer, generator, writer, and crawler modules. All tests mock external calls (LiteLLM, Firecrawl, httpx) so you do not need live API keys to run them.

pytest -v              # verbose output
pytest tests/test_writer.py  # run a specific module

Known limitations

Single-page crawl only. mcpgen scrapes one URL. Many API documentation sites spread their reference across dozens of pages. For example, the Stripe API docs have separate pages for each resource type. mcpgen will only see whatever is on the specific URL you give it. For deep coverage of multi-page docs, you need to run it multiple times on different URLs or point it at a single-page reference if one exists.

Generated selectors may be wrong. For UI mode, the LLM has to guess the CSS selectors or test IDs for page elements it has never seen. It uses heuristics from the page text. These selectors will often be wrong or break when the site updates. Expect to manually correct selectors in the generated server.py before it works reliably. The generated code at least gives you the correct structure so you only need to fix the selectors, not rewrite the whole thing.

No session management. Each tool call in UI mode opens a fresh browser, navigates, acts, and closes. There is no persistent login session. If a site requires authentication, the generated server will need manual additions to handle cookies or local storage.

Rate limits are not handled. The generated API servers do not include retry logic or rate limit backoff. If an API returns 429, raise_for_status() will propagate the exception.

Context quality depends on the page. If the documentation page you give it is poorly structured, has a lot of irrelevant content, or uses formats that Firecrawl cannot parse well (some SPAs, heavy JavaScript), the tool spec will be incomplete or inaccurate.

Development setup

The project structure:

mcpgen/
  cli.py          # Typer CLI entry point and pipeline orchestration
  crawler.py      # Firecrawl integration and URL classification
  analyzer.py     # LLM call that produces a ToolSpec from Markdown
  generator.py    # LLM call that produces server.py code from ToolSpec
  writer.py       # Writes the output folder (server.py, README, tests, etc.)
  cache.py        # JSON file cache for crawl results and tool specs
  models.py       # TypedDict definitions (ToolSpec, Tool, ToolParam)
  prompts/
    analyze.md    # System prompt for the analysis LLM call
    generate_api.md   # System prompt for API server code generation
    generate_ui.md    # System prompt for UI/Playwright server code generation
tests/
  test_analyzer.py
  test_crawler.py
  test_generator.py
  test_writer.py

The prompts directory is where the quality of generated output is determined. If you want to improve the generated servers, the system prompts are the place to start.

Dependencies are managed through pyproject.toml. To add a new dependency:

# Edit pyproject.toml, then reinstall
pip install -e .