Scout MCP

Free-first web access for AI agents. Bright Data only when a site actually blocks you.

Scout is a Model Context Protocol server that gives your AI agent a full browser and scraping toolkit. Most pages are fetched for free with a plain HTTP request. The paid Bright Data tier only kicks in when an anti-bot wall (Cloudflare, Akamai, CAPTCHA, etc.) is detected — which means most research workflows cost nothing.

No API key required to get started. Free tools work out of the box. Configure Bright Data only when you need it.

How routing works

Every fetch follows a cost ladder and stops at the first rung that works:

1. Free direct fetch  →  realistic browser headers, no cost
2. Bright Data        →  paid proxy + unlocker, only on detected block

An in-memory skip-list (persisted to disk) remembers which domains blocked the free tier so repeat calls go straight to Bright Data — no wasted attempt. Skip-list entries expire after 30 minutes so the server re-probes in case a site drops its protection.

Tools

Smart (cost-aware) tools

Bright Data tools (always paid)

Quickstart

git clone https://github.com/AndrewEstopinan/browser-mcp
cd browser-mcp
npm install && npm run build
node dist/index.js   # works immediately — no API key needed for free tools

To unlock paid tools, set BRIGHTDATA_API_KEY:

node --env-file=.env dist/index.js

Configuration

Copy .env.example to .env and fill in what you need.

Add to your MCP client

Add this to your Claude Desktop, Cursor, or other MCP client config:

{
  "mcpServers": {
    "scout": {
      "command": "node",
      "args": ["/absolute/path/to/browser-mcp/dist/index.js"],
      "env": {
        "BRIGHTDATA_API_KEY": "your_api_key_here",
        "BRIGHTDATA_UNLOCKER_ZONE": "web_unlocker1",
        "BRIGHTDATA_SERP_ZONE": "serp_api1",
        "BRIGHTDATA_BROWSER_AUTH": "brd-customer-XXXX-zone-YYYY:zzzz"
      }
    }
  }
}

For free-only use, omit the env block entirely.

Example calls

Fetch an article (content only, no nav/ads):

{ "tool": "smart_scrape", "arguments": { "url": "https://example.com/article", "data_format": "content_only" } }

Fetch 10 URLs in parallel:

{ "tool": "smart_scrape_batch", "arguments": { "urls": ["https://a.com", "https://b.com"], "concurrency": 5 } }

Crawl a docs site up to 20 pages:

{ "tool": "smart_crawl", "arguments": { "start_url": "https://docs.example.com", "max_pages": 20, "max_depth": 3 } }

Check if a page changed since last time:

{ "tool": "smart_diff", "arguments": { "url": "https://example.com/pricing" } }

Extract JSON-LD and Open Graph metadata:

{ "tool": "smart_extract", "arguments": { "url": "https://example.com/product/123" } }

Parse an RSS feed:

{ "tool": "parse_feed", "arguments": { "url": "https://example.com/feed.xml" } }

Drive a cloud browser through a CAPTCHA:

{
  "tool": "browser_scrape",
  "arguments": {
    "url": "https://hard-site.example",
    "solve_captcha": true,
    "screenshot": true,
    "actions": [
      { "type": "wait_for_selector", "selector": "#results" },
      { "type": "click", "selector": "button.load-more" }
    ]
  }
}

Responsible use

Scout accesses public web data through a paid, compliant commercial service. Respect each target site's Terms of Service and robots.txt (use the built-in check_robots tool), applicable laws, and Bright Data's Acceptable Use Policy. Don't use it to access content behind logins you aren't authorized to use, or to collect personal data without a lawful basis.

License

MIT