autopilot-mcp

A browser-automation MCP server. It hands an LLM a generic set of browser tools — navigate, screenshot, read text, run JS, click, type — that work on any URL, with each site backed by its own persistent Camoufox browser profile. Logins are filled straight from a Bitwarden vault so passwords never enter the model's context, and any sequence of steps that works can be saved as a "playbook" for one-call replay next time.

Tools

Browser (free-roam)

Each registrable domain (eTLD+1) gets its own persistent browser profile under data/profiles/<profile>/. navigate(url) auto-derives the profile from the URL; everything else takes profile explicitly.

run_js takes either an expression or a statement body — scripts using return or top-level await are wrapped in an async IIFE automatically.

For parallel work on the same site, use isolated browser instances — each clones the site's base profile so concurrent sessions don't collide:

Example: spawn_instance(url="https://accounts.example.com/...", clone_from_profile="example.com") lets each parallel branch work in its own cloned session. Always call close_instance(instance_id) when the branch is finished; timed-out instances are also cleaned up automatically.

Credentials (Bitwarden, fill-don't-reveal)

Local file server (uploads)

For sites that ask the user to upload a local file. Two paths:

1. Standard `<input type="file">` — use `attach_file(profile, selector,

path)`. Works even when the input is hidden inside a custom dropzone widget; target the input itself, not the visible drop area.

1. Pure-JS uploader (no real input element) — use the local CORS file

server below. The MCP publishes the file at an unguessable URL on 127.0.0.1; the LLM uses run_js to fetch() it inside the page, wrap the Blob in a File, and dispatch a synthetic drop event (or set it on a hidden input via DataTransfer).

Security envelope: server binds 127.0.0.1 only; tokens are uuid4 hex (122 bits of entropy); one token = one file path (no directory traversal); idle entries reaped on every request. Override the bind via AUTOPILOT_FILE_SERVER_HOST / AUTOPILOT_FILE_SERVER_PORT env vars.

Playbooks

Workflow

1. list_playbooks(url_match) — is there already a playbook for this task?
2. run_playbook(name) — if yes, run it. Done.
3. Otherwise: navigate(url) → screenshot / get_text → run_js / click / type_text.
4. On a login page: fill_login(url) — Bitwarden injects creds directly. If the form needs 2FA: get_totp(vault_item) then type_text(profile, code).
5. For SMS 2FA: open your messages-on-web client in its own profile and read the code from there.
6. After the task succeeds, save_playbook(...) so next time is one call.
7. Just signed up somewhere new? upsert_login(url, username, password) stores it and Bitwarden sync pushes to your other devices.

Credentials setup (Bitwarden)

The MCP unlocks Bitwarden with a master password stashed in the OS keyring (DPAPI-encrypted on Windows, scoped to your user). It never lands on disk outside the keyring, and never enters the model's context.

How the vault is reached

The MCP drives a `bw serve` daemon — one bw process, unlocked once, that answers over loopback HTTP. It starts on the first credential call and is locked and killed after 15 idle minutes (and on shutdown).

One daemon per box, not per MCP process. A client that starts one autopilot MCP per job ends up with several at once, so they rendezvous through data/bw-serve.json (port + a shared last-touch stamp, guarded by a lock file): the first process in spawns and owns the daemon, later ones adopt its port. Only the owner locks or kills it, and only once every process has been idle past the window — otherwise a sibling mid-login would have the vault yanked out from under it. A state file pointing at a dead port is detected by probe and replaced.

This replaced a per-operation bw <cmd> --session … design, which is broken on Bitwarden CLI ≥ 2026.3.0: bw unlock --raw returns a session that every subsequent process rejects with Vault is locked. Reads survived on a local-decryption fallback (LocalBitwardenVault); writes had no fallback and failed outright, and nothing ever synced. Reproduce the underlying bug with:

S=$(bw unlock --raw --passwordenv BW_PW); bw list items --session "$S"   # Vault is locked.

Set AUTOPILOT_BW_TRANSPORT=cli to force the old path (it still works for reads); serve to require the daemon and fail loudly if it can't start; auto (default) prefers the daemon and falls back to the CLI.

Security posture: bw serve has no authentication, so the daemon binds 127.0.0.1 on a random port (never the well-known 8087), keeps Bitwarden's origin protection on, and does not outlive its idle window. Any local process running as you could still reach it while it is up — that is the same exposure as the keyring master password itself, but keep the idle window short.

Staying in sync

vault_status() reports lock state, item count, and sync_age_minutes. The daemon syncs on start, before the first read once the local snapshot is older than AUTOPILOT_BW_SYNC_TTL_MINUTES (default 30), and after every write. Without that TTL the snapshot silently drifts: on the CLI path sync only ever ran after a write, and writes were failing, so the vault could go weeks stale and logins added in the Bitwarden app simply did not exist as far as the model could tell.

TOTP

Codes are computed locally with pyotp from the seed stored on the vault item. Bitwarden's own bw get totp / GET /object/totp are Premium-gated and answer Premium status is required to use this feature. without it. Only items whose list_logins row shows has_totp: true can produce a code.

One-time setup for a fresh machine, top to bottom:

1. Install the Bitwarden CLI

winget install --id Bitwarden.CLI --accept-source-agreements --accept-package-agreements

winget puts bw.exe on PATH via a shim — open a new shell afterward so the update takes effect. (No winget? npm install -g @bitwarden/cli, or grab a binary from <https://bitwarden.com/download/>.) Verify:

bw --version   # e.g. 2026.3.0
bw status      # {"status":"unauthenticated", ...}

2. Log in

Interactive — only your terminal sees the master password.

bw login

Prompts for email, master password, and a two-step token. On success bw status reports "status":"locked" — leave it locked; the MCP unlocks on demand.

3. Stash the master password in the OS keyring

Keep it out of .env and off the command line. After uv sync, stash it at the hidden prompt:

uv run python -c "import keyring, getpass; keyring.set_password('autopilot-mcp', 'bw_master', getpass.getpass('Master password: ')); print('stored')"

This writes to service autopilot-mcp, username bw_master. Confirm without printing the value:

uv run python -c "import keyring; v = keyring.get_password('autopilot-mcp', 'bw_master'); print(f'present={v is not None} length={len(v) if v else 0} backend={keyring.get_keyring().__class__.__name__}')"
# present=True length=<your pw length> backend=WinVaultKeyring

4. Smoke-test the unlock loop

Runs the real path — keyring read, daemon start, unlock, list, lock — without printing the password:

uv run python -c "
import credentials
c = credentials.BitwardenClient()
print(c.vault_status())
c.lock()
"

A transport: serve, status: unlocked, and a non-zero item_count mean setup is done. A transport: cli line means the daemon could not start — serve_unavailable in the same output says why.

Maintenance

• Rotate the master password — re-stash; the entry is overwritten in place:

  uv run python -c "import keyring, getpass; keyring.set_password('autopilot-mcp', 'bw_master', getpass.getpass('New master password: '))"

• Remove the keyring entry (the MCP then fails at startup until restored):

  uv run python -c "import keyring; keyring.delete_password('autopilot-mcp', 'bw_master')"

• `bw` fell off PATH — open a new shell (winget's PATH update doesn't reach

already-open shells); if still missing, re-run the install from step 1.

• Force a re-sync — vault_status(sync=True) from the MCP, or bw sync

from a shell. Only needed if the vault was edited elsewhere and you don't want to wait out AUTOPILOT_BW_SYNC_TTL_MINUTES.

• Duplicate entries for one site — several logins can legitimately share a

URL (one per account). Lookups refuse to guess between them; pass account="<username>" to fill_login / get_totp, or vault_item=<id> from the ids the error lists.

• Log out — bw logout drops the account from local bw state; repeat

steps 2–3 to restore.

Initial browser session setup

Each profile gets one persistent browser profile the first time it's opened. For sites where you want the session pre-established (to handle 2FA challenges / "remember me" outside the MCP flow):

uv run python scripts/manual_login.py <url>

A visible Camoufox window opens at the URL. Log in, complete 2FA, check "remember me", close the window. The profile at data/profiles/<eTLD+1>/ persists across headless MCP invocations.

Environment variables

All optional — defaults are sane for local use.

Credentials are pulled from Bitwarden — there are no per-site username/password environment variables.

Development

uv sync --extra dev
uv run camoufox fetch
uv run ruff check .
uv run pytest
uv run python server.py    # stdio mode