🎬 Screenwright

Subtitled tutorial videos, scripted. An MCP server that turns a list of steps into a narrated screen recording — by driving a real browser (Playwright) or an Android / Flutter emulator (adb). Captions are burned into the video and exported as an .srt.

The name is a play on Playwright (the engine under the hood) + screen.

steps[]  ──►  Playwright drives the browser  ──►  caption overlay synced per step
         ──►  video recorded  ──►  ffmpeg → .mp4  +  .srt sidecar

✨ Features

• Scripted walkthroughs — goto, click, fill, type, press, hover, scroll, wait.
• Presentation demos — generated 16:9 product tours with slide content, right-side action panel, zooms, transitions, subtitles and optional voice-over.
• Synced subtitles — attach a caption to any step; it's burned into the video and written to a timed .srt.
• Real voice-over (optional) — turn captions into a spoken narration with ElevenLabs or OpenAI TTS. Each line is pre-synthesized and the screen is held while it plays, so audio stays perfectly synced to the subtitles. Configure once via env.
• Self-contained output — bundled ffmpeg (via ffmpeg-static) produces a clean H.264 .mp4.
• Robust — per-action timeouts; mark a step optional so a missing selector doesn't abort the take.
• *Web and mobile — Playwright for the web; `adb` screen-record + ffmpeg caption-burn for Android / Flutter* apps (no app source needed).
• Use it from an AI (MCP tool) or straight from Node.

📦 Requirements

• Node.js 18+
• A Chromium build for Playwright: npx playwright install chromium (run automatically on npm install).

🚀 Install

# from GitHub
npm install -g github:0xjesus/screenwright

# or clone
git clone https://github.com/0xjesus/screenwright.git
cd screenwright && npm install

🔌 Use as an MCP server

Add it to your MCP client. Claude Desktop / Claude Code (claude_desktop_config.json or .mcp.json):

{
  "mcpServers": {
    "screenwright": {
      "command": "node",
      "args": ["/absolute/path/to/screenwright/src/index.js"]
    }
  }
}

(If installed globally, you can use "command": "screenwright" with no args.)

Then just ask your assistant to record a tutorial — it calls the `record_tutorial` tool.

Tool: record_tutorial

Step = { caption?, narration?, action, url?, selector?, text?, key?, deltaY?, delayMs?, timeoutMs?, dwellMs?, optional? }. selector is any Playwright selector — CSS, text=…, xpath=…, or :has-text(…).

Tool: record_presentation_demo 🧑‍🏫

Use this when you want the polished "product tour / launch presentation" format around a real browser walkthrough. This tool is intentionally not slide-only: it first drives the real product with Playwright, records that browser flow, then embeds the recorded demo inside the presentation stage.

• left side: title, metric card, bullets or code
• right side: real operated product demo, plus optional app/image/terminal scenes
• bottom: burned-in subtitle bar + .srt
• optional: ElevenLabs/OpenAI voice-over and background music

Demo = { baseUrl?, steps, viewport?, headless?, channel?, minDurationMs?, finalHoldMs? }, where steps[] uses the same browser actions as record_tutorial.

For long desktop demos, set:

{
  "layout": {
    "mode": "demo-fullscreen",
    "overlay": "compact",
    "demoCoverage": 1,
    "demoFit": "contain",
    "protectDemo": true,
    "safeArea": { "top": 64, "bottom": 118 },
    "syncToDemo": true,
    "demoLoop": false,
    "technicalFullscreen": true,
    "pauseDemoDuringNonDemo": true,
    "defaultSceneDurationMs": 12000
  },
  "demo": {
    "viewport": { "width": 1920, "height": 1080 },
    "minDurationMs": 120000,
    "finalHoldMs": 4000,
    "steps": [
      { "action": "goto", "url": "/SpareParts/LandingRequests/Window", "dwellMs": 3000 },
      { "action": "click", "selector": ".sp-window-open-client", "dwellMs": 5000 }
    ]
  }
}

demo-fullscreen is the desktop walkthrough mode: the operated browser recording becomes the main canvas instead of a small right-side device. By default it uses protectDemo: true, reserving safe areas for branding and captions so explanatory UI never covers the real product screen. It also defaults technicalFullscreen: true, so technical/table/diagram scenes render as clean full slides instead of overlapping the demo stage. pauseDemoDuringNonDemo: true lets those technical scenes appear between demo segments and the real demo resumes where it left off. Keep demoFit: "contain" when every pixel of the app matters; use "cover" only when cropping is acceptable. syncToDemo: true stretches the first demo scene to the raw recording length so a long real flow is visible once instead of being cut or looped.

If demo.steps is missing, the tool fails. If brand.colors.primary or brand.colors.accent is missing, the tool fails. At least one scene must use stage.kind: "demo" or omit stage, otherwise the tool fails because the real operated product demo would not be visible. This keeps generated demos grounded in the real product and the project's visual identity.

Scene = { eyebrow?, title, caption?, narration?, durationMs?, metric?, points?, actions?, code?, stage? }.

stage.kind can be:

• demo — show the real browser demo recorded from demo.steps
• video — show a provided video file
• mock-app — generated app-like phone panel
• image — show a real screenshot/image, with optional focus ring
• terminal — show command/code output in the right panel

Example:

{
  "output": "out/presentation-demo.mp4",
  "brand": {
    "name": "My SDK",
    "mark": "M",
    "tag": "Product tour",
    "colors": { "primary": "#0057B8", "accent": "#E1251B" }
  },
  "demo": {
    "baseUrl": "http://localhost:3000",
    "steps": [
      { "action": "goto", "url": "/", "caption": "Open the app.", "networkIdle": false },
      { "action": "click", "selector": "text=Start", "dwellMs": 1000 }
    ]
  },
  "scenes": [
    {
      "eyebrow": "Launch",
      "title": "My SDK turns a complex workflow into one API",
      "caption": "My SDK turns a complex workflow into one API.",
      "metric": {
        "value": "1 API",
        "label": "Install, connect, execute, and show receipts.",
        "tags": ["SDK", "Flutter", "MCP"]
      },
      "points": [
        { "title": "Install", "body": "Add the package." },
        { "title": "Connect", "body": "Open the wallet flow." },
        { "title": "Execute", "body": "Send the transaction." },
        { "title": "Receipt", "body": "Show status and support states." }
      ],
      "actions": [
        { "title": "Introduce", "body": "Start with the value." },
        { "title": "Show flow", "body": "Highlight the active action." },
        { "title": "Export", "body": "Return MP4 and SRT." }
      ],
      "stage": {
        "kind": "demo",
        "title": "Reference app",
        "subtitle": "The right side shows the operated app."
      }
    }
  ]
}

From Node: import { recordPresentationDemo } from 'screenwright/presentation'. Run the local example with node scripts/run-presentation-example.js.

Tool: record_android_tutorial 📱

Same idea, for a Flutter / Android app on an emulator or device. Captions can't be injected into a native app, so the screen is recorded with adb screenrecord and the captions are burned in afterwards with ffmpeg (libass) from the synced .srt.

Requires: Android platform-tools (adb) + a running emulator/device (it must show in adb devices). For Flutter: flutter emulators --launch <id>, then run your app.

Step actions (coordinates are device pixels) — each may carry a caption and dwellMs:

• tap — { x, y }
• text — { text }
• swipe — { x1, y1, x2, y2, durationMs? }
• key — { key } (BACK, HOME, ENTER, MENU, APP_SWITCH, or a KEYCODE_*)
• launch — { package, activity? }
• wait — { dwellMs }

See `examples/android-flutter.json`. From Node: import { recordAndroidTutorial } from 'screenwright/android'.

🎙️ Voice-over (TTS)

Turn your captions into a real spoken narration — same tool, both drivers (web + Android). Provider and voice are set once via env (the same MCP config), so any record_* call with captions gets narrated automatically. Each line is synthesized before recording and the screen is held while it plays → the voice stays locked to the burned-in subtitles.

Configure in your MCP server env (or shell):

{
  "mcpServers": {
    "screenwright": {
      "command": "node",
      "args": ["/absolute/path/to/screenwright/src/index.js"],
      "env": {
        "SCREENWRIGHT_TTS_PROVIDER": "elevenlabs",
        "ELEVENLABS_API_KEY": "sk_...",
        "ELEVENLABS_VOICE_ID": "21m00Tcm4TlvDq8ikWAM",
        "ELEVENLABS_MODEL": "eleven_multilingual_v2"
      }
    }
  }
}

OpenAI instead: SCREENWRIGHT_TTS_PROVIDER=openai, OPENAI_API_KEY=..., OPENAI_TTS_VOICE=onyx, OPENAI_TTS_MODEL=gpt-4o-mini-tts.

Per-call override (wins over env), plus a narration field when the spoken line should differ from the on-screen caption:

await recordAndroidTutorial({
  output: 'out/tour.mp4',
  tts: { provider: 'elevenlabs', voiceId: '…', speed: 1.0, stability: 0.5 },
  steps: [
    { action: 'launch', package: 'com.acme.app', caption: 'Meet Acme', narration: 'This is Acme — your market copilot.', dwellMs: 1500 },
    { action: 'tap', x: 540, y: 1925, caption: 'Generate a signal', dwellMs: 1000 },
  ],
});
// → { mp4, srt, durationMs, steps, captions, narration: { provider, lines } }

• `tts` fields: provider (elevenlabs|openai), apiKey?, voiceId?/voice?, model?, speed?, stability?, similarityBoost?, style?, tailPadMs? (silence after each line, default 450).
• Requires the bundled ffmpeg (already a dependency) to mix + mux the audio. Multilingual voices (e.g. eleven_multilingual_v2) narrate Spanish/English captions cleanly.
• No tts and no env provider → silent captions, exactly as before.

🧪 Use from Node (no MCP)

node scripts/run-example.js                 # records examples/demo.json
node scripts/run-example.js examples/capleton.json

import { recordTutorial } from 'screenwright/recorder';

await recordTutorial({
  output: 'out/tour.mp4',
  baseUrl: 'https://playwright.dev',
  steps: [
    { action: 'goto', url: '/', caption: 'Welcome to the tour 🎬', dwellMs: 2600 },
    { action: 'scroll', deltaY: 600, caption: 'Each step can carry a synced subtitle…', dwellMs: 2400 },
    { action: 'goto', url: '/docs/intro', caption: '…burned into the video and exported as .srt', dwellMs: 3000 },
  ],
});
// → { mp4, srt, durationMs, steps, captions }

💡 Tips

• Don't double your subtitles. The .mp4 already has burned-in captions. Players like VLC auto-load an .srt that shares the video's name and draw it on top. Screenwright defaults the sidecar to *.captions.srt (a non-matching name) so that doesn't happen — keep it that way, or disable the subtitle track in your player. Want selectable-only subs? Pass burnIn: false for a clean video + the .srt.
• Black video in VLC on Linux? That's a GPU/output glitch, not the file. Launch with vlc --avcodec-hw=none --vout=xcb_x11, or set those in VLC → Preferences.
• Headless still records — no display needed.

📝 License

MIT © 0xjesus