e2e-skills — Playwright and Cypress E2E Test Generation, Review, and Debugging

Find Playwright and Cypress E2E tests that pass CI but prove nothing, generate new end-to-end coverage, and turn failing test reports into root-cause fixes — as Agent Skills for Claude Code, Codex, and other [`AGENTS.md`](https://agents.md) coding agents.

E2E tests that always pass are worse than no tests — they give false confidence while real bugs slip through. e2e-skills is an AI agent testing toolkit for Playwright and Cypress: generate end-to-end tests, review existing specs for false positives and test smells, debug flaky E2E failures, and turn noisy CI reports into root-cause fixes. It runs as an Agent Skills bundle for Claude Code and Codex (and other AGENTS.md-compatible runtimes via the skills CLI) by @voidmatcha, catching what CI misses: tests that pass but prove nothing, and failures that are hard to trace.

Four complementary skills cover the full E2E testing lifecycle, from Playwright test generation to Cypress test review and failure debugging:

1. `playwright-test-generator` — generates Playwright E2E tests from scratch, from coverage gap analysis to passing, reviewed tests
2. `e2e-reviewer` — static analysis of existing Playwright and Cypress specs; flags 24 anti-patterns (P0 silent always-pass, P1 poor diagnostics, P2 maintenance) that can make tests pass CI while missing real regressions
3. `playwright-debugger` — diagnoses failures from playwright-report/ and classifies root causes (flaky timing, selector drift, auth, environment mismatch, and more)
4. `cypress-debugger` — same for Cypress report files

Contents

• Install · Workflow · Standalone scanner · Proven in OSS
• Skills: generator · reviewer · playwright-debugger · cypress-debugger
• FAQ · License

Install

# Claude Code + Codex (most common)
npx skills add voidmatcha/e2e-skills --skill '*' -g -a claude-code -a codex

# Codex plugin — via the skills CLI (Codex only)
npx skills add voidmatcha/e2e-skills --skill '*' -g -a codex

# Claude Code plugin — marketplace
/plugin marketplace add voidmatcha/e2e-skills
/plugin install e2e-skills@voidmatcha

# Claude Code plugin — manual clone
git clone https://github.com/voidmatcha/e2e-skills.git ~/.claude/skills/e2e-skills

# Every agent the skills CLI supports (55+ hosts)
npx skills add voidmatcha/e2e-skills --skill '*' -g --agent '*'

The Codex command above is the Codex plugin install: the skills CLI places the bundle in ~/.agents/skills/, where Codex auto-discovers it and reads .codex-plugin/plugin.json (the interface block). There is no separate codex plugin marketplace add step — a native marketplace entry would require duplicating the shared skills/ tree into a per-plugin subdirectory (Codex marketplace plugins cannot reference the repo root, openai/codex#17066), so the CLI route is the supported Codex path.

Quick Example

You: Review my Playwright tests in apps/viewer/src/test/

e2e-reviewer:

  [P0] settings.spec.ts:88, 99 — #4h One-shot URL read
    expect(page.url()).toEqual(`${baseURL}/${id}-public`);   // sync read, no auto-retry
    → fix: await expect(page).toHaveURL(`${baseURL}/${id}-public`);
    (also removes redundant `await page.waitForTimeout(1000)` above)

  [P0] fileUpload.spec.ts:67 — #16 Missing await on action
    page.getByRole('button', { name: 'Delete' }).click();   // fire-and-forget, races next line
    → fix: await page.getByRole('button', { name: 'Delete' }).click();

  Total: 3 P0 (2 #4h, 1 #16), 0 P1, 0 P2 in 24 spec files.
  P1/P2 candidates (not yet flagged as bugs): 20× positional .nth() selectors, 5× direct page.click(selector).

Real findings from a recent typebot.io scan — silent always-pass bugs your test suite was hiding.

Workflow

1. Run playwright-test-generator → generate with approval → auto-reviewed by e2e-reviewer
2. Generated tests fail → playwright-debugger invoked automatically after 3 fix attempts
3. Existing tests: e2e-reviewer → fix → re-run
4. Tests fail → playwright-debugger or cypress-debugger → fix → re-run

Standalone Scanner

./skills/e2e-reviewer/scripts/scan.sh path/to/tests

Three tiers run in priority order: (1) eslint-plugin-playwright / eslint-plugin-cypress — uses your local install if present, otherwise auto-downloads via npx --yes (set E2E_SMELL_NO_ESLINT_DOWNLOAD=1 to disable); (2) ast-grep Tree-sitter rules for FP-prone patterns — uses ast-grep / sg on PATH if present, otherwise auto-downloads via npx --yes @ast-grep/cli (set E2E_SMELL_NO_AST_GREP_DOWNLOAD=1 to disable); (3) bundled regex coverage for grep-detectable P0/P1/P2 patterns and gaps the lint plugins miss — Cypress cy.on('uncaught:exception', () => false) blanket suppression (#3b), {timeout:0}.should("not.exist") (#4g), and cross-framework heuristics. See docs/e2e-test-smells.md for the full P0/P1/P2 model. Use // JUSTIFIED: <reason> on (or in the comment block directly above) an intentional pattern to suppress it in the bundled scanner output; the eslint tier does not parse JUSTIFIED markers — pair with an eslint-disable comment there if needed. The eslint tier also runs under a hang watchdog (E2E_SMELL_ESLINT_TIMEOUT_SECS, default 300s) and never blocks Tier 2/3 coverage when it fails.

The e2e-reviewer skill adds what no lint can reach: semantic checks (name-assertion mismatch, missing Then, YAGNI/zombie specs, POM consistency, auth setup analysis) and fix guidance with band-aid awareness. Run eslint-plugin-playwright / eslint-plugin-cypress as your every-commit baseline; invoke the skill for PR review, suspected silent-pass bugs, or before bulk fixes.

Scanner findings are candidates, not verdicts

Static detection (lint, ast-grep, regex) can only flag candidates. It cannot tell a real silent-always-pass from a harmless one, and it both over- and under-reports:

• False positives — an expect() inside an awaited Promise.all([...]) (the assertion is awaited), or a smell inside a test.fixme/describe.skip block that never runs, both look like bugs to a grep but ship nothing.
• Blind spots — expect(page.getByRole('alert')).toBeTruthy() asserts an always-truthy Locator object (the DOM is never queried), yet no off-the-shelf lint rule flags a bare locator-as-truthy.
• Beyond any lint — a test that wraps addEdge() in try/catch and asserts only inside the catch passes forever, because addEdge never throws (it calls onError and returns the list unchanged), so the catch body never runs. No grep or AST rule can know a function doesn't throw — only reading the code does. (Real case: xyflow graph-utils.cy.ts.)

That gap is the whole point of the judgment layer. e2e-reviewer verifies each finding — reading the surrounding code, the CI config, and the test's intent to decide whether a real bug actually ships green and whether retries could mask it — instead of dumping raw hits. Scanner-positive is not the same as merge-worthy: every one of the merged PRs below survived that verification, and several flagged-but-backstopped candidates were correctly rejected before they ever became a PR. Detection is cheap and commoditised; the verification and the fix are where the value is.

Proven in Open Source

Seven real merged PRs, not synthetic examples:

Beyond those merged PRs, the skill was iterated and validated against 100+ open-source Playwright and Cypress test suites (many 1k+ stars) in a local testbed — zero GitHub side effects, no forks or PRs opened during research. Real findings from those scans drove concrete rule changes: the 4.4 cycle-count rule, the 4.2 PR-culture cross-check, the Phase 2 retry-wrapper skip, the legacy cypress/integration/**/*.js glob coverage, and the awaited-locator (expect(await locator)) variant of the missing-await check all came from observed agent behavior and real anti-patterns surfaced across those runs. See the contribution roadmap for merged, in-review, and queued PRs (with before/after lessons on the merged ones).

Skill 1: playwright-test-generator — Test Generation

Generates Playwright E2E tests from scratch for any project. Starts from coverage gap analysis, explores the live app via agent-browser tools, designs scenarios with your approval, and auto-reviews generated tests with e2e-reviewer.

When to Use

• You have a page or feature with no E2E coverage
• You want to bootstrap a test suite for an existing app
• You need to quickly add tests before a release

Usage

Generate playwright tests
Generate playwright tests for the login page
Write e2e tests for the settings page
Add playwright coverage for checkout flow

Pipeline

1. Detect environment — config, baseURL, test dir, POM structure, existing conventions doc
2. Coverage gap analysis — user picks target (skipped when target given as argument)
3. Live browser exploration — via agent-browser tools (no hallucinated selectors); accessible-name reality check for label-less inputs
4. Scenario design + approval gate — shows plan and locator table before any code
5. Code generation — POM + spec or flat spec, auto-detected from project conventions; writes must be route-stubbed (see Network Determinism in code-rules.md)
6. Conventions & seed scaffolding (first run on a project) — appends a project-adapted E2E section to AGENTS.md and designates a seed spec, so future AI-generated tests (Claude Code, Codex, Playwright Agents) stay consistent
7. YAGNI audit + e2e-reviewer — removes unused locators, catches P0 issues before first run
8. TS compile + test run — 3 auto-fix attempts on failure (heal-by-intent locator re-resolution), then hands off to playwright-debugger

Skill 2: e2e-reviewer — Quality Review

Catches issues in E2E tests that pass CI but fail to catch real regressions.

When to Use

• Your tests always pass but bugs still slip through to production
• Tests pass CI but you suspect they miss real regressions
• Your test suite is fragile — tests break on every UI change
• You want to audit test quality before a release or code review
• You're reviewing Playwright or Cypress specs

Usage

Review my E2E tests
Audit the spec files in tests/
Find weak tests in my test suite
My tests always pass but miss bugs
Tests pass CI but miss regressions
My tests are fragile and break on every UI change
We have coverage but bugs still slip through

24 Patterns Detected — Grouped by Severity

#### P0 — Must Fix (silent always-pass)

Tests pass when the feature is broken. No real verification is happening.

#### P1 — Should Fix (poor diagnostics / wastes CI time)

Tests work but mislead developers, waste CI time, or set up future regressions.

#### P2 — Nice to Fix (maintenance / robustness)

Weak but not wrong — addressed when refactoring.

References

Playwright best practices · Cypress best practices · Testing Library guiding principles

Skill 3: playwright-debugger — Playwright Failure Debugger

Diagnoses Playwright test failures from a playwright-report/ directory — whether failures happened locally or in CI. Classifies root causes and provides concrete fixes.

When to Use

• You have a playwright-report/ directory (local or downloaded from CI) with failures to understand
• Tests pass locally but fail in CI
• You're dealing with flaky or intermittent test failures
• You get TimeoutError or locator not found without a clear cause

Usage

Debug these failing tests
Why did these tests fail?
Tests pass locally but fail in CI

Note: Provide the report as a local path. Download CI artifacts manually from GitHub Actions and pass the directory path — automatic artifact fetching is not supported.

15 Root Cause Categories

Debug Workflow

1. Extract — parse results.json for failed tests, error messages, duration
2. Classify — map each failure to F1–F15 using error signals (most failures resolved here)
3. Trace — if still unclear, extract trace.zip and inspect step-by-step: failed actions, DOM snapshots, network errors, JS console errors
4. Fix — concrete code suggestion per failure, P0/P1/P2 priority

Skill 4: cypress-debugger — Cypress Failure Debugger

Diagnoses Cypress test failures from mochawesome or JUnit report files. Classifies root causes and provides concrete fixes.

When to Use

• You have a cypress/reports/ directory (local or downloaded from CI) with failures to understand
• Cypress tests pass locally but fail in CI
• You're dealing with flaky or intermittent Cypress failures
• You get Timed out retrying or Expected to find element without a clear cause

Usage

Debug these failing Cypress tests
Why did these Cypress tests fail?
Analyze cypress/reports/
Cypress tests pass locally but fail in CI

15 Root Cause Categories

Debug Workflow

1. Extract — parse mochawesome.json or JUnit XML for failed tests, error messages, duration
2. Classify — map each failure to F1–F15 using error signals (most failures resolved here)
3. Screenshot/Video — if still unclear, inspect cypress/screenshots/ and cypress/videos/
4. Fix — concrete code suggestion per failure, P0/P1/P2 priority

FAQ

What is e2e-skills?

e2e-skills is an open-source AI agent testing toolkit for Playwright and Cypress. It bundles four Agent Skills that generate end-to-end tests, review existing specs for silent always-pass anti-patterns, and debug flaky failures — running inside Claude Code, Codex, and other AGENTS.md-compatible AI coding agents.

How do I find Playwright or Cypress tests that pass but don't actually test anything?

Run the e2e-reviewer skill (or its standalone scanner, scan.sh) against your spec directory. It flags 24 anti-patterns grouped by severity (P0/P1/P2) — including missing await on assertions, one-shot isVisible() reads, matcher-less expect(), and committed .only leaks — that let a test stay green while the feature it covers is broken.

How is this different from eslint-plugin-playwright or eslint-plugin-cypress?

The eslint plugins are your every-commit baseline for syntactic rules, and the scanner runs them first (Tier 1). e2e-reviewer adds what lint cannot reach: semantic checks (name-assertion mismatch, missing-Then, auth-setup analysis, zombie specs) plus fix guidance that avoids band-aids, layered on top with ast-grep and regex coverage.

Does it work with Cypress as well as Playwright?

Yes. Both are first-class: test generation and the richest review target Playwright, while review and failure debugging fully cover Cypress (mochawesome and JUnit reports).

Can it debug flaky tests that only fail in CI?

Yes. playwright-debugger and cypress-debugger read your report files (playwright-report/, cypress/reports/) and classify each failure into 15 root-cause categories — flaky timing, selector drift, test isolation, environment mismatch, hydration race, and more — with a concrete fix per failure.

How do I review AI-generated E2E tests?

Point e2e-reviewer at the generated specs. AI-written tests frequently contain confident-looking but silent always-pass assertions; the reviewer surfaces them with before/after fixes before they reach your main branch.

Which AI coding agents are supported?

Claude Code (plugin marketplace or the skills CLI), Codex, and any agent the skills CLI supports via AGENTS.md (55+ hosts). Install once, use everywhere.

Does it support test frameworks other than Playwright and Cypress?

No — Playwright and Cypress only, by design. See framework scope for the rationale.

License

Apache-2.0. See LICENSE.