Create, run, and maintain Momentic E2E tests and modules, which are serialized to disk as *.test.yaml and *.module.yaml files. Momentic uses fast, accurate AI…
SaferSkills independently audited momentic-test (Agent Skill) and scored it 100/100 (green). The audit ran 55 deterministic rules across Security, Supply Chain, Maintenance, Transparency, and Community; it found 0 high-severity and 0 lower-severity findings. The full rule-by-rule trace and per-finding evidence are below. Free, methodology-open.
Findings & checks · 0 flagged
Every scanned point with the score it earned and what moved between them.
First recorded scan — no prior version to compare against.
The primary manifest — the file an agent reads to learn what this artifact does.
Momentic turns structured natural language into browser automation using forked Playwright, CDP, and custom agents. Interactive steps resolve natural-language targets, assertions can use multimodal models, and AI actions can complete broader goals.
Tests use *.test.yaml; reusable modules use *.module.yaml. Test IDs live in the test file's id field. Check momentic.config.yaml and test-level metadata before assuming project defaults apply.
fileType: momentic/test/v2 and fileType: momentic/module/v2 identify v2 files. Treat missing or different fileType as deprecated v1. Never edit v1 YAML directly; persist changes through momentic_test_splice_steps.
A v2 test selects its environment with defaultEnv, and --env <name> overrides it. There is no fallback to the only environment in the config. The base URL comes from --url-override, then the test's own url, then env.BASE_URL, so a test that sets url keeps it in every environment. With no base URL the run fails: "Cannot run test with no base URL and no BASE_URL variable defined in its environment".
V2 can reference modules, JavaScript, and auth state with relative paths. Paths resolve from the YAML file containing the reference. Use ./... or ../..., never absolute paths or ~. Before moving, renaming, or deleting a referenced file, grep its path and update or remove every reference. Do not add internal or generated fields to v2 YAML.
Prefer these compact workflows unless the user requests something else:
or references are uncertain -> reload or restart if validating -> run the relevant range. This is fastest and produces the clearest reviewable diff.
prerequisites -> preview a logical checkpoint -> splice it -> validate the saved range -> terminate. This grounds each edit in the live app without repeatedly running the whole test.
momentic_test_create -> author a known v2 sequencein one YAML edit, or use MCP step-by-step when discovery is needed. momentic_session_start only opens an existing test.
Use MCP as the default interactive interface: it returns structured results, screenshots, artifacts, step refs, and pollable execution. Preview and run tools wait 30 seconds by default, then return a stepRunnerId while work continues. Leave timeoutSeconds unset for most calls, especially single preset steps. Set it below 30 seconds only when the coding agent's tool-call timeout is shorter. Raise it only to wait longer within a larger tool-call timeout; poll returned handles with momentic_poll_runner.
Polling has a separate timeoutSeconds (default 0, maximum 30). Pass the returned stepRunnerId and use a nonzero wait within the coding agent's tool-call timeout instead of repeatedly polling without a wait.
Confirm the test goal, user-visible success criteria, start environment, auth, and env requirements. Ask before previewing or running any step or AI Action that may submit, purchase, delete, send, create, or cause another non-idempotent side effect. If approved, execute it at most once. Also ask before editing a shared module, restarting a long flow, or running an expensive full test.
AI cannot see the target or the user requires selector-level precision.
under Test execution behavior for the exceptions.
change, validate the contract before dependent actions.
waitForUrl for URL contracts, page/element checks for stable text orstructure, and AI assertions for semantic visual state.
individual control; for rapid sequences, see Time-sensitive interactions.
style. Do not weaken a test to hide a broken app or service.
before / steps / after structure unless the testintent requires a change.
AI Action V3 takes a natural-language goal and determines the browser steps.
replay steps. Use for repeatable flows.
cache: false in v2 or use--disable-cache with AI_ACTION_DYNAMIC. Prefer this for chats, agentic workflows, exploratory tests, and other dynamic or nondeterministic behavior.
Treat “make sure you can do X” as goal-based testing unless the user specifies a route. Use granular steps when controls, intermediate assertions, deterministic replay, or a risky side effect needs individual control. Keep existing granular tests granular unless asked to change strategy.
For rapid sequences such as back-to-back sends or interrupting during startup, prefer one focused AI Action with explicit ordering and instructions not to wait between interactions. Separate targeting steps can outlast the window being tested. Keep prerequisite setup and stable final checks outside that action; pair it with video-backed run assertions for transient UI states.
AI Action does not guarantee timing. Inspect action timestamps and recorded evidence to verify the intended state was actually exercised. A successful final result, or an assertion that infers a transient state without observing it, is not proof of coverage. If the exact race window matters and cannot be reliably exercised, use deterministic tests or controlled test fixtures rather than claiming coverage from a passing end state.
Momentic caches resolved selectors, text, coordinates, and other metadata so most runs avoid repeated AI calls. AI assertions may also reuse past-result memory. Stale cache or memory can explain a fast wrong-element match or a repeated borderline verdict.
Cache is scoped by git metadata, including branch. Protected branches read cache but do not write it unless --save-cache is used or CI is set.
add DANGEROUS_FORCE_DYNAMIC or --disable-cache to locator steps; built-in cache validation handles changing content. Disabling the cache is not a substitute for clarifying a vague description or fixing a timing or race condition. Add it only when told to, or when a cached locator already caused a failure that neither fix resolves.
CacheId into the exact step being spliced.Smart waiting runs before targeting steps and defaults to five seconds. Within that window, do not add sleeps. For longer or semantic readiness, wait for a URL or use a page, element, or AI check that describes the required positive state. Checks retry until their timeout; increase that timeout instead of polling the UI with JavaScript. See the browser configuration for project-wide timing settings.
Each run has a test-scoped env that persists across steps and modules.
saveAs; in MCP step strings, use--env-key. Use setVariable(name, value) in JavaScript when saving several values.
env.NAME in JavaScript and module input expressions. Use{{ env.NAME }} in string fields, but not inside JavaScript source.
strings and use env.X for variables.
Use native click, type, hover, wait, and check steps for browser behavior. They retain Momentic's smart waiting, cache, traces, and retries. Do not use browser JavaScript to click or type, and do not poll the UI when a check can retry until its timeout.
Use Node JavaScript for test setup, data generation, APIs, databases, OTP/email/SMS, and Momentic helpers. Use Browser JavaScript to access window, document, client-side state, or inject page scripts when no native step expresses the operation. Keep one-off code short and follow nearby project conventions for reusable v2 scripts. See the JavaScript guide for runtime APIs and timeout details.
Playwright evaluate callbacks run in the browser and cannot close over Node values. Pass values as callback arguments or use customWindow._momentic.
V2 is human-editable. Tests use before, steps, and after; modules use steps. Each step has one command key, with options nested beneath it. Durations are milliseconds, and step IDs are not stored in YAML.
Make the smallest edit and match nearby syntax. If syntax is unclear, consult the generated v2 format reference at infra/scripts/v2-json-schema/build/v2-format-reference.md when working in the Momentic monorepo, or https://static.momentic.ai/v2-format-reference.md in an installed skill. Run npx momentic lint when schema or file-reference risk warrants it; momentic app and momentic run also lint automatically.
After a disk edit, reload the active MCP test if the tool is available; otherwise terminate and restart the session. momentic_test_get reads persisted state but does not refresh an active session unless its description says so.
Use the tool descriptions and the Step Authoring Guide returned by session start for current arguments and step syntax. Do not duplicate that reference material from this skill.
Do not call context tools as session-start boilerplate. Use them on demand and follow their descriptions, especially when they offer filters or return large project data.
momentic_session_start by itself. Its TestContent provides the active steps and runnable IDs.
momentic_get_artifacts only to discover project paths, tests, modules, orenvironments that are not already known. Avoid redundant refreshes.
momentic_test_get to inspect persisted state before starting, or to reada different test. Prefer active session and splice responses afterward.
momentic_get_environment_variables only when a step needs envdata that is not already known.
momentic_module_recommend -> momentic_module_get only when evaluatingreuse; recommendation invokes AI and is not required to start a session.
V2 YAML has no runnable step IDs. Use IDs returned by session start or later splice responses.
Only one browser operation may run per session. Do not issue concurrent preview, run, state, or splice operations against the same session.
session. This avoids authoring against the wrong page state.
or submission complete. Checkpoints make the browser state reusable and keep persistence calls reviewable.
momentic_preview_steps for adjacent steps. It supports the same preset,AI Action, and module steps as momentic_preview_step.
state only when the screenshot or locator result is insufficient; request it again once if the page may still be settling.
CacheId only to itsexact step; cacheless steps are normal. Read the splice response immediately for the active step refs.
stepRunnerId, poll it instead of starting another browser operation.
prerequisites, changed step, and next dependent contract. Reset first when accumulated state could mask a failure.
Batch obvious low-risk fields. Preview uncertain locators individually. For submit, purchase, delete, send, or other non-idempotent actions, preview setup, splice the checkpoint, then run the approved saved action at most once.
Do not reset between routine edits. For nested steps, use the parent chain returned by the active session or splice response.
For a logical flow of roughly four or more reusable steps, check momentic_module_recommend -> momentic_module_get -> reuse or inline. Reuse reduces duplicated setup and keeps tests focused on their unique contract.
Modules cannot contain modules. Ask before changing a shared module. Respect its declared parameters, defaults, and enum values; module inputs are JavaScript fragments, not {{ }} templates.
state capture once if the page is settling.
or increase the existing check's timeout. Checks already retry; do not poll with JavaScript.
browser.smartWaitingTimeoutMs. Do not use it to mask one slow assertion.
timeout, not a sleep.
resetSession: true.Retry or reset once after a browser, CDP, page-load, screenshot, or snapshot timeout. If it persists or affects several browser tools, stop changing the test. Do not hide resource pressure with waits, weaker assertions, cache flags, or repeated browser calls.
Signals include an unresponsive page, document-tree or page-execution timeout, empty HTML snapshot, browser or inspector crash, and data-momentic-id timeout. A page-load timeout alone may instead indicate the app or network.
Inspect CPU, memory, and top processes with available host tools. Check the browser, Momentic, app server, compiler/bundler, database, and unrelated workloads. For completed runs, correlate with attempts/<n>/assets/resource-usage.ndjson when present.
Report the error, step, URL, retry result, resource pressure, largest consumers, and supported cause. If evidence cannot distinguish the host, app, services, or Momentic, say so.
stable description based on visible text, role, and nearby context.
moving, or outside the viewport originate in Playwright's actionability enforcement, which Momentic surfaces; they are not failures in Momentic's locator AI. Fix the page state or prerequisite. Use force only when bypassing actionability is part of the intended behavior.
as some rich-text editors, browser.visualActions: true uses coordinate-based actions instead. It avoids those checks but also gives up their stability guarantees. See Visual actions.
meaning. Make assertions concrete about region, object, count, or state.
assertVisually when the condition is visual and fully inside thecurrent viewport, e.g. an event block sitting in the 11:30–12:00 row of a calendar grid, or a detail inside a video frame or canvas drawing. Scroll the target into view if necessary before calling it; otherwise use assert.
change-over-time comparisons, use video-backed run assertions.
env.X versus{{ env.X }} at the consumer.
After about three attempts at the same failure, stop and ask the user for direction.
~30 seconds. Free. No account. Every finding cites a rule and a line of evidence.