verification-loop — independently scanned and version-tracked by SaferSkills.
SaferSkills independently audited verification-loop (Agent Skill) and scored it 100/100 (green). The audit ran 55 deterministic rules across Security, Supply Chain, Maintenance, Transparency, and Community; it found 0 high-severity and 0 lower-severity findings. The full rule-by-rule trace and per-finding evidence are below. Free, methodology-open.
Findings & checks · 0 flagged
Every scanned point with the score it earned and what moved between them.
First recorded scan — no prior version to compare against.
The primary manifest — the file an agent reads to learn what this artifact does.
Observe the running product. Stay blind to how it was built.
The verification loop is a standing autonomous reviewer that drives shipped work from "orc says done" to "verified green on prod". It observes the deployed product via a headless browser, assigns typed evidence to each acceptance criterion, judges cross-vendor (Codex primary, Claude fallback), and loops to convergence — filing correctives for hollow work, escalating taste/blockers, and never touching the build.
builder's reasoning. The judge receives only the typed evidence artifact — never the worker's explanation.
DOM/screenshot observation. A grep is auto-fail for a UI criterion. No criterion closes without observed, type-matched evidence.
~/.claude/ledger/verified/<spec_id>.yml — a separate namespace the builder's set/register commands never touch.
The loop converges only if orc is running and consuming correctives. If filed items sit unconsumed for N ticks, escalate to NEEDS-HUMAN.jsonl — do not file forever into the void.
Run with: node /path/to/verification-loop/tick.mjs --config <project> [--spec NNN-slug] [--dry-run] [--force]
PROGRESS.jsonl entry. No newsha → idle-cheap return, no browser spun.
to NEEDS-HUMAN.jsonl and halt. Never silent-continue.
acquire() storageState once per day (7-day TTL). Pullacceptance criteria from the spec file. pinSpecSha() to guard against silent scope reduction.
selectObservationLayer() routes to DOM (aria snapshot +innerText), VISION (screenshot + bounded binary question), or DOM_INTERACTION. Run callApi() for backend criteria. Both verify_periods. Run runIpt() when a gaming trigger fires.
judge(criterion, evidenceText, {runCodex, runClaude}). Token/reasoncontradiction → flag UNCLEAR → re-judge once → escalate if still unclear. Judge calls are sequential (never concurrent — subscription rate limit).
CONFIRMED → recordVerdict() + spec_ledger.py verify NNN CONFIRMED --judge codex --evidence <ref>HOLLOW / MISSING / REGRESSION → escalate corrective to NEEDS-HUMAN.jsonl(≤3 attempts; on exhaustion → BOUNCED, escalate)
DATA-GAP / NOT-RUN → ops noteSUSPECTED-GAMING / TASTE / blocker → escalate, never spinprobe.mjs catches 502/503 deploy windows, retries onceafter 30s, tags DEPLOY_IN_PROGRESS. Never cry P0 on a deploy window.
detectScopeReduction() → escalate anymissing-with-no-evidence criteria. appendProgress(). Reschedule cost-aware.
| Verdict | Meaning | Action |
|---|---|---|
CONFIRMED | Criterion observed working | Write to verified/ namespace |
HOLLOW | Exists in code, doesn't work | File corrective |
MISSING | Not implemented | File corrective |
REGRESSION | Was working, now broken | File corrective |
NOT-RUN | Operational step skipped | Ops note |
DATA-GAP | Code ok, source data absent | Ops note |
TASTE | Subjective judgement call | Escalate to human |
SUSPECTED-GAMING | IPT metamorphic relation failed | Escalate; R7 2nd-case rule before labelling systemic |
UNCLEAR | Token/reason contradiction in judge output | Re-judge once; escalate if still unclear |
BOUNCED | Trial budget (≤3) exhausted | Escalate to human |
runs/<date>/)| File | Purpose |
|---|---|
PROGRESS.jsonl | Append-only event log — sha, criteria checked, verdicts. Resume by reading. |
VERIFICATION-LEDGER.jsonl | Per-criterion verdict + evidence ref. Source of truth for "checked". |
NEEDS-HUMAN.jsonl | Loud escalation list — taste, blockers, gaming, exhausted budgets. |
SPEC-PINS.json | Criteria set pinned at handover. Scope-reduction guard. |
Verifier-owned verdict files: ~/.claude/ledger/verified/<spec_id>.yml
cd /path/to/do-it/verification-loop
set -a; source /path/to/your/.env; set +a
node tick.mjs --config <your-project> --spec <NNN-slug> --criterion "returns 200" --dry-runnode tick.mjs — uses probe.mjs / shoot.mjs / api.mjsfor observations. No MCP.
when you want to drive a browser in the current conversation. Do NOT wire the MCP into the autonomous cron tick.
An escalation unresolved for >2 ticks should trigger a human notification (append to NEEDS-HUMAN.jsonl with reason: unresolved_escalation), not another corrective attempt.
verification-loop/config/<your-project>.json — the only project-specific surface. Swap in another project's config to reuse the harness on a different app. See verification-loop/config/README.md for all config fields.
~30 seconds. Free. No account. Every finding cites a rule and a line of evidence.