submit-smoke — independently scanned and version-tracked by SaferSkills.
SaferSkills independently audited submit-smoke (Agent Skill) and scored it 96/100 (green). The audit ran 55 deterministic rules across Security, Supply Chain, Maintenance, Transparency, and Community; it found 0 high-severity and 1 lower-severity findings. The full rule-by-rule trace and per-finding evidence are below. Free, methodology-open.
Findings & checks · 1 flagged
The text {match} tells the agent to skip the normal "ask the user first" gate. Used adversarially it removes the human-in-the-loop check before destructive or sensitive actions, turning a normally-gated agent into a fire-and-forget executor.
Every scanned point with the score it earned and what moved between them.
First recorded scan — no prior version to compare against.
The primary manifest — the file an agent reads to learn what this artifact does.
Run .skills/submit-smoke/scripts/submit_smoke.sh from the project root. It is the fast gate that answers ONE question before a real benchmark: does Enter actually submit?
The driver used to write ${prompt}\r in a single chunk; Claude Code's bracketed-paste path coalesced it and the \r landed as a newline INSIDE the input box, so ~6 of 8 turns silently never ran. The box sat at "turn 0%" and the settle-on-silence heuristic could not tell "claude finished" from "claude never started." The fix (benchmark/lib/turn_submit.js) types the text, pauses, sends Enter as its OWN keystroke, and CONFIRMS each turn against the proxy's --turn-log before advancing — escalating \r -> \n -> \r\n only when confirmation never arrives. This skill verifies that fix end to end on real claude, cheaply, so a swallowed Enter can never silently under-drive a headline run again.
# default: 3 turns, passthrough, Haiku, port 8799
.skills/submit-smoke/scripts/submit_smoke.sh
# bump the confirmation window for a slower model (Opus streams later):
.skills/submit-smoke/scripts/submit_smoke.sh --model claude-opus-4-7 --confirm-sec 20 --turns 3The script prints a verdict block and exits non-zero on FAIL:
confirmed when a new non-ping turn-log record OR a sustained PTY output burst appears within --confirm-sec).
/v1/messages records the proxyactually wrote. Because the arm is passthrough (no keep-alive pings), this is an exact count of turns that truly hit Anthropic.
"\r" means CR-as-its-own-keystroke submits once separated from the text (the happy path). Any "\n" / "\r\n" winning means CR alone still did not submit and the fallback carried it — a real finding to note before the big run.
PASS = confirmed N/M with N==M, AND real records >= turns, AND the driver exited 0. Anything else is FAIL (it points you at drive.log and proxy.log in the run dir).
Spends real Anthropic tokens for --turns turns via the real claude binary — keep it tiny (default 3) and on Haiku. Runs on port 8799 by default so it never collides with a dev proxy on 8787. Each run writes a self-contained runs/submit-smoke-<ts>/ (turn-log, proxy log, drive log, state) at 0600.
benchmark/bin/drive_pty.js or benchmark/lib/turn_submit.js.claude CLI upgrade (the TUI's paste / keyboard handling can move)..skills/ab session — a green submit smoke is theprecondition for trusting an A/B block's turn counts.
~30 seconds. Free. No account. Every finding cites a rule and a line of evidence.