skill-tester — independently scanned and version-tracked by SaferSkills.
SaferSkills independently audited skill-tester (Agent Skill) and scored it 100/100 (green). The audit ran 55 deterministic rules across Security, Supply Chain, Maintenance, Transparency, and Community; it found 0 high-severity and 0 lower-severity findings. The full rule-by-rule trace and per-finding evidence are below. Free, methodology-open.
Findings & checks · 0 flagged
Every scanned point with the score it earned and what moved between them.
First recorded scan — no prior version to compare against.
The primary manifest — the file an agent reads to learn what this artifact does.
TDD for documentation: watch the agent fail without the skill, write the skill, watch the agent pass.
Dispatch a subagent with this prompt, substituting <skill-name> and <scenario>:
You are working on <scenario>. Do NOT load any skills. <pressure>
What do you do?Motivational pressure — the agent knows the rule and is tempted to skip it:
Epistemic pressure — the agent is not tempted, it is misled. Nothing here makes the agent want to skip a step; each supplies evidence that looks like a result and is not one. A rule against skipping a check never fires, because the agent believes the check already passed:
that goes green when work starts, an exit 0 from a tool whose engine failed to load, an empty result set from a query that never ran, a progress line reporting scope as though it were an outcome.
success. "My differential test found 0 differences" is the hardest signal for an agent to doubt, and the one most worth doubting: it proves the cases written down passed, never that the right cases were written.
The cost of the contradicting fact is now an admission, not just a correction.
one field in a sixty-line dump the agent has skimmed three times already.
Record exact rationalizations the agent uses to skip the rule, and for an epistemic scenario, the exact inference it treats as proof.
A RED that passes is a broken test, not a passing skill. The baseline exists to reproduce the failure; a scenario the unaided agent handles correctly has measured nothing, and treating it as evidence the skill works is the same error as trusting a scanner that reports clean because its rules never loaded. When RED passes, the scenario is too easy — harden it or discard it. Never write the counter-text against a scenario that never failed.
Epistemic scenarios go easy in specific ways, each the inverse of the pressure it is meant to apply:
trap. Paste the whole output it really appeared in, at the length it really had, with the field unremarked.
an invented error string — an agent that notices the fake answers the fake rather than the scenario, and the run is wasted. Capture real output from a real run.
assertion need history: give the agent the earlier turns where it built the check, or told the user the thing was green, before showing it the artefact.
doubt the scenario is supposed to test for. Ask what the agent would report.
Scenario provenance beats scenario invention. The failures worth encoding are the ones this repository has already had — a wrong turn taken once is a wrong turn the text did not prevent, and it comes with the real artefact attached. The mechanism behind that preference is authorship: a scenario composed by the agent that wrote the fix telegraphs, because it is built out of the same understanding the fix came from. Build it from the transcript of the failure instead.
Write down what GREEN requires before dispatching the GREEN run, in a form that can be checked against a transcript by someone who did not write it. State what counts as a fail, not only what counts as a pass — "names the ambiguity among several other caveats" is a fail where "refuses to count the signal as a gate" is the bar.
Grading after the run is how a partial pass becomes a pass. The agent reads its own text back, finds the idea present somewhere in the reply, and marks it blocked. A criterion fixed in advance has no room for that.
Also decide in advance what an unrelated correct catch is worth: nothing. A baseline that stumbles into caution on some other axis narrows the RED/GREEN gap without the skill contributing anything, and the GREEN run then looks better for a reason that is not the text.
Motivational scenarios reproduce readily — an agent told it is on its tenth task under deadline will rationalize, and the rationalizations are recordable. The epistemic ones above have so far not. Four scenarios drawn from four real failures in this repository were each handled correctly by an unaided subagent, which then diagnosed the scenario's own weaknesses unprompted.
The likely reason is positional, not intellectual. The agent that made those mistakes had built the failing check itself, had already told the user the thing was green, and met the contradicting field on turn forty inside output it had skimmed three times. A fresh subagent handed a description of that situation is not in it: the artefact arrives flagged as the subject of a question, which is the one condition under which it is easy.
So a passing RED on an epistemic scenario is weak evidence either way. It does not show the guidance is unnecessary, and it does not license writing the counter-text anyway. What it does show is that the scenario failed to put the agent where the failure happens. Record the pass, keep the counter-text tied to the real incident that motivated it, and say in the write-up that the method did not reproduce it — an unvalidated counter honestly labelled beats a validated one that was graded into existence.
Write the skill or command body (skills/<skill-name>/SKILL.md, or skills/nitpicker/commands/<command>.md for a nitpicker command). Address each rationalization from RED explicitly. Then dispatch the same subagent again, this time with the skill loaded. Confirm each RED rationalization is blocked. If a new loophole emerges, add an explicit counter to the skill and re-run.
Refactor the skill body for clarity and precision. Then dispatch the same scenario again (skill still loaded). Confirm all GREEN scenarios still pass and no new loopholes have appeared. If they have, add counters and re-run.
hardened or discarded, never written up as a pass
uv run scripts/validate-skill.py skills/<skill-name>/SKILL.md (for a nitpicker command: skills/nitpicker/SKILL.md — it validates the command files too)~30 seconds. Free. No account. Every finding cites a rule and a line of evidence.