compare-runs-fa3470 — independently scanned and version-tracked by SaferSkills.
SaferSkills independently audited compare-runs-fa3470 (Agent Skill) and scored it 100/100 (green). The audit ran 55 deterministic rules across Security, Supply Chain, Maintenance, Transparency, and Community; it found 0 high-severity and 0 lower-severity findings. The full rule-by-rule trace and per-finding evidence are below. Free, methodology-open.
Findings & checks · 0 flagged
Every scanned point with the score it earned and what moved between them.
First recorded scan — no prior version to compare against.
The primary manifest — the file an agent reads to learn what this artifact does.
<runIdA> <runIdB>.$VENV_PYTHON -c "
import json
from pathlib import Path
def load(run_id):
d = Path('.longmemeval-benchmarks/runs') / run_id
return {
'meta': json.load(open(d / 'run-meta.json')),
'card': json.load(open(d / 'scorecard.json')),
}
A = load('<RUN_A>')
B = load('<RUN_B>')
print(json.dumps({'A': A, 'B': B}, indent=2))
"If scorecard.json is missing for a run, invoke the report skill on it first.
Print markdown:
# Run Comparison
| | A ({runA}) | B ({runB}) | Delta |
|---|---|---|---|
| variant | ... | ... | |
| target model | ... | ... | |
| judge model | ... | ... | |
| n_evaluated | ... | ... | +/- |
| overall_accuracy | ... | ... | +/-.XXX |
| single-session-user | ... | ... | +/- |
| single-session-assistant | ... | ... | +/- |
| multi-session | ... | ... | +/- |
| temporal-reasoning | ... | ... | +/- |
| knowledge-update | ... | ... | +/- |
| preference | ... | ... | +/- |
| abstention | ... | ... | +/- |Also diff harness_config (model / plugins / skills / mcp_servers) and surface added/removed entries.
Print the top-3 question types with the largest positive and negative deltas.
~30 seconds. Free. No account. Every finding cites a rule and a line of evidence.