icml-artifact-evaluation — independently scanned and version-tracked by SaferSkills.
SaferSkills independently audited icml-artifact-evaluation (Agent Skill) and scored it 100/100 (green). The audit ran 55 deterministic rules across Security, Supply Chain, Maintenance, Transparency, and Community; it found 0 high-severity and 0 lower-severity findings. The full rule-by-rule trace and per-finding evidence are below. Free, methodology-open.
Findings & checks · 0 flagged
Every scanned point with the score it earned and what moved between them.
First recorded scan — no prior version to compare against.
The primary manifest — the file an agent reads to learn what this artifact does.
ICML does not let artifacts sit outside the paper's scientific argument. Reproducibility and code availability are explicitly considered in decision-making, and accepted submissions may publish the original supplementary material on OpenReview.
notebook, or supplementary manuscript.
cards, licenses, and personal paths.
deadline.
whether to consult appendices or supplementary material.
mapping.
not contain private, illegal, or unreleasable content.
to a public repository or archive and be linked in the paper/OpenReview code URL field.
ICML double-blind review means a single deanonymizing artifact can trigger a desk reject, so audit the package the way an adversarial reviewer would.
| Leak vector | Where it hides | Mitigation |
|---|---|---|
| Repo ownership | Anonymous-repo account name, commit author | Use a fresh anonymized host, strip git history |
| File metadata | PDF author field, notebook kernel, model card | Clear metadata, rename author paths |
| Hard-coded paths | Cluster usernames in scripts and logs | Replace with placeholders before zipping |
| External links | Personal site, non-anonymous URL, shortener | Remove or route through an anonymous mirror |
A paper shipping an adaptive optimizer includes training scripts, a pretrained checkpoint, and a proof-checking notebook. The review package gives minimal commands, expected runtime, and the hardware assumption so a reviewer can map a command to a benchmark number, while the checkpoint filename and the notebook kernel are scrubbed of the lab name. Because accepted ICML supplements can become public, the team confirms the checkpoint is releasable and the license is stated before the deadline, then plans the public repository and OpenReview code URL for camera-ready.
[Artifact role] code / data / model / benchmark / proof / none
[Review package] sufficient / incomplete / unsafe
[Anonymity risks] <metadata, repo, filenames, links>
[Decision relevance] <why reviewers need it>
[Public release plan] <repo/archive/license/code URL>~30 seconds. Free. No account. Every finding cites a rule and a line of evidence.