ijcai-reproducibility — independently scanned and version-tracked by SaferSkills.
SaferSkills independently audited ijcai-reproducibility (Agent Skill) and scored it 100/100 (green). The audit ran 55 deterministic rules across Security, Supply Chain, Maintenance, Transparency, and Community; it found 0 high-severity and 0 lower-severity findings. The full rule-by-rule trace and per-finding evidence are below. Free, methodology-open.
Findings & checks · 0 flagged
Every scanned point with the score it earned and what moved between them.
First recorded scan — no prior version to compare against.
The primary manifest — the file an agent reads to learn what this artifact does.
Use this before submission and again before camera-ready. Reopen the current reproducibility page; IJCAI's rubric and checklist can change.
convincing when resources can be shared safely.
proofs for novel claims.
include or promise release of new datasets only when legally possible.
criteria, seeds or repeat strategy, software versions, compute infrastructure, and runtime.
not depend on supplementary material for central claims.
IJCAI's rating moves from irreproducible to credible to convincing. Because the PC spans symbolic, search, planning, KR, multi-agent, and learning work, the evidence that earns "convincing" differs by contribution. Map each result to the right column before drafting.
| Contribution | Minimum for "credible" | What lifts it to "convincing" |
|---|---|---|
| New algorithm (search/planning/CSP) | Pseudocode plus complexity claim in body | Runnable code, instance generator, seeds, version pins |
| Theoretical result | Assumptions and proof sketch in body | Full appendix proofs, citations to formal tools |
| Multi-agent / game-theoretic | Protocol, agent counts, payoffs | Released simulator, opponent policies, seeds |
| Dataset contribution | Source, licensing, collection described | Public release or controlled-access path, datasheet |
| Learning experiment | Hyperparameters and splits in body | Code, environment file, compute, repeat strategy |
A coordination-protocol paper claims faster convergence but reports no seeds and ships no simulator. To reach "convincing": state the agent count sweep and payoff matrix in the body; appendix the convergence proof sketch; release an anonymized simulator with fixed seeds and the opponent policies; pin library versions. Without the simulator the result stays "credible" at best, which an IJCAI reviewer may flag as a significance discount.
the Technical Appendix; do not bury the theorem statement.
result tables, since IJCAI search/planning reviewers re-run when skeptical.
reproduction; never imply a release you cannot deliver by camera-ready.
because reviewers are not required to open the supplement.
[Result inventory] <claim -> evidence location>
[Rubric target] convincing / credible / currently weak
[Missing details] <algorithm/theory/data/compute/hyperparameters/seeds>
[Paper fixes] <must be in main PDF>
[Supplement fixes] <optional or supporting evidence>~30 seconds. Free. No account. Every finding cites a rule and a line of evidence.