arxiv-preflight — independently scanned and version-tracked by SaferSkills.
SaferSkills independently audited arxiv-preflight (Agent Skill) and scored it 100/100 (green). The audit ran 55 deterministic rules across Security, Supply Chain, Maintenance, Transparency, and Community; it found 0 high-severity and 0 lower-severity findings. The full rule-by-rule trace and per-finding evidence are below. Free, methodology-open.
Findings & checks · 0 flagged
Every scanned point with the score it earned and what moved between them.
First recorded scan — no prior version to compare against.
The primary manifest — the file an agent reads to learn what this artifact does.
Source: https://github.com/aipoch/medical-research-skills
Submission-readiness review for a manuscript before arXiv upload. The goal is not to judge whether a paper was AI-written from style. The goal is to find concrete, locatable, reviewable evidence of unchecked LLM output, hallucinated references, placeholder data, unsupported claims, and arXiv-policy risks — and to hand the author a fix list.
Anchor on the actual arXiv stance: authors may use generative AI tools but are fully responsible for the content and must disclose significant use per field norms; AI tools cannot be listed as authors. See references/arxiv_policy_notes.md for the exact language.
This skill is intentionally platform-neutral. It should work for any AI coding/review agent that can read files and run local Python scripts; it does not require OpenAI-specific UI metadata.
Requires Python 3.9+.
scripts/extract_manuscript_text.py on the input. It merges \input/\include, strips LaTeX commands while preserving section/figure/table/citation keys, and falls back to PDF text extraction. Output: manuscript.json with sections, figures, tables, cite_keys, numbers, and warnings. Treat unresolved or skipped includes as review risks.scripts/scan_ai_artifacts.py against the extracted text. It flags LLM meta-comments, prompt residue, placeholder content, TODO/TBD/XX%, and AI-listed-as-author. Patterns live in references/ai_artifact_patterns.md — extend the list when the user reports new failure modes. Any hit is a BLOCKER unless the user explicitly justifies it.scripts/verify_references.py on .bib (or extracted reference list). It performs (a) structural BibTeX checks, (b) external metadata lookup against Crossref / arXiv / OpenAlex / Semantic Scholar (in that order), (c) cite-key vs. \cite reconciliation. Grading and API strategy: references/reference_verification.md. If network is unavailable, mark this section `INCOMPLETE` rather than skipping silently.manuscript.json to review extracted numbers, labels, and references. This version exposes the raw structures for agent-assisted consistency review; it does not fully automate semantic claim verification. Surface candidate inconsistencies for the author to confirm, never rewrite their claims.scripts/scan_ai_artifacts.py includes a limited disclosure heuristic for substantive LLM use. For nuanced cases, compare detected AI-assistance signals against the manuscript's Acknowledgments / Methods / Ethics sections yourself. Recommend disclosure only when significant use is evident; do not push boilerplate disclosure on every paper. Calibration: references/arxiv_policy_notes.md.scripts/generate_preflight_report.py to merge all JSON outputs into a single Markdown report. Format and decision rules: references/report_template.md.BLOCKER — evidence that should stop submission: LLM meta-comment in body text, AI listed as author, reference that cannot be matched in any external database, unfilled placeholder in a results table.HIGH — likely integrity or policy issue: DOI/title mismatch, cite-key pointing at wrong paper, a strong claim with no supporting number in the manuscript.MEDIUM — inconsistency, missing metadata, unclear disclosure, broken \ref.LOW — polish, formatting, capitalization.BLOCKER → HOLD.HIGH/MEDIUM → PASS_WITH_FIXES.LOW or clean → PASS.Never upgrade a finding's severity to be safe and never silently soften it. If you are unsure, mark it MEDIUM and explain.
.tex/.bib; for PDF input, use the PDF file plus extracted-text line unless page-aware extraction is available. Include a verbatim short snippet (≤ 25 words). A finding without a location is a suggestion, not a finding — move it to "Recommended Cleanup".MEDIUM. Title + authors + year all unmatched across all four databases is BLOCKER.INCOMPLETE; the overall decision cannot be PASS.BLOCKER is more costly than a missed LOW.references/arxiv_policy_notes.md — when calibrating disclosure recommendations or moderation findings.references/ai_artifact_patterns.md — when extending or interpreting the artifact scanner's hits.references/reference_verification.md — when interpreting citation lookup output, deciding API order, or handling rate-limit / network failures.references/report_template.md — when writing or regenerating the final report.This skill currently ships: text extraction, AI-artifact scan, BibTeX + DOI/arXiv-ID reference verification, extracted label/citation/number structures, and Markdown report generation. Out of scope for v1: full semantic peer review, full claim/result verification, automated moderation judgement, AI-writing probability scoring, automatic rewriting, automated arXiv upload, large-scale plagiarism detection. If the user asks for these, say so explicitly rather than approximating.
~30 seconds. Free. No account. Every finding cites a rule and a line of evidence.