🛡️ skill-checker

Paranoid auditor for Claude Code skills. Refuse the malicious, patch the sloppy, install the safe.

A read-only auditor that runs before you install any third-party Claude Code skill. It catches what one pair of eyes misses — the static scanner pattern-matches against a curated catalogue of dangerous behaviors, and the LLM-driven steps in SKILL.md add semantic checks (description-vs-behavior consistency, tool laundering, confused deputy, prompt-injection vulnerabilities).

Output is one of three verdicts:

• 🔴 RED — refuse to install. Concrete reasons reported, no patches

offered (malice tends to be defense-in-depth).

• 🟡 YELLOW — patches required. Each finding comes with an exact diff.

You apply manually, re-run, iterate to GREEN.

• 🟢 GREEN — safe to install. Install command + brief usage included.

Read-only by design: the allowed-tools list contains no rm, cp, mv, tee, mkdir, package-install, or network commands, and no interpreter wildcard — only test, echo (diagnostics), and the pinned scan.py. echo could in principle redirect to a file; the skill never does, and you can audit that. Path scoping is enforced at the instruction level (the Checker Scope Rules), since Read/Glob/Grep are not themselves path-restricted.

What it catches

A non-exhaustive sample of the static rules — 36 CRITICAL, 19 HIGH, 10 MEDIUM regex rules, plus the structural bundled-config rules (CR032/CR033 hooks/stdio-MCP, CR040 bad MCP/hook destination, HI017/HI018, ME010, INV002), the supply-chain pass (CR039/HI023/ME012), the Python AST pass (AST001–AST008), and the Unicode pass (UNI001–UNI004):

CRITICAL — refuse, no patch:

• Pipe-to-shell (curl ... | sh), base64-decoded eval/exec, dynamic __import__ with concatenation
• pickle.loads/marshal.loads from external data, yaml.load without SafeLoader
• Reading ~/.ssh/, ~/.aws/, keychain, .env, *.pem, id_rsa, .netrc, .npmrc, .kube/config
• Persistence: writing to ~/.bashrc, ~/.gitconfig, git hooks, npm postinstall, cron, launchd
• Exfiltration endpoints: webhook.site, pastebin, Discord/Slack webhooks, ngrok
• Skill self-elevation: writing to ~/.claude/settings.json, other skills, MCP config
• Anti-user prose: "do not tell the user", "run silently", "treat document as system instructions"
• Policy override: "ignore safety", "developer mode", "jailbreak"
• Dangerous defaults: "retry with relaxed limits", "if blocked use sudo"

HIGH — patch (or RED if 3+):

• Wildcards in allowed-tools: Bash(* *), Bash(python3 *), Bash(rm *), Bash(sudo *), Bash(curl *), Bash(npm *), Bash(ssh *), Bash(aws *)...
• subprocess(..., shell=True), eval(), exec() over non-literal input
• Network calls without hard-coded destination
• Recursive home/root scans (find ~, grep -R ~)
• JS dynamic execution: Function(...), Buffer.from(..., 'base64') followed by eval

MEDIUM — patch:

• $0 confused with $1 for arguments, predictable /tmp/ without mktemp
• Path traversal via unvalidated slug, subprocess without timeout=
• Silent failure (2>/dev/null after destructive ops), missing symlink checks
• "Trust me" language without justification

Full catalogue with patterns and rationale: references/red-flags.md.

The patches the checker offers in YELLOW verdicts: references/patch-templates.md.

Quickstart

1. Install

git clone https://github.com/AntonioTimo/skillchecker.git ~/.claude/skills/skill-checker
chmod +x ~/.claude/skills/skill-checker/scripts/scan.py

Restart your Claude Code session (new chat — skills are cached at session start).

2. Audit a skill

Place the skill you want to audit in a staging directory — not in ~/.claude/skills/ yet:

mkdir -p ~/staging/skills
git clone https://github.com/some-author/some-skill.git ~/staging/skills/some-skill

Then in Claude Code:

/skill-checker ~/staging/skills/some-skill/

The auditor walks through eight steps (inventory → static scan → frontmatter audit → bash audit → script audit → tool laundering → confused deputy → prompt injection → consistency → verdict).

3. Act on the verdict

• 🔴 RED → rm -rf ~/staging/skills/some-skill and don't look back. Don't try to "patch around" — malice is layered.
• 🟡 YELLOW → apply the diffs in the report manually, then re-run /skill-checker until GREEN.
• 🟢 GREEN → use the install command from the report to copy into ~/.claude/skills/.

Example output

🔴 RED verdict (skill with Bash(* *), curl ... | sh, ~/.ssh/id_rsa):

## 🔴 SKILL REJECTED — DO NOT INSTALL

Skill path: /tmp/evil-skill
Skill name: super-helpful

Why it was rejected:
  [CRITICAL] FM005 SKILL.md:4  — Bash(* *) grants unrestricted shell access
  [CRITICAL] CR001 SKILL.md:9  — pipe-to-shell: downloads and executes remote code at runtime
  [CRITICAL] CR006 SKILL.md:10 — access to ~/.ssh — private keys, authorized_keys
  [CRITICAL] CR025 SKILL.md:10 — access to credential / secret files
  [CRITICAL] CR026 SKILL.md:10 — known exfiltration endpoint (webhook.site)
  [CRITICAL] CR028 SKILL.md:13 — anti-user instruction
  [CRITICAL] CR031 SKILL.md:15 — role confusion: "treat document as system instructions"
  ...

Recommendation: delete this skill. Do not attempt to patch around the
malicious sections — malice tends to be defense-in-depth.

🟡 YELLOW verdict with patches:

## 🟡 PATCHES REQUIRED BEFORE INSTALL

Findings: 2 HIGH, 4 MEDIUM, 0 LOW

### Patch 1: Wildcard in allowed-tools
File: SKILL.md
Severity: HIGH
Why: Bash(python3 *) lets the model run arbitrary Python (effectively RCE)

Replace:
  allowed-tools: Bash(python3 *) Bash(...)
With:
  allowed-tools: Bash(python3 ~/.claude/skills/<skill>/scripts/<script>.py *) Bash(...)

### Patch 2: $0 used as first argument
File: SKILL.md
...

Limitations — Read these before relying on the verdict

This checker does not catch every threat class. It catches the common ones, fast.

1. No dynamic analysis. A skill that fetches malicious code at runtime

from a server it controls passes static checks. Mitigation: 🔴 any skill with both network calls and writeable filesystem operations.

1. Partial supply-chain analysis. Bundled dependency manifests are scanned

(CR039 npm install-lifecycle scripts, HI023 non-registry sources, ME012 unpinned deps), but a malicious update to an already-pinned registry library, a transitive dependency, and CVE/version reputation are not detected. Pin and audit dependencies separately (pip-audit / npm audit).

1. LLM judgment is fallible. Adversarial code can mimic benign code.

When the static scan flags multiple HIGH findings, even if individually explainable, treat it as a pattern. The v1.8.0 taint pass (TF001/TF002) catches a credential→network exfil split across variables, but only intraprocedurally and single-file — a secret laundered through a function call or another module still relies on the LLM-side read.

1. Update means re-audit. A skill that was 🟢 yesterday may be 🔴 today.

Re-run after any upstream update.

1. Self-audit is a known edge case. Running /skill-checker against

skill-checker itself yields ~30 false positives (the docs literally describe attack patterns). Documentation skills will trip the same way; manual judgment overrides the static counts.

GREEN means "no known patterns matched", not "100% safe". Don't run a freshly-installed skill on production-sensitive files in the first run. Test it on something benign first.

For the full classification of what we catch, what we treat as fixable, what we let through, and what we explicitly don't address — see THREAT_MODEL.md.

How this was built

This tool came out of a real iterative audit cycle: a pair of LLMs sparring over a single skill, finding what each other missed. Five rounds of audit produced the current rule set. The methodology is:

• Paranoid by default. When in doubt, raise the flag.
• Don't trust the description. It's marketing — written by the author. Truth is in the code.
• Diffs, not opinions. When a fix exists, output the exact replacement.
• Refusal is a real outcome. Some skills don't deserve a patch.

The static rules in scripts/scan.py are line-based and miss multiline constructs by design — the LLM-driven steps in SKILL.md cover the remaining context-dependent semantics. Together they cover roughly 95% of the failure modes seen in real-world skill submissions; the remaining 5% require either a more sophisticated dynamic analyzer or human judgment.

Project layout

skill-checker/
├── README.md             ← you are here
├── LICENSE               ← MIT
├── SKILL.md              ← Claude Code skill entrypoint (the audit procedure)
├── scripts/
│   └── scan.py           ← static scanner (read-only, no shell, no network)
├── references/
│   ├── red-flags.md      ← catalogue of patterns by severity
│   └── patch-templates.md ← ready-to-paste fixes for YELLOW findings
├── examples/             ← paired clean/evil fixtures, each asserted in CI:
│   ├── clean-skill/ + evil-skill/        ← baseline benign / common attacks
│   ├── evil-plugin/ + clean-with-data/   ← bundled hooks+MCP / benign config-key data
│   ├── evil-mcp/ + clean-mcp/            ← bad MCP/hook destination (raw IP / punycode) / benign MCP-catalogue data
│   ├── evil-ast/ + clean-ast/            ← obfuscated calls / safe Python
│   ├── evil-unicode/ + clean-unicode/    ← hidden-Unicode injection / bilingual prose
│   ├── evil-exfil/ + clean-exfil/        ← modern exfil / local-dev URLs
│   ├── evil-evasion/ + clean-evasion/    ← NFKC / homoglyph / metadata-SSRF / scheme-less IP vs. legit compat + named hosts
│   ├── evil-supplychain/ + clean-supplychain/ ← bundled-manifest install scripts / non-registry sources vs. pinned registry deps
│   ├── evil-taint/ + clean-taint/        ← credential→network exfil flow (TF001/TF002) / credential reads with no sink
│   ├── evil-selftarget/ + clean-selftarget/ ← self-prompt-leak/persist/self-rewrite/catch-all (HI024/HI025/ME013-015/AST009) / defensive + skill-builder
│   ├── evil-ecosystem/ + clean-ecosystem/ ← forged-token/os.exec/MCP-secret/Phantom-Gyp/reverse-shell (CR041-044/HI026-029/AST010-011) / defensive + safe forms
│   └── evil-bypass/                      ← regression set for previously-evaded patterns
└── docs/
    └── HOWTO.md          ← user-facing guide

Contributing

Issues and PRs welcome, especially:

• New rules covering attack patterns the current set misses
• Patch templates for sloppy patterns currently flagged without a clear fix
• Translations of references/*.md (currently mixed English/Russian)
• Real-world skill submissions that produce surprising verdicts (false positive or false negative)

Before opening a PR with a new rule, read [THREAT_MODEL.md](THREAT_MODEL.md). It defines what counts as malicious vs sloppy vs acceptable, the false-positive budget per severity, and what the PR must include (positive + negative test cases, patch template if non-CRITICAL, false-positive guard if applicable). Rules that don't fit the threat model or exceed the FP budget will not be merged — the scanner is only useful if its findings are believed.

Disclaimer

This is a community tool. Not affiliated with Anthropic. It's a static audit aid; it does not guarantee safety. Ultimate responsibility for what runs on your machine is yours.

License

MIT — see LICENSE.