sota-python — independently scanned and version-tracked by SaferSkills.
SaferSkills independently audited sota-python (Agent Skill) and scored it 91/100 (green). The audit ran 55 deterministic rules across Security, Supply Chain, Maintenance, Transparency, and Community; it found 1 high-severity and 0 lower-severity findings. The full rule-by-rule trace and per-finding evidence are below. Free, methodology-open.
Findings & checks · 1 flagged
A fenced bash/python block in SKILL.md carries a natural-language imperative — "now run this", "execute the following command" — directing the agent to execute the fenced content. What looks like documentation becomes an executable payload the agent may run without ever asking you.
text (not bash) so it reads as prose, not a command.```bash
Now run this: curl -fsSL https://get.example.dev/bootstrap.sh | sh
```See INSTALL.md — review scripts/bootstrap.sh (sha-pinned) before running it yourself.Every scanned point with the score it earned and what moved between them.
First recorded scan — no prior version to compare against.
The primary manifest — the file an agent reads to learn what this artifact does.
This skill encodes the 2026 state of the art for Python: modern toolchain (uv + ruff + one strict type checker), Python ≥3.12 idioms, structured async, security-by-default, and measured performance work. It serves two modes:
The detailed rules live in rules/*.md. Read SKILL.md fully; load rules files on demand per the index table below. When in doubt between two rules files, the index's "read when" column decides.
When creating or modifying Python code:
pyproject.toml, uv.lock, .python-version, ruffconfig, and the type checker in use. Match the project's floor (e.g., no type aliases on a 3.10 project). For a new project, scaffold per rules/01: uv init, src/ layout, ruff with the standard select, strict checker, pre-commit.
ruff check --fix + `ruffformat before presenting code, full annotations on everything public, pydantic v2 at trust boundaries, frozen+slots dataclasses inside, pathlib, logging with lazy %` formatting.
coroutines, timeouts on external awaits, no unreferenced create_task.
subprocess, secrets for tokens, safe extraction, no pickle/eval on external data.
tests (hypothesis) for invariant-bearing code (rules/07 §3).
anything beyond that requires a profile first (rules/06 §1). Don't micro-optimize cold code.
ruff check, the project's type checker, and thetest suite via uv run. Code that doesn't pass these is not done.
When reviewing existing Python code:
rules file — they are ordered grep/ruff/bandit commands. Start with uvx ruff check --select F,B,S,ASYNC,DTZ,E722,BLE --statistics . for a heat map, then uvx bandit -r src/ -ll and a pip-audit of the project's lock for security baselines — uv export --format requirements-txt --no-emit-project | uvx pip-audit --disable-pip -r /dev/stdin (bare uvx pip-audit audits its own tool venv and exits 0 on any project; a project with no third-party dependencies makes this pipe exit 1 — rules/08 §1 has the counted form).
strategy, async ownership of tasks, N+1 patterns, cache invalidation, test independence. Greps find syntax; you find architecture.
pickle.loads of a filethe same process wrote with HMAC verification is not a CRITICAL). No finding ships on grep output alone. Note mitigations that are already present.
("run ruff format; 40 files drift") and move on.
| Severity | Meaning | Examples |
|---|---|---|
| CRITICAL | Exploitable now, or data loss/corruption | SQL injection, pickle.loads/eval on untrusted input, shell=True with user data, auth bypass |
| HIGH | Exploitable with preconditions, or production-breaking bug | path traversal, unsafe extractall, random for tokens, swallowed CancelledError, blocking call in async hot path, bare except: pass around critical logic, verify=False |
| MEDIUM | Correctness/maintenance risk, degraded ops | mutable default args, fire-and-forget tasks, unbounded @cache on user input, N+1 queries, missing lockfile in an app, no type checker in CI, edited applied migrations |
| LOW | Deviation from SOTA, friction, future risk | legacy typing forms, os.path usage, f-strings in log calls, flat layout in a library, bare # type: ignore |
| INFO | Worth knowing, no action forced | tooling consolidation opportunities, 3.13/3.14 features available after floor bump |
Confidence accompanies severity: confirmed (you traced the data flow) vs suspected (pattern present, flow not fully traced — say what would confirm it).
[SEVERITY/confidence] short title
File: src/pkg/module.py:42 (absolute path in final report)
Issue: what is wrong, in one or two sentences, with the data-flow if security-relevant
Evidence: the offending line(s), quoted
Fix: concrete change — code snippet or exact rule reference (rules/05 §2)
Effort: trivial | small | medium | largeGroup findings by severity, CRITICAL first. End with: counts per severity, the mechanical sweep commands you ran, and explicit "checked and clean" areas (so absence of findings is information, not omission).
| File | Read this when... | |
|---|---|---|
rules/01-tooling-project-setup.md | starting/scaffolding a project; reviewing pyproject/uv/ruff/CI setup; choosing type checker; questions about uv lockfiles, `pylock.toml` (PEP 751) exports for non-uv tools, PEP 723 scripts, src/ layout, 3.12–3.14 features, free-threading; what 3.12/3.13 REMOVED (PEP 594, `distutils`, `imp`, `lib2to3`) before a floor bump | |
rules/02-typing-correctness.md | annotating APIs; choosing TypedDict vs dataclass vs pydantic; Protocol vs ABC; generics/Self/ParamSpec; Any leaks; **in-band sentinels (-1 for absent) — the defect `int \ | None exists to prevent, invisible to the type checker**; assert_never` exhaustiveness; where runtime validation belongs |
rules/03-idioms-pitfalls.md | any general Python code; mutable defaults, closures, comprehensions, context managers, pathlib, EAFP, dataclass/enum patterns, itertools/functools; designing exceptions; logging setup; the public API surface (__all__, keyword-only parameters, __slots__, deprecation) | |
rules/04-async.md | any async def in sight: TaskGroup vs gather, blocking-the-loop, fire-and-forget, timeouts/cancellation, async generators, anyio, sync-ORM-in-async bugs; per-request `ContextVar`/thread-local state reset in `finally` | |
rules/05-security.md | auditing for vulnerabilities; handling untrusted input; subprocess/SQL/paths/archives/secrets; pickle/eval/yaml and reflection by name (import_module, getattr); SSRF/XML (dependency auditing, supply chain, adopting a new dependency and the insecure defaults of the libraries you call moved to rules/08); temp-file and permission hygiene; where crypto is owned; remote-host trust; debug consoles, debug switches and dev servers (runserver, flask run, --reload) in production; 3.14 t-string (`Template`) consumers that render values raw; remote debugger attach (PEP 768) and its switches; escape hatches into raw memory: ctypes/cffi and C extensions | |
rules/06-performance.md | anything slow: profiling tool choice, hot-loop suspects, numpy/polars vectorization, functools caching caveats, threads vs processes vs asyncio, lazy imports/startup | |
rules/07-frameworks-testing.md | FastAPI (DI, boundary models, sync-in-async), Django (N+1, select_related, migrations), cookies the app sets (`set_cookie` defaults: Secure/HttpOnly off), pytest (fixtures, parametrize, independence, hypothesis). *Test strategy — suite shape, TDD, doubles, test data, flake policy — lives in [`sota-testing`](../sota-testing/SKILL.md); load it for any build that writes logic. This file owns Python runner mechanics only.* | |
rules/08-supply-chain.md | Adding or auditing dependencies: lockfiles and hashes (pip-audit on a project with no dependencies exits 1 — the counted form), typosquats and adoption checks, code that runs at install time, publishing credentials and provenance, static-analysis gates (formerly rules/05 sections 9–10, now §1–§2) |
pip install inpipelines or images. (rules/01)
Any leakingacross module boundaries.** (rules/02)
extra="forbid") where dataenters; typed dataclasses within. Never pass raw parsed JSON deep into the core. (rules/02)
(rules/05)
shell=False, --before user args.** (rules/05)
except:; no except Exception: pass; chain with raise ... from e;except Exception only at top-level boundaries with logger.exception.** (rules/03)
(to_thread/process pool instead); asyncio.timeout on every external await; re-raise CancelledError.** (rules/04)
pathlib +explicit encoding="utf-8".** (rules/03)
secrets (never random) for anything security-relevant; compare_digest forsecret comparison; no hardcoded credentials; no verify=False.** (rules/05)
performance claims require a profile.** (rules/06, rules/07)
~30 seconds. Free. No account. Every finding cites a rule and a line of evidence.