sota-testing — independently scanned and version-tracked by SaferSkills.
SaferSkills independently audited sota-testing (Agent Skill) and scored it 100/100 (green). The audit ran 55 deterministic rules across Security, Supply Chain, Maintenance, Transparency, and Community; it found 0 high-severity and 0 lower-severity findings. The full rule-by-rule trace and per-finding evidence are below. Free, methodology-open.
Findings & checks · 0 flagged
Every scanned point with the score it earned and what moved between them.
First recorded scan — no prior version to compare against.
The primary manifest — the file an agent reads to learn what this artifact does.
Expert-level, language-agnostic rules for producing and auditing production test suites. Per-language runner/tooling details (pytest, go test, cargo test, vitest/jest) live in the language skills (sota-python, sota-golang, sota-rust, sota-javascript-typescript) — this skill defines the strategy, design discipline, and quality bar those tools execute against. Every rule states the why; every rules file ends with an audit checklist of yes/no questions and grep-able smells.
Two consumers, one source of truth:
follow the rules as defaults, not suggestions. Deviate only with an explicit comment justifying the deviation.
checklists, classify by severity, report in the finding format below.
testing (see index). A service touching HTTP + DB + a message queue needs 01, 02, 03, 04.
rules/01): what counts as a unit here, wherethe integration boundary is, which 3–10 flows deserve e2e. Write that decision down (CONTRIBUTING.md or a test README) so the next contributor doesn't relitigate it.
TODO: assert something placeholders. A merged test with no assertion is worse than no test — it manufactures false confidence.
write the failing test first — it is the only proof the fix fixes anything.
I/O semantics you don't own (DBs, brokers, caches) — see rules/04.
sleep in a test that verifies a timeout), comment why inline.
Audit the suite, not just the tests: shape, doubles discipline, data management, CI health, and what is missing (untested risk) all count.
Severity conventions:
(always-green), mocks asserting mock behavior, disabled/skipped tests hiding known-broken production behavior, coverage gates gamed by meaningless tests.
shared mutable state between tests, order-dependent tests, real time/network/randomness without injection, flaky tests un-quarantined and retried-to-green, e2e suite owning logic the unit layer should own, mocking internals so refactors break hundreds of tests.
snapshot dumps nobody reviews, sleeps instead of waits, fixture data with irrelevant noise, missing negative-path tests on critical flows.
violations, missing parameterization of near-duplicate tests.
Finding format (one per line):
file:line | rule-id | severity | finding and concrete fixExample:
tests/orders_test.py:88 | 02-determinism | High | uses datetime.now(); inject a fixed clock so the test cannot fail at month boundaries
tests/api/user.spec.ts:12 | 03-mock-boundary | High | mocks internal UserValidator; test the real validator, mock only the HTTP gatewayEnd every audit with: findings table, top-3 risks, and a prioritized fix list (quick wins vs structural).
| File | Read this when... |
|---|---|
rules/01-strategy-and-shape.md | choosing pyramid/trophy/honeycomb, defining unit vs integration boundaries, deciding what NOT to test, risk-based prioritization, budgeting test cost |
rules/02-test-design-quality.md | writing or reviewing any test: behavior-over-implementation, AAA, naming, one logical assertion, determinism (clock/random/network), test smells catalog, snapshot discipline |
rules/03-doubles-and-test-data.md | deciding mock vs fake vs stub, fixing over-mocked suites, building test data (builders/factories vs fixtures), seeding test DBs, using production data |
rules/04-integration-contract-system.md | testing against real DBs/brokers (Testcontainers-style), contract testing between services (Pact, schema-based), API testing, migrations, message/queue tests, ephemeral environments |
rules/05-e2e-and-ui.md | building or pruning an e2e suite: critical-path selection, selector strategy, auto-waiting, page objects/screenplay, visual regression, when to delete e2e tests |
rules/06-property-fuzzing-mutation.md | going beyond examples: property-based testing (what properties to encode), fuzzing parsers, mutation testing ROI, approval testing for legacy code, chaos pointer |
rules/07-suite-health-and-ci.md | flaky-test policy and quarantine, coverage philosophy (ratchets not targets), speed budgets, parallelization correctness, CI sharding, failure triage |
test is deleted or inverted is a Critical finding. Verify the failure mode when writing (break the code, watch it go red — TDD gives this for free).
pure refactor (no behavior change) breaks the test, the test is wrong.
seed or inject RNGs, fake the network. Nondeterminism is how flakes are born.
test must pass alone, in any order, and in parallel with its siblings.
gateway/port), never internals, and don't mock types you don't own — wrap them, then fake the wrapper. Verify fakes against the real thing.
behavior (rejects_expired_card, not test_payment_2).
not in-memory lookalikes with different semantics. SQLite is not Postgres.
role/testid selectors and auto-waiting — never sleep(), never a dumping ground for cases a lower layer can cover.
never silently retried-to-green forever, never deleted without a root-cause label (ordering / async / time / infra / test bug / real bug).
read the uncovered lines, and never write a test whose only purpose is to move the number.
~30 seconds. Free. No account. Every finding cites a rule and a line of evidence.