writing-tests — independently scanned and version-tracked by SaferSkills.
SaferSkills independently audited writing-tests (Agent Skill) and scored it 100/100 (green). The audit ran 55 deterministic rules across Security, Supply Chain, Maintenance, Transparency, and Community; it found 0 high-severity and 0 lower-severity findings. The full rule-by-rule trace and per-finding evidence are below. Free, methodology-open.
Findings & checks · 0 flagged
Every scanned point with the score it earned and what moved between them.
First recorded scan — no prior version to compare against.
The primary manifest — the file an agent reads to learn what this artifact does.
The rationale and the same rules in human-facing form live in the handbook: Backend coding conventions › Testing (docs/published/handbook/engineering/conventions/backend-coding.md). This skill is the operational gate — run it before writing tests. It carries the decision procedure plus a catalog of the bug shapes we actually ship.
Before writing any test, answer in one sentence:
What realistic regression does this test catch that no existing test already catches?
If you can't answer it concretely — name the bug, the code path, the input that would break — do not write the test. "Increases coverage", "good practice", and "the function exists" are not answers.
A good answer sounds like: _"if someone makes parse_filters drop the team_id clause, this fails"_ or _"empty-cohort input used to 500; this locks in the 400."_ That is a test worth keeping.
Aim each test at a failure mode we actually hit, not a hypothetical. The bugs PostHog ships and reverts cluster into a handful of shapes — cataloged with the test that catches each, and the failure modes no unit test should, in references/mistakes-we-make.md. If your test doesn't map to one of them, be skeptical it's worth keeping.
Most low-value tests fall into one of these. Recognize and skip them.
Don't test getters, setters, constants, dataclass field assignment, that Django saved a row, that DRF serialized a field, or that a library does what its docs say. You're testing someone else's code, not yours.
A test that just mirrors the implementation — asserting which private methods were called, in what order, with mocks wired to match the current code — fails on every refactor and catches no real bug. Test _observable behavior through the public interface_ (return value, persisted state, emitted event, HTTP response), not the choreography that produces it. See Change-Detector Tests Considered Harmful and Prefer Testing Public APIs.
Ten tests that exercise the same path with different data are one parameterized test. If a new test is a variation of an existing one, it's a @parameterized case (Python) or a test.each row (Jest) — not a new test function. AI-generated suites bloat here first: hundreds of near-identical cases turning a 30s suite into 3 minutes.
Don't add tests to hit a number; an uncovered line is information, not a defect. If the only reason to test a branch is the coverage report, the branch probably doesn't need a test — or the code is dead and should be deleted instead.
When the answer is "don't write it," the right move is often to extend an existing test (add a parameterized case) or delete code rather than test it. Both shrink the suite's surface.
A test that earns its place still has to be cheap. Cost is a ladder; each rung is roughly an order of magnitude slower and flakier than the one below:
pure function / unit → kea logic test → Django TestCase → ClickHouse-backed test → Playwright e2e
cheapest most expensiveThe goal is a ratio, not a cap: many tests at the bottom, very few at the top. A thousand pure unit tests are cheaper and more reliable than ten that boot Django, which are cheaper than one Playwright run. Want more coverage? Add it at the bottom.
When something is hard to test cheaply, that's a design signal — extract, don't escalate. If the only way to exercise your logic is to stand up a database, a request, and a render, the logic is tangled with its dependencies. Pull it into a pure function or a kea logic and test _that_ directly: you get a faster test and better-factored code at once. Escalating to the next rung is the last resort, not the default.
logic.actions / logic.values), not a full component render, whenever the behavior lives in the logic.A render() + DOM-query test is for things only the DOM can show.
TransactionTestCase flushes the DB between tests instead of rolling back a transaction — dramatically slower, and a common source of cross-test interference. It's a Postgres-isolation choice, orthogonal to which datastore you touch: a ClickHouse-backed test is still a plain TestCase (ClickhouseTestMixin sets ClickHouse up), so reaching ClickHouse is not a reason to switch. Common cases that people reach for TransactionTestCase to solve usually have cheaper alternatives:
transaction.on_commit side effects → use TestCase + self.captureOnCommitCallbacks(execute=True).thread_sensitive) → async_to_sync(...), not asyncio.run(...).Use TransactionTestCase only when the regression genuinely requires committed transaction boundaries that TestCase hides.
parameterized library — don't copy-paste test bodies.Don't mock your own internal helpers (that's how change-detector tests are born).
describe block per file (house rule).test.each for variations rather than copied test bodies.A snapshot over a large rendered tree or serialized blob is a change-detector test that bloats the repo and breaks on every unrelated change. Snapshot a small, intentional value, or assert specific fields instead.
Use fake timers, wait_for / waitFor on a real condition, or freeze_time. A sleep is a flake waiting to happen, and it slows every run.
Tests must pass in any order and in isolation; don't rely on state a previous test left behind.
A permanently-skipped test is dead weight — delete it or fix it.
it.only / describe.only).It doesn't skip one test, it skips every _other_ test in the file — turning the suite green on a sliver.
The PR template prompts for this under "How did you test this code?" — state the justification where a reviewer will see it. One line per group of tests is enough:
_"Added 3 cases to test_cohort_query covering empty / single / oversized cohorts — guards the 500 we just fixed; couldn't extend an existing test because none exercised the empty path."_If you can't write that line, you've found a test that shouldn't be in the PR.
/fixing-flaky-tests (reproduce, root-cause, validate). Use this skill too only if the fix adds or substantially changes coverage./playwright-test for the mechanics.~30 seconds. Free. No account. Every finding cites a rule and a line of evidence.