ln-638-test-oracle-effectiveness-auditor — independently scanned and version-tracked by SaferSkills.
SaferSkills independently audited ln-638-test-oracle-effectiveness-auditor (Agent Skill) and scored it 100/100 (green). The audit ran 55 deterministic rules across Security, Supply Chain, Maintenance, Transparency, and Community; it found 0 high-severity and 0 lower-severity findings. The full rule-by-rule trace and per-finding evidence are below. Free, methodology-open.
Findings & checks · 0 flagged
Every scanned point with the score it earned and what moved between them.
First recorded scan — no prior version to compare against.
The primary manifest — the file an agent reads to learn what this artifact does.
Paths: File paths (references/,../ln-*) are relative to this skill directory.
Type: L3 Worker
Specialized worker auditing whether tests have meaningful oracles that would catch real product defects.
STRENGTHEN_ORACLE, DELETE_WEAK_ORACLE, or ADD_MUTATION_EVIDENCEMANDATORY READ: Load references/audit_worker_core_contract.md.
Receives contextStore with: tech_stack, testFilesMetadata, codebase_root, output_dir.
Detection policy: use two-layer detection (candidate scan, then context verification); load references/two_layer_detection.md only when the verification method is ambiguous.
1) Parse Context: Extract tech stack, test file list, codebase_root, and output_dir from contextStore 2) Find weak-oracle candidates (Layer 1): Scan automated tests for no assertions, vague assertions, snapshot-only tests, excessive mocks, and mutation-test survivors when reports exist 3) Context Analysis (Layer 2 -- MANDATORY): For each candidate, ask:
4) Collect Findings: Record each confirmed issue with severity, location, effort, action, and recommendation 5) Calculate Score: Count violations by severity, calculate compliance score (X/10) 6) Write Report: Build full markdown report in memory per references/templates/audit_worker_report_template.md, write to {output_dir}/ln-638--global.md in single Write call 7) Return Summary: Return minimal summary to coordinator (see Output Format)
What: Tests with no assertions, vague assertions, or assertions that would pass for many broken implementations
Detection:
expect, assert, should, or framework-specific verificationtoBeTruthy, toBeDefined, not.toThrow, or status-only checks when domain values matterLayer 2:
Severity: HIGH for critical logic, MEDIUM otherwise
Recommendation: Assert the product-specific output, state transition, side effect, emitted event, or persisted data that would fail under a real regression
Effort: S-M Action: STRENGTHEN_ORACLE, or DELETE_WEAK_ORACLE if the test has no product signal
What: Tests whose expected result is not tied to product behavior
Detection:
Severity: MEDIUM
Recommendation: Replace structural assertions with behavior-level assertions derived from requirements, examples, fixtures, or golden data
Effort: S-M Action: STRENGTHEN_ORACLE
What: Snapshot tests without semantic assertions for the product behavior being protected
Detection:
toMatchSnapshot, image snapshots, serialized JSON snapshots, or golden files with no nearby semantic assertionLayer 2:
Severity: MEDIUM
Recommendation: Keep snapshots only as secondary evidence; add semantic assertions for key behavior or delete the snapshot-only test if it has no unique signal
Effort: S-M Action: STRENGTHEN_ORACLE or DELETE_WEAK_ORACLE
What: Test mocks the system under test or internal collaborators so aggressively that the real behavior is not exercised
Detection:
Layer 2:
ln-635; this check is about proof strength, not dependency controlSeverity: HIGH when the test appears to cover critical logic but proves only mocks, MEDIUM otherwise
Recommendation: Exercise real local behavior and mock only external boundaries; assert the product contract instead of implementation calls
Effort: M Action: STRENGTHEN_ORACLE
What: Use mutation reports or equivalent evidence to detect tests that do not fail when production behavior changes
Detection:
Severity: LOW for missing mutation evidence alone, MEDIUM when surviving mutants confirm weak assertions
Recommendation: Add mutation-style evidence for critical local logic only; do not require mutation testing for the whole suite
Effort: M-L Action: ADD_MUTATION_EVIDENCE
MANDATORY READ: Load references/audit_scoring.md.
Severity mapping:
MANDATORY READ: Load references/templates/audit_worker_report_template.md.
Write JSON summary per references/audit_summary_contract.md. In managed mode the caller passes both runId and summaryArtifactPath; in standalone mode the worker generates its own run-scoped artifact path per shared contract.
Write report to {output_dir}/ln-638--global.md with category: "Oracle Effectiveness" and checks: assertion_strength, meaningful_oracle, snapshot_oracle, over_mocking, mutation_style_evidence.
Return summary per references/audit_summary_contract.md.
When summaryArtifactPath is absent, write the standalone runtime summary under .hex-skills/runtime-artifacts/runs/{run_id}/evaluation-worker/{worker}--{identifier}.json and optionally echo the same summary in structured output.
Report written: .hex-skills/runtime-artifacts/runs/{run_id}/audit-report/ln-638--global.md
Score: X.X/10 | Issues: N (C:N H:N M:N L:N)Apply the already-loaded references/audit_worker_core_contract.md.
STRENGTHEN_ORACLE, DELETE_WEAK_ORACLE, or ADD_MUTATION_EVIDENCE.Apply the already-loaded references/audit_worker_core_contract.md.
{output_dir}/ln-638--global.md (atomic single Write call)Version: 1.0.0 Last Updated: 2026-05-09
~30 seconds. Free. No account. Every finding cites a rule and a line of evidence.