Evaluate Findings

Assess external feedback (code reviews, AI suggestions, PR comments) with adversarial verification. Triage findings into actionable verdicts. Do not apply fixes.

Step 1: Assess Each Finding

If you already assessed a finding earlier in this session and recorded a verdict of Skip or Escalate — for example when an iterating loop re-runs review and the same finding resurfaces — do not re-adjudicate it from scratch. When the re-reported finding matches one you already judged (same location and substance) and presents no new evidence beyond what your recorded reason already accounts for, keep that verdict and reason without re-reading the code, re-verifying, or routing it to the Devil's Advocate in Step 2. Assess fresh only when the finding raises materially new evidence, or when you have not judged it before in this session.

When several findings rest on a shared premise — for example a source-of-truth choice — verify that premise once before adjudicating them individually. Findings whose premise holds proceed through normal per-finding verification; when it fails, they are all Skip, citing the refuted premise.

When a plan governs the work, re-read the decisions it records before adjudicating. Having read it earlier in the session does not count: once it falls out of context, a recorded decision is indistinguishable from no decision at all.

When the repo keeps an improvements backlog (.turbo/improvements.md at the repo root or, inside a linked worktree, at the main checkout's root), search it before adjudicating for the paths and symbols the findings name and for entries on the same subject, leaving out any entry the work in hand sets out to implement. Read every matching entry: run the search uncapped, splitting it into one search per finding when the combined output is too large to read, since output cut off or capped before its end settles nothing about the entries it left out. An entry that parks all or part of the change a finding asks for records a decision to defer it. When that finding holds and acting on it would make the change now, treat it as one that would reverse a decision the user made earlier, naming the entry as the original decision and quoting any condition it records for taking the change up. An entry that only shares the finding's area leaves the verdict to the finding's merits. In both cases, weigh what the entry records about that code when verifying the claim and assessing severity.

For each finding:

1. Read the referenced code at the mentioned location — include the full function or logical block, not just the flagged line
2. Check whether the code has diverged — if the finding references code that no longer exists or has since changed, skip it and note the divergence.
3. Determine scope — clarify whether the issue was introduced by the PR/changeset or is pre-existing.

• Pre-existing issues in earlier commits on the same feature branch are in-scope by default — the entire branch is one coherent unit of work. Judge these on their merits like any in-scope finding.
• Findings genuinely outside the branch's work are the user's call to include. Assign Escalate so the user decides whether to widen the changeset. Reserve Skip for changes whose cost wildly dwarfs the benefit.

1. Verify the claim against the actual code — does the issue genuinely exist?

• When the finding offers a concrete example as evidence — a claimed mishandled input, a claimed wrong output — verify that example independently: a finding can hold in substance while its example does not. Keep the finding and record the correction beside it; drop it only when the claim rests on that example alone.
• When the finding places a defect on one code path on the strength of something true of another path that shares code with it, whether a defect observed there or a state asserted to hold throughout the shared operation, that holds for the other path only. Trace the flagged path to the point where the defect would fire and confirm the state it needs is already in place there (a failure raised before the step that establishes that state never finds it in place), and Skip the claim for that path when it is not.
• When the finding claims an outcome that a sequence of steps produces, such as an exploit, a bypass, or an exhausted or replenished limit, trace the sequence end to end through the state each step changes (what it consumes, restores, writes, or grants) before assigning Apply. Confirming that its preconditions hold leaves the claim open. When the trace shows the outcome does not arise, Skip, citing the step that prevents it. When a step's effect cannot be traced from the code, assign Escalate, naming that step in the Issue cell.
• When the finding asserts a compatibility property, establish two things before assigning Apply: what the existing check actually enforces, and what real counterparts produce today. A claim stronger than the check enforces is a premise error rather than a defect — Skip, citing what the check enforces, or narrow the finding to the property it does enforce and record the narrowing beside it. When neither can be established from the code, the artifacts, or authoritative documentation, keep the finding Escalate.
• When the finding cites a rule or convention, read the cited text, then look for a place that already applied it before this changeset — the same file, or the nearest files the rule also governs. Where the text alone leaves the reading open, read the rule the way that application reads it; where no such application exists, judge on the text alone.
• When the finding rests on a premise that reading the code in question cannot settle — what a platform API returns at runtime, or what a value measures once the system runs — establish that premise before assigning Apply or Escalate, using a targeted search or count over the source, the implementation behind the premise (the platform's or dependency's own source, or an open-source counterpart of the component checked against the shipped binary as installed on this machine where the two may differ), or a measurement from a surface already running in this session. A premise of this kind reads as sound whether or not it holds, so confidence in the finding is no substitute. When the premise is that a guard, check, or rule is redundant and the evidence is a comparison of runs with and without it, those runs must include the inputs it exists for, built on purpose where the condition it protects is hardest to meet. Runs that never reach those inputs leave the premise unsettled. When nothing available settles it, assign Escalate on that ground, naming the premise as unverified in the Issue cell.
• When the finding, or the verdict you reach on it, rests on an absence beyond the referenced code — nothing else references a symbol, no caller remains, the code is dead — establish it with a search whose every match you read. Output cut off or capped before its end settles nothing about the matches it left out: re-run the search uncapped, splitting it into several runs that together cover the same paths when the full output is too large to read. State the paths searched and the match count in the Issue cell.

1. Assess severity:

If the upstream reviewer already assigned a priority (P0-P3), map it: P0→Critical, P1→High, P2→Medium, P3→Low. Then re-assess based on what the actual code reveals. The upstream level is a starting point, not a binding constraint. When the re-assessed severity differs from the upstream level, note the change and the reason.

If the finding has no upstream priority, assess severity from scratch.

1. Assign a verdict and confidence:

Also assign an internal confidence level — High, Medium, or Low — reflecting how certain you are about the verdict. Confidence is used solely to route findings to the Devil's Advocate in Step 2. It does not appear in the output.

Escalate guidance: When a finding questions whether behavior is intentional and neither docs, specs, nor code comments clarify the intent, assign Escalate. Do not autonomously accept or reject findings that hinge on product intent. If a counterpart implementation exists elsewhere, suggest checking it for consistency.

Conflict guidance: When two findings disagree about whether the code should change at all (one suggests a change to it, another the opposite change or none), treat the conflict as input, not a reason to skip. Verify each against the code and judge each on its merits as usual. If both are defensible and the choice is a genuine trade-off, assign Escalate to both, naming the opposing options so the user can decide.

An affirmation that something is correct is not a finding and carries no evidentiary weight; agreement among reviewers, or a reviewer's authority, does not settle whether a problem exists, nor whether a remedy the reviewers converged on works. When reviewers disagree on whether something is a problem at all — including one asserting it is fine while another flags it — treat the question as unresolved and verify it against the code, without letting the affirmation substitute for verification. When reviewers agree the code should change but propose opposing remedies, assign the verdict on the finding's own merits and name the opposing remedies in the Issue cell. Prefer a remedy whose measurement reports a result concrete enough to re-run over one resting on reading or on a reviewer's authority; a bare claim to have measured ranks no higher than reading. Where no remedy reports one, say so in that cell rather than choosing between them.

A reviewer's report that it could not verify something is a claim to check, not a fact to accept. Attempt the check independently, especially when the reported inability is what justifies skipping a verification step.

Verdict guidance:

• A verdict records whether the finding is real: genuine defect, in scope, at what severity. A finding can hold while the remedy proposed for it does not, so a verdict never certifies the remedy. Judge the remedy's cost and scope where the bullets below call for that, and leave whether it works to be checked when it is applied. Where naming the likely direction helps the user, put it in the Issue cell flagged as unverified.
• Never auto-dismiss findings about security defaults, permission escalation, or fail-open vs fail-closed behavior. Always surface these even if the behavior appears intentional.
• Readability and clarity improvements that genuinely make code cleaner are valid. Do not auto-classify cosmetic changes as subjective.
• Removing a comment that adds no information beyond the code is a valid Apply, not a subjective preference. Keep only comments that capture a constraint the code cannot express.
• Be skeptical of "defensive coding" suggestions that wrap natural code in verbose guards without evidence of real-world failures. Apply a hardening finding only when it names a failure scenario reachable in this deployment, whatever severity the reviewer attached; when the plan's Context bounds the system (a single operator, no concurrent writers, a handful of invited users), a scenario that bound rules out is a Skip, citing the bound.
• Machinery is scope. A finding whose fix adds a lease, lock, queue, versioning scheme, state machine, or new persistent entity expands the project even when the requirement count stays flat. Assign Escalate regardless of confidence; "making states explicit" or "staying within the approved plan" does not make the machinery proportionate. When a stated bound rules the machinery's failure scenario out entirely, Skip instead, citing the bound.
• A finding that would reverse a decision the user made earlier — in discussion or recorded in the artifact — is Escalate, naming the original decision and the new evidence beside it. Judge by the outcome rather than the wording of the option the user chose: a reversal leaves the user with something materially different from what they chose. A finding that refutes only the factual premise the user's choice rested on, leaving the chosen outcome intact, is a premise correction: confirm the refutation against whichever of the code, the governing artifact, or authoritative documentation the premise turns on, and when none settles it, keep the finding Escalate. Otherwise assign Apply unless another bullet independently calls for Escalate, and add a callout below the table naming both the corrected premise and the chosen outcome it leaves standing.
• In an iterating loop, a structural Apply triggers another full iteration; count that iteration in the change's cost when applying the Skip cost test. A finding that targets code or text introduced by an earlier iteration's accepted finding, and names no defect in it, is churn: Skip.
• When a comment at the code in question records why a behavior is unobservable through every reachable path, verify that record against the code before honoring it. A coverage finding re-reporting the gap, naming no newly reachable path that would observe the behavior, is Skip, citing the record.
• Weight reviewer authority. Feedback from trusted reviewers (repository maintainers or admins) should be treated with higher credibility even when phrased softly.
• Plan deviation is not a verdict. Do not reject a finding on the grounds that it departs from a plan's prescribed shape. When the plan records a load-bearing reason for that shape, assign Escalate so the user can weigh the trade-off. When the plan is silent on why, or the recorded reason reads like "path of least deviation" or "minimal change", treat the shape as a default and judge the finding on its own merits.

Step 2: Devil's Advocate

After the initial assessment, challenge uncertain findings from a different angle.

Spawn when any finding has Medium or Low confidence. Send only those findings to the subagent. High-confidence findings pass through unchallenged. Skip this step entirely if all findings are High confidence.

Capture git status --short, git diff --cached | git hash-object --stdin, git diff | git hash-object --stdin, and git symbolic-ref --short -q HEAD before spawning.

Launch a single subagent (model: "opus", no name). Wait for it to report before continuing; do not relaunch it if it has not yet reported. Provide the Medium/Low-confidence findings with their file locations, claims, and initial verdicts. Instruct the subagent to challenge each finding: try to prove it wrong, or confirm it with evidence.

Evidence standards: A refutation counts only when it rests on a defense, guarantee, or documented behavior the subagent located and read, or on behavior it observed by running the code; an expectation that a framework, caller, or type already handles the case returns Inconclusive and leaves the initial verdict standing. Confirmed applies to the claim the finding stands or falls on. Establishing the premise beneath that claim leaves it open: that code reads a value settles nothing about whether a test can control that value. When the subagent has established only the premise, it returns Inconclusive. Evidence that a test fails when its subject is changed settles only that the test pins the behavior; whether the pinned behavior is the required one stays open. Where the test was written alongside its implementation, their agreement is guaranteed by construction and carries no evidence about the requirement. A finding resting on that evidence is Confirmed only when the requirement itself has been checked against the code's production consumer, or against the governing plan; when neither is reachable, the subagent returns Inconclusive.

Protect the shared tree: The subagent's prompt must direct it to treat the shared working tree and its git index as read-only; an experiment that needs a scratch project runs in a temp directory outside the repo, or in an isolated git worktree created there and discarded afterward. HEAD stays where it is: read other refs with git show <ref>:<path> rather than git checkout or git switch. Refer to that directory or worktree by absolute path in every command and join chained steps with &&, so a failed step cannot leave the rest running in the shared checkout. Run teardown and verification as their own commands. Give that worktree its own dependency install rather than reaching the shared tree's install by any route: removing a worktree deletes through symlinks, and a redirected suite writes into the shared install. When its own install is not possible, the check is left unrun and reported as such. A check that runs in the shared checkout invokes an already-installed runner directly wherever a package-manager wrapper would front it, since such a wrapper reads as read-only while reconciling the shared install before it runs. Confine dependency installs and reconciliation to an isolated worktree. Every test runner the subagent starts, in a temp directory, a worktree, or the shared checkout, runs in its own process group under a timeout enforced from outside the runner. Before teardown, the subagent stops the process group of every runner it started, since stopping a runner can leave the processes it spawned alive. Afterward the subagent verifies that git worktree list no longer shows the worktree, that git status --short shows what it showed at the start, and that HEAD is still on the branch it started on. Its teardown deletes the temp directory, if any, and it confirms that path no longer exists, reporting the path of one it could not delete. It also confirms that no process from those groups, and none whose command line names the temp directory or worktree path, if any, is still running, and reports by PID any process it could not stop. When it cannot list processes, it reports that check as unrun and names those process groups and the temp directory or worktree path, if any. After any check, in a worktree or in the shared checkout, it verifies that the shared tree's dependency directory still resolves (a destroyed install leaves git status unchanged, since it is gitignored). Damage the subagent cannot repair is reported with the exact repair command in place of findings.

Verify the tree: re-run all four commands when the subagent returns, including when it terminates early or reports incomplete results, and confirm alongside them that the shared tree's dependency directory still resolves, which the four cannot see. Delete what the subagent created, revert what it modified or staged, and return HEAD to the captured branch, leaving everything the pre-spawn capture already showed untouched.

The subagent picks research tools based on claim type:

Use whatever documentation tools are available. The specific tools vary by project setup.

Budget: max 2 research actions per finding. If the first action is conclusive, skip the second.

Subagent Verdicts

The subagent returns per finding:

• Confirmed — found evidence supporting the claim (with source)
• Disputed — found counter-evidence (with source and explanation)
• Inconclusive — no definitive evidence either way

Step 3: Reconciliation

Merge subagent results with the initial assessment:

• Confirmed: verdict and severity stand. Note the evidence source.
• Disputed: if originally Apply, downgrade to Skip or Escalate. Re-assess severity if the evidence changes the impact picture. Show both perspectives.
• Inconclusive: verdict and severity stand, note the uncertainty.

Findings not investigated by the subagent keep their original verdict.

For Apply findings, document the issue and location. For Escalate findings, note what information would resolve the ambiguity. For Skip findings, document why.

Step 4: Format Output

Summarize the evaluated findings in a table:

When Step 2 ran (any finding was investigated by the Devil's Advocate subagent), add an Investigated column:

Where Investigated shows:

• (empty) — not investigated by subagent
• Confirmed (source) — subagent found supporting evidence
• Disputed: [reason] — subagent found counter-evidence

For findings whose severity was re-assessed from the upstream level, append the change in the Severity cell (e.g., "High (was Medium)").

Carry each verdict into the Verdict column exactly as assessed, Escalate included.

For disputed findings, add a callout below the table showing both perspectives. For each finding, indicate scope in the Issue column (e.g., "Pre-existing:" prefix).

Then use the TaskList tool and proceed to any remaining task.