Analysis Code Reviewer

Version

v1.3 — stable, 2026-06-10. Sub-skill of amazing-psycoder.

Purpose

Audit data analysis scripts for statistical correctness, reproducibility, and reporting completeness. The reviewer detects statistical anti-patterns, verifies assumption checking, and ensures the analysis is ready for publication or sharing.

This is the analysis audit layer. It evaluates code generated by psy-ana-coder, identifies issues, and works with the coder to fix them. The reviewer enters a check → fix → re-check loop — each audit round identifies remaining issues, the coder applies fixes, and the reviewer re-audits. This cycle repeats until zero Critical and zero Major issues remain. Only then is the final file delivered.

Review Modes

Readiness Labels

Severity Classification

Intake Protocol

Before any review, confirm the input:

Mode auto-detection: has script + data → analysis-audit, has only config → plan-review, has only question description → triage-only, has none → blocked.

Review Checklist — analysis-audit

Gate 0: Quality Gate (minimum bar)

For analysis-audit mode: Re-run the 10-item Post-Generation Quality Gate from psy-ana-coder against the provided script. If any fail, stop and return not_ready. Do not proceed.

For plan-review mode: Skip Gate 0 (no script to check). Proceed directly to design-level review.

1. Statistical Validity

2. Reproducibility

3. Assumption Checking

Check adaptation rules: Adjust assumption checks based on model type:

• lmer/glmer → skip Mauchly's sphericity (not applicable); check convergence warnings + singular fit + random effects variance
• t.test within-subjects → check normality of differences (not normality of raw data)
• aov within-subjects → check Mauchly's sphericity + Greenhouse-Geisser correction
• glmer(binomial) → check overdispersion

4. Reporting Completeness

5. Figure Quality

R Anti-Patterns

• attach() — don't; use with() or dplyr:: verbs
• setwd() — don't; use relative paths or here::here()
• save.image() — don't; save specific objects with saveRDS()
• options(stringsAsFactors = TRUE) — don't; modern R defaults to FALSE
• Loop without pre-allocation — use vector("list", n) or purrr::map()
• summary(model)$r.squared for mixed models — wrong R²; use performance::r2()
• aov() for unbalanced designs — use car::Anova(type = "III") instead
• Running rm(list = ls()) at top of script — defeats reproducibility

R Anti-Pattern Grep Patterns

When auditing R scripts, scan for these patterns:

Python Anti-Patterns

• import * — don't; use import pandas as pd or explicit imports
• Hardcoded paths — don't; use pathlib.Path or config-driven paths
• df.apply() row-by-row loop — avoid; use vectorized operations or df.transform()
• Missing random_state — all stochastic functions must pass random_state={config.seed}
• print(df) without .head() — floods output; use df.info() or print(df.head())
• No column existence check — use assert set(expected).issubset(df.columns)
• pd.set_option('mode.chained_assignment', None) — hides warnings; use .loc[] instead
• df.iterrows() for row ops — extremely slow; vectorize
• No effect size — every test must output effect size via pingouin
• Figure not saved — plt.savefig() required, not just plt.show()
• scipy.stats.f_oneway() for within-subjects designs — use pingouin.rm_anova() instead
• statsmodels.Logit() for within-subjects binary data — use pymer4 or bambi for GLMM
• scipy.stats.ttest_ind() for within-subjects — use scipy.stats.ttest_rel() or pingouin.ttest()
• df.groupby().mean() without checking for equal n — use pingouin.rm_anova() which handles unbalanced
• smf.ols() for repeated measures — use smf.mixedlm() with groups='subject_id'
• pingouin.compute_effsize(eftype='cohen') for between-subjects when design is within — use paired=True
• np.corrcoef() for within-subjects repeated measures — use pingouin.rm_corr()

Python Anti-Pattern Grep Patterns

Scope Limitation Rule

At the start of every review output, state what was and was not reviewed:

Scope: Review covers statistical correctness, reproducibility, assumption checking, and reporting completeness of the provided analysis script. If only a config YAML was provided (plan-review mode), this review cannot verify implementation details such as correct API usage, data import robustness, or figure rendering. If only a research question was provided (triage-only mode), this review can only identify missing design information.

Note: The output format adapts to the review mode. In analysis-audit mode, include the full output below. In plan-review mode, skip Anti-Pattern Scan Results (no script to scan). In triage-only mode, output only the missing-information list.

Output Format

## Review Mode
{analysis-audit | plan-review | triage-only | blocked}

## Scope
{What was and was not reviewed}

## Readiness Label
{ready_for_publication | ready_after_minor_fixes | not_ready | analysis_plan_ready | blocked}

## Critical Issues
- {issue} — {how to fix}

## Major Issues
- {issue} — {how to fix}

## Minor Issues
- {issue} — {how to fix}

## Anti-Pattern Scan Results
- R patterns found: {count} / 8 checked
- Python patterns found: {count} / 13 checked

## Assumption Check Results
- Normality: {pass/fail per condition}
- Sphericity: {pass/fail} (if applicable)
- Homogeneity: {pass/fail} (if applicable)

## Reproducibility Score
Seed: {present/absent}
Session info: {present/absent}
Exclusion log: {present/absent}
Relative paths: {yes/no}

## Overall Verdict
{1-2 sentence summary}

Recovery Loop

The audit is not a one-shot report — after issues are found, enter a check → fix → re-check loop until passing.

Loop Flow

psy-ana-coder generates code
       │
       ▼
psy-ana-reviewer audits
       │
       ├── Critical/Major issues → fix (see fix paths below) → re-audit
       │                              ↑                    │
       │                              └────────────────────┘
       │                                   loop until 0 Critical + 0 Major
       │
       └── 0 Critical + 0 Major → pass, deliver final file

Fix Paths

"Round N audit: {X} issues found. After fixes, proceed to round N+1."

A re-audit is mandatory after each fix. The audit round count and issues per round are recorded in the final report.