MNE Decoding / MVPA & BCI (grill → analyze → critic)

Multivariate decoding of neurophysiology data via the MNE MCP server. This skill is skeptical by design: the most damaging decoding mistakes — data leakage and a wrongly assumed chance level — produce a clean, plausible accuracy curve without any error, so the discipline is to grill the cross-validation before fitting and critique before believing.

Companion skills: mne-mcp-guard for technical execution safety; mne-methodology-critic for Phase 3. Loaded objects persist in one MNE session. Decoding needs scikit-learn.

PHASE 1 — GRILL (before fitting anything)

Do not fit a classifier until these are answered. If the user can't answer one, propose a sensible default and explicitly flag the open risk — never silently choose.

What is being decoded

• Which two (or more) conditions / labels, and what is the scientific claim tied to decodability?
• Class balance and sizes — n trials per class, per subject? (Imbalance silently inflates

accuracy and breaks the nominal chance level.)

• Feature space: sensors × time? band power? source space? What is the classifier actually seeing?

The two questions that decide validity

• Cross-validation structure. Subject-level (leave-one-subject-out) or trial-level? If

trial-level on pooled multi-subject data, do trials from one subject leak across train/test folds (⇒ identity decoding, inflated)? Is the split stratified by class? (This + leakage are the two fatal errors here.)

• Is every transform fit INSIDE the fold? Scaling, feature selection, ICA, PCA, even baseline

z-scoring must be fit on training data only within each CV fold (use an sklearn Pipeline). Anything fit on the full dataset before CV = leakage ⇒ optimistic, invalid.

Inference plan (pin this down NOW, not after seeing results)

• Chance level — established by label permutation (shuffle labels, re-decode many times), not

the nominal 1/n_classes. Imbalance and small n move true chance off 1/n.

• Multiple comparisons across time — a classifier per time point ⇒ many tests; plan a

cluster-based permutation test of scores-vs-chance, not per-time-point thresholding.

• Temporal-generalization claims. Will off-diagonal generalization be read as

maintenance / reactivation of a representation? That is a strong claim — state it in advance and guard it (it can also reflect a slow/sustained component, not reactivation).

• Metric: ROC-AUC / balanced accuracy (imbalance-robust) over raw accuracy?

PHASE 2 — ANALYZE

1. Capability + look first. mne_check_status (confirm scikit-learn present); inspect class

counts (epochs["condA"], epochs["condB"]) — balance and totals decide metric and CV.

1. Structured decoding (preferred path). Read

structured-decoding.md for mne_decode parameters, grouped-CV examples, time windows, classifier options and output axes. Use named arguments: mne_decode(cond_a="a", cond_b="b", scoring="roc_auc", cv=5). Read the PNG, per-fold scores and split diagnostics. A curve crossing 0.5 is descriptive, not a significance result. The tool supports binary labels only.

1. Custom path (full control via `mne_run_code`). Build a leakage-free Pipeline and slide it

over time, cross-validated:

   from sklearn.pipeline import make_pipeline
   from sklearn.preprocessing import StandardScaler
   from sklearn.linear_model import LogisticRegression
   from mne.decoding import SlidingEstimator, Scaler, Vectorizer, cross_val_multiscore
   X = epochs.get_data();  y = epochs.events[:, 2]          # transforms fit INSIDE folds
   clf = make_pipeline(StandardScaler(), LogisticRegression(max_iter=1000))
   sl  = SlidingEstimator(clf, scoring="roc_auc", n_jobs=-1)
   scores = cross_val_multiscore(sl, X, y, cv=5).mean(0)    # (n_times,)

1. Temporal generalization. Use mne_decode(method="generalizing", ...); output axes are

(train_time, test_time), plotted with test time horizontal and train time vertical. Read the diagonal for same-time decoding. Off-diagonal scores show cross-time generalization, not proof of maintenance or reactivation. Crop with tmin/tmax to bound quadratic scoring cost; do not select the window after inspecting scores and then treat it as confirmatory.

1. CSP for oscillatory BCI. Band-pass first, then mne.decoding.CSP inside the pipeline:

   from mne.decoding import CSP
   clf = make_pipeline(CSP(n_components=4), LogisticRegression(max_iter=1000))

1. Establish an empirical null by permutation: shuffle labels only within valid exchangeability

blocks (for example within participant when the design permits), re-run the complete CV pipeline, build the null distribution of scores; compare the observed curve to it (sklearn.model_selection.permutation_test_score for a single window). For group inference, subjects, not overlapping CV folds, are independent observations. Predefine the multiple-comparison correction across time or the train/test matrix. For independent-subject mean curves/matrices, use mne_decoding_group_test after reading group-inference.md. It provides group sign flips with max-t or cluster correction, not within-subject label permutations or nested tuning.

1. Archive the equivalent code + figures (the mne-analyst archiving convention).

Best-practice reminders: ROC-AUC / balanced accuracy under imbalance; Vectorizer to flatten features for plain sklearn estimators; report per-class n and the CV scheme explicitly.

PHASE 3 — CRITIC (before believing the result)

Hand the design + result to `mne-methodology-critic` (invoke the skill, or dispatch it as a subagent with references/methodology-checklist.md). For decoding work it will specifically check:

• data leakage — any transform fit on the full dataset before CV (FAIL);
• nominal chance vs label-permutation chance level;
• class imbalance — raw accuracy vs balanced accuracy / ROC-AUC, class sizes reported;
• trial-level CV leaking trials of one subject across folds (vs leave-one-subject-out);
• temporal-generalization off-diagonal over-interpretation (maintenance / reactivation);
• multiple comparisons across time — cluster-based permutation vs per-time-point thresholding.

Report its BLOCK / REVISE / PASS verdict to the user and act on it before stating conclusions.

See references/decoding-methods.md for deeper recipes (sliding vs generalizing estimators, leakage-free pipelines, CSP, RSA, mTRF encoding, and permutation/cluster inference).