Research Design (eursr-research-design)

ESR is a quantitative journal exacting about whether the comparative or longitudinal design actually identifies the mechanism from eursr-theory-building and rules out the leading confound. The design must connect the cross-level hypothesis to evidence that a single cross-section could not provide.

When to trigger

• Specifying the comparative frame, the panel structure, sampling, or the identification strategy
• A reviewer questioned causal claims, generalization, selection, measurement comparability, or a confound
• Justifying why your design adjudicates the rival account from eursr-literature-positioning

Comparative / cross-national

• Justify the country set by design logic (institutional contrast, regime types, most/least-similar),

not by data availability alone; say what variation each context contributes.

• Measurement equivalence is the first reviewer demand: establish that constructs mean the same

across countries (configural/metric/scalar invariance for latent scales; harmonized coding for education via ISCED/CASMIN, occupation via ISCO/ISEI/EGP).

• Macro N is small. With ~20-30 countries, country-level effects rest on few degrees of freedom —

design the macro hypothesis so it does not over-claim from a handful of clusters (see eursr-data-analysis).

Panel / longitudinal / event-history

• State what the panel buys. Within-person change (fixed effects), duration/timing (event history),

or growth (latent growth) — match the estimator to the theoretical quantity.

• Attrition and selection into and out of the panel must be addressed (weights, IPW, sensitivity).
• For staggered policy exposure, use heterogeneity-robust DiD (Callaway-Sant'Anna, Sun-Abraham,

Borusyak et al.), not naive TWFE.

Causal inference where feasible

• Much of ESR is observational; distinguish description, association, and causation honestly. If

causal, state the assumptions (ignorability, parallel trends, exclusion) and defend them; report a sensitivity bound (how strong an unobserved confounder would have to be).

Multilevel / SEM

• Specify the level structure (individuals in countries/regions/cohorts), the random effects, and why a

multilevel model is warranted; for measurement, build the latent model before the structural one.

The adjudication test (ESR-specific)

For the single strongest rival explanation: "If the rival were true rather than my argument, the cross-national (or over-time) pattern would look like ___; instead it looks like ___." If you cannot write it, the comparative/panel design does not yet identify the contribution.

What ESR referees demand of each design

Worked micro-example (illustrative)

A comparative study argues that vocational specificity smooths the school-to-work transition.

Country set: most-different welfare/training regimes (e.g., dual-system vs. general-education systems),
  chosen for institutional contrast, not convenience
Measurement: education harmonized via ISCED; vocational specificity coded from program-level data
Design: cross-national + cohort variation; cross-level interaction (specificity × individual track)
Disconfirming pattern sought: if signaling (not skills) drove it, the advantage would vanish once firms
  learn quality → instead it persists across the early career, as the specificity argument predicts
Macro-N caution: ~24 countries → country-level claim kept modest; SEs / df handled in data-analysis

The country set is design-driven, the measures are comparable, and the design specifies what pattern would falsify the argument.

Referee pushback → ESR-specific fix

• "Measures aren't comparable across countries." → Test invariance; report partial invariance and

what it permits; use harmonized coding schemes.

• "You infer too much from ~20 countries." → Re-state the macro claim modestly; use df-appropriate

inference (see eursr-data-analysis).

• "Association dressed as causation." → Restate what the design identifies; add a sensitivity bound or

placebo; drop causal verbs you cannot defend.

Calibration anchors

• Measurement equivalence is the comparative gate. A cross-national claim built on non-equivalent

scales is the most common fatal design flaw at ESR.

• The adjudication sentence is the test. If you can't write "if the rival were true the pattern

would look like ___," the comparison/panel does not yet earn the contribution.

• Identification honesty travels. Stating plainly what observational European data can and cannot

establish reads as strength to a quantitative panel.

Execution bridge (StatsPAI / Stata MCP)

Estimate and audit the design, don't only describe it. Full map: execution-with-mcp. ESR is comparative quantitative sociology; cross-country panels with confounded institutions — foreground fixed effects and clustering.

• detect_design → recommend → fit with as_handle=true → audit_result.
• Observational causal claims: staggered DiD (callaway_santanna / sun_abraham +

bacon_decomposition + honest_did_from_result); IV (effective_f_test + anderson_rubin_ci); RDD (rdrobust + mccrary_test).

• Experiments: randomization-based inference, romano_wolf for many-outcome

family-wise control, and mediate for mediation (not naive controlling-away).

• Sensitivity: oster_delta / sensemakr for observational claims.

Report the effect size in interpretable units; route the full battery to the appendix/supplement. A run end-to-end (synthetic data, real returns) is in the JF execution walkthrough.

Anti-patterns

• A country set chosen by data availability and dressed up as theory-driven
• Cross-national latent comparisons with no measurement-invariance check
• Over-claiming country-level effects from a handful of clusters
• Naive TWFE on staggered policy timing; ignoring panel attrition
• A design that cannot distinguish your mechanism from the leading alternative

Output format

【Design】comparative / panel / event-history / causal / multilevel-SEM
【What it identifies】description / association / causation
【Comparability / assumption】invariance or key assumption + how defended
【Rival ruled out】the adjudication sentence
【Macro-N / attrition / sensitivity】planned
【Next】eursr-data-analysis

Supplementary resources

• ../../resources/external_tools.md — multilevel / SEM / event-history / DiD tooling
• ../../resources/code/ — reproducible Stata + Python causal-inference skeleton (DiD/IV/RDD/DML)
• ../../resources/official-source-map.md — ESR methodological expectations