alterlab-statistical-analysis — independently scanned and version-tracked by SaferSkills.
SaferSkills independently audited alterlab-statistical-analysis (Agent Skill) and scored it 100/100 (green). The audit ran 55 deterministic rules across Security, Supply Chain, Maintenance, Transparency, and Community; it found 0 high-severity and 0 lower-severity findings. The full rule-by-rule trace and per-finding evidence are below. Free, methodology-open.
Findings & checks · 0 flagged
Every scanned point with the score it earned and what moved between them.
First recorded scan — no prior version to compare against.
The primary manifest — the file an agent reads to learn what this artifact does.
A systematic process for testing hypotheses and quantifying relationships. Conduct hypothesis tests (t-test, ANOVA, chi-square), regression, correlation, and Bayesian analyses with assumption checks and APA reporting. For academic research.
Use when:
a priori power analysis; multiple-comparison strategy.
remediation when violated.
Bayesian alternatives with Bayes Factors.
practical significance.
SELECT a test? → Test Selection Guide
CHECK assumptions? → Assumption Checking
RUN analysis? → Running Statistical Tests + references/code_examples.md
REPORT results? → Reporting Results + references/apa_report_templates.mdWorked code for every step is in references/code_examples.md.
Quick reference (full decision tree: references/test_selection_guide.md):
Two groups — independent + normal → independent t-test; independent + non-normal → Mann-Whitney U; paired + normal → paired t-test; paired + non-normal → Wilcoxon signed-rank; binary outcome → chi-square or Fisher's exact.
3+ groups — independent + normal → one-way ANOVA; independent + non-normal → Kruskal-Wallis; paired + normal → repeated-measures ANOVA; paired + non-normal → Friedman.
Relationships — two continuous → Pearson (normal) or Spearman (non-normal); continuous outcome + predictors → linear regression; binary outcome + predictors → logistic regression.
Bayesian alternatives exist for all of the above, giving direct probability statements, Bayes Factors, and the ability to support the null. See references/bayesian_statistics.md.
ALWAYS check assumptions before interpreting results. Use the bundled scripts/assumption_checks.py (comprehensive_assumption_check() and targeted functions — code in references/code_examples.md). It performs outlier detection (IQR + z-score), normality testing (Shapiro-Wilk + Q-Q), homogeneity of variance (Levene + box plots), and interpretation.
When violated:
severe → transform or non-parametric.
robust SE or WLS.
Detail: references/assumptions_and_diagnostics.md.
Primary libraries: scipy.stats (core tests), statsmodels (regression/diagnostics), pingouin (tests + effect sizes), pymc + arviz (Bayesian).
Copy-paste examples for t-test, ANOVA + post-hoc, linear regression with full diagnostics, and a Bayesian t-test: references/code_examples.md.
Effect sizes quantify magnitude; p-values only indicate existence. Always report them, with CIs. Benchmarks (guidelines only — context matters):
| Test | Effect Size | Small | Medium | Large |
|---|---|---|---|---|
| T-test | Cohen's d | 0.20 | 0.50 | 0.80 |
| ANOVA | η²_p | 0.01 | 0.06 | 0.14 |
| Correlation | r | 0.10 | 0.30 | 0.50 |
| Regression | R² | 0.02 | 0.13 | 0.26 |
| Chi-square | Cramér's V | 0.07 | 0.21 | 0.35 |
Calculation code and CI computation: references/code_examples.md. Full guidance: references/effect_sizes_and_power.md.
Run a priori power analysis to size a study before collecting data, and sensitivity analysis afterward to report the smallest detectable effect. Avoid post-hoc power. Code: references/code_examples.md. Guidance: references/effect_sizes_and_power.md.
Follow APA style (references/reporting_standards.md). Report descriptive statistics (M, SD, n), test statistics (name, statistic, df, exact p), effect sizes with CIs, assumption checks, and all planned analyses.
Ready-to-adapt report paragraphs for t-test, ANOVA, multiple regression, and Bayesian analysis: references/apa_report_templates.md.
Consider Bayesian methods when you have prior information, want direct probability statements, have small samples or sequential data, need to quantify evidence for the null, or have a complex (hierarchical, missing-data) model. Key advantages: intuitive interpretation, evidence for the null, flexibility, full uncertainty quantification. Comprehensive guide (priors, Bayes Factors, credible intervals, convergence checks): references/bayesian_statistics.md.
Pre-register analyses; always check assumptions; report effect sizes with CIs; report all planned analyses including non-significant ones; distinguish statistical from practical significance; visualize before and after; check regression/ANOVA diagnostics; run sensitivity analyses; share data and code; be transparent about violations and decisions.
P-hacking; HARKing; ignoring assumptions; confusing significance with importance; not reporting effect sizes; cherry-picking results; misinterpreting p-values; uncorrected multiple comparisons; ignoring missing-data mechanism (MCAR/MAR/MNAR); overinterpreting non-significant results.
references/test_selection_guide.md)scripts/assumption_checks.py)references/reporting_standards.md)references/)test_selection_guide.md — decision tree for choosing a testassumptions_and_diagnostics.md — checking and handling violationseffect_sizes_and_power.md — effect sizes and power analysisbayesian_statistics.md — complete Bayesian guidereporting_standards.md — APA reporting rulescode_examples.md — worked Python for tests, diagnostics, Bayesian, power, effect sizesapa_report_templates.md — ready-to-adapt APA results paragraphsscripts/)assumption_checks.py — automated assumption checking with visualizations(comprehensive_assumption_check, check_normality, check_homogeneity_of_variance, check_linearity, detect_outliers)
Key textbooks: Cohen (1988), Field (2013), Gelman & Hill (2006), Kruschke (2014). Online: APA Style Guide (https://apastyle.apa.org/), Cross Validated (stats.stackexchange.com).
~30 seconds. Free. No account. Every finding cites a rule and a line of evidence.