Brand scoping

Outputs live under ~/notes/Klaviyo/<brand>/<subfolder>/, where <brand> is resolved at run time from the connected Klaviyo account's organizationName attribute (via mcp__claude_ai_Klaviyo__klaviyo_get_account_details). For ExampleBrand, this resolves to ExampleBrand. If multiple brands are connected, ask Will which brand the current run is for.

/klaviyo-ab

Treats A/B testing as a closed loop — setup writes a test record to disk, read mode reads the record + Klaviyo report to call the winner. One skill, two phases.

Phase 1 — setup

Flow

1. Parse the test request — variable (subject / send time / CTA copy / CTA placement / preheader / from-name / template), what's being tested (campaign type or specific campaign id), target segment.
2. Write the hypothesis — frame as If we change <variable> from A to B, then <metric> will move because <reason>. If the user didn't supply a reason, ask — no hypothesis = no test.
3. Pick the primary metric — open rate for subject/preheader/from-name tests; CTR for body/CTA tests; revenue per recipient (RPR) for "which version actually drove money" tests; conversion rate for landing-page tests.
4. Compute sample size — for the chosen metric, current baseline (pull from mcp__claude_ai_Klaviyo__klaviyo_query_metric_aggregates), minimum detectable effect (default 15% lift unless specified), 95% confidence, 80% power. Show the math. If the segment is too small to detect the effect, surface that.
5. Generate variants — A = control (whatever's currently used), B = the change. For subject-line tests, B should be a meaningfully different angle, not a one-word tweak.
6. Setup output — save the test record to ~/notes/Klaviyo/<brand>/ab-tests/[YYYY-MM-DD]-[slug]/test.md with all fields. Include the Klaviyo setup instructions (since campaign A/B is configured in Klaviyo's UI per-campaign).
7. Print summary — show the test, the setup steps, the date to come back for the read.

Setup output (test.md)

# A/B Test — [slug] — YYYY-MM-DD

**Status:** SETUP
**Variable:** ...
**Target:** [campaign id or flow step]
**Audience:** [segment, N profiles]
**Primary metric:** [open / CTR / RPR / conversion]
**Hypothesis:** ...

## Variants
- **A (control):** ...
- **B (variant):** ...

## Sample size math
- Baseline metric: X%
- MDE (minimum detectable effect): Y% lift
- Required per variant: N
- Available: N (split 50/50)
- Statistical setup: 95% confidence, 80% power
- Expected read date: YYYY-MM-DD

## Klaviyo setup steps
1. ...
2. ...

Phase 2 — read

Flow

1. Locate the test record — find the most recent test marked SETUP for the named campaign/flow, or take an explicit slug.
2. Pull the results — mcp__claude_ai_Klaviyo__klaviyo_get_campaign_report for the campaign (with variant data). For flow tests, klaviyo_get_flow_report per variant step.
3. Compute significance — two-proportion z-test for rate metrics, t-test for revenue metrics. Show the p-value, the lift, and the absolute numbers.
4. Call the winner — only if p < 0.05 AND lift > the threshold from the hypothesis. Otherwise: INCONCLUSIVE (sample too small) or NO EFFECT (lift below MDE despite significance).
5. Update the test record — append ## Results section with full numbers and verdict.
6. Print verdict + recommendation — one of: WINNER: B (+X% lift, p=Y) / WINNER: A (B didn't beat control) / INCONCLUSIVE — extend the test or larger sample needed / NO EFFECT — change is real-world neutral. Plus what to do next.

What this skill does NOT do

• Does not auto-apply the winner. Will decides whether to roll it out.
• Does not run the test mechanically — Klaviyo's A/B feature is configured per-campaign in their UI; this skill produces the brief + math + read.
• Does not run multivariate tests — two variants only.
• Does not test below sample-size threshold — surfaces "can't reach significance" up front.

Failure modes

• No hypothesis given → ask Will. No reasoning = no test.
• Sample size too small → surface, ask whether to widen the audience, extend the duration, or pick a bigger MDE.
• Test record missing on read → ask Will for the slug or campaign id; don't fabricate baseline.