creating-experiments — independently scanned and version-tracked by SaferSkills.
SaferSkills independently audited creating-experiments (Agent Skill) and scored it 96/100 (green). The audit ran 55 deterministic rules across Security, Supply Chain, Maintenance, Transparency, and Community; it found 0 high-severity and 1 lower-severity findings. The full rule-by-rule trace and per-finding evidence are below. Free, methodology-open.
Findings & checks · 1 flagged
The text {match} tells the agent to skip the normal "ask the user first" gate. Used adversarially it removes the human-in-the-loop check before destructive or sensitive actions, turning a normally-gated agent into a fire-and-forget executor.
Every scanned point with the score it earned and what moved between them.
First recorded scan — no prior version to compare against.
The primary manifest — the file an agent reads to learn what this artifact does.
This skill walks through the 3-step flow for creating a new A/B test experiment.
Create the experiment as a draft quickly, then iterate on metrics and configuration. The user gets a tangible draft immediately and can refine it.
Gather these before calling experiment-create:
description)The flag is auto-created — do NOT create one separately.
"product". The "web" value is reserved for no-code experiments configured visually with the PostHogtoolbar in a browser; it cannot be meaningfully driven via MCP. If a user asks for a no-code/toolbar experiment, point them to the PostHog UI instead of creating one here.)
If the user gives enough context to infer these, don't ask — just proceed.
This is about rollout configuration.
Before asking any rollout question, load `configuring-experiment-rollout`. The disambiguation wording, recommendations, and post-answer branches live there — do not formulate rollout questions yourself, and do not assume an example you remember covers the user's path.
Key decision points (covered in detail by configuring-experiment-rollout):
If the user doesn't mention rollout specifics, use defaults: 50/50 control/test, 100% rollout.
This is about analytics and metrics. Load the `configuring-experiment-analytics` skill for guidance.
Do NOT configure metrics on creation. Metrics are not passed to experiment-create — they are added afterwards via experiment-update. This keeps the creation call lightweight.
When the user specifies metrics upfront, acknowledge them and add them immediately after creation. When they don't, create the draft and then guide them through metric setup as a follow-up.
Call experiment-create with:
{
"name": "Descriptive experiment name",
"feature_flag_key": "kebab-case-key",
"description": "Hypothesis: [what you expect to happen]",
"parameters": {
"feature_flag_variants": [
{ "key": "control", "name": "Control", "split_percent": 50 },
{ "key": "test", "name": "Test", "split_percent": 50 }
],
"rollout_percentage": 100
}
}Two different percentages — do NOT mix them up:
feature_flag_variants[].split_percent — how users inside the experiment are split across variants (must sum to 100, recommended to have an even split).parameters.rollout_percentage — what fraction of all users enter the experiment at all (0-100, defaults to 100).Key details:
"control". Minimum 2, maximum 20 variants.rollout_percentage defaults to 100 if omitted.stats_config if the user requests Frequentist.experiment-create response includes _posthogUrl — always display this link so the user can view and configure the experiment in the UI.configuring-experiment-analytics skill.experiment-launch tool.~30 seconds. Free. No account. Every finding cites a rule and a line of evidence.