Benchmark Engine

Benchmarks account-level adoption metrics against segment peers. Produces customer-safe comparison data that shows where the account stands relative to similar companies. Part of the Product Adoption & Value Realisation pillar.

This is a comparison and reporting skill. It computes how an account measures against its cohort. It does not advise on what to do about the comparison -- that is consultative work the CSM leads.

When to Run

• Triggered: By cc-qbr-deck-builder (for QBR benchmark slides), by pa-adoption-tracker (for gap analysis context)
• On-demand: When a CSM needs benchmark data for a customer conversation
• Scheduled: Monthly benchmark refresh across all accounts

Core Execution Logic

Step 1: Identify Benchmark Cohort

Match the account to its segment cohort using the same segmentation as bi-health-score:

• Product tier (determines feature universe)
• Company size band (determines expected usage intensity)
• Industry (if sufficient data -- minimum 20 accounts per industry cohort)
• Contract age cohort (normalises for maturity -- a 3-month account should not be compared to a 3-year account)

If the precise cohort has fewer than 20 accounts, fall back to the next broader segment (e.g., drop industry, then drop contract age). Flag the fallback in the output.

Step 2: Compute Benchmark Metrics

For each metric, compute the cohort distribution and the account's position within it:

For each metric, compute:

• Account value (current period)
• Cohort 25th percentile
• Cohort median (50th percentile)
• Cohort 75th percentile
• Account percentile rank within the cohort

Step 3: Classify Position

Never use "bottom quartile" or "underperforming" in customer-facing language. Frame gaps as opportunities, not deficiencies.

Step 4: Generate Comparison Views

Produce three comparison formats:

Snapshot: Current position across all metrics (for QBR slides) Trend: Position change over the last 3-4 quarters (for progress narrative) Gap analysis: Where the account trails peers most, ordered by potential impact (for adoption planning)

Step 5: Generate Suggested Framing

For each metric, produce a one-sentence framing the CSM can use in conversation:

• Leading metrics: "Your team's [metric] is in the top quartile -- that is driving the outcomes you are seeing in [related outcome]"
• Lagging metrics: "Similar companies typically achieve [benchmark value] on [metric] -- your team is at [current value], which suggests there is room to drive more value from [related feature]"

These are suggestions for the CSM to adapt, not scripts to read verbatim.

Output Format

{
  "account_id": "string",
  "benchmark_date": "2026-03-10",
  "cohort": {
    "definition": "mid-market / medium / established",
    "cohort_size": 48,
    "fallback_applied": false
  },
  "metrics": [
    {
      "metric": "feature_adoption_breadth",
      "account_value": 0.65,
      "cohort_p25": 0.42,
      "cohort_median": 0.58,
      "cohort_p75": 0.72,
      "percentile_rank": 68,
      "position": "above_average",
      "trend_4q": "improving",
      "customer_framing": "Your team uses more features than most similar companies -- 65% breadth vs. 58% median. This depth of adoption is a strong indicator of value realisation."
    },
    {
      "metric": "active_user_ratio",
      "account_value": 0.62,
      "cohort_p25": 0.55,
      "cohort_median": 0.71,
      "cohort_p75": 0.84,
      "percentile_rank": 34,
      "position": "below_average",
      "trend_4q": "stable",
      "customer_framing": "Similar companies at your tier typically have 71% of their licensed users active. Your team is at 62% -- there is an opportunity to drive more value by expanding adoption across the team."
    }
  ],
  "summary": {
    "leading_metrics": ["feature_adoption_breadth", "key_workflow_completions"],
    "lagging_metrics": ["active_user_ratio"],
    "overall_position": "above_average",
    "key_opportunity": "Active user ratio is the primary gap. If addressed, it would move the account from above average to top quartile overall."
  },
  "customer_safe": true
}

Handoff to Human

Confidence and Limitations

• High confidence for percentile computation given sufficient cohort size (20+ accounts) -- the math is deterministic
• Medium confidence for cohort matching -- segmentation accuracy directly affects benchmark relevance. An account miscategorised into the wrong tier or size band produces misleading comparisons
• Medium confidence for suggested framing -- the language is appropriate for the metric position but may not fit the relationship tone or conversation context
• Low confidence for small cohorts (<20 accounts). Results are noisy and may mislead. Always flag small cohort size in the output
• Benchmarks are descriptive, not prescriptive. Being below median is not inherently bad if the customer's use case requires less intensive usage (e.g., a customer using the product for a narrow, specialised purpose may have low breadth scores but high depth on the features they use)
• Cannot benchmark on dimensions not tracked in product analytics (relationship quality, strategic alignment, internal satisfaction)
• Customer-facing benchmarks must be anonymised. Never expose individual account data. Never name specific peer companies. Present as "companies similar to yours"
• Benchmark data reflects the cohort's current state, which is not necessarily "good." If the entire segment underutilises a feature, the median is low. An account at the median is "average for the segment" but may still be underperforming against what is possible

Dependencies

Required:

• Product analytics API (feature-level usage data across all accounts for cohort computation)
• Segment benchmark dataset (shared infrastructure with bi-health-score)
• CRM segmentation data (tier, company size, industry, contract age)

Strongly recommended:

• pa-adoption-tracker (for feature-level detail per account)
• bi-health-score (for health context -- do not benchmark unhealthy accounts without noting the health context)

Downstream consumers:

• cc-qbr-deck-builder (benchmark slides for customer presentations)
• pa-adoption-tracker (peer adoption rates for gap analysis)
• pa-value-reporter (benchmark context for value narrative)
• bi-expansion-detector (high benchmark position as expansion readiness signal)

References

• references/benchmark-methodology.md -- Cohort definition, percentile computation, refresh cadence, and customer-safe output rules