Hiring Scorecard Builder

Unstructured interviews have poor predictive validity and amplify the biases of whoever talks most. A scorecard forces interviewers to decide what matters before meeting the candidate and to gather evidence, not impressions.

Step 1 — Define the Role's Top 4-6 Signals

Start from the first 90-day outcomes, not a list of desired traits. Ask: what does success look like concretely at 30, 60, and 90 days? Then work backward to the signals a candidate would have to demonstrate to make those outcomes likely. Good signals are observable in a conversation. Bad signals are adjectives ('smart', 'passionate', 'culture fit').

Step 2 — Map Signals to Interview Slots

Each interviewer owns 1-2 signals. Never have two interviewers independently assess the same signal with no calibration plan — duplication wastes time and creates conflicting data. Assign technical depth signals to engineers, cross-functional signals to PMs or peers, and leadership/judgment signals to the hiring manager.

Step 3 — Write the Scoring Rubric

For each signal, write what a 1 (does not meet bar), 3 (meets bar), and 5 (exceeds bar) looks like in observable terms. Example for 'navigates ambiguity': a 1 asks for requirements to be fully specified before acting; a 3 defines scope, makes assumptions explicit, and validates them; a 5 proactively creates structure where none exists and coaches others through it. The rubric must be written before interviews start.

Step 4 — Require Evidence, Not Ratings

The scorecard submission form has two fields per signal: a numeric score AND a required evidence quote from the candidate. No evidence, no score. This prevents post-hoc rationalization and makes debrief discussions concrete.

Step 5 — Run a Calibrated Debrief

Collect scores independently before the debrief meeting. Start by reading scores aloud before anyone speaks. Then discuss only the signals where scores diverge by 2 or more points — those are the interesting ones. The hiring manager does not share their view first; they synthesize last.

Escape Hatches

For a high-volume pipeline, reduce to 3 signals and a 3-point scale. For a senior or staff role, add a written exercise component and score the artifact, not the person's delivery style.