define-variables — independently scanned and version-tracked by SaferSkills.
SaferSkills independently audited define-variables (Agent Skill) and scored it 91/100 (green). The audit ran 55 deterministic rules across Security, Supply Chain, Maintenance, Transparency, and Community; it found 1 high-severity and 0 lower-severity findings. The full rule-by-rule trace and per-finding evidence are below. Free, methodology-open.
Findings & checks · 1 flagged
The text {match} is the classic direct prompt-injection phrasing. Placed in a skill body that the agent reads as trusted instructions, it tries to make the agent abandon its prior rules and follow whatever comes next — a full system-prompt override.
ignore/disregard/forget … previous instructions sentence.Every scanned point with the score it earned and what moved between them.
First recorded scan — no prior version to compare against.
The primary manifest — the file an agent reads to learn what this artifact does.
Map each exposure, outcome, covariate, and eligibility variable to a canonical guideline/consensus definition, cross-check it against prior operationalizations in comparable cohorts, then map it to the available DB variables. Call after /design-study (and /search-lit), before /write-protocol.
Missing inputs → ask once, then proceed.
Trigger: project has a project.yaml::db.dictionary_path field pointing to a machine-readable codebook (xlsx/csv/markdown), OR the user supplied a dictionary path in inputs. If neither, skip to Tier 1.
For every candidate DB variable — before touching literature — open the dictionary and record, verbatim, the sheet name, row number, and code→meaning mapping. This prevents the most common observational-study error: assuming a column code (status == 0, grade == 4) means what it intuitively reads like, when the codebook says otherwise.
Per variable:
Dict. sheet & row + Dict. verbatim columns of the operationalization table.Empirical checks (value distributions, cross-tabs with related columns) are useful for sanity testing after the verbatim codebook meaning is recorded — never as a substitute for it.
Recommend committing a DICTIONARY_FIRST_POLICY.md at the project root (or shared-config path) with the canonical dictionary path and the escalation contact.
Exit gate: before Tier 1, cross-check every row's Dict. sheet & row and Dict. verbatim against the source dictionary; no DB-backed row may be left blank.
Look the variable up in references/common_definitions.md (hepatology, metabolic/endocrine, renal, pulmonary, cardiovascular, oncology/imaging incidentalomas, alcohol exposure). On a hit, record the guideline, year, canonical cutoff, and BibTeX key. Done — no /search-lit call.
/search-lit (focused queries only)For variables NOT in Tier 1, OR when subgroup justification is needed (Asian-specific cutoff, pediatric, young-adult, pregnancy, etc.), call /search-lit with one query per variable — never a general sweep, which buries the signal. Query pattern:
"{construct} definition {cohort type} {subgroup qualifier}"
e.g., "obstructive sleep apnea prevalence Korean health screening cohort"Stop searching a variable once the first 1-2 papers converge on the same definition. If more than five variables need Tier 2, list them and confirm with the user before running the rest.
Every definition, cutoff, and era anchor must come from a verified source — a clinical guideline, a peer-reviewed paper with DOI, or an established registry data dictionary. Never take a phenotype threshold from the model's prior or a reference from memory. Before finalizing, run /verify-refs on the accumulated BibTeX to confirm every citation exists in PubMed/CrossRef. A choice with no canonical source is flagged Ad-hoc: yes, justified in 1-2 sentences, and confirmed by the user before it propagates into /write-protocol or /analyze-stats.
Write {project_root}/variable_operationalization.md (or the path the user specifies) from templates/variable_operationalization.md. Required structure:
| Variable | Role | Dict. sheet & row | Dict. verbatim | Canonical source | Definition | Cutoff | DB vars | Implementation | Ad-hoc? |
Role: exposure / outcome / covariate / eligibilityDict. sheet & row: e.g. 5-1.복부초음파 r12 — mandatory if a DB dictionary existsDict. verbatim: full code→meaning string copied from the dictionary — mandatory under the same conditionCanonical source: BibTeX key (e.g., @rinella2023_aasld_masld), so downstream skills can re-verifyDefinition: one line, verbatim from the guideline where possibleCutoff: numeric + unitsDB vars: exact dictionary column names usedImplementation: SQL/pandas-style pseudocode (e.g., bmi>=25 & (b_tg>=150 | b_hdl<40))Ad-hoc?: yes/no. If yes, justification below the tableOut of scope: statistical analysis → /analyze-stats; manuscript drafting → /write-paper; data cleaning / missingness → /clean-data; sample size → /calc-sample-size.
Role = covariate and Implementation = "IF status == 'never' THEN dose = 0 ELSE measured_value" — and adjust on the categorical status variable, reserving the continuous dose for an exposed-only secondary analysis. /clean-data (categorical-implied-zero flag) and /analyze-stats ("Covariate Pitfalls") enforce this downstream.~30 seconds. Free. No account. Every finding cites a rule and a line of evidence.