Harness Engineering Audit

harness-engineering-audit is a Codex skill for auditing repositories against OpenAI-style harness engineering and agentic-development readiness.

It is audit-first and strict, but aggressive about safe follow-up work. It inventories a repo, scores the harness-engineering surface, writes report artifacts, and produces an OMX-ready handoff so auto-approved low-risk fixes can move through $ralplan, $team, and $ralph. Root AGENTS.md improvements are P0 when the audit finds missing, stale, oversized, or under-informative instruction surfaces.

What it checks

• AGENTS.md and nested instruction files
• .codex/config.toml, MCP servers, hooks, prompts, and rules
• repo-scoped skills in .agents/skills
• OMX contexts, plans, and workflow docs
• validation commands and package scripts
• stack profile and installed tooling signals
• approval-gated official/high-trust upgrade recommendations
• CI workflows
• docs authority and index coverage
• docs-to-production and visual product spec pack readiness for generated PNG/SVG targets, stories, screen contracts, tokens, component packs, visual QA, and production-readiness deltas
• glossary, terminology, ADR, and domain-language surfaces that keep agent vocabulary consistent
• doc gardening / knowledge-base readiness, including source boundaries, generated/synthesized docs, indexes/logs, and stale/contradiction/orphan checks
• progressive-disclosure guidance that keeps hot-path instructions concise
• deterministic automated checks versus judgment-based automated/human review boundaries
• lifecycle classification and readiness registry coverage for broader harness/Symphony concepts
• lane-pack registry coverage for grounded source-of-truth surfaces, including universal core lanes plus confidence-gated stack-detected UI/UX, backend/API, data, security, performance, infra, AI/ML/CV, docs, and QA lanes
• generated artifact lifecycle
• Symphony orchestration readiness (static signals for workflow contracts, task-state handoffs, workspace isolation, agent runner guidance, observability, validation, and recovery)
• scaffolding / preview / legacy entropy
• cross-agent surfaces such as Claude, Cursor, Gemini CLI, Cline, Windsurf, and Copilot files

Install with GitHub CLI

Install into the current repo as a Codex project skill:

gh skill install ryne2010/harness-engineering-audit \
  skills/harness-engineering-audit \
  --agent codex \
  --scope project

Install a pinned release:

gh skill install ryne2010/harness-engineering-audit \
  skills/[email protected] \
  --agent codex \
  --scope project

Install globally for your user:

gh skill install ryne2010/harness-engineering-audit \
  skills/[email protected] \
  --agent codex \
  --scope user

Download manually

Download the latest GitHub release ZIP, unzip it, and copy the skill into a repo:

mkdir -p .agents/skills
cp -R /path/to/harness-engineering-audit/skills/harness-engineering-audit \
  .agents/skills/harness-engineering-audit

Or install from a local clone:

gh skill install ./skills/harness-engineering-audit --agent codex --scope project

Safe skill updates

Audit runs check the skill's update status by default and include the result in .codex/reports/harness-engineering-audit/update-status.json plus the main report. This is report-only: the skill does not auto-update by default.

To update this one skill globally for your user, prefer the explicit install command:

gh skill install ryne2010/harness-engineering-audit skills/harness-engineering-audit --agent codex --scope user --force

Or use the skill's explicit self-update flag:

python3 ~/.codex/skills/harness-engineering-audit/scripts/run_audit.py . --self-update --update-scope user

After a successful self-update, restart Codex / rerun the audit so the updated files are loaded. Project-scoped installs should generally be updated intentionally through a repo change and PR. Avoid gh skill update --all for this use case because it may touch unrelated system/manual skills and may fail on skills without GitHub metadata.

Run the audit

From a target repo root after installing the skill.

Project-scoped install:

python3 .agents/skills/harness-engineering-audit/scripts/run_audit.py .

User-scoped install:

python3 ~/.codex/skills/harness-engineering-audit/scripts/run_audit.py .

The audit replaces the previous generated report set at:

.codex/reports/harness-engineering-audit/
  inventory.json
  scorecard.json
  stack-inventory.json
  tool-inventory.json
  upgrade-recommendations.json
  upgrade-recommendations.md
  web-verification-queue.json
  source-trust-policy.md
  update-status.json
  report.md
  findings.md
  recommended-fixes.md
  agents-priority.md
  omx-handoff.md
  next-step.md
  next-step.json
  docs-to-production-readiness.md
  premortem.md
  premortem.json
  failure-mode-register.yaml
  harness-gap-matrix.yaml
  capability-adapter-report.yaml
  instruction-trust-report.md
  recommended-omx-prompts/
  prompts/
    deep-interview.md
    ralplan.md
    team.md
    ralph.md
    symphony-adoption.md
    tool-upgrade-ralplan.md

Use the generated handoff with OMX. In an interactive OMX run, the skill should present the generated default next stage from next-step.json so you do not need to copy/paste a long prompt. To resume later, type only:

$harness-engineering-audit continue

Upgrade recommendations are report-only by default: tools are recommended, follow-up web verification is requested, human approval is required, and the audit never executes install/config commands or claims local-script web verification.

The primary artifacts also state the user-visible workflow status: what happened, what did not happen, what needs approval, mode side effects, and the optional approval-gated tool-upgrade branch. next-step.json exposes the same details as mode_summary, approval_state, tool_recommendation_state, and current_step_explanation.

The docs-to-production family now audits visual-product-spec workflows: generated PNG/SVG/Figma targets must be treated as design-intent evidence, translated into screen contracts, mapped to user stories, connected to token/component packs, and validated through workbench, E2E, visual, accessibility, and production-readiness gates.

When the script runs in an interactive terminal without --mode, it prompts for the audit level. The default choice is minimal, which maps to report-only audit; non-interactive automation also defaults to this report-only level.

Explicit setup modes are available when you want the skill to create low-risk harness infrastructure:

AUDIT_SCRIPT=.agents/skills/harness-engineering-audit/scripts/run_audit.py
# For a user-scoped install, use:
# AUDIT_SCRIPT=~/.codex/skills/harness-engineering-audit/scripts/run_audit.py

python3 "$AUDIT_SCRIPT" . --mode safe-setup
python3 "$AUDIT_SCRIPT" . --mode force-ideal-harness
python3 "$AUDIT_SCRIPT" . --mode symphony-repo-local
python3 "$AUDIT_SCRIPT" . --mode symphony-live-handoff
python3 "$AUDIT_SCRIPT" . --mode full-orchestration
python3 "$AUDIT_SCRIPT" . --mode product-harness-setup

Mode side-effect summary:

The default --mode minimal / --mode audit remains report-only. safe-setup creates docs-only lane packs under docs/harness/** and never creates .codex/agents. Stack lanes include activation confidence and recommendation policy fields so weak layout signals can remain advisory instead of inflating missing-lane work. full-orchestration is explicit opt-in for project custom-agent TOML and stronger orchestration contracts; it still does not spawn agents or execute live commands. --force remains only the output-directory overwrite flag.

Manual commands are still generated for portability:

$ralplan "Read .codex/reports/harness-engineering-audit/prompts/ralplan.md and follow it."
$team "Read .codex/reports/harness-engineering-audit/prompts/team.md and follow it."
$ralph "Read .codex/reports/harness-engineering-audit/prompts/ralph.md and follow it."
$deep-interview "Read .codex/reports/harness-engineering-audit/prompts/deep-interview.md and follow it." # medium/high-risk questions only

Optional Codex plugin wrapper

This repo includes an optional plugin wrapper under:

plugins/harness-engineering-audit/

The plugin wrapper is secondary. The recommended install path today is gh skill install. See docs/PLUGIN_DISTRIBUTION.md for the plugin marketplace flow.

Repository layout

harness-engineering-audit/
  README.md
  LICENSE
  CHANGELOG.md
  Makefile
  pyproject.toml
  docs/
    INSTALL.md
    PUBLISHING.md
    PLUGIN_DISTRIBUTION.md
    USAGE.md
    OMX_WORKFLOW.md
    RELEASE_CHECKLIST.md
  skills/
    harness-engineering-audit/
      SKILL.md
      agents/openai.yaml
      release.json
      scripts/*.py
      assets/*
      references/*
  plugins/
    harness-engineering-audit/
      .codex-plugin/plugin.json
      skills/harness-engineering-audit/
  .agents/plugins/marketplace.json
  tests/check_skill_mirror.py
  tests/smoke/run_skill_smoke.py

Validate this repo

make validate

Equivalent commands:

python3 -m py_compile skills/harness-engineering-audit/scripts/*.py scripts/release_skill_workflow.py tests/smoke/run_skill_smoke.py tests/smoke/run_release_workflow_smoke.py tests/check_skill_mirror.py
python3 tests/check_skill_mirror.py
python3 tests/smoke/run_skill_smoke.py
python3 tests/smoke/run_release_workflow_smoke.py

Publish

Normal releases are published by .github/workflows/release-skill.yml after the required main-branch workflow succeeds. The release workflow derives the vX.Y train from skill/project metadata, checks release-relevant changes since the latest matching tag, runs make validate, stamps package-local release.json, validates mirror parity and the package, then publishes the next patch tag. Release decision logic lives in scripts/release_skill_workflow.py and is covered by git-history smoke tests.

Release validation uses smoke fixtures and package dry-run validation. It should not run harness-engineering-audit against this source repo as its own quality signal.

Local validation:

gh skill publish --dry-run

Manual publish should be used only as an intentional fallback:

gh skill publish --tag vX.Y.Z

Full publishing steps are in docs/PUBLISHING.md.

License

MIT