helmdeck-debug — independently scanned and version-tracked by SaferSkills.
SaferSkills independently audited helmdeck-debug (Agent Skill) and scored it 100/100 (green). The audit ran 55 deterministic rules across Security, Supply Chain, Maintenance, Transparency, and Community; it found 0 high-severity and 0 lower-severity findings. The full rule-by-rule trace and per-finding evidence are below. Free, methodology-open.
Findings & checks · 0 flagged
Every scanned point with the score it earned and what moved between them.
First recorded scan — no prior version to compare against.
The primary manifest — the file an agent reads to learn what this artifact does.
<!-- Canonical helmdeck integration-debugger skill. Re-run scripts/configure-openclaw.sh (OpenClaw) or scripts/configure-claude.sh (Claude Code) after any helmdeck release to re-stamp helmdeckVersion and pick up new pipelines/packs. The companion helmdeck skill teaches an agent to USE the packs; this skill teaches an agent to DEBUG them. -->
When invoked, you run a diagnostic sweep across all helmdeck pipelines and capability packs, classify what you find, and produce a findings report with one ready-to-file GitHub issue per real bug. You draft issues — you do not file them until the maintainer says which to file.
Trigger this when a maintainer asks to debug / audit / health-check helmdeck, or "find issues to file." For normal pack usage, that's the separate helmdeck skill.
These are the recurring, real classes (each has bitten this project). Look for all four:
Description promises more thanits steps actually do. Canonical example: builtin.grounded-blog once said "fact-check + rewrite + publish" but content.ground only cites claims (it does not rewrite voice/structure) and blog.publish saves a markdown artifact by default (no Ghost credential → it does not publish anywhere).
without a guard, so a bad value produces bad output instead of an error. Canonical example: a literal {{TITLE}} once published a post titled {{TITLE}}. (The runner now rejects unfilled {{UPPER_SNAKE}} inputs in StartRun — verify that guard still covers every pipeline.)
OutputSchema doesn't matchwhat its handler actually emits (key missing, or wrong type — e.g. declaring number for a field the handler emits as an object). Handler-direct unit tests bypass Engine.Execute, so OutputSchema.Validate never runs on them.
(handler_failed/internal/invalid_output) for what is really caller input. internal/pipelines/classify.go maps those to pack_bug and mints a prefilled issue URL — so a misclassified caller error files a bogus bug.
Three capabilities gate which passes you can run. Probe and state plainly what you will and won't do:
internal/pipelines/seed.go andinternal/packs/builtin/. Required for the static pass (classes 1–4 need to read the definitions — REST does NOT expose pack input/output schemas). No source → say "static pass skipped — no source checkout."
GET {HELMDECK_URL}/healthz (expect{"status":"ok"}) and GET {HELMDECK_URL}/version. Required for the live pass. HELMDECK_URL defaults to http://localhost:3000.
sweep (see the note in Step C).
Enumerate, then check the four classes.
Enumerate:
internal/pipelines/seed.go → Builtins().internal/packs/builtin/*.go (the Name: field of each*packs.Pack).
(GET /api/v1/pipelines, GET /api/v1/packs) and flag any seed ↔ registry drift (a builtin in one but not the other).
Class 1 — oversold descriptions. For each pipeline in seed.go, compare its Description string to the actual step(...) packs and their inputs. Red flags: "rewrite" while a content.ground step is rewrite:false; "publish" while the terminal blog.publish step has no credential/host (it saves an artifact); any verb the steps don't perform. Use the #321 ground/blog wording as the bar for "honest." Do the same for each pack's own Description.
Class 2 — silent-bad-output inputs. For every ${{ inputs.* }} reference in seed.go, confirm the StartRun placeholder guard (internal/pipelines/runs.go, validateInputsFilled / unfilledPlaceholderRE) still applies — it runs for all pipelines centrally, so the check is "does any new input path bypass it?" Flag any input that lands in a user-visible field (title/body/filename) with no validation.
Class 3 — schema vs handler. For each pack *.go, read its OutputSchema declaration and the map its handler actually returns; flag mismatches (missing key, declared type ≠ emitted type). Then flag any Async:true or handler-direct-tested pack that lacks a contract test asserting real output matches the schema (see internal/packs/builtin/output_schema_contract_test.go for the pattern). These are the cases Engine.Execute's validation never sees in unit tests.
Class 4 — misclassification. Read internal/pipelines/classify.go. For any pack that returns CodeHandlerFailed/CodeInternal/CodeInvalidOutput on what is actually caller input (wrong shape, missing field, bad URL), flag it — it would be classified pack_bug and mint a bogus prefilled issue. Corroborate with the live pass: a pipeline that fails pack_bug on the obviously-safe fixture inputs below is a prime class-4 (or genuine class-3) candidate.
REST-direct is the primary path (works anywhere a control-plane is up, judges by the authoritative failure_class).
POST {HELMDECK_URL}/api/v1/auth/login{"username":"<admin>","password":"<pass>"} → {token}. Creds from HELMDECK_USER/HELMDECK_PASS env, or HELMDECK_ADMIN_PASSWORD in deploy/compose/.env.local. Send the token as Authorization: Bearer <token>.
GET /api/v1/pipelines → list of ids (don't hardcode —new/community pipelines auto-covered).
POST /api/v1/pipelines/{id}/run with{"inputs": <minimal-safe inputs>} → run_id. Use the fixture table below; for a pipeline not in it, read its ${{ inputs.* }} refs via GET /api/v1/pipelines/{id} and synthesize a minimal value of the right shape (default strings to the example.com / Hello-World fixtures).
GET /api/v1/pipelines/{id}/runs/{run_id} every ~5s until statusis succeeded or failed (deadline ~900s — narrate/video pipelines take minutes).
succeeded → pass.failed + failure_class == pack_bug → real bug → draft an issue.failed + failure_class ∈ {caller_fixable, transient, state_changed} →"ran, not a bug" — report it but do NOT draft an issue. The wiring worked; the input / environment / outside world was the cause. (This is what keeps a keyless stack — no Firecrawl/ElevenLabs/Ghost/fal — from generating a flood of bogus drafts.)
Minimal-safe pipeline inputs (kept in sync with scripts/validate-openclaw.sh):
| pipeline id | inputs |
|---|---|
builtin.grounded-deck | {"markdown":"# Helmdeck\n\nHelmdeck runs browser automation and capability packs inside sandboxed containers."} |
builtin.grounded-blog | {"markdown":"# Helmdeck\n\nHelmdeck runs browser automation in sandboxed containers.","title":"Helmdeck overview"} |
builtin.research-deck | {"query":"mnemonics memory techniques"} |
builtin.research-narrate | {"query":"mnemonics memory techniques"} |
builtin.research-podcast | {"query":"mnemonics memory techniques"} |
builtin.scrape-ground-blog | {"url":"https://example.com","title":"Example"} |
builtin.research-ground-deck | {"query":"mnemonics memory techniques"} |
builtin.doc-ground-blog | {"source_url":"https://example.com","title":"Example"} |
builtin.scrape-deck | {"url":"https://example.com"} |
builtin.research-blog | {"query":"mnemonics memory techniques","title":"Mnemonics"} |
builtin.repo-presentation | {"repo_url":"https://github.com/octocat/Hello-World.git"} |
builtin.repo-readme-podcast | {"repo_url":"https://github.com/octocat/Hello-World.git"} |
builtin.html-video | {"composition_html":"<html><body><h1>Hello from helmdeck</h1></body></html>"} |
Never run these during a sweep (write ops that create real external resources): github.create_issue, github.post_comment, github.create_release, repo.push, email.send. Test those by hand against throwaway resources.
Cost/time + optional deeper sweep. A full pipeline sweep is 10–20+ minutes and spends LLM/ElevenLabs/fal credits (the narrate/video pipelines are minutes each). Offer a quick pipelines-light mode (skip research-*, *-narrate, repo-presentation, html-video) and warn before a full sweep. When OpenClaw is present and the maintainer wants the per-pack agent-round-trip coverage, shell out to bash scripts/validate-openclaw.sh --pipelines-only (or --pack <name>) and parse its pass/bug tallies — do not reimplement it.
Assemble one markdown report:
static | live), bugclass (1–4), failure_class (live), severity, one-line finding.
to gh issue create, in the same shape classify.go uses:
pack <name> failed (<error_code>) for live pack bugs, or[<pipeline-or-pack>] <one-line> for static findings.
bug (add area/packs or area/pipelines when clear).exact pipeline id + inputs you ran (live), or the file path + the definition/handler mismatch (static).
issues to file (e.g. 'file 1 and 3', or 'file all')."*
Only on explicit confirmation, file the chosen issues — one at a time — via the gh CLI (gh issue create --title … --body … --label bug, repo tosin2013/helmdeck) when present, else the helmdeck__github_create_issue pack with repo: tosin2013/helmdeck. Dedup first: gh issue list --search "<title>" --state open (or github.list_issues) and skip / link rather than duplicate.
internal (a helmdeck bug, not user error).pack_bug on the safe fixture inputs above.pack_bug error (class 4).HELMDECK_*_ENABLED unset) — configuration.caller_fixable/transient/state_changed — the wiring worked.confirmation of which items to file.
land in an issue body.
fixtures; the write-op skip list above is non-negotiable.
~30 seconds. Free. No account. Every finding cites a rule and a line of evidence.