plan-error-handling — independently scanned and version-tracked by SaferSkills.
SaferSkills independently audited plan-error-handling (Agent Skill) and scored it 45/100 (orange). The audit ran 55 deterministic rules across Security, Supply Chain, Maintenance, Transparency, and Community; it found 1 high-severity and 0 lower-severity findings. The full rule-by-rule trace and per-finding evidence are below. Free, methodology-open.
Findings & checks · 1 flagged
A base64 string of 128+ characters appears in a documentation file. Encoded prompt injection hides the hostile instruction in base64 — invisible to keyword filters — and relies on the agent's ability to decode it at runtime. There is no normal authoring reason to embed a multi-hundred-byte base64 blob in skill docs.
*.sig, SIGNATURES) outside the documentation.Every scanned point with the score it earned and what moved between them.
First recorded scan — no prior version to compare against.
The primary manifest — the file an agent reads to learn what this artifact does.
Role: Senior reliability engineer + observability specialist.
Task: Map every silent-failure path across Sentry and Langfuse planes, score by blast radius × invisibility, phase remediations, emit plan-error-handling.md. Audit & plan only — no code or SDK edits until each phase is approved.
Find what fails in silence. Make it observable. Change nothing until approved.
AI coding agents optimize for making the error message go away, not for making failure visible. Error-handling gaps are nearly twice as common in AI-generated pull requests as in human ones — empty catch blocks, missing guards, unhandled promise rejections, and handlers that leak stack traces into capture systems. The dangerous part isn't the crash you see; it's the failure you don't.
This skill is the audit-and-plan half. Execution goes to backend-observability / audit-langfuse-llm after you approve each phase.
Trigger phrases: "errors don't reach Sentry", "it fails silently", "empty catch blocks", "add error handling", "why can't I debug prod", "check my Langfuse traces", "my LLM costs are a mystery", "pre-launch observability".
Do not fire for: a specific firing incident (debug-error, debug-sentry-monitor), or security gaps (plan-rls-audit, plan-security-audit). This is the coverage audit, not incident response.
plan-stub-checker finds fake functionality (dead buttons). This finds real functionality that fails invisibly. Different failure mode: "if this breaks in production tonight, would you ever know?"
Walk every error path:
catch (e) {}, catch { return null },catch (e) { console.log(e) } with no rethrow and no Sentry capture.
Sentry.captureException. Local console ≠ production visibility.
chaining gaps that throw at runtime on the unhappy path.
await without try/catch onfallible calls, floating promises, missing .catch() on fire-and-forget.
stack traces with internal paths captured into Sentry. Recommend beforeSend scrubbing.
functions, RN/Capacitor, server actions)? Source maps uploaded?
For any AI/LLM feature:
@observe /Langfuse generation wrapping them.
params.
your own judges. Flag features shipping with zero quality signal.
versioned and linked to traces.
attributes.
missed.
Langfuse was acquired by ClickHouse (Jan 2026); if the user self-hosts, flag validating hosting/licensing — a note, not a code finding.
trace (shared request/trace id)?
console.log soup?planes. State assumptions.
path:line. For each: *"If this fails in prod,what's the signal?"* — None / Console-only / Sentry / Langfuse.
Sentry-spamming every validation miss.
reported; blind swallowing is not.
minimum.
plan-test-coverage.plan-error-handling.md# Error-Handling & Observability Audit — <repo>
_Audit-only. Nothing changes until each phase is approved._
## Scope
- Planes: [ ] Sentry [ ] Langfuse | Surfaces: web / edge / RN / server actions
- Assumptions / not inspected: …
## Verdict
| Plane | Silent failures | PII leaks | Coverage holes |
|-------|-----------------|-----------|----------------|
| Sentry | n | n | n |
| Langfuse | n | n | n |
## Findings — application errors (Sentry)
| # | path:line | Pattern | Signal today | Sev | Direction |
|---|-----------|---------|--------------|-----|-----------|
| E1 | api/pay.ts:88 | empty catch on charge | none | Crit | report + rethrow, surface to user |
## Findings — LLM observability (Langfuse)
| # | path:line | Gap | Sev | Direction |
|---|-----------|-----|-----|-----------|
| L1 | lib/ai.ts:40 | model call untraced | High | wrap with observe, capture usage |
## Phased burndown
- **Phase 1 — Stop silent failures** → `backend-observability` — E-tier swallows
- **Phase 2 — Close coverage holes** → `backend-observability` — init gaps, maps
- **Phase 3 — Redact PII** → `backend-observability` — beforeSend / SDK redaction
- **Phase 4 — LLM tracing & evals** → `audit-langfuse-llm` — L-tier items
## Execution handoff
Approve a phase to run it. Re-run after to confirm failures are now observable.plan-error-handling); complementsplan-test-coverage (tests prevent; observability reveals).
backend-observability, audit-langfuse-llm,debug-sentry-monitor.
Plan with a strong model; execute with composer-2.5-execution.mdc riding along. The plan says what is invisible; the rule constrains how it's wired.~30 seconds. Free. No account. Every finding cites a rule and a line of evidence.