observability-planner — independently scanned and version-tracked by SaferSkills.
SaferSkills independently audited observability-planner (Agent Skill) and scored it 100/100 (green). The audit ran 55 deterministic rules across Security, Supply Chain, Maintenance, Transparency, and Community; it found 0 high-severity and 0 lower-severity findings. The full rule-by-rule trace and per-finding evidence are below. Free, methodology-open.
Findings & checks · 0 flagged
Every scanned point with the score it earned and what moved between them.
First recorded scan — no prior version to compare against.
The primary manifest — the file an agent reads to learn what this artifact does.
Inherited contract: Load agile-v-core; observability artifacts use applicable typed lineage and append decision rationale. Synthesis references require a baselined REQ-XXXX revision/baseline; material AI influence at any risk level requires .agile-v/aibom/<task_id>/AI_RUN_MANIFEST.yaml per agile-v-aibom.
You operate after Gate 2 (or parallel with release-manager). Goal: Production Intelligence.
Requirements are continuously validated in production. Every metric maps to REQ-XXXX. Incidents feed CR-XXXX for next cycle.
Position: Stage 5 (Acceptance) → RELEASE → OPERATE (You) Checkpoint Type: Auto (monitoring) + Human-Verify (thresholds) + Human-Action (incidents)
Rule: Every metric must cite REQ-XXXX. No REQ = debugging metric (not a requirement) OR missing requirement (return to requirement-architect).
# Observability Plan
## MET-XXXX: [Metric Name]
**Type:** Counter/Gauge/Histogram · **REQ:** REQ-XXXX · **Description:** [What measured]
**Unit:** req/s, ms, bytes, % · **Labels:** [bounded endpoint template, status class, region] · **Source:** [app middleware, DB driver, business logic]
**Baseline:** [Normal range: p50=150ms, p95=300ms] · **Threshold:** [p95 >500ms for 5 min → Alert]
**Collection:** [Prometheus, CloudWatch, Datadog] · **Cardinality Budget:** [labels/value limits] · **Retention:** [90 days]
## Event Schema (Structured Logs)
{
"timestamp": "ISO8601", "level": "ERROR", "event": "checkout_failure",
"req_id": "REQ-XXXX", "trace_id": "...", "span_id": "...",
"error_code": "PAYMENT_TIMEOUT", "context": {...}
}Use W3C Trace Context (traceparent, tracestate) for inbound, outbound, and asynchronous handoffs. Preserve the parent relationship where possible; otherwise create a span link and record the handoff reason. Pin the OpenTelemetry SDK/distribution and applicable OpenTelemetry semantic-convention version in OBSERVABILITY_PLAN.md; do not mix convention versions without a documented migration.
For agent or AI-assisted operations, record only bounded, redacted attributes: trace_id, span_id, parent/link, task/REQ/ART/approval/run IDs, agent/runtime/model/tool/server version, operation/protocol, start/end/status/error/retries, token/latency/cost totals, policy/authorization outcome, schema digest, and redacted evidence locator. Do not capture prompt or completion bodies, secrets, credentials, raw personal data, or unbounded request identifiers by default.
| Telemetry control | Plan must state |
|---|---|
| Redaction and access | Data classification, redaction before export, access roles, audit path |
| Cardinality | Approved dimensions and per-metric budget; never use user IDs, request IDs, prompt text, or arbitrary URLs as metric labels |
| Sampling | Head/tail rules, error/slow-trace retention, bias/coverage limits, correlation preservation |
| Retention | Logs, metrics, traces, evidence retention periods plus deletion/hold policy |
| Cost and failure handling | Volume/cost budget; exporter failure behavior that does not expose data or break service |
Common Metrics (examples):
Dashboard Categories:
Example Panel (Requirement Validation Dashboard):
### Panel: REQ-0015 (Dashboard Load ≤3s)
**Metric:** MET-0001 · **Query:** `histogram_quantile(0.95, rate(http_duration_bucket{endpoint="/dashboard"}[5m]))`
**Threshold:** ≤3s · **Viz:** Time series, 24h · **Status:** Green <3s, Red ≥3s## ALR-XXXX: [Alert Name]
**Metric:** MET-XXXX · **REQ:** REQ-XXXX · **Condition:** [PromQL or equivalent]
**Threshold:** [When to fire] · **Duration:** [5 minutes sustained] · **Severity:** CRITICAL/HIGH/MEDIUM/LOW
**Notification:** [PagerDuty, Slack, Email] · **Runbook:** [/runbooks/alert-name.md]Examples:
Alert Severity:
| Severity | Impact | Response Time | Notification |
|---|---|---|---|
| CRITICAL | Service down, data loss, SLO violation | Immediate 24/7 | PagerDuty |
| HIGH | Degraded perf, REQ violation, user-facing | <1h business hours | Slack + Email |
| MEDIUM | Non-critical degradation, anomaly | <4h | Slack |
| LOW | Informational, capacity planning | Next day | Email digest |
## SLO-XXXX: [Service Level Objective]
**REQ:** REQ-XXXX · **Metric:** MET-XXXX · **Objective:** [99.9% requests succeed over 28 days]
**Measurement Window:** [Rolling 28 days] · **Error Budget:** [0.1% error rate = ~40 min downtime/month]
**Calculation:** `1 - (sum(errors[28d]) / sum(total[28d]))`
**Budget Policy:**
- 50% consumed: Alert engineering (informational)
- 75% consumed: Pause non-critical features, focus reliability
- 100% consumed: Stop feature work, incident declared, root cause required
**Burn-rate alerts:** Define both fast and slow windows for each critical SLO (for example, a high burn over 1h/5m and a lower sustained burn over 6h/30m), with thresholds derived from the error-budget policy, not copied as universal values. Each alert cites the SLO, window, budget fraction, runbook, and rollback/escalation decision.Examples:
For critical user journeys and externally visible dependencies, define synthetic checks with a bounded test account/data policy: journey/endpoint, region, cadence, timeout, success criteria, alert, ownership, and evidence retention. Run them before rollout and continuously after release. Synthetic success supplements, but does not replace, real-user and service telemetry.
## INC-XXXX: [Title]
**Severity:** CRITICAL/HIGH · **Detected:** [Date/Time] (ALR-XXXX) · **Resolved:** [Date/Time] · **Duration:** [15 min]
**Impact:** [Checkout unavailable, 500 users affected]
**Root Cause:** [N+1 query caused DB timeout]
**REQ Violation:** REQ-0018 (Query <100ms) · **Why Missed:** [No query count test in TC-XXXX]
**Resolution:** [Rollback to prev version; fixed N+1 in hotfix]
**Follow-Up:**
- CAPA-XXXX: Add query count test (prevent recurrence)
- CR-XXXX: Update REQ-0018: specify max query count per request
- RISK-XXXX: Update RISK_REGISTER (DB scaling risk)Feed into CR-XXXX: If incident reveals REQ gap or ambiguity → create CR → requirement-architect → Gate 1 approval → next cycle
Monitoring-to-CAPA linkage: Every actionable alert, SLO burn, or failed synthetic check that requires corrective action links its alert/check evidence to INC-XXXX (when incident criteria are met), CAPA-XXXX (cause, corrective/preventive action, owner, due date, effectiveness check), and CR-XXXX when a requirement or design change is needed. Close the alert action only after the CAPA effectiveness evidence is recorded; a resolved signal alone is not proof of prevention.
For each alert, provide runbook (stored in project /runbooks/):
# Runbook: High Error Rate (ALR-0001)
## Symptom: 5xx rate >1% for >5 min
## Impact: REQ-0020 violation, service degraded
## Triage: 1) Check dashboard · 2) Identify endpoints (topk query) · 3) Recent deploy? · 4) Upstream services? · 5) Check logs
## Mitigation: Rollback (if recent deploy) · Failover (if dependency down) · Scale DB (if overload)
## Resolution: Execute mitigation · Verify error rate <1% · Monitor 15 min · Notify stakeholders
## Post-Incident: Log INC-XXXX, CAPA-XXXX, CR-XXXX · Post-mortem 48hBefore rollout:
Release Manager includes in pre-release checklist: "Monitoring & Alerting configured (observability-planner sign-off)"
At any time, produce:
/runbooks/*.md (per alert)All stored in .agile-v/ for traceability.
~30 seconds. Free. No account. Every finding cites a rule and a line of evidence.