sota-observability — independently scanned and version-tracked by SaferSkills.
SaferSkills independently audited sota-observability (Agent Skill) and scored it 100/100 (green). The audit ran 55 deterministic rules across Security, Supply Chain, Maintenance, Transparency, and Community; it found 0 high-severity and 0 lower-severity findings. The full rule-by-rule trace and per-finding evidence are below. Free, methodology-open.
Findings & checks · 0 flagged
Every scanned point with the score it earned and what moved between them.
First recorded scan — no prior version to compare against.
The primary manifest — the file an agent reads to learn what this artifact does.
Make every production system answerable. Two questions define success:
from a trace ID, without adding new instrumentation.
only when users are hurt, each linked to a runbook and a dashboard that narrows cause in minutes.
This skill covers structured logging, metrics, distributed tracing, SLOs and alerting, and operational readiness — both how to build them correctly and how to audit them adversarially. Telemetry is a product with users (on-call engineers) and costs (storage, cardinality, attention). Treat both.
When writing or modifying code, apply the rules files as design constraints, not afterthoughts. Workflow:
consumer: which SLI does it affect, what one wide event describes a unit of work, what spans bound its external calls.
configure SDK/exporters only at the application entry point. Follow OTel semantic conventions for names and attributes.
trace_id, outcome, durations, and business context. Debug logs are supplementary, sampled, and disposable.
traceparent over HTTP, injectedinto queue message headers, restored in consumers and scheduled jobs.
serializers for PII/secrets; fail closed on unknown object dumps.
value set. No IDs, no URLs, no user input in labels.
correct liveness/readiness semantics, dashboard panels answering the questions the feature raises, burn-rate alerts wired to the SLO, runbook entry for each new alert.
an error — confirm the trace, the wide event, the metric, and the alert all show it, and that they cross-link (exemplars, trace_id in logs).
Assess an existing codebase/deployment. Read rules/06 first for the full playbook; sample real code paths, do not trust README claims.
Severity conventions:
| Severity | Meaning | Examples |
|---|---|---|
| CRITICAL | Blind during incidents, or telemetry is itself a hazard | Secrets/PII in logs; no error visibility at all; liveness check hits the database (restart storms); unauthenticated debug/pprof endpoints |
| HIGH | Materially slows MTTR or breaks at scale | No correlation/trace IDs; unbounded label cardinality; cause-based paging alerts with no runbooks; readiness == liveness; percentiles averaged across instances |
| MEDIUM | Degrades signal quality or cost discipline | Wrong log levels (ERROR for expected events); no exemplars; head-only sampling losing all error traces; dashboards as vanity walls; no log sampling on hot paths |
| LOW | Hygiene and polish | Inconsistent field names; missing OTel semantic conventions; unpinned dashboard queries; noisy Sentry grouping |
Finding format (one per finding):
[SEVERITY] <short title>
Where: <file:line, config path, or dashboard/alert name>
Evidence: <exact code/config snippet or observed behavior>
Impact: <what fails during an incident or at scale, concretely>
Fix: <specific change, with code/config if short>
Effort: <S/M/L>Conclude every audit with the two-question verdict: can on-call currently answer "why is this request slow?" and "what broke at 3am?" — YES/PARTIAL/NO, with the shortest path to YES.
| File | Read this when... |
|---|---|
rules/01-structured-logging.md | Writing or reviewing log statements, choosing levels, designing wide events/canonical log lines, configuring redaction, sampling, or controlling log spend |
rules/02-metrics.md | Adding Prometheus/OTel metrics, choosing counter vs gauge vs histogram, designing labels, computing percentiles, applying RED/USE, linking metrics to traces via exemplars |
rules/03-tracing.md | Instrumenting with OpenTelemetry, deciding what gets a span, propagating context across HTTP/queues/jobs, choosing head vs tail sampling, using (or avoiding) baggage |
rules/04-slos-alerting.md | Defining SLIs/SLOs, error budgets, writing burn-rate alerts, reviewing alert quality, fighting alert fatigue, deciding page vs ticket |
rules/05-operational-readiness.md | Implementing health endpoints, exposing graceful degradation, securing debug endpoints, continuous profiling, Sentry-style error tracking, building dashboards |
rules/06-audit-playbook.md | Auditing a codebase's observability posture end-to-end; common gaps catalog; scoring and reporting |
to a request is gossip, not evidence.
it is WARN or below. Level discipline is alert discipline upstream.
by call-site vigilance. Redaction is infrastructure, not convention.
duration, and business context — the canonical log line you grep at 3am.
text. Cardinality explosions take down the monitoring you need most.
quantiles. A dashboard of avg(p99) is fiction.
traceparent propagated across every HTTP hop, queue, and async job.
Conflating them turns one slow dependency into a cluster-wide restart storm.
user pain (SLO burn rate, multi-window), not on causes (CPU, pod restarts).
(tail-sample to keep errors/slow), review cost monthly, delete signals nobody queries.
~30 seconds. Free. No account. Every finding cites a rule and a line of evidence.