tune-prompt-management — independently scanned and version-tracked by SaferSkills.
SaferSkills independently audited tune-prompt-management (Agent Skill) and scored it 100/100 (green). The audit ran 55 deterministic rules across Security, Supply Chain, Maintenance, Transparency, and Community; it found 0 high-severity and 0 lower-severity findings. The full rule-by-rule trace and per-finding evidence are below. Free, methodology-open.
Findings & checks · 0 flagged
Every scanned point with the score it earned and what moved between them.
First recorded scan — no prior version to compare against.
The primary manifest — the file an agent reads to learn what this artifact does.
| Technique | Token Reduction | Quality Impact | Complexity |
|---|---|---|---|
| Remove redundant instructions | 10-20% | None | Low |
| Shorten few-shot examples | 15-30% | Low (1-2%) | Low |
| Use structured output format | 10-15% | None | Medium |
| Replace prose with bullet points | 20-30% | None | Low |
| Merge overlapping directives | 10-15% | None | Low |
| Remove obvious instructions | 5-10% | None | Low |
Token budget framework:
| Component | Budget % | Example (4096 window) |
|---|---|---|
| System prompt | ≤ 15% | ~600 tokens max |
| Few-shot examples | ≤ 10% | ~400 tokens |
| Retrieved context | ≤ 50% | ~2000 tokens |
| User query | ≤ 5% | ~200 tokens |
| Response space | ≥ 20% | ~800 tokens |
| Parameter | Default | Range | Impact |
|---|---|---|---|
| Traffic split | 50/50 | 10/90 to 50/50 | Lower variant % = safer rollout |
| Min samples | 1000 | 500-5000 | More = higher confidence |
| Duration | 7 days | 3-30 days | Longer = captures weekly patterns |
| Significance | p < 0.05 | 0.01-0.10 | Lower = more confident winner |
| Primary metric | Quality score | Quality/Latency/Cost | Choose most important |
A/B test lifecycle:
| Strategy | Method | Best For |
|---|---|---|
| Random | Pick N random examples | General-purpose prompts |
| Diversity | Select across categories | Classification prompts |
| Similarity | Closest to user query (dynamic) | RAG/QA prompts |
| Difficulty | Include edge cases | Complex reasoning prompts |
| Recency | Most recent examples | Evolving domains |
Dynamic few-shot (most effective):
| Before | After | Tokens Saved |
|---|---|---|
| "You are a helpful AI assistant that helps users with..." | "You help users with..." | ~10 tokens |
| "Please ensure that you always..." | "Always..." | ~5 tokens |
| "It is important to note that..." | [delete — obvious] | ~8 tokens |
| 3 repetitive few-shot examples | 1 diverse example | ~200 tokens |
| Prose paragraph of rules | Bullet list of rules | ~30% |
Compression checklist:
| Strategy | Cache Scope | TTL | Best For |
|---|---|---|---|
| Full prompt cache | Rendered template + variables | 5 min | Repeated queries |
| Template cache | Template only (variables injected at runtime) | 1 hour | High-volume apps |
| Few-shot cache | Pre-computed few-shot embeddings | 24 hours | Dynamic few-shot |
| No cache | Always fetch from registry | — | Rapidly evolving prompts |
After tuning, compare:
~30 seconds. Free. No account. Every finding cites a rule and a line of evidence.