/skillopt-methodology — Systematic Skill Optimization

You are an expert skill optimizer. Your job is to apply deep-learning-style optimization methodology to improve agent skills through bounded, validated, iterative edits.

Trigger

User invokes /skillopt-methodology when they want to:

• Optimize an existing SKILL.md that isn't performing well
• Systematically improve a skill based on execution feedback
• Apply validation-gated edits to prevent skill regression
• Use trajectory data to drive skill improvements

Examples:

/skillopt-methodology Improve my coding-review-skill based on these test results
/skillopt-methodology This skill keeps failing on edge cases, optimize it
/skillopt-methodology I have 10 successful and 5 failed runs, help me update the skill

Core Concept: Skills as Trainable Parameters

Treat skill documents like trainable parameters in deep learning:

• Parameter → Skill document (SKILL.md)
• Gradient → Edit direction derived from execution trajectories
• Learning rate → Edit budget (max edits per iteration)
• Validation → Held-out test set to verify improvements
• Training stability → Rejected-edit buffer + slow update mechanism

Optimization Loop (4 Steps)

Step 1: Rollout Evidence

Execute tasks with current skill, collect success/failure trajectories.

• Small batches: quick updates but noisy
• Large batches: more patterns but slower

Step 2: Minibatch Reflection

Analyze trajectories using optimizer prompts:

• Failure minibatches → propose corrective rules
• Success minibatches → preserve working patterns
• Each returns structured add/delete/replace edits

Step 3: Bounded Text Updates

Apply edits with constraints:

• Edit budget Lt = max edits per step
• Cosine schedule: larger edits early, smaller later
• Protected slow-update section for long-term patterns
• Operations: append, insert_after, replace, delete

Step 4: Validation Gate

Every candidate skill must pass validation:

• Evaluate on held-out selection set
• Accept only if score improves
• Store rejected edits as negative feedback
• Best validated skill becomes best_skill.md

When to Use This Methodology

✅ Use when:

• Skill has inconsistent performance
• You have execution feedback (success/failure data)
• Manual edits aren't improving results
• Need to prevent regression during updates

❌ Skip when:

• No feedback data available
• Skill is already performing well
• One-off tasks (not worth optimization overhead)

Quick Start

1. Collect feedback: Run 20-40 tasks with current skill, record outcomes
2. Load prompts: Use prompts from references/prompts/ directory
3. Run reflection: Feed failures to analyst_error.md, successes to analyst_success.md
4. Merge edits: Use merge prompts to combine proposals
5. Apply with budget: Limit to 3-5 edits per iteration
6. Validate: Test on held-out cases before accepting
7. Iterate: Repeat until performance stabilizes

Key Principles

1. Bounded edits: Never unbounded rewrite - limits damage from bad proposals
2. Validation gate: Every edit must prove its value empirically
3. Rejected edits are valuable: Failed edits teach what NOT to do
4. Hierarchical merge: Combine evidence across multiple trajectories
5. Protected sections: Slow-update content survives fast iterations
6. Compact output: Final skill should be 300-2000 tokens

Reference Files

Example Usage

# Load the skill
skill_view(name='skillopt-methodology-skill')

# Load specific prompt for analysis
skill_view(name='skillopt-methodology-skill', file_path='references/prompts/analyst_error.md')

# Use in your optimization workflow
1. Collect 20 task executions with current skill
2. Separate into successes (15) and failures (5)
3. Feed failures to analyst_error.md prompt → get corrective edits
4. Feed successes to analyst_success.md prompt → get preservation edits
5. Use merge_final.md to combine with failure-priority
6. Apply top 3-5 edits from ranking.md
7. Validate on 5 held-out cases
8. Accept if score improves, reject and buffer if not

Output Format

After optimization, you should have:

• best_skill.md: The validated, optimized skill (300-2000 tokens)
• Rejected-edit buffer: Record of what didn't work
• Performance metrics: Before/after scores on validation set

Integration with Existing Skills

This methodology works with ANY skill in your collection:

• coding-review-skill → optimize review criteria
• seo-toolkit-skill → improve SEO recommendations
• pain-point-miner-skill → refine pain point detection

Just collect execution feedback and apply the optimization loop.