Context Compression Strategies

When agent sessions generate millions of tokens of conversation history, compression becomes mandatory. The naive approach is aggressive compression to minimize tokens per request. The correct optimization target is tokens per task: total tokens consumed to complete a task, including re-fetching costs when compression loses critical information.

When to Use / Not Use

Use when:

• Agent sessions exceed context window limits
• Codebases exceed context windows (5M+ token systems)
• Designing conversation summarization strategies
• Debugging cases where agents "forget" what files they modified
• Building evaluation frameworks for compression quality

Do NOT use when:

• Diagnosing why context is degrading -> use context-degradation
• KV-cache optimization or context partitioning -> use context-optimization
• Learning foundational context theory -> use context-fundamentals
• File-based offloading or scratch pads -> use filesystem-context

Decision Tree

Why do you need compression?
├── Agent sessions hitting context limits
│   ├── What matters most?
│   │   ├── File tracking + decision history (long sessions) -> Anchored Iterative Summarization
│   │   ├── Maximum token savings (short sessions, low re-fetch cost) -> Opaque Compression
│   │   └── Readability + phase boundaries -> Regenerative Full Summary
│   └── Not sure? -> Start with Anchored Iterative (best quality trade-off)
├── Need to measure if compression is working
│   └── Probe-based evaluation (§Probe-Based Evaluation)
├── When to trigger compression?
│   └── See §Compression Trigger Strategies
└── Not about reducing size? -> See related skills

Core Compression Approaches

Key insight: Structure forces preservation. Dedicated summary sections act as checklists the summarizer must populate, preventing silent information drift.

Anchored Iterative Summarization

1. Define explicit summary sections matching your agent's needs
2. On first compression trigger, summarize truncated history into sections
3. On subsequent compressions, summarize only new truncated content
4. Merge new summary into existing sections (not full regeneration)
5. Track which information came from which compression cycle

When to Use Each Approach

Structured Summary Sections

## Session Intent
[What the user is trying to accomplish]

## Files Modified
- auth.controller.ts: Fixed JWT token generation
- config/redis.ts: Updated connection pooling
- tests/auth.test.ts: Added mock setup for new config

## Decisions Made
- Using Redis connection pool instead of per-request connections
- Retry logic with exponential backoff for transient failures

## Current State
- 14 tests passing, 2 failing
- Remaining: mock setup for session service tests

## Next Steps
1. Fix remaining test failures
2. Run full test suite
3. Update documentation

Compression Trigger Strategies

Recommendation: Sliding window + structured summaries for most coding agent use cases.

The Artifact Trail Problem

Artifact trail integrity is the weakest dimension across all compression methods (2.2-2.5/5.0 in evaluations). Coding agents need to track:

• Which files were created, modified, or read-only
• Function names, variable names, error messages

Solution: Supplement structured summaries with a separate artifact index or dedicated file-state tracking in agent scaffolding.

Tokens-Per-Task vs Tokens-Per-Request

Traditional metrics target tokens-per-request. This is the wrong optimization. When compression loses critical details, the agent must re-fetch information, re-explore approaches, and waste tokens recovering context.

The right metric is tokens-per-task: total tokens consumed from task start to completion. A compression strategy saving 0.5% more tokens per request but causing 20% more re-fetching costs more overall.

Probe-Based Evaluation

Traditional metrics (ROUGE, embedding similarity) fail for functional quality. Probe-based evaluation directly measures functional quality by asking questions after compression:

Six Evaluation Dimensions

1. Accuracy: Are technical details correct? (largest variation between methods)
2. Context Awareness: Does response reflect current conversation state?
3. Artifact Trail: Does agent know which files were read or modified? (universally weak)
4. Completeness: Does response address all parts of the question?
5. Continuity: Can work continue without re-fetching?
6. Instruction Following: Does response respect stated constraints?

Anti-Patterns

Guidelines

1. Optimize for tokens-per-task, not tokens-per-request
2. Use structured summaries with explicit sections for file tracking
3. Trigger compression at 70-80% context utilization
4. Implement incremental merging rather than full regeneration
5. Test compression quality with probe-based evaluation
6. Track artifact trail separately if file tracking is critical
7. Accept slightly lower compression ratios for better quality retention
8. Monitor re-fetching frequency as a compression quality signal

Integration

• context-degradation - Compression is a mitigation strategy for degradation
• context-optimization - Compression is one optimization technique among many
• memory-systems - Compression relates to scratchpad and summary memory patterns

References

Internal reference:

• Evaluation Framework Reference - Detailed probe types and scoring rubrics

External resources:

• Factory Research: Evaluating Context Compression for AI Agents (December 2025)
• Research on LLM-as-judge evaluation methodology (Zheng et al., 2023)