tune-model-serving-aks — independently scanned and version-tracked by SaferSkills.
SaferSkills independently audited tune-model-serving-aks (Agent Skill) and scored it 100/100 (green). The audit ran 55 deterministic rules across Security, Supply Chain, Maintenance, Transparency, and Community; it found 0 high-severity and 0 lower-severity findings. The full rule-by-rule trace and per-finding evidence are below. Free, methodology-open.
Findings & checks · 0 flagged
Every scanned point with the score it earned and what moved between them.
First recorded scan — no prior version to compare against.
The primary manifest — the file an agent reads to learn what this artifact does.
| Method | VRAM Reduction | Quality Loss | Speed | Best For |
|---|---|---|---|---|
| None (FP16) | Baseline | 0% | 1x | Max quality, enough VRAM |
| GPTQ (4-bit) | ~75% | 1-3% | 1.2-1.5x | Most production workloads |
| AWQ (4-bit) | ~75% | 1-2% | 1.3-1.6x | Balanced quality/speed |
| GGUF (Q4_K_M) | ~75% | 2-4% | 1.1-1.3x | CPU/mixed inference |
| bitsandbytes (8-bit) | ~50% | <1% | 0.9x | Quality-sensitive, limited VRAM |
Decision rule: Start with AWQ 4-bit. If quality drops >3% on eval set, switch to 8-bit or FP16.
| Parameter | Default | Range | Impact |
|---|---|---|---|
max-model-len | Auto | 2048-32768 | Lower = more concurrent requests |
gpu-memory-utilization | 0.9 | 0.7-0.95 | Higher = more KV cache, fewer OOMs |
max-num-batched-tokens | Auto | 2048-8192 | Higher = more throughput, more latency |
max-num-seqs | 256 | 16-512 | Concurrent sequences in batch |
enable-prefix-caching | False | True/False | True = faster TTFT for repeated prefixes |
tensor-parallel-size | 1 | 1-8 | Multi-GPU parallelism |
| Strategy | Configuration | When to Use |
|---|---|---|
| HPA on GPU util | Scale at 80% GPU | General-purpose serving |
| HPA on queue depth | Scale at 10 pending | Bursty workloads |
| KEDA on custom metric | Scale on tokens/sec | Fine-grained control |
| Scheduled scaling | Pre-scale at peak hours | Predictable traffic patterns |
| Cluster autoscaler | Add nodes when pods pending | Node-level elasticity |
HPA configuration:
| Aspect | Regular | Spot | Savings |
|---|---|---|---|
| Price | 100% | 10-40% of regular | 60-90% |
| Eviction | Never | Possible (30s notice) | — |
| SLA | 99.95% | None | — |
| Use case | Production serving | Batch inference, dev/test | — |
Spot strategy:
Cost breakdown (A100 80GB, ~$3.67/hr):
| Component | Calculation | Cost |
|---|---|---|
| GPU compute | $3.67/hr ÷ 500 tok/s = | ~$0.000002/token |
| Cluster mgmt | ~$0.10/hr (AKS free tier) | Negligible |
| Networking | ~$0.01/GB egress | ~$0.0001/request |
| Storage (model) | ~$0.018/GB/month | ~$50/month (per model) |
| ACR | ~$5/month (Basic) | Fixed |
Monthly estimate (10M tokens/day):
Optimization strategies:
After tuning, compare before/after:
~30 seconds. Free. No account. Every finding cites a rule and a line of evidence.