evaluate-model-serving-aks — independently scanned and version-tracked by SaferSkills.
SaferSkills independently audited evaluate-model-serving-aks (Agent Skill) and scored it 100/100 (green). The audit ran 55 deterministic rules across Security, Supply Chain, Maintenance, Transparency, and Community; it found 0 high-severity and 0 lower-severity findings. The full rule-by-rule trace and per-finding evidence are below. Free, methodology-open.
Findings & checks · 0 flagged
Every scanned point with the score it earned and what moved between them.
First recorded scan — no prior version to compare against.
The primary manifest — the file an agent reads to learn what this artifact does.
| Metric | Target | How to Measure |
|---|---|---|
| Throughput | ≥ 500 tokens/sec per GPU | Load test with concurrent requests |
| Time to First Token (TTFT) | < 500ms | Request-level timing |
| Tokens Per Second (TPS) | ≥ 30 per request | Streaming response timing |
| GPU utilization | 70-90% under load | nvidia-smi / DCGM metrics |
| Pod restart count | 0 during test | kubectl get pods |
| Scale-up time | < 3 minutes | HPA scaling latency |
| P99 latency | < 5 seconds | Tail latency under load |
| Error rate | < 0.1% | HTTP 5xx / total requests |
# Single request latency
python evaluation/benchmark.py --endpoint $ENDPOINT --requests 1 --concurrent 1
# Measure TTFT and TPS
python evaluation/benchmark.py --endpoint $ENDPOINT --requests 10 --concurrent 1 --metrics ttft,tps# Ramp up to 50 concurrent requests
python evaluation/benchmark.py --endpoint $ENDPOINT \
--requests 500 --concurrent 50 --duration 300s \
--output evaluation/load-test-report.jsonMonitor during test:
# GPU utilization (in another terminal)
kubectl exec -it llm-serving-xxx -- nvidia-smi --loop=5
# Pod metrics
kubectl top pods -l app=llm-servingkubectl delete pod llm-serving-xxx → verify auto-recoverypython evaluation/benchmark.py --endpoint $ENDPOINT --full-report --output evaluation/report.json| Result | Action |
|---|---|
| All PASS | Approve for production traffic |
| Throughput < 300 tok/s | Enable continuous batching, check GPU type |
| TTFT > 1s | Reduce model size or use tensor parallelism |
| GPU util < 50% | Right-size to smaller GPU VM |
| GPU util > 95% | Add replicas or upgrade GPU VM |
| Pod restarts > 0 | Check OOM, increase memory limits |
| Symptom | Root Cause | Fix |
|---|---|---|
| Low throughput | Batching disabled | Enable continuous batching in vLLM config |
| High TTFT | Cold KV cache | Enable prefix caching |
| OOM crashes | VRAM exceeded | Use smaller model or AWQ quantization |
| Uneven load across pods | Sticky sessions | Use round-robin load balancing |
| Slow scale-up | Large container image | Pre-pull images, use init containers |
~30 seconds. Free. No account. Every finding cites a rule and a line of evidence.