evaluate-call-center-voice-ai — independently scanned and version-tracked by SaferSkills.
SaferSkills independently audited evaluate-call-center-voice-ai (Agent Skill) and scored it 100/100 (green). The audit ran 55 deterministic rules across Security, Supply Chain, Maintenance, Transparency, and Community; it found 0 high-severity and 0 lower-severity findings. The full rule-by-rule trace and per-finding evidence are below. Free, methodology-open.
Findings & checks · 0 flagged
Every scanned point with the score it earned and what moved between them.
First recorded scan — no prior version to compare against.
The primary manifest — the file an agent reads to learn what this artifact does.
| Metric | Target | How to Measure |
|---|---|---|
| Intent recognition accuracy | ≥ 95% | Test set of labeled utterances |
| Response latency (STT+LLM+TTS) | < 2 seconds | End-to-end timing |
| STT word error rate (WER) | < 10% | Reference transcript comparison |
| Call resolution rate | ≥ 70% | Scenario completion tracking |
| Escalation to human | < 30% | Fallback trigger rate |
| CSAT prediction | ≥ 4.0/5.0 | Post-call sentiment analysis |
| Barge-in success | ≥ 90% | Interrupt detection tests |
| Content safety pass rate | 100% | Content filter evaluation |
Create test cases in evaluation/test-set.jsonl:
{"id": "tc001", "audio": "samples/greeting.wav", "expected_intent": "greeting", "expected_response_contains": "help", "category": "normal"}
{"id": "tc002", "audio": "samples/billing.wav", "expected_intent": "billing_inquiry", "expected_response_contains": "account", "category": "normal"}
{"id": "tc003", "audio": "samples/angry.wav", "expected_intent": "escalation", "expected_response_contains": "transfer", "category": "edge"}
{"id": "tc004", "audio": "samples/background_noise.wav", "expected_intent": "greeting", "expected_response_contains": "help", "category": "adversarial"}Minimum: 20 test cases across normal, edge, and adversarial scenarios.
python evaluation/eval.py --test-set evaluation/test-set.jsonl --metrics intent_accuracypython evaluation/eval.py --test-set evaluation/test-set.jsonl --metrics latencyBreakdown target:
Run scenario simulations:
python evaluation/eval.py --test-set evaluation/test-set.jsonl --metrics resolutionpython evaluation/eval.py --test-set evaluation/test-set.jsonl --metrics safetypython evaluation/eval.py --test-set evaluation/test-set.jsonl --output evaluation/report.json --ci-gate| Result | Action |
|---|---|
| All metrics PASS | Deploy to production |
| Latency WARN (2-3s) | Optimize model routing, enable caching |
| Intent accuracy < 90% | Retrain intent classifier, expand training data |
| Resolution < 60% | Add more intents, improve response templates |
| Safety FAIL | Block deployment, fix content filters |
| Symptom | Root Cause | Fix |
|---|---|---|
| Low intent accuracy on noisy audio | No noise robustness training | Add noisy samples to training set |
| High tail latency (p99 > 5s) | Cold start on Speech Service | Implement warm-up pings |
| CSAT drops after deployment | Response quality regression | Compare eval metrics before/after |
| Barge-in fails intermittently | VAD threshold too high | Lower silence detection threshold |
| Wrong language detected | Missing language hints | Set expected language in STT config |
~30 seconds. Free. No account. Every finding cites a rule and a line of evidence.