fai-tune-10-content-moderation — independently scanned and version-tracked by SaferSkills.
SaferSkills independently audited fai-tune-10-content-moderation (Agent Skill) and scored it 82/100 (green). The audit ran 55 deterministic rules across Security, Supply Chain, Maintenance, Transparency, and Community; it found 2 high-severity and 0 lower-severity findings. The full rule-by-rule trace and per-finding evidence are below. Free, methodology-open.
Findings & checks · 2 flagged
The text {match} is the classic direct prompt-injection phrasing. Placed in a skill body that the agent reads as trusted instructions, it tries to make the agent abandon its prior rules and follow whatever comes next — a full system-prompt override.
ignore/disregard/forget … previous instructions sentence.The text {match} is the classic direct prompt-injection phrasing. Placed in a skill body that the agent reads as trusted instructions, it tries to make the agent abandon its prior rules and follow whatever comes next — a full system-prompt override.
ignore/disregard/forget … previous instructions sentence.Every scanned point with the score it earned and what moved between them.
First recorded scan — no prior version to compare against.
The primary manifest — the file an agent reads to learn what this artifact does.
solution-plays/10-content-moderation/config/
├── categories.json # Content category severity thresholds
├── blocklists.json # Custom blocklist configuration
├── prompt-shields.json # Prompt injection detection settings
├── review.json # Human review workflow config
└── guardrails.json # Quality and safety thresholds// config/categories.json
{
"categories": {
"Hate": { "severity_threshold": 2, "action": "block" },
"Violence": { "severity_threshold": 2, "action": "block" },
"Sexual": { "severity_threshold": 2, "action": "block" },
"SelfHarm": { "severity_threshold": 2, "action": "block" }
},
"severity_levels": {
"0": "safe",
"2": "low — may need review",
"4": "medium — likely harmful",
"6": "high — definitely harmful"
},
"default_action": "block",
"log_all_detections": true
}Severity tuning:
| Setting | Conservative | Balanced | Permissive |
|---|---|---|---|
| Hate threshold | 0 (block all) | 2 | 4 |
| Violence threshold | 0 | 2 | 4 |
| Sexual threshold | 0 | 2 | 4 |
| SelfHarm threshold | 0 | 0 | 2 |
// config/blocklists.json
{
"blocklists": [
{
"name": "competitor-names",
"description": "Block competitor brand mentions in bot responses",
"terms": ["CompetitorA", "CompetitorB"],
"match_type": "exact_or_substring"
},
{
"name": "internal-projects",
"description": "Block internal project codenames",
"terms": ["ProjectX", "OperationY"],
"match_type": "exact"
}
],
"max_blocklist_terms": 10000,
"case_sensitive": false
}// config/prompt-shields.json
{
"prompt_injection_detection": true,
"jailbreak_detection": true,
"indirect_attack_detection": true,
"detection_model": "default",
"action_on_detection": "block_and_log",
"custom_patterns": [
{ "pattern": "ignore previous instructions", "severity": "high" },
{ "pattern": "you are now", "severity": "medium" },
{ "pattern": "system prompt", "severity": "high" }
],
"groundedness_detection": {
"enabled": true,
"threshold": 0.7,
"action": "flag_for_review"
}
}// config/review.json
{
"auto_approve_below_severity": 0,
"require_review_above_severity": 2,
"review_queue": "content-moderation-review",
"reviewer_roles": ["content_moderator", "compliance_officer"],
"sla_review_hours": 4,
"escalation_after_hours": 8,
"review_actions": ["approve", "reject", "modify", "escalate"],
"appeal_process": true
}// config/guardrails.json
{
"quality": {
"false_positive_rate_max": 0.05,
"false_negative_rate_max": 0.01,
"detection_latency_ms": 200
},
"safety": {
"block_rate_alert_threshold": 0.10,
"review_backlog_alert": 100,
"real_time_monitoring": true
},
"cost": {
"max_api_calls_per_minute": 1000,
"max_review_queue_size": 500
}
}| Check | Expected | Command | ||
|---|---|---|---|---|
| All 4 categories configured | true | `jq '.categories | keys | length' config/categories.json` |
| Prompt Shields enabled | true | jq '.prompt_injection_detection' config/prompt-shields.json | ||
| Detection latency | <=200ms | Monitor via Application Insights | ||
| False negative rate | <=1% | Run evaluation with adversarial test set |
| Issue | Cause | Fix |
|---|---|---|
| Too many false positives | Thresholds too low | Increase severity threshold to 2 |
| Missed harmful content | Thresholds too high | Lower to 0 for SelfHarm category |
| Prompt injections passing | Shields disabled | Enable all three detection types |
| Review backlog growing | SLA too long | Reduce sla_review_hours or add reviewers |
~30 seconds. Free. No account. Every finding cites a rule and a line of evidence.