iris-eval — independently scanned and version-tracked by SaferSkills.
SaferSkills independently audited iris-eval (Agent Skill) and scored it 100/100 (green). The audit ran 55 deterministic rules across Security, Supply Chain, Maintenance, Transparency, and Community; it found 0 high-severity and 0 lower-severity findings. The full rule-by-rule trace and per-finding evidence are below. Free, methodology-open.
Findings & checks · 0 flagged
Every scanned point with the score it earned and what moved between them.
First recorded scan — no prior version to compare against.
The primary manifest — the file an agent reads to learn what this artifact does.
Iris is an MCP server that scores every agent output for quality, safety, and cost. 12 built-in deterministic eval rules. No LLM-as-judge. No SDK. No code changes.
npx @iris-eval/mcp-serverOr add to your MCP config:
{
"mcpServers": {
"iris": {
"command": "npx",
"args": ["-y", "@iris-eval/mcp-server"]
}
}
}Log an agent execution trace with spans, tool calls, and metrics.
Use when: You want to record what an agent did for later analysis.
Evaluate agent output quality using 12 built-in deterministic rules across 4 categories:
Use when: You want a quality score for any agent response.
Query stored traces with filters, pagination, and summary statistics.
Use when: You want to review past agent executions or track quality trends over time.
Each rule fires independently with a clear pass/fail result. No opaque LLM scoring — every score is deterministic and reproducible.
| Category | Rule | What It Checks |
|---|---|---|
| Completeness | topic_consistency | Output addresses the prompt topic |
| Completeness | response_complete | Response is substantive, not truncated |
| Completeness | expected_coverage | Key expected elements are present |
| Relevance | language_match | Output language matches input |
| Relevance | output_format_valid | Structured output matches expected schema |
| Relevance | sentiment_appropriate | Tone matches context |
| Safety | no_pii | No credit cards, SSNs, phone numbers, emails leaked |
| Safety | no_hallucination_markers | No fabricated citations, invented facts |
| Safety | no_injection_patterns | No prompt injection attempts in output |
| Safety | no_blocklist_words | No prohibited terms |
| Cost | cost_under_threshold | Execution cost within budget |
| Cost | latency_under_threshold | Response time within limits |
| Environment Variable | Default | Description |
|---|---|---|
| IRIS_API_KEY | (none) | API key for HTTP transport auth |
| IRIS_DB_PATH | ./iris.db | SQLite database path |
| IRIS_LOG_LEVEL | info | Logging verbosity |
| IRIS_DASHBOARD | true | Enable real-time dashboard |
| IRIS_PORT | 3001 | Dashboard port |
See the examples/ directory for detailed walkthroughs:
~30 seconds. Free. No account. Every finding cites a rule and a line of evidence.