Kensa — independently scanned and version-tracked by SaferSkills.
SaferSkills independently audited Kensa (Plugin) and scored it 100/100 (green). The audit ran 55 deterministic rules across Security, Supply Chain, Maintenance, Transparency, and Community; it found 0 high-severity and 0 lower-severity findings. The full rule-by-rule trace and per-finding evidence are below. Free, methodology-open.
Findings & checks · 0 flagged
Every scanned point with the score it earned and what moved between them.
Aggregate score unchanged between these scans.
The primary manifest — the file an agent reads to learn what this artifact does.
<div align="center">
<br> <img src="https://raw.githubusercontent.com/satyaborg/kensa/main/assets/banner.png" alt="kensa - the open source agent evals harness" width="800"> <br><br>
<p>Kensa is the open source harness for evaluating agents.</p>
<p> <a href="https://github.com/satyaborg/kensa/actions/workflows/ci.yml"><img src="https://img.shields.io/github/actions/workflow/status/satyaborg/kensa/ci.yml?label=CI&style=flat-square" alt="CI"></a> <a href="https://pypi.org/project/kensa/"><img src="https://img.shields.io/pypi/v/kensa?style=flat-square" alt="PyPI"></a> <a href="https://www.python.org/downloads/"><img src="https://img.shields.io/python/required-version-toml?tomlFilePath=https%3A%2F%2Fraw.githubusercontent.com%2Fsatyaborg%2Fkensa%2Fmain%2Fpyproject.toml&style=flat-square" alt="Python"></a> <a href="LICENSE"><img src="https://img.shields.io/github/license/satyaborg/kensa?style=flat-square" alt="License"></a> <a href="https://pepy.tech/projects/kensa"><img src="https://img.shields.io/pepy/dt/kensa?style=flat-square" alt="Downloads"></a> </p>
</div>
Agents are non-deterministic. Prompts drift. Tools change. Models behave differently. Any change can make them slower, more expensive, or just plain unreliable.
kensa gives coding agents, like Claude Code, a repeatable loop to eval your agents, and catch regressions every time you make a change.
Run these from the root of your Python agent repo.
Open your coding agent and paste:
Run `uvx kensa init --cli --agent all`, then use the audit-evals skill and
follow the eval lifecycle.The agent installs the CLI, scaffolds .kensa/, drops in the five skills, and runs your first eval. Works with Claude Code, Codex, Cursor, OpenCode, and Gemini CLI.
If you want to control the install step but still let your coding agent drive the eval workflow:
uvx kensa init --cli --agent allThen in Claude Code, Codex, Cursor, OpenCode, or Gemini CLI:
> /audit-evalsThe skill captures a real run, generates scenarios, runs evals, and reports back.
If you want to skip the coding-agent loop entirely and drive kensa as a regular CLI:
uvx kensa init # dev dep + bare .kensa/
kensa capture -i "<example input>" -- <your agent> # record one real run as a trace
kensa generate # synthesize scenarios from the capture
kensa eval # run + judge + reportIf you primarily use Claude Code, install via the plugin marketplace:
/plugin marketplace add satyaborg/kensa
/plugin install kensaTell your coding agent what you want:
| You say | Kensa does |
|---|---|
| "Evaluate this agent" | Audit setup, create or reuse scenarios, and run evals. |
| "Why are evals failing?" | Inspect results and traces, then diagnose the root cause. |
| "Add coverage for tool use" | Write scenario YAML with tool or trajectory checks. |
| "The judge seems wrong" | Create or validate structured judge prompts. |
use, tokens, cost, and latency captured.
Zero code changes. kensa captures LLM calls, tool use, tokens, cost, and latency without modifying your agent. OpenTelemetry (OTel) compatible.
<details> <summary>Provider extras</summary>
uv add "kensa[anthropic]"
uv add "kensa[openai]"
uv add "kensa[langchain]"
uv add "kensa[all]"</details>
| Command | What it does |
|---|---|
kensa init | Scaffold .kensa/ (bare; pass --example for a demo agent + scenario) |
kensa capture -i "<input>" -- <cmd> | Record one real agent run as a trace |
kensa doctor | Check instrumentation, config, and environment readiness |
kensa generate | Synthesize scenario YAMLs from captured traces via an LLM |
kensa eval | Run + judge + report in one command |
kensa report | Show the latest results in terminal, Markdown, JSON, or HTML |
See the CLI docs for run, judge, analyze, mcp, and the full command reference.
One-liner for Claude Code (run from your project root):
claude mcp add kensa -- uvx kensa-mcpFor other JSON-based MCP clients, add to your project's .mcp.json or .cursor/mcp.json:
{
"mcpServers": {
"kensa": {
"command": "uvx",
"args": ["kensa-mcp"]
}
}
}For Codex, add to your project-scoped .codex/config.toml:
[mcp_servers.kensa]
command = "uvx"
args = ["kensa-mcp"]See the MCP server docs for tools, resources, and manual config.
If you want to author evals yourself:
kensa init
kensa doctorScenarios live in .kensa/scenarios/*.yaml and point at your agent entrypoint with run_command.
id: classify_ticket
input: "Our entire team can't log in. SSO has returned 502 since 7am."
run_command: [python, agent.py] # input is appended as the final argv element
checks:
- type: trajectory
params:
steps:
- tool: classify_ticket
max_steps: 1
max_tokens: 2000
- type: output_matches
params: { pattern: "^P[123]$" }
criteria: |
P1 is for outages or data loss affecting multiple users.For complete examples, see examples/. See the scenario docs and checks docs for the full field and check reference.
- name: Run evals
run: uv run kensa eval --format markdownIf you only use deterministic checks, you do not need API keys. If you use criteria or judge, add judge LLM provider secrets in CI.
examples/ has sample agents and scenariosCONTRIBUTING.md covers local development~30 seconds. Free. No account. Every finding cites a rule and a line of evidence.