Skip to main content
Generated from traces or written from scratch, your evals live in your repo as plain, executable pytest files. Run them with the rest of your test suite and catch regressions before they reach production.
Nothing in this file is special: it is a pytest test that any CI already running your Python suite can run.
Prefer the guided path? Quickstart installs Kensa, finds and approves the production function or class used to start a conversation, and lands your first eval.

Run your first eval

Add Kensa, run kensa init, and turn one realistic case into a passing eval.

Learn the mental model

Understand how cases, traces, assertions, judges, and trials fit together.

Drive it from your agent

The kensa-evals skill walks Claude Code, Codex, or Cursor through the eval lifecycle.

Look up a command

init, doctor, connect, import, and eval - every flag in one place.

How it works

Traces in

Import minimized, redacted trace evidence from Langfuse or a JSON / JSONL / OTLP export - or capture it locally.

Behavior out

Your coding agent mines imports into reviewable eval ideas you approve and materialize as pytest files.

Assertions gate the judge

Deterministic assertions run first. The judge(...) call only runs if they pass, so obvious regressions never spend tokens.

Ship in CI

Evals are plain pytest. Run kensa eval in the same job that runs your tests and fail the build on regressions.

Where to start

Data flow

Traces from real runs become regression tests. Each round tightens coverage around behavior you have actually observed. Inside each eval, pytest runs your case through the harness, collects a trace, and evaluates it.

Compatible coding agents

Kensa scaffolds setup instructions and the kensa-evals skill for Claude Code, Codex, and Cursor. If none are detected, kensa init still prints a copyable setup prompt.

License

Apache 2.0.
Last modified on August 16, 2026