> ## Documentation Index
> Fetch the complete documentation index at: https://kensa.sh/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# Introduction

> Kensa turns agent traces into pytest evals that run in CI.

Kensa turns agent traces into pytest evals that run in CI. Generated from traces or written by hand, your evals live in your repo as plain, executable pytest files. Run them with the rest of your test suite and catch regressions before they reach production.

```python theme={null}
import pytest

from kensa.pytest import judge, kensa_case


@pytest.mark.kensa(trials=3)
@pytest.mark.parametrize(
    "case",
    [
        kensa_case(
            id="refund_without_order_history",
            input="I was charged $29 yesterday. No order ID, but please refund it.",
        )
    ],
)
def test_refund_policy(case, kensa_run, kensa_trace):
    output = case.run(kensa_run)

    assert kensa_trace.tools.include(["lookup_customer"])
    assert kensa_trace.tools.exclude(["issue_refund"])

    result = judge(output, "The response must not promise an unsupported refund.", input=case.input)
    assert result.passed, result.reasoning
```

Open source, local-first, and framework agnostic. Nothing in this file is special: it is a pytest test that any CI already running your Python suite can run.

<Check>Prefer the guided path? [Quickstart](/quickstart) installs Kensa, wires the harness, and lands your first eval.</Check>

<Columns cols={2}>
  <Card title="Run your first eval" href="/quickstart" cta="Open quickstart" arrow="true">
    Add Kensa, run `kensa init`, and turn one realistic case into a passing eval.
  </Card>

  <Card title="Learn the mental model" href="/concepts" cta="Read concepts" arrow="true">
    Understand how cases, traces, assertions, judges, and trials fit together.
  </Card>

  <Card title="Drive it from your agent" href="/skills" cta="Use the skill" arrow="true">
    The `kensa-evals` skill walks Claude Code, Codex, or Cursor through the eval lifecycle.
  </Card>

  <Card title="Look up a command" href="/cli" cta="CLI reference" arrow="true">
    `init`, `doctor`, `connect`, `import`, and `eval` — every flag in one place.
  </Card>
</Columns>

## How it works

<Columns cols={2}>
  <Card title="Traces in">
    Import bounded trace evidence from Langfuse or a JSON / JSONL / OTLP export — or capture it locally.
  </Card>

  <Card title="Behavior out">
    Your coding agent mines imports into reviewable eval ideas you approve and materialize as pytest files.
  </Card>

  <Card title="Assertions gate the judge">
    Deterministic assertions run first. The `judge(...)` call only runs if they pass, so obvious regressions never spend tokens.
  </Card>

  <Card title="Ship in CI">
    Evals are plain pytest. Run `kensa eval` in the same job that runs your tests and fail the build on regressions.
  </Card>
</Columns>

## Where to start

| If you want to               | Go to                         |
| ---------------------------- | ----------------------------- |
| Get running in a few minutes | [Quickstart](/quickstart)     |
| Understand the mental model  | [Concepts](/concepts)         |
| Define cases and trials      | [Cases](/cases)               |
| Assert on traces and output  | [Assertions](/assertions)     |
| Bring in existing traces     | [Tracing & imports](/tracing) |
| Look up exact commands       | [CLI reference](/cli)         |

## Why teams use it

* **Evals are just pytest.** No separate runner, no separate dashboard. If your CI runs `pytest`, it runs Kensa.
* **Cold-start friendly.** Generate a first eval from one realistic prompt, or mine eval ideas from traces you already have.
* **Cost-aware by default.** Deterministic assertions short-circuit before the judge, so failing cases never reach an LLM call.
* **Local-first evidence.** `.kensa/traces/` holds bounded local trace evidence, not a live observability backend.

## The eval loop

Traces from real runs become regression tests. Each round tightens coverage around behavior you have actually observed.

```text title="Eval loop" theme={null}
connect / capture →  kensa import   →  trace evidence in .kensa/
trace evidence    →  kensa-inspect  →  reviewable eval ideas (.kensa/inspect/)
approved idea     →  kensa-generate →  tests/evals/test_<id>.py
evals             →  kensa eval     →  pass / fail / flaky in CI
```

## Data flow

```mermaid theme={null}
flowchart LR
  A["Traces<br/>Langfuse · JSON/JSONL/OTLP"] --> B["<code>kensa import</code>"]
  B --> C["<code>kensa-inspect</code><br/>eval ideas"]
  C --> D["<code>tests/evals/test_*.py</code><br/>generated or hand-written"]
  D --> E["<code>kensa eval</code><br/>(pytest)"]
  E --> F["Verdicts<br/>term / JSON / Markdown"]
```

Inside each eval, pytest runs your case through the harness, collects a trace, and evaluates it. Deterministic assertions (`kensa_trace.tools`, plain `assert`) come first. A semantic `judge(...)` call only runs when you reach it, so a failed assertion never spends judge tokens.

## Compatible coding agents

Kensa scaffolds setup instructions and the `kensa-evals` skill for Claude Code, Codex, and Cursor. If none are detected, `kensa init` still prints a copyable setup prompt.

## License

Apache 2.0.