Tracing & imports

Kensa turns observed behavior into evals. Trace evidence reaches Kensa two ways: imported from an observability provider or export file, or captured locally from an instrumented run. Imports become minimized, redacted evidence under .kensa/traces/ that your coding agent can mine into evals.

.kensa/traces/ is local evidence, not an observability backend. Kensa imports bounded exports to build CI regression tests; it does not stream or store your production telemetry.

Importing from a provider

Langfuse is supported as a live connection. Connect once, then import.

kensa connect langfuse           # verify credentials, save non-secret metadata
kensa import --from langfuse --since 7d --limit 200

kensa connect saves connection metadata to .kensa/connections/<provider>.json. API keys are read from environment variables (or a configured dotenv) at run time and are never written to that file. By default connect verifies access before saving; pass --configure-only to skip the check.

Provider	Connect	Key env vars
Langfuse	`kensa connect langfuse`	`LANGFUSE_PUBLIC_KEY`, `LANGFUSE_SECRET_KEY`

Kensa automatically chooses between the legacy traces API and observations v2. Use --langfuse-mode legacy_traces or --langfuse-mode observations_v2 only when you need to force one path. See the CLI reference for every connect and import flag.

Importing from a file

You do not need a live connection. kensa import reads bounded export files in several formats:

kensa import --from json   --source traces.json
kensa import --from jsonl  --source spans.jsonl
kensa import --from otlp   --source otlp-export.json
kensa import --from langfuse --source langfuse-export.json

`--from`	Source
`json`	A JSON array of spans
`jsonl`	One span per line
`otlp`	OpenTelemetry Protocol export
`langfuse`	A Langfuse export file (or a live connection)

Imports are normalized to Kensa’s TraceView schema and written to .kensa/traces/imports/<provider>-<timestamp>.jsonl, with a .manifest.json recording provenance and redaction, and a latest.json pointer.

Evidence minimization and redaction

Import has two privacy boundaries:

Kensa projects provider payloads into a minimal TraceView. It retains trace and span relationships, names and kinds, tool names, timing and status, input and output, and model usage. Arbitrary provider metadata is dropped.
Kensa scans retained values with deterministic recognizers, detect-secrets, Presidio, and spaCy NER. PII and secrets become typed placeholders such as [PERSON_1] before the artifact is written.

Redaction is mandatory and fails closed. Missing dependencies or model readiness block imports, trace access, inspection, and generation. Missing or mismatched manifests also block artifact access. kensa init installs kensa[redaction] and prepares the small English model by default. Select the larger model when recall matters more than download size:

kensa init --redaction-model large

[tool.kensa].redaction_model selects the model. Kensa verifies the wheel checksum on download and validates the cache before use. It writes no readiness file.

Capturing traces locally

If you do not have an observability provider, instrument your agent and let Kensa record spans directly. The recording helpers are exported from the top-level package:

from kensa import instrument, record_tool_call, record_llm_call, record_span

instrument()

with record_tool_call(
    "lookup_customer",
    arguments={
        "customer_id": "cus_123",
        "include": {"charges": True},
    },
) as tool_call:
    customer = lookup_customer("cus_123")
    tool_call.set_result({"found": True, "customer": customer})

with record_llm_call(provider="openai", model="gpt-5.4-mini"):
    reply = client.responses.create(...)

Use result= when the result already exists before the context:

with record_tool_call(
    "read_cache",
    arguments={"key": "customer:cus_123"},
    result={"hit": True},
):
    pass

Arguments and results must be strict JSON values. Kensa validates and snapshots them before writing canonical kensa.tool.args and kensa.tool.result span attributes, so later caller mutation cannot change the evidence. set_result(...) records at most one result. Supplying result= and then calling set_result(...), or calling set_result(...) twice, raises RuntimeError. If the tool body raises before recording a result, the original exception and traceback escape unchanged. The tool call remains observable with status == "error" and result_recorded is False. During kensa eval, these context managers preserve the current operation when a trial times out. Set KENSA_TRACE_DIR so spans are written to disk, then import them like any other JSONL source:

KENSA_TRACE_DIR=.kensa/traces python my_agent.py
kensa import --from jsonl --source .kensa/traces/spans.jsonl

OTLP HTTP export

Opt in with otlp_endpoint= or KENSA_OTLP_ENDPOINT. Local JSONL capture stays on, so each span lands both locally and at the endpoint:

from kensa import instrument

instrument(
    trace_dir=".kensa/traces",
    otlp_endpoint="http://localhost:4318/v1/traces",
    otlp_headers={"authorization": "Bearer local-token"},
)

Or from the environment:

KENSA_OTLP_ENDPOINT=http://localhost:4318/v1/traces \
KENSA_OTLP_HEADERS=authorization=Bearer\ local-token \
python my_agent.py

Standard OTEL_EXPORTER_OTLP_* variables never enable export, so instrument() stays a no-op in a process that only configures OpenTelemetry for its own exporter. Credentials come from otlp_headers= or KENSA_OTLP_HEADERS only; your application’s OTEL_EXPORTER_OTLP_HEADERS is never sent to a Kensa endpoint. Timeout, compression, and certificate settings still read the standard variables. Without a local directory, spans also go to .kensa/traces/spans.jsonl. An unreachable endpoint is logged and does not discard the local span or raise. Requests default to a two-second timeout, overridable with otlp_timeout_s or the standard timeout variables, which the Python exporter reads in seconds. Both exporters ship the same spans, so expect duplicates if your application already exports to the same collector.

OTLP export sends raw telemetry before Kensa’s import-time redaction. Treat the endpoint as sensitive, use TLS for non-local collectors, and configure authentication through OTLP headers.

Local capture files are raw telemetry and can contain sensitive payloads. Treat only successfully imported artifacts with valid redaction manifests as safe evidence.

Inside an eval, the same helpers feed the live result.trace accessor on successful runs. The case-aware agent returned by kensa_run(case) wraps real tool and model calls with record_tool_call / record_llm_call. Kensa adds one source-attributed outer response span for every attempted agent or simulator response, while the existing trace totals include activity from both sources. See Pytest plugin.

Attaching evidence from a target-owned agent

Keep the deployed agent in its own repository or service and translate its native result into AgentRunEvidence inside your adapter. Kensa records the supplied data but never imports, invokes, or connects to the target agent.

from kensa.pytest import (
    AgentEvent,
    AgentRunEvidence,
    ConversationResponse,
    EffectPolicy,
    EvidenceCompleteness,
    ExecutionAttestation,
    StateObservation,
    TraceReference,
    attach_agent_run,
)


class TargetAdapter:
    def __init__(self, client):
        self.client = client

    async def respond(self, messages):
        external = await self.client.run(messages)
        attach_agent_run(
            AgentRunEvidence(
                run_id=external.run_id,
                attestation=ExecutionAttestation(
                    revision=external.revision,
                    environment="agent-staging",
                    effects=EffectPolicy.SANDBOXED,
                ),
                events=tuple(
                    AgentEvent(
                        id=event.id,
                        parent_id=event.parent_id,
                        sequence=index,
                        kind=event.kind,
                        name=event.name,
                        input=event.input,
                        output=event.output,
                        attributes=event.attributes,
                        status=event.status,
                        started_at_ns=event.started_at_ns,
                        ended_at_ns=event.ended_at_ns,
                    )
                    for index, event in enumerate(external.events)
                ),
                trace=TraceReference(
                    provider=external.trace_provider,
                    trace_id=external.trace_id,
                    url=external.trace_url,
                ),
                trajectory_completeness=EvidenceCompleteness.COMPLETE,
                state=(
                    StateObservation(
                        name="account",
                        value=external.account_state,
                        source="staging-database",
                        observed_at_ns=external.observed_at_ns,
                    ),
                ),
                state_completeness=EvidenceCompleteness.COMPLETE,
            )
        )
        return ConversationResponse(output=external.output)

Call attach_agent_run() after the target finishes and before respond() returns. With no active Kensa runtime it is a no-op, so the same adapter works in application tests. If a Kensa runtime exists, attachment before case.run(), after the operation returns, or from a late background task raises KensaCaseError. Each run_id is immutable: an equal duplicate is ignored and conflicting evidence is rejected. Attachment lifecycle errors and conflicting run_id evidence are KensaCaseError values. Even when the conversation boundary wraps them, Kensa records them as harness/case_contract, not as agent execution failures. Invalid AgentRunEvidence model construction originates inside the target adapter and is therefore an agent/execution error by default when it escapes through respond(). If the adapter knows that invalid evidence is its own contract defect, translate it explicitly:

from pydantic import ValidationError

from kensa.pytest import AgentRunEvidence, KensaEvalError


try:
    evidence = AgentRunEvidence.model_validate(external.evidence)
except ValidationError as exc:
    raise KensaEvalError(
        "Target adapter returned invalid agent-run evidence.",
        category="harness",
        kind="agent_run_contract",
        evidence={"validation_errors": exc.error_count()},
    ) from exc

Kensa does not classify arbitrary Pydantic validation errors as harness failures because the target agent can use Pydantic for its own behavior.

Evidence reference

Model	Fields
`AgentRunEvidence`	`schema_version`, `run_id`, `attestation`, ordered `events`, optional `trace`, `trajectory_completeness`, ordered `state`, `state_completeness`, `incomplete_reason`
`ExecutionAttestation`	target `revision`, named `environment`, and `effects`
`TraceReference`	provider name, provider-owned `trace_id`, and optional `url`
`AgentEvent`	provider-owned `id` and optional parent, increasing `sequence`, `kind`, `name`, JSON `input`, `output`, and `attributes`, `status`, and optional Unix epoch nanosecond timestamps
`StateObservation`	`name`, JSON `value`, `source`, and optional Unix epoch nanosecond observation time

Event kind is one of llm, tool, handoff, retrieval, action, state, or span. Event status is completed, failed, or cancelled. Completeness is complete, pending, partial, or unavailable; any value other than complete requires an incomplete_reason. Trajectory and state completeness are independent. Effect policies are target-supplied facts:

Value	Meaning
`none`	The evaluated path had no effectful tool capability
`captured`	Effectful calls were intercepted without reaching the dependency
`sandboxed`	Effects reached the isolated environment named by `environment`
`live`	Effects could reach the named non-isolated environment

Attaching effects="live" does not make a network call or approve, reject, authorize, sandbox, or roll back an effect. Point adapters at test or sandbox environments unless live effects are deliberately part of the evaluation policy. Trajectory events become normalized spans in result.trace. Tool events appear in result.trace.tools, and LLM events count toward result.trace.llm_turns. A finite nonnegative kensa.cost_usd event attribute contributes to known cost. If any local or external LLM event has no valid cost, known_cost_usd still sums priced events while cost_available is false and cost_usd is unavailable. StateObservation contains actual state read from the target-owned environment and stays under result.trace.agent_runs[*].state. It never becomes a span or trial outcome. An AgentEvent(kind="state") is only a reported trajectory transition, not proof of actual state.

Attached evidence is written unredacted to result JSON, trace JSONL, xdist, failure, and timeout artifacts. It can contain prompts, tool arguments, tool results, and state values. Protect .kensa artifacts and avoid production secrets.

From traces to evals

Once evidence is imported, your coding agent turns it into evals. The kensa-inspect skill reads the redacted evidence and proposes reviewable eval ideas as a YAML queue under .kensa/inspect/, which the CLI can read and validate:

kensa traces list         # list imported trace IDs
kensa traces sample       # preview one trace
kensa inspect list        # read the eval-idea review queue
kensa inspect lint        # validate the queue against the imported traces

Approve the ideas worth keeping (change status: pending to status: approved), then the kensa-generate skill materializes them as tests/evals/test_<id>.py. No traces yet? Capture a local run (below), or let the skill seed a first eval from one realistic prompt.

OpenTelemetry compatibility

Spans are standard OpenTelemetry. Kensa annotates them with kensa.span.kind, kensa.tool.name, and kensa.llm.provider / kensa.llm.model attributes. record_llm_call also emits standard GenAI attributes. operation_name defaults to chat and accepts embeddings, generate_content, or text_completion; use span_kind=SpanKind.INTERNAL for an in-process model. Anything that emits OTLP can be imported with --from otlp.

Getting started

Reference

Workflows

Releases

Tracing & imports

Importing from a provider

Importing from a file

Evidence minimization and redaction

Capturing traces locally

OTLP HTTP export

Attaching evidence from a target-owned agent

Evidence reference

From traces to evals

OpenTelemetry compatibility

​Importing from a provider

​Importing from a file

​Evidence minimization and redaction

​Capturing traces locally

​OTLP HTTP export

​Attaching evidence from a target-owned agent

​Evidence reference

​From traces to evals

​OpenTelemetry compatibility

Importing from a provider

Importing from a file

Evidence minimization and redaction

Capturing traces locally

OTLP HTTP export

Attaching evidence from a target-owned agent

Evidence reference

From traces to evals

OpenTelemetry compatibility