Skip to main content

Module evaluator

Module evaluator 

Source
Expand description

Agent evaluation against a golden corpus.

Compares agent output to expected results from a golden corpus and computes evaluation metrics: accuracy, precision, recall, F1, and per-case pass/fail. The comparison is deterministic and string-based (exact match, substring match, or token overlap).

Structs§

CaseResult
Result of evaluating a single case.
EvalMetrics
Aggregate evaluation metrics.

Enums§

MatchMethod
Comparison method for matching agent output to expected.

Functions§

compute_metrics
Compute aggregate metrics from a list of case results.
eval_case
Evaluate a single case.
evaluate_corpus
Evaluate an agent’s outputs against a golden corpus.
score_case
Score a single case’s output against expected, returning a [0.0, 1.0] score.