Expand description
Agent evaluation against a golden corpus.
Compares agent output to expected results from a golden corpus and computes evaluation metrics: accuracy, precision, recall, F1, and per-case pass/fail. The comparison is deterministic and string-based (exact match, substring match, or token overlap).
Structs§
- Case
Result - Result of evaluating a single case.
- Eval
Metrics - Aggregate evaluation metrics.
Enums§
- Match
Method - Comparison method for matching agent output to expected.
Functions§
- compute_
metrics - Compute aggregate metrics from a list of case results.
- eval_
case - Evaluate a single case.
- evaluate_
corpus - Evaluate an agent’s outputs against a golden corpus.
- score_
case - Score a single case’s output against expected, returning a [0.0, 1.0] score.