> ## Documentation Index
> Fetch the complete documentation index at: https://docs.trymaitai.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Evaluation Criteria

> The reusable checks that score Test Runs: deterministic, LLM-judge, and agent-trajectory.

**Evaluation Criteria** are the reusable checks that score your [Test Runs](/test/test_sets/execution).
You define them once in your company catalog, then select which ones apply when you create a run.

## Where to manage criteria

In the Portal, go to `Test > Test Criteria`. You'll see:

* **System criteria**: built-in checks (like **Exact Match**). Read-only, but you can clone one to
  customize it.
* **Custom criteria**: checks you create for your own quality bar.

## Criteria types

### Deterministic

Computed directly from the output. No LLM involved, fully repeatable:

* **Exact Match**: output equals expected output
* **Contains**: output contains an expected substring
* **Regex Match**: output matches a pattern
* **BLEU / ROUGE / Cosine Similarity / Levenshtein**: text-similarity scores against the expected
  output

### LLM judge

A model grades the output against your instructions. Three scoring modes:

* **Boolean**: PASS / FAIL against a yes/no question
* **Numeric**: an integer rubric over a configurable range (default **1–5**), optionally with named
  levels (e.g. `1 = Poor` … `5 = Perfect`)
* **Tags**: classify each output into one of N categories

LLM-judge criteria let you pick the evaluation model and, where relevant, map which fields to compare.

### Agent trajectory (agent runs)

For [agent](/build/agents/create_agent) runs, criteria can score *how* the agent worked, not just
its final answer:

* **Tool checks**: required tools were called, only whitelisted tools were used, tools ran in the
  expected sequence
* **Shape checks**: step count, sub-agent depth, failure rate within bounds
* **Budget checks**: latency and cost ceilings respected
* **Outcome checks**: an LLM judge or a custom **Python predicate** over the final result

## How criteria attach to a run

You select criteria in the **Evaluations** step of the Test Run wizard:

* **Request** and **Workflow** runs default to **Exact Match**; add more as needed.
* **Agent** runs require at least one criterion (trajectory, outcome, or custom).

Each item is then scored against every selected criterion. How those results roll up into a single
per-item percent (and the run's match/pass rate) is covered in
[Scoring & Criteria](/test/test_sets/rubrics).