Interpreting Results

Once a Test Run completes, its detail page gives you three layers: run-level summary metrics, per-item scores, and side-by-side comparisons.

​

Run summary (macro view)

For Request and Workflow runs, the summary tiles are:

• # Items: total items in the run
• Completed %: how many items finished executing
• Match Rate %: matched / (matched + mismatched), over items that have expected output
• Error Rate %: items that errored during execution

A banner breaks the match rate down into matched / mismatched / without-expected / errored. For Agent runs, the summary instead shows a Pass Rate: the average pass rate across the run’s criteria, since agent runs are scored by criteria rather than structural matching.

​

Requests table (per-item view)

The items table is where you debug individual cases:

• Status for each item execution
• Score: a normalized 0–100% pill; hover it for the criteria breakdown (each criterion’s pass/fail, numeric value, or tag)
• Actions: inspect the item, and compare the expected output against the run’s actual output

​

Criteria breakdown

The per-criterion breakdown adapts to the criterion type:

• Boolean → a pass rate, a failure count, and a pass/fail bar
• Numeric → a score distribution (a table + bar over the score buckets, colored low→high)
• Tag → a categorical distribution across the tag values

Use this to see which criterion is driving a low score, then jump into those items.

​

Comparing runs

Because runs share the test set as their axis, you can compare them directly:

• Across runs: see item-by-item score shifts between two or more runs, including runs against different consumers (e.g. a model baseline vs. an agent).
• A single item across runs: track how one specific scenario changed over time.

Next: Regression Monitoring.