> ## Documentation Index
> Fetch the complete documentation index at: https://docs.trymaitai.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Execute Test Runs

> Monitor model performance improvements with Test Runs

Test Runs let you run a Test Set against a chosen **consumer**: a model, a workflow, or an agent,
and score it, so you can track improvements (and regressions) over time. The same set can be run
against different consumers, which makes those runs directly comparable.

If you want the full reference docs for Test Runs, start here:

* [Test Run Execution](/test/test_sets/execution)
* [Interpreting Results](/test/test_sets/results)
* [Regression Monitoring](/test/test_sets/regression)

## Creating a Test Run

1. Navigate to your Test Set Overview page
2. Click **New Test Run** in the upper right corner of the Test Runs table

<img src="https://mintcdn.com/maitai-cbe2cd27/kcIewhDoyU5XOGPZ/images/test_runs/new_test_run_button.png?fit=max&auto=format&n=kcIewhDoyU5XOGPZ&q=85&s=b0cefdc6725a9174ea114071d70e32d3" alt="New Test Run Button" width="2416" height="1212" data-path="images/test_runs/new_test_run_button.png" />

3. Configure your Test Run:
   * Choose a **consumer**: **Request** (a model + generation params), **Workflow**, or **Agent**.
   * Add a descriptive name to identify this run later.
   * Pick the [evaluation criteria](/test/test_sets/criteria) that will score the run.

<img src="https://mintcdn.com/maitai-cbe2cd27/kcIewhDoyU5XOGPZ/images/test_runs/test_run_configuration.png?fit=max&auto=format&n=kcIewhDoyU5XOGPZ&q=85&s=c0c222cd93ca86bc79733cc5fec12faf" alt="Test Run Configuration" width="2414" height="1442" data-path="images/test_runs/test_run_configuration.png" />

4. Review your settings and click **Create Test Run**
5. You'll be redirected back to the Test Set Overview where your new Test Run will appear at the top of the table

<Note>
  Test Runs may take a few minutes to complete. While the status is `PENDING`, you won't be able to access the Test Run details.
</Note>

## Understanding Test Run Results

Once your Test Run completes, you can analyze the results in detail:

### Overview

Each Test Run shows headline metrics, for model and workflow runs a **match rate** (share of items whose output matched the expected output), and for agent runs a **pass rate** across the run's criteria.

<img src="https://mintcdn.com/maitai-cbe2cd27/kcIewhDoyU5XOGPZ/images/test_runs/test_run_overview.png?fit=max&auto=format&n=kcIewhDoyU5XOGPZ&q=85&s=b8d3228fa03e809ab81a7133ae6a0a91" alt="Test Run Overview" width="2332" height="896" data-path="images/test_runs/test_run_overview.png" />

### Detailed Analysis

Click on any completed Test Run to see:

1. **Details Section**
   * Configuration used for the Test Run
   * Description and execution timestamps
   * Overall performance metrics

2. **Results Section**
   * Macro-level metrics across all items (match/pass rate, completed %, error %)
   * Per-criterion breakdowns, pass/fail, numeric score distributions, or tag distributions
   * Hover a segment for detailed breakdowns

<img src="https://mintcdn.com/maitai-cbe2cd27/kcIewhDoyU5XOGPZ/images/test_runs/test_run_results.png?fit=max&auto=format&n=kcIewhDoyU5XOGPZ&q=85&s=933c7e1467c3f0715cd4de59250ed9c5" alt="Test Run Results" width="2410" height="1520" data-path="images/test_runs/test_run_results.png" />

3. **Requests Table**
   * Individual scores for each request
   * Hover over scores to see evaluation criteria
   * Compare original vs Test Run responses side-by-side

<Tip>
  Use the "Compare" feature to understand exactly how configuration changes affected specific responses.
</Tip>

## Comparing runs (baseline vs candidate)

On a Test Set, you can compare runs side-by-side, including runs against *different* consumers. This is the fastest way to answer:

* “Did this change improve the overall match rate?”
* “Which specific items got better or worse?”

<Note>
  Want to learn more about building Test Sets? Start with [Test Set Creation](/test/test_sets/creation).
</Note>
