> ## Documentation Index
> Fetch the complete documentation index at: https://docs.trymaitai.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Quality Loop (Fault, Fix, then Regression Test)

> A repeatable Portal workflow to fix failures and prevent them from coming back

This recipe connects the most common “day 2” Portal workflows:

* find failures with **Evaluations**
* inspect a **Request**
* refine a **Sentinel**
* promote the case into a **Test Set**
* verify changes with a **Test Run**

## 1) Find a real failure

In the Portal, go to `Observe > Sentinels`, then:

* Open a Sentinel to see the requests it has flagged
* Filter to **FAULT** results
* Open a failing request to see what triggered the Sentinel

See: [Monitoring with Sentinels](/observe/sentinels/monitoring)

## 2) Understand the failure on the Request page

On the Request page:

* Review the request messages and response
* Scroll to **Evaluation** to see the per-sentinel results for this request
* If a failure includes a **Suggested Correction**, treat it as debugging guidance (not an automatic change)

See: [Evaluating a Request](/observe/sentinels/request_evaluations)

## 3) Promote the request into a Test Set

On the Request page, click **Add to Test Set**:

* Choose an existing Test Set, or create one inline
* Add tags (this flow enforces a max of **5** tags)
* Submit

See: [Test Set Creation](/test/test_sets/creation)

## 4) Correct the expected response (when production output was wrong)

If the original production response is wrong, update the Test Set’s expected response:

1. Open the Test Set.
2. Go to **Requests**.
3. Use the request **Edit** action to update the **final assistant message** (and tool calls if applicable).

This makes the Test Set reflect “what should have happened”, which is what you want for regression testing.

## 5) Verify with a Test Run (baseline vs candidate)

From the Test Set:

1. Go to **Test Runs**
2. Click **New Test Run**
3. Configure the candidate model/config and start the run
4. When complete, use **Compare Runs** and per-request comparison to see what changed

See:

* [Test Run Execution](/test/test_sets/execution)
* [Interpreting Results](/test/test_sets/results)

## 6) Fix the root cause with a Sentinel (when applicable)

If the failure is something you want to enforce at the Intent Group level:

1. Open the Intent Group.
2. Go to **Sentinels**.
3. Edit the sentinel (or create a new one) and iterate until it matches your expectation.

See:

* [Sentinels](/observe/sentinels/overview)
* [Sentinel Configuration](/observe/sentinels/configuration)
* [Regenerating Sentinels](/observe/sentinels/regeneration)

## 7) Close the loop

After updating Sentinels and/or configuration:

* Re-check `Observe > Sentinels` (filter by sentinel + `FAULT`) to confirm the fix is behaving as expected on real traffic.
* Re-run the relevant Test Set to ensure you didn’t introduce regressions.