An Evaluation Run is a batch job that evaluates a date range of real requests for a specific Application + Intent using a chosen set of Sentinels.Documentation Index
Fetch the complete documentation index at: https://docs.trymaitai.com/llms.txt
Use this file to discover all available pages before exploring further.
Where to find Evaluation Runs
In the Portal, open the Evaluation page to see your past runs and create a new one.Create a new Evaluation Run
Click New Evaluation, then complete the wizard:- 1) Application: pick the application to evaluate.
- 2) Intent: pick the intent (action type).
- 3) Dates: choose a start/end date range.
- The UI shows an estimated request count for the selected range.
- 4) Sentinels: choose which sentinels to run.
- High-severity sentinels are preselected by default.
- You can optionally open a selected sentinel and make run-specific edits (the UI marks edited sentinels with
*in the selector).
- 5) Review: confirm the run inputs, then click Run Evaluation.
In the Evaluation Run wizard, “editing” a sentinel is a temporary override for that run (it does not update the saved sentinel configuration). To permanently change a sentinel, edit it from the Intent Group’s Sentinels tab.
Run statuses
Evaluation Runs can show statuses like:- PENDING: queued/running
- DONE: completed
- ERROR: failed
Review an Evaluation Run
Open a run to see:- Configuration: application, intent, date range, and the selected sentinels (you can view each sentinel definition).
- Results: total request count, pass count, fault count, and pass rate.
- Requests: a paginated list of evaluated requests, with per-request pass/fail.