Skip to main content
An Evaluation is what you get when Maitai runs one or more Sentinels against a request (and the request’s final assistant response), producing per-sentinel pass/fail signals you can review in the Portal.

​
Where evaluations show up in the Portal

You’ll typically encounter evaluation results in three places:
  • On a single request: the Request detail page has an Evaluation section with one or more evaluation blocks (including one-off “manual” evaluations you run yourself). See Evaluating a Request.
  • Across many requests: Test > Evaluations shows a global table of evaluation results across your applications. See Evaluation Results.
  • As a batch job: Evaluation Runs let you evaluate a date range of real requests for an Application + Intent using a selected set of Sentinels. See Evaluation Runs.

​
Relationship to Sentinels

  • A Sentinel is the rule/prompt/config that defines what to check (and optionally, how to qualify inputs or suggest corrections).
  • An Evaluation Result is the output of running a Sentinel on a specific request.
If you’re new to Sentinels, start with Sentinels.

​
Statuses you’ll see

At the per-sentinel level, evaluation results commonly include:
  • PASS: the sentinel believes the request/response meets the expectation.
  • FAULT: the sentinel flagged a failure.
  • ERROR: the sentinel could not evaluate (e.g., evaluation error).
  • NA: not applicable for this request.
At the request level, many tables summarize a request as PASS unless any sentinel returns FAULT.

​
Suggested corrections

Some FAULT results include a suggested correction in the UI. This is separate from whether Maitai automatically applies corrections during inference (that behavior is controlled by configuration).