Skip to main content

Documentation Index

Fetch the complete documentation index at: https://docs.trymaitai.com/llms.txt

Use this file to discover all available pages before exploring further.

Monitors continuously sample live production traffic, judge each sampled invocation against a configurable set of rules, and store the verdicts so you can review them in the Portal. A Monitor defines:
  • What to judge (the runner — a model + prompt, a workflow, or the production output itself)
  • How to decide the verdict (the resolution — boolean groups of conditions that map to error / warning / pass)
  • What to watch (one or more targets — intents, workflows, or agents — each with its own sample rate)
Monitors are scoped to your Company. The same Monitor can watch many targets, and each target sets its own sampling percentage.

​
Where to manage Monitors

In the Portal, go to Test > Monitors. From there you can:
  • Create a new Monitor (+ New Monitor)
  • Open any existing Monitor to edit configuration, attach/detach targets, publish versions, and view recent runs
  • Pause sampling or activate a Monitor without deleting it

​
How Monitors compare to Sentinels

SentinelsMonitors
ScopePer-Intent-GroupCross-resource (intent / workflow / agent)
TriggerEvery qualifying request (always-on)Sampled at a configurable rate
JudgeAlways an LLM directive against the request/responseChoice of model, workflow, or direct evaluation of the output
Where verdicts surfaceEvaluationsMonitors area (Activity chart, At-a-Glance, Runs)
Use Sentinels when you want to evaluate every request for a specific Intent Group with an LLM judge. Use Monitors when you want to sample across higher-level surfaces (intents, workflows, agents), use a workflow or a direct rule as the judge, or watch the same kind of failure across multiple resources.

​
Monitor lifecycle

  1. Draft — newly created or edited Monitors land here. Sampling is off.
  2. Active — sampling fires for every attached target at its configured rate.
  3. Paused — Monitor exists and keeps history, but no new runs are produced until reactivated.
Each save to the Monitor’s name, description, or config bumps has_unpublished_changes=true. You can publish a version when you’re ready to freeze a snapshot for review or rollback. Next: Configuration.