Skip to main content
Monitors continuously sample live production traffic, judge each sampled invocation against a configurable set of rules, and store the verdicts so you can review them in the Portal. A Monitor defines:
  • What to judge (the runner, a model + prompt, a workflow, or the production output itself)
  • How to decide the verdict (the resolution, boolean groups of conditions that map to error / warning / pass)
  • What to watch (one or more targets (intents, workflows, or agents) each with its own sample rate)
Monitors are scoped to your Company. The same Monitor can watch many targets, and each target sets its own sampling percentage.

​
Where to manage Monitors

In the Portal, go to Test > Monitors. From there you can:
  • Create a new Monitor (+ New Monitor)
  • Open any existing Monitor to edit configuration, attach/detach targets, publish versions, and view recent runs
  • Pause sampling or activate a Monitor without deleting it

​
How Monitors compare to Sentinels

Use Sentinels when you want to evaluate every request for a specific Intent Group with an LLM judge. Use Monitors when you want to sample across higher-level surfaces (intents, workflows, agents), use a workflow or a direct rule as the judge, or watch the same kind of failure across multiple resources.

​
Monitor lifecycle

  1. Draft: newly created or edited Monitors land here. Sampling is off.
  2. Active: sampling fires for every attached target at its configured rate.
  3. Paused: Monitor exists and keeps history, but no new runs are produced until reactivated.
Each save to the Monitor’s name, description, or config bumps has_unpublished_changes=true. You can publish a version when you’re ready to freeze a snapshot for review or rollback. Next: Configuration.