Skip to main content
This recipe is for when you want to enforce a quality bar (or catch a specific failure mode) across an Intent Group.

​
1) Start from an Intent Group

In the Portal:
  1. Open Intent Groups.
  2. Select the Intent Group you care about.
  3. Go to the Sentinels tab.

​
2) Generate a new Sentinel

Click New Sentinel, then fill out:
  • Expectation: what must be true for “good” output
  • Condition: when this expectation applies
If the generator asks follow-up questions, answer them and submit again until a Sentinel is created. See: Generating Sentinels

​
3) Configure it like a reviewer

On the Sentinel edit page, tune:
  • Severity
  • Evaluation directive + Examples
  • Optional sections like Qualification and Correction
  • Include options (system prompt, context, input, tool calls, JSON schema)
See: Sentinel Configuration

​
4) Iterate with regeneration (when the sentinel is “directionally wrong”)

If a section needs a bigger rewrite, use the Portal’s regeneration flow and provide feedback (and expectation/condition if prompted), then re-review and Update Sentinel. See: Regenerating Sentinels

​
5) Validate on real traffic

Use Test > Evaluations to validate quickly:
  • Add a Sentinel filter (optionally scoped to FAULT)
  • Expand rows to see the per-sentinel results
  • Open failing requests to see what triggered the Sentinel
See: Evaluation Results

​
6) Monitor over time

Once the Sentinel is live:
  • Use the Intent Group’s Fault Rate chart to spot regressions.
  • Configure notification preferences (if enabled in your Portal) under Settings > Notifications.
See: Monitoring with Sentinels