Resource hierarchy
- Company: the top-level tenant for your team.
- Application: a product/service you’re integrating (ex:
checkout,support_bot). - Intent: a specific action type within an Application (also called an ApplicationAction).
Cross-application grouping
- Intent Group: a company-wide grouping of Intents by shared
action_type(so you can manage monitoring/testing across many apps at once).
Requests and sessions
- Request: one chat completion (messages to model output).
- Session: a set of requests grouped by
session_id(highly recommended). Sessions are the main way to debug multi-turn behavior in the Portal.
Agents
An Agent is a configurable AI runtime that can plan, execute capabilities, and synthesize results. Each agent has an Execution Mode, chosen when you build it:- Reasoning (
reasoning): a multi-step orchestration loop that plans actions, processes results, and iterates until ready to respond. - Route (
route): single-step capability selection, where each request goes straight to one capability.
Quality and testing objects
- Sentinel: an intent-group-level quality check (what to evaluate, severity, optional correction guidance).
- Evaluation: the result of running one or more Sentinels against a request.
- Test Set: a type-less collection of
input + expected outputitems (from requests, workflow runs, agent tasks, or authored by hand) for regression testing. - Test Run: executes a Test Set against a chosen consumer (a model, workflow, or agent) and scores it with evaluation criteria.
- Evaluation Criteria: reusable checks (deterministic, LLM-judge, or agent-trajectory) that score a Test Run.
Auto-creation on first request
When Maitai receives a request with a newapplication / intent pair for your company, it will create the corresponding Application/Intent resources so they show up in the Portal automatically.