> ## Documentation Index
> Fetch the complete documentation index at: https://docs.trymaitai.com/llms.txt
> Use this file to discover all available pages before exploring further.
# Logging (Bring Your Own Inference)
> Run inference with your own provider client, then send Maitai the raw request/response with maitai.log() for monitoring, debugging, and Test Sets, without routing inference through Maitai.
Already running inference with your own provider client (OpenAI, Anthropic, Gemini, or any OpenAI-compatible API) and don't want to move it behind Maitai's inference service? Use **`maitai.log()`** to hand Maitai the request/response pair after each call. Maitai normalizes the native payloads by `provider` and runs them through the **same pipeline as proxied traffic**, so your existing client keeps doing inference while you still get Maitai observability: [Sentinels](/observe/sentinels/overview), [Sessions](/observe/agent_sessions), [request history](/observe/request_overview), and data for [Test Sets](/test/test_sets/creation).
Maitai never calls the model on this path, you do. That means **no provider keys are needed** by the Maitai client (only your `MAITAI_API_KEY`), and inference latency is entirely yours. Logging is fire-and-forget and is engineered to **never raise into your application**.
## How it works
Call OpenAI / Anthropic / Gemini (or any OpenAI-compatible API) exactly as you do today, with your own keys.
Pass the native `request` and `response` along with the `intent`, `application`, and `provider`. The call returns immediately; the send happens on a background thread (sync) or task (async).
The native payloads are converted into Maitai's OpenAI-compatible schema based on `provider` and stored as **PROD** traffic (inference location `CLIENT`).
The logged pair flows through the same pipeline as Maitai-routed requests, so Sentinels evaluate it, Sessions thread it, and it's available for debugging and building Test Sets.
## Quickstart
Run inference with your own client, then log the pair. Reusing the same `request` dict you pass to your provider keeps the logged inputs perfectly in sync with what you sent.
```python Python (sync) theme={null}
import maitai
from openai import OpenAI
maitai_client = maitai.Maitai() # only needs MAITAI_API_KEY
openai_client = OpenAI() # your own client + OPENAI_API_KEY
request = {
"model": "gpt-4o",
"messages": [{"role": "user", "content": "Summarize this text..."}],
}
# You run inference, not Maitai.
response = openai_client.chat.completions.create(**request)
# Fire-and-forget: returns immediately, never raises.
maitai_client.log(
intent="SUMMARIZATION",
application="demo_app",
provider="openai",
request=request,
response=response, # SDK objects are serialized automatically
session_id="YOUR_SESSION_ID",
metadata={"source": "byo_inference"},
)
```
```python Python (async) theme={null}
import maitai
from openai import AsyncOpenAI
maitai_client = maitai.MaitaiAsync() # only needs MAITAI_API_KEY
openai_client = AsyncOpenAI() # your own client + OPENAI_API_KEY
request = {
"model": "gpt-4o",
"messages": [{"role": "user", "content": "Summarize this text..."}],
}
response = await openai_client.chat.completions.create(**request)
await maitai_client.log(
intent="SUMMARIZATION",
application="demo_app",
provider="openai",
request=request,
response=response,
session_id="YOUR_SESSION_ID",
metadata={"source": "byo_inference"},
)
```
## Provider examples
Send the provider's **native** payloads, Maitai converts them server-side. `request` and `response` accept either plain dicts or the provider SDK's objects (they're serialized for you).
```python OpenAI-compatible theme={null}
# provider="openai" covers OpenAI and any OpenAI-compatible API
# (Groq, Together, vLLM, ...). Point your own client wherever you like.
import maitai
from openai import OpenAI
maitai_client = maitai.Maitai()
openai_client = OpenAI()
request = {
"model": "gpt-4o",
"messages": [{"role": "user", "content": "Summarize this text..."}],
}
response = openai_client.chat.completions.create(**request)
maitai_client.log(
intent="SUMMARIZATION",
application="demo_app",
provider="openai",
request=request,
response=response,
session_id="YOUR_SESSION_ID",
)
```
```python Anthropic theme={null}
import maitai
import anthropic
maitai_client = maitai.Maitai()
anthropic_client = anthropic.Anthropic()
request = {
"model": "claude-sonnet-4-20250514",
"max_tokens": 1024,
"system": "You are a helpful assistant.",
"messages": [{"role": "user", "content": "Summarize this text..."}],
}
message = anthropic_client.messages.create(**request)
maitai_client.log(
intent="SUMMARIZATION",
application="demo_app",
provider="anthropic",
request=request,
response=message, # native Anthropic Message; tool_use + thinking are mapped
session_id="YOUR_SESSION_ID",
)
```
```python Gemini theme={null}
import maitai
from google import genai
maitai_client = maitai.Maitai()
genai_client = genai.Client()
request = {
"model": "gemini-2.0-flash",
# Structured contents normalize cleanly into messages.
"contents": [
{"role": "user", "parts": [{"text": "Summarize this text..."}]}
],
}
response = genai_client.models.generate_content(**request)
maitai_client.log(
intent="SUMMARIZATION",
application="demo_app",
provider="gemini",
request=request,
response=response, # native GenerateContentResponse
session_id="YOUR_SESSION_ID",
)
```
## Parameters
The Intent / action type for the request (also called `action_type`). Scopes Sentinels, configuration, and quality tooling.
The Application reference name (shown in the Portal). Groups your traffic.
Source provider for the pair, used to normalize the native payloads. One of `openai`, `anthropic`, or `gemini`. Use `openai` for any **OpenAI-compatible** API (Groq, Together, vLLM, etc.) where payloads are already in OpenAI shape.
The provider's native request payload, the dict you sent to your client, or the SDK request object. Serialized automatically.
The provider's native response, the SDK response object (e.g. OpenAI `ChatCompletion`, Anthropic `Message`, Gemini `GenerateContentResponse`) or its dict form. Serialized automatically.
Optional but recommended. Groups related requests into a [Session](/observe/agent_sessions). If omitted, Maitai generates one server-side.
Optional caller-supplied identifier for correlating the logged request with your own systems.
Optional end-user identifier stored with the request.
Optional metadata tags stored with the request for filtering and debugging.
Optional per-call client-side sampling rate between `0.0` and `1.0`. Overrides the default (see [Sampling and kill-switch](#sampling-and-kill-switch)). The surviving rate is recorded so server-side aggregates can correct for sampling.
Optional latency metrics. Provide a `RequestTimingMetric` (`from maitai.models.metric import RequestTimingMetric`) to record response time.
Epoch seconds when you issued the request. Required for latency to be recorded.
Epoch seconds when the response completed. Required for latency to be recorded.
Epoch seconds of the first streamed chunk, used to derive time-to-first-token.
Timing is only recorded when **both** `time_request_start` and `time_request_end` are set; partial metrics are ignored.
## Behavior and guarantees
* **Non-blocking.** `maitai_client.log(...)` returns immediately. Sync clients send on a background thread; async clients schedule a task.
* **Never raises into your code.** Bad inputs, network errors, and serialization failures are swallowed and reported to Maitai's internal error metric. Instrumentation can't take down your request path.
* **No inference, no inline corrections.** Maitai does not call the model and does not apply corrections here (corrections require server-side inference, see [Model Request](/sdk/chat)). [Sentinels](/observe/sentinels/overview) still evaluate the stored traffic asynchronously.
* **Stored as PROD / CLIENT.** Logged pairs are recorded as production traffic with the inference location marked `CLIENT`, so they're indistinguishable downstream from Maitai-routed requests for monitoring and Test Sets.
## Sampling and kill-switch
Dial volume without touching call sites using environment variables:
| Variable | Default | Effect |
| ------------------------ | ------- | ------------------------------------------------------------------------------------------ |
| `MAITAI_LOGGING_ENABLED` | `true` | Set to a falsey value (`0`, `false`, `no`, `off`, empty) to disable all logging. |
| `MAITAI_LOG_SAMPLE_RATE` | `1.0` | Fraction of events to send (`0.0`–`1.0`). A per-call `sample_rate` overrides this default. |
Short-lived processes (scripts, lambdas) don't need to do anything special, queued logs are drained automatically at interpreter exit, so in-flight events aren't dropped.
## Raw HTTP (any language)
The `maitai.log()` helper ships in the Python SDK. From Node, Go, or any other stack, `PUT` the same payload to the endpoint directly:
```
PUT https://api.trymaitai.ai/chat/completions/log
```
The HTTP body uses the underlying field names: `intent` maps to `action_type` and `application` maps to `application_ref_name`. `request` and `response` are the provider's native payloads as JSON.
```bash cURL theme={null}
curl --request PUT \
--url https://api.trymaitai.ai/chat/completions/log \
--header "Content-Type: application/json" \
--header "x-api-key: $MAITAI_API_KEY" \
--data '{
"provider": "openai",
"action_type": "SUMMARIZATION",
"application_ref_name": "demo_app",
"session_id": "YOUR_SESSION_ID",
"metadata": { "source": "byo_inference" },
"request": {
"model": "gpt-4o",
"messages": [
{ "role": "user", "content": "Summarize this text..." }
]
},
"response": {
"id": "chatcmpl_example",
"object": "chat.completion",
"created": 1730000000,
"model": "gpt-4o",
"choices": [
{
"index": 0,
"message": { "role": "assistant", "content": "Here is the summary..." },
"finish_reason": "stop"
}
],
"usage": { "prompt_tokens": 42, "completion_tokens": 18, "total_tokens": 60 }
}
}'
```
```javascript Node (fetch) theme={null}
const request = {
model: "gpt-4o",
messages: [{ role: "user", content: "Summarize this text..." }],
};
// You run inference with your own client.
const response = await openai.chat.completions.create(request);
// Fire-and-forget: don't await if you don't want to block.
fetch("https://api.trymaitai.ai/chat/completions/log", {
method: "PUT",
headers: {
"Content-Type": "application/json",
"x-api-key": process.env.MAITAI_API_KEY,
},
body: JSON.stringify({
provider: "openai",
action_type: "SUMMARIZATION",
application_ref_name: "demo_app",
session_id: "YOUR_SESSION_ID",
request,
response,
metadata: { source: "byo_inference" },
}),
}).catch(() => {}); // logging should never break your request path
```
On success the endpoint returns `{ "success": true }`. It returns `400` for caller mistakes, an unsupported `provider`, or a missing `action_type` / `application_ref_name`.
For **anthropic** and **gemini**, send each provider's native request/response shape (Anthropic Messages API, google-genai `GenerateContent`). Maitai maps finish reasons, `tool_use`/`function_call` to tool calls, usage, and Anthropic `thinking` into its OpenAI-compatible schema.
## When to use this vs. routing through Maitai
Logging is the right fit when you must keep inference on your own infrastructure or client but still want full observability. If you can route inference through Maitai instead (`server_side_inference=true`, the default), you additionally unlock server-side features like automatic corrections and `input_safety_score`, see [Model Request](/sdk/chat).