Alerts

Alerts help teams react when production behavior crosses a threshold. Use them for signals that need action without waiting for someone to inspect Monitor manually.

​

Useful alert signals

Good alert conditions are tied to an operational question: Use filters and charts to tune the condition before enabling an alert.

​

Before creating an alert

Check that:

• The metric is visible and stable in Monitor.
• The relevant traces have meaningful names and metadata.
• The threshold represents user impact, not temporary noise.
• The alert has an owner and response path.
• The team knows where to inspect evidence after it fires.

For quality issues, pair alerts with evaluators and datasets. For latency or cost issues, make sure model, prompt, tool, and environment metadata is available so responders can isolate the cause.

​

Respond to an alert

When an alert fires:

1. Open Monitor for the alert window.
2. Open the relevant traces and inspect representative spans.
3. Check whether a Behavior captures the repeated pattern.
4. Add important spans to datasets or update evaluators.
5. Start Improve only when the fix belongs in prompt behavior.