Monitoring thresholds

A threshold says when something your app does is too slow, too frequent or failing too often, such as a route's p95 duration. You declare thresholds in your app's source, in a monitoring file for each domain, so they are reviewed with the code and versioned with each build rather than configured in a monitoring tool. There are two kinds, because only one can be judged by a single process.

Thresholds per request

These judge one request: its duration, its number of queries, whether it failed. The runtime keeps each request's spans in memory until the request ends, then sends:

• the full trace when a threshold was crossed, the request failed or faulted, or its path is new to this process;
• otherwise metrics only, plus one sample trace for each path each minute, so there is always a normal request to compare a bad one with.

telemetry: { detail: } sets both (Telemetry): full_traces: names the first list and samples: the second.

A process that crashes loses the requests it hasn't sent yet, and long jobs send on a timeout. This keeps the volume low without losing anything structural: the compiler knows every path, so the number of per-path metrics can't grow without bound.

Thresholds across requests

These judge many requests together: a p95 over a window, an error rate, the oldest waiting delivery. One instance can't judge them for a fleet, so the compiler generates rules for your monitoring tool into build/monitoring/: OpenSLO and Prometheus rules, and other tools' only when asked. The build manifest carries the same thresholds, so a tool that reads it needs no generated rules. The rules are applied when you deploy, and a change made in the tool is overwritten, as with Terraform.

Setting and changing them

• Starting values come from what the compiler knows, such as the queries on each path from the query report, then from production once the app has run. Local timings aren't production timings, so locally the compiler reports change on the same fixtures: "30% slower", "2 more queries".
• An agent proposes changes from observed data as an ordinary change to the source, through the MCP server (The developer console and MCP).
• Loosening needs a person. Raising or removing a threshold is how an agent would silence an alert, so it needs an acceptance and appears in the approval report. Tightening one doesn't.
• Per build and per flag value. Thresholds belong to a build. Each flag value takes its own branch, so it has its own path and metrics, with no extra tagging.

OpenTelemetry stays the transport; only what is sent changes.