# Monitoring thresholds

A threshold says when something your app does is too slow, too frequent
or failing too often, such as a route's p95 duration. You declare
thresholds in your app's source, in a monitoring file for each domain, so
they are reviewed with the code and versioned with each build rather
than configured in a monitoring tool. There are two kinds, because only
one can be judged by a single process.

## Thresholds per request

These judge one request: its duration, its number of queries, whether
it failed. The runtime keeps each request's spans in memory until the
request ends, then sends:

- the full trace when a threshold was crossed, the request failed or
  faulted, or its path is new to this process;
- otherwise metrics only, plus one sample trace for each path each
  minute, so there is always a normal request to compare a bad one with.

`telemetry: { detail: }` sets both ([Telemetry](telemetry.md)):
`full_traces:` names the first list and `samples:` the second.

A process that crashes loses the requests it hasn't sent yet, and long
jobs send on a timeout. This keeps the volume low without losing
anything structural: the compiler knows every path, so the number of
per-path metrics can't grow without bound.

## Thresholds across requests

These judge many requests together: a p95 over a window, an error rate,
the oldest waiting delivery. One instance can't judge them for a fleet,
so the compiler generates rules for your monitoring tool into
`build/monitoring/`: OpenSLO and Prometheus rules, and other tools' only
when asked. The build manifest carries the same thresholds, so a tool
that reads it needs no generated rules. The rules are applied when you
deploy, and a change made in the tool is overwritten, as with Terraform.

## Setting and changing them

- **Starting values** come from what the compiler knows, such as the
  queries on each path from the query report, then from production once
  the app has run. Local timings aren't production timings, so locally
  the compiler reports change on the same fixtures: "30% slower",
  "2 more queries".
- **An agent proposes changes** from observed data as an ordinary
  change to the source, through the MCP server
  ([The developer console and MCP](the-developer-console-and-mcp.md)).
- **Loosening needs a person.** Raising or removing a threshold is how
  an agent would silence an alert, so it needs an
  [acceptance](acceptances.md) and appears in
  the approval report. Tightening one doesn't.
- **Per build and per flag value.** Thresholds belong to a build. Each
  flag value takes its own branch, so it has its own path and metrics,
  with no extra tagging.

OpenTelemetry stays the transport; only what is sent changes.