How budgets are counted

The full rules for ai: { limits: } and how each call is counted against them. The common case is under Budgets.

• limits: takes per_request:, per_day: { tokens: cost: }, per_caller_per_day: and store:. Token limits are whole numbers written out, 1 to a trillion. cost: is a Money setting, so a setting can hold Money (What a setting can hold), read from its canonical text, "GBP 20.00". ai: without limits:, limits: without per_day:, and per_day: without tokens: are semantic.ai_budget_missing, since the tokens are the hard cap and the cost only an estimate. Every other mistake in it is ai.settings_invalid.

• The main store holds the counts. It is the database store: names; else main (a declared one, or the one an app with none has); else the app's only database. With several databases and none of them main, store: is required.

• The usage table, jadpo_ai_usage, has one row per day (UTC) and window: the app's, each caller's, and each prompt and model's. The table is asked for once ai: checks cleanly, so a migration creates it; an ai: that still has errors asks for none. Its rows hold counts, the prompt's name, the model's and a digest of who the caller is (a principal's key, or address <ip> without one). They never hold a prompt, an answer or an address. The tick, the runtime's regular housekeeping (The tick), deletes rows last changed 62 days before, so jadpo ai usage always has this month and the last.

• Before sending (after the too_long check, and after the call takes its step, so a rerun's steps line up), Jadpo works out the most the call could use: its rendered prompt's estimated tokens (four bytes a token) and max_output:, each times attempts:, since every attempt resends the prompt. That is checked against the request's count, which the invocation keeps (each action a test calls is a request of its own). Then it is taken, all at once, from the day's app row and, for a call with a caller or an address, the caller's row.

• How it is taken. One conditional INSERT … ON CONFLICT DO UPDATE … WHERE statement covers every window that applies. It admits the call only while every row's used and held tokens, and the app's cost, stay within its limit. Without a caller window it updates one row; with one it updates both together. When a row can't take it, the call is over_budget, and nothing changes: no rows or timestamps, and any tentative new rows are rolled back. A repair or a retry that needs more uses the same statement, and adds to the call's held amount only when it succeeds. A store fault is unavailable. A call refused for its budget is answered over_budget with nothing sent, and its step is kept so a rerun answers the same.

• After the call, it is charged what each answer's usage says, attempts included: the built-in driver's usage.prompt_tokens, completion_tokens and prompt_tokens_details.cached_tokens, or a package's where its usage: says. An answer that says nothing is charged the estimate of what was sent and what came back. An attempt with no answer is charged what it sent. One statement charges the app, the caller when there is one, and the prompt and model together, and releases the app's and caller's holds. A count that can't be written leaves every hold taken, so the limit is kept more strictly, not less. Tokens are input plus output; cached tokens are part of the input.

• Every call counts: in jadpo test (so a test can use up a limit, each test having its own database), in jadpo dev whether faked, sandboxed or local, and in production. So the development per_day limit on sandboxed prompts is the app's own limits, counted in the development database. A call given back on a rerun is never charged again.

• ai.embed and ai.stream count too. The most an embedding could use is its texts' estimate, with no answer and one attempt, counted under the prompt name embed. It is charged the answer's usage.prompt_tokens, or that estimate. A stream is charged the whole of the most it could use (its prompt's estimate and max_output:) when it is opened, since what it used is never read from it.

• jadpo ai record isn't counted: it is the developer's own one call, made outside any running app and its store.

• models: { prices: } names a constant of the package's libraries listing, for each model:

  • model:, its name after its provider;
  • currency:, a Currency;
  • input:, cached: and output:, each a price per million tokens in the currency's major unit, to six places;
  • from:, the day the price holds from.

  A call uses its model's row with the latest from: on or before its day. Cost is counted in millionths of the currency, each kind of token rounded up. A model with no price, the built-in driver's included, adds nothing to the estimate, though its tokens still count. A price in another currency than cost:'s can't be compared with it, so the call is misconfigured, and nothing is sent. ai: { prices: } overrides a package's price (Streams, embeddings and the OpenAI and Anthropic packages).

• jadpo ai usage reads jadpo_ai_usage from the store's database: its connection's setting, a running jadpo dev's, or --url. It shows the day, today's tokens against per_day and its calls, and today's cost. Then a table of each prompt and model, with this month's calls, today's tokens, the month's tokens and the month's cost (– without a price). The footer has today's tokens and share of the limit, or a failure footer once the day's tokens are spent. --json gives the same with input, output and cached tokens. A database that can't be read is ai.usage_unavailable.