How a run is kept

This topic is the detail of how a batches.each run is stored, resumed and checked. Most apps never need it: Batches says what a batch does. The rules for a file's run and an export's run build on this one (How a file's run is kept, How an export's run is kept).

A batch is a page

Each transaction is one page of from:, at the size its query's page: reads. So 60 items in pages of 3 commit 20 times.

Where a run is kept

A run is kept in the store from:'s query reads, in three system tables:

• jadpo_batch_runs has one row per run: which call started it, the shape it was started under, its key:, what it saved, the tenant it is limited to, how deep in a chain of events it started, the cursor of the next page, its counts, its state and its runner's lease;
• jadpo_batch_problems holds each problem;
• jadpo_batch_cancels holds each cancel asked for.

Only a store some batch walks has these tables. Starting a run writes that store, and each:'s and finished:'s actions may write no other (store.action_spans_stores, One store per action), so an item's writes and the run's position always commit together. A run always has its query's store, so it never needs events: { store: }.

Starting

The run's row is written in the starting action's transaction, and the run is left to start after the response. An action that rolls back leaves no row, so nothing runs. A start needs no outbox row: the run's row is as durable, and the tick picks up a start a host dropped, as the outbox's sweep picks up a delivery.

A start with a key: that finds a run of the same call with the same key, compared by its contract JSON, in the same tenant, gives back that run, whatever its state and whatever else the call was given. A start without a key: is always a new run.

In an app with a tenant, key: is unique for each call, run's tenant and key, so two tenants' runs with the same key never meet. The row keeps the tenant: all zeros when the run reaches no tenant's records, or the app has no tenant. A later start never finds a row with no tenant kept, since nothing says which tenant its key was given in. A retry then starts a new run, rather than one tenant being handed another's.

Saved values

each:'s and finished:'s arguments may be the item (or the summary), a field inside it, or a value that reads nothing of it. That last kind is worked out as the run starts and saved with it. An argument that works something out of the item is type.batch_each_invalid: that work belongs in the action.

from:'s query arguments are saved as a request carries them, so an Input may be among them. What each: and finished: capture, and key:, are saved as an entity field holds a value, so an Input there is type.batch_capture_not_storable. A secret is never saved.

The saved values are a JSON array of each value's own JSON (its type's contract JSON), with null for none. They are kept with the encoding they were written in (jadpo_batch_runs.encoding) and read with the standard JSON reader, a number keeping the digits it was written with.

Which call started a run

A run names the batches.each call that started it by what the call is, in identity keys that survive a rename (Stable identity): the action starting it, the query from: walks, and the action each: runs, as in action check_stock from query products_to_check each action check_product. Adding or moving another call leaves the name as it was. A second call of the same three in one action adds its place among them ( #2).

The run's shape, what it saves and calls, is kept as canonical type text, naming declared types by their identity keys, with a digest. So a release that changes the compiler's own internals leaves every running batch's shape as it was.

Problems

An item whose action fails is undone on its own and becomes a problem. Its row is its place in the walk, counting from 1. Its column is empty, since a query's item has no columns (the file helpers fill it). Its rule is the failure's code.

An item whose action faults with a defect or an overflow is a problem too, undone on its own as a failure is. Its rule is the fault's classification, internal_fault or int_overflow.

Some faults belong to no single item: a store that can't be reached, a deadline, or a change made by another transaction. These undo the whole page. A page undone by another transaction's change runs again, up to three times; otherwise the run waits for the tick.

Items that make outside calls

When each:'s action makes outside calls, itself or through an action it calls, each item is a batch of its own, so no call goes out while a transaction is open (All-or-nothing actions). Each item goes like this:

1. The run's place is read in a statement of its own, locking nothing.
2. The item is read in a snapshot: a query's page of one item, or a file's next row. Its first call closes the snapshot.
3. Its action runs as the system, as a subscriber's delivery does.
4. In the transaction its writes are in, the run's row is locked and held to the position the item was read at. The item's problem, if it failed, and the run's next position commit with its writes.

A query's run is then at the item's own cursor, and a file's run just after its row, so a run resumed after a crash meets the same item at the same place. Each call's key is made from the run and that place (the item's cursor, or where its row begins in the file), so the item's calls go out again under the keys they were first sent with. A change by another transaction runs the item again, up to three times, as it would a page.

A finished: whose action makes outside calls runs the same way: its calls are keyed from the run, and the run's end commits with its writes.

An unkeyed call that may have acted (outcome_unknown) is the item's problem, and is never sent again. One lost with its runner waits for the step record that several changing calls keep, as a subscriber's does (Several changing calls in one action).

Who runs it

Each page's read, its items and finished: act as the system (Background work and tenants), and read the clock as their page began. Each page begins at repeatable read when each:'s or finished:'s action loads a record, and at read committed otherwise.

A run keeps how deep in a chain of events the action that started it was, and its items run at that depth. So what they emit is one deeper than the event that started the run, and a loop through a batch stops at 16 deep, as any loop of events does. Each item's action, and finished:'s, may emit as many events as one action may, 1,000, however many the items before it in the page emitted.

Finishing

finished: runs once after the last page, in a transaction of its own. It is given the ImportSummary with every problem in row order, and the run becomes finished in the same transaction. If its action fails, what it wrote is undone and the run is failed with the failure's code. A cancelled run doesn't run finished:.

Cancelling

batches.cancel writes a row of its own in the calling action's transaction and never touches the run's row, so it never waits for the page in hand. The next page finds it and ends the run cancelled. Until then batches.status says running.

Status

batches.status reads the counts and the state in the calling action's transaction. A BatchRun that no store keeps is a fault. Both batches.status and batches.cancel find a run by its id alone, so they need the app's runs in one store.

The estimated total

A query's run counts its query as its first page is read, in that page's transaction, as the system limited to the run's tenant. It runs the query's statement with no page limit or cursor, so the count is what the walk would meet if nothing changed. Later pages keep that count.

A file's run works its estimate out with each page: the items done so far, scaled by the file's size over how far into the file they end, and never fewer than are done. For a gzip file, the size is what it is unpacking to, at the rate so far.

estimated_total is none until then. Once the run has finished it is the items done, and once it failed or was cancelled, the last estimate. It is kept in jadpo_batch_runs.estimated_total. A count the store refuses leaves the run without one rather than stopping it. A query whose records Jadpo checks again after its statement, such as one with geo. conditions (Geospatial queries), has none.

An order an update can change

A from: whose query orders by a field that can be updated, before any id, is type.batch_order_updatable. It is reported at the from: and declined on the action that starts the run. A field can't be updated when it is id, is marked immutable: true, or is generated with Generated.on_create. What comes after id, or after a unique field that can't be updated, is passed over, since it never moves a record.

The check doesn't look at which updates the program makes, so adding one elsewhere never makes a run's order wrong after the fact. A policy with no update: doesn't count either, since adding one is a one-line change.

Carried on over time

A runner holds its run by a lease lasting what is left of its budget plus a minute (Running background work). The lease is renewed with each page it commits. The runner goes page after page while its budget lasts, then gives its lease back, and a fresh invocation carries on from the position that committed. If the budget ends in the middle of a page, that page is dropped and nothing it did is kept. A run that couldn't commit a page in a whole budget gives its lease back and waits for the tick, which picks up every running run whose lease has lapsed.

What a deploy changes

A deploy may change a run's call: its query, its actions or what it saves. Or the run's saved values may be in an encoding this release doesn't read, such as a later release's after a rollback. Either way, the run ends failed and says why ("its saved values were written in encoding 2, which this release doesn't read"), rather than reading what it saved as something else.

A run with no encoding recorded, whose call and shape match, is read with its values as encoding 1's. A run kept before calls were named by identity has a name no call now gives, and ends failed because "it was started by a release that named its batches.each call another way". To list such runs before a deploy, run jadpo runs unfinished (The jadpo command).

Kept 30 days

In each hourly cleanup pass, in every store that keeps batch runs, a run that ended (finished, failed or cancelled) 30 days ago or more is deleted, with its problems and its cancel. So are cancels as old for runs no store keeps, which happens when a cancel is asked of a BatchRun a field held after its run was deleted. A run still going is never deleted, however old.

Names

batches is a standard namespace, so, as with every standard namespace, no variable may be named batches.

Not built yet

A page type allowing more than the 1,000 items of one batch (Limits and runtime faults) isn't split into smaller batches yet; PageSize stops at 100. batches.status and batches.cancel in an app whose runs are kept in no store, or in more than one, are toolchain.not_built_yet.