How an export's run is kept
On this page
This topic is what a file helper's write run does beyond a query's run
(How a run is kept). It covers csv.write,
tsv.write, json_lines.write and json.write's file form. Where
Batches and
Data formats
leave a choice open, the run takes the most cautious one. Data formats
also says how each row is written.
Where it is kept
As for batches.each: in the store from:'s query reads, which
finished:'s action writes too.
Each page is stored after its page commits
No outside call goes out while a transaction is open (The seam every outside call goes through), so each page goes in three steps:
- A transaction locks the run's row and reads, as the system, the page after the position that committed. It commits, having written nothing.
- With no transaction open,
each:works out each item's row, and the page's rows are put in the store as a piece of the file of their own, under a key made from the run's id and the piece's number. - A second transaction locks the run's row again, holds it to the position the page was read from, and commits the problems and the position after the page: the next page's cursor, and the pieces kept so far with their bytes and rows.
A run stopped between the put and that commit puts the piece again when it resumes, under the same key, so the file never holds a row twice.
each: is pure
each:'s body is one expression, the row, worked out from the item and
what the run saved. attempt can only start a statement, so the body
can't fail. What can stop it is a fault, an overflow among them. That
makes the item a problem, by its place in the walk, with the fault's
classification as its rule, and its row isn't written.
What it saves
The run saves each of the action's parameters and variables that
each: reads, once, as an entity field holds a value
(type.batch_capture_not_storable otherwise), beside name: and
format:. finished:'s captures and key: are saved as for
batches.each.
Finishing
Once every page is written, the pieces are joined into the file, with no
transaction open, by the store's join
(Files).
finished: then runs in a transaction of its own, given the
ExportSummary, whose file holds the store, the key, name:, the
size and the type. The file's key is made from the run's id, so a run
that joins again finds the file whole.
Cancelling
A cancel ends the run at its next page, as for a query's run. Once that
end commits, the file's pieces, and the file if they were joined, are
dropped. If the store can't be reached then, they stay. Deleting files
nothing points at comes with the rest of the files capability.
A store with no room
A store with no room pauses the run, as one that can't be reached does. What committed stands, and the tick carries the run on.
Markdown for agents: /docs/export-batch-runs.md.Built from Jadpo main at a707efb.