The Cloudflare Queues driver

Use the Cloudflare Queues driver to receive a Cloudflare queue in an app deployed to Workers. Cloudflare pushes each batch of messages to the Worker, and each message becomes its event for your subscribers, as with any queue (Queues):

example

// app.jadpoconnections: {    warehouse_queue: {        driver: Driver.cloudflare_queues        queue: "warehouse-events"        dead_letter_queue: "warehouse-events-dead"    }}

The connection. driver: Driver.cloudflare_queues names one queue by queue:, its name in Cloudflare, and dead_letter_queue:, another queue Cloudflare moves a message to once its retries are spent, so none is dropped: both required, both written out, since the consumer is written into the Wrangler file when the app is built, each lowercase letters, digits and dashes, starting and ending with a letter or digit, at most 63 characters (Jadpo's own form, since Wrangler checks no queue name and takes any text). The two differ, and no two connections name one queue. It takes no domains:, since nothing but receiving: reads it, and reaches no host, so the network report lists none. Problems are connection.key_missing, connection.key_invalid and connection.value_invalid; receiving: takes it as it takes a package's connection.
A batch. The Workers host's queue consumer, the Worker's queue, starts the isolate as fetch does and hands each batch to the runtime's seam: the queue's name and each message's body, as text (a text body as sent, bytes as UTF-8 with an invalid sequence replaced, so it parks as unreadable, any other as its JSON), with the times it was handed over. An isolate that can't start, or whose stores aren't current, retries each message after the delay its attempts give, keeping nothing. Each message is read and kept as the tick keeps a fetched one ("How queues are received"), and answered in the batch's order: acknowledged once its event and id commit, as a repeat, or once parked; retried when the store can't take it now, after 10 seconds doubled for each earlier attempt, at most an hour. The host answers each message on its own, never the batch whole, then runs the kept events' deliveries after answering, in waitUntil and handed on past its 30 seconds. Each batch with a message retried logs queue.retried with the queue's name, how many and a code (not_started, not_received, store_unavailable), never a body. A batch from a queue the program doesn't receive is retried whole.
The tick. Nothing is fetched from a queue Cloudflare pushes; the tick still reads again the parked messages an operator retried with jadpo deliveries retry, as for any queue.
Long retries and the dead-letter queue. Every message the consumer retries is one the Worker can't take yet: its isolate hasn't started, its stores are behind while a deploy migrates them, or the store is unavailable; one it can read and can't keep is parked, not retried. The delay doubles up to an hour and the consumer has 30 retries, about 22 hours: long enough to outlast a deploy's migration, and short enough that the message reaches the dead-letter queue inside the 24 hours a queue keeps a message on Workers' Free plan (4 days by default on a paid one, at most 14), every value within Cloudflare's limits (100 retries, a delay of at most 24 hours). The Worker consumes the dead-letter queue too: each message it hands back is parked whole in jadpo_parked_messages as dead_lettered, with its id and its event when they can be read and no event kept, and acknowledged, so jadpo deliveries lists it and retry has the next tick read it again with the build running then; one the store can't park now is retried on the same schedule, 100 times, since a message whose retries are spent there is gone. So one queue's dead-letter queue is no other connection's queue or dead-letter queue (connection.value_invalid). A message whose retry() throws isn't acknowledged by default: the handler throws once every message is answered, so Cloudflare hands again each one not acknowledged.
The Wrangler file. Each such queue is a consumer under queues: { consumers: [] }: its queue, max_batch_size 10 (as many as one receive asks a package's queue for), max_batch_timeout 5 seconds, max_retries 30 and its dead_letter_queue; and the dead-letter queue is a consumer too, with the same batch, 100 retries, Cloudflare's most, and no dead-letter queue of its own. The setting names are Wrangler's own: jadpo build writes them into the Wrangler file as JSON, and Wrangler's deploy --dry-run accepts it without a warning.
Natively. A server off Workers can't receive the binding: it reads no message from the queue, only the parked ones an operator retried. A native deployable that runs Trigger.subscribers, where the app's queues are received, is refused the driver (Targets); a native server reads a Cloudflare queue through a package over its HTTP pull API instead.

Related