Events from outside

Besides the events your actions emit, four sources outside the app produce events: webhooks, a package's own events, polls and queues. Each event is written to the app's outbox and delivered to subscribers like any other, with the same retries.

Webhooks

A connection with a webhook secret generates a route, POST /webhooks/<connection> (Webhooks as events). For each request, the runtime checks the signature, allowing five minutes of clock difference, drops repeats by the provider's event id, and writes the event to the outbox. Subscribers run after the commit.

Every webhook event carries the provider's occurred_at. Webhooks may arrive out of order, which the app handles by ignoring an event older than what it stored; a recipe shows how.

A webhook that needs a reply runs the connection's named action inside the request instead (Webhooks that need a reply). That event is never kept in the outbox, and its reply is kept with its id for a repeat.

An event's id is recorded in jadpo_received_events, and its outbox row and deliveries are written in the same transaction, in the store events: { store: } names or the only store of records. within: is counted from when it arrived. The tick deletes the ids after 30 days.

A package's own events

A package's public events are subscribed to with on:, like the app's own. These are its webhooks' events, and the richer events its own subscribers emit from them, such as Stripe.SubscriptionRenewed. An event without public reaches only the package's own subscribers, and subscribing to one is a compile error.

A package emits only about the outside world, never as a side effect of the app calling it. When calling a package should announce something, the app's action emits its own event there, where a reviewer sees it.

A package's subscriber hears its webhooks' events once for each connection that receives them, and its action calls the API through that connection. What the action emits is kept in the outbox where the webhooks are, and delivered to the app's subscribers of the package's public events (Recipes).

Polls

Use a poll to turn an API without webhooks into events. polls: in a package's provider.jadpo names an operation, a default interval, and the event emitted when the answer changes. The runtime runs a poll only while something subscribes to its event, keeps what it last saw (in the database, like webhook de-duplication) and emits only changes. The connection can change the interval: polls: { latest_rates: 6h } (Polls).

Polls are for feeds that don't depend on the app's data, such as rates or a price list. To check each of the app's own records, write a schedule that calls the package; a package can ship one as a recipe.

The tick calls each poll the app subscribes to through its connection once it is due. What the poll last saw is kept in jadpo_polls beside the outbox, and its event and deliveries are kept with the new digest in one transaction.

Queues, through receiving

receiving reads messages from a queue and turns each into one of the app's events. It is the inbound twin of publishing.

app.jadpo:

// app.jadpoapplication {    connections: {        warehouse_queue: {            package: AmazonSqs.v2012_11_05            credential: { ... }            queue_url: config.warehouse_queue_url        }    }    receiving: {        warehouse_queue: {            type_at: "type"            id_at: "message_id"            events: [Warehouse.OrderShipped, Warehouse.StockLow]        }    }}

Here the warehouse_queue connection reads an Amazon SQS queue. Each message's type field says which event it is, and its message_id field is its id.

• What a queue names. The field giving each message's type (type_at:), the field giving its id (id_at:), and the events it may carry. A queue of one kind lists one event and no type_at:. The events come from another Jadpo app's generated package, or are declared by the app, in a domain's events/ or in its own integration in shared/providers/, each with its wire_name:. The queue's package only knows how to fetch messages.
• Acknowledged after the commit. Each message becomes its event for the app's subscribers. It is acknowledged once the transaction keeping its event and its id commits, and its subscribers are then delivered from the outbox. Repeats are dropped by the message id, stored in the database.
• Never dropped. A message whose type isn't listed is parked for an operator. A message that keeps failing is left to the queue's own dead-letter queue, and parked for an operator. On Cloudflare Queues, the Worker reads the dead-letter queue and parks each message it hands back as dead_lettered. A message that can't be kept is parked whole in jadpo_parked_messages, which jadpo deliveries lists, retries and dismisses.
• How often. Queues are read on the tick, at least every 10 seconds (Queues).

Which kind of connection reads which queue:

• An HTTP API is a package. Amazon SQS and Google Pub/Sub's pull API are packages.
• SNS and Pub/Sub's push are webhooks, through a package with a handshake.
• Anything that needs its own protocol or a platform binding is a built-in driver. Cloudflare Queues is a built-in driver on Workers, its consumer written into the Wrangler configuration, and a package (its HTTP pull API) from a native server. Kafka is a built-in driver in a later release.

The Cloudflare Queues driver (A Workers deployable) is a connection with driver: Driver.cloudflare_queues, queue: and dead_letter_queue:. Its consumer is handed each batch of messages and answers for each one: acknowledged once kept, as a repeat or parked, or retried when the store can't take it yet. A message the Worker can't take yet is retried for about 22 hours, its delay doubling up to an hour, so it outlasts a deploy's migration. The dead-letter queue's consumer parks what reaches it.