Receiving from outside

A provider tells your app that something happened in one of two ways: it sends a webhook, or it puts a message on a queue. Either way, the message reaches your app as an event, and your subscribers handle it as they handle your app's own events (Subscribers). You don't write a route for either.

Webhooks as events

A webhook is a request a provider such as Stripe sends your app when something happens there, such as a payment going through. The package declares the provider's events and how it signs them, so your app only gives the connection a webhook secret. With a secret set, Jadpo:

1. generates the route POST /webhooks/<connection> in the app's webhooks API (Webhooks). You can change its path, but you can't switch it off;
2. checks the signature over the raw body, and refuses a timestamp more than five minutes old;
3. drops repeats, by the provider's event id;
4. writes each event to the app's outbox, the table that holds events until their subscribers run, so subscribers run after the transaction commits. The package marks whether each event is public, which decides whether your app's subscribers hear it.

A subscriber then takes the event as it takes one of your own:

// domains/payments/subscribers/payment_succeeded.jadposubscriber payment_succeeded {    on: Stripe.PaymentIntentSucceeded    run: mark_order_paid(event)}

The signature schemes are built in:

• SignatureScheme.stripe;
• SignatureScheme.standard_webhooks, used by Svix and others;
• SignatureScheme.hmac_sha256, with the header that holds the signature named;
• basic authentication, for Postmark, which signs nothing;
• Twilio's HMAC-SHA1;
• AWS SNS's certificates.

Every event carries occurred_at, when the provider says it happened. Providers can deliver events out of order. Your app handles that by ignoring an event older than what it has stored, and a recipe shows how.

A package declares what its provider sends in provider.jadpo's webhooks: block. check: names the signature scheme (SignatureScheme.stripe, .standard_webhooks, .hmac_sha256 with header:, .basic, .twilio, .slack or .sns). event_id:, event_type: and occurred_at: say where in the body each of those is, and handshake: names a handshake when the provider needs one. Each event is a type in the package's events/ folder, with its wire_name: (the name the provider gives that kind of event) and occurred_at: Instant. A connection gives webhooks: { secret: }. The runtime answers the generated route and keeps the events in the outbox for your subscribers. The details are under How webhooks are received and Webhook replies, handshakes and further signature schemes.

Here is a package for Acme Pay, which signs its webhooks as Stripe does, with one event:

/// Acme Pay's API, which signs its webhooks as Stripe does.provider {    protocol: Protocol.json_http    hosts: {        live: "https://api.acme-pay.example"        sandbox: "https://sandbox.acme-pay.example"    }    credential: CredentialScheme.bearer    webhooks: {        check: SignatureScheme.stripe        event_id: "id"        event_type: "type"        occurred_at: "created"    }}

/// A payment was taken.public type PaymentSucceeded = Event {    wire_name: "payment.succeeded"    occurred_at: Instant    payment_id: PaymentId {        wire_name: "paymentId"    }    pence: Int {        min: 0    }}

A connection to it with webhooks: { secret: config.payments_webhook_secret } gets the route POST /webhooks/payments. jadpo routes lists it as generated webhook, with signature as its authentication. A subscriber takes its events with on: AcmePay.PaymentSucceeded. The body's paymentId and pence become the event's fields, and its id, type and created are read from the places webhooks: names. Every other member of the body is left out, as it is from a provider's answer to a call.

Webhooks that need a reply

Some providers wait for an answer in the webhook's own response: Twilio waits for call instructions, and Slack for a message. For these, the package's event names its reply type with reply:, a type in the package's types/ folder. The connection's webhooks: { replies: { } } names the app's action that answers, keyed by the event's name in lower_snake_case:

// packages/twilio/v2010/events/incoming_call.jadpopublic type IncomingCall = Event {    wire_name: "voice.incoming"    reply: CallReply    from: Text    to: Text}// app.jadpophone: {    package: Twilio.v2010    domains: [support]    credential: config.twilio_credential    webhooks: {        secret: config.twilio_auth_token        replies: {            incoming_call: support.answer_call        }    }}

The action takes the event and returns the reply type. It runs inside the webhook's request, once the signature is checked, and it can read and write like any action. Its reply is sent as the package's JSON and kept with the event's id, so a repeat of the same webhook gets the stored reply and the action doesn't run again. Twilio's own XML format waits for an XML byte format in a later Jadpo release.

An event that needs a reply isn't kept in the outbox, so no subscriber hears it: subscribing to one is subscriber.event_invalid.

What a reply may show. The action runs as the system, with no tenant, and its reply goes back to whoever sent the webhook. So Policy treats the reply as an output to Access.public: everything it carries, or that decided it, must be something anyone at all may read. That includes whether records are there, because the system can find every record. A reply that counts or tests for records only the system may read is refused, and so is a failure raised because a record is there or not. A failure is sent back the same way as a reply, so it is held to the same rule. The details are under Webhook replies, handshakes and further signature schemes.

Here Acme Voice signs its webhooks as Twilio does, over a form body, and waits to be told what a caller hears:

use supportapplication {    connections: {        phone: {            package: AcmeVoice.v1            domains: [support]            credential: config.phone_key            host: config.phone_address            webhooks: {                secret: config.phone_auth_token                replies: {                    incoming_call: support.answer_call                }            }        }    }}

/// Acme Voice's API, which signs its webhooks as Twilio does.provider {    protocol: Protocol.json_http    credential: CredentialScheme.bearer    webhooks: {        check: SignatureScheme.twilio        event_id: "CallSid"        event_type: "CallStatus"    }}

/// What a caller hears.public type CallReply = Object {    spoken: Text    hang_up: Bool {        wire_name: "hangUp"    }}

/// A call is ringing, and waits to be told what to do.public type IncomingCall = Event {    wire_name: "ringing"    reply: CallReply    occurred_at: Instant    call_sid: Text {        wire_name: "CallSid"    }    from: Text {        wire_name: "From"    }}

use support/entities { PhoneLog }use shared/providers/acme_voice/// Answers a call Acme Voice rings through, noting who rang.action answer_call(event: AcmeVoice.IncomingCall) -> AcmeVoice.CallReply {    var noted_call: PhoneLog = PhoneLog.create {        set: {            call_sid: event.call_sid            caller: event.from            rang_at: event.occurred_at        }    }    return AcmeVoice.CallReply {        spoken: "Thanks for calling."        hang_up: noted_call.caller != ""    }}

A ringing call is answered 200 with {"spoken":"Thanks for calling.","hangUp":true}. If Acme Voice sends the same webhook again, it gets the same reply without the action running again.

Handshakes

Some providers check an address before they send to it: AWS SNS, Slack and Microsoft Graph. The package names the handshake in webhooks: (handshake: WebhookHandshake.sns, .slack or .microsoft_graph), and the runtime answers it, so you write nothing for it. New handshakes come with Jadpo releases.

• SNS sends a subscription confirmation, signed as its messages are and from the connection's topic. The runtime confirms it by visiting its address on AWS's SNS host.
• Slack sends a challenge, answered once its signature is checked (SignatureScheme.slack).
• Microsoft Graph sends a validation, which Graph doesn't sign. Its token is answered as plain text.

SNS's own messages are checked against the certificate the runtime fetches from AWS's SNS host over HTTPS, never from a private address, and keeps. A handshake signed in a way its provider doesn't sign is package.provider_invalid. The details are under Webhook replies, handshakes and further signature schemes.

A package receiving an SNS topic's messages:

/// AWS SNS, which signs each message with a certificate of AWS's own.provider {    protocol: Protocol.json_http    credential: CredentialScheme.bearer    webhooks: {        check: SignatureScheme.sns        event_id: "MessageId"        event_type: "Message.type"        handshake: WebhookHandshake.sns    }}

/// An order was shipped.public type OrderShipped = Event {    wire_name: "order.shipped"    occurred_at: Instant    shipment: Shipment {        wire_name: "Message"    }}

The connection's secret is the topic's ARN, the name AWS gives it. SNS's Message member holds JSON written as text, and is read as the JSON it holds, so event_type: and the event's fields can reach inside it.

Queues

A queue is the other way a provider can send your app messages. Your app reads a queue directly, through the receiving capability, the inbound twin of publishing (Capabilities). A queue names the field that gives each message's type, the field that gives its id, and the events it may carry. A queue that carries one kind of message lists one event and no type_at:.

// app.jadporeceiving: {    warehouse_queue: {        type_at: "type"        id_at: "message_id"        events: [Warehouse.OrderShipped, Warehouse.StockLow]    }}

Each message becomes its event for your app's subscribers:

• The events come from another Jadpo app's generated package, or your app declares them, in a domain's events/ or its own integration, each with its wire_name:. The queue's package or driver only knows how to fetch messages.
• A message is acknowledged only once the transaction that keeps its event, and hands it to the subscribers, has committed. Repeats are dropped by message id, which is stored in the database.
• A message whose type isn't listed is parked for an operator, never dropped. A message that keeps failing is left to the queue's dead-letter queue (a second queue that holds messages that couldn't be delivered) and parked. jadpo deliveries lists what is parked.

receiving: takes type_at:, id_at: and events:. An event of your app's gives a wire_name:, and a package provides the capability in provides/receiving.jadpo. The details are under How queues are received.

Which queues are packages. The usual rule decides: a service reached over an HTTP API is a package, and anything that needs its own protocol or a platform binding is a built-in driver. So:

• Amazon SQS and Google Pub/Sub pull are packages. The SQS package makes its calls in AWS's JSON protocol, signed with AWS Signature Version 4.
• SNS and Pub/Sub push are webhooks, through a package with a handshake.
• Cloudflare Queues is a built-in driver on Workers, a binding whose consumer is generated into the Wrangler configuration. From a native server it is a package, over its HTTP pull API. The driver is a connection with driver: Driver.cloudflare_queues, queue: and dead_letter_queue: (The Cloudflare Queues driver).
• Kafka becomes a built-in driver in a later release.

Senders with no package

A system that signs its webhooks but has no package gets your app's own integration in shared/providers/, written in the package format, by the golden rule. A partner your company has an agreement with is a Service, calling an ordinary route with a service key (Service credentials).