Dashboard
WebhooksWebhook Deduplication & Retry Policy | BotSubscription
Webhooks

Deduplication & Retries

BotSubscription uses an at-least-once delivery model: we attempt to deliver every webhook event, and we retry transient failures until your server acknowledges receipt or the retry window is exhausted.

If you are integrating outbound webhooks to synchronize events with your own custom database, backend application, or external services, handling retry patterns and implementing deduplication protects your server from processing the same event multiple times.

Response Semantics

Our outbound delivery worker classifies every attempt by the HTTP status code your server returns (or by the network-level error when no response arrives). Each attempt resolves to one of three outcomes:

  • Success — 2xx. Any status code from 200 to 299 returned within the 10-second per-attempt timeout. The event is acknowledged and removed from the active queue, and no further attempts are made.
  • Retry (transient). A failure we expect to recover on its own. The event is re-queued on the backoff schedule below. This covers:
    • Connection, timeout, and DNS errors (including any response slower than the 10-second timeout).
    • 408 Request Timeout and 429 Too Many Requests.
    • Any 5xx server error.
    • 404 Not Found and 410 Gone — a missing path is often a temporary outage (a receiver mid-deploy, or a briefly suspended host) that later recovers, so we keep retrying.
  • Dropped (permanent) — 400, 401, 403, 405, 406, 422. A caller-side error that re-sending cannot fix. The signed payload is byte-for-byte identical on every attempt, so a malformed, unauthenticated, or unauthorized request would fail the same way every time. The event is discarded immediately, with no retries. If your endpoint returns one of these codes, fix your request handling and recover any missed events through reconciliation.
Tip

Discarding unprocessable events: If your receiver encounters a bad payload or missing data that retrying will not fix (e.g., references a deleted external account), return a 2xx success code anyway. This stops further retries, and you can log the discard or raise an internal alert instead of clogging the queue. Prefer this over returning a 4xx, so the delivery is recorded as acknowledged rather than dropped.

Retry Policy & Schedule

When an attempt is classified as a transient failure, our worker waits and tries again, using a capped exponential backoff:

  • The delay starts at about 30 seconds and doubles with each retry — roughly 30s, 1m, 2m, 4m, 8m, and so on.
  • Once the gap reaches 8 hours, it stays there: we keep retrying about every 8 hours.
  • We continue for roughly 3 days. If the event still hasn't been accepted by then, it is permanently discarded.

Each delay also gets a small random reduction (jitter), so the values above are approximate upper bounds. Jitter spreads out endpoints that fail at the same time, so they don't all retry in lockstep.

Idempotency and Deduplication

Because we use at-least-once delivery, your endpoint will occasionally receive duplicate webhook events. This typically happens when your server successfully processes an event but suffers a network disconnect right before returning the 2xx HTTP response.

The Idempotency Key

Use the event envelope's id field (a unique UUID v4) as your idempotency key. This id is generated once and reused verbatim on every retry — the id, created_at, body, and signature are byte-for-byte identical across all delivery attempts of the same event — so it reliably identifies duplicates.

  1. Query Store: When you receive a webhook event, check if the event id is present in your local database log of recently-processed event IDs.
  2. Handle Duplicate: If the event id already exists, immediately return a 2xx status code. Do not re-run any side effects (e.g., allocating credits or extending access).
  3. Process New: If the id is new, save the ID to your store, execute your side-effects, and return a 200 OK status.
  4. Log Retention: Maintain your processed event logs for at least 3 days to ensure coverage across the entire retry window.

Event Ordering

Outbound webhooks may not arrive in chronological order. If a network retry delays an earlier event, a newer event may arrive and be processed first.

For example, if a member cancels and immediately resubscribes:

  • subscription.cancelled is sent, but fails and goes into retry.
  • subscription.created is sent and succeeds on its initial attempt.
  • subscription.cancelled is retried and succeeds.

If your receiver simply updates your database based on the latest arrived webhook, you might accidentally mark an active subscriber as cancelled.

Always check the created_at timestamp (ISO 8601 UTC) inside the event envelope. Only update your database state if the incoming event's created_at timestamp is newer than the timestamp of the last processed state on that member's record.

Next Steps

Keep building your developer integration with our reference guides:

Last updated: