Webhook

What is a webhook?

A webhook is a way for one system to notify another about an event by calling an HTTP endpoint. The subscriber registers a URL, and the provider sends an HTTP request, typically a POST with a JSON body, to that URL every time the event happens: a payment succeeds, a pull request is merged, an alert fires, an incident is resolved.

Webhooks are often described as a "reverse API". With a normal API, the client asks repeatedly whether something has changed (polling). With a webhook, the provider pushes the change when it happens. That saves requests and reduces delay, but it moves responsibility for availability, security and correctness to the receiving endpoint.

How webhooks work

1. The subscriber registers an endpoint URL and selects event types, and both sides agree on a shared secret.
2. When an event occurs, the provider builds a payload with an event ID, type, timestamp and data.
3. The provider signs the payload and sends it to the endpoint.
4. The receiver verifies the signature, stores or enqueues the event and quickly responds with a 2xx status.
5. If the response is missing, times out or is not 2xx, the provider retries later.

A typical request looks like this (illustrative):

POST /hooks/alerts HTTP/1.1
Content-Type: application/json
Webhook-Id: evt_01J9X4K2
Webhook-Timestamp: 1760515200
Webhook-Signature: v1,K5oZfzN95Z9UVu1EsfQmfVNQhnkZ2pj9o9NDN/H/pI4=

{"id": "evt_01J9X4K2", "type": "alert.triggered", "data": {"service": "checkout", "severity": "critical"}}

Verifying webhook signatures with HMAC

Because a webhook endpoint is a public URL, the receiver must verify that a request really comes from the provider and was not modified. The common approach is an HMAC signature: the provider computes an HMAC, usually HMAC-SHA256, over the raw request body (often together with a timestamp and message ID) using the shared secret, and sends it in a header. The receiver recomputes the HMAC and compares.

• Compute the signature over the raw bytes of the body, before any JSON parsing or reformatting.
• Compare with a constant-time function, such as hmac.compare_digest in Python, to avoid timing attacks.
• Include a timestamp in the signed content and reject requests outside a short tolerance window, for example five minutes, to prevent replay attacks.
• Support secret rotation by accepting signatures from the old and new secret for a transition period.

Header names differ by provider: GitHub uses X-Hub-Signature-256, Stripe uses Stripe-Signature with a timestamp, and the open Standard Webhooks specification defines webhook-id, webhook-timestamp and webhook-signature.

Retries and idempotency

Networks fail, receivers deploy and endpoints time out, so providers retry failed deliveries, usually with exponential backoff and jitter over hours or days. This makes most webhook systems at-least-once: the same event can arrive more than once, and events can arrive out of order.

• Make processing idempotent. Store the event ID and skip events you have already handled. A unique constraint on the ID in the database is a simple, robust solution.
• Acknowledge fast, process later. Verify, persist to a queue and return 2xx within a few seconds. Long processing in the request causes timeouts and duplicate deliveries.
• Do not rely on order. Use timestamps or version numbers in the payload, or fetch the current state from the provider's API.
• Use status codes deliberately. Return 2xx once the event is stored, 4xx for requests that will never succeed such as an invalid signature, and 5xx or a timeout only when a retry makes sense. A provider may respect rate limiting responses such as 429.

Monitoring webhooks

Treat webhook receivers as production services. Track request rate per event type, signature failures, response status and latency, and the depth of the processing queue. On the sender side, monitor delivery success, retry counts and endpoints that are disabled after repeated failures. Because a missing webhook produces no error on the receiving side, also watch for silence: if you normally get events every few minutes, an alert on "no events received" catches broken subscriptions, similar to heartbeat monitoring.

Common mistakes

• Skipping signature verification, or verifying after parsing so the bytes no longer match.
• Doing heavy work synchronously and timing out, which triggers retries and duplicates.
• Treating every delivery as unique and charging a customer or creating an incident twice.
• Logging full payloads that contain personal data or secrets.

How Obseria helps

Obseria traces webhook receivers like any other endpoint, so you see latency, errors and downstream calls per event type, and synthetic API checks can test that endpoints respond. Workflows in Obseria Automation can be started by webhooks, incidents can be forwarded to existing paging tools via webhook, and users can subscribe to status page updates by webhook.