A supplier changes an order status and sends a webhook to the ERP. The endpoint responds, but the application crashes before applying the update. Later, the same event arrives again, followed by an older status from a delayed delivery. A simple controller that immediately writes every received payload can leave the business record inconsistent.
Webhooks are a useful way to receive external events, but the receiving application needs a deliberate intake and processing model. It must establish who sent the message, preserve accepted work and decide how the event relates to current business state. Those responsibilities remain with the consumer even when the provider offers a convenient delivery service.
Read the provider's delivery contract first
Document the provider's event types, authentication or signature scheme, timeout expectations and retry behaviour. Check whether it promises ordering, how it identifies deliveries and whether missed events can be retrieved later. Do not transfer assumptions from one provider to another because both use HTTP POST requests.
GitHub's webhook documentation is a useful concrete example: it describes validation, prompt responses and asynchronous processing practices. Its details apply to GitHub deliveries, while the same questions should be answered from the official documentation of the provider actually being integrated. GitHub webhook best practices.
For a hypothetical distributor receiving supplier fulfilment events, record which system owns the authoritative status. Decide whether a notification contains the complete state needed for a change or should trigger an authenticated lookup of the latest source record. That choice affects both correctness and recovery.
Validate the delivery at the intake boundary
Use HTTPS and implement the provider's documented verification mechanism. For a signed webhook, preserve the representation required by that signature scheme and use the recommended validation approach. Reformatting the body before verification can change the bytes being checked and invalidate the comparison.
GitHub documents validating payload signatures with a webhook secret and a timing-safe comparison. Follow the exact scheme for the chosen provider rather than inventing a generic header that appears secure. Validating GitHub webhook deliveries.
Apply size limits and validate the event type and payload shape before accepting work. Keep credentials and sensitive content out of routine logs. A valid signature establishes something about the delivery's origin and integrity; the application must still verify that the event refers to a resource and operation within the intended integration scope.
Acknowledge only after preserving accepted work
Separate receiving a delivery from completing its business processing. Where asynchronous handling is appropriate, durably record the accepted event or enqueue it through a dependable mechanism before returning the response that tells the provider it was received. Otherwise a process crash can turn a successful acknowledgement into lost work.
Store enough context to investigate and safely process the event: provider identity, delivery identifier, event type, received time and an appropriate payload or reference. Choose retention and access rules according to the information involved. An event archive should not become an uncontrolled copy of customer data.
Bound intake work so the endpoint can meet the provider's response expectations without performing long external calls inline. If the application cannot preserve the event, return the appropriate failure according to the contract. Avoid claiming acceptance merely to make the provider's delivery dashboard look healthy.
Handle duplicates and ordering deliberately
Define an idempotency key appropriate to the provider and event semantics. Repeated processing of the same accepted event should not create another shipment or repeat an irreversible action. Persist the relevant processing state so protection survives a restart rather than depending only on an in-memory collection.
Consider delivery identity separately from resource version. Several different events can refer to the same order, and the order's current state may already be newer than one delayed event. Use documented versions, timestamps with understood semantics or a current-state lookup to decide whether the incoming update still applies.
For the distributor, an older “packed” event should not casually replace a confirmed “dispatched” state. Write the allowed transition rules and test reordered deliveries. If the provider cannot supply enough ordering evidence, choose a reconciliation strategy that relies on its authoritative record instead of guessing.
Make failures reviewable and replay controlled
Record processing outcomes and distinguish temporary failures from invalid data or unsupported contract changes. Give operational staff a view of unresolved events with a clear owner and next step. A growing log file is not a usable exception queue when customer orders are waiting.
Support replay through a controlled operation that checks previous effects and records why another attempt is being made. Replaying after a code fix may be appropriate, but the consumer must remain safe if part of the original attempt succeeded. Avoid editing stored payloads casually to force a successful result.
Use representative tests for invalid signatures, duplicate deliveries, unknown events, out-of-order updates and a crash after applying a business effect. These cases exercise the contract that matters during real integration incidents, rather than only demonstrating that a sample payload can be deserialised.
Reconcile beyond the delivery channel
Webhooks should be accompanied by a way to detect missing business outcomes where the provider and workflow allow it. Compare relevant source records or retrieve changes from a known checkpoint. Decide how the application establishes that it has caught up after an outage or a subscription configuration error.
Monitor age of unresolved events and discrepancies in business state alongside delivery failures. A provider can report successful HTTP responses while the ERP remains behind. Connect those signals to support procedures that explain both the communication path and the underlying order outcome.
Worktechlabs can help implement external event integrations with verified intake, durable processing and recoverable exceptions. Pair webhook handling with API contract management and reliable messaging so the application can explain what arrived, what changed and what still requires attention.
Official sources and further reading
- GitHub: webhook best practices — a provider's documented delivery and processing expectations.
- GitHub: validating webhook deliveries — signature verification for GitHub's specific scheme.
