Help

How Drum webhooks work

A webhook is Drum telling your system that something changed, instead of your system asking. Most of what is surprising about Drum’s webhooks follows from one aim: your system should always end up with the latest state of a record, even when a network or an endpoint fails along the way.

From a change to a request

When someone saves a project, invoice or cost, Drum finishes saving it first. Only once the change is safely stored does Drum look for subscriptions that want the event. A change that is rolled back, such as a save that fails part way, never sends anything.

For each matching subscription, Drum records a delivery holding the body to send, then hands it to a background worker. The person in Drum never waits on your endpoint, and a slow or broken endpoint never slows Drum down.

The worker checks where the URL points before it connects, signs the body with the subscription’s secret, and sends it. Your response decides what happens next: a 2xx is a success, a temporary-looking failure is tried again, and anything else is recorded as failed.

Why events can repeat

Drum cannot know for certain whether your endpoint processed a request. A response can be lost after your code has already done its work, and Drum then sends the same delivery again. Retrying is the only way to make sure an event is not lost, so Drum delivers each event at least once, and sometimes more.

The fix belongs at the receiving end: process each event in a way that gives the same result if it runs twice. Recording event, data.type, data.id and data.attributes.updated_at, and skipping a combination you have seen, does that.

Why events can arrive out of order

Each delivery is sent on its own. A delivery that is retried after a timeout can arrive after a later one that went through first. Two systems receiving the same events can also see them in different orders.

So the order of arrival tells you nothing. The record’s updated_at does. It has six decimal places, so even two saves in the same second can be told apart. Keep the newest state you have seen for each record, and ignore an older one when it turns up late.

Why quick changes are combined

One action in Drum can save a record several times in quick succession: a status change that also updates dates, or a total recalculated after a line is added. Sending every save would flood your endpoint with near-identical events, most of them out of date by the time they arrive.

Instead, Drum sends the first .updated event for a record straight away, then waits 5 seconds. If the record changed again in that window, Drum sends one more event with the record as it is at the end. You hear about a change quickly, and the last thing you hear is the current state.

Events that mark a moment rather than a state are never combined: a record created, a project completed, an opportunity won or lost, companies merged, a trigger fired. Each of those is worth knowing about on its own.

Why the body carries the whole record

Each event carries a copy of the record, not just its id. Most receivers can act on the event alone, without calling back to the API. The copy is taken when the event is recorded, so a delivery that is retried later still carries the state it was created with. That is another reason to compare updated_at before applying an event.

The payload is a fixed set of fields, the same for everyone. It does not change with the permissions of the person who made the change or who created the subscription. That is why some sensitive values, such as an opportunity’s deal value, are never in it. When you need more, read the record through the REST API, which applies the token user’s permissions.

Why requests are signed

Your endpoint has a public address, so anyone can send it a request that looks like Drum’s. The signature proves two things: the request came from someone who knows the subscription’s secret, and the body was not changed on the way.

The timestamp is part of what is signed. A request captured and sent again later still has a valid signature, but its timestamp is old. Rejecting old timestamps closes that gap.

Why some addresses are refused

Drum sends requests from inside its own network. If a webhook could point at a private address, someone could use Drum to reach machines that are not meant to be reachable from the internet. So Drum resolves the URL’s host name before every attempt, refuses private, loopback and cloud metadata addresses, and connects only to the address it checked. A subscription whose URL points somewhere it should not is disabled.

Why subscriptions disable themselves

An endpoint that has failed 10 deliveries in a row is almost certainly broken or gone. Carrying on would build up thousands of failed deliveries that nobody is watching. Drum stops sending and emails the person who created the subscription, so a person decides when it is fixed. Re-activating it resets the count, and failed deliveries can be sent again one by one.