Webhook subscription management: the subscription as a first-class resource
Most webhook systems start with a single WEBHOOK_URL in a config table, and every scaling problem described in the webhook architecture fundamentals reference eventually traces back to that shortcut. The moment a second consumer wants a different subset of events, or one endpoint starts timing out and takes the whole dispatch loop with it, the config value has to become a row — an addressable, versioned, independently governable object with its own identity, its own secret, its own filter, and its own health. That object is the subscription, and treating it as a first-class resource is what makes the rest of the delivery pipeline tractable.
This page covers the shape of that resource: the fields it must carry, the state machine it moves through, how it is scoped in a multi-tenant provider, and how the dispatcher reads it. The three deep dives underneath handle the hard parts individually — proving the consumer owns the URL in registering and validating webhook endpoints, narrowing what each subscriber receives in event type filtering and subscription scoping, and shutting off dead endpoints in auto-disabling failing webhook endpoints.
The subscription record and its mandatory columns
A subscription is defined by six pieces of state, and omitting any one of them creates a class of bug that is expensive to retrofit. The identifier must be opaque, stable, and safe to print in logs and delivery headers, because consumers will quote it in support tickets; a UUIDv7 or prefixed ULID (sub_01J...) gives you both uniqueness and rough creation ordering. The tenant reference is the scoping key that every query filters on. The endpoint URL is stored normalised, never as the raw string the consumer typed. The event type selection is the filter the dispatcher evaluates. The secret reference points at a signing key in a secrets manager rather than holding the key inline. The status is the single authoritative flag the dispatcher trusts.
Two columns deserve extra discipline. Store endpoint_url in canonical form — lowercase scheme and host, explicit default port stripped, no fragment, query string preserved verbatim — and keep the raw submission in a separate audit column. Without normalisation, https://api.example.com/hooks and https://API.example.com:443/hooks/ are two subscriptions that double-deliver every event. For secret_ref, hold a pointer rather than the key: an entry like projects/p/secrets/whsec-sub_01J.../versions/4 lets you rotate the signing secret without a schema migration, which is exactly the indirection that makes zero-downtime webhook secret rotation possible.
Everything else — retry counters, last response code, last success timestamp — belongs in a sibling subscription_health table rather than the subscription row itself. Delivery workers write those columns on every attempt, and mixing hot write traffic into the row that the dispatcher reads on every event turns a cheap indexed read into a contention hotspot.
The registration, verification, activation, suspension lifecycle
A subscription is never simply “on”. It moves through an explicit state machine whose transitions are all triggered by named events, and the value of writing it out is that the dispatcher only ever has to answer one question: is this row active? Everything else — is the URL proven, has it been failing, did the owner turn it off — collapses into that single predicate.
A newly created subscription starts in pending. It carries a verification token with a short TTL and receives no real traffic at all. When the consumer echoes the challenge back it becomes active. If the token expires unclaimed, it moves to revoked and is garbage-collected — a pending row that never verifies is either a typo or an abandoned integration, and neither should linger. From active, the delivery pipeline can move it to suspended when the consecutive-failure counter crosses its threshold; a human or an automated probe moves it back. The owner can also revoke it outright, which is terminal.
The trade-off worth naming here is suspend versus revoke. Suspension is reversible and should preserve the subscription’s identity, secret, and filter so that re-enabling is a one-field update; revocation is terminal and should free the URL for re-registration. Systems that only implement deletion force consumers to re-onboard after every outage, which means new subscription IDs, new secrets, and a support ticket. Systems that only implement suspension accumulate zombie rows that quietly consume dispatch-time filter evaluations forever. You need both, plus the archival sweep that moves long-idle suspended rows out of the hot table.
Multi-tenant scoping and the ownership boundary
In any provider serving more than one customer, the subscription table is a cross-tenant object and therefore an authorization surface. Three scoping models show up in practice, and the choice determines how much blast radius a single bug carries.
Tenant-level scoping gives each customer account one set of subscriptions covering all of its resources. It is the simplest model and the right default. Resource-level scoping adds an optional resource_id so a subscription can be narrowed to one project, store, or connected account — necessary for platforms whose customers themselves have customers. Delegated scoping lets a third-party integrator hold subscriptions on behalf of many tenants, which requires the subscription to carry both the owning application and the tenant it observes, and requires the consent record that authorised the link.
| Scoping model | Query predicate | Best for | Main risk |
|---|---|---|---|
| Tenant-level | tenant_id = $1 |
Single-account SaaS integrations | Noisy: consumer receives every resource’s events |
| Resource-level | tenant_id = $1 AND resource_id = $2 |
Platforms with sub-accounts or projects | Subscription sprawl; one row per resource |
| Delegated (app-owned) | app_id = $1 AND tenant_id = $2 |
Marketplace and partner integrations | Consent revocation must cascade to delivery |
| Environment-partitioned | tenant_id = $1 AND env = 'live' |
Providers with test and live key pairs | Test events leaking into production endpoints |
Whichever model you pick, enforce it at the storage layer, not in the application service. Postgres row-level security with a current_setting('app.tenant_id') policy makes a forgotten WHERE clause a failed query instead of a cross-tenant data leak. The subscription API is also the place where an unvalidated URL becomes a server-side request forgery vector, since a consumer who registers http://169.254.169.254/ is asking your dispatcher to fetch your own metadata service — the controls for that live in preventing SSRF in outbound webhook delivery.
How subscription state drives the dispatcher
The dispatcher’s job on each domain event is to turn one event into N delivery jobs. The subscription table is the routing table for that expansion, and the critical design rule is that status and tenant are query predicates, not runtime checks. If you load all subscriptions and filter in application code, a suspended endpoint still costs you a filter evaluation, a job enqueue, and eventually a discarded delivery — multiplied by every event for as long as the row exists.
Cache the routing table, but cache it correctly. A per-tenant snapshot with a 5–15 second TTL removes the lookup from the hot path at the cost of a bounded window where a just-suspended endpoint still receives events. That window is acceptable for suspension and unacceptable for revocation, so publish an invalidation message on revoke and on secret rotation while letting ordinary status changes expire naturally. Where events for one subscriber must arrive in order, the routing decision also has to preserve the partition key — see message ordering guarantees for why the subscription id usually makes a poor ordering key.
Authenticating and validating writes to the subscription API
The subscription API is a privileged surface: whoever can write a row can redirect a customer’s event stream. Four controls belong on it from day one.
Validate the URL before persisting, rejecting non-HTTPS schemes, private and link-local address ranges, and hostnames that resolve to internal space. Require re-verification whenever endpoint_url changes, so an attacker with a stolen API key cannot silently repoint an existing verified subscription. Generate the signing secret server-side and return it exactly once at creation, never on subsequent reads — a GET /subscriptions/:id that returns the secret turns every read-scoped token into a forgery capability, which undermines the whole HMAC signature verification chain. Finally, write an append-only audit record for every mutation, capturing actor, source IP, and the before and after values of the URL and event-type list.
Input validation deserves a schema rather than hand-rolled checks. Coerce and constrain in one place, reject unknown keys so a typo in event_types never silently produces a subscription that matches nothing, and cap the size of the event-type array to stop a consumer registering ten thousand patterns that must be evaluated per event.
Production rollout checklist for a subscription service
Create the subscription table and status enum
Ship the table with the status enum, a unique constraint on (tenant_id, endpoint_url, resource_id) to prevent accidental duplicates, and a partial index on (tenant_id) WHERE status = 'active' so the dispatch query never scans suspended rows.
Enforce endpoint verification before first delivery
Default status to pending at the database level, not in application code, so any code path that inserts a row produces an unverified subscription until the handshake completes.
Push the status predicate into the dispatch query
Move status and tenant filtering into SQL. Measure the plan: the dispatch lookup should be an index scan returning tens of rows, not a sequential scan filtered in the application.
Wire suspension and re-enable into the delivery pipeline
Have delivery workers write outcomes to subscription_health and let a single transition function own the status change, so suspension logic exists in exactly one place.
Expose subscription state to the endpoint owner
Return status, last_delivery_at, last_status_code, and consecutive_failures from the API, and surface the same fields in the dashboard. Owners who can see a red endpoint fix it without opening a ticket.
The following implementation covers the store and the dispatch query in one runnable module.
import { Pool, PoolClient } from 'pg';
import { z } from 'zod';
import { randomUUID } from 'node:crypto';
export type SubscriptionStatus = 'pending' | 'active' | 'suspended' | 'revoked';
export interface Subscription {
id: string;
tenantId: string;
endpointUrl: string;
eventTypes: string[];
secretRef: string;
status: SubscriptionStatus;
}
const pool = new Pool({ connectionString: process.env.DATABASE_URL });
// Reject unknown keys so a misspelled field never silently creates a
// subscription that matches nothing.
export const CreateSubscriptionInput = z
.object({
endpointUrl: z.string().url().max(2048),
eventTypes: z.array(z.string().regex(/^[a-z][a-z0-9_.]*(\.\*)?$/)).min(1).max(64),
resourceId: z.string().uuid().optional(),
})
.strict();
export type CreateSubscriptionInput = z.infer<typeof CreateSubscriptionInput>;
/** Canonical form: lowercase scheme+host, default port stripped, no fragment. */
export function normaliseEndpoint(raw: string): string {
const url = new URL(raw);
if (url.protocol !== 'https:') {
throw new Error(`Endpoint must use https, got ${url.protocol}`);
}
url.hostname = url.hostname.toLowerCase();
url.hash = '';
if (url.port === '443') url.port = '';
if (url.pathname.length > 1 && url.pathname.endsWith('/')) {
url.pathname = url.pathname.slice(0, -1);
}
return url.toString();
}
export async function createSubscription(
tenantId: string,
input: CreateSubscriptionInput,
): Promise<{ subscription: Subscription; verificationToken: string }> {
const parsed = CreateSubscriptionInput.parse(input);
const endpointUrl = normaliseEndpoint(parsed.endpointUrl);
const id = randomUUID();
const verificationToken = randomUUID().replace(/-/g, '');
const client: PoolClient = await pool.connect();
try {
await client.query('BEGIN');
// Scope the transaction so row-level security policies apply.
await client.query('SELECT set_config($1, $2, true)', ['app.tenant_id', tenantId]);
const { rows } = await client.query<Subscription>(
`INSERT INTO webhook_subscriptions
(id, tenant_id, resource_id, endpoint_url, event_types, secret_ref, status,
verification_token, verification_expires_at)
VALUES ($1, $2, $3, $4, $5, $6, 'pending', $7, now() + interval '24 hours')
RETURNING id, tenant_id AS "tenantId", endpoint_url AS "endpointUrl",
event_types AS "eventTypes", secret_ref AS "secretRef", status`,
[
id,
tenantId,
parsed.resourceId ?? null,
endpointUrl,
parsed.eventTypes,
`whsec/${id}`,
verificationToken,
],
);
await client.query(
`INSERT INTO subscription_audit (subscription_id, action, payload)
VALUES ($1, 'created', $2)`,
[id, JSON.stringify({ endpointUrl, eventTypes: parsed.eventTypes })],
);
await client.query('COMMIT');
return { subscription: rows[0], verificationToken };
} catch (err) {
await client.query('ROLLBACK');
throw err;
} finally {
client.release();
}
}
/**
* The dispatch routing query. Status and tenant are predicates, not runtime
* checks: suspended and pending rows never leave the database.
*/
export async function routeEvent(
tenantId: string,
eventType: string,
): Promise<Subscription[]> {
const wildcard = `${eventType.split('.')[0]}.*`;
const { rows } = await pool.query<Subscription>(
`SELECT id, tenant_id AS "tenantId", endpoint_url AS "endpointUrl",
event_types AS "eventTypes", secret_ref AS "secretRef", status
FROM webhook_subscriptions
WHERE tenant_id = $1
AND status = 'active'
AND (event_types && ARRAY[$2, $3, '*']::text[])`,
[tenantId, eventType, wildcard],
);
return rows;
}
export async function transitionStatus(
subscriptionId: string,
from: SubscriptionStatus[],
to: SubscriptionStatus,
reason: string,
): Promise<boolean> {
// Guarded transition: the UPDATE only applies from an expected source state,
// so concurrent workers cannot resurrect a revoked subscription.
const { rowCount } = await pool.query(
`UPDATE webhook_subscriptions
SET status = $3, status_reason = $4, status_changed_at = now()
WHERE id = $1 AND status = ANY($2::text[])`,
[subscriptionId, from, to, reason],
);
return rowCount === 1;
}
Failure modes in subscription state management
| Failure mode | Impact | Mitigation |
|---|---|---|
| Duplicate subscriptions for one normalised URL | Every event delivered twice; consumer-side deduplication masks the cost until volume spikes | Unique constraint on (tenant_id, endpoint_url, resource_id) plus URL normalisation before insert |
| Cached routing table outlives a revoke | Events keep flowing to an endpoint the owner deleted, often after an offboarding | Publish explicit invalidation on revoke and secret rotation; keep TTL expiry only for benign transitions |
| Status changed without a guarded predicate | A slow suspension worker overwrites a newer revoke, silently re-enabling a dead endpoint | Conditional UPDATE ... WHERE status = ANY($from) and treat a zero row count as a lost race |
| Secret stored inline in the subscription row | Rotation requires a migration; a read-scoped API token leaks forgery material | Store a secret_ref pointer and resolve it in the signer at delivery time |
| Unbounded event-type arrays | One subscriber with thousands of patterns makes filter evaluation the dispatch bottleneck | Cap array length in the schema and reject at the API boundary, not at dispatch |
| Health counters written into the hot subscription row | Write contention on the row the dispatcher reads for every event | Split delivery outcomes into a subscription_health table keyed by subscription id |
Debugging a subscription that is not receiving events
- Confirm the row’s
statusisactive— apendingrow that never completed verification is the single most common cause. - Compare the stored
endpoint_urlagainst what the consumer expects, character for character, after normalisation. - Run the dispatch query by hand with the tenant id and the exact event type; if it returns zero rows the problem is the filter, not delivery.
- Check
event_typesfor a pattern that looks right but is not, such asinvoice.*when the emitted type isinvoices.paid. - Verify the tenant scoping columns — a subscription created against a test-mode key never sees live-mode events.
- Inspect
consecutive_failuresandlast_status_codeinsubscription_healthto distinguish “never routed” from “routed and failing”. - Force-invalidate the routing cache for the tenant and re-emit a test event to rule out a stale snapshot.
- Read the audit trail for the subscription: a recent URL change resets verification and silently stops delivery until re-confirmed.
Operating subscriptions in CI and production
Treat the state machine as testable logic. A contract test that walks a subscription through every legal transition, and asserts that every illegal one is rejected, catches the class of bug where a new feature adds a status write that bypasses the transition function. Run it in CI against a real Postgres instance so the guarded UPDATE semantics are exercised rather than mocked.
Migrations that touch the subscription table need the same care as any other online schema change: add the column, backfill in batches, then add the constraint. Adding a NOT NULL status column to a live subscription table without a default has taken down dispatch loops. Instrument four metrics — subscriptions by status, verification completion rate, mean time from pending to active, and suspensions per day — and alert on the second and fourth. A drop in verification completion usually means a docs or SDK regression; a spike in suspensions means either a bad release on your side or a shared platform outage on the consumer side. Wire those signals into the same pipeline described in webhook observability and monitoring rather than building a parallel dashboard.
Frequently Asked Questions
Can two subscriptions in the same tenant point at the same endpoint URL?
Only if they differ on resource_id, which is exactly what the unique constraint on tenant_id, endpoint_url and resource_id permits. That is legitimate under resource-level scoping, but the consumer then receives two separately signed deliveries for any event both rows match, so their handler has to read the subscription id out of the delivery headers to tell them apart. If the two rows were meant to be one subscription with a wider filter, merge the event-type arrays instead.
What happens to events emitted while a subscription sits in suspended?
Nothing leaves the dispatcher, because the routing query never selects the row, so the real question is whether those events remain recoverable. A provider that keeps an event archive can replay the suspended window when the row is re-enabled; a provider without one has quietly dropped that customer's data with no way to reconstruct it. Decide the retention window before you ship suspension, because it is the only thing support can honestly promise afterwards.
Does editing the event-type list require re-verification the way a URL change does?
No. Verification proves control of a destination, and a filter edit does not move the destination, so it is an ordinary audited update. It does need a cache invalidation, though: a customer who widens their filter and immediately fires a test event will otherwise be routed by a snapshot still holding the old array and will report the edit as broken.
Should the API expose a separate public identifier instead of the primary key?
One identifier is enough provided it was chosen to be safe in the open. A prefixed ULID or UUIDv7 carries no secret material and no sequential customer count, so it can appear in delivery headers, support tickets, and logs without leaking anything. Minting a second external id doubles the lookup paths and creates the class of incident where a support engineer and a database query are discussing different rows without realising it.
What should a read of the subscription return in place of the signing secret?
Return a non-reversible descriptor: the secret reference's version number, when it was created, and at most a short fingerprint of the key. That is enough for an engineer to confirm which secret their consumer is verifying against, without handing a read-scoped token the material to forge deliveries. If the customer has genuinely lost the value, the answer is a rotation rather than a read.
What has to happen when a tenant closes their account with subscriptions still active?
Revocation should cascade inside the same transaction that closes the account, and the tenant's routing snapshot has to be invalidated explicitly rather than left to expire. The part teams forget is the queue: delivery jobs enqueued moments before the revoke are already in flight, so the delivery worker needs its own status check when it dequeues, not just the dispatcher's predicate.
Why one status enum rather than separate verified and enabled booleans?
Boolean pairs make illegal states representable — verified and revoked, unverified and enabled — and then every reader of the record has to remember the same conjunction. A single enum with guarded transitions makes those combinations unwritable, and it lets the dispatch path use a partial index on one value. The cost is that adding a state becomes a migration rather than a new column default, which is the right amount of friction for something this load-bearing.
Related
- Registering and validating webhook endpoints — URL normalisation and the challenge-response handshake that moves a subscription out of
pending. - Event type filtering and subscription scoping — taxonomies, wildcard matching, and evaluating filters at dispatch time.
- Auto-disabling failing webhook endpoints — consecutive-failure counters, suspend thresholds, and a safe re-enable path.
- Event schema design — the payload contract that the event types in a subscription refer to.
- Webhook architecture fundamentals — the wider design discipline this resource model sits inside.