Webhook deliveries are failing: how to inspect and retry them
The delivery log records every attempt with status, latency and the truncated response body. How to read it, and the auto-disable that ends a bad subscription.
When an Infrai webhook stops arriving, three reads tell you why without guessing: GET /v1/account/webhooks/get/{id} for the subscription’s current health, GET /v1/account/webhooks/deliveries/{id} for the per-attempt log, and POST /v1/account/webhooks/test/{id} to try one right now. The delivery record keeps the HTTP status, the latency and a truncated copy of what your endpoint actually said.
That last part is what turns “webhooks are broken” into a five-minute diagnosis.
Start with the subscription’s health
curl -sS "https://api.infrai.cc/v1/account/webhooks/get/whk_7Uu2kQxWvR4mBn8d" \
-H "Authorization: Bearer ${INFRAI_API_KEY}"
{
"ok": true,
"data": {
"webhook_id": "whk_7Uu2kQxWvR4mBn8d",
"url": "https://ops.example.com/hooks/infrai",
"events": ["email.delivered", "email.bounced"],
"active": true,
"status": "active",
"retry_policy": "default",
"last_delivery_at": "2026-09-21T02:40:12Z",
"last_delivery_status": "failed",
"failure_count_24h": 37,
"auto_disabled_at": null,
"created_at": "2026-09-14T09:00:00Z",
"updated_at": "2026-09-21T02:40:12Z"
}
}
Four fields answer most of it.
last_delivery_status tells you whether the most recent attempt worked, failure_count_24h distinguishes a blip from a pattern, and auto_disabled_at is the one that explains total silence — a subscription that keeps failing gets turned off, and after that nothing is being attempted at all, which is why “we stopped getting webhooks an hour ago” and “we have been failing verification for a day” are usually the same incident seen from two ends.
active: true with a high failure count means it’s still trying. auto_disabled_at set means it isn’t.
Read the attempts
curl -sS "https://api.infrai.cc/v1/account/webhooks/deliveries/whk_7Uu2kQxWvR4mBn8d" \
-H "Authorization: Bearer ${INFRAI_API_KEY}"
Each item carries event, status, attempt, http_status, latency_ms, error, a truncated response_body, and replay_of when it was a retry of an earlier delivery. Read them together and the failure names itself.
| What you see | Almost certainly |
|---|---|
http_status: 401 | your signature check is rejecting valid deliveries |
http_status: 404 | the route moved; the subscription still points at the old path |
error set, no http_status | connection or TLS failure — DNS, certificate, firewall |
latency_ms near the 8000 ms budget | your handler is doing the work before replying |
2xx but duplicated event ids | you aren’t deduplicating; retries look like new events |
The latency row is the most common self-inflicted one. Delivery has an eight-second budget, so a handler that processes before acknowledging fails under load even though the code is correct — acknowledge with a 204 first, then process out of band.
Test without waiting
curl -sS -X POST "https://api.infrai.cc/v1/account/webhooks/test/whk_7Uu2kQxWvR4mBn8d" \
-H "Authorization: Bearer ${INFRAI_API_KEY}"
{
"ok": true,
"data": {
"webhook_id": "whk_7Uu2kQxWvR4mBn8d",
"event": "system.maintenance.scheduled",
"delivered": false,
"http_status": 401,
"latency_ms": 96,
"error": "http_401",
"delivery_id": "whd_3kQ9mVzR1sXbNt",
"tested_at": "2026-09-21T02:52:00Z"
}
}
A 401 on the test is the fastest possible proof that verification is the problem rather than networking. The usual cause: hashing a re-serialised JSON body instead of the raw bytes, because the signature covers exactly what arrived.
One triage function
import os
import requests
API = "https://api.infrai.cc"
KEY = os.environ["INFRAI_API_KEY"]
SESSION = requests.Session()
SESSION.headers.update({"Authorization": f"Bearer {KEY}", "Content-Type": "application/json"})
def triage(webhook_id: str) -> dict:
"""Health, then the last attempts, then a live probe — in the order that
narrows the problem fastest."""
health = SESSION.get(f"{API}/v1/account/webhooks/get/{webhook_id}", timeout=20)
health.raise_for_status()
hook = health.json()["data"]
log = SESSION.get(f"{API}/v1/account/webhooks/deliveries/{webhook_id}", timeout=20)
log.raise_for_status()
attempts = log.json()["data"].get("items", [])
probe = SESSION.post(f"{API}/v1/account/webhooks/test/{webhook_id}", timeout=25)
live = probe.json().get("data", {}) if probe.ok else {}
statuses = [a.get("http_status") for a in attempts[:20]]
slow = [a for a in attempts[:20] if (a.get("latency_ms") or 0) > 5000]
return {
"auto_disabled_at": hook.get("auto_disabled_at"),
"failure_count_24h": hook.get("failure_count_24h"),
"recent_statuses": statuses,
"slow_attempts": len(slow),
"live_probe": {"delivered": live.get("delivered"), "http_status": live.get("http_status"),
"error": live.get("error")},
"verdict": _verdict(hook, statuses, slow, live),
}
def _verdict(hook, statuses, slow, live) -> str:
if hook.get("auto_disabled_at"):
return "subscription auto-disabled after repeated failures — fix the endpoint, then re-enable"
if live.get("http_status") in (401, 403):
return "endpoint rejects the signature — verify against the RAW body bytes"
if live.get("http_status") == 404:
return "endpoint path is wrong — update the subscription URL"
if len(slow) > 3:
return "handler too slow — acknowledge first, process after"
if live.get("delivered"):
return "delivering now; earlier failures look transient"
return "connection-level failure — check DNS, TLS and firewall"
if __name__ == "__main__":
for key, value in triage(os.environ["WEBHOOK_ID"]).items():
print(f"{key:>20}: {value}")
Bringing it back
Once the endpoint is fixed, PATCH /v1/account/webhooks/update/{id} re-activates and reconfigures in one call — it takes url, events, active, retry_policy and headers:
curl -sS -X PATCH "https://api.infrai.cc/v1/account/webhooks/update/whk_7Uu2kQxWvR4mBn8d" \
-H "Authorization: Bearer ${INFRAI_API_KEY}" \
-H "Content-Type: application/json" \
-d '{"url": "https://ops.example.com/hooks/infrai/v2", "active": true, "retry_policy": "aggressive"}'
retry_policy: "aggressive" retries harder, which is right for an endpoint with occasional brief unavailability and wrong for one that’s slow — retrying a timeout more often just multiplies the load that caused it.
The limitation you have to design around
There’s no bulk replay. You can inspect past deliveries and re-test the subscription, but “resend every event from the four hours we were down” isn’t a call you can make, and replay_of only appears on retries the platform itself performed.
So make your consumer able to recover from a gap by polling the source of truth: GET /v1/email/event/list for mail activity, GET /v1/account/usage for spend, GET /v1/errors/list for captured errors. Webhooks then become an optimisation over polling rather than the only path — which is the right architecture anyway.
Everything in this loop is on one credential, and that’s the part that doesn’t reproduce across vendors: the events come from the same account as the email or job that produced them, the queue you hand accepted events to is POST /v1/queue/publish, and the alert when failure_count_24h climbs goes out through POST /v1/email/send. One key, one bill, one usage view.
All the webhook routes report billing_class: free in discovery, so diagnosing this costs nothing per call. The capabilities emitting events are the billable part — GET /v1/discovery carries the live rate and GET /v1/account/usage your spend (verified 2026-09-21), and those rates trend downward as vendor contracts improve.