Retrying failed webhooks: delayed republish, cron sweep, or push retries

Why a nack is not a backoff, how to build the retry ladder from delayed publishes, and when a once-a-day cron redrive of the dead-letter queue is the right safety net.

Two different jobs get confused here. The retry ladder — try again in 1 minute, then 5, then 30 — belongs in the queue, as a message published with a delay. The sweep — go find everything that exhausted its attempts last night and give it one more shot after the partner fixed their TLS cert — is the thing a cron job is genuinely good at. Infrai gives you both on one key, and the piece most people get wrong is that a nack isn’t the first one.

Here’s the detail that decides your design.

A nack retries immediately

We ran it: consume a message, nack it with requeue: true, and the very next consume call hands it straight back with delivery_count incremented. There’s no pause. For a partner API that just returned 503 because they’re mid-deploy, that means your three attempts burn in under a second and the message is in the dead-letter queue before their pod finished restarting.

So nack is the right call for this delivery failed, someone try again, and the wrong tool for wait, then try again.

Backoff is a publish with delay_seconds, which accepts anything from 0 to 604800 — a full week of ladder if you want one.

import process from "node:process";

const BASE = "https://api.infrai.cc";
const key = process.env.INFRAI_API_KEY;
if (!key) throw new Error("set INFRAI_API_KEY before running the sender");
const headers = { Authorization: `Bearer ${key}`, "Content-Type": "application/json" };

// 1m, 5m, 30m, 2h, 12h — five attempts spread over about fifteen hours.
const LADDER = [60, 300, 1800, 7200, 43200];

async function api(path, payload) {
  const res = await fetch(`${BASE}${path}`, { method: "POST", headers, body: JSON.stringify(payload) });
  const out = await res.json();
  if (out.ok === false) throw new Error(`${path}: ${out.error.code} — ${out.error.message}`);
  return out.data;
}

export async function deliverBatch() {
  const { items } = await api("/v1/queue/consume", { queue: "webhook-out", max_messages: 10 });

  for (const msg of items) {
    const { url, event, attempt = 0 } = msg.payload;
    let ok = false;
    try {
      const res = await fetch(url, {
        method: "POST",
        headers: { "Content-Type": "application/json", "X-Event-Id": event.id },
        body: JSON.stringify(event),
        signal: AbortSignal.timeout(10_000),
      });
      ok = res.status >= 200 && res.status < 300;
      if (!ok) console.warn(`subscriber ${url} answered ${res.status} for ${event.id}`);
    } catch (err) {
      console.warn(`subscriber ${url} unreachable: ${err.message}`);
    }

    if (!ok && attempt < LADDER.length) {
      const retry = { queue: "webhook-out", payload: { url, event, attempt: attempt + 1 }, delay_seconds: LADDER[attempt] };
      await api("/v1/queue/publish", retry);
    } else if (!ok) {
      await api("/v1/queue/publish", { queue: "webhook-dead", payload: { url, event, attempts: attempt } });
    }
    await api("/v1/queue/ack", { queue: "webhook-out", message_id: msg.message_id });
  }
  return items.length;
}

Notice the shape: whatever happened, the original message is acked, and the retry exists as a new delayed message. That keeps the lease short — a message you hold for two hours while sleeping is a message that gets redelivered to a second worker the moment the visibility timeout expires, and you’d get a QUEUE_MESSAGE_NOT_IN_FLIGHT on the ack that follows.

The trade-off is real and you should know it: republishing resets delivery_count, so the queue’s own attempt counter stops being the source of truth. Carry attempt in the payload, as above, or your ladder will never terminate.

The DLQ, and the sweep that empties it

Configure the failure lane when you create the queue, and let three failed deliveries be the automatic exit condition.

{
  "name": "webhook-out",
  "type": "standard",
  "dead_letter_queue": "webhook-out-dlq",
  "max_retries": 3,
  "visibility_timeout_default": 60
}

A dead-lettered message isn’t gone — the DLQ is an ordinary queue, so you read it with the same consume call, by name:

export INFRAI_API_KEY="your_infrai_api_key"

curl -sS "https://api.infrai.cc/v1/queue/stats/webhook-out" \
  -H "Authorization: Bearer ${INFRAI_API_KEY}"

{
  "ok": true,
  "data": {
    "queue": "webhook-out",
    "message_count": 0,
    "available_count": 0,
    "in_flight_count": 0,
    "delayed_count": 12,
    "dlq_count": 4,
    "oldest_message_age_seconds": 0
  }
}

dlq_count is the number your on-call alert should watch. When it moves, one message per affected subscriber usually explains the whole spike — a rotated secret, an expired certificate, a partner who moved to a new hostname without telling anyone.

Moving one message back is a single call, and it’s free:

curl -sS -X POST "https://api.infrai.cc/v1/queue/dlq/redrive/webhook-out" \
  -H "Authorization: Bearer ${INFRAI_API_KEY}" \
  -H "Content-Type: application/json" \
  --data '{"message_id":"qmsg_54CQ2suupH09MIbkg94Tf39l"}'

That returns {"queue": "webhook-out", "redriven": 1, "message_id": "..."} and the message is available again with its delivery counter reset. Worth flagging from our testing: the per-message form is the one to build on. The batch forms — omitting message_id, or passing since — returned an INVALID_ARGUMENT on our account rather than moving a batch, and GET /v1/queue/dlq/list/{queue} came back with an empty items array while dlq_count said four. Until that settles, drain the DLQ by consuming it under its own name and re-publishing what you want to retry; that path works today and costs one publish per message.

If you don’t want a worker at all

A queue can push to you instead. Subscribe a public HTTPS endpoint and Infrai does the delivery, the retries and the dead-lettering:

curl -sS -X POST "https://api.infrai.cc/v1/queue/push_subscribe/{queue}" \
  -H "Authorization: Bearer ${INFRAI_API_KEY}" \
  -H "Content-Type: application/json" \
  --data @subscribe.json

subscribe.json carries url, an optional secret for HMAC signing, max_retries, visibility_timeout and dead_letter_queue. The endpoint must be publicly reachable and is SSRF-checked, so localhost and RFC1918 addresses are refused — if your consumer lives inside a VPC with no ingress, polling is your only option here.

Choosing between the three

The honest boundary: if delayed HTTP delivery is the entire problem you have, QStash is purpose-built for it and you should look there first. SQS with a dead-letter queue is the right answer if your infrastructure is already AWS-shaped, and Temporal is what you want when a failed webhook has to roll back three other steps. Infrai’s queue wins when the retry is one part of something bigger — the same key sends the notification email when a subscriber is disabled, stores the request/response pair for support, and files the error.

What the retries cost

Publishing is the only billable call: $0.00002 each, verified 2026-07-26. Consume, ack, nack, stats and redrive are free and rate-limited rather than metered, which is why redelivery costs nothing and a five-rung ladder costs five publishes. A subscriber that goes down for a day and takes 4,000 events with it, retried five times each, is 20,000 publishes.

Check what you actually spent, by capability:

curl -sS "https://api.infrai.cc/v1/account/usage" \
  -H "Authorization: Bearer ${INFRAI_API_KEY}"

The response carries total_cost, total_calls and a breakdown array keyed by capability, so queue.publish shows up as its own line. Rates drift downward over time and discount campaigns run, so treat the figure above as an upper bound rather than a promise, and read the endpoint on the day you’re budgeting.

References

• Infrai queue API reference
• QUEUE_MESSAGE_NOT_IN_FLIGHT error reference
• Amazon SQS dead-letter queues
• Cron job alternatives for delayed webhooks