A dead-letter redrive runbook for small Node.js SaaS teams

Find dead background jobs on Infrai, replay one by id or drain the whole backlog, and confirm the queue cleared — plus the boundaries worth knowing before you automate it.

Redrive is the boring half of a queue and the half you’ll actually be woken up for. A partner’s certificate expires, four hundred jobs exhaust their attempts, the certificate gets fixed, and now someone has to put those four hundred jobs back. On Infrai that recovery is three calls — read the count, list the dead messages, replay them — and none of the three costs anything.

For a team of five running a handful of background jobs, this is about as small as the operation gets.

Step one: know the count before you know the cause

A dead-letter queue is an ordinary queue, so you create it first and then point the working queue at it by name. max_retries is the nack budget: exceed it and the message moves across.

curl -sS -X POST "https://api.infrai.cc/v1/queue/create" \
  -H "Authorization: Bearer ${INFRAI_API_KEY}" \
  -H "Content-Type: application/json" \
  -d '{"name":"billing-jobs-dead","type":"standard"}'

curl -sS -X POST "https://api.infrai.cc/v1/queue/create" \
  -H "Authorization: Bearer ${INFRAI_API_KEY}" \
  -H "Content-Type: application/json" \
  -d '{"name":"billing-jobs","type":"standard","dead_letter_queue":"billing-jobs-dead","max_retries":2}'

The response echoes what the queue actually stored, which is the fastest way to confirm the two options landed:

{
  "ok": true,
  "data": {
    "name": "billing-jobs",
    "type": "standard",
    "visibility_timeout_default": 300,
    "max_message_size_kb": 256,
    "max_receive_count": 2,
    "dlq_name": "billing-jobs-dead"
  }
}

Leave dead_letter_queue out and you still get one — the default companion is named after the parent with a .dlq suffix. The count you page on lives in the stats route:

curl -sS "https://api.infrai.cc/v1/queue/stats/billing-jobs" \
  -H "Authorization: Bearer ${INFRAI_API_KEY}"

{
  "ok": true,
  "data": {
    "queue": "billing-jobs",
    "message_count": 0,
    "available_count": 0,
    "in_flight_count": 0,
    "delayed_count": 0,
    "dlq_count": 12,
    "oldest_message_age_seconds": 0
  }
}

A jump in dlq_count while message_count stays flat is the signature of a systemic failure — one broken dependency, many identical victims. A slowly-climbing dlq_count with healthy throughput is the other story entirely: a few malformed payloads that will never succeed and shouldn’t be replayed.

Step two: read the dead messages before you touch them

GET /v1/queue/dlq/list/{queue} takes the parent queue’s name and returns what is sitting in its dead-letter queue, so the item count lines up with the dlq_count you just read:

curl -sS "https://api.infrai.cc/v1/queue/dlq/list/billing-jobs" \
  -H "Authorization: Bearer ${INFRAI_API_KEY}"

{
  "ok": true,
  "data": {
    "items": [
      {
        "message_id": "qmsg_ID9gGfSmspd7d7ZN62joWPNY",
        "queue": "billing-jobs-dead",
        "payload": { "invoice_id": "inv_204", "action": "charge" },
        "status": "available",
        "delivery_count": 0,
        "published_at": "2026-07-27T11:44:11Z"
      }
    ],
    "next_cursor": null
  }
}

Reading them is the point of the exercise. Half the value of a dead-letter queue is triage: you look at ten payloads and learn whether you’re replaying an outage or deleting a bug.

Step three: replay one, or replay the batch

Redrive takes the parent queue’s name in the path. Send a message_id and exactly one message moves back:

curl -sS -X POST "https://api.infrai.cc/v1/queue/dlq/redrive/billing-jobs" \
  -H "Authorization: Bearer ${INFRAI_API_KEY}" \
  -H "Content-Type: application/json" \
  -d '{"message_id":"qmsg_ID9gGfSmspd7d7ZN62joWPNY"}'

{
  "ok": true,
  "data": { "queue": "billing-jobs", "redriven": 1, "message_id": "qmsg_ID9gGfSmspd7d7ZN62joWPNY" }
}

Omit message_id and the whole dead-letter queue goes back in one call, with redriven reporting how many moved. since narrows that to messages dead-lettered at or after an ISO-8601 timestamp, which is the difference between replaying this morning’s outage and replaying three weeks of accumulated poison. An id that isn’t in the dead-letter queue comes back as a 404 QUEUE_MESSAGE_NOT_FOUND rather than a silent success, so a loop can tell “already replayed” from “moved”.

The selective version — read, judge, replay what deserves it — is about thirty lines of Node 22:

import process from "node:process";

const BASE = "https://api.infrai.cc";
const PARENT = "billing-jobs";
const KEY = process.env.INFRAI_API_KEY;
if (!KEY) throw new Error("INFRAI_API_KEY is not set");
const H = { Authorization: `Bearer ${KEY}`, "Content-Type": "application/json" };

async function call(path, init) {
  const res = await fetch(`${BASE}${path}`, { headers: H, ...init });
  const out = await res.json();
  if (!out.ok) throw new Error(`${path}: ${out.error.code} — ${out.error.message}`);
  return out.data;
}

// Replay everything whose payload passes a sanity check; leave the rest for a human.
export async function redriveSelectively(accept = () => true) {
  const { items } = await call(`/v1/queue/dlq/list/${PARENT}`, { method: "GET" });
  let moved = 0;
  let skipped = 0;
  for (const msg of items) {
    if (!accept(msg.payload)) {
      skipped++;
      console.warn(`skipping ${msg.message_id}: ${JSON.stringify(msg.payload).slice(0, 120)}`);
      continue;
    }
    await call(`/v1/queue/dlq/redrive/${PARENT}`, { method: "POST", body: JSON.stringify({ message_id: msg.message_id }) });
    moved++;
  }
  return { moved, skipped };
}

const result = await redriveSelectively((p) => typeof p?.invoice_id === "string");
console.log(`redriven ${result.moved}, left alone ${result.skipped}`);

Run that behind a scheduled job once an hour and you have an automatic redrive service; run it by hand after an incident and you have a runbook. The accept predicate is doing real work — replaying a payload that failed because it’s malformed just burns the retry budget again and lands it back where it started. Then confirm with the same stats call you started from: dlq_count should fall to zero and available_count should rise by what you moved.

Where a specialist is still the better buy

Buy BullMQ if your jobs are already Node-side and you want a job dashboard, priorities and repeatable jobs against a Redis you’re running anyway — its failed-set tooling is deeper than a redrive endpoint and it costs nothing extra once Redis exists. Amazon SQS is the better pick when the rest of the system is in one AWS account and the queue needs to sit inside that IAM boundary. Switching queue vendors purely for redrive ergonomics would be a poor trade-off in either direction.

The call after the redrive is on the same key

A redrive puts the work back; it doesn’t tell you why the work died. That follow-up — capture the poison payload as an error event with POST /v1/errors/capture, then page the on-call with POST /v1/email/send — runs on the same key you just used for the redrive, with no second account, no second SDK and no second invoice at the end of the month. Nothing in this runbook needed a vendor beyond the one already in your env file.

That is the part a single-purpose queue library can’t hand you, and it’s worth more than any per-message rate.

Limits worth knowing, and what it costs

Delivery is at-least-once, so a redriven message can arrive twice — a duplicate delivery is normal operation, not a fault, and your worker has to be idempotent for redrive to be safe at all. Key each job on something stable (the invoice id, not the message id) and check before you act.

max_retries counts nacks, not wall-clock time. If a dependency is down for an hour, three fast failures burn the budget in seconds and the whole batch lands in the dead-letter queue — that’s the trade-off for a delivery counter that doesn’t know why you failed. Space your worker’s own retries out before you nack.

max_messages on consume tops out at 10, so draining a large backlog is a loop rather than one big call.

Every call in this runbook is free: create, stats, dead-letter list, redrive, consume, ack and nack. Publish is the metered one, at $0.00002 per message (verified 2026-07-27), and a new account starts with $2 of credit. Rates here have moved downward over time, so read today’s number instead of trusting this line:

curl -sS "https://api.infrai.cc/v1/discovery" \
  -H "Authorization: Bearer ${INFRAI_API_KEY}" \
  | jq '.capabilities[] | select(.id | startswith("queue.")) | {id, billable: .billing.is_billable, price: .billing.price_usd}'

References

• Infrai queue API reference
• QUEUE_MESSAGE_NOT_FOUND error
• Dead-letter queues in Amazon Simple Queue Service
• Svix: DLQ redrive