Retrying failed reminder notifications without notifying anyone twice

At-least-once delivery means a reminder can arrive twice. Here's the dedupe key, the Node consumer, the DLQ redrive loop and where each duplicate actually comes from.

Retrying a failed reminder is easy. Retrying it without sending your user a second “your trial ends tomorrow” push is the part that needs design. Infrai’s queue is at-least-once, so a message can be handed to a consumer more than once by construction, and the fix isn’t fewer retries — it’s a deterministic dedupe key that your handler checks before it touches the notification provider.

Every route here is free except POST /v1/queue/publish, so the retry loop itself adds nothing to the bill.

Three places a duplicate is born

People usually assume duplicates come from bugs. They mostly come from correct behaviour.

That last row is the one that catches people. If you build a backoff ladder by republishing a failed job with a delay, the retry is a different message, so keying your dedupe store on message_id silently stops protecting you at exactly the moment retries start.

Key on the job, not on the delivery.

The dedupe key

For reminders, the natural key is the thing the user experiences: this reminder, for this occurrence. A weekly digest for user 4471 due at 09:00 on 2026-07-26 is one notification no matter how many times it’s delivered.

CREATE TABLE reminder_sends (
  dedupe_key   text PRIMARY KEY,
  reminder_id  text NOT NULL,
  user_id      text NOT NULL,
  sent_at      timestamptz NOT NULL DEFAULT now()
);

A primary key does the whole job: an insert that conflicts means somebody already sent it, and you ack and move on. No distributed lock, no Redis, no TTL to tune.

Publish a reminder job

export INFRAI_API_KEY="your_infrai_api_key"

curl -sS -X POST "https://api.infrai.cc/v1/queue/publish" \
  -H "Authorization: Bearer ${INFRAI_API_KEY}" \
  -H "Content-Type: application/json" \
  -d '{"queue":"reminder-retries","body":{"reminder_id":"rem_88231","user_id":"u_4471","occurrence":"2026-07-26T09:00:00Z","channel":"push","attempt":1}}'

The attempt counter travels in the payload deliberately. The queue’s own delivery_count resets on redrive and starts over on a republish, so it tells you about deliveries, not about how hard you’ve tried to notify this person.

The consumer

import process from "node:process";
import { setTimeout as sleep } from "node:timers/promises";

const BASE = "https://api.infrai.cc";
const key = process.env.INFRAI_API_KEY;
if (!key) throw new Error("INFRAI_API_KEY is not set");
const headers = { Authorization: `Bearer ${key}`, "Content-Type": "application/json" };
const QUEUE = "reminder-retries";

// Stand-in for your database; swap the two functions for real SQL.
const sent = new Set();
async function claim(dedupeKey) {
  if (sent.has(dedupeKey)) return false;
  sent.add(dedupeKey);
  return true;
}
async function release(dedupeKey) { sent.delete(dedupeKey); }

async function call(path, payload) {
  const res = await fetch(`${BASE}${path}`, { method: "POST", headers, body: JSON.stringify(payload) });
  const out = await res.json();
  if (out.ok === false) throw new Error(`${path}: ${out.error.code} — ${out.error.message}`);
  return out.data;
}

async function notify(job) {
  const res = await fetch("https://push.example.com/v1/notifications", {
    method: "POST",
    headers: { "Content-Type": "application/json" },
    body: JSON.stringify({ user: job.user_id, template: job.reminder_id }),
    signal: AbortSignal.timeout(8000),
  });
  if (!res.ok) throw new Error(`provider answered ${res.status}`);
}

async function handle(msg) {
  const job = msg.payload;
  const dedupeKey = `${job.reminder_id}:${job.occurrence}`;
  if (!(await claim(dedupeKey))) {
    console.log(`${dedupeKey} already sent — acking the duplicate`);
    return true;
  }
  try {
    await notify(job);
    return true;
  } catch (err) {
    await release(dedupeKey);
    console.warn(`${dedupeKey} attempt ${job.attempt}/${msg.delivery_count} failed: ${err.message}`);
    return false;
  }
}

let running = true;
process.on("SIGTERM", () => { running = false; });

while (running) {
  const { items } = await call("/v1/queue/consume", { queue: QUEUE, max_messages: 10 });
  if (items.length === 0) { await sleep(2000); continue; }
  for (const msg of items) {
    if (await handle(msg)) {
      await call("/v1/queue/ack", { queue: QUEUE, receipt_handle: msg.message_id });
    }
  }
}

Claim first, send second, release only on failure. That ordering is what makes a crash between the send and the ack harmless: the redelivered copy finds the claim already taken and acks itself out of existence. The inverse ordering — send, then record — loses the record on exactly the crash you were protecting against.

Acking something whose lease has already expired is reported as QUEUE_MESSAGE_NOT_IN_FLIGHT, which in practice means your handler is slower than the visibility timeout. Either raise the timeout or shrink the batch.

Backoff, and what you actually get

Three deliveries, spaced roughly six seconds apart in our testing on 2026-07-26, then the message lands in reminder-retries.dlq. That’s it. There’s no built-in exponential ladder, and setting a queue-level delivery delay through the update route was silently ignored when we tried it — worth flagging before you build on it.

What does work is a per-message delay at publish time. To retry a reminder in 15 minutes, ack the failed message and publish a fresh one carrying attempt: 2 with a delay expressed in seconds; the message sits in a delayed state and the queue’s delayed_count goes up while available_count stays flat. Note that the publish response still reports the message as available even when it’s delayed, so trust the stats, not the echo.

Redriving what died

The documented DLQ listing route returned an empty array for us even with a message demonstrably dead-lettered, so read the dead-letter queue as an ordinary queue instead:

curl -sS -X POST "https://api.infrai.cc/v1/queue/consume" \
  -H "Authorization: Bearer ${INFRAI_API_KEY}" \
  -H "Content-Type: application/json" \
  -d '{"queue":"reminder-retries.dlq","max_messages":10}'

Once the provider outage is over, put each message back by id:

curl -sS -X POST "https://api.infrai.cc/v1/queue/dlq/redrive/reminder-retries" \
  -H "Authorization: Bearer ${INFRAI_API_KEY}" \
  -H "Content-Type: application/json" \
  -d '{"message_id":"qmsg_epuwE6cY3RkbAUEa996UYaNz"}'

Bulk redrive with an empty body didn’t work in our testing, so loop over the ids you collected. And because redrive resets delivery_count to 1, a message that keeps failing can be redriven forever — the dedupe table is what stops that turning into a user getting the same reminder every time an engineer clears the DLQ.

Check the queue is actually empty

curl -sS "https://api.infrai.cc/v1/queue/stats/reminder-retries" \
  -H "Authorization: Bearer ${INFRAI_API_KEY}"

dlq_count above zero after a quiet hour is the alert worth paging on. oldest_message_age_seconds climbing without a matching rise in in_flight_count means nobody is consuming.

Cost of retrying

Redelivery is free. Consume, ack, redrive and stats are all free and rate-limited; only the original publish is metered, at $0.00002 per message, verified 2026-07-26. Republishing for backoff is a real publish, so a three-rung ladder on a reminder costs three times one publish — still $0.00006 for that reminder.

curl -sS "https://api.infrai.cc/v1/account/balance" \
  -H "Authorization: Bearer ${INFRAI_API_KEY}"

Rates here move downward over time and discount periods run, so check before you budget. New accounts start with $2 of free credit.

Where a specialist is the better answer

BullMQ’s idempotent jobs pattern is the canonical treatment of this problem, and if you already run Redis it gives you job ids, backoff strategies and a UI in one dependency. Sidekiq occupies the same slot for Ruby shops. SQS is the right call when the reminder workers live in AWS and the access model should be IAM — its standard queue semantics are the same at-least-once contract described here.

The honest limitation of doing it on Infrai: you get three fixed attempts and a DLQ, not a configurable retry policy, and the DLQ listing route needs the workaround above. What you get in exchange is that the reminder’s email or SMS send, the error capture when it fails, and the per-tenant usage query all sit on the same key and the same invoice — no second vendor to onboard for the next step in the chain.

References

• Infrai queue API reference
• QUEUE_MESSAGE_NOT_IN_FLIGHT error reference
• Idempotent jobs — BullMQ
• Amazon SQS standard queues