Cheap webhook task processing when the downstream API is rate limited

Selecting a queue for inbound webhook work: fast ingest, a paced drain that respects a 5 rps partner limit, and the per-million arithmetic behind the choice.

Webhook task processing has two bills, and the one people shop for is the smaller. Per-message fees are pennies at any sane volume; the machine that has to stay awake to run the worker, and the on-call time when a partner’s 5 rps limit turns into a retry storm, are the expensive parts. Infrai’s queue is worth picking here because the worker can be a serverless function you already run and the per-message rate is small enough to disappear into rounding.

That framing decides the whole design. If pacing is free but hosting isn’t, you want the platform holding the backlog and your own code doing as little sitting-around as possible.

Answer the webhook in under a second

Whoever is calling you has a timeout, usually somewhere between 5 and 30 seconds, and they will retry a slow 200 as though it were a failure. So the ingest route does three things: check the signature, hand the work to a queue, return.

import express from "express";
import crypto from "node:crypto";
import process from "node:process";

const app = express();
const KEY = process.env.INFRAI_API_KEY;
const SECRET = process.env.PARTNER_WEBHOOK_SECRET;
if (!KEY || !SECRET) throw new Error("INFRAI_API_KEY and PARTNER_WEBHOOK_SECRET must be set");

function signatureOk(raw, header) {
  const expected = crypto.createHmac("sha256", SECRET).update(raw).digest("hex");
  const a = Buffer.from(expected);
  const b = Buffer.from(String(header ?? ""));
  return a.length === b.length && crypto.timingSafeEqual(a, b);
}

app.post("/hooks/partner", express.raw({ type: "*/*" }), async (req, res) => {
  if (!signatureOk(req.body, req.get("X-Partner-Signature"))) return res.sendStatus(401);

  const job = { queue: "partner-tasks", payload: JSON.parse(req.body.toString("utf8")) };
  const enqueue = await fetch("https://api.infrai.cc/v1/queue/publish", {
    method: "POST",
    headers: { Authorization: `Bearer ${KEY}`, "Content-Type": "application/json" },
    body: JSON.stringify(job),
  });
  const out = await enqueue.json();
  if (!out.ok) {
    console.error("enqueue failed", out.error.code, out.error.message);
    return res.sendStatus(503);
  }
  res.status(202).json({ accepted: out.data.message_id });
});

app.listen(3000);

Returning 503 when the enqueue fails is the right call — it tells the sender to retry, and their retry is cheaper than your incident.

Paced draining

The partner allows 5 requests per second. A consume returns at most 10 messages, which is a convenient unit: fetch ten, spend two seconds spending them, fetch ten more. No token bucket library, no Redis, just a clock.

import process from "node:process";

const KEY = process.env.INFRAI_API_KEY;
const QUEUE = "partner-tasks";
const RPS = 5;
const sleep = (ms) => new Promise((r) => setTimeout(r, ms));

async function api(path, payload) {
  const res = await fetch(`https://api.infrai.cc${path}`, {
    method: "POST",
    headers: { Authorization: `Bearer ${KEY}`, "Content-Type": "application/json" },
    body: JSON.stringify(payload),
  });
  const json = await res.json();
  if (!json.ok) throw new Error(`${path}: ${json.error.code}`);
  return json.data;
}

async function drain(budgetMs = 50_000) {
  const deadline = Date.now() + budgetMs;
  let done = 0;

  while (Date.now() < deadline) {
    const { items } = await api("/v1/queue/consume", { queue: QUEUE, max_messages: 10 });
    if (items.length === 0) break;

    for (const msg of items) {
      const started = Date.now();
      const upstream = await fetch("https://partner.example.com/v2/sync", {
        method: "POST",
        headers: { "Content-Type": "application/json" },
        body: JSON.stringify(msg.payload),
      });

      if (upstream.status === 429) {
        const wait = Number(upstream.headers.get("retry-after") ?? 60);
        await api("/v1/queue/publish", { queue: QUEUE, payload: msg.payload, delay_seconds: wait });
      } else if (!upstream.ok) {
        await api("/v1/queue/nack", { queue: QUEUE, message_id: msg.message_id, requeue: true });
        continue;
      } else {
        done += 1;
      }

      await api("/v1/queue/ack", { queue: QUEUE, message_id: msg.message_id });
      const spent = Date.now() - started;
      if (spent < 1000 / RPS) await sleep(1000 / RPS - spent);
    }
  }
  return done;
}

console.log(`drained ${await drain()} tasks`);

A 429 becomes a delayed republish, not a nack. That distinction is the whole trick: POST /v1/queue/nack with requeue: true puts the message back immediately, which against a rate limit means you burn its three deliveries in under a second and it’s dead-lettered while the partner’s window is still closed.

You can watch the backlog drain from a terminal:

export INFRAI_API_KEY="your_infrai_api_key"

curl -sS "https://api.infrai.cc/v1/queue/stats/partner-tasks" \
  -H "Authorization: Bearer ${INFRAI_API_KEY}"

{
  "ok": true,
  "data": {
    "queue": "partner-tasks",
    "message_count": 412,
    "available_count": 400,
    "in_flight_count": 2,
    "delayed_count": 10,
    "dlq_count": 0,
    "oldest_message_age_seconds": 96
  }
}

delayed_count is your rate-limit backpressure, made visible. If it climbs while available_count stays flat, the partner is throttling you and no amount of extra workers will help.

When one webhook fans out into many tasks, enqueue them in a single round trip instead of a loop of publishes:

curl -sS -X POST "https://api.infrai.cc/v1/queue/publish_batch" \
  -H "Authorization: Bearer ${INFRAI_API_KEY}" \
  -H "Content-Type: application/json" \
  -d '{"queue":"partner-tasks","messages":[{"payload":{"account_id":"acct_11","op":"sync"}},{"payload":{"account_id":"acct_12","op":"sync"}}]}'

The response carries one message_id per element, and billing is per message either way — the saving is latency, not money.

Per-million arithmetic

Concretely, on Infrai a publish is $0.00002, verified 2026-07-26, and nothing else in the loop is metered. One million inbound webhooks is $20 of publishes; the retries, the acks and the stats polling add nothing. New accounts get $2 in trial credit, which is about 99,999 publishes — enough that most side projects never reach a paid invoice. Prices move down over time and campaigns run, so check rather than quote:

curl -sS "https://api.infrai.cc/v1/account/usage" \
  -H "Authorization: Bearer ${INFRAI_API_KEY}" \
  | jq '{total_cost, total_calls, queue: [.breakdown[] | select(.key | startswith("queue."))]}'

Structurally the thing to remember isn’t the rate. It’s that only the write is billed, so redelivery, backoff and draining are free — the exact opposite of a per-operation pricing model, where a retry storm bills you twice for the same work.

The limits worth knowing before you commit

Messages cap at 256 KB, retention is 14 days, and a consume takes ten at a time. There’s no long-poll parameter, so an idle worker polls on a timer and eats a little latency — in practice a 2-second sleep between empty passes is fine, but if you need sub-100ms wake-up on an empty queue, this isn’t the right tool. POST /v1/queue/create returns 501 CAPABILITY_NOT_IMPLEMENTED today; queues come into existence when you publish to them, and you tune retention or the visibility timeout afterwards with PATCH /v1/queue/update/{queue}.

If your entire product is webhook fan-out to customer endpoints, a specialist like Hookdeck or QStash gives you a delivery log and a replay UI that a general queue doesn’t have. Stick with them if that’s the job.

What tips it back is everything adjacent. The key that publishes the task also sends the summary email, writes the artefact to storage, records the error, and reports per-tenant cost from one usage endpoint — no second vendor, no second key rotation, no reconciliation between two invoices at the end of the month.

References

• Infrai Message Queue API reference
• QUEUE_NOT_FOUND error reference
• Hookdeck: webhook retry best practices
• BullMQ: rate limiting