Bucket events to a thumbnail worker: webhook, queue, retries

Wire object-storage notifications into a Node queue so a worker derives thumbnails after upload — with the upload-leg constraint stated up front.

Push the original in through your own backend, subscribe the bucket to a notification URL, and let that URL do exactly one thing: drop a job on a queue. A worker then reads the original, writes the derived sizes under a separate prefix, and acks. On Infrai that is four storage routes plus one queue — and the upload leg carries a constraint you should know before you draw the diagram.

The constraint sits on the first hop. POST /v1/storage/bucket/set_cors/{bucket} stores an allowed-origin rule set and GET /v1/storage/bucket/get/{bucket} reads it back, but a preflight OPTIONS against a presigned PUT URL still answers 403 with no Access-Control-Allow-Origin, so a tab can’t PUT into the bucket today. If the true browser-to-storage leg is what you’re after, Cloudflare R2 or S3 will give it to you. If the original passes through your Node process anyway — which it usually does when you want size and MIME validation before anything is stored — the rest of this pipeline is unchanged.

The chain, in one line each

Browser → your POST /uploads route → PUT /v1/storage/object/put/{bucket}/{key} → bucket notification → your /hooks/storage endpoint → POST /v1/queue/publish → worker → derived objects.

Two of those hops are the interesting ones. The notification is at-least-once and its payload is not something you should trust blindly; the queue is where retries actually live.

Writing the original

Object bytes go up base64-encoded inside a JSON body. Build the payload in a file rather than inlining it — a 4 MB photo becomes a 5.3 MB base64 string and shell argument limits bite well before that.

IMG_B64=$(base64 -i lakeside.png | tr -d '\n')
cat > /tmp/put-original.json <<JSON
{"data_base64":"$IMG_B64","content_type":"image/png"}
JSON

curl -s -X PUT "https://api.infrai.cc/v1/storage/object/put/hub6-media-originals/originals/usr_8f21/2026/07/lakeside.png" \
  -H "Authorization: Bearer $INFRAI_API_KEY" \
  -H "Content-Type: application/json" \
  -H "Idempotency-Key: original-usr_8f21-lakeside-v1" \
  --data-binary @/tmp/put-original.json

Send the Idempotency-Key as a header. A replay of the same key with the same bytes returns the first result instead of writing a second object, and a replay with different bytes under that key is refused with IDEMPOTENCY_KEY_CONFLICT — which is the behaviour you want when a mobile client retries a flaky upload three times. Derive the key from something stable, like the user id plus a content hash. Derive it from a timestamp and every retry mints a fresh original.

Subscribing the bucket

curl -s -X POST "https://api.infrai.cc/v1/storage/bucket/set_notification/hub6-media-originals" \
  -H "Authorization: Bearer $INFRAI_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"events":["object.created","multipart.completed"],"target":{"url":"https://api.yourapp.com/hooks/storage"}}'

Supported event names are object.created, object.deleted and multipart.completed. Subscriptions are per bucket and reversible: GET /v1/storage/bucket/notifications/{bucket} lists what’s registered and DELETE /v1/storage/bucket/notification/delete/{bucket}/{subscription_id} takes one back off, so a staging endpoint you wired up in March isn’t yours forever. Register against a hostname you own and gate the handler on the bucket name regardless.

The callback arrives as a JSON POST carrying an X-Infrai-Event header and nothing else to authenticate itself with, which makes verify-then-act the honest pattern: confirm the key with a free head before the worker spends anything. Treat the field names below as the illustrative shape and confirm them against the storage reference before you depend on any of them:

{
  "event": "object.created",
  "type": "object.created",
  "bucket": "hub6-media-originals",
  "key": "originals/usr_8f21/2026/07/lakeside.png",
  "size_bytes": 2841733,
  "etag": "2cd8bde463f5d82aae0f0cec061d6b8f",
  "occurred_at": "2026-07-26T01:21:45Z"
}

The receiver: enqueue and get out

The endpoint that receives a storage event should not resize anything. It should validate, publish, and return 200 in a few milliseconds, because a slow webhook handler turns provider retries into a stampede.

import express from "express";

const app = express();
const API = "https://api.infrai.cc";
const KEY = process.env.INFRAI_API_KEY;
if (!KEY) throw new Error("INFRAI_API_KEY is not set");

app.post("/hooks/storage", express.json({ limit: "256kb" }), async (req, res) => {
  const evt = req.body ?? {};
  const kind = evt.event ?? evt.type;
  const bucket = evt.bucket ?? evt.bucket_name;
  const key = evt.key ?? evt.object_key;

  if (kind !== "object.created" || bucket !== "hub6-media-originals" || !key?.startsWith("originals/")) {
    return res.status(200).json({ ignored: true });
  }

  const r = await fetch(`${API}/v1/queue/publish`, {
    method: "POST",
    headers: { authorization: `Bearer ${KEY}`, "content-type": "application/json" },
    body: JSON.stringify({
      queue: "hub6-thumbnail-jobs",
      payload: { bucket, key, sizes: [320, 800] },
      idempotency_key: `thumb:${key}:${evt.etag ?? "na"}`,
    }),
  });

  if (!r.ok) {
    console.error("enqueue failed", r.status, await r.text());
    return res.status(500).json({ ok: false });
  }
  res.status(200).json({ ok: true });
});

app.listen(3000);

The idempotency_key is derived from the object key plus its etag. Re-delivery of the same event produces the same key, so a duplicate webhook doesn’t become a duplicate job.

The worker

POST /v1/queue/consume leases messages for a visibility window; POST /v1/queue/ack removes them. Anything you don’t ack comes back, which is what you want when a worker dies mid-resize.

import sharp from "sharp";

const API = "https://api.infrai.cc";
const KEY = process.env.INFRAI_API_KEY;
if (!KEY) throw new Error("INFRAI_API_KEY is not set");
const H = { authorization: `Bearer ${KEY}`, "content-type": "application/json" };

async function call(method, path, body) {
  const r = await fetch(`${API}${path}`, { method, headers: H, body: body ? JSON.stringify(body) : undefined });
  const j = await r.json();
  if (!r.ok || j.ok === false) throw new Error(`${method} ${path} -> ${r.status} ${JSON.stringify(j.error ?? j)}`);
  return j.data;
}

async function derive(msg) {
  const { bucket, key, sizes } = msg.payload;
  const src = await call("GET", `/v1/storage/object/get/${bucket}/${key}`);
  const original = Buffer.from(src.data_base64, "base64");
  const stem = key.replace(/^originals\//, "").replace(/\.[a-z0-9]+$/i, "");

  for (const width of sizes) {
    const out = await sharp(original).resize({ width, withoutEnlargement: true }).webp({ quality: 82 }).toBuffer();
    await call("PUT", `/v1/storage/object/put/${bucket}/derived/${stem}/thumb_${width}.webp`, {
      data_base64: out.toString("base64"),
      content_type: "image/webp",
    });
  }
}

for (;;) {
  const batch = await call("POST", "/v1/queue/consume", { queue: "hub6-thumbnail-jobs", max_messages: 5, visibility_timeout: 120 });
  if (!batch.items.length) { await new Promise((r) => setTimeout(r, 2000)); continue; }
  for (const msg of batch.items) {
    try {
      await derive(msg);
      await call("POST", "/v1/queue/ack", { queue: "hub6-thumbnail-jobs", message_id: msg.message_id });
    } catch (err) {
      console.error("derive failed", msg.message_id, "attempt", msg.delivery_count, err.message);
    }
  }
}

Note what makes a retry safe here: the derived key is a pure function of the original key and the width. Re-running the job overwrites the same three objects instead of accumulating thumb_320 (1).webp. If a message keeps failing, delivery_count climbs and the queue’s max_retries eventually parks it in the DLQ, which is where you want a corrupt upload to end up rather than in a hot loop.

Verifying it actually ran

curl -s -X GET "https://api.infrai.cc/v1/storage/object/list/hub6-media-originals?prefix=derived/usr_8f21/" \
  -H "Authorization: Bearer $INFRAI_API_KEY"

curl -s -X GET "https://api.infrai.cc/v1/storage/object/head/hub6-media-originals/derived/usr_8f21/2026/07/lakeside/thumb_320.webp" \
  -H "Authorization: Bearer $INFRAI_API_KEY"

head is the cheap probe your monitoring should use — it’s free, and it returns found: false with HTTP 200 rather than a 404, so check the field and not the status code.

What a run costs

Per image, one worker read plus N derived writes — and those two legs aren’t billed in the same unit. On Infrai, object/put is counted at $0.0001 per call, while object/get is metered on egress at $0.104 per GB of response body. So the read leg is priced by how heavy the original is that your worker pulls down, not by how many jobs ran, and a pipeline over 12 MB phone photos costs a different shape of money than one over 300 KB avatars. A queue publish is $0.00002 per message; presign, head, list and the notification routes are free but rate-limited. New accounts start with a $2 credit. Read today’s rates rather than trusting this paragraph:

curl -s -X GET "https://api.infrai.cc/v1/discovery" \
  -H "Authorization: Bearer $INFRAI_API_KEY" \
  | jq '.capabilities[] | select(.id | startswith("storage.object")) | {id, price: .billing.price_usd}'

Those figures were verified 2026-07-27, and the direction of travel is downward — rate cuts and campaigns happen, so what you read is as likely to be lower as equal. Bytes at rest are metered separately again, per GB-month.

When something else is the better pick

If images are the entire product and you want on-the-fly transformation URLs, stick with Cloudinary — a derived-object pipeline is strictly more work. The argument for doing it on Infrai isn’t that resizing is cheaper; it’s that the bucket, the queue, the DLQ and the cost attribution sit behind one credential and one invoice. The next question is already answered on the same key: POST /v1/email/send tells the user their gallery is ready, POST /v1/errors/capture catches the sharp exception, and POST /v1/cron/create schedules the sweep for orphaned originals — no second vendor, no second bill.

References

• Infrai storage API reference
• Infrai queue API reference
• BullMQ workers guide