Give each transcode to exactly one homelab box, and reclaim it if that box dies

Lease-based queue delivery is how you spread ffmpeg work across machines safely. What Infrai's visibility timeout, delivery count and dead-letter queue do in practice.

What you want is a lease, not a lock. Each worker asks Infrai’s queue for a message, gets an exclusive hold on it for a fixed number of seconds, and either acknowledges it or lets the hold lapse. While the lease is live no other machine can see that message; when a box dies mid-transcode the lease expires on its own and the next worker to poll picks the job up, with a delivery_count that tells you it’s a second attempt.

Say the honest thing up front: this is at-least-once delivery, not exactly-once. You get “one machine at a time”, which is the guarantee that stops two boxes writing the same output file. Genuine exactly-once needs the output write to be idempotent, and that part is yours — name the output after the asset id and overwrite, and a duplicate attempt becomes wasted CPU rather than a corrupt MP4.

Set the queue up once

Create the dead-letter queue first, then the working queue that points at it. The lease length is the parameter that decides whether this works at all, so size it against your slowest job, not your median one.

curl -X POST https://api.infrai.cc/v1/queue/create \
  -H "Authorization: Bearer ${INFRAI_API_KEY}" \
  -H "Content-Type: application/json" \
  -d '{"name": "transcode-dlq"}'

curl -X POST https://api.infrai.cc/v1/queue/create \
  -H "Authorization: Bearer ${INFRAI_API_KEY}" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "transcode",
    "type": "standard",
    "dead_letter_queue": "transcode-dlq",
    "max_retries": 3,
    "visibility_timeout_default": 2700,
    "message_retention_days": 4
  }'

2700 seconds is 45 minutes — generous for a 1080p H.264 pass on a mini PC, and deliberately longer than the worst case. Undersize it and you get the failure this whole design exists to avoid: box A is still encoding when the lease lapses, box B starts the same asset, and they race to write the same file. Oversize it and a dead box’s job sits idle for up to the lease length before anyone retries it. That trade-off is the real design decision here.

Defaults worth knowing: visibility_timeout_default is 300 seconds if you don’t set it, retention is 14 days, and a message body caps at 256 KB — so you enqueue a path or an object key, never the video.

Enqueue one message per asset

curl -X POST https://api.infrai.cc/v1/queue/publish \
  -H "Authorization: Bearer ${INFRAI_API_KEY}" \
  -H "Content-Type: application/json" \
  -d '{
    "queue": "kh-cron-transcode",
    "payload": {"asset_id": "vid_001", "preset": "h264_1080p", "src": "s3://ingest/vid_001.mkv"},
    "idempotency_key": "transcode-vid_001-h264_1080p"
  }'

{
  "ok": true,
  "data": {
    "message_id": "qmsg_ajoYxNd3l30i1mdA7BNRr2bl",
    "queue": "kh-cron-transcode",
    "payload": {"asset_id": "vid_001", "preset": "h264_1080p", "src": "s3://ingest/vid_001.mkv"},
    "status": "available",
    "delivery_count": 0,
    "published_at": "2026-07-26T05:06:33.682572Z"
  }
}

The idempotency_key is your protection against the publisher double-submitting, which is a different problem from the consumer double-running. Both need handling and neither solves the other.

Scanning an ingest folder is a job for a schedule, and it’s on the same key:

curl -X POST https://api.infrai.cc/v1/cron/create \
  -H "Authorization: Bearer ${INFRAI_API_KEY}" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "ingest-scan",
    "task": "https://example.com/hooks/scan-ingest",
    "cron_expr": "*/10 * * * *",
    "timezone": "UTC",
    "overlap_policy": "skip",
    "timeout_seconds": 120
  }'

The worker

Runs on every box. No coordination between machines, no shared filesystem lock, no leader election — the lease does that work.

import json
import os
import subprocess
import sys
import time
import urllib.error
import urllib.request

KEY = os.environ.get("INFRAI_API_KEY")
if not KEY:
    sys.exit("INFRAI_API_KEY is not set")

QUEUE = os.environ.get("TRANSCODE_QUEUE", "kh-cron-transcode")
BASE = "https://api.infrai.cc"
OUT_DIR = os.environ.get("TRANSCODE_OUT", "/srv/media/out")


def post(path, body):
    req = urllib.request.Request(
        BASE + path,
        method="POST",
        data=json.dumps(body).encode(),
        headers={"Authorization": f"Bearer {KEY}", "Content-Type": "application/json"},
    )
    try:
        with urllib.request.urlopen(req, timeout=30) as resp:
            return json.loads(resp.read())["data"]
    except urllib.error.HTTPError as exc:
        detail = json.loads(exc.read() or b"{}").get("error", {})
        raise RuntimeError(f"{path} -> {exc.code} {detail.get('message', '')}") from exc


def transcode(payload):
    asset = payload["asset_id"]
    dest = os.path.join(OUT_DIR, f"{asset}.mp4")
    tmp = dest + ".part"
    subprocess.run(
        ["ffmpeg", "-y", "-i", payload["src"], "-c:v", "libx264", "-preset", "medium", tmp],
        check=True,
    )
    os.replace(tmp, dest)  # atomic: a duplicate attempt overwrites, never interleaves
    return dest


while True:
    batch = post("/v1/queue/consume", {"queue": QUEUE, "max_messages": 1, "visibility_timeout": 2700})
    if not batch["items"]:
        time.sleep(5)
        continue

    msg = batch["items"][0]
    print(f"lease {msg['message_id']} attempt {msg['delivery_count']}", flush=True)
    try:
        out = transcode(msg["payload"])
        result = post("/v1/queue/ack", {"queue": QUEUE, "message_id": msg["message_id"]})
        print("done", out, "acked" if result["acked"] else "LEASE ALREADY EXPIRED", flush=True)
    except subprocess.CalledProcessError as exc:
        print(f"ffmpeg failed rc={exc.returncode}", flush=True)
        post("/v1/queue/nack", {"queue": QUEUE, "message_id": msg["message_id"], "requeue": True})

Three details in there are load-bearing. max_messages is capped at 10 by the API — ask for 50 and you get a 400 saying so — but for long jobs you want 1 anyway, because a batch of five leases all expire on the same clock while you’re still on the first. The .part rename makes the output atomic. And the acked flag gets checked: acknowledging a message whose lease already lapsed returns HTTP 200 with acked: false rather than an error, so a worker that ignores the response will happily believe it finished a job that’s already been handed to someone else.

Watching a box die

We measured this rather than assuming it. With a 30-second lease, a message consumed and then abandoned came back on the next poll after the lease expired, with delivery_count incremented from 1 to 2 — no intervention, no sweeper job.

The queue’s counters make this observable without instrumenting the workers:

curl https://api.infrai.cc/v1/queue/stats/kh-cron-transcode \
  -H "Authorization: Bearer ${INFRAI_API_KEY}"

{
  "queue": "kh-cron-transcode",
  "message_count": 2,
  "available_count": 0,
  "in_flight_count": 2,
  "delayed_count": 0,
  "dlq_count": 1,
  "oldest_message_age_seconds": 0
}

An in_flight_count that never falls is your signal that a machine went away mid-lease. A rising dlq_count is a poison asset — a truncated source file that kills ffmpeg every time.

Draining the dead letters

A message that exhausts max_retries, or that you nack with requeue: false, lands on the queue you named. Read it by consuming that queue directly:

curl -X POST https://api.infrai.cc/v1/queue/consume \
  -H "Authorization: Bearer ${INFRAI_API_KEY}" \
  -H "Content-Type: application/json" \
  -d '{"queue": "kh-cron-transcode-dlq", "max_messages": 10}'

Worth flagging: in our testing GET /v1/queue/dlq/list/{queue} came back with an empty items array even when stats reported dlq_count: 1, so consuming the DLQ by name is the reliable route today. Fix the source file, republish, and the asset re-enters the normal path.

Where something else fits better

If your transcode fleet is really a workflow — probe, then transcode three renditions, then package, then notify — a durable workflow engine models that better than a flat queue, and you’d be better off there. A single long shell command per asset is exactly the shape a lease queue is for.

What it costs

Publishing is the only billable step: $0.00002 per message on Infrai as verified 2026-07-26, with consume, ack, nack, stats and the whole cron namespace free and rate-limited. Ten thousand transcodes is about $0.20 in publishes. New accounts start with $2 in credit, which covers roughly 99,999 publishes before you pay anything.

curl "https://api.infrai.cc/v1/discovery" \
  -H "Authorization: Bearer ${INFRAI_API_KEY}" \
  | python3 -c "import json,sys; print([c['billing'] for c in json.load(sys.stdin)['capabilities'] if c['id']=='queue.publish'])"

Rates here have trended down and campaigns run, so the live figure may be lower than the one printed above — read it from that call rather than from this page.

The cost argument isn’t really the point though. Once the fleet works, the next three things you’ll want are somewhere to put the finished MP4, an email when a batch completes, and a record of the ffmpeg failures. On Infrai those are routes on the key you already have, which is a different proposition from adding three more vendors to a homelab you built to avoid paying for things.

References

• https://docs.infrai.cc/en/api/queue/
• https://docs.infrai.cc/en/api/cron/
• https://github.com/OptimalBits/bull/discussions/2098
• https://dev.to/sfundomhlungu/building-a-distributed-video-transcoding-system-with-nodejs-3ko2