Long-running jobs and the 15-minute wall: checkpoint instead of extending

Your platform kills the function at 900 seconds and the message lease expires under it. Split the work on a cursor so every chunk finishes inside both clocks.

A job that takes forty minutes cannot be made to fit in a fifteen-minute function by asking for more time. The move that works is to stop treating it as one job: give each unit of work a cursor, do as much as fits in a bounded window, publish the continuation, and let the next worker pick up where you stopped. On Infrai’s queue that’s one extra publish per chunk and no orchestration layer.

The reason people reach for a bigger timeout first is that there appear to be two knobs. There are actually two clocks, and Infrai only lets you turn one of them.

Two clocks, and which one you control

The first is your compute platform’s ceiling. AWS Lambda stops at 900 seconds, most serverless HTTP handlers stop far sooner, and a scheduled HTTP job on Infrai carries a timeout_seconds that sits at 300 by default. The second is the message lease: when a worker consumes a message it becomes invisible for the queue’s visibility_timeout_default, and if the worker hasn’t acked by then, the message comes back and someone else starts the same job.

Here’s the limitation that shapes the whole design: there’s no call to extend a lease mid-flight. SQS has ChangeMessageVisibility and Sidekiq workers can heartbeat; the queue namespace here has nothing equivalent, so a lease is a promise you make once, at consume time.

What you can do is widen it for the whole queue.

export INFRAI_API_KEY="your_infrai_api_key"

curl -sS -X POST "https://api.infrai.cc/v1/queue/create" \
  -H "Authorization: Bearer ${INFRAI_API_KEY}" \
  -H "Content-Type: application/json" \
  -d '{"name":"ledger-export","type":"standard","dlq":"ledger-export-dlq"}'

curl -sS -X PATCH "https://api.infrai.cc/v1/queue/update/ledger-export" \
  -H "Authorization: Bearer ${INFRAI_API_KEY}" \
  -H "Content-Type: application/json" \
  -d '{"visibility_timeout_default":1800}'

{
  "ok": true,
  "data": {
    "name": "ledger-export",
    "type": "standard",
    "message_retention_days": 14,
    "max_message_size_kb": 256,
    "visibility_timeout_default": 1800,
    "max_receive_count": 3,
    "dlq_name": "ledger-export-dlq"
  }
}

We’ve set that as high as 7200 on a test queue without complaint. Two caveats come with a wide lease, though. A worker that dies silently now holds its message hostage for the full half hour before anyone retries it, and max_receive_count stays at 3 regardless of what you pass at create time — we tried 8 and got 3 back — so three crashed attempts at half an hour each is ninety minutes before the job dead-letters.

Chunk on a cursor, not on a count

“Process 1,000 rows per message” breaks the first time a row takes ten seconds. Chunk on a resumable position and a time budget instead: the message says where to start, the worker decides where to stop.

curl -sS -X POST "https://api.infrai.cc/v1/queue/publish" \
  -H "Authorization: Bearer ${INFRAI_API_KEY}" \
  -H "Content-Type: application/json" \
  -d '{"queue":"ledger-export","body":{"export_id":"exp_2261","account_id":"acct_88","cursor":null,"rows_done":0}}'

The worker below runs for at most 240 seconds of work per message, well inside both a 300-second function ceiling and the 1800-second lease. When its budget runs out it publishes the next chunk and only then acks the current one — that ordering is the whole safety argument. Publish-then-ack means a crash in between costs you a duplicate chunk; ack-then-publish means a crash in between silently loses the rest of the export.

// export-worker.mjs — node 22
import process from "node:process";
import { setTimeout as sleep } from "node:timers/promises";

const API = "https://api.infrai.cc";
const QUEUE = "ledger-export";
const WORK_BUDGET_MS = 240_000;       // stop well before the function ceiling
const PAGE = 500;

const key = process.env.INFRAI_API_KEY;
if (!key) throw new Error("set INFRAI_API_KEY");
const headers = { Authorization: `Bearer ${key}`, "Content-Type": "application/json" };

async function post(path, payload) {
  const res = await fetch(`${API}${path}`, { method: "POST", headers, body: JSON.stringify(payload) });
  const json = await res.json();
  if (!res.ok || json.ok === false) throw new Error(`${path} ${res.status}: ${json?.error?.message ?? "unknown"}`);
  return json.data;
}

// Your own paginated source. Returns { rows, nextCursor } with nextCursor null at the end.
async function fetchPage(accountId, cursor) {
  const url = new URL("https://api.example.com/ledger/entries");
  url.searchParams.set("account_id", accountId);
  url.searchParams.set("limit", String(PAGE));
  if (cursor) url.searchParams.set("cursor", cursor);
  const res = await fetch(url, { signal: AbortSignal.timeout(20_000) });
  if (!res.ok) throw new Error(`ledger page failed ${res.status}`);
  const body = await res.json();
  return { rows: body.items, nextCursor: body.next_cursor ?? null };
}

async function writeRows(exportId, rows) {
  const res = await fetch(`https://api.example.com/exports/${exportId}/append`, {
    method: "POST",
    headers: { "Content-Type": "application/json", "Idempotency-Key": `${exportId}:${rows[0]?.id ?? "empty"}` },
    body: JSON.stringify({ rows }),
  });
  if (!res.ok) throw new Error(`append failed ${res.status}`);
}

async function runChunk(job) {
  const deadline = Date.now() + WORK_BUDGET_MS;
  let { cursor, rows_done: done } = job;
  while (Date.now() < deadline) {
    const { rows, nextCursor } = await fetchPage(job.account_id, cursor);
    if (rows.length) await writeRows(job.export_id, rows);
    done += rows.length;
    cursor = nextCursor;
    if (!cursor) return { finished: true, cursor: null, done };
  }
  return { finished: false, cursor, done };
}

async function pass() {
  const { items } = await post("/v1/queue/consume", { queue: QUEUE, max_messages: 1 });
  for (const msg of items) {
    const job = msg.payload;
    try {
      const state = await runChunk(job);
      if (!state.finished) {
        await post("/v1/queue/publish", {
          queue: QUEUE,
          body: { ...job, cursor: state.cursor, rows_done: state.done },
        });
        console.log(`export ${job.export_id}: handed off at ${state.done} rows`);
      } else {
        console.log(`export ${job.export_id}: finished, ${state.done} rows`);
      }
      await post("/v1/queue/ack", { queue: QUEUE, receipt_handle: msg.message_id });
    } catch (err) {
      console.error(`export ${job.export_id} delivery ${msg.delivery_count}: ${err.message}`);
      // no ack — the lease lapses and the chunk is retried from its own cursor
    }
  }
  return items.length;
}

for (;;) {
  if (!(await pass())) await sleep(5000);
}

max_messages: 1 is deliberate. Leasing ten long-running messages at once means nine of them are ticking down their visibility timeout while the worker is busy with the first, which is exactly how you end up with duplicate chunks under load.

Sizing the budget

Keep the lease at roughly two to three times the work budget. Tighter and a slow-but-healthy chunk gets redelivered while it’s still running; much looser and a genuinely dead worker parks the job for ages.

Note the last row. Even on a box with no timeout at all, chunking is worth it, because deploys, OOM kills and spot reclamations are ceilings too — they just arrive unannounced.

Watching a long export

curl -sS "https://api.infrai.cc/v1/queue/stats/ledger-export" \
  -H "Authorization: Bearer ${INFRAI_API_KEY}"

in_flight_count of 1 with a slowly rising oldest_message_age_seconds is a healthy long chunk. dlq_count climbing means chunks are dying three times — read them by consuming ledger-export-dlq under its own name, since acking a message whose lease already expired returns QUEUE_MESSAGE_NOT_IN_FLIGHT and is the usual sign your budget is too close to the lease.

What the handoffs cost

Each continuation is one publish at $0.00002, verified 2026-07-26; consume, ack, stats and queue updates are free and rate-limited. A 40-minute export in 4-minute chunks is 10 publishes — $0.0002 for the whole run, which means chunk size is an engineering decision rather than a budget one. New accounts start with $2 of credit. Rates on this surface have moved down over time, so confirm before planning:

curl -sS "https://api.infrai.cc/v1/discovery" \
  -H "Authorization: Bearer ${INFRAI_API_KEY}" \
  | jq '.capabilities[] | select(.id=="queue.publish") | .billing'

When to reach for something heavier

Temporal exists for precisely this problem and solves it properly: activity heartbeats, automatic resumption, and a durable execution history you can query. If your long job has branches, compensation steps or human approvals, stop hand-rolling continuations and use it. Sidekiq and Celery both let a long worker heartbeat against a lease, which is a genuine advantage over what’s available here — that’s the trade-off for having no broker to run. And if the work is embarrassingly parallel rather than sequential, don’t chunk at all: publish one message per unit and let concurrency do the work.

References

• Infrai queue API reference
• QUEUE_MESSAGE_NOT_IN_FLIGHT error reference
• AWS Lambda quotas, including the 900-second maximum
• Detecting activity failures — Temporal docs