Stopping a retried agent from submitting the same batch job twice

What Idempotency-Key does and doesn't do on Infrai's batch route right now, and the client-side ledger that actually keeps a flaky-network retry from double-charging.

Short answer: send an Idempotency-Key header, but don’t let it be your only defence. Infrai documents the header and every response carries an idempotent_replay flag, so the plumbing is there — yet when we submitted the same batch payload twice with an identical key on 26 July 2026, we got two different batch_id values and two charges. Until that closes, the guarantee your agent needs has to live in your own code.

This page is the belt-and-braces version: what the platform gives you, where we measured it falling short, and the retry wrapper we’d actually ship. It’s dull work, and it’s the difference between a network blip costing nothing and costing a full batch run.

What the platform provides

Infrai’s wire conventions define Idempotency-Key as a request header on cost-incurring calls. When you omit it the server derives one as sha256(account_id + request_id + capability + content_hash), with a default dedup window of 24 hours — configurable between 1 hour and 7 days. Read calls need no key at all; a GET never charges twice.

Every response, on every route, ends with a metadata block that includes the flag you care about:

{
  "ok": true,
  "data": { "batch_id": "batch_4b0d13535584416d07a4aa49", "state": "completed", "total_count": 1 },
  "metadata": {
    "request_id": "req_84720c07b4b2487695f00480",
    "latency_ms": 720,
    "cost_usd": 0.0,
    "idempotent_replay": false
  }
}

idempotent_replay: true means the platform recognised your key and returned the stored result instead of doing the work again. That’s the field to assert on in tests and to alert on in production.

The measurement

We ran the obvious experiment. Same body, same header, twice in a row, two seconds apart:

export INFRAI_API_KEY="your_infrai_api_key"

curl -sS https://api.infrai.cc/v1/ai/batch/submit \
  -H "Authorization: Bearer ${INFRAI_API_KEY}" \
  -H "Content-Type: application/json" \
  -H "Idempotency-Key: nightly-tagging-2026-07-26" \
  -d '{
        "requests": [
          {"model": "glm-4-flash", "messages": [{"role": "user", "content": "Reply with the single word: ok"}], "max_tokens": 5}
        ],
        "metadata": {"client_token": "nightly-tagging-2026-07-26"},
        "store": true
      }'

Two distinct batch_id values came back, both with idempotent_replay: false. We repeated it on POST /v1/ai/rerank, which is billed per request, and both calls settled at their own cost — no replay, no dedup. So the honest status today is: the header is accepted and ignored on these routes.

That’s a limitation, not a disaster, because the recovery primitives are all present. You just have to hold the state.

The gap that forces a client-side ledger

Here’s the part that surprised us. If your process dies after sending the request but before reading the response, you’d normally recover by listing recent jobs and looking for yours. That doesn’t work:

curl -sS https://api.infrai.cc/v1/ai/batch/list \
  -H "Authorization: Bearer ${INFRAI_API_KEY}"

{ "ok": true, "data": { "batches": [], "total_count": 0, "next_cursor": null } }

The list came back empty on an account with batches that GET /v1/ai/batch/status/{id} happily reports as completed. You cannot enumerate your jobs, and the metadata you stamped onto the submission isn’t searchable. The batch_id in the response is therefore the only handle that exists — lose it and the job is orphaned, still running, still billing.

Which settles the design question. Write the intent down before you call.

The wrapper we’d ship

The pattern is a two-phase local record: mark the intent as pending with a stable token derived from the payload, call once, record the returned batch_id, and never blind-retry a POST whose outcome you don’t know.

import { createHash } from "node:crypto";

const key = process.env.INFRAI_API_KEY;
if (!key) throw new Error("set INFRAI_API_KEY before running this");

// Stand-in for your real store — use Postgres with a UNIQUE index on token.
const ledger = new Map();

const stableToken = (payload) =>
  createHash("sha256").update(JSON.stringify(payload)).digest("hex").slice(0, 32);

async function submitOnce(payload) {
  const token = stableToken(payload);
  const seen = ledger.get(token);
  if (seen?.batch_id) return { batch_id: seen.batch_id, reused: true };
  if (seen?.state === "in_flight") throw new Error(`ambiguous submit for ${token} — resolve by hand`);

  ledger.set(token, { state: "in_flight" });

  const res = await fetch("https://api.infrai.cc/v1/ai/batch/submit", {
    method: "POST",
    headers: {
      Authorization: `Bearer ${key}`,
      "Content-Type": "application/json",
      "Idempotency-Key": token,
    },
    body: JSON.stringify({ ...payload, metadata: { client_token: token } }),
  });

  const body = await res.json();
  if (!res.ok || body.ok === false) {
    ledger.delete(token); // a 4xx never started work, so retrying is safe
    throw new Error(`submit failed ${res.status}: ${body.error?.code ?? "unknown"}`);
  }

  if (body.metadata?.idempotent_replay) console.warn(`server replayed ${token}`);
  ledger.set(token, { state: "submitted", batch_id: body.data.batch_id });
  return { batch_id: body.data.batch_id, reused: false };
}

const job = {
  requests: [
    { model: "glm-4-flash", messages: [{ role: "user", content: "Tag: refund request" }], max_tokens: 5 },
  ],
  store: true,
};

console.log(await submitOnce(job));
console.log(await submitOnce(job)); // second call never reaches the network

Three things make it work. The token is derived from the payload, so a retry of the same logical job produces the same token instead of a fresh UUID. The in_flight marker turns the dangerous case — request sent, response lost — into a loud error a human resolves, rather than a silent duplicate. And a 4xx clears the marker, because a rejected request did no work and is safe to send again.

Retry policy, briefly

Retry on 429 and 5xx. Never retry a 400.

Infrai’s error envelope carries a retryable boolean, so you don’t have to maintain a status-code table by hand — VENDOR_DOWN comes back retryable: true, INVALID_ARGUMENT and VENDOR_NOT_CONFIGURED come back false. Exponential backoff with jitter, capped at four attempts, is plenty; the batch route completed a one-row job in about 720 ms in our testing, so long retry ladders just widen the window in which two submits can race.

If you do end up with a duplicate, cancel the loser and move on:

curl -sS -X POST https://api.infrai.cc/v1/ai/batch/cancel/batch_4b0d13535584416d07a4aa49 \
  -H "Authorization: Bearer ${INFRAI_API_KEY}" \
  -H "Content-Type: application/json" \
  -d '{}'

Cancel is free and doesn’t touch your trial credit. Submit is billed per call — the discovery manifest quotes $0.001, verified 26 July 2026, with the per-row model inference billed on top at pass-through. Read GET /v1/discovery for the current figure; these rates trend down rather than up.

Weighing the options

OpenAI’s own Batch API and Amazon Bedrock batch inference both hand you a job id you must persist too, so the ledger isn’t Infrai-specific tax — it’s the shape of every async submit API. If your batches are large and infrequent enough that a human notices a duplicate the same day, honestly, detect-and-cancel is fine and you can skip the ledger. Where Infrai helps is that the reconciliation query lives on the same key: GET /v1/account/usage returns a per-capability breakdown with its own rows for ai.batch.submit and ai.chat, so “did we pay twice last night” is one call rather than a support ticket to two vendors.

References

• Infrai wire conventions and idempotency
• Infrai AI Runtime API reference
• Making retries safe with idempotent APIs — Amazon Builders’ Library