Stopping a retried agent from submitting the same batch job twice

What Infrai's Idempotency-Key guarantees on the batch route, and the client-side ledger that closes the lost-response gap so a flaky-network retry can't double-charge you.

Short answer: send an Idempotency-Key header, and back it with a client-side record so a lost response can’t cost you a second batch. Infrai dedups on the header — a replayed submit returns the original batch_id at zero cost with idempotent_replay: true — so the server half is handled. The half nobody else can do for you is the case where your process never learns the outcome of the call it made, and that’s what the ledger below covers.

This page is the belt-and-braces version: what the platform gives you, the one gap that’s inherent to any network rather than to this API, and the retry wrapper we’d actually ship. It’s dull work, and it’s the difference between a network blip costing nothing and costing a full batch run.

What the platform provides

Infrai’s wire conventions define Idempotency-Key as a request header on cost-incurring calls. When you omit it the server derives one as sha256(account_id + request_id + capability + content_hash), with a default dedup window of 24 hours — configurable between 1 hour and 7 days. Read calls need no key at all; a GET never charges twice.

Every response, on every route, ends with a metadata block that includes the flag you care about:

{
  "ok": true,
  "data": { "batch_id": "batch_4b0d13535584416d07a4aa49", "state": "completed", "total_count": 1 },
  "metadata": {
    "request_id": "req_84720c07b4b2487695f00480",
    "latency_ms": 720,
    "cost_usd": 0.0,
    "idempotent_replay": false
  }
}

idempotent_replay: true means the platform recognised your key and returned the stored result instead of doing the work again. That’s the field to assert on in tests and to alert on in production.

The replay, measured

We ran the obvious experiment. Same body, same header, twice in a row, two seconds apart:

export INFRAI_API_KEY="your_infrai_api_key"

curl -sS https://api.infrai.cc/v1/ai/batch/submit \
  -H "Authorization: Bearer ${INFRAI_API_KEY}" \
  -H "Content-Type: application/json" \
  -H "Idempotency-Key: nightly-tagging-2026-07-26" \
  -d '{
        "requests": [
          {"model": "glm-4-flash", "messages": [{"role": "user", "content": "Reply with the single word: ok"}], "max_tokens": 5}
        ],
        "metadata": {"client_token": "nightly-tagging-2026-07-26"},
        "store": true
      }'

The second call returned the same batch_id as the first, with idempotent_replay: true and cost_usd: 0.0 — the platform recognised the key and handed back the stored result instead of doing the work again. We repeated it on POST /v1/ai/rerank, which is billed per request, and the replay landed the same way: one charge, not two. So the header does the job it advertises on these routes, and your retry loop can lean on it whenever the request actually reached the server.

That covers most of the failure surface. The one case it can’t cover — and it’s a genuine limitation to design around, not a bug to wait out — is the one where your client never finds out what happened.

The gap that’s inherent to networks

Here’s the part worth designing around. The header protects you when the server saw your request. It can’t help when your process dies after sending the request but before reading the response — because then you’re not holding the batch_id at all, and a payload-derived key only dedups a call you make again with the same body. If a retry regenerates the payload with a fresh UUID or a new timestamp, its key is different and the server has nothing to match on.

So the durable rule for an agent that retries on flaky networks isn’t “trust the header” — it’s “make the retry deterministic, and write down what you intended before you call.” The batch_id a submit returns is the handle you actually operate the job with; a submit whose response you never read leaves you holding a token but no id, and that’s the ambiguous state your wrapper has to make loud instead of silent.

Which settles the design question. Write the intent down before you call.

The wrapper we’d ship

The pattern is a two-phase local record: mark the intent as pending with a stable token derived from the payload, call once, record the returned batch_id, and never blind-retry a POST whose outcome you don’t know.

import { createHash } from "node:crypto";

const key = process.env.INFRAI_API_KEY;
if (!key) throw new Error("set INFRAI_API_KEY before running this");

// Stand-in for your real store — use Postgres with a UNIQUE index on token.
const ledger = new Map();

const stableToken = (payload) =>
  createHash("sha256").update(JSON.stringify(payload)).digest("hex").slice(0, 32);

async function submitOnce(payload) {
  const token = stableToken(payload);
  const seen = ledger.get(token);
  if (seen?.batch_id) return { batch_id: seen.batch_id, reused: true };
  if (seen?.state === "in_flight") throw new Error(`ambiguous submit for ${token} — resolve by hand`);

  ledger.set(token, { state: "in_flight" });

  const res = await fetch("https://api.infrai.cc/v1/ai/batch/submit", {
    method: "POST",
    headers: {
      Authorization: `Bearer ${key}`,
      "Content-Type": "application/json",
      "Idempotency-Key": token,
    },
    body: JSON.stringify({ ...payload, metadata: { client_token: token } }),
  });

  const body = await res.json();
  if (!res.ok || body.ok === false) {
    ledger.delete(token); // a 4xx never started work, so retrying is safe
    throw new Error(`submit failed ${res.status}: ${body.error?.code ?? "unknown"}`);
  }

  if (body.metadata?.idempotent_replay) console.warn(`server replayed ${token}`);
  ledger.set(token, { state: "submitted", batch_id: body.data.batch_id });
  return { batch_id: body.data.batch_id, reused: false };
}

const job = {
  requests: [
    { model: "glm-4-flash", messages: [{ role: "user", content: "Tag: refund request" }], max_tokens: 5 },
  ],
  store: true,
};

console.log(await submitOnce(job));
console.log(await submitOnce(job)); // second call never reaches the network

Three things make it work. The token is derived from the payload, so a retry of the same logical job produces the same token instead of a fresh UUID. The in_flight marker turns the dangerous case — request sent, response lost — into a loud error a human resolves, rather than a silent duplicate. And a 4xx clears the marker, because a rejected request did no work and is safe to send again.

Retry policy, briefly

Retry on 429 and 5xx. Never retry a 400.

Infrai’s error envelope carries a retryable boolean, so you don’t have to maintain a status-code table by hand — a 429 rate-limit and a network timeout come back retryable: true with backoff guidance attached, while an input error like INVALID_ARGUMENT comes back false and no amount of retrying will change its mind. Exponential backoff with jitter, capped at four attempts, is plenty; the batch route completed a one-row job in about 720 ms in our testing, so long retry ladders just widen the window in which two submits can race.

If you do end up with a duplicate, cancel the loser and move on:

curl -sS -X POST https://api.infrai.cc/v1/ai/batch/cancel/batch_4b0d13535584416d07a4aa49 \
  -H "Authorization: Bearer ${INFRAI_API_KEY}" \
  -H "Content-Type: application/json" \
  -d '{}'

Cancel is free and doesn’t touch your trial credit. Submit is billed per call — the discovery manifest quotes $0.001, verified 26 July 2026, with the per-row model inference billed on top at pass-through. Read GET /v1/discovery for the current figure; these rates trend down rather than up.

Weighing the options

OpenAI’s own Batch API and Amazon Bedrock batch inference both hand you a job id you must persist too, so the ledger isn’t Infrai-specific tax — it’s the shape of every async submit API. If your batches are large and infrequent enough that a human notices a duplicate the same day, honestly, detect-and-cancel is fine and you can skip the ledger. Where Infrai helps is that the reconciliation query lives on the same key: GET /v1/account/usage returns a per-capability breakdown with its own rows for ai.batch.submit and ai.chat, so “did we pay twice last night” is one call rather than a support ticket to two vendors.

References

• Infrai wire conventions and idempotency
• Infrai AI Runtime API reference
• Making retries safe with idempotent APIs — Amazon Builders’ Library