Summarising a folder of documents from Node: batch submit, poll, export

A working Node 22 example for bulk document summarisation on Infrai: the blocking submit call, the states a batch can settle in, and how the JSONL export comes back.

The route you want is POST /v1/ai/batch/submit: one call carries an array of chat requests, each one a document plus a summarise instruction, and Infrai fans them out at pass-through cost. Six documents through it returned six summaries with state: "completed" and a per-row token count. But the word “async” in the query needs a correction before you build anything, because the submit call blocks until the whole batch has finished.

Our six-document run held the HTTP connection for 31.7 seconds. That’s the single most important fact on this page — Infrai’s batch endpoint is a fan-out primitive, not an offline queue, and if you call it from an Express handler your user is watching a spinner the whole time.

Pin a concrete model id in every row

Each element of requests is an ordinary chat request payload: model, messages, max_tokens. The model field here wants a concrete id — the auto / cheapest / smartest routing aliases belong to the /v1/chat/completions surface, so take the id from the catalogue instead:

curl -s "https://api.infrai.cc/v1/ai/models?capability=chat&available=true" \
  -H "Authorization: Bearer $INFRAI_API_KEY" \
| jq '[.data.models[] | {id, owned_by, price_input_per_mtok, price_output_per_mtok}] | .[0:6]'

That’s also where the money lives. Twenty-two chat models were available when we read it on 26 July 2026, and the spread between them is enormous — glm-4-flash from Zhipu listed at $0.00 per million tokens both ways, while the frontier Western ids cost real money per document. Submitting the batch itself is $0.001 per call regardless of how many rows it carries, and new accounts get $2 of free credit. Rates here move down and promotions run, so read that endpoint rather than this paragraph. The durable point is the arbitrage: swapping glm-4-flash for gpt-5-mini is a string change in one field, not a re-integration, because the row payload is the ordinary chat request shape.

The submit call

curl -s -X POST https://api.infrai.cc/v1/ai/batch/submit \
  -H "Authorization: Bearer $INFRAI_API_KEY" \
  -H 'content-type: application/json' \
  -d '{
    "requests": [
      {"model":"glm-4-flash","max_tokens":60,"messages":[
        {"role":"system","content":"Summarise the document in one sentence, then list any dates or amounts."},
        {"role":"user","content":"Master services agreement. Term 24 months, auto-renewing for 12 unless either party gives 60 days notice. Payment net 45."}]},
      {"model":"glm-4-flash","max_tokens":60,"messages":[
        {"role":"system","content":"Summarise the document in one sentence, then list any dates or amounts."},
        {"role":"user","content":"Statement of work for a data migration. Fixed fee of 48,000 USD, invoiced in three milestones."}]}
    ],
    "metadata": {"job": "doc-summaries"},
    "store": true
  }'

{
  "ok": true,
  "data": {
    "batch_id": "batch_04d6acc3176fcc7af91cedb6",
    "state": "completed",
    "total_count": 6
  }
}

Three fields: batch_id, state, total_count. Write your client against those.

Save that batch_id before you do anything else — it’s the handle for status, results and export, and none of them are reachable without it. Persist it in the same transaction that marks the documents as queued, so a crash between the submit and the write doesn’t strand work you’ve already paid for.

Terminal isn’t only “completed”

A batch where some rows succeed and some fail settles as partial, and partial is final — nothing further will happen to it. So don’t allow-list the endings you expect; treat anything outside the running states as done and inspect the counts. Submit a mixed batch — one row naming a model the catalogue doesn’t carry, two good — and you get this:

curl -s https://api.infrai.cc/v1/ai/batch/status/batch_04d6acc3176fcc7af91cedb6 \
  -H "Authorization: Bearer $INFRAI_API_KEY" | jq '.data'

{
  "batch_id": "batch_cd7f73eb601ac357072d31ef",
  "state": "partial",
  "progress": 0.6667,
  "total_count": 3,
  "completed_count": 2,
  "failed_count": 1
}

Two of three rows produced a summary; the third failed on its own and took nothing else with it. That per-row isolation is the reason to prefer one batch over three loose requests when a document set is heterogeneous — a malformed row is a line in the results, not a dead job. POST /v1/ai/batch/cancel/{id} is safe to call against a batch that has already settled; it hands back the current state.

The Node worker

This runs outside your request cycle — a queue consumer, a cron target, an npm script. It chunks the folder, keeps the connection budget sane, and records every id.

import { readFile, readdir, writeFile } from "node:fs/promises";
import path from "node:path";
import process from "node:process";

const BASE = "https://api.infrai.cc";
const KEY = process.env.INFRAI_API_KEY;
if (!KEY) throw new Error("INFRAI_API_KEY is not set");

const MODEL = process.env.SUMMARY_MODEL ?? "glm-4-flash";
const CHUNK = 20;              // rows per submit; each row costs seconds of wall clock
const RUNNING = new Set(["queued", "running", "connecting"]);

async function api(method, route, body) {
  const res = await fetch(`${BASE}${route}`, {
    method,
    headers: { authorization: `Bearer ${KEY}`, "content-type": "application/json" },
    body: body === undefined ? undefined : JSON.stringify(body),
  });
  const json = await res.json();
  if (!res.ok) throw new Error(`${route} -> ${res.status} ${json?.error?.code ?? ""} ${json?.error?.message ?? ""}`);
  return json.data;
}

const row = (text) => ({
  model: MODEL,
  max_tokens: 160,
  messages: [
    { role: "system", content: "Summarise the document in one sentence, then list any dates or amounts." },
    { role: "user", content: text },
  ],
});

export async function summariseFolder(dir) {
  const names = (await readdir(dir)).filter((n) => n.endsWith(".txt")).sort();
  const summaries = [];

  for (let i = 0; i < names.length; i += CHUNK) {
    const slice = names.slice(i, i + CHUNK);
    const texts = await Promise.all(slice.map((n) => readFile(path.join(dir, n), "utf8")));

    const submitted = await api("POST", "/v1/ai/batch/submit", {
      requests: texts.map(row),
      metadata: { job: "doc-summaries", offset: String(i) },
      store: true,
    });
    await writeFile(`batch-${submitted.batch_id}.json`, JSON.stringify({ batch_id: submitted.batch_id, files: slice }, null, 2));
    // The id is the only handle on this work — write it down before polling.

    let state = submitted.state;
    while (RUNNING.has(state)) {
      await new Promise((r) => setTimeout(r, 2000));
      state = (await api("GET", `/v1/ai/batch/status/${submitted.batch_id}`)).state;
    }

    const results = await api("GET", `/v1/ai/batch/results/${submitted.batch_id}`);
    for (const item of results.items) {
      const file = slice[item.request_index];
      if (!item.ok) {
        console.error(`${file}: ${item.error?.code ?? "unknown"} — ${item.error?.message ?? ""}`);
        continue;
      }
      summaries.push({ file, summary: item.result.content, tokens: item.result.usage.total_tokens });
    }
    console.log(`chunk ${i / CHUNK + 1}: state=${state}, kept ${summaries.length} summaries`);
  }

  return summaries;
}

const out = await summariseFolder(process.argv[2] ?? "./documents");
await writeFile("summaries.json", JSON.stringify(out, null, 2));

The while loop is deliberately harmless: because submit already blocked until the work finished, state is usually terminal on the first read and the loop never runs. Leave it in anyway — the shape is right if the endpoint ever becomes genuinely asynchronous, and it costs one comparison.

Getting the results out

Two routes, and they answer different questions. GET /v1/ai/batch/results/{id} gives you structured items with ok, result.content and result.usage per row, paged by next_cursor. The export gives you the same thing as JSONL for a data pipeline:

curl -s -X POST https://api.infrai.cc/v1/ai/batch/export/batch_04d6acc3176fcc7af91cedb6 \
  -H "Authorization: Bearer $INFRAI_API_KEY" \
  -H 'content-type: application/json' \
  -d '{"format":"jsonl"}' | jq -r '.data.content' > summaries.jsonl

The JSONL arrives inline, in data.content, as one string:

{
  "ok": true,
  "data": {
    "batch_id": "batch_04d6acc3176fcc7af91cedb6",
    "format": "jsonl",
    "content": "{\"request_index\": 0, \"ok\": true, \"result\": {\"content\": \"The document outlines a 24-month master services agreement...\", \"finish_reason\": \"stop\", \"model\": \"glm-4-flash\", \"usage\": {\"prompt_tokens\": 83, \"completion_tokens\": 84, \"total_tokens\": 167}}, \"error\": null}\n"
  }
}

The whole export arrives inline in data.content, so you redirect it to a file yourself rather than following a link. For six short summaries that’s convenient. For forty thousand it isn’t, and that’s the honest boundary of this endpoint.

If you need 40,000 documents summarised on a nightly schedule and a day of latency is fine, you’d be better off with OpenAI’s Batch API — the 50% discount is hard to argue with, and true fire-and-forget submission is something this endpoint doesn’t support. If you’re summarising the twelve files a user just uploaded, a plain loop is simpler than either, and OpenRouter will give you a wider model list to loop over. This endpoint’s niche is the middle: bounded batches where you want one call, one bill and per-row error reporting.

What it buys you beyond that is the second question being free. The same key that ran the summaries stores the source documents, schedules the nightly run, queues the retries and captures the failures — one credential and one usage view instead of four vendors to reconcile per tenant.

Verify a finished batch before you trust your own plumbing:

curl -s https://api.infrai.cc/v1/ai/batch/results/batch_04d6acc3176fcc7af91cedb6 \
  -H "Authorization: Bearer $INFRAI_API_KEY" \
| jq '{rows: (.data.items | length), failures: [.data.items[] | select(.ok == false)] | length}'

If failures is anything but zero, read error.message on those rows before you blame the batch: in our testing the cause was almost always the row’s own model id or message content, and the surrounding rows had completed fine.

References

• Infrai AI runtime API reference
• Infrai documentation home
• OpenAI Batch API guide