Cheap text summarization: per-1K-token rates versus cost per document

Rate cards quote tokens; your budget is in documents. How to convert one into the other with a measurement, and when batching summaries actually pays off.

Per-1K-token pricing is a shopping unit. What a startup actually needs is cost per summarised document, and the conversion needs exactly one measurement: how many tokens your typical document really is. Measure that once, multiply by the input rate, add a small output term, and you have a number you can put in a spreadsheet. Infrai gives you the token count and the current per-model rates from two free calls, so the whole exercise takes about five minutes.

Summarisation is also the friendliest workload for cheap models, because the asymmetry works in your favour: thousands of input tokens, a couple of hundred output tokens. Models priced low on input and high on output — which is most of the cheap tier — are at their best here.

Measure one real document, not a guess

A support thread that reads as “about 600 words” came back at 988 prompt tokens when we counted it. That’s a factor of roughly 1.5 tokens per word, and it’s the kind of thing you want measured rather than assumed, because a 30% error in this number is a 30% error in every projection built on it.

import { readFile } from "node:fs/promises";

const KEY = process.env.INFRAI_API_KEY;
if (!KEY) throw new Error("INFRAI_API_KEY is not set");

const document = await readFile(process.argv[2] ?? "./sample-thread.txt", "utf8");

const res = await fetch("https://api.infrai.cc/v1/ai/tokens/count", {
  method: "POST",
  headers: { Authorization: `Bearer ${KEY}`, "Content-Type": "application/json" },
  body: JSON.stringify({
    model: "gpt-5-mini",
    messages: [
      { role: "system", content: "Summarise in three bullets and end with the next action." },
      { role: "user", content: document },
    ],
  }),
});
if (!res.ok) throw new Error(`token count failed: ${res.status} ${await res.text()}`);

const { data } = await res.json();
const words = document.trim().split(/\s+/).length;
console.log(`${words} words -> ${data.prompt_tokens} tokens (${(data.prompt_tokens / words).toFixed(2)} per word)`);

Local tokenizers get you the same figure without the round trip; tiktoken is the obvious one. The server-side count matters when you’re comparing models, because it uses the tokenizer of whichever vendor the model string routes to.

Turn the rate card into cost per thousand documents

export INFRAI_API_KEY="your_infrai_api_key"

curl -sS "https://api.infrai.cc/v1/ai/models?capability=chat&available=true" \
  -H "Authorization: Bearer ${INFRAI_API_KEY}"

Take a 1,000-token document and a 150-token summary, apply the input and output rates from that response, and the picture stops being abstract. Read 2026-07-26:

Thirty-four times between the top and bottom paid rows for the same summary. That spread, not any individual figure, is the reason to keep model in configuration rather than in code — rates move, and in this market they move down, so rerun the call before you commit.

The honest caveat: the free row is free because it’s a small model. It writes perfectly decent bullet points for a support thread and falls short on a dense legal document where the summary has to preserve conditions. Sample fifty of your own documents at two tiers before choosing; that experiment costs pennies and settles arguments no benchmark can.

One pass or map-reduce?

Anything that fits the context window should go in one call. Splitting a document into pieces and summarising the summaries costs more tokens, not fewer, because every intermediate summary is written and then read again.

curl -sS -X POST "https://api.infrai.cc/v1/chat/completions" \
  -H "Authorization: Bearer ${INFRAI_API_KEY}" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "glm-4-flash",
    "max_tokens": 120,
    "messages": [
      {"role": "system", "content": "Summarise the support thread in three bullet points and end with the next action."},
      {"role": "user", "content": "Customer reports the CSV export button spins forever since Tuesday. Agent asked for a HAR file. The request 504s after 30 seconds. Engineering suspects the report query is unindexed."}
    ]
  }'

{
  "choices": [{ "message": { "content": "- CSV export hangs since Tuesday\n- HAR shows a 504 after 30s\n- Unindexed report query suspected\nNext: add the index and retest" }, "finish_reason": "stop" }],
  "usage": { "prompt_tokens": 66, "completion_tokens": 51, "total_tokens": 117 },
  "infrai": { "cost_usd": 0.0, "vendor": "zhipu", "model": "glm-4-flash", "cache": false }
}

Map-reduce earns its keep above the context limit — a 200-page report, a month of chat logs. Cap the fan-out and hold the reduce step to a stronger model, since that’s the call that decides what survives:

import os
import requests

KEY = os.environ["INFRAI_API_KEY"]
BASE = "https://api.infrai.cc"
HEADERS = {"Authorization": f"Bearer {KEY}", "Content-Type": "application/json"}


def summarise(text, model, instruction, max_tokens=200):
    body = {
        "model": model,
        "max_tokens": max_tokens,
        "messages": [
            {"role": "system", "content": instruction},
            {"role": "user", "content": text},
        ],
    }
    r = requests.post(f"{BASE}/v1/chat/completions", headers=HEADERS, json=body, timeout=60)
    r.raise_for_status()
    payload = r.json()
    return payload["choices"][0]["message"]["content"], payload.get("infrai", {}).get("cost_usd", 0.0)


def map_reduce(sections):
    spend = 0.0
    partials = []
    for section in sections:
        text, cost = summarise(section, "glm-4-flash", "Summarise this section in two sentences.")
        partials.append(text)
        spend += cost
    final, cost = summarise(
        "\n\n".join(partials), "gpt-5-mini", "Merge these section summaries into one brief.", 400
    )
    return final, spend + cost


if __name__ == "__main__":
    brief, total = map_reduce(["First section text.", "Second section text."])
    print(brief)
    print(f"spent {total:.6f} USD")

Nightly digests belong in a batch

If nobody is waiting for the summary, don’t hold a connection open for it. One submit carries the whole night’s work, and the job endpoints around it are free.

curl -sS -X POST "https://api.infrai.cc/v1/ai/batch/submit" \
  -H "Authorization: Bearer ${INFRAI_API_KEY}" \
  -H "Content-Type: application/json" \
  -d '{
    "requests": [
      {"model":"glm-4-flash","max_tokens":150,"messages":[{"role":"system","content":"Summarise this thread for a daily digest."},{"role":"user","content":"Thread A: billing question, resolved by refund."}]},
      {"model":"glm-4-flash","max_tokens":150,"messages":[{"role":"system","content":"Summarise this thread for a daily digest."},{"role":"user","content":"Thread B: export timeout, escalated to engineering."}]}
    ],
    "batch_timeout": 3600,
    "metadata": {"job": "nightly-digest", "tenant": "tenant_42"},
    "store": true
  }'

The response gives you batch_id, state and total_count. Poll the status route until the state is completed, page the results with next_cursor, and pull the whole set as JSONL when you want to hand it to something else:

curl -sS -X POST "https://api.infrai.cc/v1/ai/batch/export/{id}" \
  -H "Authorization: Bearer ${INFRAI_API_KEY}" \
  -H "Content-Type: application/json" \
  -d '{"format": "jsonl"}'

Each result row carries its own ok, cost_usd and vendor, so a failed item is one index to retry rather than a whole job to rerun.

Then check what it really cost

Estimates are estimates. This is the ledger:

curl -sS "https://api.infrai.cc/v1/account/usage/timeseries" \
  -H "Authorization: Bearer ${INFRAI_API_KEY}"

Daily buckets with cost and call counts, so a jump in summarisation spend shows up as a shape rather than as a surprise at the end of the month.

When something else is the better buy

If you need one model and nothing else, buying from the vendor directly removes a hop — OpenAI’s own batch API is well documented and its 24-hour turnaround discount is real. If your documents can’t leave the building, a local model under Ollama costs nothing per token and summarisation is forgiving enough that small models do fine. If you want the widest catalogue with per-request routing rules, OpenRouter goes deeper on that than we do.

Infrai’s trade-off is the aggregator’s: new frontier models land here later than at the vendor, and there’s no live quota reading, so you bound spend with a budget cap rather than preflighting capacity. What you get back is that switching model is a string, and the queue that schedules the digest, the storage that holds it and the email that sends it are already on the same key and the same invoice — which is usually where the actual engineering time goes.

References

• Infrai AI runtime API reference
• Infrai documentation
• OpenAI Batch API guide
• tiktoken tokenizer