Cheap, reliable LLM JSON extraction: count tokens before you pick a model

Token counting, a free cross-model cost compare call, and the batch-or-realtime decision for structured extraction in Node — including where JSON mode still lets you down.

The cheapest structured extraction that still works is a small model plus a validator, not a large model plus optimism. Measure the prompt, price that exact prompt across candidate models, send anything you can wait for through a batch job, and parse-and-check every response before it reaches your database. Infrai exposes the first two of those as free endpoints, so the measuring part costs nothing.

Both POST /v1/ai/tokens/count and POST /v1/ai/cost/compare are free and rate-limited, and neither consumes the new-account trial — you can profile a whole extraction schema before you spend a cent.

Measure the prompt, not a guess about the prompt

Extraction prompts are front-loaded: a long system message describing your schema, plus a document. That system message is paid for on every single call, which is why trimming it is usually worth more than switching models.

export INFRAI_API_KEY="your_infrai_api_key"

curl -sS -X POST "https://api.infrai.cc/v1/ai/tokens/count" \
  -H "Authorization: Bearer ${INFRAI_API_KEY}" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gpt-5-mini",
    "messages": [
      {"role": "system", "content": "Return only JSON: {\"vendor\":string,\"total\":number,\"due_date\":string}"},
      {"role": "user", "content": "Invoice from Acme Ltd for 249.90 EUR due 2026-08-01"}
    ]
  }'

The reply is small on purpose:

{
  "ok": true,
  "data": { "prompt_tokens": 43, "model": "gpt-5-mini" }
}

Local tokenizers do the same job — tiktoken and the tokencost package are both fine, and they’re faster because there’s no network hop. The difference is that a server-side count uses the tokenizer the routed vendor actually applies, so it doesn’t drift when you change the model string. Use whichever you’ll keep in sync.

Price the same prompt across models in one call

This is the part the calculator sites can’t do, because they don’t have your prompt. POST /v1/ai/cost/compare takes a list of models and the real messages and returns one costed row per model, cheapest first.

curl -sS -X POST "https://api.infrai.cc/v1/ai/cost/compare" \
  -H "Authorization: Bearer ${INFRAI_API_KEY}" \
  -H "Content-Type: application/json" \
  -d '{
    "models": ["openai/gpt-4o-mini", "openai/gpt-4o", "deepseek/deepseek-chat"],
    "messages": [
      {"role": "system", "content": "Extract vendor, total and due_date as JSON."},
      {"role": "user", "content": "Invoice from Acme Ltd for 249.90 EUR due 2026-08-01"}
    ],
    "expected_output_tokens": 120
  }'

{
  "ok": true,
  "data": [
    { "model": "deepseek/deepseek-chat", "vendor_region": "china", "prompt_tokens": 20,
      "breakdown": { "input_cost": 0.0000028, "output_cost": 0.0000336, "markup": 0.0, "final": 0.0000364 } },
    { "model": "openai/gpt-4o-mini", "vendor_region": "western", "prompt_tokens": 20,
      "breakdown": { "input_cost": 0.000003, "output_cost": 0.000072, "markup": 0.00000375, "final": 0.00007875 } },
    { "model": "openai/gpt-4o", "vendor_region": "western", "prompt_tokens": 20,
      "breakdown": { "input_cost": 0.0001, "output_cost": 0.0018, "markup": 0.000095, "final": 0.001995 } }
  ]
}

Read the ratio, not the digits. The flagship row is roughly 25 times the mini row and about 55 times the China-origin row for identical work — and that ordering has held while individual rates moved. A caveat worth knowing before you wire this into a dashboard: the compare endpoint’s model table is narrower than the served catalogue, so a model id that routes fine through chat can still come back MODEL_NOT_FOUND here. Check what the catalogue serves separately.

What the catalogue costs today

curl -sS "https://api.infrai.cc/v1/ai/models?capability=chat&available=true" \
  -H "Authorization: Bearer ${INFRAI_API_KEY}"

Verified 2026-07-26, per million tokens, input / output:

Those are readings from the endpoint above, not a rate card — rerun the call on the day you decide. Rates in this market move down, and discount campaigns run on the China-origin tier especially, so what you find is more likely to be lower than higher.

The extraction call, with the validator attached

JSON mode gets you syntactically valid JSON. It does not get you your JSON. Here’s the shape that survives production: ask a cheap model, validate against the schema you actually need, and escalate only the failures.

const BASE = "https://api.infrai.cc";
const KEY = process.env.INFRAI_API_KEY;
if (!KEY) throw new Error("INFRAI_API_KEY is not set");

const SYSTEM = 'Return only JSON: {"vendor":string,"total":number,"due_date":string}';

function valid(row) {
  return row && typeof row.vendor === "string" && typeof row.total === "number"
    && /^\d{4}-\d{2}-\d{2}$/.test(String(row.due_date));
}

async function extractOnce(text, model) {
  const payload = {
    model,
    response_format: { type: "json_object" },
    max_tokens: 200,
    messages: [{ role: "system", content: SYSTEM }, { role: "user", content: text }],
  };
  const res = await fetch(`${BASE}/v1/chat/completions`, {
    method: "POST",
    headers: { Authorization: `Bearer ${KEY}`, "Content-Type": "application/json" },
    body: JSON.stringify(payload),
  });
  if (!res.ok) throw new Error(`chat failed: ${res.status} ${await res.text()}`);
  const json = await res.json();
  try {
    return { row: JSON.parse(json.choices[0].message.content), spend: json.infrai?.cost_usd ?? 0 };
  } catch {
    return { row: null, spend: json.infrai?.cost_usd ?? 0 };
  }
}

export async function extract(text) {
  const cheap = await extractOnce(text, "glm-4-flash");
  if (valid(cheap.row)) return cheap.row;
  const strong = await extractOnce(text, "gpt-5-mini");
  if (valid(strong.row)) return strong.row;
  throw new Error("extraction failed schema validation twice");
}

Every chat response carries an infrai object with cost_usd, vendor, region and model, so the escalation rate and its price are observable per call rather than at month end.

One finding from our own testing worth flagging, because it cuts against what the model cards imply: response_format of type json_schema is not enforced uniformly across the catalogue. On glm-4-flash a strict schema request came back as formatted prose, while plain json_object on the same model returned clean, parseable JSON. Don’t assume the schema is a contract — the validator is the contract, and the schema is a hint that helps.

Batch or realtime is a latency decision

Price barely moves between the two. What moves is how long the caller waits and how many connections you hold open.

Submitting is one call carrying many requests:

curl -sS -X POST "https://api.infrai.cc/v1/ai/batch/submit" \
  -H "Authorization: Bearer ${INFRAI_API_KEY}" \
  -H "Content-Type: application/json" \
  -d '{
    "requests": [
      {"model":"glm-4-flash","response_format":{"type":"json_object"},"messages":[{"role":"system","content":"Return only JSON: {\"vendor\":string,\"total\":number}"},{"role":"user","content":"Invoice from Acme Ltd for 249.90 EUR"}]},
      {"model":"glm-4-flash","response_format":{"type":"json_object"},"messages":[{"role":"system","content":"Return only JSON: {\"vendor\":string,\"total\":number}"},{"role":"user","content":"Receipt: Globex, 12.40 USD"}]}
    ],
    "metadata": {"job": "invoice-backfill", "tenant": "tenant_42"},
    "store": true
  }'

You get back batch_id, state and total_count. Poll GET /v1/ai/batch/status/{id} until the state is completed, then page GET /v1/ai/batch/results/{id} with next_cursor. Each row carries ok, result, cost_usd, vendor and request_index, so a partial failure is a list of indices to retry rather than a lost job. Status, results, list, export and cancel are all free; only the submit is billed, at a per-call rate you can read from GET /v1/discovery.

Then check what it actually cost, which is the only number that isn’t an estimate:

curl -sS "https://api.infrai.cc/v1/account/usage" \
  -H "Authorization: Bearer ${INFRAI_API_KEY}"

The response groups spend by capability — ai.chat and the rest on one line each — which is where the per-tenant question gets answered if you put a tenant id in metadata.

Where something else is the better buy

If your extraction targets a fixed schema and strict enforcement is non-negotiable, OpenAI’s structured outputs on their own API give you a harder guarantee than any aggregator can pass through, and that’s worth the integration. If the documents are sensitive enough that they shouldn’t leave your network at all, Ollama with a small local model removes the question entirely — slower, and you own the GPU, but the compliance conversation ends. And if you need Claude specifically for long, messy documents, check the catalogue first: Anthropic models aren’t in the served list right now, so you’d be better off going direct for that one workload.

The trade-off with routing through Infrai is the usual aggregator one. New frontier models land here after they land at the vendor, and the cheap tier is cheap because the models are small — forgiving on invoices, less forgiving on nested contracts. Benchmark against fifty of your own documents before you commit; that costs a few cents and settles the argument.

What you get in exchange is that the next decision is a string change, and the queue, storage and error tracking around the extractor are already on the same key and the same bill.

References

• Infrai AI runtime API reference
• MODEL_NOT_FOUND error reference
• OpenAI structured outputs guide
• tokencost: token price estimates for 400+ LLMs