Receipts and invoices through a vision LLM: dependable enough, and which model

Vision models read photographed receipts well enough to assist an approver, not to post to the ledger unattended. How to choose one and gate the output.

Dependable enough to pre-fill an expense claim and put a human on the exceptions — not dependable enough to post straight to the ledger. That’s the honest answer, and the deciding variable is photo quality, not model brand. On Infrai you pick a vision model from the same catalogue as everything else, send the image inline, and gate the result on your own validation rather than the model’s confidence.

The selection process has a step most guides skip: check that the model you picked can actually serve vision on your account today. Several of the catalogue’s vision-flagged models weren’t servable for image input when we tested, which is a much bigger accuracy problem than any benchmark delta.

Pick from what’s actually servable

export INFRAI_API_KEY="your_infrai_api_key"

curl -sS "https://api.infrai.cc/v1/ai/models?capability=chat&available=true" \
  -H "Authorization: Bearer ${INFRAI_API_KEY}" \
  | tr ',' '\n' | grep -A1 '"vision"'

{
  "data": [
    { "id": "qwen3-vl-plus", "owned_by": "alibaba_intl", "price_input_per_mtok": 0.2, "price_output_per_mtok": 1.6, "vision": true, "vision_status": "verified" },
    { "id": "qwen-vl-plus", "owned_by": "alibaba_intl", "price_input_per_mtok": 0.21, "price_output_per_mtok": 0.63, "vision": true, "vision_status": "verified" },
    { "id": "qwen-vl-max", "owned_by": "alibaba_intl", "price_input_per_mtok": 0.8, "price_output_per_mtok": 3.2, "vision": true, "vision_status": "verified" },
    { "id": "glm-4v-plus", "owned_by": "zhipu", "price_input_per_mtok": 0.56, "price_output_per_mtok": 0.56, "vision": true, "vision_status": "verified" },
    { "id": "gpt-5-mini", "owned_by": "openai", "price_input_per_mtok": 0.25, "price_output_per_mtok": 2.0, "vision": true, "vision_status": "verified" }
  ]
}

Two fields do the work. vision: true says the model accepts images at all — send an image to one without it and you get MODEL_NO_IMAGE_INPUT rather than a hallucinated answer. vision_status separates verified (we’ve run a real image through it) from declared (the vendor says so).

Here’s the caveat that costs an afternoon: verified in the catalogue still doesn’t mean routable right now. On 2026-07-26 both gpt-5-mini and glm-4v-plus answered image requests with a 503 and VENDOR_NOT_CONFIGURED for the vision capability, while the qwen*-vl-* family served every request. Probe your shortlist with one real receipt before you write the model id into config.

Probe first, trust the flag second.

Send the image inline

A remote URL is the obvious approach and it’s the one that failed for us — the vendor couldn’t fetch a public image host and returned a download error. Base64 in the request body is slower to build and far more reliable.

IMG_B64=$(base64 -i receipt.jpg | tr -d '\n')

curl -sS -X POST "https://api.infrai.cc/v1/chat/completions" \
  -H "Authorization: Bearer ${INFRAI_API_KEY}" \
  -H "Content-Type: application/json" \
  -d "{
    \"model\": \"qwen3-vl-plus\",
    \"max_tokens\": 300,
    \"response_format\": {\"type\": \"json_object\"},
    \"messages\": [{\"role\": \"user\", \"content\": [
      {\"type\": \"text\", \"text\": \"Extract merchant, date, total_amount, currency, tax_amount. Use null for anything you cannot read.\"},
      {\"type\": \"image_url\", \"image_url\": {\"url\": \"data:image/jpeg;base64,${IMG_B64}\"}}
    ]}]
  }"

Downscale first. A 900-pixel-wide JPEG of a till receipt cost 479 prompt tokens in our testing; the 3 MB original a phone camera produces costs far more and reads no better.

What “dependable” actually looked like

{
  "model": "qwen3-vl-plus",
  "choices": [{ "message": { "role": "assistant", "content": "```json\n{\n  \"merchant\": \"ijssel land ziekenhuis\",\n  \"date\": null,\n  \"total_amount\": null,\n  \"currency\": null\n}\n```" }, "finish_reason": "stop" }],
  "usage": { "prompt_tokens": 479, "completion_tokens": 39, "total_tokens": 518 },
  "infrai": { "cost_usd": 9.525e-05, "vendor": "alibaba_intl", "region": "china", "model": "qwen3-vl-plus" }
}

Three things to take from that. It read the merchant off a creased, low-contrast photo. It returned null for the fields it couldn’t read instead of inventing them, because the prompt told it to — that instruction is the single highest-value line in the whole pipeline. And the JSON came wrapped in a Markdown fence despite response_format: {"type":"json_object"}, so a naive JSON.parse throws.

json_schema is weaker still. We sent a schema requiring merchant and total_amount and got back an object with institution, document_type and patient_number — keys the schema never mentioned. Treat response format as a strong hint on these models, never as a contract, and validate every field yourself.

The extraction function you’d ship

import { readFile } from "node:fs/promises";

const KEY = process.env.INFRAI_API_KEY;
if (!KEY) throw new Error("INFRAI_API_KEY is not set");

const FIELDS = ["merchant", "date", "total_amount", "currency", "tax_amount"];

function parseLoosely(text) {
  const cleaned = text.trim().replace(/^```(?:json)?/i, "").replace(/```$/, "").trim();
  try { return JSON.parse(cleaned); } catch { return null; }
}

export async function extractReceipt(path, model = "qwen3-vl-plus") {
  const b64 = (await readFile(path)).toString("base64");
  const res = await fetch("https://api.infrai.cc/v1/chat/completions", {
    method: "POST",
    headers: { Authorization: `Bearer ${KEY}`, "Content-Type": "application/json" },
    body: JSON.stringify({
      model,
      max_tokens: 300,
      response_format: { type: "json_object" },
      messages: [{
        role: "user",
        content: [
          { type: "text", text: `Extract ${FIELDS.join(", ")} from this receipt as JSON. Use null for anything unreadable. No commentary.` },
          { type: "image_url", image_url: { url: `data:image/jpeg;base64,${b64}` } },
        ],
      }],
    }),
  });
  if (!res.ok) throw new Error(`vision call failed: ${res.status} ${await res.text()}`);

  const body = await res.json();
  const parsed = parseLoosely(body.choices[0].message.content);
  if (!parsed) return { status: "review", reason: "unparseable model output", cost_usd: body.infrai?.cost_usd };

  const missing = FIELDS.filter((f) => parsed[f] === null || parsed[f] === undefined);
  const amount = Number(parsed.total_amount);
  const sane = Number.isFinite(amount) && amount > 0 && amount < 100000;
  return {
    status: missing.length === 0 && sane ? "auto" : "review",
    missing,
    data: parsed,
    cost_usd: body.infrai?.cost_usd,
    model: body.model,
  };
}

const result = await extractReceipt("receipt.jpg");
console.log(JSON.stringify(result, null, 2));

Everything that isn’t a complete, arithmetically plausible record goes to review. In an expense workflow that’s not a compromise — an approver was always going to look at the claim, and pre-filled fields with three of five confirmed is still most of the typing saved. Notice what the gate is made of: presence of every required field, a total that parses as a number, and a range check that rejects both zero and an implausible six-figure lunch. None of it asks the model how confident it is, because a vision model’s stated confidence tends to be a fluent sentence rather than a calibrated probability, and treating it as one is how teams end up posting a mis-read decimal into the general ledger. Add your own rules as you learn the failure shapes — currency mismatched against the claimant’s country, a date in the future, a total that doesn’t equal the line items you also asked for.

Cheap to run, cheap to be wrong about.

Cost, and doing a month at once

curl -sS "https://api.infrai.cc/v1/ai/models?capability=chat&available=true" \
  -H "Authorization: Bearer ${INFRAI_API_KEY}" | head -c 400

Verified 2026-07-26, qwen3-vl-plus was $0.20 per Mtok in and $1.60 out, which put a single downscaled receipt at roughly $0.0001 — about a hundred receipts per cent. gpt-5-mini sat at $0.25 in, and the Western vision models generally run several times the China-origin ones for this kind of work. Vision pricing has fallen steadily, so the live catalogue will probably beat these figures.

Month-end is a batch, and vision rows batch fine — we ran one and the item came back with its own cost_usd:

curl -sS "https://api.infrai.cc/v1/ai/batch/results/batch_6cbddd7f878a09ca04d318e1" \
  -H "Authorization: Bearer ${INFRAI_API_KEY}"

{
  "ok": true,
  "data": {
    "items": [{ "ok": true, "request_index": 0, "cost_usd": 0.0001364, "vendor": "alibaba_intl", "result": { "model": "qwen3-vl-plus", "usage": { "prompt_tokens": 466, "total_tokens": 493 } } }],
    "total_count": 1,
    "next_cursor": null
  }
}

Submit with POST /v1/ai/batch/submit, poll GET /v1/ai/batch/status/{id}, page the results, and route anything with ok: false or a missing field into the same review queue.

Where this falls short

Totals are the risky field — a smudged decimal is worth real money, and the benchmark literature on invoice extraction finds exactly this pattern, strong on merchant and date, weaker on amounts and line items. If finance needs guaranteed capture with an audit trail and a support contract, a specialist invoice vendor is the right purchase and we’d say so. If you’re already all-in on one hyperscaler, Gemini and OpenAI both read documents well through their own SDKs.

The case for doing it here is the rest of the workflow: the receipt image lands in storage, the extraction runs on the same key, the batch runs on the same key, the approval email goes out on the same key, and every one of those lines shows up in one usage view attributed to one tenant. The model is a string in a config file, so when a cheaper vision model appears you change the string.

References

• Infrai AI runtime API reference
• MODEL_NO_IMAGE_INPUT error reference
• Multi-modal vision vs text-based parsing for invoice processing
• Invoice OCR benchmark: LLMs vs OCR engines