Cheap semantic search over your docs: embeddings and rerank, priced properly

Indexing is a one-off in cents; queries are the recurring bill. Measured per-million-token and per-request rates, the catalogue gotcha, and where the region caveat bites.

An “ask your docs” search has two bills with completely different shapes, and conflating them is why people over-shop for embedding models. Indexing is a one-off charge you pay when the corpus changes. Querying is forever. On Infrai both sit behind the same key — POST /v1/embeddings for vectors, POST /v1/ai/rerank for ordering — and once you separate the two, the cheap answer turns out to be a modest embedding model with a reranker in front of it, not a premium embedding model on its own.

That’s the recommendation up front. Below is the arithmetic that produced it, measured against a live account on 2026-07-26, plus one catalogue quirk that will make you think embeddings aren’t available when they are.

The catalogue quirk, first

If you check what’s servable before you build — and you should — this is what you’ll see:

export INFRAI_API_KEY="your_infrai_api_key"

curl -sS "https://api.infrai.cc/v1/ai/models?capability=embed&available=true" \
  -H "Authorization: Bearer ${INFRAI_API_KEY}"

{ "object": "list", "capability": "embed", "available_only": true, "count": 0, "data": [] }

Zero models. The obvious conclusion is wrong: the embedding route works fine, it just isn’t tracked in the model catalogue, which covers chat, image and speech models. Pin an OpenAI model id and you’ll get a model_not_found from upstream, which is where most people give up.

Ask for automatic routing instead and you get vectors.

curl -sS -X POST "https://api.infrai.cc/v1/embeddings" \
  -H "Authorization: Bearer ${INFRAI_API_KEY}" \
  -H "Content-Type: application/json" \
  -d '{"model":"auto","input":["how do I rotate an api key","the office is closed on holidays"]}'

{
  "object": "list",
  "model": "text-embedding-v4",
  "usage": { "prompt_tokens": 15, "total_tokens": 15 },
  "infrai": {
    "cost_usd": 0.00000105,
    "vendor": "alibaba_intl",
    "region": "china",
    "model": "text-embedding-v4",
    "markup_pct": 0,
    "cache": false
  }
}

Two vectors of 1,024 dimensions each, from text-embedding-v4. Omitting model entirely does the same thing. Keep "auto" in your code anyway — it documents the intent, and it keeps working when the routing target changes underneath you.

Work out your own rate rather than trusting a table

Every response carries usage.total_tokens and infrai.cost_usd, so you can divide one by the other and stop guessing.

const KEY = process.env.INFRAI_API_KEY;
if (!KEY) throw new Error("INFRAI_API_KEY is not set");

const sample = Array.from({ length: 5 }, () => "word ".repeat(400));

const res = await fetch("https://api.infrai.cc/v1/embeddings", {
  method: "POST",
  headers: { Authorization: `Bearer ${KEY}`, "Content-Type": "application/json" },
  body: JSON.stringify({ model: "auto", input: sample }),
});
if (!res.ok) throw new Error(`embed failed: ${res.status} ${await res.text()}`);

const json = await res.json();
const tokens = json.usage.total_tokens;
const perMtok = (json.infrai.cost_usd / tokens) * 1e6;

console.log(`${json.data.length} vectors, ${json.data[0].embedding.length} dims, ${tokens} tokens`);
console.log(`$${perMtok.toFixed(4)} per million tokens on ${json.infrai.model} (${json.infrai.vendor})`);

Verified 2026-07-26 that returned $0.07 per million tokens — 2,010 tokens billed at $0.0001407, with no per-request floor distorting small calls. Embedding rates have fallen steadily for two years and will probably be lower by the time you run it, which is the point of computing rather than quoting.

Be straight about where that sits in the market. OpenAI’s small embedding model is cheaper per token than this one, and Voyage price aggressively for retrieval-specific models; both publish live pricing pages, linked at the bottom. If you’re embedding hundreds of millions of tokens a month and nothing else, go and buy from them directly — that’s a real cost difference and it would be dishonest to bury it.

Here’s why it usually doesn’t decide anything.

The one-off is trivial. The queries are the bill.

Take a documentation corpus of 10,000 pages, a dozen 300-token chunks each. That’s 36 million tokens to index once.

Halving your embedding rate saves about $1.30 a month on that shape. Getting the ranking right saves you the support tickets. That’s the whole argument for spending attention on rerank instead of on embedding-model shopping.

Rerank is priced per request, not per candidate

This is the number that surprised us most, so we measured it twice.

curl -sS -X POST "https://api.infrai.cc/v1/ai/rerank" \
  -H "Authorization: Bearer ${INFRAI_API_KEY}" \
  -H "Content-Type: application/json" \
  -d '{
    "query": "how do I rotate an API key without downtime",
    "candidates": [
      "Key rotation: create a new key, deploy it, then revoke the old one.",
      "Our office is closed on public holidays.",
      "Invoices are issued monthly on the first.",
      "To revoke a credential, call the keys endpoint with the key id."
    ],
    "top_k": 2
  }'

{
  "ok": true,
  "data": {
    "ranked": [
      { "index": 0, "score": 0.719662 },
      { "index": 3, "score": 0.514673 }
    ],
    "metadata": {
      "latency_ms": 106,
      "vendor": "alibaba_intl",
      "vendor_region": "china",
      "cost_usd": 0.0001
    }
  }
}

Four candidates cost $0.0001. Fifty candidates cost $0.0001. Twenty results back cost $0.0001. So the sizing question isn’t “can I afford 50 candidates” — it’s how many your retrieval layer can produce without hurting recall. Send more.

Two details to get right. candidates is an array of plain strings, and top_k defaults to 10, so a bare request over 50 documents silently returns only the top ten — set it explicitly or you’ll conclude there’s a ceiling that isn’t there. The index in each result is the position in the array you sent, which means the join back to your own records is yours to write:

const KEY = process.env.INFRAI_API_KEY;
if (!KEY) throw new Error("INFRAI_API_KEY is not set");

export async function askYourDocs(question, candidates) {
  const res = await fetch("https://api.infrai.cc/v1/ai/rerank", {
    method: "POST",
    headers: { Authorization: `Bearer ${KEY}`, "Content-Type": "application/json" },
    body: JSON.stringify({
      query: question,
      candidates: candidates.map((c) => c.text),
      top_k: 5,
    }),
  });
  if (!res.ok) throw new Error(`rerank failed: ${res.status} ${await res.text()}`);

  const { data } = await res.json();
  const top = data.ranked.map((r) => ({ ...candidates[r.index], score: r.score }));

  const answer = await fetch("https://api.infrai.cc/v1/chat/completions", {
    method: "POST",
    headers: { Authorization: `Bearer ${KEY}`, "Content-Type": "application/json" },
    body: JSON.stringify({
      model: "glm-4-air",
      max_tokens: 300,
      messages: [
        { role: "system", content: "Answer only from the passages. Cite the doc id you used." },
        { role: "user", content: `${question}\n\n${top.map((t) => `[${t.id}] ${t.text}`).join("\n\n")}` },
      ],
    }),
  });
  if (!answer.ok) throw new Error(`answer failed: ${answer.status} ${await answer.text()}`);

  const json = await answer.json();
  return { text: json.choices[0].message.content, sources: top, spend: json.infrai?.cost_usd ?? 0 };
}

Retrieval, reranking, generation and cost attribution — one credential, one usage endpoint, one invoice. The alternative is an embedding vendor, a rerank vendor and an inference vendor, each with its own key rotation and its own monthly reconciliation.

The caveat that will decide it for some of you

Both routes came back with "region": "china" in our testing. That’s a routing property, not a contractual residency guarantee, and there’s no per-country pin on the embeddings path today. If your DPA names an EU region for text sent to a processor, this isn’t the right tool and no amount of price advantage fixes that — buy embeddings and rerank in-region from a vendor who will sign for it.

Two smaller limitations. The embedding model is chosen for you, so you can’t pin a specific dimensionality or a retrieval-tuned variant the way Voyage lets you. And because embeddings aren’t in the model catalogue, you can’t preflight the rate from GET /v1/ai/models — you learn it from a real call, which is what the script above does.

If none of that binds you, the shortest path to a working docs search is: index once with model: "auto", retrieve 50 candidates from whatever store you already have, rerank them for a hundredth of a cent, and spend the money you saved on a better generation model. Ollama on a spare box is a perfectly reasonable fourth option if your corpus never leaves the building.

References

• Infrai AI runtime API reference
• MODEL_NOT_FOUND error reference
• OpenAI API pricing
• Voyage AI pricing