Cheap RAG in Node: cost the chunks, the index and the answer separately

A RAG budget has three independent line items and only one of them is embeddings. How to measure each with Infrai's free token and cost endpoints before writing the pipeline.

Most RAG cost surprises come from the wrong line item. Embeddings are a one-off charge measured in cents per thousand documents; the recurring bill is the generation call, because every query stuffs several retrieved chunks into a prompt you pay for again and again. Budget the three parts separately — indexing, retrieval, generation — and the optimisation targets sort themselves out. Infrai’s token-count and cost endpoints are free, which makes that arithmetic cheap to do up front.

The whole pipeline sits on one key: embed, upsert, query, rerank, generate. That matters less for the price than for the fact that you can measure each step before you commit to a chunking strategy.

Step one: count the tokens you’re about to index

Chunk size drives everything downstream — index size, retrieval precision, and how many tokens each answer prompt carries. Measure a real document rather than assuming 4 characters per token.

const KEY = process.env.INFRAI_API_KEY;
if (!KEY) throw new Error("INFRAI_API_KEY is not set");

function chunk(text, words = 220) {
  const parts = text.split(/\s+/);
  const out = [];
  for (let i = 0; i < parts.length; i += words) out.push(parts.slice(i, i + words).join(" "));
  return out;
}

async function countTokens(text) {
  const res = await fetch("https://api.infrai.cc/v1/ai/tokens/count", {
    method: "POST",
    headers: { Authorization: `Bearer ${KEY}`, "Content-Type": "application/json" },
    body: JSON.stringify({ model: "gpt-5-mini", messages: [{ role: "user", content: text }] }),
  });
  if (!res.ok) throw new Error(`token count failed: ${res.status}`);
  const { data } = await res.json();
  return data.prompt_tokens;
}

const handbook = await (await fetch("https://raw.githubusercontent.com/nodejs/node/main/README.md")).text();
const chunks = chunk(handbook);
const counts = await Promise.all(chunks.slice(0, 20).map(countTokens));
const mean = counts.reduce((a, b) => a + b, 0) / counts.length;

console.log(`${chunks.length} chunks, mean ${Math.round(mean)} tokens`);
console.log(`whole corpus at 10,000 documents ≈ ${Math.round((mean * chunks.length * 10000) / 1e6)} Mtok`);

That last line is the number worth writing on the whiteboard. A 220-word chunk lands around 300 tokens; ten thousand documents of a dozen chunks each is roughly 36 Mtok to embed once — and the same 36 Mtok is what you’d re-pay on every re-chunk, which is the real argument for getting chunk size right early.

Step two: price the embedding pass

Start from the catalogue, because the model you pin decides both the price and the vector width your collection has to match:

export INFRAI_API_KEY="your_infrai_api_key"

curl -sS "https://api.infrai.cc/v1/ai/models?capability=embed&available=true" \
  -H "Authorization: Bearer ${INFRAI_API_KEY}"

Then embed. The route is the OpenAI-compatible one, so the request is the shape your existing code already emits:

curl -sS -X POST "https://api.infrai.cc/v1/embeddings" \
  -H "Authorization: Bearer ${INFRAI_API_KEY}" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "auto",
    "input": ["Refunds are issued to the original payment method.", "Bank transfers need an IBAN first."]
  }'

The response carries the vectors plus a usage block and Infrai’s own cost line:

{
  "object": "list",
  "data": [{ "object": "embedding", "index": 0, "embedding": [0.0237, 0.0661, "…1024 floats"] }],
  "model": "text-embedding-v4",
  "usage": { "prompt_tokens": 12, "total_tokens": 12 },
  "infrai": { "cost_usd": 0.00000084, "vendor": "alibaba_intl", "region": "china", "cache": false }
}

Verified 2026-07-26: model: "auto" resolved to text-embedding-v4 at 1024 dimensions, priced $0.07 per Mtok, with the exact charge for the call echoed back in infrai.cost_usd. Run the 36 Mtok from step one through that rate and the entire index costs about $2.50. Pin a specific model id in production rather than riding auto, because vectors from two models can’t be compared and a silent routing change would quietly poison retrieval. Prices here drift downward and discount campaigns run, so read GET /v1/ai/models?capability=embed&available=true for today’s figure rather than trusting this paragraph.

That $2.50 is also the number that makes the rest of the article’s point: indexing is a rounding error next to generation.

Step three: the index and the search

The vector side is self-hosted on the same key. Prove the wiring with a tiny 4-dimensional collection before you spend anything on real embeddings — it takes about a minute.

curl -sS -X POST "https://api.infrai.cc/v1/vector/collection/create" \
  -H "Authorization: Bearer ${INFRAI_API_KEY}" \
  -H "Content-Type: application/json" \
  -d '{"collection":"rag_smoke_test","dimension":4,"metric":"cosine"}'

curl -sS -X POST "https://api.infrai.cc/v1/vector/upsert" \
  -H "Authorization: Bearer ${INFRAI_API_KEY}" \
  -H "Content-Type: application/json" \
  -d '{
    "collection": "rag_smoke_test",
    "vectors": [
      {"id":"handbook-1","embedding":[0.12,0.04,0.91,0.33],"metadata":{"doc":"handbook","page":7}},
      {"id":"handbook-2","embedding":[0.88,0.10,0.02,0.41],"metadata":{"doc":"handbook","page":12}}
    ]
  }'

Then query it with a vector and read the scores:

curl -sS -X POST "https://api.infrai.cc/v1/vector/query" \
  -H "Authorization: Bearer ${INFRAI_API_KEY}" \
  -H "Content-Type: application/json" \
  -d '{"collection":"rag_smoke_test","embedding":[0.11,0.05,0.90,0.30],"top_k":2,"include_metadata":true}'

{
  "ok": true,
  "data": {
    "collection": "rag_smoke_test",
    "matches": [
      { "id": "handbook-1", "score": 0.9995688, "metadata": { "doc": "handbook", "page": 7 } },
      { "id": "handbook-2", "score": 0.26008149, "metadata": { "doc": "handbook", "page": 12 } }
    ]
  }
}

For the real collection, set dimension to whatever your pinned model emits — 1024 for text-embedding-v4 — and every other call is identical. Upserts and queries are billed per call at fractions of a cent, with the current figures readable from discovery:

curl -sS "https://api.infrai.cc/v1/discovery" \
  -H "Authorization: Bearer ${INFRAI_API_KEY}" | grep -o '"vector[^}]*}' | head -20

A query costs about twice an upsert, and both are dwarfed by the generation call they feed. That ratio is the durable fact; the digits move, generally downward.

Step four: reranking beats a bigger model

The cheapest way to cut generation cost isn’t a cheaper model — it’s sending fewer, better chunks. Pull top_k: 20 from the index, rerank, keep four.

curl -sS -X POST "https://api.infrai.cc/v1/ai/rerank" \
  -H "Authorization: Bearer ${INFRAI_API_KEY}" \
  -H "Content-Type: application/json" \
  -d '{
    "query": "how do I rotate an API key",
    "candidates": [
      "Key rotation is done from the account keys endpoint.",
      "Our office is in Singapore.",
      "Billing runs monthly on the first."
    ],
    "top_k": 2
  }'

It returns ranked as {index, score} pairs, trimmed to the top_k you asked for — in our testing the relevant chunk scored 0.74 against 0.31 for the noise, on a call that costs $0.0001. Dropping from 20 chunks to 4 removes roughly 4,800 prompt tokens from every single query, which at any model tier is a far larger saving than the rerank costs.

Step five: price the answer call

curl -sS -X POST "https://api.infrai.cc/v1/ai/cost/estimate" \
  -H "Authorization: Bearer ${INFRAI_API_KEY}" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "openai/gpt-4o-mini",
    "messages": [
      {"role": "system", "content": "Answer only from the supplied context. Cite the page."},
      {"role": "user", "content": "Context chunk one. Context chunk two. Question: how are refunds issued?"}
    ],
    "expected_output_tokens": 250
  }'

The breakdown splits input_cost from output_cost, and for RAG the input side is usually the larger of the two — which is exactly why chunk count matters more than output length.

Offline enrichment — chunk summaries, synthetic questions, per-document tags — belongs in a batch job rather than a loop of live calls:

const KEY = process.env.INFRAI_API_KEY;
if (!KEY) throw new Error("INFRAI_API_KEY is not set");

const chunks = ["Refunds are issued to the original payment method.", "Bank transfers need an IBAN first."];
const payload = {
  requests: chunks.map((text) => ({
    model: "glm-4-flash",
    max_tokens: 60,
    messages: [
      { role: "system", content: "Write one search-friendly question this passage answers." },
      { role: "user", content: text },
    ],
  })),
  metadata: { job: "chunk-enrichment" },
  store: true,
};

const res = await fetch("https://api.infrai.cc/v1/ai/batch/submit", {
  method: "POST",
  headers: { Authorization: `Bearer ${KEY}`, "Content-Type": "application/json" },
  body: JSON.stringify(payload),
});
if (!res.ok) throw new Error(`batch submit failed: ${res.status}`);
const { data } = await res.json();
console.log(`batch ${data.batch_id} state=${data.state} count=${data.total_count}`);

Poll GET /v1/ai/batch/status/{id}, then page GET /v1/ai/batch/results/{id} with the returned cursor. Both are free, so a long job costs you nothing to watch.

Limitations, and when to buy elsewhere

The vector API takes an embedding you computed — it won’t chunk, embed and index a document for you in one call, so the orchestration above is yours to write. There’s no hybrid keyword-plus-vector scoring and no managed filter tuning, so if you need BM25 fused with dense retrieval, a specialist store like Pinecone or Qdrant will beat this on features. And if your corpus is small and already in Postgres, pgvector in the database you’re already running is simpler and cheaper than any API — that’s the honest answer for a few thousand documents.

Self-hosting the embedding model under Ollama is the other real option, and it drives the marginal token cost to zero if you have somewhere to run it. At $0.07 per Mtok the API side has to be very large before that trade pays for the operational work.

What you get here instead is that the embedding, the retrieval, the batch enrichment, the storage for source documents and the error tracking around the pipeline sit behind one key and land on one bill, with per-tenant attribution as a query rather than a reconciliation exercise. For a RAG feature inside a product — as opposed to a standalone search product — that usually matters more than any single line item.

References

• Infrai vector API reference
• Infrai AI runtime API reference
• OpenAI embeddings guide
• Ollama embedding models