Shrinking embedding dimensions: measured recall loss at 512, 256 and 64

Truncating 1024-dim vectors quarters your index footprint. We measured what it costs in recall on a live Infrai collection, and where the losses stop being acceptable.

You can shrink them, and for a million documents the arithmetic is worth taking seriously: 1024 float32 dimensions is 4.1 GB of raw vector payload, 256 is 1.0 GB, 64 is 262 MB. We ran the shrink on an Infrai collection and measured it — at 256 dimensions every one of our eight test questions still returned the same single best chunk, while roughly one in six of the remaining top-5 slots churned.

Where it gets interesting is how you shrink, and what the tail of the result list does when you do.

The shrink belongs in your ingest code

Start from what the platform actually hands you. The served embedding model is text-embedding-v4, and one call returns 1024 floats per input:

curl -s https://api.infrai.cc/v1/embeddings \
  -H "Authorization: Bearer $INFRAI_API_KEY" \
  -H 'content-type: application/json' \
  -d '{"model":"auto","input":"What is the safest way to add a NOT NULL column?"}' \
| jq '{model, dims: (.data[0].embedding | length)}'

{
  "model": "text-embedding-v4",
  "dims": 1024
}

That width is a property of the model, so the place to shrink is between the embed call and the upsert — a step your ingest loop owns.

Which is less annoying than it sounds, because text-embedding-v4 is a Matryoshka-style model: it’s trained so the leading coordinates carry most of the signal. Slice the prefix, renormalise to unit length, store that. Two lines.

import process from "node:process";

const BASE = "https://api.infrai.cc";
const KEY = process.env.INFRAI_API_KEY;
if (!KEY) throw new Error("INFRAI_API_KEY is not set");

/** Matryoshka-style prefix truncation: keep the first `d` coords, re-unit-normalise. */
export function shrink(vector, d) {
  const head = vector.slice(0, d);
  const norm = Math.hypot(...head);
  if (norm === 0) throw new Error("degenerate vector after truncation");
  return head.map((x) => x / norm);
}

async function post(path, body) {
  const res = await fetch(`${BASE}${path}`, {
    method: "POST",
    headers: { authorization: `Bearer ${KEY}`, "content-type": "application/json" },
    body: JSON.stringify(body),
  });
  const json = await res.json();
  if (!res.ok) throw new Error(`${path} ${res.status} ${json?.error?.code ?? ""} ${json?.error?.message ?? ""}`);
  return json;
}

export async function backfill(chunks, dim, collection) {
  const vectors = [];
  // The embeddings route takes at most ten strings per input array.
  for (let i = 0; i < chunks.length; i += 10) {
    const slice = chunks.slice(i, i + 10);
    const out = await post("/v1/embeddings", { model: "auto", input: slice.map((c) => c.text) });
    out.data.forEach((row, j) => {
      vectors.push({ id: slice[j].id, embedding: shrink(row.embedding, dim), metadata: { text: slice[j].text } });
    });
  }
  const result = await post("/v1/vector/upsert", { collection, vectors });
  return result.data.upserted;
}

A collection’s dimension is fixed when you create it, so shrinking an existing index is a rebuild rather than an edit — new collection, backfill, swap the name your app reads. POST /v1/vector/collection/create is free, which makes running two side by side during a cutover cheap:

curl -s -X POST https://api.infrai.cc/v1/vector/collection/create \
  -H "Authorization: Bearer $INFRAI_API_KEY" \
  -H 'content-type: application/json' \
  -d '{"collection":"dimlab_256","dimension":256,"metric":"cosine"}'

Get the arithmetic wrong and the upsert tells you exactly where, which is more helpful than it sounds when a backfill is halfway through 40,000 chunks:

{
  "ok": false,
  "error": {
    "code": "VECTOR_DIMENSION_MISMATCH",
    "http_status": 400,
    "message": "vector.upsert: vector 'x' has dim 3, collection expects 256",
    "retryable": false
  }
}

What we actually lost

Forty chunks of platform documentation, eight questions, five parallel collections built from the same 1024-dimension embeddings by truncating to each width. Recall is measured against the 1024 ranking, since that’s the thing being degraded. Read on 26 July 2026.

The shape of that is the useful part. Rank 1 is stubborn — it survived a sixteen-fold cut in every question but one. The tail is where the damage lands, and the tail is exactly what a generous top_k feeds into your prompt.

Here’s the same question at 64 dimensions, where the whole query vector fits in a code block:

curl -s -X POST https://api.infrai.cc/v1/vector/query \
  -H "Authorization: Bearer $INFRAI_API_KEY" \
  -H 'content-type: application/json' \
  -d '{"collection":"dimlab_64","top_k":5,"include_metadata":false,"embedding":[-0.0384,0.1616,0.0056,-0.1209,-0.0023,-0.0587,0.2279,0.0567,-0.0468,0.162,0.1247,-0.3071,-0.0999,0.0202,0.2255,0.2061,-0.3287,-0.3009,-0.1256,0.0334,0.1969,-0.0266,0.1395,-0.0748,0.0443,0.0985,0.0687,-0.1865,0.0015,-0.004,-0.0191,0.0932,0.2613,-0.09,-0.0606,0.0337,-0.0016,-0.0073,0.1018,0.1115,0.0311,-0.0057,-0.0584,0.1394,-0.0782,0.305,0.0521,-0.146,0.1291,0.0455,0.0471,-0.0804,-0.0391,0.093,-0.0793,-0.0175,0.0467,0.0755,-0.0132,-0.0363,0.0204,0.0111,0.0862,0.0661]}'

{
  "ok": true,
  "data": {
    "matches": [
      { "id": "doc-20", "score": 0.81307016 },
      { "id": "doc-19", "score": 0.60483025 },
      { "id": "doc-16", "score": 0.59576962 },
      { "id": "doc-10", "score": 0.54683592 },
      { "id": "doc-4", "score": 0.52394202 }
    ]
  }
}

At 1024 the same question returned doc-20, doc-19, doc-16, doc-4, doc-14. Three identical, then doc-10 — a chunk about pagination cursors — walks into fourth place.

Similarity scores inflate as you truncate

This is the failure mode that will actually break something in production, and it isn’t visible in a recall number.

Watch the fifth-place score in the table: 0.459 at full width, 0.474 at 256, 0.548 at 64. Top-1 barely moves (0.785 → 0.783 → 0.800). Fewer coordinates means fewer chances for two vectors to disagree, so everything looks more similar, and the spread between a real answer and a bystander compresses.

An absolute cutoff you tuned against 1024-wide vectors — score > 0.5, say — therefore lets more through once you truncate. Re-tune it per width, or switch to a relative rule and keep whatever scores within some fraction of the top hit.

Adding a reranking pass restores most of the ordering you gave up, which is the pairing we’d recommend below 256 dimensions (see the rerank measurements).

Run the check on your own corpus before you commit — forty chunks is an illustration, not a benchmark:

import os, json, requests

BASE = "https://api.infrai.cc"
KEY = os.environ["INFRAI_API_KEY"]
H = {"authorization": f"Bearer {KEY}", "content-type": "application/json"}

def search(collection, embedding, k=5):
    payload = {"collection": collection, "embedding": embedding, "top_k": k, "include_metadata": False}
    r = requests.post(f"{BASE}/v1/vector/query", headers=H, json=payload, timeout=30)
    r.raise_for_status()
    return [m["id"] for m in r.json()["data"]["matches"]]

def recall_at_k(baseline_col, shrunk_col, query_vectors, dim, k=5):
    hits = 0
    for full in query_vectors:
        head = full[:dim]
        norm = sum(x * x for x in head) ** 0.5
        small = [x / norm for x in head]
        base = set(search(baseline_col, full, k))
        hits += len(base.intersection(search(shrunk_col, small, k)))
    return hits / (k * len(query_vectors))

if __name__ == "__main__":
    vectors = json.load(open("query-vectors.json"))
    print(round(recall_at_k("dimlab_1024", "dimlab_256", vectors, 256), 3))

What shrinking saves you here, honestly

Not an Infrai storage line, because there isn’t one. The vector routes bill per call — POST /v1/vector/upsert at $0.0001 and POST /v1/vector/query at $0.0002, with $2 of credit on a new account — and no per-gigabyte meter, so a 256-dimension index and a 1024-dimension index of the same document count cost the same to run. Confirm today’s numbers rather than trusting this paragraph; rates drift downward and campaigns run:

curl -s https://api.infrai.cc/v1/discovery \
  -H "Authorization: Bearer $INFRAI_API_KEY" \
| jq '[.capabilities[] | select(.module == "vector") | {id, method, path, billing}]'

So if your bill is a Pinecone pod bill or a Qdrant Cloud node bill, dimension reduction is the right lever and this article’s numbers tell you what it costs in quality — but the saving lands on their invoice, not ours. That’s the honest version. What Infrai changes is the surrounding work: the embeddings call, the index, the rerank hop, the cron job that runs the backfill and the object store holding the source documents are one key and one bill, rather than four accounts to reconcile per tenant.

Where we’d draw the line

512 is close to free — 90% of the top-5 set preserved for half the bytes. 256 is the sweet spot if a reranker sits behind it. Below 128 you’re trading real quality for storage that, at a million vectors, is already under a gigabyte.

Some honest limits. This index doesn’t support scalar or binary quantization, so float32 truncation is the only compression lever exposed; if you need int8 or 1-bit vectors with rescoring, Milvus and Qdrant both ship that natively and would be the better pick. metric is fixed at creation alongside dimension, so a change means a rebuild either way. And prefix truncation only works on models trained for it — do not assume it transfers to whatever embedding model you migrate to next.

Verify the collection you built before you point traffic at it:

curl -s "https://api.infrai.cc/v1/vector/collection/get?collection=dimlab_256" \
  -H "Authorization: Bearer $INFRAI_API_KEY" | jq '.data'

A vector_count short of your chunk count means a batch failed silently somewhere in the backfill loop, which is the most common way one of these migrations goes wrong.

References

• Infrai vector API reference
• VECTOR_DIMENSION_MISMATCH error reference
• Optimization of embeddings storage for RAG systems using quantization and dimensionality reduction