Reranking 50 search candidates: request, response and mapping back

The exact rerank payload Infrai accepts, the ranked array it returns, and how index-based results join back to your documents — with the ten-result ceiling we hit.

Send { "query": "...", "documents": ["...", "..."] } to POST /v1/ai/rerank on Infrai and you get back data.ranked, an array of { index, score } sorted best-first. The index is the position of that string in the array you sent, so joining back to your own records is an array lookup — no ids travel to the reranker and none come back. One detail matters more than the payload shape: the response is capped at ten entries.

That ceiling is the thing most rerank write-ups skip, and it changes how you’d wire a 50-candidate pipeline. Everything here was measured against a live account on 26 July 2026.

The request

Documents are plain strings. Nothing else.

export INFRAI_API_KEY="your_infrai_api_key"

curl -sS https://api.infrai.cc/v1/ai/rerank \
  -H "Authorization: Bearer ${INFRAI_API_KEY}" \
  -H "Content-Type: application/json" \
  -d '{
        "query": "how do I rotate an API key",
        "documents": [
          "Rotating a project key issues a new secret and keeps the old one valid for 24 hours.",
          "Our office is closed on public holidays.",
          "Billing invoices are generated on the first of each month."
        ]
      }'

{
  "ok": true,
  "data": {
    "ranked": [
      { "index": 0, "score": 0.738931 },
      { "index": 2, "score": 0.299001 },
      { "index": 1, "score": 0.136277 }
    ],
    "metadata": {
      "request_id": "req_d896473f8f3b4c88bd56e089",
      "latency_ms": 120,
      "vendor": "alibaba_intl",
      "vendor_region": "china",
      "cache_layer": "none"
    }
  },
  "metadata": { "cost_usd": 0.0001, "vendor": "alibaba_intl", "idempotent_replay": false }
}

Note what is not in there: no document echo, no id field, no vendor-side handle. The reranker sees an unlabelled list and answers in positions. That’s a feature for a search stack — you never have to sanitise your primary keys before shipping text to a vendor — but it means the join is yours to write.

Passing structured documents fails. We tried [{ "id": "doc-1", "text": "…" }] and the upstream rejected it with InvalidParameter: Input should be a valid string. Flatten first, keep a parallel array, map by position.

Mapping 50 candidates back to your documents

Here is the whole loop in Node 22: candidate objects from your search layer, a flattened text array, one rerank call, then a join that reattaches every field the reranker never saw.

const key = process.env.INFRAI_API_KEY;
if (!key) throw new Error("set INFRAI_API_KEY before running this");

// Whatever your search returned. Only `text` is sent upstream.
const candidates = Array.from({ length: 50 }, (_, i) => ({
  doc_id: `kb-${1000 + i}`,
  url: `https://example.internal/kb/${1000 + i}`,
  text: i === 17
    ? "API keys are rotated from the console; the previous secret stays valid for 24 hours."
    : `Unrelated filler paragraph about office logistics, number ${i}.`,
}));

const res = await fetch("https://api.infrai.cc/v1/ai/rerank", {
  method: "POST",
  headers: { Authorization: `Bearer ${key}`, "Content-Type": "application/json" },
  body: JSON.stringify({
    query: "how do I rotate an API key",
    documents: candidates.map((c) => c.text),
  }),
});

const payload = await res.json();
if (!res.ok || payload.ok === false) {
  console.error("rerank failed:", res.status, payload.error?.code, payload.error?.message);
  process.exit(1);
}

const reordered = payload.data.ranked.map(({ index, score }, position) => ({
  rank: position + 1,
  score,
  ...candidates[index],
}));

console.log(`asked for 50, got ${reordered.length} back`);
for (const row of reordered.slice(0, 3)) {
  console.log(`#${row.rank} ${row.doc_id} ${row.score.toFixed(4)} ${row.url}`);
}

Run that and the log line reads asked for 50, got 10 back. The genuinely relevant document came first at 0.7305; the next nine clustered between 0.24 and 0.26, which is the shape you want — a clear leader and an undifferentiated tail.

The ceiling, and what top_n does not do

We sent 50 documents with top_n: 25, then 12 documents with no top_n at all, then 3 documents with top_n: 2. Results: 10, 10 and 3. So the parameter is accepted and ignored, and the response tops out at ten ranked entries. return_documents: true is equally inert — no text comes back either way.

The practical consequence is a design constraint, not a bug to work around.

If ten is enough for your final answer — a chat citation panel, the context window of a RAG prompt — send your whole candidate set and take what comes back. If you need a full 50-item reordering, one call won’t do it, and that’s a real limitation to plan around: shard the candidates into groups, rerank each, and merge on score. Scores from separate calls are broadly comparable in our testing but not formally calibrated, so treat a merged ordering as a heuristic rather than a guarantee.

The sharded version in Python 3, five calls of ten:

import json
import os
import urllib.error
import urllib.request

KEY = os.environ["INFRAI_API_KEY"]
QUERY = "how do I rotate an API key"
CANDIDATES = [
    {"doc_id": f"kb-{1000 + i}", "text": f"Knowledge base paragraph number {i}."}
    for i in range(50)
]


def rerank(texts):
    body = json.dumps({"query": QUERY, "documents": texts}).encode()
    req = urllib.request.Request(
        "https://api.infrai.cc/v1/ai/rerank",
        data=body,
        headers={"Authorization": f"Bearer {KEY}", "Content-Type": "application/json"},
        method="POST",
    )
    try:
        with urllib.request.urlopen(req, timeout=30) as resp:
            return json.load(resp)["data"]["ranked"]
    except urllib.error.HTTPError as exc:
        raise SystemExit(f"rerank failed: {exc.code} {exc.read().decode()[:200]}")


merged = []
for start in range(0, len(CANDIDATES), 10):
    shard = CANDIDATES[start:start + 10]
    for row in rerank([c["text"] for c in shard]):
        merged.append((row["score"], shard[row["index"]]["doc_id"]))

merged.sort(reverse=True)
print(f"merged {len(merged)} scored documents; best: {merged[0]}")

Vendor, pinning and price

The route is served by alibaba_intl (the qwen_intl binding) out of a China region by default, at roughly 120 ms upstream and 210-270 ms end-to-end from Europe. Vendor pinning is where the surface is thinner than the catalogue implies: "vendor": "cohere" returned VENDOR_NOT_CONFIGURED (503), and pinning "model": "gte-rerank" came back Model.AccessDenied. Send the plain body and let the default vendor serve it.

Two catalogue quirks are worth flagging before you build monitoring on them:

curl -sS "https://api.infrai.cc/v1/ai/models?capability=rerank&available=true" \
  -H "Authorization: Bearer ${INFRAI_API_KEY}"

{ "object": "list", "capability": "rerank", "available_only": true, "count": 0, "data": [] }

That count: 0 arrives even though the route works and bills — the model catalogue only enumerates chat, image and TTS models today. For the authoritative billing class, read discovery instead:

curl -sS https://api.infrai.cc/v1/discovery \
  -H "Authorization: Bearer ${INFRAI_API_KEY}" \
  | grep -o '"id": *"ai.rerank"[^}]*}[^}]*}' | head -1

Rerank is billable per request rather than per document — the catalogue quotes about $0.00105 and the calls we made settled at $0.0001 each, verified 26 July 2026. New accounts get $2 of free credit, which is a few thousand rerank calls before you pay anything. Rates on this surface drift downward and campaign discounts run, so read GET /v1/discovery for today’s figure rather than trusting a number in an article.

Is a cross-encoder even the right tool?

If your requirement is “reorder all 50 and give me every score”, go direct to Cohere’s Rerank API or the rerank models in Amazon Bedrock; both return the full set and let you pin a model version, and we’d rather point you there than have you shard calls to fake it. OpenRouter’s rerank passthrough is another honest option if you’re already routing chat through it. What Infrai buys you is that the reranker, the vector index it draws from, and the object store holding the source documents all sit behind the one key and one invoice — so adding relevance tuning to an existing pipeline is a new call, not a new vendor relationship.

References

• Infrai AI Runtime API reference
• Infrai vector search API
• Cohere Rerank API reference