Embed, upsert and query with the model and the index on one key

A working RAG loop in three calls, with the dimension that must match, the metric to pick, and the metadata field that makes answers citable.

A retrieval loop needs an embedding model and a vector store, and those are usually two vendors with two keys. On Infrai they’re one: POST /v1/embeddings on the OpenAI-compatible surface produces the vectors, POST /v1/vector/collection/create makes somewhere to put them, POST /v1/vector/upsert stores them and POST /v1/vector/query searches.

Four calls, one credential, and one number that has to match on both sides.

Create the collection with the right dimension

curl -sS -X POST "https://api.infrai.cc/v1/vector/collection/create" \
  -H "Authorization: Bearer ${INFRAI_API_KEY}" \
  -H "Content-Type: application/json" \
  -d '{"collection": "docs", "dimension": 1536, "metric": "cosine", "metadata": {"owner": "docs-team"}}'

{
  "ok": true,
  "data": {
    "collection": "docs",
    "dimension": 1536,
    "metric": "cosine",
    "vector_count": 0,
    "state": "ready",
    "raw_vendor_payload": null
  }
}

dimension must equal the output size of whatever embedding model you use, and it’s fixed for the life of the collection. Get it from one real embedding call rather than from documentation — then a model change means a new collection and a re-index, which is a known cost rather than a surprise.

metric accepts cosine, euclidean and dotproduct.

Cosine is the right default for text embeddings, and the reason is worth knowing rather than taking on faith: it compares the direction of two vectors and ignores their magnitude, so a three-paragraph explanation and a one-sentence summary of the same idea land close together — whereas a magnitude-sensitive metric systematically favours longer text, and your retrieval quietly becomes a length contest.

Embed and upsert

curl -sS -X POST "https://api.infrai.cc/v1/embeddings" \
  -H "Authorization: Bearer ${INFRAI_API_KEY}" \
  -H "Content-Type: application/json" \
  -d '{"model": "auto", "input": ["An API gateway routes requests to services."]}'

curl -sS -X POST "https://api.infrai.cc/v1/vector/upsert" \
  -H "Authorization: Bearer ${INFRAI_API_KEY}" \
  -H "Content-Type: application/json" \
  -d '{
    "collection": "docs",
    "vectors": [
      {"id": "guide-gateway#0", "embedding": [0.0123, -0.0456, 0.0789],
       "metadata": {"url": "https://docs.example.com/gateway", "title": "Gateway basics",
                    "text": "An API gateway routes requests to services.", "tenant": "t_northwind"}}
    ]
  }'

Two habits worth adopting immediately. Deterministic ids — <source>#<chunk> — mean a re-index overwrites rather than duplicating, because the same id overwrites. And storing the chunk text in metadata means a query result is directly usable: you get the passage back with the score instead of holding a second lookup table keyed by id.

Query

curl -sS -X POST "https://api.infrai.cc/v1/vector/query" \
  -H "Authorization: Bearer ${INFRAI_API_KEY}" \
  -H "Content-Type: application/json" \
  -d '{
    "collection": "docs",
    "embedding": [0.0119, -0.0461, 0.0802],
    "top_k": 5,
    "filter": {"tenant": "t_northwind"},
    "include_metadata": true
  }'

{
  "ok": true,
  "data": {
    "items": [
      {"id": "guide-gateway#0", "score": 0.91,
       "metadata": {"url": "https://docs.example.com/gateway", "title": "Gateway basics",
                    "text": "An API gateway routes requests to services.", "tenant": "t_northwind"}}
    ],
    "next_cursor": null
  }
}

include_metadata: true is what makes the answer citable — the URL comes back with the passage, so the model can attribute rather than assert.

The whole loop

import os

import requests

API = "https://api.infrai.cc"
KEY = os.environ["INFRAI_API_KEY"]
COLLECTION = os.environ.get("RAG_COLLECTION", "docs")
SESSION = requests.Session()
SESSION.headers.update({"Authorization": f"Bearer {KEY}", "Content-Type": "application/json"})


def embed(texts: list[str], model: str = "auto") -> list[list[float]]:
    resp = SESSION.post(f"{API}/v1/embeddings", json={"model": model, "input": texts}, timeout=120)
    resp.raise_for_status()
    return [row["embedding"] for row in resp.json()["data"]]


def ensure_collection(dimension: int) -> dict:
    resp = SESSION.post(
        f"{API}/v1/vector/collection/create",
        json={"collection": COLLECTION, "dimension": dimension, "metric": "cosine"},
        timeout=60,
    )
    body = resp.json()
    if body.get("ok"):
        return body["data"]
    # Already there is success for this purpose; anything else is not.
    if body.get("error", {}).get("code") in {"VECTOR_COLLECTION_EXISTS", "ALREADY_EXISTS"}:
        return SESSION.get(f"{API}/v1/vector/collection/get",
                           params={"collection": COLLECTION}, timeout=30).json()["data"]
    raise RuntimeError(body["error"]["code"])


def index(chunks: list[dict], tenant: str) -> int:
    """chunks: [{"id", "text", "url", "title"}]. Embed in one call, upsert in one
    call — a request per chunk is the most common way an indexing job becomes slow
    and expensive at the same time."""
    embeddings = embed([chunk["text"] for chunk in chunks])
    ensure_collection(len(embeddings[0]))
    vectors = [
        {"id": chunk["id"], "embedding": embedding,
         "metadata": {"url": chunk["url"], "title": chunk["title"],
                      "text": chunk["text"], "tenant": tenant}}
        for chunk, embedding in zip(chunks, embeddings)
    ]
    resp = SESSION.post(f"{API}/v1/vector/upsert",
                        json={"collection": COLLECTION, "vectors": vectors}, timeout=180)
    resp.raise_for_status()
    return len(vectors)


def search(question: str, tenant: str, top_k: int = 5) -> list[dict]:
    embedding = embed([question])[0]
    resp = SESSION.post(
        f"{API}/v1/vector/query",
        json={"collection": COLLECTION, "embedding": embedding, "top_k": top_k,
              "filter": {"tenant": tenant}, "include_metadata": True},
        timeout=60,
    )
    resp.raise_for_status()
    return [{"score": item.get("score"), "url": (item.get("metadata") or {}).get("url"),
             "text": (item.get("metadata") or {}).get("text")}
            for item in resp.json()["data"].get("items", [])]


def answer(question: str, tenant: str) -> str:
    hits = search(question, tenant)
    context = "\n\n".join(f"[{h['url']}]\n{h['text']}" for h in hits)
    resp = SESSION.post(
        f"{API}/v1/chat/completions",
        json={"model": "auto", "messages": [
            {"role": "system", "content": "Answer only from the context. Cite the URLs you used."},
            {"role": "user", "content": f"Context:\n{context}\n\nQuestion: {question}"}]},
        timeout=120,
    )
    resp.raise_for_status()
    return resp.json()["choices"][0]["message"]["content"]


if __name__ == "__main__":
    print(answer("What does an API gateway do?", tenant="t_northwind"))

The embedding call, the index, the search and the model that writes the answer are the same SESSION and the same key. There’s no second client, no second credential in the environment, and the cost of the whole loop is one line in GET /v1/account/usage.

The three decisions that matter

Dimension is the one that bites hardest, because the failure comes much later than the mistake: an index built against one model and queried with embeddings from another returns plausible-looking nonsense.

Limitations

There’s no hybrid search: no BM25 keyword component to combine with vector similarity, so an exact-term query — a product code, an error code — may not retrieve the chunk containing it. Weaviate’s native hybrid search exists for exactly that, and Qdrant and Pinecone both expose richer filtering and index tuning than the four fields here.

Nor is there a reranking step inside the query. POST /v1/ai/rerank is a separate call on the same key if you want one, which is the usual fix for “the right chunk was fourth”.

Query and upsert bill per call at rates live in GET /v1/discovery/vector.query (verified 2026-09-21), and collections accrue standing storage rent while they exist — so a collection you stopped using is worth deleting. Platform rates drift downward as vendor contracts improve.

References

• Infrai vector API reference
• Infrai AI runtime API reference
• Infrai documentation home