OpenAI-compatible API

Run AI inference through infrai with the OpenAI SDK you already use — change only the base URL. Chat, embeddings, images, speech (TTS/ASR), moderations and models in the exact OpenAI wire format, with smart cross-vendor routing, one wallet and one bill.

Drop-in: point the OpenAI SDK at infrai

infrai serves AI inference in the exact OpenAI request/response shape. Point any OpenAI client at https://api.infrai.cc/v1 with your infrai project key as the Bearer token — no new SDK, no new knowledge. Every response adds an extra top-level infrai object (cost / vendor / model) that a vanilla client simply ignores.

Python

python

from openai import OpenAI

client = OpenAI(
    base_url="https://api.infrai.cc/v1",   # the ONLY change vs vanilla OpenAI
    api_key="ifr_...",          # your infrai project key
)

resp = client.chat.completions.create(
    model="auto",   # balanced routing across healthy vendors ("cheapest" / "smartest" also valid)
    messages=[{"role": "user", "content": "Explain quicksort in one sentence."}],
)
print(resp.choices[0].message.content)
# infrai cost/vendor extension — a vanilla OpenAI client just ignores it:
print(resp.infrai["cost_usd"], resp.infrai["vendor"], resp.infrai["model"])

JavaScript / TypeScript

javascript

import OpenAI from "openai";

const client = new OpenAI({
  baseURL: "https://api.infrai.cc/v1",   // the only change vs vanilla OpenAI
  apiKey: process.env.INFRAI_API_KEY,
});

const resp = await client.chat.completions.create({
  model: "auto",
  messages: [{ role: "user", content: "Explain quicksort in one sentence." }],
});
console.log(resp.choices[0].message.content);
console.log(resp.infrai.cost_usd, resp.infrai.vendor);

curl

bash

curl https://api.infrai.cc/v1/chat/completions \
  -H "Authorization: Bearer $INFRAI_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "auto",
    "messages": [{"role": "user", "content": "Explain quicksort in one sentence."}]
  }'

model="auto" lets infrai route to the best/cheapest healthy vendor; pass task / prefer via extra_body, or pin a real model like "gpt-4o-mini". The old custom /v1/ai/chat shapes are retired — use the OpenAI surface.

Endpoints

The standard OpenAI endpoints, all under the /v1 base URL. Each maps to an infrai capability with full smart routing, failover, caching and billing.

Endpoint	infrai capability	Purpose
`POST /v1/chat/completions`	`ai.chat`	Chat completions — text, vision and tool calls, with streaming.
`POST /v1/embeddings`	`ai.embed`	Text embeddings for search and RAG.
`POST /v1/images/generations`	`ai.image`	Image generation from a text prompt.
`POST /v1/moderations`	`ai.moderation`	Classify whether content violates usage policy.
`POST /v1/audio/speech`	`ai.tts`	Text-to-speech — synthesize spoken audio.
`POST /v1/audio/transcriptions`	`ai.asr`	Speech-to-text — transcribe an audio file.
`GET /v1/models`	`ai.models.list`	List every model a verified, routable vendor serves.
`GET /v1/models/{model}`	`ai.models.get`	Retrieve one model's details (pricing, context, availability).

Smart routing rides the model field

infrai's reason to exist — cross-vendor routing — survives the OpenAI model field. Use a routing mode to let infrai choose, or pin a concrete model (optionally a vendor) to take control.

python

# 1) Routing MODES — the complete set; infrai picks the vendor/model
#    (SSOT: infrai-spec/enums/model_routing.yaml; also listed by GET /v1/models):
model="auto"        # balanced default, honors task / prefer (== prefer "balanced")
model="cheapest"    # cheapest healthy model for the task
model="smartest"    # flagship-tier model

# 2) Pin a concrete model (any model a verified vendor serves):
model="gpt-4o-mini"
model="deepseek-chat"
model="dashscope/qwen-plus"   # "vendor/model" pins the vendor too

# task / prefer ride OpenAI's extra_body (ignored by vanilla clients):
client.chat.completions.create(
    model="auto",
    messages=[...],
    extra_body={"task": "coding", "prefer": "cheapest"},
)

Chat request parameters — generated from the contract, so the model type below is the exact value set the gateway accepts:

Name	Type	Required	Description
`model`	`string`	Optional	Model to use. 'auto' (default) lets infrai smart-route across healthy vendors; 'vendor/model' pins a specific vendor and model. `e.g. auto`
`messages`	`object[]`	Required	OpenAI chat messages: a list of {role, content}.
`temperature`	`number`	Optional
`max_tokens`	`integer`	Optional
`top_p`	`number`	Optional
`stream`	`boolean`	Optional	Server-Sent Events streaming when true.
`tools`	`object[]`	Optional
`response_format`	`object`	Optional

Chat: streaming, vision and tools

Streaming, vision (image inputs) and tool/function calling all work exactly as in the OpenAI SDK.

Streaming (Server-Sent Events)

python

stream = client.chat.completions.create(
    model="auto",
    messages=[{"role": "user", "content": "Write a haiku about smart routing."}],
    stream=True,   # real token-by-token Server-Sent Events
)
for chunk in stream:
    print(chunk.choices[0].delta.content or "", end="", flush=True)

Vision — image input

python

resp = client.chat.completions.create(
    model="auto",   # infrai resolves to an available vision model
    messages=[{
        "role": "user",
        "content": [
            {"type": "text", "text": "What is in this image?"},
            {"type": "image_url",
             "image_url": {"url": "https://example.com/cat.jpg"}},
        ],
    }],
)
print(resp.choices[0].message.content)

Tool / function calling

python

resp = client.chat.completions.create(
    model="auto",
    messages=[{"role": "user", "content": "What's the weather in Paris?"}],
    tools=[{
        "type": "function",
        "function": {
            "name": "get_weather",
            "parameters": {"type": "object",
                           "properties": {"city": {"type": "string"}}},
        },
    }],
)
print(resp.choices[0].message.tool_calls)

Embeddings

Create embeddings and pipe them straight into the infrai vector store for retrieval-augmented generation.

python

resp = client.embeddings.create(
    model="auto",
    input=["first document", "second document"],
)
vector = resp.data[0].embedding
# → feed straight into POST /v1/vector/upsert for retrieval (RAG)

Images, speech and transcription

Image generation, text-to-speech and speech-to-text use the standard OpenAI methods. Audio speech returns binary audio; transcription accepts a multipart file upload or base64 JSON.

Image generation

python

img = client.images.generate(
    model="auto",
    prompt="a red panda coding at a laptop, studio lighting",
    size="1024x1024",
)
print(img.data[0].url or img.data[0].b64_json[:32])

Text-to-speech (TTS)

bash

curl https://api.infrai.cc/v1/audio/speech \
  -H "Authorization: Bearer $INFRAI_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model": "auto", "voice": "alloy", "input": "Hello from infrai."}' \
  --output speech.mp3

Speech-to-text (ASR)

python

with open("meeting.mp3", "rb") as audio:
    tr = client.audio.transcriptions.create(model="auto", file=audio)
print(tr.text)

The infrai extension: cost, vendor and cache

Every response carries an extra top-level infrai object so you can see exactly what a call cost, which vendor and model served it, the region, and whether it was a cache hit. A vanilla OpenAI client ignores the extra field; a power user reads it.

json

{
  "id": "chatcmpl-...",
  "object": "chat.completion",
  "model": "deepseek-chat",
  "choices": [{ "message": { "role": "assistant", "content": "..." } }],
  "usage": { "prompt_tokens": 12, "completion_tokens": 30, "total_tokens": 42 },
  "infrai": {
    "cost_usd": 0.0000123,
    "vendor": "deepseek",
    "model": "deepseek-chat",
    "region": "china",
    "cache": false,
    "request_id": "req_..."
  }
}

The same facts also ride response headers — X-Infrai-Request-Id, X-Infrai-Vendor and X-Infrai-Cost-Usd — so you can observe cost and routing without parsing the body.

Listing models

Enumerate the live, routable model menu. /v1/models is OpenAI-standard; /v1/ai/models is the infrai-native, public list with availability and per-million-token pricing, filterable by capability.

bash

# OpenAI-standard: every model a verified, routable vendor serves:
curl https://api.infrai.cc/v1/models -H "Authorization: Bearer $INFRAI_API_KEY"

# infrai-native (public, richer): availability + per-Mtok price, filterable:
curl "https://api.infrai.cc/v1/ai/models?capability=vision"

Errors

Failures come back in the OpenAI error shape — an error object with message, type and code — so the OpenAI SDK raises the APIError your code already handles. 401 is an authentication error, 402 maps to insufficient quota (top up your wallet), 429 is a rate limit, and 5xx is an upstream vendor error after failover. See the full error catalog for the machine-readable codes.

Drop-in: point the OpenAI SDK at infrai

Endpoints

Smart routing rides the model field

Chat: streaming, vision and tools

Embeddings

Images, speech and transcription

The infrai extension: cost, vendor and cache

Listing models

Errors

Next steps