Publishing thousands of realtime updates without tripping limits

Batch publishing, coalescing on the sender, and the backoff that keeps a burst from becoming a queue of retries. Plus the stride nobody sets.

A job that finishes ten thousand items and publishes ten thousand Infrai realtime events is fighting two limits at once: the platform’s rate limit, and the browser’s ability to render. POST /v1/realtime/publish/batch takes many messages in one request, which solves the first — and coalescing on your side solves the second, which is the one your users notice.

Batch first, then coalesce, then back off. In that order, because each step reduces the work the next has to do.

Batch instead of looping

curl -sS -X POST "https://api.infrai.cc/v1/realtime/publish/batch" \
  -H "Authorization: Bearer ${INFRAI_API_KEY}" \
  -H "Content-Type: application/json" \
  -d '{
    "messages": [
      {"channel": "orders:live", "event": "message.published", "data": {"type": "order.created", "id": "ord_8821"}},
      {"channel": "orders:live", "event": "message.published", "data": {"type": "order.created", "id": "ord_8822"}},
      {"channel": "stats:live", "event": "message.published", "data": {"type": "counter", "orders_today": 412}}
    ]
  }'

{
  "ok": true,
  "data": { "published": 3 }
}

One request, three deliveries, and published confirms the count. Different channels in the same batch are fine, which matters for a worker updating a resource view and an aggregate counter in the same tick.

Coalesce before you batch

Here’s the thing most implementations miss. If your job updates a progress counter a thousand times, the client only ever renders the latest value — so nine hundred and ninety-nine of those publishes were work for nobody.

Keep the newest value per key and flush on a timer:

const API = "https://api.infrai.cc";
const KEY = process.env.INFRAI_API_KEY;
if (!KEY) throw new Error("INFRAI_API_KEY is not set");

const MAX_BATCH = 100;
const FLUSH_MS = 250;

// key -> newest message. A later update for the same key REPLACES the earlier
// one: for a counter or a progress value, only the latest is meaningful, and
// sending the intermediate states costs money and renders nothing.
const pending = new Map();
let timer = null;

export function queueUpdate(key, channel, type, payload) {
  pending.set(key, { channel, event: "message.published", data: { type, ...payload } });
  if (!timer) timer = setTimeout(flush, FLUSH_MS);
}

export async function flush() {
  timer = null;
  if (pending.size === 0) return 0;
  const messages = [...pending.values()].slice(0, MAX_BATCH);
  for (const key of [...pending.keys()].slice(0, MAX_BATCH)) pending.delete(key);

  const published = await publishBatch(messages);
  if (pending.size > 0) timer = setTimeout(flush, FLUSH_MS);
  return published;
}

async function publishBatch(messages, attempt = 0) {
  const res = await fetch(`${API}/v1/realtime/publish/batch`, {
    method: "POST",
    headers: { authorization: `Bearer ${KEY}`, "content-type": "application/json" },
    body: JSON.stringify({ messages }),
  });
  const body = await res.json();
  if (body.ok) return body.data.published;

  const code = body.error?.code;
  const retryable = ["RATE_LIMIT_ACCOUNT", "RATE_LIMIT_USER", "RATE_LIMIT_VENDOR", "VENDOR_TIMEOUT", "NETWORK_ERROR"];
  if (!retryable.includes(code) || attempt >= 5) return 0;

  // Honour Retry-After when present; otherwise exponential with jitter, capped
  // at 45s so a single flush can never park the worker for minutes.
  const header = Number(res.headers.get("retry-after"));
  const delay = Number.isFinite(header) && header > 0
    ? header * 1000
    : Math.min(45_000, 2 ** attempt * 500) + Math.random() * 250;
  await new Promise((r) => setTimeout(r, delay));
  return publishBatch(messages, attempt + 1);
}

A 250 ms flush interval is four updates a second per key — smoother than any UI needs and a 99.9% reduction on a tight loop.

What to do when you’re limited anyway

import os
import random
import time

import requests

API = "https://api.infrai.cc"
KEY = os.environ["INFRAI_API_KEY"]
RETRYABLE = {"RATE_LIMIT_ACCOUNT", "RATE_LIMIT_USER", "RATE_LIMIT_VENDOR", "VENDOR_TIMEOUT", "NETWORK_ERROR"}
MAX_SLEEP = 45
SESSION = requests.Session()
SESSION.headers.update({"Authorization": f"Bearer {KEY}", "Content-Type": "application/json"})


def publish_batch(messages: list[dict], attempts: int = 5) -> int:
    for attempt in range(attempts):
        resp = SESSION.post(f"{API}/v1/realtime/publish/batch",
                            json={"messages": messages}, timeout=20)
        body = resp.json()
        if body.get("ok"):
            return body["data"]["published"]
        code = body.get("error", {}).get("code")
        if code not in RETRYABLE:
            raise RuntimeError(f"not retryable: {code}")
        retry_after = resp.headers.get("retry-after")
        delay = float(retry_after) if retry_after else min(MAX_SLEEP, 2 ** attempt)
        time.sleep(delay + random.uniform(0, 0.4))
    return 0


def chunked(items: list[dict], size: int = 100):
    for i in range(0, len(items), size):
        yield items[i:i + size]


if __name__ == "__main__":
    updates = [{"channel": "orders:live", "event": "message.published",
                "data": {"type": "order.created", "id": f"ord_{n}"}} for n in range(250)]
    total = sum(publish_batch(chunk) for chunk in chunked(updates))
    print(f"published {total}")

The 45-second ceiling is deliberate. Uncapped exponential backoff puts the fifth retry minutes away and the tenth beyond a quarter of an hour, which turns a rate limit into a worker that appears hung.

Separate the burst from the interactive path

An account-level limit is shared. A backfill publishing a million updates can starve the publishes your live users depend on, and the fix is not a bigger batch.

Two levers, both on the same key. Give the batch workload its own credential with POST /v1/account/keys/create so its consumption is visible and separable. And put the burst behind POST /v1/queue/publish so the consumer sets the pace instead of the producer — a worker draining a queue at a steady rate is inherently better behaved than one looping over a result set.

Limitations

There’s no delivery guarantee: publish reaches clients connected at that moment, and there’s no replay for one that was reconnecting. For a progress counter that’s fine by construction. For anything a user must not miss, keep a durable record they can fetch and treat the event as a hint.

Ably’s QoS levels and message history exist precisely for the cases where that isn’t acceptable, and if a missed message is a defect in your product rather than a stale pixel, it’s the better tool. Batch size is also bounded by the request itself rather than by a documented per-batch maximum, so chunk at a size you’ve tested rather than sending ten thousand in one body.

Publishing is the billable part — live per-call and per-batch rates in GET /v1/discovery/realtime.publish and GET /v1/discovery/realtime.publish.batch (verified 2026-09-21), with channel and token management reporting billing_class: free. Coalescing is the biggest cost lever available to you, and it’s larger than any rate change; those rates drift downward over time anyway, so read them live from your own account.

References

• Infrai realtime API reference
• RATE_LIMIT_ACCOUNT error reference
• Infrai documentation home