Dashboard queries timing out over long ranges? Fix the shape

Why a 30-day service-health panel times out, which of the three usual causes you have, and how bounding the window and pre-aggregating at write time fixes it on Infrai.

A panel that renders instantly over six hours and times out over thirty days is telling you something specific: the query does work proportional to the range, and nobody capped it. Raising the timeout buys a week. The durable fix is to decide the aggregation before the data lands and to bound every read explicitly — and on Infrai both halves are available, because GET /v1/metrics/query takes a from/to window and returns the aggregate for exactly that window.

Bounding the read is the easy half. The half that actually decides whether your board survives a year of growth happens at the write, and that’s where most of this article goes.

The three reasons long-range panels die

Diagnosing which one you have takes five minutes. Halve the range: if latency halves too, it’s downsampling. Drop one tag: if it gets dramatically faster, it’s cardinality. If neither moves the needle, look at the matcher.

Pagination, which people reach for next, isn’t the answer to any of the three. Paging a 30-day scan still scans 30 days.

One read, one window

export INFRAI_API_KEY="your_infrai_api_key"

curl -sS -G -X GET "https://api.infrai.cc/v1/metrics/query" \
  -H "Authorization: Bearer ${INFRAI_API_KEY}" \
  --data-urlencode "name=svc.checkout.latency_ms" \
  --data-urlencode "agg=p99" \
  --data-urlencode "tag.svc=checkout" \
  --data-urlencode "from=2026-07-25T00:00:00Z" \
  --data-urlencode "to=2026-07-26T00:00:00Z"

{
  "ok": true,
  "data": {
    "name": "svc.checkout.latency_ms",
    "agg": "p99",
    "points": [{ "ts": "2026-07-26T00:00:00Z", "value": 514.0 }]
  }
}

The five aggregations are avg, sum, count, p50 and p99. Ask for something outside that set — max, say — and you get a 400 rather than a number, which is the behaviour you want on a dashboard: a typo in an aggregation name should break the panel loudly, not draw a plausible line.

Because the window is a parameter, a day tile and a 30-day tile cost the same call shape and differ only in how much data sits behind them. That’s what makes the board’s worst case predictable — you decide the range, rather than inheriting whatever the default retention happens to be.

The service-health board

Thirty daily buckets, three aggregations each, issued concurrently. Every one of them is a free read.

const API = "https://api.infrai.cc";
const KEY = process.env.INFRAI_API_KEY;
if (!KEY) throw new Error("INFRAI_API_KEY is not set");

const headers = { Authorization: `Bearer ${KEY}` };
const AGGS = new Set(["avg", "sum", "count", "p50", "p99"]);

async function agg(name, aggregate, { from, to, tags }) {
  if (!AGGS.has(aggregate)) throw new Error(`"${aggregate}" is not a supported aggregation`);
  const qs = new URLSearchParams({ name, agg: aggregate, from, to });
  for (const [k, v] of Object.entries(tags)) qs.set(`tag.${k}`, v);
  const res = await fetch(`${API}/v1/metrics/query?${qs}`, { headers });
  if (!res.ok) throw new Error(`${name} ${aggregate} -> HTTP ${res.status}`);
  const { data } = await res.json();
  return data.points[0]?.value ?? null;
}

const dayWindows = (n) =>
  Array.from({ length: n }, (_, i) => {
    const day = new Date(Date.now() - (n - 1 - i) * 86_400_000).toISOString().slice(0, 10);
    const from = `${day}T00:00:00Z`;
    const to = new Date(new Date(from).getTime() + 86_400_000).toISOString();
    return { day, from, to };
  });

export async function healthBoard(service, days = 30) {
  const rows = await Promise.all(dayWindows(days).map(async ({ day, from, to }) => {
    const window = { from, to, tags: { svc: service } };
    const [p50, p99, samples] = await Promise.all([
      agg("svc.checkout.latency_ms", "p50", window),
      agg("svc.checkout.latency_ms", "p99", window),
      agg("svc.checkout.latency_ms", "count", window),
    ]);
    return { day, p50, p99, samples: samples ?? 0 };
  }));
  return { service, days, rows: rows.filter((r) => r.samples > 0) };
}

console.log(JSON.stringify(await healthBoard("checkout", 3), null, 2));

Ninety small reads for a month of p50, p99 and volume, none of them touching more than a day of samples. A day with no traffic comes back with an empty points array, which the filter turns into an absent row rather than a zero — a fake zero on a latency chart is a lie, and it’s the one that makes people distrust the whole board.

Splitting the month into day windows rather than asking for one 30-day aggregate isn’t ceremony: you want thirty numbers to plot, and a single call over the whole range gives you one. Pick the bucket size your chart actually renders.

Where the slow call really is

Ingest is what you should be watching, and it’s measurable. In our testing a 200-point batch to POST /v1/metrics/batch came back in roughly 4 seconds, while pushing 1,000 points in a single request took about 15 — linear, and long enough to blow a default HTTP client timeout. So chunk it, with a timeout and a retry:

const API = "https://api.infrai.cc";
const KEY = process.env.INFRAI_API_KEY;
if (!KEY) throw new Error("INFRAI_API_KEY is not set");

const CHUNK = 200;

async function sendChunk(points, attempt = 1) {
  const ac = new AbortController();
  const timer = setTimeout(() => ac.abort(), 30_000);
  try {
    const res = await fetch(`${API}/v1/metrics/batch`, {
      method: "POST",
      headers: { Authorization: `Bearer ${KEY}`, "Content-Type": "application/json" },
      body: JSON.stringify({ points }),
      signal: ac.signal,
    });
    if (res.status === 429 || res.status >= 500) throw new Error(`retryable HTTP ${res.status}`);
    if (!res.ok) throw new Error(`batch -> HTTP ${res.status} ${await res.text()}`);
    const { data } = await res.json();
    if (data.accepted !== points.length) {
      console.warn(`accepted ${data.accepted}/${points.length} — reconcile before you trust the chart`);
    }
    return data.accepted;
  } catch (err) {
    if (attempt >= 4) throw err;
    await new Promise((r) => setTimeout(r, 500 * 2 ** attempt));
    return sendChunk(points, attempt + 1);
  } finally {
    clearTimeout(timer);
  }
}

export async function flush(points) {
  let accepted = 0;
  for (let i = 0; i < points.length; i += CHUNK) {
    accepted += await sendChunk(points.slice(i, i + CHUNK));
  }
  return accepted;
}

Comparing accepted against what you sent is cheap insurance. A partial acceptance that nobody reconciles shows up weeks later as a suspiciously smooth week on the chart.

{
  "ok": true,
  "data": { "accepted": 200 },
  "metadata": { "request_id": "req_a74e9ff1ec3b40e3a663fe20", "latency_ms": 3967 }
}

Then verify from the outside rather than trusting the writer. Sample count over today’s window is the honest check, because it goes up only when rows really landed:

curl -sS -G -X GET "https://api.infrai.cc/v1/metrics/query" \
  -H "Authorization: Bearer ${INFRAI_API_KEY}" \
  --data-urlencode "name=svc.checkout.latency_ms" \
  --data-urlencode "agg=count" \
  --data-urlencode "tag.svc=checkout" \
  --data-urlencode "from=2026-07-26T00:00:00Z" \
  --data-urlencode "to=2026-07-27T00:00:00Z"

The cost of doing it this way

Reads are free and rate-limited; writes are $0.001 per point, on POST /v1/metrics/report and POST /v1/metrics/batch alike, since batching is billed per point rather than per request. Batching saves round trips, not money. A service emitting one pre-aggregated latency sample per minute is 43,200 points a month, about $43 — which is the real argument for rolling up in your own process and pushing a summary every five minutes instead of a sample per request. New accounts get $2 of credit to test the shape first. Verified 2026-07-26:

curl -sS "https://api.infrai.cc/v1/account/usage" \
  -H "Authorization: Bearer ${INFRAI_API_KEY}"

Rates in this catalogue tend to move down and discounts run, so today’s number may be lower than what’s printed here.

When the answer is a different tool

The limitation worth flagging is expressiveness. There’s no query language here — no rate() over a window, no arithmetic between two series, no recording rules, no drag-select on a panel that re-issues an arbitrary step. You get five aggregations, a tag filter and a window. Prometheus plus Grafana is the better pick when exploration is the job and someone on the team is happy owning retention policy and storage; Datadog and CloudWatch hide the rollup machinery behind a UI, which is worth paying for once nobody wants to own it at all.

What you get in exchange is a board whose worst case is the same on day 1 and day 400, and one key that also covers the logs, the queue and the error tracker sitting next to it.

References

• Infrai metrics API reference
• Infrai documentation home
• Prometheus instrumentation practices
• Grafana: query and transform data