Cheap failure alerts for a Node/Express API: counters instead of log search

Log platforms bill on ingested volume, so alerting off log search gets expensive fast. Derive a counter at the log call site and poll it instead — with working Express code.

The cheapest way to get “tell me when my Express API starts failing” is not to search logs at all. Ship the logs wherever you like, but at the same call site where you write logger.error(...), increment a named counter and push it to Infrai’s POST /v1/metrics/batch. Alerting then reads a number over a time window instead of scanning text, and reading is free.

That inversion is the whole cost story. Log platforms price on bytes ingested and events indexed, which means your alerting bill scales with how chatty your app is — a debug log left on during an incident is a line item. A counter is 40 bytes regardless of how verbose the underlying error was, and Datadog or Grafana Loki are still there for the forensic search afterwards if you want them.

Where the money actually goes

Prometheus deserves a fair hearing here. If you already run Kubernetes and someone on the team knows PromQL, scraping /metrics off each pod is cheaper than any API and gives you real range queries. The reason small teams don’t do it is that “already run” is doing a lot of work in that sentence.

Counting errors where they happen

Express error middleware is the natural chokepoint — every unhandled route error and every next(err) passes through it exactly once.

The one design decision that matters is what you send. Don’t ship one metric point per failure; roll them up in memory first, keyed by the tag set, and ship one point per distinct combination per flush. A burst of 4,000 identical timeouts becomes a single point with value: 4000.

import express from "express";

const API = "https://api.infrai.cc";
const KEY = process.env.INFRAI_API_KEY;
if (!KEY) throw new Error("INFRAI_API_KEY is not set");

// key -> {tags, value}. Aggregating here is what keeps the bill flat during
// an incident: the point count depends on distinct tag sets, not on volume.
let bucket = new Map();

export function countFailure(route, status, code) {
  const tags = { route, status: String(status), code, service: "checkout-api" };
  const key = JSON.stringify(tags);
  const entry = bucket.get(key) ?? { tags, value: 0 };
  entry.value += 1;
  bucket.set(key, entry);
}

async function flush() {
  if (!bucket.size) return;                 // healthy minute: zero points, zero cost
  const current = bucket;
  bucket = new Map();                       // swap before the await, never after
  const points = [...current.values()].map(({ tags, value }) => ({
    name: "http.failure",
    value,
    type: "counter",
    tags,
  }));
  try {
    const res = await fetch(`${API}/v1/metrics/batch`, {
      method: "POST",
      headers: { authorization: `Bearer ${KEY}`, "content-type": "application/json" },
      body: JSON.stringify({ points }),
    });
    if (!res.ok) console.error(`metrics flush failed: HTTP ${res.status}`);
  } catch (err) {
    console.error("metrics flush threw", err);
  }
}
setInterval(flush, 60_000).unref();

const app = express();
app.get("/checkout", (_req, _res) => { throw new Error("payment gateway timeout"); });
app.use((err, req, res, _next) => {
  console.error({ msg: err.message, route: req.path });
  countFailure(req.path, 500, err.code ?? "unhandled");
  res.status(500).json({ error: "internal_error" });
});
app.listen(3000);

Note the map swap before the await. Without it a slow flush and a burst of errors race, and you either double-send points or drop them.

The raw shape of that batch call, if you want to try it before wiring the middleware:

export INFRAI_API_KEY="your_infrai_api_key"

curl -sS -X POST "https://api.infrai.cc/v1/metrics/batch" \
  -H "Authorization: Bearer ${INFRAI_API_KEY}" \
  -H "Content-Type: application/json" \
  -d '{
    "points": [
      {"name": "http.failure", "value": 7, "type": "counter",
       "tags": {"route": "/checkout", "status": "500", "service": "checkout-api"}},
      {"name": "http.request", "value": 812, "type": "counter",
       "tags": {"route": "/checkout", "service": "checkout-api"}}
    ]
  }'

{
  "ok": true,
  "data": { "accepted": 2 },
  "metadata": { "request_id": "req_d1084e5f140e4df0a4f56d9a", "latency_ms": 34 }
}

accepted is a count of points, not a boolean. Compare it against the length of what you sent and log the mismatch — it’s two lines and it turns a silent gap in your dashboard into something you find out about.

Polling for the alert

Reads are free, and the query route takes a time range, so the alert query is the same question you’d ask a human: how many failures in the last five minutes?

curl -sS "https://api.infrai.cc/v1/metrics/query?name=http.failure&agg=sum&window=5m&tag.service=checkout-api" \
  -H "Authorization: Bearer ${INFRAI_API_KEY}"

{
  "ok": true,
  "data": {
    "name": "http.failure",
    "agg": "sum",
    "points": [
      { "ts": "2026-07-26T09:45:00Z", "value": 3.0 },
      { "ts": "2026-07-26T09:50:00Z", "value": 41.0 }
    ]
  }
}

since and until take absolute ISO timestamps if you’d rather pin the range than roll it. An unknown agg is rejected with a 400 rather than quietly averaging something, which is worth knowing the first time you typo p95 for p99.

A watcher that pages a Slack incoming webhook, with no state file to keep:

const API = "https://api.infrai.cc";
const KEY = process.env.INFRAI_API_KEY;
const HOOK = process.env.SLACK_WEBHOOK_URL;
if (!KEY || !HOOK) throw new Error("set INFRAI_API_KEY and SLACK_WEBHOOK_URL");

const until = new Date();
const since = new Date(until.getTime() - 5 * 60_000);

const url = new URL(`${API}/v1/metrics/query`);
url.searchParams.set("name", "http.failure");
url.searchParams.set("agg", "sum");
url.searchParams.set("since", since.toISOString());
url.searchParams.set("until", until.toISOString());
url.searchParams.set("tag.service", "checkout-api");

const res = await fetch(url, { headers: { authorization: `Bearer ${KEY}` } });
if (!res.ok) {
  console.error(`query failed: HTTP ${res.status}`);
  process.exit(1);
}
const { data } = await res.json();
const failures = data.points.reduce((n, p) => n + p.value, 0);

if (failures >= 10) {
  await fetch(HOOK, {
    method: "POST",
    headers: { "content-type": "application/json" },
    body: JSON.stringify({ text: `checkout-api: ${failures} failures in the last 5 minutes` }),
  });
}
console.log(`failures=${failures}`);

Run it from cron with the key in the environment:

# /etc/cron.d/failure-watch — every five minutes
*/5 * * * * app INFRAI_API_KEY=your_infrai_api_key SLACK_WEBHOOK_URL=https://hooks.slack.com/services/T000/B000/xxxx /usr/bin/node /srv/app/failure-watch.mjs >> /var/log/failure-watch.log 2>&1

That script is stateless, which matters more than it looks: a watcher that keeps counters on disk gives you a wrong answer after every deploy that wipes the box.

The arithmetic, and where it stops being cheap

Reads are free. Writes — POST /v1/metrics/report and POST /v1/metrics/batch — are $0.001 per metric point, verified 2026-07-26. Batching many points into one request saves round trips and connection overhead, not money; what saves money is the in-memory rollup above, because it decides how many points exist at all.

Run the number yourself rather than trusting a date-stamped paragraph:

curl -sS "https://api.infrai.cc/v1/discovery" \
  -H "Authorization: Bearer ${INFRAI_API_KEY}" \
  | jq '.capabilities[] | select(.id | startswith("metrics.")) | {id, path, price: .billing.price_usd, unit: .billing.unit, free: .billing.free}'

Do the multiplication before you commit. One instance emitting three distinct tag sets a minute is 129,600 points in a 30-day month, or about $130 — which is what tells you that a route tag on a REST API with sixty paths is a mistake, and that a healthy service should be emitting almost nothing. Bucket the route into a handful of classes, drop tags you’ll never filter on, and the same instance costs a couple of dollars. New accounts get $2 in credit to test the shape, and rates on this platform have trended downward, so what you read today may be lower than what’s written here.

The trade-off you’re accepting

You cannot ask a counter a question you didn’t think of in advance. “Show me every 502 from last Tuesday that mentioned this order id” is a log search problem, and no amount of tagging replaces it — that’s the drawback of the whole approach, not just of one vendor. Keep your logs somewhere greppable, even if it’s just files on disk with logrotate, or push them to POST /v1/logs/ingest on the same key and search them with since/until when you need the text.

Two more limits worth knowing. Nothing here evaluates a rule for you: there’s no server-side alerting, no escalation policy, no on-call rota, so if the box running the cron dies your alerting dies quietly with it — Grafana or a Datadog monitor with PagerDuty behind it solves that properly, and it’s worth buying once more than one person carries a pager. And GET /v1/metrics/query is rate-limited per account; a watcher hammering it every second will start seeing RATE_LIMIT_ACCOUNT with Retry-After on the response, while a five-minute cron sits comfortably inside the envelope.

What tips the decision for small teams isn’t the per-point price. It’s that the same key already covers error capture, log search, cron, queues and object storage, so the day your alert wants to attach a stack trace or file an artefact, you’re not opening another account.

References

• https://docs.infrai.cc/en/api/metrics/
• https://docs.infrai.cc/errors/RATE_LIMIT_ACCOUNT/
• https://betterstack.com/community/guides/logging/best-nodejs-logging-libraries/
• https://last9.io/blog/node-js-logging-libraries/