Alerting on repeats in Express: count deltas and resolve as your ack

Capture server exceptions from one Express handler, then poll for the two kinds of repetition that matter — a group still climbing, and a resolved group that came back.

Capturing exceptions is the easy half. The half that decides whether anyone trusts the setup is the alert rule, and “an error happened” is a terrible rule — a healthy Express API throws a few times an hour and nobody should hear about it. What you want to hear about is repetition: a fingerprint whose event count is climbing fast, or one you’d already marked fixed showing up again. Infrai’s errors namespace gives you both from free read routes, with POST /v1/errors/resolve/{error_group_id} doubling as your acknowledgement.

That last part is the trick worth stealing. There’s no ack field, no snooze, no “I’m on it” button in an API like this — but resolve is a flag you can set, and we measured that a new event with the same fingerprint flips it straight back to unresolved. Ack and un-ack, both server-side, no state of your own.

”Repeated” means two different things

Conflate the first and last rows and you get an alert channel that’s on fire permanently. The threshold isn’t a moral judgement, it’s arithmetic: pick events-per-minute, not events.

Capture, deliberately coarse

Rate arithmetic only works if the same failure lands in the same group. That means the fingerprint carries the route template and the error type — and nothing that varies per request.

// app.mjs — Express 5, Node 22.
import express from "express";

const app = express();
const CAPTURE = "https://api.infrai.cc/v1/errors/capture";

app.get("/orders/:id/charge", async (req, res) => {
  const outcome = await chargeOrder(req.params.id);
  res.json(outcome);
});

async function chargeOrder(id) {
  if (id === "8812") {
    const err = new TimeoutError("upstream payments.charge exceeded 8000ms");
    err.stack = `TimeoutError: ${err.message}\n    at chargeOrder (/app/src/orders/charge.js:74:9)`;
    throw err;
  }
  return { id, charged: true };
}

class TimeoutError extends Error {
  constructor(message) { super(message); this.name = "TimeoutError"; }
}

app.use(async (err, req, res, next) => {
  const key = process.env.INFRAI_API_KEY;
  const template = `${req.method} ${req.route?.path ?? req.path}`;
  res.status(502).json({ error: "upstream_failed" });
  if (!key) return;
  try {
    const r = await fetch(CAPTURE, {
      method: "POST",
      headers: { authorization: `Bearer ${key}`, "content-type": "application/json" },
      signal: AbortSignal.timeout(2000),
      body: JSON.stringify({
        message: err.stack ?? `${err.name}: ${err.message}`,
        exception: err.name,
        fingerprint: `${template}:${err.name}`,
        environment: process.env.NODE_ENV ?? "development",
        release: process.env.APP_RELEASE ?? "dev",
      }),
    });
    if (!r.ok) console.error("[capture] HTTP", r.status);
  } catch (e) {
    console.error("[capture] failed:", e.message);
  }
});

app.listen(3000, () => console.log("listening on :3000"));

req.route?.path is /orders/:id/charge, not /orders/8812/charge. Get that wrong and every order id opens its own group, every group has a count of 1, and no delta rule can ever fire — the alerting problem you’d then spend a week debugging is really a fingerprint problem.

Here’s the same event as a curl, so you can seed a group before writing the watcher:

export INFRAI_API_KEY="your_infrai_api_key"

curl -sS -X POST "https://api.infrai.cc/v1/errors/capture" \
  -H "Authorization: Bearer ${INFRAI_API_KEY}" \
  -H "Content-Type: application/json" \
  -d '{
    "message": "TimeoutError: upstream payments.charge exceeded 8000ms\n    at chargeOrder (/app/src/orders/charge.js:74:9)",
    "exception": "TimeoutError",
    "fingerprint": "POST /orders/:id/charge:TimeoutError",
    "environment": "production",
    "release": "api@2026.07.24"
  }'

Resolve, and what happens when it comes back

Resolving takes the group id from that response. The body is optional — an empty JSON object is fine — and the response isn’t a bare {ok}, it’s the whole updated group:

curl -sS -X POST "https://api.infrai.cc/v1/errors/resolve/{error_group_id}" \
  -H "Authorization: Bearer ${INFRAI_API_KEY}" \
  -H "Content-Type: application/json" \
  -d '{}'

Then we captured the same fingerprint again, from a later release, and re-read the group:

curl -sS "https://api.infrai.cc/v1/errors/group_detail/errgrp_xwHPx82iGJxmHC7RDzP76ly2" \
  -H "Authorization: Bearer ${INFRAI_API_KEY}" \
  | jq '{count: .data.count, resolved: .data.is_resolved, releases: .data.releases}'

{
  "count": 2,
  "resolved": false,
  "releases": ["api@2026.07.24", "api@2026.07.25"]
}

is_resolved went back to false on its own, and releases now lists both — the deploy you thought fixed it and the one where it reappeared. That two-element array is the most useful regression evidence this API produces, and you get it for free on every read.

Worth flagging: activity_log on the group came back empty even after a resolve, so “who acked this, and when” isn’t a question the API answers. If your process needs that audit trail, keep it on your side.

The watcher

// watch-repeats.mjs — run every 2 minutes.
import { readFile, writeFile } from "node:fs/promises";

const API = "https://api.infrai.cc";
const KEY = process.env.INFRAI_API_KEY;
const HOOK = process.env.SLACK_WEBHOOK_URL;
const SNAP = process.env.SNAPSHOT ?? "./counts.json";
const RATE_LIMIT_PER_MIN = Number(process.env.RATE_LIMIT_PER_MIN ?? 3);
if (!KEY || !HOOK) throw new Error("set INFRAI_API_KEY and SLACK_WEBHOOK_URL");

async function get(path) {
  const r = await fetch(`${API}${path}`, {
    headers: { Authorization: `Bearer ${KEY}` },
    signal: AbortSignal.timeout(10_000),
  });
  if (!r.ok) throw new Error(`${path}: HTTP ${r.status}`);
  return (await r.json()).data;
}

const prev = await readFile(SNAP, "utf8").then(JSON.parse).catch(() => ({}));
const now = Date.now();
const open = (await get("/v1/errors/groups?status=unresolved&limit=50")).groups ?? [];
const alerts = [];
const next = {};

for (const g of open) {
  const before = prev[g.error_group_id];
  next[g.error_group_id] = { count: g.count, at: now, acked: before?.acked ?? false };
  if (!before) continue;
  const minutes = Math.max((now - before.at) / 60_000, 0.5);
  const rate = (g.count - before.count) / minutes;
  if (before.acked && g.count > before.count) {
    alerts.push(`*regression* ${g.title.split("\n")[0].slice(0, 90)} — resolved, now back (${g.count} total)`);
    next[g.error_group_id].acked = false;
  } else if (rate >= RATE_LIMIT_PER_MIN) {
    alerts.push(`*burning* ${g.title.split("\n")[0].slice(0, 90)} — ${rate.toFixed(1)}/min, ${g.count} total`);
  }
}

if (alerts.length) {
  const r = await fetch(HOOK, {
    method: "POST",
    headers: { "Content-Type": "application/json" },
    body: JSON.stringify({ text: alerts.join("\n") }),
  });
  if (!r.ok) throw new Error(`slack: HTTP ${r.status}`);
}
await writeFile(SNAP, JSON.stringify(next), "utf8");
console.log(`${open.length} open, ${alerts.length} alerts`);

Mark acked: true in the snapshot at the moment you resolve a group — from your fix script, your deploy pipeline, wherever you call the resolve route — and the watcher turns any subsequent growth on that id into a regression alert rather than yet another “still burning” line. Two rules, one file, no rules engine.

Set RATE_LIMIT_PER_MIN from what your service actually does, not from a blog post. Three a minute is loud for a service handling 5 rps and silent for one handling 500.

# /etc/systemd/system/watch-repeats.timer
[Unit]
Description=Poll Infrai error groups for repeats

[Timer]
OnBootSec=2min
OnUnitActiveSec=2min

[Install]
WantedBy=timers.target

What this doesn’t do

There are no server-side alert rules here, so the threshold lives in your code and gets deployed like code — fine for one team, awkward for twenty. There’s no snooze, no escalation, no rotation, and no per-user acknowledgement, only the binary resolved flag. And the detection floor is your polling interval; nothing pushes.

Datadog monitors do this properly, with anomaly detection, composite conditions and a maintenance window, and if you’re already paying for Datadog you should use it rather than this. Sentry’s issue alerts sit closer to what’s described here, with “seen more than N times in M minutes” as a first-class rule and a UI for muting. Stick with the poller when the whole ops surface is one Node service and you’d rather own thirty lines than a rules editor.

What it costs to run

Reads — groups, group_detail, list, search — are free and rate-limited, so a two-minute poll costs nothing at any interval. Resolve is free too. Only capture is metered: $0.00005 per event, verified 2026-07-26, with $2 of free credit on a new account, which is around 39,999 events before you’re charged. Rates on this catalogue trend downward and campaigns run, so the live number may well be lower than the printed one:

curl -sS "https://api.infrai.cc/v1/account/usage" \
  -H "Authorization: Bearer ${INFRAI_API_KEY}" | jq '{period, total_cost}'

One capture per failed request is also the reason to keep the 4xx you caused out of the reporter — validation errors are traffic, not incidents. The structure to remember: metered writes, free reads and resolves, and the same credential already reaching cron, queues and email, so scheduling this watcher and mailing its output stay on one bill.

References

• Infrai errors API reference
• Express error handling guide
• Datadog metric monitors