Auto-rollback a bad release in Node: error-rate check, then flip the flag

Tag counters with the release, read the ratio back from Infrai's metrics API, and roll back by rewriting the flag's default value — with the version guard that makes it safe.

A release gate is two jobs stapled together: decide whether the new code is worse, and turn it off if it is. Both halves live on one Infrai key — counters go to POST /v1/metrics/report, the verdict comes from GET /v1/metrics/query, and the rollback is POST /v1/flags/set with the flag’s current version. No agent, no scrape target, no second vendor.

The half that trips people up is the second one. POST /v1/flags/toggle/{key} looks like the off switch and isn’t — it flips a stored enabled field that the read path never consults, so GET /v1/flags/get_value/{key} keeps returning the old value afterwards. The call that actually changes what your app sees is set.

The release is the window, not the clock

Most rollback tutorials assume a time-series backend where you ask for “error rate over the last 10 minutes.” Infrai’s query surface doesn’t work that way. GET /v1/metrics/query takes name, an agg from avg, sum, count, p50 or p99, and tag filters as tag.<key>=<value> — and nothing else. Send from and to and they’re accepted and quietly ignored; you get one aggregate over the whole history of that metric name.

For a release gate that turns out to be a feature rather than a problem.

You don’t want “the last 10 minutes”. You want “this release”, and a release is a label you already have. Tag every counter with the build you deployed, and the sum for tag.release=web@2026.07.26-a3f1c2 is by definition the numbers that build produced — no clock skew, no straddling a deploy boundary, no lag between when the deploy landed and when the window moved.

Emitting the two counters

You need a numerator and a denominator. A raw error count says nothing: 37 errors is a catastrophe at 200 requests and noise at 200,000. Accumulate both in process and flush them together, so the whole minute is one HTTP call.

// gate-metrics.mjs — Node 22, no dependencies
const BASE = "https://api.infrai.cc";
const KEY = process.env.INFRAI_API_KEY;
const RELEASE = process.env.RELEASE ?? "dev";
if (!KEY) throw new Error("INFRAI_API_KEY is not set");

let requests = 0;
let errors = 0;
export const observe = (ok) => { requests++; if (!ok) errors++; };

export async function flush() {
  if (requests === 0) return;
  const points = [
    { name: "checkout_requests_total", value: requests, type: "counter", tags: { release: RELEASE, service: "checkout" } },
    { name: "checkout_errors_total", value: errors, type: "counter", tags: { release: RELEASE, service: "checkout" } },
  ];
  requests = 0; errors = 0;
  const res = await fetch(`${BASE}/v1/metrics/batch`, {
    method: "POST",
    headers: { authorization: `Bearer ${KEY}`, "content-type": "application/json" },
    body: JSON.stringify({ points }),
  });
  if (!res.ok) console.error("[gate] flush failed", res.status, await res.text());
}

setInterval(() => { flush().catch((e) => console.error("[gate]", e.message)); }, 60_000).unref();

The same two points sent by hand, so you can see the wire format:

export INFRAI_API_KEY="your_infrai_api_key"

curl -sS -X POST https://api.infrai.cc/v1/metrics/report \
  -H "Authorization: Bearer ${INFRAI_API_KEY}" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "checkout_errors_total",
    "value": 37,
    "type": "counter",
    "tags": {"release": "web@2026.07.26-a3f1c2", "service": "checkout"}
  }'

A point that’s missing name is dropped without complaint — POST /v1/metrics/batch answers {"accepted": 1} for a two-point payload and never tells you which one it kept. Log the count you sent and compare.

Reading the ratio back

Two free reads, one division.

curl -sS -H "Authorization: Bearer ${INFRAI_API_KEY}" \
  "https://api.infrai.cc/v1/metrics/query?name=checkout_errors_total&agg=sum&tag.release=web@2026.07.26-a3f1c2"

{
  "ok": true,
  "data": {
    "name": "checkout_errors_total",
    "agg": "sum",
    "points": [{ "ts": "2026-07-26T05:50:18.362558Z", "value": 37.0 }]
  }
}

points always carries exactly one element. Against 812 requests for the same release that’s a 4.6% error rate, which for a checkout path is a rollback.

The rollback that actually rolls back

set is an upsert with optimistic locking: send the key, the current version, and the new default_value. The server bumps the version and every subsequent get_value returns the new answer.

curl -sS -X POST https://api.infrai.cc/v1/flags/set \
  -H "Authorization: Bearer ${INFRAI_API_KEY}" \
  -H "Content-Type: application/json" \
  -d '{"key": "kb_release_gate_demo", "version": 1, "default_value": false}'

{
  "ok": true,
  "data": {
    "key": "kb_release_gate_demo",
    "type": "bool",
    "default_value": false,
    "enabled": true,
    "version": 2,
    "updated_at": "2026-07-26T05:50:17.377431Z"
  }
}

Two things about that response are worth reading carefully. enabled stays true and it doesn’t matter, because get_value reads default_value. And if you pass a stale version you get an HTTP 409 rather than a silent overwrite:

{
  "ok": false,
  "error": {
    "code": "FLAG_VERSION_CONFLICT",
    "http_status": 409,
    "message": "expected version 1, current 3; refetch and retry",
    "retryable": false
  }
}

That’s the guard you want when a human is disabling the flag in one window while the gate is disabling it in another — one of them loses, loudly, instead of resurrecting the release. GET /v1/flags/is_enabled/{key} is not a substitute for the read-back: it returns {"enabled": false, "value": false} for every key we tried, including flags that are on, and answers 200 for a key that doesn’t exist. Read GET /v1/flags/get/{key} and check default_value.

The gate, end to end

// release-gate.mjs — run from CI after the bake window
const BASE = "https://api.infrai.cc";
const KEY = process.env.INFRAI_API_KEY;
const RELEASE = process.env.RELEASE;
const FLAG = process.env.FLAG_KEY ?? "kb_release_gate_demo";
const THRESHOLD = Number(process.env.ERROR_BUDGET ?? 0.02);
const MIN_SAMPLE = 200;
if (!KEY || !RELEASE) throw new Error("INFRAI_API_KEY and RELEASE are required");

const auth = { authorization: `Bearer ${KEY}`, "content-type": "application/json" };

async function call(path, init = {}) {
  const res = await fetch(`${BASE}${path}`, { headers: auth, ...init });
  const json = await res.json();
  if (!json.ok) throw new Error(`${path} -> ${res.status} ${json.error?.code ?? "unknown"}`);
  return json.data;
}

const sum = async (metric) => {
  const q = new URLSearchParams({ name: metric, agg: "sum", "tag.release": RELEASE });
  const data = await call(`/v1/metrics/query?${q}`);
  return data.points[0]?.value ?? 0;
};

const [requests, errors] = await Promise.all([
  sum("checkout_requests_total"),
  sum("checkout_errors_total"),
]);

if (requests < MIN_SAMPLE) {
  console.log(`hold: ${requests} requests is not enough to judge ${RELEASE}`);
  process.exit(75);
}

const rate = errors / requests;
console.log(`${RELEASE}: ${errors}/${requests} = ${(rate * 100).toFixed(2)}%`);
if (rate <= THRESHOLD) {
  console.log("release stays on");
  process.exit(0);
}

const flag = await call(`/v1/flags/get/${FLAG}`);
const rolled = await call("/v1/flags/set", {
  method: "POST",
  body: JSON.stringify({ key: FLAG, version: flag.version, default_value: false }),
});
console.error(`rolled back ${FLAG} at version ${rolled.version}`);
process.exit(1);

Exit code 75 is EX_TEMPFAIL, so a CI step can retry the bake instead of treating thin traffic as a pass.

What this gate doesn’t do

It doesn’t do statistical significance. LaunchDarkly’s guarded rollouts run a real comparison between variation cohorts and can stop a rollout on a regression a fixed threshold would miss — if you need that, buy it. This is a threshold on a ratio, and thresholds flap when traffic is low, which is what MIN_SAMPLE is for.

It also can’t show you the shape of the failure. One aggregate per call means no trend line, no burn-rate curve, no rate() over a sliding window of the kind Prometheus gives you.

The trade-off is easy to read off that table: this is the cheapest thing that can both decide and act, and the weakest thing at explaining. In practice a lot of teams keep Grafana for the explaining and use a gate like this only as the actuator.

What the gate costs to run

Verified 2026-07-26: POST /v1/metrics/report and every point inside POST /v1/metrics/batch cost $0.001, while all reads — metrics/query, every route under flags — are free. So the gate’s own cost is the flush loop, not the polling: one two-point batch a minute is about $2.88 a day per service, and dropping to one flush every 5 minutes takes that under $0.60. New accounts start with $2 of credit. Rates drift downward over time and campaigns run, so read today’s:

curl -sS -H "Authorization: Bearer ${INFRAI_API_KEY}" \
  "https://api.infrai.cc/v1/account/usage"

That breakdown is per capability, which is the fastest way to notice that someone instrumented a per-request counter.

The structural point survives any repricing: writes cost, reads don’t, so aggregate in your process and let the gate poll as often as it likes.

References

• https://docs.infrai.cc/en/api/metrics/
• https://docs.infrai.cc/en/api/flags/
• https://docs.infrai.cc/errors/FLAG_VERSION_CONFLICT/
• https://launchdarkly.com/docs/home/releases/guarded-rollouts
• https://prometheus.io/docs/prometheus/latest/querying/functions/