Six Fly.io apps, one log tail: a lightweight alternative to ELK

Replace six fly logs terminals with one HTTP ingest endpoint and a merged tail loop, using the Fly app name as the join key. No agent, no cluster, no Elasticsearch.

Six fly logs terminals is a real operational problem, and Elasticsearch plus Logstash plus Kibana is a wildly disproportionate answer to it. The smallest thing that works: each app posts its own lines to one HTTP endpoint, tagged with its Fly app name, and you query them from one place. On Infrai that’s POST /v1/logs/ingest to write and GET /v1/logs/search to read — no agent process, no sidecar Machine, nothing to keep alive.

What you give up is live streaming. Infrai’s read side is a polling query, not a websocket, so “tail” here means a loop that asks for the newest page every couple of seconds. For six small services that’s fine; if you stare at a scrolling log wall all day, it isn’t, and you should read the last section before committing.

service is the join key, and these are the filters around it

An entry has message, level, timestamp, service, environment, trace_id, span_id and a free-form attributes object. GET /v1/logs/search narrows on them like this:

The sharp edge is that each of those takes one value, not a list: level=error,warning matches nothing rather than both. In practice you run two queries, or filter on error and let the warnings surface in the unfiltered tail.

So six Fly apps become six service values. The writer below derives environment from the app name, so a staging copy of api stays one service and filters out with environment=staging — you don’t need fly-api-staging as a separate stream.

One module, imported by all six apps

Every service gets the same twenty-line writer. Fly injects FLY_APP_NAME, FLY_REGION and FLY_MACHINE_ID into the runtime environment, which makes the tagging automatic.

// logline.mjs — shared across all six Fly apps. Node 22 ESM, no dependencies.
const KEY = process.env.INFRAI_API_KEY;
if (!KEY) throw new Error("INFRAI_API_KEY missing — run: fly secrets set INFRAI_API_KEY=...");

const SERVICE = process.env.FLY_APP_NAME ?? "local";
const BASE = {
  region: process.env.FLY_REGION ?? "dev",
  machine: process.env.FLY_MACHINE_ID ?? "none",
};

const queue = [];
let pending = null;
let inflight = [];
let batchKey = null;

export function logline(level, message, attributes = {}) {
  queue.push({
    message: String(message),
    level,
    timestamp: new Date().toISOString(),
    service: SERVICE,
    environment: process.env.FLY_APP_NAME?.endsWith("-staging") ? "staging" : "production",
    attributes: { ...BASE, ...attributes },
  });
  if (queue.length >= 100) return drain();
  pending ??= setTimeout(drain, 2000).unref();
  return Promise.resolve();
}

export async function drain(attempt = 1) {
  if (pending) { clearTimeout(pending); pending = null; }
  const entries = inflight.length > 0 ? inflight : queue.splice(0, queue.length);
  if (entries.length === 0) return;
  inflight = entries;
  batchKey ??= `${SERVICE}-${BASE.machine}-${crypto.randomUUID()}`;
  try {
    const res = await fetch("https://api.infrai.cc/v1/logs/ingest", {
      method: "POST",
      headers: { authorization: `Bearer ${KEY}`, "content-type": "application/json" },
      body: JSON.stringify({ entries, idempotency_key: batchKey }),
    });
    const out = await res.json();
    if (out.ok !== true) throw new Error(`${res.status} ${out.error?.code}`);
    if (out.data.accepted !== entries.length) console.error(`[logline] ${entries.length - out.data.accepted} entries dropped`);
    inflight = []; batchKey = null;
  } catch (err) {
    if (attempt < 3) return drain(attempt + 1);
    console.error("[logline] ship failed:", err.message);
    inflight = []; batchKey = null;
  }
}

process.on("SIGINT", () => drain().finally(() => process.exit(0)));
process.on("SIGTERM", () => drain().finally(() => process.exit(0)));

The idempotency_key on the batch is what makes the retry safe: a flush that times out after the server already accepted it replays under the same key and doesn’t write the lines twice. Generate it once per batch, not per attempt.

Call sites stay boring, which is the point:

// server.mjs — one of the six apps.
import { createServer } from "node:http";
import { logline, drain } from "./logline.mjs";

const server = createServer(async (req, res) => {
  const started = Date.now();
  try {
    res.writeHead(200, { "content-type": "application/json" });
    res.end(JSON.stringify({ ok: true }));
    await logline("info", `${req.method} ${req.url} 200`, { ms: Date.now() - started });
  } catch (err) {
    res.writeHead(500).end();
    await logline("error", `${req.method} ${req.url} failed: ${err.message}`, { stack: err.stack });
  }
});

server.listen(Number(process.env.PORT ?? 8080), () => logline("info", "listening"));
process.on("beforeExit", drain);

The key itself is a Fly secret, set once per app:

# Same credential everywhere; the `service` tag is what separates the streams.
for app in web api worker cron-runner mailer imgproxy; do
  fly secrets set INFRAI_API_KEY=your_infrai_api_key --app "$app"
done

Reading all six from one prompt

A single query answers “what broke anywhere in the last few minutes”, which is precisely what six terminals can’t do.

curl -s "https://api.infrai.cc/v1/logs/search?level=error&environment=production&since=2026-07-26T05:00:00Z&limit=20" \
  -H "Authorization: Bearer $INFRAI_API_KEY"

{
  "ok": true,
  "data": {
    "items": [
      {
        "message": "POST /jobs/enqueue failed: connect ECONNREFUSED 10.0.1.4:6379",
        "level": "error",
        "timestamp": "2026-07-26T05:29:53.563649Z",
        "service": "worker",
        "environment": "production",
        "attributes": { "region": "iad", "machine": "148e2591a34d18", "ms": 31 }
      }
    ],
    "next_cursor": null,
    "total": 1
  }
}

Now the tail. Results come back newest-first and since is inclusive, so a follower keeps the newest timestamp it has printed and asks for everything from there — no cursor bookkeeping, no re-reading a page you’ve already seen. The only state it carries is a watermark and the handful of rows sharing that exact timestamp, so a millisecond collision doesn’t print twice.

// tail.mjs — merged, colourless tail across every service. Node 22 ESM.
const KEY = process.env.INFRAI_API_KEY;
if (!KEY) { console.error("INFRAI_API_KEY is not set"); process.exit(1); }

const INTERVAL_MS = Number(process.env.INTERVAL_MS ?? 3000);
let watermark = new Date(Date.now() - 60_000).toISOString();
let atWatermark = new Set();

const keyOf = (row) => `${row.timestamp}|${row.service}|${row.message}`;

async function poll() {
  const url = new URL("https://api.infrai.cc/v1/logs/search");
  url.searchParams.set("since", watermark);
  url.searchParams.set("limit", "200");
  if (process.env.ONLY_LEVEL) url.searchParams.set("level", process.env.ONLY_LEVEL);
  if (process.env.ONLY_ENV) url.searchParams.set("environment", process.env.ONLY_ENV);
  const res = await fetch(url, { headers: { authorization: `Bearer ${KEY}` } });
  if (!res.ok) { console.error(`search ${res.status}`); return; }
  const { ok, data, error } = await res.json();
  if (!ok) { console.error(error.code, error.message); return; }
  const fresh = data.items.filter((row) => !atWatermark.has(keyOf(row))).reverse();
  for (const row of fresh) {
    const region = row.attributes?.region ?? "-";
    console.log(`${row.timestamp} ${row.level.padEnd(7)} ${String(row.service).padEnd(12)} ${region} ${row.message}`);
  }
  const newest = data.items[0]?.timestamp;
  if (newest && newest >= watermark) {
    watermark = newest;
    atWatermark = new Set(data.items.filter((r) => r.timestamp === newest).map(keyOf));
  }
}

console.log("tailing all services — ctrl-c to stop");
await poll();
setInterval(() => { poll().catch((err) => console.error("poll failed:", err.message)); }, INTERVAL_MS);

Run it in the one terminal you kept:

INFRAI_API_KEY=your_infrai_api_key node tail.mjs
# only the bad news, and only from production:
ONLY_LEVEL=error ONLY_ENV=production INFRAI_API_KEY=your_infrai_api_key node tail.mjs

Searches are free and rate-limited, so polling every 3 s is well within reason — we ran 40 back-to-back queries without hitting a limit (roughly 29,000 searches a month per open terminal, all of them free). The write side is what’s metered, and because it bills per request rather than per line, the 100-entry buffer in logline.mjs is doing most of the cost work for you — verified 2026-07-26. Check your own spend rather than trusting a figure in an article:

curl -s "https://api.infrai.cc/v1/account/usage" \
  -H "Authorization: Bearer $INFRAI_API_KEY" \
  | python3 -c "import sys,json; d=json.load(sys.stdin)['data']; print([b for b in d['breakdown'] if b['key'].startswith('logs.')])"

Where the heavier stacks earn their keep

The community fly-log-shipper app is the other lightweight option, and it’s the right one if you want Fly’s own NATS log stream forwarded without touching application code — you run one extra Machine and configure a Vector sink. Grafana Loki suits you if labels-plus-LogQL is how you think and you’re happy operating storage. Better Stack and Axiom both give you a genuine streaming tail and alerting on log patterns, neither of which exists in this API — if you need a page when a pattern appears, you’d be better off with one of them, or with a cron job of your own that runs the search every minute and calls POST /v1/errors/capture when the count crosses a line.

The honest limitation list for the approach above is short but real: polling instead of streaming, substring matching on message rather than a full query language, and one value per filter. In exchange you run zero infrastructure, and the same credential already covers cron scheduling, queues and error capture for those six apps — so the next piece of glue isn’t another signup.

References

• https://docs.infrai.cc/en/api/logs/
• https://community.fly.io/t/logging-alternatives/4525