Sweeping thousands of stale temp objects with batched delete calls

A per-object delete loop is the wrong tool. Batch up to 1000 keys per call, page the bucket with a cursor, then stop the mess recurring with a lifecycle rule.

Stop looping. Infrai exposes POST /v1/storage/object/delete_batch/{bucket}, which takes an array of keys and removes up to 1000 of them in one request — so four thousand stale temp files is four calls, not four thousand. It’s a free route, the same as object/list, so the whole sweep costs nothing but a few seconds of wall clock. Then set a lifecycle rule so you never have to do it again.

Here’s the shape of it, against a real bucket:

export INFRAI_API_KEY="your_infrai_api_key"

curl -sS -X POST "https://api.infrai.cc/v1/storage/object/delete_batch/kb-tmpsweep-0726" \
  -H "Authorization: Bearer ${INFRAI_API_KEY}" \
  -H "Content-Type: application/json" \
  -d '{"keys":["tmp/2026-05/job_1.tmp","tmp/2026-05/job_2.tmp"]}'

{
  "ok": true,
  "data": {
    "deleted": ["tmp/2026-05/job_1.tmp", "tmp/2026-05/job_2.tmp"],
    "errors": []
  }
}

The cap is 1000, and it fails hard

We pushed this to find the edge. A batch of exactly 1000 keys is accepted; 1001 comes back as a whole-request rejection with an unhelpfully empty key field:

{
  "ok": true,
  "data": {
    "deleted": [],
    "errors": [{ "key": "", "code": "INVALID_ARGUMENT" }]
  }
}

Note the ok: true. The HTTP status is 200 and the envelope says success, because the batch endpoint reports per-key outcomes rather than failing the request — which means a sweeper that only checks res.ok will report “4000 deleted” while having deleted nothing. Read data.errors, always, and chunk at 1000 or below. We use 500 in the code below to keep individual requests under a couple of seconds; a full 1000-key call took roughly 14 seconds in our testing against a bucket in eu-central-1.

Paging the bucket to build the key list

GET /v1/storage/object/list/{bucket} is cursor-paginated. Scope it with prefix, cap the page with limit, and follow next_cursor until it’s null — the cursor is simply the last key returned, so it’s stable across runs.

curl -sS "https://api.infrai.cc/v1/storage/object/list/kb-tmpsweep-0726?prefix=tmp/&limit=2" \
  -H "Authorization: Bearer ${INFRAI_API_KEY}"

{
  "ok": true,
  "data": {
    "items": [
      {
        "bucket_id": "bkt_c64065d7f55a4e7e9d91ad",
        "key": "tmp/2026-05/job_3.tmp",
        "size_bytes": 5,
        "etag": "188a0ff1e830261d0253b185a1c0f94f",
        "content_type": null,
        "metadata": null,
        "last_modified": "2026-07-26T00:57:51Z"
      }
    ],
    "next_cursor": "tmp/2026-05/job_4.tmp"
  }
}

Two things about that payload matter for a sweeper. content_type comes back null on list rows even when the object has one recorded — if your delete rule depends on MIME type you’ll need a head per key, which is free but serial and slow. And last_modified is the only age signal available, so “stale” has to mean “not written recently”, not “not read recently”.

The sweeper

Complete script. It pages the prefix, keeps keys older than a cutoff, deletes them in chunks, and refuses to proceed if any chunk reports a per-key error.

const API = "https://api.infrai.cc";
const TOKEN = process.env.INFRAI_API_KEY;
if (!TOKEN) throw new Error("INFRAI_API_KEY is not set");

const BUCKET = "kb-tmpsweep-0726";
const PREFIX = "tmp/";
const MAX_AGE_DAYS = 7;
const CHUNK = 500;

async function listPage(cursor) {
  const q = new URLSearchParams({ prefix: PREFIX, limit: "1000" });
  if (cursor) q.set("cursor", cursor);
  const res = await fetch(`${API}/v1/storage/object/list/${BUCKET}?${q}`, {
    method: "GET",
    headers: { Authorization: `Bearer ${TOKEN}` },
  });
  const json = await res.json();
  if (!res.ok || json.ok !== true) throw new Error(`list: HTTP ${res.status} ${JSON.stringify(json.error ?? json)}`);
  return json.data;
}

async function deleteChunk(keys) {
  const res = await fetch(`${API}/v1/storage/object/delete_batch/${BUCKET}`, {
    method: "POST",
    headers: { Authorization: `Bearer ${TOKEN}`, "Content-Type": "application/json" },
    body: JSON.stringify({ keys }),
  });
  const json = await res.json();
  if (!res.ok || json.ok !== true) throw new Error(`delete_batch: HTTP ${res.status}`);
  return json.data;
}

const cutoff = Date.now() - MAX_AGE_DAYS * 86400_000;
const doomed = [];
let cursor = null;
do {
  const page = await listPage(cursor);
  for (const o of page.items) {
    if (Date.parse(o.last_modified) < cutoff) doomed.push(o.key);
  }
  cursor = page.next_cursor;
} while (cursor);

console.log(`${doomed.length} objects older than ${MAX_AGE_DAYS} days under ${PREFIX}`);
if (process.env.DRY_RUN !== "0") {
  console.log("DRY_RUN — set DRY_RUN=0 to actually delete");
  process.exit(0);
}

let removed = 0;
for (let i = 0; i < doomed.length; i += CHUNK) {
  const chunk = doomed.slice(i, i + CHUNK);
  const out = await deleteChunk(chunk);
  removed += out.deleted.length;
  if (out.errors.length > 0) {
    console.error(`chunk ${i / CHUNK}: ${out.errors.length} errors, first=${JSON.stringify(out.errors[0])}`);
    process.exit(1);
  }
}
console.log(`removed ${removed} objects`);

The dry run defaulting to on is deliberate. A prefix typo in a delete script is the classic way to turn a cleanup task into an incident, and printing the count first costs one extra run of a free endpoint.

Deleting a key that isn’t there isn’t an error worth panicking about — it comes back in errors as STORAGE_OBJECT_NOT_FOUND while the rest of the batch succeeds. That makes retries safe: rerun a failed sweep and the already-deleted keys report not-found, the survivors go away, and nothing double-charges.

Confirm the bucket actually shrank

curl -sS "https://api.infrai.cc/v1/storage/bucket/usage/kb-tmpsweep-0726" \
  -H "Authorization: Bearer ${INFRAI_API_KEY}"

{
  "ok": true,
  "data": {
    "byte_count": 25,
    "object_count": 5,
    "as_of": "2026-07-26T00:59:50.363834Z"
  }
}

Object count is the number to watch, since it’s what your storage bill is partly a function of, and it settles within a second or two of the delete.

Better: never accumulate them again

A sweep is a fix for the backlog. The fix for the pattern is a lifecycle rule that expires the tmp/ prefix on the platform’s own schedule, at no cost and with no cron of yours to babysit:

curl -sS -X POST "https://api.infrai.cc/v1/storage/bucket/set_lifecycle/kb-tmpsweep-0726" \
  -H "Authorization: Bearer ${INFRAI_API_KEY}" \
  -H "Content-Type: application/json" \
  -d '{"rules":[{"prefix":"tmp/","expire_days":1},{"prefix":"exports/","expire_days":7}]}'

The catch is that this call replaces the bucket’s entire rule set rather than merging into it. Send every rule you want every time — we watched a two-rule configuration vanish on a live bucket because a later call carried only one rule — and read the result back with GET /v1/storage/bucket/get/{bucket} to confirm what stuck.

What the sweep costs

Verified 2026-07-26: object/list, object/delete, object/delete_batch, bucket/usage and set_lifecycle are all free per call on Infrai, rate-limited rather than metered, and they don’t consume the $2 credit new accounts start with. Writes are where the meter runs — object/put at $0.0001 per call, reads at $0.0002, roughly double. Storage and egress are billed by GB on top. Prices in this business move down and campaigns run, so treat the figures above as a floor to check rather than a promise:

curl -sS "https://api.infrai.cc/v1/discovery" \
  -H "Authorization: Bearer ${INFRAI_API_KEY}" \
  | python3 -c "import json,sys; d=json.load(sys.stdin); print([(c['id'], c['billing'].get('price_usd','free')) for c in d['capabilities'] if c['id'].startswith('storage.object')])"

Where this falls short

There’s no server-side “delete everything under this prefix” primitive, so you pay for a full listing pass before you can delete — with a million objects that’s a thousand list pages before the first key goes. There’s no object versioning either, which means a batch delete is final; no recycle bin, no restore. And per-key errors ride inside a 200 response, which is friendly to partial success and hostile to naive error handling.

If your objects already live in S3, aws s3 rm s3://bucket/tmp/ --recursive wraps the same 1000-key DeleteObjects call with local parallelism and is a perfectly good tool for a one-off; for tens of millions of objects, an S3 Batch Operations job driven by an inventory manifest beats anything you’ll script by hand. Self-hosting MinIO gets you mc rm --recursive --force and full control of the retention policy, at the price of running the storage yourself.

What Infrai adds here isn’t a faster delete — it’s that the cron entry firing this sweep weekly, the alert when a chunk reports errors, and the bucket itself are one account, one key and one bill. If object storage is the only piece of infrastructure you need, a dedicated provider is cheaper and you should use one.

References

• Infrai storage API reference
• Deleting multiple objects (Amazon S3 user guide)
• Node.js global fetch and URLSearchParams