Document retention on object storage: prefixes, expiry rules, erasure

Choosing retention per document class, the one-day floor on expiry rules, why the rule set is replaced rather than merged, and what to do about erasure requests.

Retention is a filing decision before it’s an API call. Group documents by who is allowed to delete them — the user, the clock, or nobody — give each group its own key prefix, and attach an expiry rule to the prefixes the clock owns. Infrai’s storage namespace does this with one free call per bucket, and the rules are evaluated by the vendor rather than by a job you have to keep alive.

The trap is that an expiry rule is not an erasure mechanism. It’s a broom.

Sort by who owns the deletion

Three classes cover most SaaS document storage, and they want different treatment. User-uploaded originals are the user’s to delete, so a rule that expires them silently is a support ticket. Machine-generated exports are disposable the moment the download link goes cold. Backups have a compliance number attached and shouldn’t be reachable by the same code path that serves the app.

Splitting backups into their own bucket isn’t fussiness. It means a mistaken rule on the app bucket can’t reach them, and the bucket-level usage figure becomes a real signal instead of a blend.

The rule set is a document, not a list you append to

One call writes retention for a whole bucket:

export INFRAI_API_KEY="your_infrai_api_key"

curl -sS -X POST "https://api.infrai.cc/v1/storage/bucket/set_lifecycle/docvault-2026" \
  -H "Authorization: Bearer ${INFRAI_API_KEY}" \
  -H "Content-Type: application/json" \
  -d '{"rules":[{"prefix":"tmp/","expire_days":1},{"prefix":"exports/","expire_days":7}]}'

{
  "ok": true,
  "data": {
    "bucket_id": "bkt_b971926ffeee4fdfa90177",
    "name": "docvault-2026",
    "acl": "private",
    "cors_rules": [],
    "lifecycle_rules": [
      { "prefix": "tmp/", "expire_days": 1 },
      { "prefix": "exports/", "expire_days": 7 }
    ]
  }
}

Send a single rule next month and the other rules are gone — the array you post becomes the array the bucket has. So never call it with a literal; read the current rules, merge, write back, and log what changed. This is the helper worth having in your repo on day one, in Node 22:

const KEY = process.env.INFRAI_API_KEY;
if (!KEY) throw new Error("INFRAI_API_KEY is not set");
const BASE = "https://api.infrai.cc";

async function call(path, init = {}) {
  const res = await fetch(`${BASE}${path}`, {
    ...init,
    headers: { Authorization: `Bearer ${KEY}`, "Content-Type": "application/json", ...(init.headers ?? {}) },
  });
  const json = await res.json();
  if (!json.ok) throw new Error(`${path} -> ${json.error?.code}: ${json.error?.message}`);
  return json.data;
}

async function upsertRule(bucket, prefix, expireDays) {
  const current = await call(`/v1/storage/bucket/get/${bucket}`, { method: "GET" });
  const rules = (current.lifecycle_rules ?? []).filter((r) => r.prefix !== prefix);
  rules.push({ prefix, expire_days: expireDays });
  const updated = await call(`/v1/storage/bucket/set_lifecycle/${bucket}`, {
    method: "POST",
    body: JSON.stringify({ rules }),
  });
  console.log("lifecycle now:", JSON.stringify(updated.lifecycle_rules));
  return updated.lifecycle_rules;
}

await upsertRule("docvault-2026", "exports/", 7);

One day is the floor

expire_days is a whole number of days and one is the smallest accepted value. If your product promises a download link that dies in 30 minutes, the rule can’t deliver it — you need a short presign window for the link and your own sweeper for the bytes. That’s the same constraint S3 imposes, and the reason “expires in 15 minutes” almost always means the URL rather than the object.

Deletion also isn’t instant at the boundary. A rule with expire_days: 1 removes objects roughly a day after they were written, not at midnight sharp, so don’t build a test that asserts a precise second.

An erasure request can’t wait for the broom

When a user asks for their documents to be removed, the clock is the wrong instrument. List the prefix, delete the batch, then verify — all three routes are free, so a full-account purge costs nothing but the time it takes to page through keys.

import os
import sys
import requests

BASE = "https://api.infrai.cc"
KEY = os.environ.get("INFRAI_API_KEY")
if not KEY:
    sys.exit("INFRAI_API_KEY is not set")
HEADERS = {"Authorization": f"Bearer {KEY}", "Content-Type": "application/json"}


def list_prefix(bucket, prefix):
    keys, cursor = [], None
    while True:
        params = {"prefix": prefix}
        if cursor:
            params["cursor"] = cursor
        r = requests.get(f"{BASE}/v1/storage/object/list/{bucket}", headers=HEADERS, params=params, timeout=30)
        body = r.json()
        if not body.get("ok"):
            sys.exit(f"list failed: {body.get('error')}")
        keys.extend(o["key"] for o in body["data"]["items"])
        cursor = body["data"].get("next_cursor")
        if not cursor:
            return keys


def erase(bucket, user_id):
    keys = []
    for prefix in (f"docs/{user_id}/", f"exports/{user_id}/", f"tmp/{user_id}/"):
        keys.extend(list_prefix(bucket, prefix))
    if not keys:
        print("nothing to erase")
        return
    for chunk_start in range(0, len(keys), 100):
        chunk = keys[chunk_start:chunk_start + 100]
        r = requests.post(
            f"{BASE}/v1/storage/object/delete_batch/{bucket}",
            headers=HEADERS,
            json={"keys": chunk},
            timeout=60,
        )
        data = r.json()
        if not data.get("ok"):
            sys.exit(f"delete_batch failed: {data.get('error')}")
        print(f"deleted {len(data['data']['deleted'])}, errors {len(data['data']['errors'])}")


erase("docvault-2026", "usr_301")

Keys that were already gone come back in errors with STORAGE_OBJECT_NOT_FOUND rather than failing the whole batch, so the script is safe to re-run after a timeout.

Proving the policy did something

Auditors and support engineers ask the same question — is it actually gone? Two free reads answer it:

curl -sS "https://api.infrai.cc/v1/storage/object/list/docvault-2026?prefix=exports/" \
  -H "Authorization: Bearer ${INFRAI_API_KEY}"

curl -sS "https://api.infrai.cc/v1/storage/bucket/usage/docvault-2026" \
  -H "Authorization: Bearer ${INFRAI_API_KEY}"

{
  "ok": true,
  "data": { "byte_count": 21, "object_count": 1, "as_of": "2026-07-26T01:02:11Z" }
}

Chart object_count per bucket weekly and a retention rule that silently stopped applying shows up as a line that stops falling.

What retention here doesn’t cover

There’s no object versioning, no object lock, and no legal hold. Anything with write-once-read-many in the requirement — financial records under SEC 17a-4, evidence preservation, a regulator’s word “immutable” — is not a good fit for this surface, and you’d be better off with S3 Object Lock in compliance mode, or Backblaze B2 and Wasabi, both of which expose object lock over the S3 API. Also note that a bucket’s stated region is metadata: our buckets recorded eu-central-1 and still served from an Asia-Pacific host, so contractual data-residency belongs elsewhere too.

What you do get is a retention policy that lives beside everything else: the same key runs the cron that triggers the erasure job, records the audit entry, and mails the user their confirmation.

The cost of keeping things

Lifecycle rules, listings, head and deletes are free; writes and reads bill per call; stored bytes accrue rent per GB-month, which is the line that grows if a rule quietly stops matching. Check both together:

curl -sS "https://api.infrai.cc/v1/discovery" \
  -H "Authorization: Bearer ${INFRAI_API_KEY}" | grep -o '"storage.bucket.set_lifecycle[^}]*}'

curl -sS "https://api.infrai.cc/v1/account/usage" \
  -H "Authorization: Bearer ${INFRAI_API_KEY}"

New accounts get $2 free credit to try the whole loop, and storage rates keep drifting down across the market, so treat any figure in an article — including ours — as an upper bound and read the live one.

References

• Infrai storage API reference
• Expiring objects — Amazon S3 lifecycle considerations
• Locking objects with S3 Object Lock