100 MB to 1 GB documents: when to switch to multipart upload

Size bands, the 33% base64 tax on single-call uploads, and a resumable Node driver for PDFs and zips landing in a private Infrai bucket.

Below roughly 32 MB, send the document in one call and move on. Above it, open a multipart session — not because a single request will fail, but because a failure at minute two costs you the whole upload instead of one 16 MiB slice. Infrai’s storage module offers both shapes behind the same key: PUT /v1/storage/object/put/{bucket}/{key} takes the bytes inline, and POST /v1/storage/multipart/create/{bucket} opens a resumable session for the 100 MB contracts and 1 GB evidence bundles that break it.

The threshold isn’t a hard limit. It’s the point where retry cost, memory and wall-clock time all start pushing the same direction.

Choosing by size

In our testing on 26 July 2026, a 40 MB PDF sent as a single call took just over a minute end to end. Nothing was wrong — that’s simply what it costs to base64 the bytes, ship 53 MB of JSON and have the far side decode it.

The single-call path, and its 33% tax

object/put carries the file as data_base64, which inflates the payload by a third. For a 2 MB contract that’s noise. For a 200 MB one it’s 67 MB of pure encoding overhead on the wire, and it’s the main reason the size bands above exist at all.

Build the payload as a file rather than trying to interpolate megabytes into a shell argument:

import base64
import json
import sys

src, dest = sys.argv[1], sys.argv[2]
with open(src, "rb") as handle:
    raw = handle.read()

payload = {
    "data_base64": base64.b64encode(raw).decode("ascii"),
    "content_type": "application/pdf",
}
with open(dest, "w", encoding="utf-8") as out:
    json.dump(payload, out)

print(f"{len(raw)} bytes -> {len(payload['data_base64'])} base64 chars")

export INFRAI_API_KEY="your_infrai_api_key"

python3 build_payload.py ./msa-2026.pdf ./payload.json

curl -sS -X PUT \
  "https://api.infrai.cc/v1/storage/object/put/kb-docs-upload-0726/contracts/tenant_12/msa-2026-a91c.pdf" \
  -H "Authorization: Bearer ${INFRAI_API_KEY}" \
  -H "Content-Type: application/json" \
  --data-binary @payload.json

{
  "ok": true,
  "data": {
    "bucket_id": "bkt_a666e5a16725465fbbd456",
    "key": "contracts/tenant_12/msa-2026-a91c.pdf",
    "size_bytes": 2400009,
    "etag": "89eaf2e0910e07174eaf562876d8a886",
    "content_type": "application/pdf",
    "metadata": null,
    "last_modified": "2026-07-26T01:12:17.648908Z"
  }
}

Above the threshold: a session your process can survive

For a 1 GB bundle the interesting failure isn’t a dropped packet, it’s a deploy restarting the worker at part 37. Persist the upload_id and every completed part number and ETag as you go, and resuming becomes a matter of skipping what’s already stored. AWS’s multipart overview documents the same contract for S3, so this driver ports with a change of base URL.

import { open, stat } from "node:fs/promises";
import { loadJob, saveJob, savePart } from "./jobs-repo.mjs";

const API = "https://api.infrai.cc";
const BUCKET = "kb-docs-upload-0726";
const CHUNK = 16 * 1024 * 1024;

const token = process.env.INFRAI_API_KEY;
if (!token) throw new Error("INFRAI_API_KEY is not set");
const headers = { Authorization: `Bearer ${token}`, "Content-Type": "application/json" };

export async function uploadDocument(jobId, filePath, key) {
  let job = await loadJob(jobId);

  if (!job.upload_id) {
    const opened = await fetch(`${API}/v1/storage/multipart/create/${BUCKET}`, {
      method: "POST",
      headers,
      body: JSON.stringify({ key, content_type: "application/pdf" }),
    });
    const session = await opened.json();
    if (session.ok === false) throw new Error(session.error.code);
    job = await saveJob(jobId, { upload_id: session.data.upload_id, parts: [] });
  }

  const done = new Map(job.parts.map((p) => [p.part_number, p]));
  const { size } = await stat(filePath);
  const file = await open(filePath, "r");

  try {
    const buffer = Buffer.allocUnsafe(CHUNK);
    for (let n = 1, offset = 0; offset < size; n++, offset += CHUNK) {
      if (done.has(n)) continue;

      const { bytesRead } = await file.read(buffer, 0, CHUNK, offset);
      const signed = await fetch(
        `${API}/v1/storage/multipart/presign_part/${job.upload_id}/${n}`,
        { method: "POST", headers, body: "{}" },
      );
      const slot = await signed.json();

      const sent = await fetch(slot.data.url, { method: "PUT", body: buffer.subarray(0, bytesRead) });
      if (!sent.ok) throw new Error(`part ${n} -> HTTP ${sent.status}`);

      const part = { part_number: n, etag: (sent.headers.get("etag") ?? "").replaceAll('"', "") };
      done.set(n, part);
      await savePart(jobId, part);
    }

    const ordered = [...done.values()].sort((a, b) => a.part_number - b.part_number);
    const committed = await fetch(`${API}/v1/storage/multipart/complete/${job.upload_id}`, {
      method: "POST",
      headers,
      body: JSON.stringify({ parts: ordered }),
    });
    const result = await committed.json();
    if (result.ok === false) throw new Error(result.error.code);
    return result.data;
  } finally {
    await file.close();
  }
}

A restart replays this function, skips the parts already in the jobs table, and finishes. No abort call in the happy path, no re-reading 900 MB you already sent.

What “private bucket” does and doesn’t mean here

Buckets are private by default, and that governs who can call the API against them. It does not mean the bytes are unreachable: an object URL without a signature still returns the file, so treat the key itself as part of the secret — random suffix, never the user’s filename, exactly as the OWASP file upload guidance recommends. That’s a real limitation to weigh if your documents are legally sensitive.

Two smaller behaviours worth knowing before you ship:

Executable and active content types are refused outright. A text/html or application/x-msdownload upload comes back HTTP 415 with STORAGE_CONTENT_TYPE_BLOCKED — sensible for a document vault, annoying if you were planning to store rendered HTML previews next to the PDFs.

{
  "ok": false,
  "error": {
    "code": "STORAGE_CONTENT_TYPE_BLOCKED",
    "http_status": 415,
    "message": "content-type 'text/html' is not allowed (active/executable)",
    "retryable": false
  }
}

And custom metadata keys must be hyphenated. {"tenant-id":"12"} round-trips fine; {"doc_kind":"msa"} breaks the upstream signature and surfaces as a 503, which is a rough way to learn a naming rule.

curl -sS \
  "https://api.infrai.cc/v1/storage/object/head/kb-docs-upload-0726/contracts/tenant_12/msa-2026-a91c.pdf" \
  -H "Authorization: Bearer ${INFRAI_API_KEY}"

The bill for a document workload

Verified 26 July 2026: a single-call write bills $0.0001, a multipart commit $0.0002, and opening a session, signing parts, listing, head and deletes are all free. New accounts carry $2 in credit. Note what that means for the size bands — a 1 GB upload in 64 parts costs the same one commit as a 40 MB one, so per-call pricing doesn’t punish you for slicing finer.

curl -sS https://api.infrai.cc/v1/discovery \
  -H "Authorization: Bearer ${INFRAI_API_KEY}" \
  | python3 -c "import json,sys; [print(c['id'], c['billing'].get('price_usd','free')) for c in json.load(sys.stdin)['capabilities'] if c['id'].startswith('storage')]"

Those rates drift down over time and campaigns run, so check rather than quote. The structural argument holds either way: the virus scan queued after the upload, the cron that expires old contracts and the per-tenant cost query all run on the same credential.

Where a specialist beats this

Regulated document retention is the clearest case. If you need WORM guarantees, S3 Object Lock in compliance mode with server-side KMS encryption is the answer and nothing here substitutes for it; Backblaze B2 offers a cheaper file-lock variant if archive cost dominates. Teams already running MinIO in their own data centre have the residency story solved and shouldn’t move. The pitch here is narrower and more practical — one key, one bill, and the queue, cron and error tracking around your upload pipeline already provisioned.

References

• Infrai storage API reference
• STORAGE_CONTENT_TYPE_BLOCKED error reference
• Uploading and copying objects using multipart upload — Amazon S3
• OWASP File Upload Cheat Sheet