Does this object already exist? Size and hash checks without downloading

One free head call returns found, size_bytes and an ETag that equals the file's MD5 — enough to build instant-upload dedupe before a single byte goes over the wire.

Use the head route. GET /v1/storage/object/head/{bucket}/{key} on Infrai returns whether the key is there, how many bytes it holds, its recorded MIME type, its last-modified time and its ETag — without transferring the object. It’s free and it came back in about 70 ms in our testing, which makes it cheap enough to call on every upload attempt.

The interesting part for dedupe is the ETag. For an object written with a single PUT, Infrai’s ETag is the MD5 of the stored bytes — we checked it both ways, hashing a 83-byte CSV and a 200,000-byte binary locally and getting exactly the values the API reported. So a client that can hash a file can decide “the server already has this” before uploading anything.

export INFRAI_API_KEY="your_infrai_api_key"

curl -sS -X GET "https://api.infrai.cc/v1/storage/object/head/kb-csv-0726/results/2026-07-26/orders_9f21.summary.csv" \
  -H "Authorization: Bearer ${INFRAI_API_KEY}"

{
  "ok": true,
  "data": {
    "found": true,
    "status": "found",
    "key": "results/2026-07-26/orders_9f21.summary.csv",
    "size_bytes": 64,
    "etag": "c20821601f570f4231a73f3b82e16fdb",
    "content_type": "text/csv",
    "metadata": {
      "sha256": "8be0b33aee2ddfb4a5769800c6dc782544a388f871919ebe07fdbd631330ea77"
    },
    "last_modified": "2026-07-26T01:10:16Z"
  }
}

A missing key is not an error

This trips people coming from boto3, where a missing object raises and you catch a 404. Here the request succeeds and the answer lives in the payload:

{
  "ok": true,
  "data": {
    "found": false,
    "status": "not_found",
    "key": "uploads/nope.csv"
  }
}

ok: true, HTTP 200, found: false. Any wrapper that treats a non-2xx as “absent” will report every object as present, forever, and you’ll only notice when your dedupe stops deduplicating. Branch on data.found.

Instant-upload detection, end to end

The pattern: hash locally, derive the key from the hash, head it, upload only on a miss. Because the key contains the digest, two users uploading the same file converge on the same object and the second upload never happens.

import { readFile } from "node:fs/promises";
import { createHash } from "node:crypto";
import { Buffer } from "node:buffer";

const TOKEN = process.env.INFRAI_API_KEY;
if (!TOKEN) throw new Error("INFRAI_API_KEY is not set");

const BUCKET = "kb-csv-0726";

async function head(key) {
  const res = await fetch(`https://api.infrai.cc/v1/storage/object/head/${BUCKET}/${key}`, {
    method: "GET",
    headers: { Authorization: `Bearer ${TOKEN}` },
  });
  const json = await res.json();
  if (!res.ok || json.ok !== true) throw new Error(`head ${key}: HTTP ${res.status}`);
  return json.data;
}

export async function storeOnce(filePath, contentType) {
  const bytes = await readFile(filePath);
  const sha = createHash("sha256").update(bytes).digest("hex");
  const md5 = createHash("md5").update(bytes).digest("hex");
  const key = `cas/${sha.slice(0, 2)}/${sha}`;

  const existing = await head(key);
  if (existing.found === true) {
    const sizeMatches = existing.size_bytes === bytes.length;
    const hashMatches = existing.etag === md5;
    if (sizeMatches && hashMatches) return { key, uploaded: false, reason: "identical object already stored" };
    console.warn(`key collision or multipart etag at ${key}: size=${existing.size_bytes} etag=${existing.etag}`);
  }

  const payload = {};
  payload.data_base64 = bytes.toString("base64");
  payload.content_type = contentType;

  const res = await fetch(`https://api.infrai.cc/v1/storage/object/put/${BUCKET}/${key}`, {
    method: "PUT",
    headers: { Authorization: `Bearer ${TOKEN}`, "Content-Type": "application/json" },
    body: JSON.stringify(payload),
  });
  const json = await res.json();
  if (!res.ok || json.ok !== true) throw new Error(`put ${key}: HTTP ${res.status} ${JSON.stringify(json.error ?? json)}`);
  return { key, uploaded: true, etag: json.data.etag, size: json.data.size_bytes };
}

console.log(await storeOnce("summary.csv", "text/csv"));

Two checks, not one. Size alone is a weak signal — plenty of distinct files share a byte count — and the ETag alone can mislead you for the reason in the next section, so requiring both keeps the false-positive rate near zero without a download.

Where the ETag-is-MD5 rule breaks

Objects assembled from multipart uploads don’t carry a plain MD5. In the S3 family the ETag for a multi-part object is a hash of the concatenated part hashes with a -N suffix, so it depends on the part size the uploader chose, not just the content. Two identical files uploaded with different part sizes get different ETags.

So treat a mismatch as unknown, never as corruption.

The durable fix is to record your own strong digest alongside the object, which is what the sha256 field in that first response is. POST /v1/storage/object/set_metadata/{bucket}/{key} attaches it:

curl -sS -X POST "https://api.infrai.cc/v1/storage/object/set_metadata/kb-csv-0726/results/2026-07-26/orders_9f21.summary.csv" \
  -H "Authorization: Bearer ${INFRAI_API_KEY}" \
  -H "Content-Type: application/json" \
  -d '{"metadata":{"sha256":"8be0b33aee2ddfb4a5769800c6dc782544a388f871919ebe07fdbd631330ea77"}}'

Two things we learned the hard way here. Metadata keys containing an underscore break the upstream signature calculation — content_sha256 fails with a 503 VENDOR_DOWN carrying SignatureDoesNotMatch, while plain sha256 succeeds. Use hyphens or single words. And setting metadata rewrites the object in place, which bumps last_modified, so don’t build an age-based lifecycle policy that a metadata update can silently reset.

Checking many keys at once

Calling head 10,000 times is correct and slow. GET /v1/storage/object/list/{bucket} walks a prefix in pages of up to 1,000 and gives you key, size and ETag per row:

curl -sS -X GET "https://api.infrai.cc/v1/storage/object/list/kb-csv-0726?prefix=results/&delimiter=/" \
  -H "Authorization: Bearer ${INFRAI_API_KEY}"

{
  "ok": true,
  "data": {
    "items": [],
    "next_cursor": null,
    "common_prefixes": ["results/2026-07-26/"]
  }
}

With delimiter=/ you get directory-style browsing: items holds only the keys directly at that level and common_prefixes names the pseudo-folders below it. Note the empty items array there — everything under results/ sits one level deeper. Drop the delimiter to get the objects themselves. One caveat: list rows report content_type as null even when the object has one, so MIME type still requires a head.

What it costs to check versus to fetch

Verified 2026-07-26: object/head, object/list, object/presign and bucket/usage are free on Infrai — rate-limited rather than metered, and they don’t consume the $2 credit new accounts get. Downloading is $0.0002 per object/get and uploading $0.0001 per object/put, so reads run about double writes and a head-before-get on a 60% cache-hit workload removes most of the read bill outright. Storage and egress are metered by GB on top. Infrastructure pricing drifts down and promotions run, so read today’s rather than mine:

curl -sS "https://api.infrai.cc/v1/discovery" \
  -H "Authorization: Bearer ${INFRAI_API_KEY}" \
  | python3 -c "import json,sys; d=json.load(sys.stdin); print([(c['id'], c['billing'].get('price_usd','free')) for c in d['capabilities'] if c['id'] in ('storage.object.head','storage.object.get','storage.object.list')])"

Limits, and when something else is the better tool

There’s no server-side deduplication. Store the same bytes under two keys and you’re billed for two copies — dedupe is a decision your code makes, which is exactly why the free head call matters. There’s also no checksum-on-upload API: you can’t ask the platform to verify a SHA-256 you supply, so the comparison happens on your side. And per the earlier caveat, the ETag equivalence holds for single-PUT objects only.

If you’re on S3 already, HeadObject answers the same question and recent S3 versions can store and return a real SHA-256 checksum computed at upload time, which is stronger than anything you can reconstruct from an ETag — for a content-addressed store at scale that’s worth the migration on its own. MinIO gives you the same API surface on hardware you control, which is the honest pick when the dataset is large, cold and yours.

Where one credential earns its keep is the rest of the loop: the queue that fans out these checks, the cron that re-verifies a sample of the store weekly, the error tracker that catches a 503 from a bad metadata key, and the bucket itself — one account, one bill, one usage view. If object storage is all you need, use a specialist and keep this pattern; it ports cleanly.

References

• Infrai storage API reference
• Uploading objects and checksums (Amazon S3 user guide)
• Fastest way to find out if a file exists in S3 (Peter Bengtsson)