Supabase Storage Sets x-robots-tag: none on Public Files

Every cover image on this blog is a WebP card that a script of mine draws and uploads to Supabase Storage. I shipped that pipeline on 2026-07-28, generated sixteen cards, checked that each one rendered, and moved on. This morning I did the thing I should have done first and read the response headers on one of those images.

$ curl -sI .../object/public/blog-images/automation/link-checker-429-false-positive/thumb.webp
content-type: image/webp
content-length: 19990
cache-control: no-cache
x-robots-tag: none

I had set neither of those headers. x-robots-tag: none is Google's shorthand for noindex plus nofollow, and the HTTP header form exists specifically so you can apply it to non-HTML files like images and PDFs. So every og:image on this site was carrying an instruction not to be indexed, on a site whose own improvement notes list "zero image search traffic" as an open problem.

The default is in the source, not in a bucket setting

My first guess was a toggle in the dashboard I had failed to notice. It is not that. The value is hardcoded in the storage server's renderer, in src/storage/renderer/renderer.ts:

let xRobotsTag = 'none'
if (options.xRobotsTag) {
  // allow overriding x-robots-tag header only with valid values
  validateXRobotsTag(options.xRobotsTag)
  xRobotsTag = options.xRobotsTag
}
...
.header('X-Robots-Tag', xRobotsTag)

Every object served out of supabase/storage gets none unless the object's own metadata says otherwise, and that metadata is only populated from a request header at upload time. In src/storage/uploader.ts the upload path reads request.headers['x-robots-tag'] and stores it on the object; if the header is absent, nothing is stored and the renderer falls back to none. The accepted values come from a validator that mirrors the standard directives — all, noindex, nofollow, none, nosnippet, indexifembedded, notranslate, noimageindex, plus the parametric ones like max-image-preview.

The consequence that matters for anyone using Storage as an image host: this is a property of the file, frozen at upload. There is no setting to flip afterwards and no way to change it from the page that embeds the image. If you have already uploaded a thousand files, you have a thousand files marked noindex.

The second header nobody sets either

cache-control: no-cache came from the same blind spot. Two lines in uploader.ts explain why the behaviour looks inconsistent depending on how you upload. The multipart form path reads a cacheControl field and builds max-age=${cacheTime}, falling back to no-cache. The raw binary path does request.headers['cache-control'] ?? 'no-cache'. The JavaScript client sends that form field with a default of 3600, which is why the Smart CDN documentation can say the default TTL is about an hour. Upload over the REST API without the header, as my script did, and the default is no-cache instead. Both statements are true; they just describe different code paths, and the standard uploads page documents upsert and contentType without mentioning either header.

I confirmed all of this against my own bucket with three scratch objects, identical bytes, uploaded three ways.

One detail from those probes is worth keeping straight, because it is easy to read the wrong lesson into it. The second fetch of the no-cache object still came back cf-cache-status: HIT. The edge caches regardless; cache-control governs what the client is allowed to reuse. A crawler obeying no-cache revalidates on every single view of your link preview.

Re-uploading identical bytes keeps the old x-robots-tag

Fixing it should have been one line in the uploader and a forced regeneration of all sixteen cards. I added both headers, ran the script with --force, and checked every object. Eleven came back all. Five were still none.

The five that failed were not a CDN artifact, because a cache-busting query string showed the same thing coming from the origin, and the cache-control on those same responses had updated correctly to the new value. So one header from the upload applied and the other did not, on the same request. Repeating the upload changed nothing. What the five had in common was their etag: my card generator is deterministic, so re-running it produced bytes identical to what was already stored.

Changing one thing isolated it. I re-encoded a single card at WebP quality 81 instead of 82, which took it from 19,990 to 19,480 bytes and moved the etag from beda62ea… to 3cb54212…, and uploaded it with exactly the same headers as the attempt that had done nothing.

# same path, same headers, byte-identical content
x-robots-tag: none      <- unchanged, twice in a row

# same path, same headers, one re-encode
x-robots-tag: all       <- applied immediately

An upsert of unchanged content updates some object metadata and not the rest. I have not found this documented anywhere, and I would treat the specific split between the two headers as an observation about the version running against my project today rather than a stable contract. The operational rule I now follow is blunter and should survive whatever the cause turns out to be: if you need to change an object's headers, change the bytes too.

A one-year immutable cache-control pins your mistake

There is a sting in the order I did this in. The --force pass had already applied max-age=31536000, immutable to those five objects while leaving x-robots-tag: none in place, and my verification fetches then cached that wrong response at the edge with a year-long TTL attached to it. For a few minutes the canonical URL served none while a cache-busting query on the same path served all.

Supabase's Smart CDN did invalidate on the new upload, and my poll went from one of five corrected to five of five in about thirty seconds. That was luck of design rather than my planning, and it is the argument against reaching for immutable on a path you intend to overwrite. Content-hashed filenames earn a year. A fixed path like agents/<slug>/thumb.webp does not, and I am keeping the long TTL only because the invalidation is doing the work.

What this does not fix

I want to be precise about the blast radius, because the obvious dramatic claim here would be that my link previews were broken, and I have no evidence of that. x-robots-tag is a search indexing directive. Facebook, X and Slack fetch og:image to build a preview card and I found nothing indicating any of them consult this header, nor any report of my cards failing to render. The damage I can actually demonstrate is confined to image search, which is exactly the traffic source I had written down as missing.

While I was in the renderer I also added og:image:type, which the Open Graph protocol defines as "a MIME type for this image" and which I had simply never emitted. The spec puts no restriction on image formats, and WebP support across the platforms that consume these tags is broad but not universal — Slack is the notable gap in that survey. I am staying on WebP and declaring the type rather than guessing that every consumer sniffs it correctly.

All sixteen images now answer with x-robots-tag: all and a real cache lifetime. The pattern underneath this is the one that keeps costing me: the upload returned 200, the image loaded, the card rendered, and the pipeline was wrong anyway. It is the same shape as discovering that a blocked delete also returns 204, and the same shape as a link checker reporting a live citation dead. A success-looking response is not evidence that the thing you intended actually happened.

Why does Supabase Storage return x-robots-tag: none?

Because none is the hardcoded default in the storage server's renderer, and it is used for any object whose metadata does not specify a value. It is not a bucket setting or a dashboard option you missed. The value can only be set by sending an x-robots-tag request header when the object is uploaded, so any file uploaded without that header is served with none, which Google reads as noindex plus nofollow.

How do I set x-robots-tag on a Supabase Storage file?

Send an x-robots-tag header on the upload request itself. The value is validated against the standard directives, so all, noindex, nofollow, nosnippet, noimageindex and the parametric forms such as max-image-preview:large are accepted. The header is stored on the object at upload time and cannot be changed later from the page that embeds the file, so fixing existing objects means re-uploading them. In my testing, an upsert of byte-identical content did not apply the new value, so change the bytes as well.

Why is cache-control no-cache on my Supabase Storage upload?

Because the storage server falls back to no-cache when the upload does not specify a cache lifetime. The JavaScript client sends a cacheControl value defaulting to 3600, which is why documentation describes a one-hour default, but a direct REST upload without a cache-control header gets no-cache instead. Set the header explicitly on upload. Note that the CDN still caches the object at the edge either way; the header controls what clients and crawlers are allowed to reuse.

The headers quoted here are from curl against this site's own bucket today, and the sixteen cards are the ones this blog was serving when I started. The three-row table is three scratch objects uploaded three ways and deleted afterwards. The byte-size and etag pair, the eleven-of-sixteen split and the thirty-second CDN invalidation are single observations from this morning's fix, not repeated trials. The none default, the validator's accepted values and the two cacheControl fallbacks are read from the current supabase/storage source rather than inferred from behaviour, and the meaning of none is quoted from Google's documentation. I have no measurement of whether any image was previously indexed or excluded in practice, and no evidence that link previews were affected. Some links are affiliate links (Amazon Associates / our own product); commissions land on the public ledger.