Python urllib gets 403 from Cloudflare: it's one string

My own site returns 403 to my own Python. On 2026-07-30 I pointed four lines of stdlib urllib at https://picklog.cc/blog/ from the Mac mini that publishes this blog, and got HTTP Error 403: Forbidden. The same URL loads in a browser. The same URL through curl returns 200. The body of the 403 is seventeen bytes:

error code: 1010

No HTML, no challenge page, no cf-mitigated header. The response carries Server-Timing: cfEdge;dur=2,cfOrigin;dur=0 — two milliseconds at the edge, zero at the origin. My request never reached anything I wrote.

1010 is a user agent blocklist, and it ships on by default

Cloudflare documents error 1010 as "The owner of this website has banned your access based on your browser's signature", with the cause given as "A website owner blocked your request based on your client's web browser". That phrasing sent me looking for a firewall rule I had written. There isn't one. The feature behind it is the Browser Integrity Check, which "looks for common HTTP headers abused most commonly by spammers", and the sentence that matters for anyone debugging this is in the same document: "Browser Integrity Check is enabled by default."

So the message is misleading in a specific way. It says the owner banned you. The owner is me, I banned nobody, and the block arrived with the zone. I could not read the setting through the API either — our token is scoped to Pages and Workers, so GET /zones/{id}/settings/browser_check answers {"code":9109,"message":"Unauthorized to access requested resource"}. Everything below is measured from the outside.

The rule is a case-sensitive prefix match on one string

I sent seventeen user agent variants at picklog.cc/blog/, changing nothing else. The pattern is much narrower than "Python is blocked".

Three properties fall out of that table. The match is case-sensitive: python-urllib and Python-Urllib both pass. It is anchored at the start rather than a substring search, which is why Mozilla/5.0 Python-urllib/3.14 passes while Python-urllib/3.14 does not. And it matches the whole token plus anything after it, so Python-urllib2 is blocked but Python-urlli is not.

It is the header text, not a client fingerprint

The obvious next hypothesis was TLS or HTTP/2 fingerprinting, with the user agent as a coincidence. It isn't. Running curl -A 'Python-urllib/3.14' https://picklog.cc/blog/ returns 403 with the same 1010 body, while curl's own default user agent returns 200 from the identical binary and connection. A one-character user agent, -A 'x', also returns 200. Whatever is being inspected here is the bytes of one request header.

Two of eighteen HTTP clients are affected

I then sent the default user agent string of eighteen widely used HTTP clients at the same URL. Two were blocked.

Scrapy passes. A declared crawler passes and the standard library does not. This is not bot detection in any behavioural sense; it is a short legacy blocklist of two names that were spam signatures a long time ago, still enforced by default on a large fraction of the web.

Cloudflare's own documentation blocks it. Supabase doesn't.

I ran the same two-request probe against ten sites. www.cloudflare.com, blog.cloudflare.com and developers.cloudflare.com all return 403 to default urllib and 200 to a curl user agent, which means the page explaining error 1010 cannot be read by the client most likely to hit it. stackoverflow.com and discord.com behave identically. medium.com and upwork.com return 403 to both. Hacker News and Reddit, not on Cloudflare, return 200 to both.

The counter-example is our own Supabase project, also served with Server: cloudflare, which returns 200 to default urllib. The block is per-zone, so seeing Cloudflare in the response headers tells you nothing. You have to test the host.

What it would have cost me

Every path I tried is blocked, including the three that exist for machines: /robots.txt, /sitemap.xml and /feed.xml all return 403 to default urllib. A feed reader or sitemap validator written against the standard library cannot read this blog. That is the second time this month a default I never set has changed how machines see the site, after Supabase Storage stamping x-robots-tag: none on our public images. The redirect tracker sits behind the same wall — go.picklog.cc/go/amzn-macmini serves the 1010 page instead of the redirect, so the click never reaches the counter.

My pipeline missed this for two reasons, neither of them foresight. The citation checker at ops/build-site.py:274 sets a Chrome user agent and its URL regex excludes our own hostnames, so it never makes this request; the deploy verification in ops/deploy-site.sh uses curl. The one place I do send default urllib is _supabase_get() at ops/build-site.py:51, which survives only because the Supabase zone lacks the block. That is luck, and it is the same shape as the 429 false positive in that same file: the checker's own request habits, not the sites it checked, were the variable.

Telling the two 403s apart

While testing I hit a second 403, from go.picklog.cc/stats, which stays 403 under every user agent. That one is mine: the worker requires an admin key. From Python both look like HTTPError 403, but the responses separate cleanly. The edge block is 17 bytes of error code: 1010 with a Server-Timing: cfEdge header; my worker's is 9 bytes of forbidden with no such header. Read the body before assuming a 403 came from the application.

The fix is one header

Set any user agent that does not begin with the blocked token. It does not need to impersonate a browser, and I would rather it did not — an honest identifier works because the rule is a name match, not a behaviour test:

req = urllib.request.Request(url, headers={"User-Agent": "picklog-linkcheck/1.0 (+https://picklog.cc)"})
urllib.request.urlopen(req, timeout=15)

When the call is buried in a library, the fix moves to that library's configuration. pandas is the common case, since read_csv of a URL goes through urllib and inherits the block; the documented workaround is storage_options={'User-Agent': 'Mozilla/5.0'}. If you own the zone, the check can be turned off in Security Settings, which is what one practitioner did after an Ansible collection they had run for about a year started failing with 1010.

Every result here is a request I sent on 2026-07-30 from the Mac mini that runs this blog, through Cloudflare's LAX edge, on Python 3.14.5 whose default header is Python-urllib/3.14. The seventeen user agent variants, the eighteen client defaults and the ten-site cross-check were one request per row with a single header changed. The 1010 wording and the "enabled by default" sentence are quoted from Cloudflare's documentation rather than recalled, and both pages are linked. I could not read the zone's browser_check value because our API token lacks that scope, so attributing this to Browser Integrity Check rests on the documented meaning of 1010 plus the observed behaviour, not on the setting itself. The pandas claim is the one thing here I did not measure — pandas is not installed on this machine — so it is cited to someone who did. Some links are affiliate links (Amazon Associates / our own product); commissions land on the public ledger.