Cloudflare Workers KV List Keys: Listing Is Not Reading

My /stats endpoint returned HTTP 500 twice this afternoon, at 121.4 s and 117.9 s. Three days ago I published a post that predicted this failure and put a date on it. Before fixing anything I wanted to check whether the prediction landed, and to find out what a list() loop actually costs. The answer was nothing like what I assumed when I wrote it.

The endpoint is the admin view of the click tracker Worker that runs this site, sitting over one KV namespace. It does the obvious thing: list every key under six prefixes, then read the value behind each key.

// ops/tracker/worker.js - the loop that died
for (const prefix of ["views:", "clicks:", "link:", "ref:", "hviews:", "hclicks:"]) {
  let cursor;
  do {
    const res = await env.STATS.list({ prefix, cursor });
    for (const k of res.keys) {
      const v = await env.STATS.get(k.name);   // one operation per key
      ...
    }
    cursor = res.list_complete ? null : res.cursor;
  } while (cursor);
}

Listing 1,246 keys costs two requests and 1.4 seconds

I pulled the whole namespace through the REST API today, following the cursor to the end. It holds 1,246 keys: 591 under views:, 305 under hviews:, 216 under clicks:, 72 under hclicks:, 40 under link:, 16 under ref:, and 6 stragglers. Fetching that entire index took two requests, 0.66 s and 0.75 s.

Reading the values behind those keys is a different order of magnitude: 1,240 separate operations, roughly two minutes of wall time, and, as of today, a crash. Listing is an index scan the platform is built for. The get() inside the loop is the part that scales with your key count, and it is easy to miss because the listing code is what you are looking at while you write it.

The page you get back is not the page you asked for

The first surprise came before any of that. I asked for 1,000 keys and got 957. Not an error, not the end of the namespace — a cursor came back pointing at 289 more. Sweeping the limit parameter across the same 1,246-key namespace:

limit requested	keys returned	short by
200	151	49
500	474	26
800	793	7
900	873	27
950	890	60
970	901	69
971	917	54
1000	957	43

The shortfall is deterministic — three consecutive runs at limit=1000 returned 957 keys ending on the same key name — but it is not proportional and not monotonic. Asking for 800 costs you 7 keys; asking for 950 costs you 60. Earlier in the session, with 43 temporary probe keys in the namespace, the same call returned 970.

So the tempting termination check is wrong in a way that will not show up in testing:

// wrong: a short page does not mean the end
if (res.keys.length < limit) break;

// right: the platform tells you, and it is the only thing that does
cursor = res.list_complete ? null : res.cursor;

On this namespace that bug silently drops 289 of 1,246 keys, or 23% of the data, with no error and no log line. Cloudflare's list keys documentation is explicit that list_complete "will be false if there are more keys to fetch, even if the keys array is empty," and offers a mechanism: recently expired or deleted keys still have to be iterated through, but are not included in what you get back.

Two experiments that failed to reproduce it

I tried to confirm that mechanism and could not. First I wrote 40 keys with expiration_ttl=60 plus 3 permanent ones under a fresh prefix. Before expiry, paging at limit=10 gave a clean 10/10/10/10/3. After expiry, at t+194 s, the prefix returned a single page of 3 with no cursor. No phantom pages.

Then I wrote 30 permanent keys, listed them, deleted the first 10 in bulk, and listed again 0.7 seconds later. Same result: the deleted keys were gone from the index immediately, no empty pages left behind. So I cannot tell you why my production namespace hands back short pages. The documented explanation is the vendor's, and both attempts to reproduce it at this scale came back negative.

The second experiment did surface something better. Listing 30 keys at limit=10 produced four pages: 10, 10, 10, and then zero. The third page was completely full and still carried a cursor; the fourth was empty and was the real terminator. Page length carries no information in either direction. A full page does not mean more data, and an empty page is the normal way a listing ends.

One more thing the docs skip: on the REST path, limit has an undocumented floor. Anything below 10 returns HTTP 400 with {"code": 10028, "message": "limit argument must be at least 10"}. The documentation states the default and maximum of 1,000 and says nothing about a minimum.

What actually killed the endpoint

The /stats failure returns error code: 1101, which is the Workers code for an uncaught exception in your own code and tells you nothing on its own. So I ran wrangler tail against the live Worker and hit the endpoint again:

outcome: exception   cpuTime: 193   wallTime: 115025
message: "Too many API requests by single Worker invocation. To configure this limit,
          refer to https://developers.cloudflare.com/workers/wrangler/configuration/#limits"
stack:   "    at Object.fetch (worker.js:122:39)"

Two details in there are worth more than the message. 193 ms of CPU against 115,025 ms of wall clock — the Worker burned a fifth of a second of compute across nearly two minutes of waiting. This is not a CPU limit or a duration limit. It is a count.

The stack line is a lie. worker.js:122 is "Content-Type": "application/zip", inside the /playbook download branch that a /stats request never enters (that branch ends at line 126; the /stats handler starts at 149). The exception surfaces at the handler boundary, not at the call that caused it. If you are chasing this error by line number you will spend the afternoon reading the wrong function.

Two endpoints, one namespace, one wall

The same Worker exposes a public summary at /public-stats that scans only views: and clicks:. Counted from the key inventory, that is about 812 operations against roughly 1,252 for /stats. Today:

endpoint	prefixes scanned	operations (counted from inventory)	result
/public-stats	2	~812	HTTP 200 in 119.820 s
/stats	6	~1,252	HTTP 500, error 1101, 121.384 s

Same code shape, same namespace, same edge, ten minutes apart. One is under 1,000 operations and merely unbearable; the other is over and throws. Cloudflare's Workers limits page gives the number: subrequests to internal Cloudflare services are capped at 1,000 per invocation on the free plan, against 50 for third-party fetches. The error message points at the limits block in your Wrangler config, where subrequests is indeed configurable — up to 10,000 by default and 10 million at most, on the paid plan. On free, the ceiling the message tells you to raise is the number you just hit.

Dating the crossing, three days after predicting it

Every counter key in this namespace is named prefix:YYYY-MM-DD:name, so the full key listing is also a growth log. Reconstructing cumulative dated keys:

The count stood at 927 on August 9 — 979 operations once the undated link: keys and the list requests are added, still under the cap. August 10 added 177 keys and took it over. My August 8 post on KV latency ended with this sentence: "At that rate the loop crosses 1,000 operations in roughly two days, and the endpoint stops being slow and starts throwing — no client timeout required." Roughly two days from August 8 is August 10. That is the day.

The prediction was right for a shaky reason. It assumed 60.1 new keys per day; the actual rate since has been about 118. The model was off by nearly a factor of two and the date still landed, so something else in the arithmetic absorbed the error. A correct date out of a wrong rate is luck, not a good model.

What this costs a visitor

The public homepage of this site calls /public-stats from the browser on every page load (ops/site/index.html:972). Every visitor triggers roughly 812 KV operations and waits about two minutes for three numbers to appear. That endpoint sits 188 operations from the same wall, which at the current rate is under two days away.

The fix is in the same documentation that describes the limit, and it is not subtle. A get() call accepts an array of up to 100 key names and counts as one operation, which turns 1,240 reads into 13. Better still, the list-keys page recommends storing small values in key metadata, which list() already returns — that collapses the whole loop to about a dozen operations and removes the per-key read entirely. My counters are integers under 8 bytes against a 1,024-byte metadata budget, so they fit.

Neither is deployed as I publish this. The tracker still runs the loop above, /stats still returns 500, and the homepage still makes every visitor wait. This is the third repair I have queued against this Worker without shipping one, after the lost concurrent increments and the free-tier write ceiling. A limit measured in the thousands feels infinitely far away on day one of a project and arrives, unannounced, on day twenty.

One footnote on tooling. wrangler kv key list returned all 1,246 keys in 2.4 seconds and has no --cursor or --limit flag at all — the CLI paginates for you and hides the concept completely. That behaviour came from workers-sdk issue #4095, opened in October 2023 and closed as completed in April 2026, with zero comments in between. Two and a half years, and the result is that the surface most people learn KV through is the one place the cursor cannot bite them.

The tracker Worker, including the loop in this post and the bulk-read rewrite I have queued for it, is part of the MMM Playbook ($12).

All measurements here were taken on 2026-08-11 from Seoul against the live picklog-go Worker and its production KV namespace: the key inventory and limit sweep through Cloudflare's REST API, the exception text through wrangler tail 4.120.1, and the endpoint timings through curl from a single location, one to three runs each. The operation counts of ~812 and ~1,252 are counted from the key inventory and the pagination pattern I observed, not reported by the platform. The growth timeline is reconstructed from date-stamped key names; the 40 undated link: keys are counted as present for the whole period, which slightly overstates early totals. The 73 probe keys written for the expiry and delete experiments have been deleted. I could not reproduce the documented expired-key mechanism behind short pages, and have not attributed my results to it.