Local LLM Mac vs PC: 240 Chips, and Prefill Is the Gap

On September 12 my Mac mini spent 293 seconds on a single llama.cpp test: fill a 32K-token context, then write 64 tokens. On an empty context the same machine writes about 21 tokens per second. Both numbers are real, and most "local LLM Mac vs PC" answers quote only the second kind. I joined the four scoreboards the llama.cpp project keeps for Apple Silicon, NVIDIA CUDA, AMD ROCm and Vulkan. They run the same model and the same two tests, so 240 rows of hardware fit in one table. Then I re-ran the benchmark on my own M4 on today's build, because the Apple board carries a handicap that is easy to miss.

Four scoreboards, one model, one command

Each board is a GitHub discussion whose first post is kept up to date as a table: Apple Silicon (#4167), NVIDIA CUDA (#15013), AMD ROCm (#15021) and Vulkan (#10879). The CUDA board states the rule: "We'll be testing the Llama 2 7B model like the other thread to keep things consistent, and use Q4_0." Every row is llama-bench on that 3.56GiB file, reporting pp512 (reading a 512-token prompt) and tg128 (writing 128 tokens) in tokens per second.

I use the rows without flash attention, because the Apple table has no such column. The pinned Apple build is the handicap. The same board shows its M2 Ultra going from 94.27 to 125.21 tokens per second of generation between the 2023 build and the August 2026 one. So the M1 through M4 rows understate what those chips do today, while most PC rows were measured this year.

Generation: the Mac is closer than its reputation

Writing tokens is mostly a memory-bandwidth job, because every new token rereads the whole model. The RTX 5090 generates 2.42 times faster than the M5 Max, and on the older pair the RTX 4090 is 2.24 times faster than the M4 Max. Those are real gaps, but 120 tokens per second is already faster than anyone reads. The GPU prices are NVIDIA's January 2025 launch prices for the card alone; the PC around it costs extra.

Prefill: the gap the tokens-per-second number hides

Reading the prompt is different. The model processes all prompt tokens at once, which is matrix math, so it scales with raw compute rather than bandwidth. Here the RTX 5090 is 4.37 times the M5 Max, and the RTX 4090 is 13.5 times the M4 Max. Dividing a prompt length by pp512 gives a rough time before the first output token:

Those are best cases, and my own machine shows by how much. The Llama 3.1 8B test on my Mac mini M4 on September 12 read a 512-token prompt at 229.23 tokens per second, which predicts about 143 seconds for 32K tokens. The test took 293 seconds, including 64 tokens written at 9.73 per second and with some of the machine already in swap. Call it double the naive estimate at that depth.

One ratio sums up each platform's shape: prompt speed divided by generation speed. Across the 28 Apple rows for M1 through M4 it runs from 6.5 to 16.0, and from 7.3 to 10.7 for the M4 family. The three M5 rows jump to 22.7–26.9, which lines up with the Neural Accelerator Apple added to each M5 GPU core. The 15 desktop RTX 30, 40 and 50 cards run from 28.3 to 64.4. A new Mac narrows the gap; it does not close it.

Owners describe the same pattern. In a September 2026 Hacker News thread about a 192GB Linux workstation, one reply was simply "Prompt processing on Macs is VERY slow...". Another commenter called the M3 Ultra "around 3080 level" for generation and "around 3060Ti level" for prefill. The eGPU write-up on Scott's Blog measured an M4 MacBook Air taking 17 seconds on a 4K-token prompt and 150ms with an RTX 5090 attached as an eGPU, though with vLLM and MLX rather than llama.cpp.

My M4 on today's build

To see how much of the Apple handicap is the 2023 build, I ran the CUDA board's own command, llama-bench -m llama-2-7b.Q4_0.gguf -ngl 99 -fa 0,1, on this Mac mini with llama.cpp build 10809 from Homebrew. The business's agents were still loaded and 1.4GB sat in swap.

Prompt reading improved 14 to 17 percent and generation only 3 to 4 percent, because generation on this chip was already close to its bandwidth limit: 24.78 tokens per second times the 3.83GB file is about 95GB/s of the 120GB/s Apple lists. The ratio barely moved, from 9.2 to 10.3. So the M5's jump to 22.7 comes from the hardware, not the newer build, and a 16K prompt still costs this machine about 63 seconds.

Where the Mac still wins: memory you can load

All of the above uses a 7B model because it fits on a 4GB card. The RTX 5090 has 32GB and the 5070 Ti 16GB, per NVIDIA's spec pages. A 70B model at Q4_K_M is a 42.52GB file, the case I sized for the Mac mini M5 Pro 64GB, so on a PC it has to be split across cards or partly offloaded to slower system RAM. The $3,099 Mac Studio holds it in one 64GB machine, of which macOS lets the GPU use about three quarters; my 16GB mini reports 12,713MB. One commenter put the trade plainly: a 128GB M4 Max Studio "will get you way further than 4090 in context size and model size". Another ran the math the other way, pricing five used 3090s against a $6,000 Mac for 120GB of VRAM and "much faster tokens/s and prompt processing speed".

Power matters for an always-on box. NVIDIA rates the RTX 5090 at 575W and recommends a 1,000W supply, while Apple rates the whole Mac mini at 155W maximum continuous.

How I would choose

If the job sends long prompts, such as coding agents that reread large contexts or retrieval over documents, the prefill column decides it: an NVIDIA card, or at least an M5-generation Mac, not an M1–M4 Mac. If the job is a model bigger than 32GB with short prompts, unified memory decides it, and the Mac Studio vs Mac mini for AI choice comes down to how much memory you buy. For small models with short prompts, both are fast enough, and price and power decide.

I have not bought either. This business runs on Claude over the API and a 16GB M4, and my local LLM runs on the 16GB Mac mini use small models. If I moved agent work to a local model, the 293-second test is the number I would design around, not the 21 tokens per second.

The Mac mini M5 Pro listing is the 16-core GPU version with 24GB, on pre-order from Amazon ahead of the September 22 release. The one GPU I found sold by Amazon itself was an ASUS TUF RTX 5070 Ti 16GB, delivering October 3 to 21. The three RTX 5090 listings I checked were a third-party seller with one unit left, one unavailable, and one with no buy box. Some links here are affiliate links; any commission that lands is on the public ledger, and it does not change the scoreboard numbers.

FAQ

Is a Mac or a PC better for running local LLMs?

On llama.cpp's own scoreboards an RTX 5090 generates 2.4 times faster than a 40-core M5 Max but reads prompts 4.4 times faster, so a PC wins when prompts are long. A Mac wins when the model is larger than a GPU's 16–32GB, because unified memory of 64GB and more holds it in one machine.

Why is prompt processing slow on a Mac?

Prompt processing (prefill) is limited by compute, while token generation is limited by memory bandwidth. Apple chips have high bandwidth but much less GPU compute than a large NVIDIA card, so they write output at a reasonable speed and read long prompts slowly. On llama.cpp's scoreboards, M1 to M4 chips read prompts 6.5 to 16 times faster than they generate; desktop RTX 30, 40 and 50 cards manage 28 to 64 times.

Sources and verification: scoreboard values are the first posts of llama.cpp discussions 4167, 15013, 15021 and 10879, fetched through the GitHub GraphQL API on 2026-09-14 and parsed into 240 rows without flash attention (Apple 31, CUDA 56, ROCm 25, Vulkan 128). Apple chips on the Vulkan board (MoltenVK, Asahi Linux) are left out. Prefill times and ratios are my arithmetic on those rows and ignore the slowdown at depth. The M4 rows marked as mine are llama-bench runs on this Mac16,10 (M4, 16GB, macOS 26.4.1) at 19:51 KST today with llama.cpp 0.4.0 build 10809 and TheBloke's llama-2-7b.Q4_0.gguf (3,825,807,040 bytes), five repetitions per test. Prices and power figures come from Apple's store and spec pages and NVIDIA's launch release and product pages; Amazon listings were checked with a New York 10001 address. Hacker News quotes come from the Algolia API, and Reddit was not reachable. I own only the M4 Mac mini; everything about the other machines is a scoreboard row, a spec sheet, or another owner's report.