Ollama vs llama.cpp: Same Engine, 21 tok/s, One Silent Trap

I expected this to be a speed test. Every ollama vs llama.cpp comparison I found ranks the two by tokens per second, so I installed Ollama 0.34.2 next to the Homebrew llama.cpp already on this 16 GB M4 Mac mini, pointed both at the same Llama 3.1 8B Q4_K_M file, and ran the same prompts 30 times. Generation speed came out at 20.91 tokens per second for Ollama and 20.54 for llama.cpp. The first 80 characters of every output matched.

The reason showed up in the process list before the first benchmark finished: Ollama was running llama.cpp. The difference that did show up was a 5,420-token prompt that Ollama answered wrongly with HTTP 200, after throwing away 3,371 tokens from the middle of it.

Ollama vs llama.cpp is now one engine in two wrappers

When the first request arrived, the Ollama log printed using llama-server for model, and ps showed the child it started:

/opt/homebrew/Cellar/ollama/0.34.2/libexec/lib/ollama/llama-server \
  --model ~/work/llm/ollama-models/blobs/sha256-7b064f58... \
  -c 4096 -np 1 --flash-attn auto -b 512 -ub 512 --context-shift --keep 4

That is llama.cpp's own HTTP server. Its startup banner reads build 10969 (391fac164), and the Homebrew formula pins exactly that tag as a resource labelled "Pinned dependency required by llama-server." The change dates to Ollama 0.30.0 on 2026-05-13, whose notes describe "improved compatibility and performance using llama.cpp," alongside the MLX engine Ollama uses for some models on Apple silicon. For a GGUF file on a Mac, Ollama now tokenizes, templates, schedules and proxies; the math happens in llama.cpp.

One detail runs against the usual framing. My standalone llama.cpp is Homebrew 0.4.0, build 10809. The copy bundled inside Ollama is 160 builds newer. Installing Ollama got me a more recent llama.cpp than installing llama.cpp did.

Same GGUF, 30 runs: the benchmark

Both servers loaded the same bartowski Q4_K_M file (4,920,739,232 bytes). A 28-line Python client sent a raw prompt, with Ollama's chat template switched off via raw: true, greedy sampling and 128 generated tokens. Each of the five repetitions started with a different first line so no run could reuse a cached prompt. The short prompt was 41 tokens; the long one was 3,626 tokens of our own persona file with a summarization request.

The third row settles it. Started with the flags Ollama passes (-c 4096 -np 1 -b 512 -ub 512 --flash-attn auto), standalone llama.cpp matched or beat Ollama by a fraction of a token per second. The 1 to 2% in the first two rows comes from configuration and build, not from anything Ollama adds, and it is the same 21 tokens per second I got from llama-bench in my Llama 3.1 8B Mac mini M4 test ten days ago. For all ten prompts, the first 80 characters of output were identical across the three setups.

The published numbers agree once you check the version. ModelPiper measured a 2 to 7% spread on an M2 Max, but with Ollama 0.23.4, before the switch. Mozilla.ai tested Ollama 0.34.0 on an M4 Max, an L40S and a Steam Deck and found all servers "within a few percent of each other on prompt processing," with build flags and shader toolchains moving results by tens of percent. So if you read that Ollama is 15% slower than llama.cpp on a Mac, check the date first.

Where they differ: defaults you never typed

The memory column is the first real difference. Plain llama-server -m file.gguf fits the context to free memory and logged n_slots = 4, n_ctx_slot = 50944. Ollama saw 11.8 GiB of GPU-addressable memory and logged default_num_ctx=4096, which matches the tiers in its context length docs (under 24 GiB gets 4k). Llama 3.1 8B's f16 KV cache costs 128 KiB per token, so that is 512 MiB for Ollama against roughly 6.2 GiB for llama-server. It explains most of the 5.3 GB RSS gap. I covered what those extra gigabytes buy in llama.cpp KV cache quantization.

Two smaller ones. ollama create from a local GGUF copied the whole 4.6 GiB file into Ollama's blob store; the blob is byte-identical, named after the file's SHA-256, so every model you import this way costs its size twice. Ollama also passes -np 1, one request at a time per model, while llama-server's automatic default is 4 slots, which on a base M4 is the slow range for llama.cpp parallel requests. For a single user, Ollama's choice is the better one.

The 5,420-token prompt Ollama answered with HTTP 200

I put one line at the top of a 5,420-token document, SECRET CODE: the vault number is 7741., and asked at the bottom what the number was. llama-server with its 50,944-token context answered 7741. Ollama at its 4,096 default gave different results depending on the endpoint:

The generate endpoint kept the first 4 tokens and the last 2,046 and dropped everything between them, then answered from what was left. The client saw a normal response. The only trace was one line in the server log:

level=WARN source=llama_server.go:318 msg="truncating input prompt" limit=2050 prompt=5421 keep=4 new=2050

The 2,050 is not arbitrary. In llama_server.go, an over-long prompt is cut to numCtx - (numCtx - numKeep) / 2, with a comment saying this matches "the old runners' first context shift." So a prompt that is one token too long loses about half the window. The chat endpoints reach llama-server untruncated and get its 400, which is the behavior I'd want everywhere. Passing "shift": false or "truncate": false on /api/generate also turned my request into a 400.

This is known upstream. Issue #18399, opened ten days ago, reports the same 2,050 on Linux with 0.34.0 and proposes a server-wide switch to refuse over-long prompts. Issue #17427 documented the half-window formula in July and was closed after a maintainer explained the two trimming stages. Issue #7043, asking for a warning the caller can see, has been open since September 2024. A search for the log string across the repo returns 75 issues. The Ollama context docs don't mention truncation at all.

The practical rule: if you call /api/generate, compare prompt_eval_count with the token count you think you sent, or set num_ctx above your longest prompt. Also don't trust Ollama's prompt speed on repeated prompts. Sending the identical long prompt again, it reported 3,622 tokens evaluated at 67,586 tokens per second, because the cached prefix counts as evaluated.

Which one I'd run on a 16 GB Mac

Speed doesn't decide it. On this machine the question is whether you want Ollama's model registry, its 4k context and its one-request-per-model default, or llama-server's flags, which you have to set yourself. I kept llama-server for the benchmarks on this blog because I can pin exactly what it does. For a single-user chat app, Ollama's defaults are easier on 16 GB than llama-server's. If you keep Ollama running as a background service, note that the Homebrew service definition adds OLLAMA_FLASH_ATTENTION=1 and OLLAMA_KV_CACHE_TYPE=q8_0, which a plain ollama serve does not, and that environment variables for a launchd-started Ollama are their own problem, covered in launchctl setenv not working.

The community argument about Ollama has mostly been about credit and support rather than speed, from the 2025 Hacker News thread on Ollama and the llama.cpp license (202 points) to a comment last month from a user who switched after Ollama dropped ROCm support for their card. On a Mac in September 2026, both tools run the same llama.cpp code at the same speed.

FAQ

Is Ollama slower than llama.cpp?

Not on this machine. With the same Llama 3.1 8B Q4_K_M file on a base M4 Mac mini, Ollama 0.34.2 generated 20.91 tokens/s and llama-server 20.54, and llama-server with Ollama's flags hit 21.06. Since version 0.30.0, Ollama runs GGUF models through llama.cpp's own llama-server, so older benchmarks showing large gaps no longer apply.

Does Ollama use llama.cpp?

Yes. Ollama 0.34.2 starts a bundled llama-server (llama.cpp build b10969) as a child process for GGUF models, with flags such as -c 4096 -np 1 --flash-attn auto. On Apple silicon it also has an MLX engine for some models.

Why does Ollama ignore part of my long prompt?

On machines with under 24 GiB of GPU memory Ollama defaults to a 4,096-token context. A longer prompt sent to /api/generate is cut to about half the window (2,050 tokens at 4,096) with HTTP 200 and only a server-log warning. Raise num_ctx or OLLAMA_CONTEXT_LENGTH, or pass "truncate": false to get an error instead.

Sources and verification: every number here is my own measurement on this Mac16,10 (base M4, 16 GB, macOS 26.4.1) on 2026-09-22 between 15:00 and 15:20 KST, with other work resident and 2.8 GB in swap. Ollama 0.34.2 was installed from Homebrew for this test and run as a plain ollama serve; llama.cpp was Homebrew 0.4.0 build 10809. Both used bartowski's Llama 3.1 8B Instruct Q4_K_M. Each benchmark configuration ran 5 times per prompt; the truncation test ran once per endpoint. The truncation formula is quoted from Ollama's source at tag v0.34.2, and the bundled llama.cpp build from the Homebrew formula and the runner's own banner. Not tested: Ollama's MLX engine, the Homebrew service environment, concurrent requests, and larger Macs.