Whisper vs Mac Dictation: 8.5% Word Errors vs 2.7% on an M4

I ran 524 LibriSpeech clips through five speech engines on this M4 Mac mini to settle whisper vs Mac dictation. The model Apple documents as the one behind system Dictation got 8.53% of words wrong. That was the worst result in the test, behind even whisper.cpp's smallest English model I tried (base.en, 4.86%). Whisper large-v3-turbo got 2.66% wrong. The best result was also Apple's: SpeechTranscriber, the newer engine in macOS 26, got 2.17%, and it finished in 79 seconds where turbo needed 568.

So "Whisper or Apple" is the wrong split. Apple ships two engines, and the Dictation key does not use the good one.

First, the run that told me nothing

My first test fed both Apple engines a sentence I generated with the say command. "Testing the legacy recognizer on device" came back as "Testing tore, go see the cognizer on tibase." from one engine and "Testing to base" from the other. That is not a fair test. Synthetic voices are not what these models were trained on. I threw it out and switched to LibriSpeech test-clean (CC BY 4.0): recordings of people reading audiobooks, each with a reference transcript.

The second failure was the old API. SFSpeechRecognizer with requiresOnDeviceRecognition = true returned this in 0.056 seconds and transcribed nothing:

Error Domain=kLSRErrorDomain Code=201 "Siri and Dictation are disabled"

This Mac is headless and Dictation was never switched on in Keyboard settings, so the legacy path is locked. The new SpeechAnalyzer modules ran anyway. SFSpeechRecognizer.authorizationStatus() stayed at 0 (not determined) the whole time, and no permission prompt appeared. That matters on a server-style Mac like mine. A permission check that nobody can answer is how a screencapture job failed on TCC on this same machine.

The setup

• Audio: LibriSpeech test-clean sorted by utterance ID, every fifth clip. That gives 524 of 2,620 clips, all 40 speakers, 10,607 reference words, 65.2 minutes of audio. Converted to 16 kHz mono WAV with afconvert.
• Apple: a 60-line Swift harness using SpeechTranscriber (preset .transcription) and DictationTranscriber (preset .longDictation). One file at a time, fully on-device.
• Whisper: whisper.cpp 1.9.5 from Homebrew on Metal, default settings (beam 5, 4 threads), with base.en, small.en and large-v3-turbo q5_0.
• Scoring: lowercase, strip punctuation, spell out digits, expand Mr/Mrs/Dr/St, then word-level edit distance against the reference.

Why call DictationTranscriber "Mac dictation"? Apple's documentation says it "uses the same speech-to-text machine learning models as system dictation features do, or as SFSpeechRecognizer does when it is configured for on-device operation." I can't press a Dictation key on a Mac with no keyboard or screen, so this is the documented model, not the keyboard feature. The feature may add things a file test can't see, like your contact names.

Results

Engine	WER	Clips with zero errors	Time for 65.2 min	Speed	Download
Apple SpeechTranscriber	2.17%	382 / 524	79 s	49x real time	OS asset
whisper large-v3-turbo q5_0	2.66%	367 / 524	568 s	6.9x	574 MB
whisper small.en	3.60%	325 / 524	192 s	20x	488 MB
whisper base.en	4.86%	278 / 524	86 s	45x	148 MB
Apple DictationTranscriber	8.53%	173 / 524	116 s	34x	OS asset

Most of the Dictation model's errors were wrong words: 725 substitutions, 84 dropped words and 96 extra words. They were not small spelling slips. Three clips, with the reference first, then the Dictation model, then SpeechTranscriber:

• "Eliza Parsons shook her head" became "Eliza Parsons Sugarhead". SpeechTranscriber got it right.
• "I boldly lighted my cheroot" became "I bold delighted my trout". SpeechTranscriber got it right.
• "A Montfichet, a Montfichet, Gamewell to the rescue" became "I'm not fishy. I'm not fishy game well to the rescue." SpeechTranscriber wrote "A Montfichet, a Montfichet game well to the rescue."

Punctuation differs too, and my error rate ignores it. SpeechTranscriber ended 519 of 524 clips with a period, question mark or exclamation point. The Dictation model did that on 68. If you paste dictated text into an email, you fix that by hand.

Whisper's turbo model is close to Apple's best engine on accuracy. The cost is time and memory. In a separate 20-clip run, whisper.cpp's peak memory was 396 MB for base.en and about 890 MB for small.en and turbo. The Apple engines barely show up in their own process (6.8 MB). The work happens in a system XPC service, localspeechrecognition, which I saw at 97 to 132 MB during the runs. On a 16 GB machine that also runs a local LLM, that difference is worth knowing.

Does this match anyone else's numbers?

Yes, closely. In July, the team behind the Inscribe transcription app benchmarked SpeechAnalyzer against Whisper on all 5,559 LibriSpeech test utterances on an M2 Pro. Their results were 2.12% for SpeechAnalyzer on test-clean and 9.02% for the legacy SFSpeechRecognizer. Mine are 2.17% and 8.53%. According to Apple's documentation, DictationTranscriber and on-device SFSpeechRecognizer use the same model, so those two pairs should line up, and they do. Their Whisper Small (3.74%, through WhisperKit) and my small.en (3.60%, through whisper.cpp) are also close. Their test had no DictationTranscriber column and no large-v3-turbo, which is the gap this run fills. Their original domain, get-inscribe.com, now redirects to lyonesse.app, so older links to that benchmark may not work. Their HN thread (570 points) is still the best discussion of it.

What people who dictate all day report

The numbers above are clean read speech. Users in recent threads describe different problems:

• One commenter in the October BetterWispr Show HN thread says NVIDIA's Parakeet is "blindingly fast" but they still use Whisper "because it just does better for accented voices." LibriSpeech is mostly native English readers, so my test can't speak to accents.
• In the Yap thread, the author points out that built-in Dictation discloses it "sends information like your voice input, contacts, and location to Apple." Apple's Dictation guide tells you to check Keyboard settings to see whether your dictation stays on the device. Apps built on SpeechAnalyzer, such as Yap, never use that path.
• In the same BetterWispr thread, another commenter says SpeechAnalyzer "works quite well" and asks whether people find it lacking. Going by my error rates, it isn't. Its weaker points are language coverage (the Inscribe write-up counts around 30 SpeechTranscriber locales) and the macOS 26 requirement.

Which one to use

• You press the Dictation key in macOS: you get the 8.5% model. It's fine for short messages. For long text, expect to fix about one word in twelve.
• You're on macOS 26 and want local English transcription: use an app built on SpeechAnalyzer/SpeechTranscriber, or write your own. In my test it was the most accurate and the fastest, and there's no model file to manage.
• You need other languages, accents, an older macOS, or Linux: use Whisper. large-v3-turbo is within half a point of Apple's best engine on clean English, but it runs about 7x slower on an M4.

The core of my harness, for anyone who wants to rerun this on their own audio:

let t = SpeechTranscriber(locale: Locale(identifier: "en-US"), preset: .transcription)
// swap in DictationTranscriber(locale:, preset: .longDictation) for the Dictation model
if let req = try await AssetInventory.assetInstallationRequest(supporting: [t]) {
    try await req.downloadAndInstall()   // first run only, no prompt
}
let analyzer = SpeechAnalyzer(modules: [t])
let text = Task { var s = ""; for try await r in t.results where r.isFinal { s += String(r.text.characters) }; return s }
let file = try AVAudioFile(forReading: url)
if let last = try await analyzer.analyzeSequence(from: file) {
    try await analyzer.finalizeAndFinish(through: last)   // without this, results never finishes
}
print(try await text.value)

If you're picking a microphone for dictation rather than an engine, I compared driver needs in USB microphone for Mac. The Mac mini has no built-in mic, so dictating on one needs something plugged in.

FAQ

Is Whisper more accurate than Mac dictation?

On clean English speech, yes. On 524 LibriSpeech clips on an M4 Mac mini, the model Apple uses for system Dictation got 8.53% of words wrong. Whisper base.en got 4.86% wrong, small.en 3.60%, and large-v3-turbo 2.66%. Apple's newer SpeechTranscriber engine, which apps can use on macOS 26, beat all of them at 2.17%.

Does Apple's SpeechAnalyzer need microphone or speech recognition permission for files?

Not in my test. With speech recognition authorization still "not determined", SpeechTranscriber and DictationTranscriber transcribed audio files on macOS 26.4.1 without a prompt. The older SFSpeechRecognizer API failed with "Siri and Dictation are disabled" because Dictation was off on that Mac. Recording live from a microphone is a separate permission.

How fast is whisper.cpp on an M4 Mac mini?

With Homebrew whisper.cpp 1.9.5 on Metal and default settings, 65.2 minutes of audio took 86 seconds with base.en, 192 seconds with small.en and 568 seconds with large-v3-turbo q5_0. That is 45x, 20x and 6.9x real time. Apple's SpeechTranscriber took 79 seconds.

How this was measured: a Mac mini M4 with 16 GB RAM, running macOS 26.4.1 headless, on 2026-10-19. Audio was LibriSpeech test-clean (every fifth utterance by ID: 524 clips, 40 speakers, 3,912 seconds) from the OpenSLR corpus under CC BY 4.0. All engines ran on-device, one after another, with nothing else heavy running. Times are wall clock for the whole set. For the Apple engines that includes setting up each clip. The WER normalizer is my own and simpler than Whisper's, so absolute numbers can differ from published ones by a few tenths of a point. The comparison between engines uses the same scorer for all five. Peak memory comes from a separate 20-clip run with /usr/bin/time -l. The Swift harness, the scorer and every per-clip transcript are saved with this post's research notes.