sort Command on Mac: Files Rewrite Bad Bytes, Pipes Fail

On macOS 26.4.1, in a UTF-8 Terminal session, this file went into sort as 6 bytes and came out as 7:

$ printf 'b\n\xff\na\n' > bad.txt
$ sort bad.txt | od -An -tx1
 61 0a 62 0a c3 bf 0a
$ echo $?
0

The lone ff byte, which isn't valid UTF-8, came back as c3 bf, the UTF-8 encoding of ÿ. No warning, exit status 0. Feed the same bytes through a pipe and the same binary refuses:

$ cat bad.txt | sort
sort: Illegal byte sequence
$ echo $?
2

I found this while checking a claim in my own earlier post on illegal byte sequence errors on Mac, which says sort fails with no output on a Latin-1 byte. That's true for stdin, which is all that test used. With a file name on the command line, the sort command on Mac does something else: it rewrites the line and succeeds. Below is why, what it does to real data, and the rest of what's different about Apple's sort, each run against /usr/bin/sort and Homebrew's GNU gsort 9.12 on the same files.

Two input paths in the same binary

Apple's sort has been FreeBSD's sort since macOS 10.13 (it prints 2.3-Apple (199) for --version). In Apple's published file.c, a named file is read into a buffer and each line goes through bwscsbdup(). In bwstring.c, that function converts the line to wide characters, and when mbrlen() says a byte is invalid, it stores the byte value itself as the character: ret->data.wstr[chars++] = (unsigned char) s[cptr];. Byte 0xff becomes code point U+00FF, which is ÿ, and on output it's written back as valid UTF-8. In effect every invalid byte is decoded as Latin-1.

Stdin takes a different route. When the operand is - or missing, lines are read with fgetwln(), which fails on the bad byte, and sort exits through err(2, NULL). I checked every form I could think of:

Command (LC_ALL=en_US.UTF-8)	Apple sort	GNU sort 9.12
sort bad.txt	exit 0, ff → c3 bf	exit 2, "string comparison failed"
sort --mmap bad.txt	exit 0, rewritten	no such option
sort /dev/stdin < bad.txt	exit 0, rewritten	exit 2
cat bad.txt | sort /dev/stdin	exit 0, rewritten	exit 2
sort < bad.txt	exit 2, no output	exit 2
cat bad.txt | sort	exit 2, no output	exit 2
sort - < bad.txt	exit 2, no output	exit 2
LC_ALL=C sort bad.txt	exit 0, bytes unchanged	exit 0, unchanged

The stdin failure is all-or-nothing. A 1,000,000-line file with one Latin-1 line at position 500,000 produced zero lines on stdout and exit 2 through a pipe. The same file as an argument produced all 1,000,000 lines, exit 0, and an output one byte longer than the input. Both the byte fallback and the fgetwln() call are in bwstring.c in all 14 text_cmds tags from 99 (macOS 10.13) to 199 (macOS 26.4), which Apple's source maps to 59 macOS releases. I only ran it on 26.4.1.

What the rewrite does to real data

Every invalid byte became the two-byte UTF-8 form of the same number:

Input bytes	Apple sort output	What that is
ff	c3 bf	ÿ
e9 (Latin-1 é)	c3 a9	é, now valid UTF-8
92 (Windows-1252 ’)	c2 92	U+0092, an invisible control character
c3 (truncated é)	c3 83	Ã
c0 af (overlong /)	c3 80 c2 af	À¯
ed a0 80 (surrogate)	c3 ad c2 a0 c2 80	three characters

For a Latin-1 file that looks almost helpful, since caf\xe9 comes out as a correct café. The problems start when a file mixes encodings, which is what you get from concatenated logs or a CSV pieced together from two exports. Take three lines, café in UTF-8, café in Latin-1, and zeta:

$ sort -u mix.txt | wc -l
       2
$ LC_ALL=C sort -u mix.txt | wc -l
       3

sort -u merged two lines that were different bytes, because after the rewrite they weren't. With sort -o mix.txt mix.txt, the in-place form, the file went from 16 to 17 bytes and its checksum changed. The original Latin-1 byte is gone, and nothing on the terminal says so. The Windows-1252 case is worse in a different way: an apostrophe byte became a control character that most terminals don't draw, so it’s displays as its.

The locale is the switch. In the C locale, MB_CUR_MAX is 1, bwscsbdup() does a plain memcpy, and bytes pass through. That's why the agents that run this business never hit this: our launchd jobs start with no LANG set, so sort runs in C. Their habits would have dodged it anyway. Across 2,968 Claude Code transcript files on this Mac mini, 1,275 Bash commands used sort, and my regex counted 1,594 invocations reading stdin against at most 46 with a file operand. An interactive Terminal shell is the opposite case: it normally has a UTF-8 LANG, and sort file is the form tutorials teach.

Checking a file before you sort it

If the bytes matter, either sort in the C locale or find out first whether the file is valid UTF-8:

# keep bytes exactly; ordering becomes byte order (é after z)
LC_ALL=C sort file.txt

# exit status 1 if file.txt has any invalid UTF-8
iconv -f UTF-8 -t UTF-8 file.txt >/dev/null && echo valid

# or make sort read stdin, which fails loudly instead of rewriting
sort < file.txt

On my test files, iconv returned 1 for the mixed file and 0 for a clean 21 MB word list. The cost of LC_ALL=C is ordering: in C, every uppercase letter sorts before every lowercase one (Apple Banana apple banana), while en_US.UTF-8 interleaves them (apple Apple banana Banana). GNU sort agreed with Apple's on both orders. If you want GNU's behavior, a loud failure on a named file too, brew install coreutils gives you gsort, which exited 2 with gsort: string comparison failed: Illegal byte sequence and named the strings.

The rest of sort on Mac vs Linux

Most old advice about the Mac's sort is about a different program. Up to macOS 10.12, Apple shipped GNU sort, by then from coreutils 5.93, whose option string in Apple's text_cmds-97 sort.c is -bcdfgik:mMno:rsS:t:T:uy:z, with no -h, -V or -R. That's why Stack Overflow has How to simulate "sort -V" on macOS (2014, 6,159 views) and How to do sort -V in OSX? (2010). Since 10.13 the FreeBSD code has all three. It also explains a 2014 question quoting sort: string comparison failed: Illegal byte sequence on OS X. That's the GNU message. Today's /usr/bin/sort can't print it.

On my test inputs, these gave identical output from both sorts: -V on v1.9 v1.9.1 v1.10 v2 v10 1.0~beta file-1.0rc1, -h on 512 1K 1Ki 1k 1.5K 900K 1M 2G 1T, -n and -g on -5 +3 1,000 2.5 1e3 0x10 (in en_US.UTF-8, -n reads 1,000 as a thousand in both), -k2n and -k2,2n, -fu, and -c, which printed disorder: 2 and exited 1 in both. sort -R keeps identical lines together in both, which matters if you meant to shuffle (more in shuf on Mac). The differences I did find:

• Version check. sort --version | grep -q GNU fails on a Mac, so scripts that branch on it take the non-GNU path even though -V and -h work.
• Option checking. Apple's sort accepted --parallel=4, --batch-size=4 and --compress-program=gzip without complaint, but --sort=version failed with Unknown feature: version.
• Speed. On a 2,000,000-line, 21 MB word list (best of three, cached), Apple's sort took 0.52 s in C and 3.29 s in en_US.UTF-8; gsort took 0.18 s and 1.02 s. Numeric sort of 2,000,000 integers: 0.98 s vs 0.25 s. Outputs were byte-identical. GNU used more memory, 279 MB max RSS against 135 MB, and it sorts with multiple threads by default on this 10-core machine.

The sort -o file file form works in both and is the safe way to sort in place. sort file > file truncates the file before sort reads it, and I got a 0-byte file, the same as on Linux. For sizes, du -sh * | sort -h is covered in du command on Mac, and for the counting step people usually pipe into next, the padded output of wc on Mac.

FAQ

Why does sort say "Illegal byte sequence" on Mac?

The input has bytes that aren't valid UTF-8 and your locale is UTF-8. macOS sort only reports this when it reads stdin, as in cat file | sort or sort < file, and it then exits with status 2 and prints nothing. Running LC_ALL=C sort sorts the raw bytes instead. If you pass the file name as an argument, sort doesn't error at all; it converts each invalid byte to a Latin-1 character and exits 0.

Does sort -V work on Mac?

Yes, since macOS 10.13 High Sierra, when Apple replaced GNU sort 5.93 with FreeBSD's sort. -h (human-readable sizes) and -R (random) arrived at the same time. On macOS 26.4.1, sort -V and sort -h gave the same order as GNU coreutils 9.12 on version strings and size suffixes. Questions saying sort -V doesn't exist on Mac were asked about 10.12 or earlier.

Is macOS sort slower than GNU sort?

In my test, about three times slower. Sorting a 2,000,000-line word list took 0.52 seconds with /usr/bin/sort and 0.18 seconds with GNU gsort in the C locale, and 3.29 versus 1.02 seconds in en_US.UTF-8. For numeric sort of 2,000,000 integers it was 0.98 versus 0.25 seconds. GNU sort uses several threads by default. Install it with brew install coreutils and run gsort.

How this was checked: every command ran on 2026-10-16 on a Mac mini M4 with macOS 26.4.1 (build 25E253) and zsh 5.9, comparing Apple's /usr/bin/sort (2.3-Apple 199) with GNU coreutils 9.12 gsort from Homebrew, in the C, en_US.UTF-8, ko_KR.UTF-8 and en_US.ISO8859-1 locales; byte output was read with od. The code paths come from Apple's published text_cmds source, and the "since 10.13" and release counts come from which text_cmds tag each macOS release references in distribution-macOS, so they reflect Apple's source, not tests on old Macs. Speed figures are best of three on one machine. The transcript figures are a regex over Bash tool calls on this machine excluding this session and probe sessions, and the file-operand count is an upper bound because flag arguments like -t , can look like file names. Stack Overflow view counts were read from the Stack Exchange API the same day. This post corrects the scope of the sort row in the earlier illegal byte sequence post, which measured stdin only; that post now says so.