Does the distinction of binary mode for files only exist to deal with pre-de-facto-standardized line endings (i.e. anything other than '\n', most notably Windows' CR LF)? Or is there another purpose I'm not seeing?
That's the only difference I'm aware of, but the C standard allow more differences.
A text stream is an ordered sequence of characters composed into lines, each line consisting of zero or more characters plus a terminating new-line character. Whether the last line requires a terminating new-line character is implementation-defined. Characters may have to be added, altered, or deleted on input and output to conform to differing conventions for representing text in the host environment. Thus, there need not be a one-to-one correspondence between the characters in a stream and those in the external representation. Data read in from a text stream will necessarily compare equal to the data that were earlier written out to that stream only if: the data consist only of printing characters and the control characters horizontal tab and new-line; no new-line character is immediately preceded by space characters; and the last character is a new-line character. Whether space characters that are written out immediately before a new-line character appear when read in is implementation-defined.
I couldn't find similar wording for C++ but I would guess it's similar.
C++ streams work a bit different though because they are normally not "written out immediately" (flushed) based on new line characters. For std::cout they might be if you don't call std::ios::sync_with_stdio(false) but otherwise flushing is more of an explicit thing that happens when you use std::flush or std::endl.
In some implementations, opening a binary file with append mode may initially position the file position indicator for the stream beyond the last data written, because of null character padding.
and for seeks,
A binary stream need not meaningfully support fseek calls with a whence value of SEEK_END.
For a text stream, either offset shall be zero, or offset shall be a value returned by an earlier successful call to the ftell function on a stream associated with the same file and whence shall be SEEK_SET
It allows the "implementation" to trim excessive space characters at the end of lines.
It's interesting. I wonder if they did it just because they thought it might be a good idea or because some vendor already did this and they didn't want to forbid it when the language was standardized.
In 1931 IBM began introducing upper-case letters and special characters [...] The Space character has no punches. [...] IBM and other manufacturers used many different 80-column card character encodings.
If each line had 80 columns and there was no new-line characters (?) and instead they just left the excess columns on each line as spaces (no punches) then it would make sense to just ignore those trailing spaces at the end of each line when reading.