Don't apply system-dependent newline conversion when writing files

[mojavelinux]

When Asciidoctor writes files (converted output and stylesheets) on a non-Unix operating system (Windows or macOS), it converts universal (Unix) newlines (i.e., line feeds) to the system-dependent newline. On Windows, it converts them to CRLF (\r\n) and on macOS, it converts them to CR (\r).

This behavior is coming from Ruby. Ruby tries to be "smart" and convert the newlines to match the system default (\n for Unix, \r for macOS, and \r\n for Windows) whenever an application writes a file to disk using File.write unless it is configured otherwise. See https://docs.ruby-lang.org/en/3.3/File.html#class-File-label-Data+Mode

UPDATE: After researching this more, while Ruby can convert LF to CR on write, that conversion has to be explicitly enabled. So the behavior described in this issue only applies to Windows.

This is not only antiquated behavior, but it's not what we have documented and communicated. Asciidoctor should not be system-dependent. It should always create files the same way, regardless of the operating system on which it is run. Asciidoctor already prepares the output exactly how we want it and Ruby should write it to the file as prepared.

There are two ways in Ruby to control the behavior I just described. The first way is to write files in what's know an "bin" mode (i.e., wb). This is not to be confused with writing binary content. It basically just means "write the content as I gave it to you and don't attempt to modify it". This is what we want. The other way is to pass the option newline: :universal to the File.write method. However, this option is poorly documented, isn't recognized in older versions of Ruby, and it still does some manipulation of the output.

Asciidoctor has to explicitly disable the newline conversion or else Ruby is going to apply it. Thus, the proposed change is to update the value of the constant FILE_WRITE_MODE as follows:

FILE_WRITE_MODE = RUBY_ENGINE_OPAL ? 'w' : 'wb:utf-8'

This mode is used for all file write operations in Asciidoctor.

The reason for the exception for Opal (and hence Asciidoctor.js) is because Node.js doesn't have this behavior.

[abelsromero]

and on macOS, it converts them to CR (\r)

For what is worth I tested 2.0.20, Ruby 3.2.2 and it create an LF HTML.

[mojavelinux]

Thanks for pointing that it. It seems like that either changed in Ruby for macOS at some point or it never worked that way. Regardless, it's inconsistent with the Ruby documentation and further justification for why we should avoid this conversion in the first place. As I stated above, it's an antiquated practice. Using a line feed as the newline is consistent with Asciidoctor's goal of universality (just as forcing UTF-8) and ensures everyone and every environment has the same experience when using the software.

[kriegaex]

It should come as no big surprise that the MacOS line break is LF, because Apple switched from CR on classic MacOS (1-9) to POSIX standard LF with MacOS X, i.e. since 2001. (I never used a Mac in my whole life, but I think it still understands the old CR.) I much doubt that there are a lot of classic MacOS systems capable of running Asciidoctor around in 2024. 😉