Page MenuHomePhabricator

dispatch: force \n for newlines on sys.std* streams (BC)

Authored by indygreg on Mar 29 2020, 4:31 PM.



The sys.std* streams behave differently on Python 3. On Python 3,
these streams are an io.TextIOWrapper that wraps a binary buffer
stored on a .buffer attribute. These TextIOWrapper instances
normalize \n to os.linesep by default. On Windows, this means
that \n is normalized to \r\n. So functions like print() which
have an implicit end='\n' will actually emit \r\n for line endings.

While most parts of Mercurial go through the ui.write() layer to
print output, some code - notably in extensions and hooks - can use
print(). If this code was using print() or otherwise writing to
sys.std* on Windows, Mercurial would emit \r\n.

In reality, pretty much everything on Windows reacts to \n just fine.
Mercurial itself doesn't emit \r\n when going through the ui layer.
Changing the sys.std* streams to not normalize line endings sounds
like a scary change. But I think it is safe. It also makes Mercurial
on Python 3 behave similarly to Python 2, which did not perform \r\n
normalization in print() by default.

.. bc:: sys.{stdout, stderr, stdin} now use \n line endings on Python 3

Diff Detail

rHG Mercurial
Automatic diff as part of commit; lint not applicable.
Automatic diff as part of commit; unit tests not applicable.

Event Timeline

indygreg created this revision.Mar 29 2020, 4:31 PM

As scary as this patch sounds, I'm pretty sure it is safe, as I believe it restores compatibility with Python 2. Changing sys.std* to be binary streams instead of text streams would be a bigger BC break. And that is not a change I want to make, as this would invalidate assumptions in 3rd party code about the behavior of these streams on Python 3!

This revision was not accepted when it landed; it landed in state Needs Review.
This revision was automatically updated to reflect the committed changes.