( )⚙ D12158 color: don't infer vt status from TERM on Windows

This is an archive of the discontinued Mercurial Phabricator instance.

color: don't infer vt status from TERM on Windows
ClosedPublic

Authored by mitchhentgesmozilla on Feb 9 2022, 1:15 PM.

Details

Summary

Previously, it was assumed that Windows environments with
"xterm" in the TERM environment variable meant that either
"virtual terminal mode" was already enabled, or that
we are running in an environment that didn't need a "virtual
terminal mode" (such as mintty, that interprets ANSI sequences
itself).

However, modern Cygwin and MSYS2 set TERM=xterm when using the
Command Prompt as the terminal, which needs "virtual terminal
mode" to be manually enabled. However, due to (issue6640),
the vtmode wasn't being enabled.

This patch ensures that we always try to enable vtmode on
Windows regardless of the state of TERM, so that:

  • ANSI-based colors work in modern Cygwin/MSYS2 (with Command Prompt), and
  • The vtmode is unnecessarily set when running in a different terminal such as mintty, but it is simply redundant and doesn't appear to have ill effects.

Diff Detail

Repository
rHG Mercurial
Branch
default
Lint
No Linters Available
Unit
No Unit Test Coverage

Event Timeline

spectral accepted this revision.Feb 9 2022, 2:55 PM
spectral added a subscriber: spectral.

This really feels like a bug that they're advertising support for xterm-compatible escape codes, and then not setting everything up to handle it properly, but for all I know it's not possible for them to do so (i.e. maybe Windows requires that each Process enable vtmode for every Console it cares about, it can't be done "for us" by these tools)

This really feels like a bug that they're advertising support for xterm-compatible escape codes, and then not setting everything up to handle it properly, but for all I know it's not possible for them to do so (i.e. maybe Windows requires that each Process enable vtmode for every Console it cares about, it can't be done "for us" by these tools)

Yeah, it looks like Cygwin sets the term to xterm-256color based on whether the console supports it, not based on whether it's currently active.
I suppose that this makes sense considering that it's possible for children processes in the same terminal to SetConsoleMode(...) and enable/disable virtual terminal support without changing the $TERM.

Going into more theoretical territory, my team is currently investigating an issue where it seems like each cygwin binary that runs is implicitly unsetting virtual terminal support, which could be related. However, even if that's changed, due to the malleable nature of "virtual terminal mode", I still think that aggresively enabling it in HG-land is a good idea :)

Alphare accepted this revision.Feb 14 2022, 5:07 AM
This revision is now accepted and ready to land.Feb 14 2022, 5:07 AM