This is an archive of the discontinued Mercurial Phabricator instance.

exewrapper: avoid directly linking against python3X.dll
ClosedPublic

Authored by mharbison72 on May 20 2021, 10:23 AM.

Details

Summary

Subsequent code calls LoadLibrary() to attempt to load the DLL, but because of
this symbol reference, there is an attempt to load the DLL used during the build
prior to _main() running. This causes the whole process to fail if the DLL
isn't in the standard search path. That also means it will never load the DLL
for HackableMercurial. (Maybe we should get rid of that for py3, since you can
install python for a user without admin rights?)

This could also be resolved by calling GetProcAddress() on the symbol and
dereferencing it, but using the environment variable is consistent with the
*.bat file since fc8a5c9ecee0. (The environment variable persists after the
interpreter is initialized.)

Far more concerning is somehow I've gotten my system into a state where setting
the flag causes any output to the pager to be lost (as if it wasn't set at all)
in MSYS, cmd.exe, WSL, and PowerShell using py3.9.0, but the environment
variable works properly. I'm sure this flag worked on some versions of py3, so
I'm not sure what's going on here. This is might be related to init config
related changes in 3.8[1], since it works with 3.7.8, but fails with 3.8.1.
Somebody who understands encoding issues better than I do should give some
thought to if we need to make some changes to our encoding strategy on Windows
with py3.

With or without the flag/envvar, there is proper output if the command is
directly paged by piping to more.com (in any environment) or less (in MSYS
and WSL), or if paging is disabled with --pager=no. Legacy mode is required
though when Mercurial decides to spin up a pager.

[1] https://bugs.python.org/issue41941

Diff Detail

Repository
rHG Mercurial
Lint
Automatic diff as part of commit; lint not applicable.
Unit
Automatic diff as part of commit; unit tests not applicable.

Event Timeline

mharbison72 created this revision.May 20 2021, 10:23 AM
mharbison72 edited the summary of this revision. (Show Details)May 21 2021, 10:54 AM

@indygreg - you might be interested in the I/O issue. PyOxidizer doesn't seem to be affected (at least when it's building with 3.9.5), but does seem to set Py_LegacyWindowsStdioFlag. Also not sure if there are other issues with the pager code. I remember you looking at it a little over a year ago and saying there were a lot if iffy things, but IDR seeing a patch series after that.

indygreg accepted this revision.May 24 2021, 9:38 PM

Oh wow - I never would have thought that the presence of this symbol would affect linking behavior. But I suppose it makes sense. The functions are probably left as unresolved references but the static variable reference forces early binding or something. I'm not an expert on how PE does symbol binding/resolution. But this patch shouldn't change behavior and if it fixes the problem, it fixes the problem.

This revision is now accepted and ready to land.May 24 2021, 9:38 PM

Oh wow - I never would have thought that the presence of this symbol would affect linking behavior. But I suppose it makes sense. The functions are probably left as unresolved references but the static variable reference forces early binding or something.

Yeah, something like that- we don't reference the functions, rather fill in function pointers of the same name with the value of GetProcAddress(). I only figured it out because dumpbin.exe listed the symbol as imported by the executable, unlike all of the other symbols looked up dynamically.

I'm wondering how far back we need to support older python versions on Windows. This was part of a series to fix things up to work in a venv (since venv creation doesn't copy the python3X.dll into itself, hg.exe always fails unless that python version is on PATH). I'd rather not support multiple init APIs, but the references to encoding in the bug report make me wonder if we do the right things in py3.8+. Also, do you think we should link against python3.dll instead (which might simplify setup.py slightly, but might lead to cryptic failures if cext is built against one version and another is loaded)?