| mharbison72 |
| hg-reviewers |
Newer versions of PyOxidizer use Python 3.9 by default. We
previously pinned the version to 3.8 to facilitate porting to
a new PyOxidizer version and diffing results.
Now that the porting work is complete, let's bump Python
to Python 3.9.
This will effectively change our Windows Inno and WiX Python 3
installers from Python 3.8 to 3.9.
.. bc::
Windows .msi and .exe installers now use Python 3.9 instead of Python 3.8.
| No Linters Available |
| No Unit Test Coverage |
I think I've hit a dead end trying to produce installers. I first updated rustup and pyoxidizer on my laptop that I've used in the past to successfully build pyoxidizer binaries, but it prints an error about a zip file with a date before 1980 not being supported. I switched to a clean system, ran .\contrib\install-windows-dependencies.ps1, and that died because it looks like MS removed the py27 compiler. (Maybe that should be cached somewhere for posterity, but I don't think I have a local copy.) Commenting those references out, I was eventually able to get everything installed, but it reproduced the same error:
py -3 contrib/packaging/packaging.py wix --pyoxidizer-target x86_64-pc-windows-msvc
running build_doc
generating doc/common.txt
generating doc/hg-ssh.8.txt
generating doc/hg-ssh.8.gendoc.txt
generating doc/hg-ssh.8.html
normalizing doc/hg-ssh.8.html to LF line endings
generating doc/hg.1.txt
generating doc/hg.1.gendoc.txt
generating doc/hg.1.html
normalizing doc/hg.1.html to LF line endings
generating doc/hgignore.5.txt
generating doc/hgignore.5.gendoc.txt
generating doc/hgignore.5.html
normalizing doc/hgignore.5.html to LF line endings
generating doc/hgrc.5.txt
generating doc/hgrc.5.gendoc.txt
generating doc/hgrc.5.html
normalizing doc/hgrc.5.html to LF line endings
downloading https://versaweb.dl.sourceforge.net/project/gnuwin32/gettext/0.14.4/gettext-0.14.4-bin.zip to C:\Users\mharbison\mercurial\build\wix-x86_64-pc-windows-msvc\gettext-0.14.4-bin.zip
C:\Users\mharbison\mercurial\build\wix-x86_64-pc-windows-msvc\gettext-0.14.4-bin.zip exists and passes integrity checks
downloading https://versaweb.dl.sourceforge.net/project/gnuwin32/gettext/0.14.4/gettext-0.14.4-dep.zip to C:\Users\mharbison\mercurial\build\wix-x86_64-pc-windows-msvc\gettext-0.14.4-dep.zip
C:\Users\mharbison\mercurial\build\wix-x86_64-pc-windows-msvc\gettext-0.14.4-dep.zip exists and passes integrity checks
resolving 1 targets
resolving target msi
resolving target app
resolving target distribution
resolving target exe
resolving Python distribution Url { url: "https://github.com/indygreg/python-build-standalone/releases/download/20210506/cpython-3.9.5-x86_64-pc-windows-msvc-shared-pgo-20210506T0943.tar.zst", sha256: "4956b8c29ab2841f04cd9aa465e3073be2107ced59636e2a82d35b0c8815e217" }
downloading https://github.com/indygreg/python-build-standalone/releases/download/20210506/cpython-3.9.5-x86_64-pc-windows-msvc-shared-pgo-20210506T0943.tar.zst
Python distribution available at C:\Users\mharbison\AppData\Local\pyoxidizer\python_distributions\cpython-3.9.5-x86_64-pc-windows-msvc-shared-pgo-20210506T0943.tar.zst
reading data from Python distribution...
pip installing to C:\Temp\pyoxidizer-pip-installqd9qZz\install
Using pip 21.1.1 from C:\Users\mharbison\AppData\Local\pyoxidizer\python_distributions\python.4956b8c29ab2\python\install\lib\site-packages\pip-21.1.1-py3.9.egg\pip (python 3.9)
Non-user install due to --prefix or --target option
Created temporary directory: C:\Temp\pip-target-4tvzyhct
Created temporary directory: C:\Temp\pip-ephem-wheel-cache-747t3i1b
Created temporary directory: C:\Temp\pip-req-tracker-0p__z57i
Initialized build tracking at C:\Temp\pip-req-tracker-0p__z57i
Created build tracker: C:\Temp\pip-req-tracker-0p__z57i
Entered build tracker: C:\Temp\pip-req-tracker-0p__z57i
Created temporary directory: C:\Temp\pip-install-7tihpiqe
Processing c:\users\mharbison\mercurial
Created temporary directory: C:\Temp\pip-req-build-_6p7lhtt
DEPRECATION: A future pip version will change local packages to be built in-place without first copying to a temporary directory. We recommend you use --use-feature=in-tree-build to test your packages with this new behavior before it becomes the default.
pip 21.3 will remove support for this functionality. You can find discussion regarding this at https://github.com/pypa/pip/issues/7555.
Added file:///C:/Users/mharbison/mercurial to build tracker 'C:\\Temp\\pip-req-tracker-0p__z57i'
Created temporary directory: C:\Temp\pip-build-env-ibaynzkz
Created temporary directory: C:\Temp\pip-standalone-pip-4zfitqro
ERROR: Exception:
Traceback (most recent call last):
File "C:\Users\mharbison\AppData\Local\pyoxidizer\python_distributions\python.4956b8c29ab2\python\install\lib\site-packages\pip-21.1.1-py3.9.egg\pip\_internal\cli\base_command.py", line 180, in _main
status = self.run(options, args)
File "C:\Users\mharbison\AppData\Local\pyoxidizer\python_distributions\python.4956b8c29ab2\python\install\lib\site-packages\pip-21.1.1-py3.9.egg\pip\_internal\cli\req_command.py", line 204, in wrapper
return func(self, options, args)
File "C:\Users\mharbison\AppData\Local\pyoxidizer\python_distributions\python.4956b8c29ab2\python\install\lib\site-packages\pip-21.1.1-py3.9.egg\pip\_internal\commands\install.py", line 318, in run
requirement_set = resolver.resolve(
File "C:\Users\mharbison\AppData\Local\pyoxidizer\python_distributions\python.4956b8c29ab2\python\install\lib\site-packages\pip-21.1.1-py3.9.egg\pip\_internal\resolution\resolvelib\resolver.py", line 103, in resolve
r = self.factory.make_requirement_from_install_req(
File "C:\Users\mharbison\AppData\Local\pyoxidizer\python_distributions\python.4956b8c29ab2\python\install\lib\site-packages\pip-21.1.1-py3.9.egg\pip\_internal\resolution\resolvelib\factory.py", line 418, in make_requirement_from_install_req
cand = self._make_candidate_from_link(
File "C:\Users\mharbison\AppData\Local\pyoxidizer\python_distributions\python.4956b8c29ab2\python\install\lib\site-packages\pip-21.1.1-py3.9.egg\pip\_internal\resolution\resolvelib\factory.py", line 200, in _make_candidate_from_link
self._link_candidate_cache[link] = LinkCandidate(
File "C:\Users\mharbison\AppData\Local\pyoxidizer\python_distributions\python.4956b8c29ab2\python\install\lib\site-packages\pip-21.1.1-py3.9.egg\pip\_internal\resolution\resolvelib\candidates.py", line 306, in __init__
super().__init__(
File "C:\Users\mharbison\AppData\Local\pyoxidizer\python_distributions\python.4956b8c29ab2\python\install\lib\site-packages\pip-21.1.1-py3.9.egg\pip\_internal\resolution\resolvelib\candidates.py", line 151, in __init__
self.dist = self._prepare()
File "C:\Users\mharbison\AppData\Local\pyoxidizer\python_distributions\python.4956b8c29ab2\python\install\lib\site-packages\pip-21.1.1-py3.9.egg\pip\_internal\resolution\resolvelib\candidates.py", line 234, in _prepare
dist = self._prepare_distribution()
File "C:\Users\mharbison\AppData\Local\pyoxidizer\python_distributions\python.4956b8c29ab2\python\install\lib\site-packages\pip-21.1.1-py3.9.egg\pip\_internal\resolution\resolvelib\candidates.py", line 317, in _prepare_distribution
return self._factory.preparer.prepare_linked_requirement(
File "C:\Users\mharbison\AppData\Local\pyoxidizer\python_distributions\python.4956b8c29ab2\python\install\lib\site-packages\pip-21.1.1-py3.9.egg\pip\_internal\operations\prepare.py", line 508, in prepare_linked_requirement
return self._prepare_linked_requirement(req, parallel_builds)
File "C:\Users\mharbison\AppData\Local\pyoxidizer\python_distributions\python.4956b8c29ab2\python\install\lib\site-packages\pip-21.1.1-py3.9.egg\pip\_internal\operations\prepare.py", line 570, in _prepare_linked_requirement
dist = _get_prepared_distribution(
File "C:\Users\mharbison\AppData\Local\pyoxidizer\python_distributions\python.4956b8c29ab2\python\install\lib\site-packages\pip-21.1.1-py3.9.egg\pip\_internal\operations\prepare.py", line 60, in _get_prepared_distribution
abstract_dist.prepare_distribution_metadata(finder, build_isolation)
File "C:\Users\mharbison\AppData\Local\pyoxidizer\python_distributions\python.4956b8c29ab2\python\install\lib\site-packages\pip-21.1.1-py3.9.egg\pip\_internal\distributions\sdist.py", line 34, in prepare_distribution_metadata
self._setup_isolation(finder)
File "C:\Users\mharbison\AppData\Local\pyoxidizer\python_distributions\python.4956b8c29ab2\python\install\lib\site-packages\pip-21.1.1-py3.9.egg\pip\_internal\distributions\sdist.py", line 62, in _setup_isolation
self.req.build_env.install_requirements(
File "C:\Users\mharbison\AppData\Local\pyoxidizer\python_distributions\python.4956b8c29ab2\python\install\lib\site-packages\pip-21.1.1-py3.9.egg\pip\_internal\build_env.py", line 198, in install_requirements
pip_runnable = ctx.enter_context(_create_standalone_pip())
File "C:\Users\mharbison\AppData\Local\pyoxidizer\python_distributions\python.4956b8c29ab2\python\install\lib\contextlib.py", line 429, in enter_context
result = _cm_type.__enter__(cm)
File "C:\Users\mharbison\AppData\Local\pyoxidizer\python_distributions\python.4956b8c29ab2\python\install\lib\contextlib.py", line 117, in __enter__
return next(self.gen)
File "C:\Users\mharbison\AppData\Local\pyoxidizer\python_distributions\python.4956b8c29ab2\python\install\lib\site-packages\pip-21.1.1-py3.9.egg\pip\_internal\build_env.py", line 63, in _create_standalone_pip
zf.write(child, child.relative_to(source.parent).as_posix())
File "C:\Users\mharbison\AppData\Local\pyoxidizer\python_distributions\python.4956b8c29ab2\python\install\lib\zipfile.py", line 1727, in write
zinfo = ZipInfo.from_file(filename, arcname,
File "C:\Users\mharbison\AppData\Local\pyoxidizer\python_distributions\python.4956b8c29ab2\python\install\lib\zipfile.py", line 517, in from_file
zinfo = cls(arcname, date_time)
File "C:\Users\mharbison\AppData\Local\pyoxidizer\python_distributions\python.4956b8c29ab2\python\install\lib\zipfile.py", line 361, in __init__
raise ValueError('ZIP does not support timestamps before 1980')
ValueError: ZIP does not support timestamps before 1980
Removed file:///C:/Users/mharbison/mercurial from build tracker 'C:\\Temp\\pip-req-tracker-0p__z57i'
Removed build tracker: 'C:\\Temp\\pip-req-tracker-0p__z57i'
error[PYOXIDIZER_PYTHON_EXECUTABLE]: calling pip install
Caused by:
command ["C:\\Users\\mharbison\\AppData\\Local\\pyoxidizer\\python_distributions\\python.4956b8c29ab2\\python\\install/python.exe", "-m", "pip", "--disable-pip-version-check", "install", "--target", "C:\\Temp\\pyoxidizer-pip-installqd9qZz\\install", "--verbose", "C:\\Users\\mharbison\\mercurial\\rust\\hgcli/../.."] exited with code 2
--> C:\Users\mharbison\mercurial\rust\hgcli\pyoxidizer.bzl:99:30
|
99 | exe.add_python_resources(exe.pip_install(["--verbose", ROOT]))
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ PythonExecutable.pip_install()
error: calling pip install
Caused by:
command ["C:\\Users\\mharbison\\AppData\\Local\\pyoxidizer\\python_distributions\\python.4956b8c29ab2\\python\\install/python.exe", "-m", "pip", "--disable-pip-version-check", "install", "--target", "C:\\Temp\\pyoxidizer-pip-installqd9qZz\\install", "--verbose", "C:\\Users\\mharbison\\mercurial\\rust\\hgcli/../.."] exited with code 2A couple of other observations- pyoxidizer is on PATH on my laptop in %USERPROFILE%/.cargo/bin- but on this fresh system, the only copy was in %PROGRAMFILES% and not on PATH. (I assume that's due to an MSI install vs what I assume was a cargo install long ago.) That caused the build to fail earlier with a generic "file not found" type message. I'd feel weird about the install script adjusting PATH; OTOH this already requires Administrator rights and installs stuff globally. (Not using an admin prompt was also one of the errors I hit- IDK if there's a way to detect this up front in the install script and exit with a clearer message.) I'm not too concerned about this stuff, but I wanted to write it down before I forget.
Yeah, I was hitting this zip file error too. I'm not sure what's causing it. But I'm actively looking into fixing it!
The refresh on this series was me applying a topic and pushing so that I could pull to the CI system for experimenting. Sorry about that- I forgot about the refresh.
The behavior seems pretty good, but I wanted to highlight a few things to see what you think.
running light to produce C:\Users\mharbison\mercurial\build\pyoxidizer\x86_64-pc-windows-msvc\release\msi\mercurial-5.8.msi C:\Users\mharbison\mercurial\build\pyoxidizer\wxs\mercurial.wxs(55) : warning LGHT1076 : ICE40: REINSTALLMODE is defined in the Property table. This may cause difficulties. C:\Users\mharbison\mercurial\build\pyoxidizer\wxs\mercurial.wxs(147) : warning LGHT1076 : ICE61: This product should remove only older versions of itself. No Maximum version was detected for the current product. (INSTALLEDMERCURIALPRODUCTS)
IDK if the REINSTALLMODE warning matters, or what it is alluding to.
This warning has existed for years AFAICT. However, the Python code is suppressing the warning via an argument to light.exe. This argument is not specified in PyOxidizer because PyOxidizer doesn't expose a mechanism to add this type of argument to light.exe invocations.
Should we address the warning? Maybe. But it would be part of a separate series.
- When I ran the installer, I got a message in the system notifications that "An app caused a problem with your default browser setting, so it was reset to Internet Explorer" as it was finishing up. I've never seen that before, am not certain what the default was set to on this system before, and it didn't happen a second time after I set the default to FireFox, uninstalled and then reran the installer.
That's super strange. I never saw this when testing locally. I can't imagine how Mercurial's MSI installer is triggering that. My guess is it has something to do with triggering the Windows installer service than Mercurial itself. Keep in mind that the WiX XML built through PyOxidizer is very similar to what we were doing before. The main difference is the auto-generated .wxs file holding the fragments for the installed files.