( )⚙ D11697 pyoxidizer: disable using in-memory resources

This is an archive of the discontinued Mercurial Phabricator instance.

Differential D11697

pyoxidizer: disable using in-memory resources
ClosedPublic

Authored by spectral on Oct 19 2021, 5:00 PM.

Details

Reviewers
Alphare
Group Reviewers
hg-reviewers
Commits
rHGc900d962e38b: pyoxidizer: disable using in-memory resources
Summary

It's possible that the errors are due to using an incompatible version of
PyOxidizer; unfortunately the README.md file in this directory says to fetch a
copy of PyOxidizer matching the commit in this pyoxidizer.bzl file, and yet the
pyoxidizer.bzl file does not actually have a commit mentioned in it.

By disabling in-memory modules, this appears to work on all platforms using the
current head version of PyOxidizer, so let's disable them for now.

Sample error (during pyoxidizer build):

error[PYOXIDIZER_PYTHON_EXECUTABLE]: adding PythonExtensionModule<name=hgext.fsmonitor.pywatchman.bser>

Caused by:
    extension module hgext.fsmonitor.pywatchman.bser cannot be loaded from memory but memory loading required
   --> ./pyoxidizer.bzl:140:5
    |
140 |     exe.add_python_resources(exe.pip_install(["--verbose", ROOT]))
    |     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ add_python_resources()


error: adding PythonExtensionModule<name=hgext.fsmonitor.pywatchman.bser>

Caused by:
    extension module hgext.fsmonitor.pywatchman.bser cannot be loaded from memory but memory loading required

Diff Detail

Repository
rHG Mercurial
Lint
Automatic diff as part of commit; lint not applicable.
Unit
Automatic diff as part of commit; unit tests not applicable.

Event Timeline

spectral created this revision.Oct 19 2021, 5:00 PM
Alphare accepted this revision.Oct 20 2021, 3:18 AM
This revision is now accepted and ready to land.Oct 20 2021, 3:18 AM
spectral added a commit: rHGc900d962e38b: pyoxidizer: disable using in-memory resources.Oct 20 2021, 5:35 AM
Closed by commit rHGc900d962e38b: pyoxidizer: disable using in-memory resources (authored by spectral). · Explain Why
This revision was automatically updated to reflect the committed changes.

Revision Contents
Changeset List

PathPackages
Mrust/hgcli/pyoxidizer.bzl (7 lines)

Diff 30949

rust/hgcli/pyoxidizer.bzl

MSI_NAME = VARS.get("MSI_NAME", "mercurial") MSI_NAME = VARS.get("MSI_NAME", "mercurial")
EXTRA_MSI_FEATURES = VARS.get("EXTRA_MSI_FEATURES") EXTRA_MSI_FEATURES = VARS.get("EXTRA_MSI_FEATURES")
SIGNING_PFX_PATH = VARS.get("SIGNING_PFX_PATH") SIGNING_PFX_PATH = VARS.get("SIGNING_PFX_PATH")
SIGNING_PFX_PASSWORD = VARS.get("SIGNING_PFX_PASSWORD", "") SIGNING_PFX_PASSWORD = VARS.get("SIGNING_PFX_PASSWORD", "")
SIGNING_SUBJECT_NAME = VARS.get("SIGNING_SUBJECT_NAME") SIGNING_SUBJECT_NAME = VARS.get("SIGNING_SUBJECT_NAME")
TIME_STAMP_SERVER_URL = VARS.get("TIME_STAMP_SERVER_URL", "http://timestamp.digicert.com") TIME_STAMP_SERVER_URL = VARS.get("TIME_STAMP_SERVER_URL", "http://timestamp.digicert.com")
IS_WINDOWS = "windows" in BUILD_TARGET_TRIPLE IS_WINDOWS = "windows" in BUILD_TARGET_TRIPLE
IS_MACOS = "darwin" in BUILD_TARGET_TRIPLE
USE_IN_MEMORY_RESOURCES = False
# Code to run in Python interpreter. # Code to run in Python interpreter.
RUN_CODE = """ RUN_CODE = """
import os import os
import sys import sys
extra_path = os.environ.get('PYTHONPATH') extra_path = os.environ.get('PYTHONPATH')
if extra_path is not None: if extra_path is not None:
# extensions and hooks expect a working python environment # extensions and hooks expect a working python environment
""" """
set_build_path(ROOT + "/build/pyoxidizer") set_build_path(ROOT + "/build/pyoxidizer")
def make_distribution(): def make_distribution():
return default_python_distribution(python_version = "3.9") return default_python_distribution(python_version = "3.9")
def resource_callback(policy, resource): def resource_callback(policy, resource):
if not (IS_WINDOWS or IS_MACOS): if USE_IN_MEMORY_RESOURCES:
resource.add_location = "in-memory" resource.add_location = "in-memory"
return return
# We use a custom resource routing policy to influence where things are loaded # We use a custom resource routing policy to influence where things are loaded
# from. # from.
# #
# For Python modules and resources, we load from memory if they are in # For Python modules and resources, we load from memory if they are in
# the standard library and from the filesystem if not. This is because # the standard library and from the filesystem if not. This is because
def make_exe(dist): def make_exe(dist):
"""Builds a Rust-wrapped Mercurial binary.""" """Builds a Rust-wrapped Mercurial binary."""
packaging_policy = dist.make_python_packaging_policy() packaging_policy = dist.make_python_packaging_policy()
# Extension may depend on any Python functionality. Include all # Extension may depend on any Python functionality. Include all
# extensions. # extensions.
packaging_policy.extension_module_filter = "all" packaging_policy.extension_module_filter = "all"
packaging_policy.resources_location = "in-memory" packaging_policy.resources_location = "in-memory"
if IS_WINDOWS or IS_MACOS: if not USE_IN_MEMORY_RESOURCES:
packaging_policy.resources_location_fallback = "filesystem-relative:lib" packaging_policy.resources_location_fallback = "filesystem-relative:lib"
packaging_policy.register_resource_callback(resource_callback) packaging_policy.register_resource_callback(resource_callback)
config = dist.make_python_interpreter_config() config = dist.make_python_interpreter_config()
config.allocator_backend = "default" config.allocator_backend = "default"
config.run_command = RUN_CODE config.run_command = RUN_CODE
# We want to let the user load extensions from the file system # We want to let the user load extensions from the file system