( )⚙ D11697 pyoxidizer: disable using in-memory resources

This is an archive of the discontinued Mercurial Phabricator instance.

Differential D11697

pyoxidizer: disable using in-memory resources
ClosedPublic

Authored by spectral on Oct 19 2021, 5:00 PM.

Details

Reviewers
Alphare
Group Reviewers
hg-reviewers
Commits
rHGc900d962e38b: pyoxidizer: disable using in-memory resources
Summary

It's possible that the errors are due to using an incompatible version of
PyOxidizer; unfortunately the README.md file in this directory says to fetch a
copy of PyOxidizer matching the commit in this pyoxidizer.bzl file, and yet the
pyoxidizer.bzl file does not actually have a commit mentioned in it.

By disabling in-memory modules, this appears to work on all platforms using the
current head version of PyOxidizer, so let's disable them for now.

Sample error (during pyoxidizer build):

error[PYOXIDIZER_PYTHON_EXECUTABLE]: adding PythonExtensionModule<name=hgext.fsmonitor.pywatchman.bser>

Caused by:
    extension module hgext.fsmonitor.pywatchman.bser cannot be loaded from memory but memory loading required
   --> ./pyoxidizer.bzl:140:5
    |
140 |     exe.add_python_resources(exe.pip_install(["--verbose", ROOT]))
    |     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ add_python_resources()


error: adding PythonExtensionModule<name=hgext.fsmonitor.pywatchman.bser>

Caused by:
    extension module hgext.fsmonitor.pywatchman.bser cannot be loaded from memory but memory loading required

Diff Detail

Repository
rHG Mercurial
Branch
default
Lint
No Linters Available
Unit
No Unit Test Coverage

Event Timeline

spectral created this revision.Oct 19 2021, 5:00 PM
Alphare accepted this revision.Oct 20 2021, 3:18 AM
This revision is now accepted and ready to land.Oct 20 2021, 3:18 AM
spectral added a commit: rHGc900d962e38b: pyoxidizer: disable using in-memory resources.Oct 20 2021, 5:35 AM
Closed by commit rHGc900d962e38b: pyoxidizer: disable using in-memory resources (authored by spectral). · Explain Why
This revision was automatically updated to reflect the committed changes.

Revision Contents
Changeset List

PathPackages
Mrust/hgcli/pyoxidizer.bzl (7 lines)

Diff 30912

rust/hgcli/pyoxidizer.bzl

MSI_NAME = VARS.get("MSI_NAME", "mercurial") MSI_NAME = VARS.get("MSI_NAME", "mercurial")
EXTRA_MSI_FEATURES = VARS.get("EXTRA_MSI_FEATURES") EXTRA_MSI_FEATURES = VARS.get("EXTRA_MSI_FEATURES")
SIGNING_PFX_PATH = VARS.get("SIGNING_PFX_PATH") SIGNING_PFX_PATH = VARS.get("SIGNING_PFX_PATH")
SIGNING_PFX_PASSWORD = VARS.get("SIGNING_PFX_PASSWORD", "") SIGNING_PFX_PASSWORD = VARS.get("SIGNING_PFX_PASSWORD", "")
SIGNING_SUBJECT_NAME = VARS.get("SIGNING_SUBJECT_NAME") SIGNING_SUBJECT_NAME = VARS.get("SIGNING_SUBJECT_NAME")
TIME_STAMP_SERVER_URL = VARS.get("TIME_STAMP_SERVER_URL", "http://timestamp.digicert.com") TIME_STAMP_SERVER_URL = VARS.get("TIME_STAMP_SERVER_URL", "http://timestamp.digicert.com")
IS_WINDOWS = "windows" in BUILD_TARGET_TRIPLE IS_WINDOWS = "windows" in BUILD_TARGET_TRIPLE
IS_MACOS = "darwin" in BUILD_TARGET_TRIPLE
USE_IN_MEMORY_RESOURCES = False
# Code to run in Python interpreter. # Code to run in Python interpreter.
RUN_CODE = """ RUN_CODE = """
import os import os
import sys import sys
extra_path = os.environ.get('PYTHONPATH') extra_path = os.environ.get('PYTHONPATH')
if extra_path is not None: if extra_path is not None:
# extensions and hooks expect a working python environment # extensions and hooks expect a working python environment
""" """
set_build_path(ROOT + "/build/pyoxidizer") set_build_path(ROOT + "/build/pyoxidizer")
def make_distribution(): def make_distribution():
return default_python_distribution(python_version = "3.9") return default_python_distribution(python_version = "3.9")
def resource_callback(policy, resource): def resource_callback(policy, resource):
if not (IS_WINDOWS or IS_MACOS): if USE_IN_MEMORY_RESOURCES:
resource.add_location = "in-memory" resource.add_location = "in-memory"
return return
# We use a custom resource routing policy to influence where things are loaded # We use a custom resource routing policy to influence where things are loaded
# from. # from.
# #
# For Python modules and resources, we load from memory if they are in # For Python modules and resources, we load from memory if they are in
# the standard library and from the filesystem if not. This is because # the standard library and from the filesystem if not. This is because
def make_exe(dist): def make_exe(dist):
"""Builds a Rust-wrapped Mercurial binary.""" """Builds a Rust-wrapped Mercurial binary."""
packaging_policy = dist.make_python_packaging_policy() packaging_policy = dist.make_python_packaging_policy()
# Extension may depend on any Python functionality. Include all # Extension may depend on any Python functionality. Include all
# extensions. # extensions.
packaging_policy.extension_module_filter = "all" packaging_policy.extension_module_filter = "all"
packaging_policy.resources_location = "in-memory" packaging_policy.resources_location = "in-memory"
if IS_WINDOWS or IS_MACOS: if not USE_IN_MEMORY_RESOURCES:
packaging_policy.resources_location_fallback = "filesystem-relative:lib" packaging_policy.resources_location_fallback = "filesystem-relative:lib"
packaging_policy.register_resource_callback(resource_callback) packaging_policy.register_resource_callback(resource_callback)
config = dist.make_python_interpreter_config() config = dist.make_python_interpreter_config()
config.allocator_backend = "default" config.allocator_backend = "default"
config.run_command = RUN_CODE config.run_command = RUN_CODE
# We want to let the user load extensions from the file system # We want to let the user load extensions from the file system