This is an archive of the discontinued Mercurial Phabricator instance.

debugextensions: gracefully handle missing __file__ attributes
ClosedPublic

Authored by mharbison72 on Nov 17 2019, 1:35 AM.

Details

Summary

This was crashing PyOxidizer.

Diff Detail

Repository
rHG Mercurial
Lint
Automatic diff as part of commit; lint not applicable.
Unit
Automatic diff as part of commit; unit tests not applicable.

Event Timeline

mharbison72 created this revision.Nov 17 2019, 1:35 AM

Interesting failure trying to load an external copy of evolve (e5d92ac69a9c):

$ ./build/apps/hg/x86_64-pc-windows-msvc/debug/hg.exe debugextensions -v
*** failed to import extension evolve from C:/Users/Matt/hg-evolve/hgext3rd\evolve: cannot import name 'compat' from 'hgext_evolve' (C:/Users/Matt/hg-evolve/hgext3rd\evolve\__init__.py)
Traceback (most recent call last):
  File "mercurial.extensions", line 294, in loadall
    load(ui, name, path, loadingtime)
  File "mercurial.extensions", line 211, in load
    mod = _importext(name, path, bind(_reportimporterror, ui))
  File "mercurial.extensions", line 119, in _importext
    mod = loadpath(path, b'hgext.%s' % name)
  File "mercurial.extensions", line 95, in loadpath
    return imp.load_module(module_name, fd, fpath, desc)
  File "imp", line 244, in load_module
    return load_package(name, filename)
  File "imp", line 216, in load_package
    return _load(spec)
  File "<frozen importlib._bootstrap>", line 696, in _load
  File "<frozen importlib._bootstrap>", line 677, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 728, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "C:/Users/Matt/hg-evolve/hgext3rd\evolve\__init__.py", line 289, in <module>
    from . import (
ImportError: cannot import name 'compat' from 'hgext_evolve' (C:/Users/Matt/hg-evolve/hgext3rd\evolve\__init__.py)
absorb
  location: c:\Users\Matt\hg3\hg\build\apps\hg\x86_64-pc-windows-msvc\debug\hg.exe
  bundled: yes

It loads fine when using PipInstallSimple to bundle it into the executable. Not sure why the name is hgext_evolve and not hgext3rd_evolve.

indygreg accepted this revision.Nov 17 2019, 6:27 PM
indygreg added a subscriber: indygreg.

The failure might be due to PyOxidizer not supporting namespace packages. I'm pretty sure that isn't implemented and nobody has asked for it yet. Workaround would be to bundle the Mercurial modules in the binary.

This revision is now accepted and ready to land.Nov 17 2019, 6:27 PM

The failure might be due to PyOxidizer not supporting namespace packages. I'm pretty sure that isn't implemented and nobody has asked for it yet. Workaround would be to bundle the Mercurial modules in the binary.

OK, I filed a feature request.

I think the modules are bundled into the binary for what I'm hacking on:

#BUILD_PATH = CWD + "/../../build/pyoxidizer"

Config(
    application_name="hg_pyO2_041",
    python_distribution=default_python_distribution(),
    embedded_python_config=EmbeddedPythonConfig(
        legacy_windows_fs_encoding=True,
        legacy_windows_stdio=True,
     sys_frozen=True,
#        sys_paths=["$ORIGIN/lib"],
#        sys_paths=["$ORIGIN"],
#        sys_paths=["C:/Users/Matt/hg"],
     unbuffered_stdio=True,
    ),
    python_run_mode=python_run_mode_eval("import hgdemandimport; hgdemandimport.enable(); import mercurial.dispatch; mercurial.dispatch.run()"),
    packaging_rules=[
        # Mercurial requires a fully featured Python because extensions may use
        # anything.
        StdlibExtensionsPolicy("all"),
        Stdlib(include_source=True),
        SetupPyInstall(
            package_path="C:/Users/Matt/hg",
            extra_global_arguments=["clean", "--all", "build"],
            # Our code doesn't yet work with the in-memory importer. So package
            # relative to the executable for now.
            #install_location="app-relative:lib",
        ),
        PipInstallSimple("hg-evolve"
        ),
    ]
)


# END OF COMMON USER-ADJUSTED SETTINGS.
#
# Everything below this is typically managed by PyOxidizer and doesn't need
# to be updated by people.

PYOXIDIZER_VERSION = "0.5.0"
PYOXIDIZER_COMMIT = "c7d9c79690d3f6ad8187e2d409c9e5e14b14d916"

I'm hacking on getting the disabled extensions to list in hg help extensions. With this config, hgext.__path__ is None. If I uncomment the sys_paths=C:/Users... line, it prints the path as C:/Users/..., so I assume that counts as unbundled modules.

It's not getting in my way or anything, so not a high priority.