This commit demonstrates the feasibility of integrating Rust into
There are two significant components of this commit:
1) Establish a Rust `hg` binary.
2) Establish a new mechanism for producing standalone Mercurial
These functions will likely get split up before this commit lands.
If you are familiar with Rust, the contents of the added rust/
directory should be pretty straightforward. We create an "hgcli"
package that implements a binary application to run Mercurial.
The output of this package is an "hg" binary.
Our Rust `hg` creates an embedded CPython interpreter and attempts
to mimim the current functionality in the `hg` Python script. An
immediate goal of the Rust implementation of `hg` is to serve as a
drop-in replacement for the `hg` script. Longer term, we can discuss
removing the Python script so `hg` is a native executable on all
platforms. This will allow us to:
* Merge `chg` functionality into `hg`
* Start implementing very early `hg` logic in Rust (config parsing,
argument parsing, repository opening, etc)
* Possibly have some commands implemented in 100% Rust, allowing us
to avoid Python interpreter startup overhead
The added code for "standalone Mercurial" is a proof-of-concept
demonstrating the direction I'd like to take Mercurial packaging
and distribution in the near future. Today, we generally treat
Mercurial as a Python application. This by-and-large works. But
as we start writing things in Rust, that will pull us away from
being Python centric. In addition, the current distribution model
is fragile because it often relies on the Python that end-users
have installed. We constantly run into problems with:
* Users running old Python versions with known bugs and lacking
features (such as modern SSL/TLS support).
* Missing optional packages that improve the Mercurial experience
(such as re2, pygments, and even the curses module on Windows).
* Python packaging fragility.
While Python is still a critical component of Mercurial and will be
for the indefinite future, I'd like Mercurial to pivot away from
being pictured as a "Python application" and move towards being
a "generic/system application." In other words, Python is just
an implementation detail.
The added Python script produces a standalone Mercurial
installation. It currently only works on Linux, MacOS, and similar
*NIX variants (although I haven't tested it thoroughly and it may
be broken on MacOS after recent changes). Essentially, the script
downloads the Python 2.7 source code, builds Python from source,
builds the Mercurial Rust and Python components, and produces a
tarball containing the Rust `hg` binary and all the support files.
The rpath of the `hg` binary is relative. So you should be able
to drop the files into any system and things "just work." That's
the goal anyway: we don't come close to fulfilling this end state.
This patch should be considered early alpha and RFC quality. The
following known issues and open investigation items exist:
* This is the first Rust code I've written. I'm almost certainly
doing many Rust things poorly. I'm pretty sure I'm a bit too
liberal with my .unwrap() usage.
* I haven't verified that the produced standalone Mercurial is
* HGUNICODEPEDANTRY likely doesn't work.
* rpath values are being hacked in post build. We should set the rpath
properly at link time.
* Sometimes running the Rust `hg` from target/<build>/hg fails
because it can't locate Python files. The `build/standalone/bin/hg`
binary in the standalone install should "just work."
* The path searching in the Rust `hg` to find libpython and
Mercurial python files is very hacky.
* Rust `hg` is using std::env::args() instead of std::env::args_os().
I think we want the latter so Rust doesn't panic if it sees
arguments that aren't UTF-8.
* main.rs is re-defining some CPython APIs that are already defined
in python27-sys. If you attempt to make python27-sys a dependency
on hgcli, Cargo complains that multiple packages are linking to
libpython. I didn't feel like going down that rabbit hole.
* cpython and python27-sys can pick up other Python installs and
libraries on the system. I've seen issues with missing PyUnicode_*
symbols because of UCS2 and UCS4 mismatch.
* No support for Windows yet (but I have a plan).
* The cpython crate seems to be optimized for writing Python
extensions, not embedding Python. As a result, the interaction
between the Rust `hg` and this crate is a bit wonky. For example,
I'm pretty sure we don't need to call `PyEval_InitThreads()`. But
the cpython crate insists a thread already exist if the Python
interpreter is already initialized. So... ???
* Python may not be receiving OS signals.