( )⚙ D9132 rust: introduce `dirstate-tree` cargo feature

This is an archive of the discontinued Mercurial Phabricator instance.

Differential D9132

rust: introduce `dirstate-tree` cargo feature
ClosedPublic

Authored by Alphare on Sep 30 2020, 1:35 PM.

Details

Reviewers
None
Group Reviewers
hg-reviewers
Commits
rHGe604a3c03ab9: rust: introduce `dirstate-tree` cargo feature
Summary

This feature gates (at compile-time) the use of the newly-added dirstate tree.

The motivation for this is that the dirstate tree is currently *very* slow;
replacing the current hashmap-based dirstate is not a viable solution in terms
of performance... and why would you be using the Rust implementation if not
for performance?

The feature will also help reviewers better understand the differences that
will slowly appear as the dirstate tree gets better.

Diff Detail

Repository
rHG Mercurial
Branch
default
Lint
No Linters Available
Unit
No Unit Test Coverage

Event Timeline

Alphare created this revision.Sep 30 2020, 1:35 PM
Alphare added a commit: rHGe604a3c03ab9: rust: introduce `dirstate-tree` cargo feature.Oct 1 2020, 12:57 AM
This revision was not accepted when it landed; it landed in state Needs Review.
Closed by commit rHGe604a3c03ab9: rust: introduce `dirstate-tree` cargo feature (authored by Alphare). · Explain Why
This revision was automatically updated to reflect the committed changes.

Revision Contents
Changeset List

Diff 22954

Makefile

$(MAKE) -f $(HGROOT)/contrib/Makefile.python PYTHONVER=$* PREFIX=$(HGPYTHONS)/$* python ) $(MAKE) -f $(HGROOT)/contrib/Makefile.python PYTHONVER=$* PREFIX=$(HGPYTHONS)/$* python )
cd tests && $(HGPYTHONS)/$*/bin/python run-tests.py $(TESTFLAGS) cd tests && $(HGPYTHONS)/$*/bin/python run-tests.py $(TESTFLAGS)
rust-tests: py_feature = $(shell $(PYTHON) -c \ rust-tests: py_feature = $(shell $(PYTHON) -c \
'import sys; print(["python27-bin", "python3-bin"][sys.version_info[0] >= 3])') 'import sys; print(["python27-bin", "python3-bin"][sys.version_info[0] >= 3])')
rust-tests: rust-tests:
cd $(HGROOT)/rust/hg-cpython \ cd $(HGROOT)/rust/hg-cpython \
&& $(CARGO) test --quiet --all \ && $(CARGO) test --quiet --all \
--no-default-features --features "$(py_feature)" --no-default-features --features "$(py_feature) $(HG_RUST_FEATURES)"
check-code: check-code:
hg manifest | xargs python contrib/check-code.py hg manifest | xargs python contrib/check-code.py
format-c: format-c:
clang-format --style file -i \ clang-format --style file -i \
`hg files 'set:(**.c or **.cc or **.h) and not "listfile:contrib/clang-format-ignorelist"'` `hg files 'set:(**.c or **.cc or **.h) and not "listfile:contrib/clang-format-ignorelist"'`

contrib/heptapod-ci.yml

- echo "python used, $PYTHON" - echo "python used, $PYTHON"
- make rust-tests - make rust-tests
rust-cargo-test-py3: rust-cargo-test-py3:
<<: *rust_cargo_test <<: *rust_cargo_test
variables: variables:
PYTHON: python3 PYTHON: python3
rust-cargo-test-py3-dirstate-tree:
<<: *rust_cargo_test
variables:
PYTHON: python3
HG_RUST_FEATURES: dirstate-tree
test-py2: test-py2:
<<: *runtests <<: *runtests
variables: variables:
RUNTEST_ARGS: " --no-rust --blacklist /tmp/check-tests.txt" RUNTEST_ARGS: " --no-rust --blacklist /tmp/check-tests.txt"
TEST_HGMODULEPOLICY: "c" TEST_HGMODULEPOLICY: "c"
test-py3: test-py3:
<<: *runtests <<: *runtests
test-py3-rust: test-py3-rust:
<<: *runtests <<: *runtests
variables: variables:
HGWITHRUSTEXT: cpython HGWITHRUSTEXT: cpython
RUNTEST_ARGS: "--rust --blacklist /tmp/check-tests.txt" RUNTEST_ARGS: "--rust --blacklist /tmp/check-tests.txt"
PYTHON: python3 PYTHON: python3
TEST_HGMODULEPOLICY: "rust+c" TEST_HGMODULEPOLICY: "rust+c"
test-py3-rust-dirstate-tree:
<<: *runtests
variables:
HGWITHRUSTEXT: cpython
RUNTEST_ARGS: "--rust --blacklist /tmp/check-tests.txt"
PYTHON: python3
TEST_HGMODULEPOLICY: "rust+c"
HG_RUST_FEATURES: "dirstate-tree"
test-py2-chg: test-py2-chg:
<<: *runtests <<: *runtests
variables: variables:
RUNTEST_ARGS: "--blacklist /tmp/check-tests.txt --chg" RUNTEST_ARGS: "--blacklist /tmp/check-tests.txt --chg"
TEST_HGMODULEPOLICY: "c" TEST_HGMODULEPOLICY: "c"

rust/README.rst

checking module policy (rust+c-allow) checking module policy (rust+c-allow)
If the environment variable ``HGWITHRUSTEXT=cpython`` is set, the Rust If the environment variable ``HGWITHRUSTEXT=cpython`` is set, the Rust
extension will be used by default unless ``--no-rust``. extension will be used by default unless ``--no-rust``.
One day we may use this environment variable to switch to new experimental One day we may use this environment variable to switch to new experimental
binding crates like a hypothetical ``HGWITHRUSTEXT=hpy``. binding crates like a hypothetical ``HGWITHRUSTEXT=hpy``.
Special features
================
You might want to check the `features` section in ``hg-cpython/Cargo.toml``.
It may contain features that might be interesting to try out.
To use features from the Makefile, use the `HG_RUST_FEATURES` environment
variable: for instance `HG_RUST_FEATURES="some-feature other-feature"`
Profiling Profiling
========= =========
Setting the environment variable ``RUST_LOG=trace`` will make hg print Setting the environment variable ``RUST_LOG=trace`` will make hg print
a few high level rust-related performance numbers. It can also a few high level rust-related performance numbers. It can also
indicate why the rust code cannot be used (say, using lookarounds in indicate why the rust code cannot be used (say, using lookarounds in
hgignore). hgignore).

rust/hg-core/Cargo.toml

version = "1.0.16" version = "1.0.16"
features = ["zlib"] features = ["zlib"]
default-features = false default-features = false
[dev-dependencies] [dev-dependencies]
clap = "*" clap = "*"
pretty_assertions = "0.6.1" pretty_assertions = "0.6.1"
tempfile = "3.1.0" tempfile = "3.1.0"
[features]
# Use a (still unoptimized) tree for the dirstate instead of the current flat
# dirstate. This is not yet recommended for performance reasons. A future
# version might make it the default, or make it a runtime option.
dirstate-tree = []

rust/hg-core/src/dirstate.rs

// dirstate module // dirstate module
// //
// Copyright 2019 Raphaël Gomès <rgomes@octobus.net> // Copyright 2019 Raphaël Gomès <rgomes@octobus.net>
// //
// This software may be used and distributed according to the terms of the // This software may be used and distributed according to the terms of the
// GNU General Public License version 2 or any later version. // GNU General Public License version 2 or any later version.
use crate::{utils::hg_path::HgPathBuf, DirstateParseError, FastHashMap}; use crate::{utils::hg_path::HgPathBuf, DirstateParseError, FastHashMap};
use std::collections::hash_map; use std::collections::hash_map;
use std::convert::TryFrom; use std::convert::TryFrom;
pub mod dirs_multiset; pub mod dirs_multiset;
pub mod dirstate_map; pub mod dirstate_map;
#[cfg(feature = "dirstate-tree")]
pub mod dirstate_tree; pub mod dirstate_tree;
pub mod parsers; pub mod parsers;
pub mod status; pub mod status;
#[derive(Debug, PartialEq, Clone)] #[derive(Debug, PartialEq, Clone)]
pub struct DirstateParents { pub struct DirstateParents {
pub p1: [u8; 20], pub p1: [u8; 20],
pub p2: [u8; 20], pub p2: [u8; 20],

rust/hg-cpython/Cargo.toml

[package] [package]
name = "hg-cpython" name = "hg-cpython"
version = "0.1.0" version = "0.1.0"
authors = ["Georges Racinet <gracinet@anybox.fr>"] authors = ["Georges Racinet <gracinet@anybox.fr>"]
edition = "2018" edition = "2018"
[lib] [lib]
name='rusthg' name='rusthg'
crate-type = ["cdylib"] crate-type = ["cdylib"]
[features] [features]
default = ["python27"] default = ["python27"]
dirstate-tree = ["hg-core/dirstate-tree"]
# Features to build an extension module: # Features to build an extension module:
python27 = ["cpython/python27-sys", "cpython/extension-module-2-7"] python27 = ["cpython/python27-sys", "cpython/extension-module-2-7"]
python3 = ["cpython/python3-sys", "cpython/extension-module"] python3 = ["cpython/python3-sys", "cpython/extension-module"]
# Enable one of these features to build a test executable linked to libpython: # Enable one of these features to build a test executable linked to libpython:
# e.g. cargo test --no-default-features --features python27-bin # e.g. cargo test --no-default-features --features python27-bin
python27-bin = ["cpython/python27-sys"] python27-bin = ["cpython/python27-sys"]