This is an archive of the discontinued Mercurial Phabricator instance.

Differential D9132

rust: introduce `dirstate-tree` cargo feature
ClosedPublic

Authored by Alphare on Sep 30 2020, 1:35 PM.

Download Raw Diff

Details

Reviewers

None

Group Reviewers

hg-reviewers

Commits

rHGe604a3c03ab9: rust: introduce `dirstate-tree` cargo feature

Summary

This feature gates (at compile-time) the use of the newly-added dirstate tree.

The motivation for this is that the dirstate tree is currently *very* slow;
replacing the current hashmap-based dirstate is not a viable solution in terms
of performance... and why would you be using the Rust implementation if not
for performance?

The feature will also help reviewers better understand the differences that
will slowly appear as the dirstate tree gets better.

Diff Detail

Repository

rHG Mercurial

Lint

Automatic diff as part of commit; lint not applicable.

Unit

Automatic diff as part of commit; unit tests not applicable.

Event Timeline

Alphare created this revision.Sep 30 2020, 1:35 PM

Herald added a reviewer: hg-reviewers. · View Herald TranscriptSep 30 2020, 1:35 PM

Herald added a subscriber: mercurial-patches. · View Herald Transcript

Alphare added a child revision: D9133: rust: start plugging the dirstate tree behind a feature gate.Sep 30 2020, 1:36 PM

Alphare added a commit: rHGe604a3c03ab9: rust: introduce `dirstate-tree` cargo feature.Oct 1 2020, 12:57 AM

This revision was not accepted when it landed; it landed in state Needs Review.

Closed by commit rHGe604a3c03ab9: rust: introduce `dirstate-tree` cargo feature (authored by Alphare). · Explain Why

This revision was automatically updated to reflect the committed changes.

Revision Contents
Changeset List

			Path	Packages
M			Makefile (2 lines)
M			contrib/heptapod-ci.yml (15 lines)
M			rust/README.rst (9 lines)
M			rust/hg-core/Cargo.toml (6 lines)
M			rust/hg-core/src/dirstate.rs (1 line)
M			rust/hg-cpython/Cargo.toml (1 line)

	Status	Author	Revision
	Closed	Alphare	D9133 rust: start plugging the dirstate tree behind a feature gate
	Closed	Alphare	D9132 rust: introduce `dirstate-tree` cargo feature

Diff 22962

Makefile

	$(MAKE) -f $(HGROOT)/contrib/Makefile.python PYTHONVER=$* PREFIX=$(HGPYTHONS)/$* python )			$(MAKE) -f $(HGROOT)/contrib/Makefile.python PYTHONVER=$* PREFIX=$(HGPYTHONS)/$* python )
	cd tests && $(HGPYTHONS)/$*/bin/python run-tests.py $(TESTFLAGS)			cd tests && $(HGPYTHONS)/$*/bin/python run-tests.py $(TESTFLAGS)

	rust-tests: py_feature = $(shell $(PYTHON) -c \			rust-tests: py_feature = $(shell $(PYTHON) -c \
	'import sys; print(["python27-bin", "python3-bin"][sys.version_info[0] >= 3])')			'import sys; print(["python27-bin", "python3-bin"][sys.version_info[0] >= 3])')
	rust-tests:			rust-tests:
	cd $(HGROOT)/rust/hg-cpython \			cd $(HGROOT)/rust/hg-cpython \
	&& $(CARGO) test --quiet --all \			&& $(CARGO) test --quiet --all \
	--no-default-features --features "$(py_feature)"			--no-default-features --features "$(py_feature) $(HG_RUST_FEATURES)"

	check-code:			check-code:
	hg manifest \| xargs python contrib/check-code.py			hg manifest \| xargs python contrib/check-code.py

	format-c:			format-c:
	clang-format --style file -i \			clang-format --style file -i \
	`hg files 'set:(.c or .cc or **.h) and not "listfile:contrib/clang-format-ignorelist"'`			`hg files 'set:(.c or .cc or **.h) and not "listfile:contrib/clang-format-ignorelist"'`

contrib/heptapod-ci.yml

	- echo "python used, $PYTHON"			- echo "python used, $PYTHON"
	- make rust-tests			- make rust-tests

	rust-cargo-test-py3:			rust-cargo-test-py3:
	<<: *rust_cargo_test			<<: *rust_cargo_test
	variables:			variables:
	PYTHON: python3			PYTHON: python3

				rust-cargo-test-py3-dirstate-tree:
				<<: *rust_cargo_test
				variables:
				PYTHON: python3
				HG_RUST_FEATURES: dirstate-tree

	test-py2:			test-py2:
	<<: *runtests			<<: *runtests
	variables:			variables:
	RUNTEST_ARGS: " --no-rust --blacklist /tmp/check-tests.txt"			RUNTEST_ARGS: " --no-rust --blacklist /tmp/check-tests.txt"
	TEST_HGMODULEPOLICY: "c"			TEST_HGMODULEPOLICY: "c"

	test-py3:			test-py3:
	<<: *runtests			<<: *runtests
	test-py3-rust:			test-py3-rust:
	<<: *runtests			<<: *runtests
	variables:			variables:
	HGWITHRUSTEXT: cpython			HGWITHRUSTEXT: cpython
	RUNTEST_ARGS: "--rust --blacklist /tmp/check-tests.txt"			RUNTEST_ARGS: "--rust --blacklist /tmp/check-tests.txt"
	PYTHON: python3			PYTHON: python3
	TEST_HGMODULEPOLICY: "rust+c"			TEST_HGMODULEPOLICY: "rust+c"

				test-py3-rust-dirstate-tree:
				<<: *runtests
				variables:
				HGWITHRUSTEXT: cpython
				RUNTEST_ARGS: "--rust --blacklist /tmp/check-tests.txt"
				PYTHON: python3
				TEST_HGMODULEPOLICY: "rust+c"
				HG_RUST_FEATURES: "dirstate-tree"

	test-py2-chg:			test-py2-chg:
	<<: *runtests			<<: *runtests
	variables:			variables:
	RUNTEST_ARGS: "--blacklist /tmp/check-tests.txt --chg"			RUNTEST_ARGS: "--blacklist /tmp/check-tests.txt --chg"
	TEST_HGMODULEPOLICY: "c"			TEST_HGMODULEPOLICY: "c"

rust/README.rst

	checking module policy (rust+c-allow)			checking module policy (rust+c-allow)

	If the environment variable ``HGWITHRUSTEXT=cpython`` is set, the Rust			If the environment variable ``HGWITHRUSTEXT=cpython`` is set, the Rust
	extension will be used by default unless ``--no-rust``.			extension will be used by default unless ``--no-rust``.

	One day we may use this environment variable to switch to new experimental			One day we may use this environment variable to switch to new experimental
	binding crates like a hypothetical ``HGWITHRUSTEXT=hpy``.			binding crates like a hypothetical ``HGWITHRUSTEXT=hpy``.

				Special features
				================

				You might want to check the `features` section in ``hg-cpython/Cargo.toml``.
				It may contain features that might be interesting to try out.

				To use features from the Makefile, use the `HG_RUST_FEATURES` environment
				variable: for instance `HG_RUST_FEATURES="some-feature other-feature"`

	Profiling			Profiling
	=========			=========

	Setting the environment variable ``RUST_LOG=trace`` will make hg print			Setting the environment variable ``RUST_LOG=trace`` will make hg print
	a few high level rust-related performance numbers. It can also			a few high level rust-related performance numbers. It can also
	indicate why the rust code cannot be used (say, using lookarounds in			indicate why the rust code cannot be used (say, using lookarounds in
	hgignore).			hgignore).

rust/hg-core/Cargo.toml

	version = "1.0.16"			version = "1.0.16"
	features = ["zlib"]			features = ["zlib"]
	default-features = false			default-features = false

	[dev-dependencies]			[dev-dependencies]
	clap = "*"			clap = "*"
	pretty_assertions = "0.6.1"			pretty_assertions = "0.6.1"
	tempfile = "3.1.0"			tempfile = "3.1.0"

				[features]
				# Use a (still unoptimized) tree for the dirstate instead of the current flat
				# dirstate. This is not yet recommended for performance reasons. A future
				# version might make it the default, or make it a runtime option.
				dirstate-tree = []

rust/hg-core/src/dirstate.rs

	// dirstate module			// dirstate module
	//			//
	// Copyright 2019 Raphaël Gomès <rgomes@octobus.net>			// Copyright 2019 Raphaël Gomès <rgomes@octobus.net>
	//			//
	// This software may be used and distributed according to the terms of the			// This software may be used and distributed according to the terms of the
	// GNU General Public License version 2 or any later version.			// GNU General Public License version 2 or any later version.

	use crate::{utils::hg_path::HgPathBuf, DirstateParseError, FastHashMap};			use crate::{utils::hg_path::HgPathBuf, DirstateParseError, FastHashMap};
	use std::collections::hash_map;			use std::collections::hash_map;
	use std::convert::TryFrom;			use std::convert::TryFrom;

	pub mod dirs_multiset;			pub mod dirs_multiset;
	pub mod dirstate_map;			pub mod dirstate_map;
				#[cfg(feature = "dirstate-tree")]
	pub mod dirstate_tree;			pub mod dirstate_tree;
	pub mod parsers;			pub mod parsers;
	pub mod status;			pub mod status;

	#[derive(Debug, PartialEq, Clone)]			#[derive(Debug, PartialEq, Clone)]
	pub struct DirstateParents {			pub struct DirstateParents {
	pub p1: [u8; 20],			pub p1: [u8; 20],
	pub p2: [u8; 20],			pub p2: [u8; 20],

rust/hg-cpython/Cargo.toml

	[package]			[package]
	name = "hg-cpython"			name = "hg-cpython"
	version = "0.1.0"			version = "0.1.0"
	authors = ["Georges Racinet <gracinet@anybox.fr>"]			authors = ["Georges Racinet <gracinet@anybox.fr>"]
	edition = "2018"			edition = "2018"

	[lib]			[lib]
	name='rusthg'			name='rusthg'
	crate-type = ["cdylib"]			crate-type = ["cdylib"]

	[features]			[features]
	default = ["python27"]			default = ["python27"]
				dirstate-tree = ["hg-core/dirstate-tree"]

	# Features to build an extension module:			# Features to build an extension module:
	python27 = ["cpython/python27-sys", "cpython/extension-module-2-7"]			python27 = ["cpython/python27-sys", "cpython/extension-module-2-7"]
	python3 = ["cpython/python3-sys", "cpython/extension-module"]			python3 = ["cpython/python3-sys", "cpython/extension-module"]

	# Enable one of these features to build a test executable linked to libpython:			# Enable one of these features to build a test executable linked to libpython:
	# e.g. cargo test --no-default-features --features python27-bin			# e.g. cargo test --no-default-features --features python27-bin
	python27-bin = ["cpython/python27-sys"]			python27-bin = ["cpython/python27-sys"]

Diff	ID	Description	Created	Lint	Unit
Base		Base
Diff 1	22954		Sep 30 2020, 1:35 PM	★	★
Diff 2	22962	rHGe604a3c03ab90d0c49264c75314c535f0cff96b2	Sep 30 2020, 12:10 PM	★	★