setup.py
33	Maybe this should be a hard error so people won't accidentally miss dependencies.

setup.py
33	I don't want to break users who simply don't have a rust toolchain installed from using the rest of fb-hgext. If `USERUST` is `1` then it's a hard failure, but if `USERUST` is not set then you just don't get rust-based code if you don't have a rust toolchain. We can change this later once rust toolchains are more commonplace and necessary.

setup.py
33	We could at least print a warning to stderr, so it's more debugable in build outputs.

Diff	ID	Description	Created	Lint	Unit
Base		Base
Diff 1	3492		Nov 14 2017, 12:39 PM	★	★
Diff 2	3737		Nov 21 2017, 1:36 PM	★	★
Diff 3	3778		Nov 22 2017, 1:15 PM	★	★
Diff 4	3833		Nov 24 2017, 11:53 AM	★	★
Diff 5	3872		Nov 27 2017, 8:00 AM	★	★
Diff 6	3890		Nov 27 2017, 4:33 PM	★	★
Diff 7	3941	rFBHGX2742730d0d39c10a837879e618241e78a343bb13	Nov 28 2017, 7:51 AM	★	★

Status	Author	Revision
Closed	mbthomas	D1528 treedirstate: prevent interference with other dirstate implementations
Closed	mbthomas	D1512 treedirstate: de-genericize Dirstate
Closed	mbthomas	D1510 treedirstate: extract serialization methods to separate module
Closed	mbthomas	D1490 treedirstate: add integration tests
Closed	mbthomas	D1434 treedirstate: add configuration options
Closed	mbthomas	D1412 setup: build treedirstate and rusttreedirstate packages
Closed	mbthomas	D1411 distutils_rust: add distutils extension to compile Rust extension modules
Closed	mbthomas	D1410 perftweaks: support treedirstate maps
Closed	mbthomas	D1409 treedirstate: implement casefolding maps for case insensitive filesystems
Closed	mbthomas	D1408 treedirstate: use vlqencoding for numbers
Closed	mbthomas	D1407 treedirstate: auto-repack treedirstate once it reaches 3x its original size
Closed	mbthomas	D1406 treedirstate: implement efficient case collision detection
Closed	mbthomas	D1405 treedirstate: allow absent non-normal sets
Closed	mbthomas	D1404 treedirstate: clear ambiguous times when writing the dirstate
Closed	mbthomas	D1403 treedirstate: better iteration using visitor pattern
Closed	mbthomas	D1402 treedirstate: cache dirstate data when iterating all files
Closed	mbthomas	D1401 treedirstate: add Python linkage
Closed	mbthomas	D1400 treedirstate: add Dirstate
Closed	mbthomas	D1399 treedirstate: add Tree
Closed	mbthomas	D1398 treedirstate: add FileStore
Closed	mbthomas	D1397 treedirstate: add Store and StoreView traits
Closed	mbthomas	D1396 treedirstate: add vecmap implementation
Closed	mbthomas	D1395 treedirstate: create empty Rust project

Diff 3941

.cargo/config

This file was moved from rust/treedirstate/.cargo/config.

Makefile

	@echo ' install - install program and man pages to PREFIX ($(PREFIX))'			@echo ' install - install program and man pages to PREFIX ($(PREFIX))'
	@echo ' clean - remove files created by other targets'			@echo ' clean - remove files created by other targets'
	@echo ' (except installed files or dist source tarball)'			@echo ' (except installed files or dist source tarball)'

	local:			local:
	$(PYTHON) setup.py \			$(PYTHON) setup.py \
	build_py -c -d . \			build_py -c -d . \
	build_clib \			build_clib \
	build_ext -i			build_ext -i \
				build_rust_ext -i

	install:			install:
	$(PYTHON) setup.py $(PURE) install --prefix="$(PREFIX)" --force			$(PYTHON) setup.py $(PURE) install --prefix="$(PREFIX)" --force

	clean:			clean:
	-$(PYTHON) setup.py clean --all clean_ext # ignore errors from this command			-$(PYTHON) setup.py clean --all clean_ext # ignore errors from this command

	deb:			deb:
	contrib/builddeb			contrib/builddeb

hgext3rd/rust/init.py

This file was added.

				from __future__ import absolute_import
				import pkgutil

				# Indicate that hgext3rd.rust is a namspace package, and other python path
				# directories may still be searched for hgext3rd.rust libraries.
				__path__ = pkgutil.extend_path(__path__, __name__)

hgext3rd/treedirstate.py

This file was moved from treedirstate/__init__.py.

	import errno			import errno
	import heapq			import heapq
	import itertools			import itertools
	import os			import os
	import random			import random
	import struct			import struct
	import string			import string

	from .rusttreedirstate import RustDirstateMap			from hgext3rd.rust.treedirstate import treedirstatemap as rusttreedirstatemap

	dirstateheader = b'########################treedirstate####'			dirstateheader = b'########################treedirstate####'
	treedirstateversion = 1			treedirstateversion = 1
	treefileprefix = 'dirstate.tree.'			treefileprefix = 'dirstate.tree.'
	useinnewrepos = True			useinnewrepos = True

	# Sentinel length value for when a nonnormalset or otherparentset is absent.			# Sentinel length value for when a nonnormalset or otherparentset is absent.
	setabsent = 0xffffffff			setabsent = 0xffffffff
	class treedirstatemap(object):			class treedirstatemap(object):
	def __init__(self, ui, opener, root, importmap=None):			def __init__(self, ui, opener, root, importmap=None):
	self._ui = ui			self._ui = ui
	self._opener = opener			self._opener = opener
	self._root = root			self._root = root
	self.copymap = {}			self.copymap = {}

	self._filename = 'dirstate'			self._filename = 'dirstate'
	self._rmap = RustDirstateMap(ui, opener)			self._rmap = rusttreedirstatemap(ui, opener)
	self._treeid = None			self._treeid = None
	self._parents = None			self._parents = None
	self._dirtyparents = False			self._dirtyparents = False
	self._nonnormalset = set()			self._nonnormalset = set()
	self._otherparentset = set()			self._otherparentset = set()
	self._packedsize = 0			self._packedsize = 0

	if importmap is not None:			if importmap is not None:

rust/treedirstate/.cargo/config

This file was moved to .cargo/config.

rust/treedirstate/Cargo.lock

	source = "registry+https://github.com/rust-lang/crates.io-index"			source = "registry+https://github.com/rust-lang/crates.io-index"

	[[package]]			[[package]]
	name = "rustc-demangle"			name = "rustc-demangle"
	version = "0.1.5"			version = "0.1.5"
	source = "registry+https://github.com/rust-lang/crates.io-index"			source = "registry+https://github.com/rust-lang/crates.io-index"

	[[package]]			[[package]]
	name = "rusttreedirstate"
	version = "0.1.0"
	dependencies = [
	"byteorder 1.1.0 (registry+https://github.com/rust-lang/crates.io-index)",
	"cpython 0.1.0 (git+https://github.com/dgrunwald/rust-cpython.git)",
	"error-chain 0.11.0 (registry+https://github.com/rust-lang/crates.io-index)",
	"itertools 0.7.2 (registry+https://github.com/rust-lang/crates.io-index)",
	"quickcheck 0.4.1 (registry+https://github.com/rust-lang/crates.io-index)",
	"tempdir 0.3.5 (registry+https://github.com/rust-lang/crates.io-index)",
	"vlqencoding 0.1.0",
	]

	[[package]]
	name = "tempdir"			name = "tempdir"
	version = "0.3.5"			version = "0.3.5"
	source = "registry+https://github.com/rust-lang/crates.io-index"			source = "registry+https://github.com/rust-lang/crates.io-index"
	dependencies = [			dependencies = [
	"rand 0.3.18 (registry+https://github.com/rust-lang/crates.io-index)",			"rand 0.3.18 (registry+https://github.com/rust-lang/crates.io-index)",
	]			]

	[[package]]			[[package]]
	name = "thread_local"			name = "thread_local"
	version = "0.2.7"			version = "0.2.7"
	source = "registry+https://github.com/rust-lang/crates.io-index"			source = "registry+https://github.com/rust-lang/crates.io-index"
	dependencies = [			dependencies = [
	"thread-id 2.0.0 (registry+https://github.com/rust-lang/crates.io-index)",			"thread-id 2.0.0 (registry+https://github.com/rust-lang/crates.io-index)",
	]			]

	[[package]]			[[package]]
				name = "treedirstate"
				version = "0.1.0"
				dependencies = [
				"byteorder 1.1.0 (registry+https://github.com/rust-lang/crates.io-index)",
				"cpython 0.1.0 (git+https://github.com/dgrunwald/rust-cpython.git)",
				"error-chain 0.11.0 (registry+https://github.com/rust-lang/crates.io-index)",
				"itertools 0.7.2 (registry+https://github.com/rust-lang/crates.io-index)",
				"quickcheck 0.4.1 (registry+https://github.com/rust-lang/crates.io-index)",
				"tempdir 0.3.5 (registry+https://github.com/rust-lang/crates.io-index)",
				"vlqencoding 0.1.0",
				]

				[[package]]
	name = "utf8-ranges"			name = "utf8-ranges"
	version = "0.1.3"			version = "0.1.3"
	source = "registry+https://github.com/rust-lang/crates.io-index"			source = "registry+https://github.com/rust-lang/crates.io-index"

	[[package]]			[[package]]
	name = "vlqencoding"			name = "vlqencoding"
	version = "0.1.0"			version = "0.1.0"

rust/treedirstate/Cargo.toml

	[package]			[package]
	name = "rusttreedirstate"			name = "treedirstate"
	version = "0.1.0"			version = "0.1.0"
	authors = ["Facebook Source Control Team <sourcecontrol-dev@fb.com>"]			authors = ["Facebook Source Control Team <sourcecontrol-dev@fb.com>"]

	[profile.release]			[profile.release]
	lto = true			lto = true

	[lib]			[lib]
	name = "rusttreedirstate"			name = "treedirstate"
	crate-type = ["cdylib"]			crate-type = ["cdylib"]

	[dependencies]			[dependencies]
	byteorder = "*"			byteorder = "*"
	error-chain = "*"			error-chain = "*"
	vlqencoding = { version = "*", path = "../vlqencoding" }			vlqencoding = { version = "*", path = "../vlqencoding" }

	[dev-dependencies]			[dev-dependencies]

rust/treedirstate/src/lib.rs


	extern crate vlqencoding;			extern crate vlqencoding;

	pub mod dirstate;			pub mod dirstate;
	pub mod errors;			pub mod errors;
	pub mod filestate;			pub mod filestate;
	pub mod filestore;			pub mod filestore;
	#[cfg(not(test))]			#[cfg(not(test))]
				#[allow(non_camel_case_types)]
	pub mod python;			pub mod python;
	pub mod store;			pub mod store;
	pub mod tree;			pub mod tree;
	pub mod vecmap;			pub mod vecmap;

	pub use errors::*;			pub use errors::*;

rust/treedirstate/src/python.rs

	// Copyright Facebook, Inc. 2017			// Copyright Facebook, Inc. 2017
	//! Python bindings for treedirstate.			//! Python bindings for treedirstate.

	use cpython::*;			use cpython::*;
	use cpython::exc;			use cpython::exc;
	use dirstate::Dirstate;			use dirstate::Dirstate;
	use errors::ErrorKind;			use errors::ErrorKind;
	use errors;			use errors;
	use filestate::FileState;			use filestate::FileState;
	use std::cell::RefCell;			use std::cell::RefCell;
	use std::path::PathBuf;			use std::path::PathBuf;
	use store::BlockId;			use store::BlockId;
	use tree::{Key, KeyRef};			use tree::{Key, KeyRef};

	py_module_initializer!(			py_module_initializer!(
	rusttreedirstate,			treedirstate,
	initrusttreedirstate,			inittreedirstate,
	PyInit_rusttreedirstate,			PyInit_treedirstate,
	\|py, m\| {			\|py, m\| {
	m.add_class::<RustDirstateMap>(py)?;			m.add_class::<treedirstatemap>(py)?;
	Ok(())			Ok(())
	}			}
	);			);

	fn callback_error(py: Python, mut e: PyErr) -> ErrorKind {			fn callback_error(py: Python, mut e: PyErr) -> ErrorKind {
	let s = e.instance(py)			let s = e.instance(py)
	.extract::<String>(py)			.extract::<String>(py)
	.unwrap_or_else(\|_e\| "unknown exception".to_string());			.unwrap_or_else(\|_e\| "unknown exception".to_string());
	ErrorKind::CallbackError(s)			ErrorKind::CallbackError(s)
	}			}

	py_class!(class RustDirstateMap \|py\| {			py_class!(class treedirstatemap \|py\| {
	data repodir: PathBuf;			data repodir: PathBuf;
	data dirstate: RefCell<Dirstate<FileState>>;			data dirstate: RefCell<Dirstate<FileState>>;
	data casefolderid: RefCell<Option<usize>>;			data casefolderid: RefCell<Option<usize>>;

	def __new__(			def __new__(
	_cls,			_cls,
	_ui: &PyObject,			_ui: &PyObject,
	opener: &PyObject			opener: &PyObject
	) -> PyResult<RustDirstateMap> {			) -> PyResult<treedirstatemap> {
	let repodir = opener.getattr(py, "base")?.extract::<String>(py)?;			let repodir = opener.getattr(py, "base")?.extract::<String>(py)?;
	let dirstate = Dirstate::new();			let dirstate = Dirstate::new();
	RustDirstateMap::create_instance(			treedirstatemap::create_instance(
	py,			py,
	repodir.into(),			repodir.into(),
	RefCell::new(dirstate),			RefCell::new(dirstate),
	RefCell::new(None))			RefCell::new(None))
	}			}

	def clear(&self) -> PyResult<PyObject> {			def clear(&self) -> PyResult<PyObject> {
	let mut dirstate = self.dirstate(py).borrow_mut();			let mut dirstate = self.dirstate(py).borrow_mut();

setup.py

	from distutils.version import LooseVersion			from distutils.version import LooseVersion
	from distutils.cmd import Command			from distutils.cmd import Command
	from distutils.core import setup, Extension			from distutils.core import setup, Extension
	import distutils			import distutils
				from distutils_rust import RustExtension, BuildRustExt
	import fnmatch			import fnmatch
	from glob import glob			from glob import glob

	import os, shutil, sys			import os, shutil, sys

	iswindows = os.name == 'nt'			iswindows = os.name == 'nt'
	WERROR = "/WX" if iswindows else "-Werror"			WERROR = "/WX" if iswindows else "-Werror"
	WSTRICTPROTOTYPES = None if iswindows else "-Werror=strict-prototypes"			WSTRICTPROTOTYPES = None if iswindows else "-Werror=strict-prototypes"
	WALL = "/Wall" if iswindows else "-Wall"			WALL = "/Wall" if iswindows else "-Wall"
	STDC99 = "" if iswindows else "-std=c99"			STDC99 = "" if iswindows else "-std=c99"
	STDCPP0X = "" if iswindows else "-std=c++0x"			STDCPP0X = "" if iswindows else "-std=c++0x"
	WEXTRA = "" if iswindows else "-Wextra"			WEXTRA = "" if iswindows else "-Wextra"
	WCONVERSION = "" if iswindows else "-Wconversion"			WCONVERSION = "" if iswindows else "-Wconversion"
	PEDANTIC = "" if iswindows else "-pedantic"			PEDANTIC = "" if iswindows else "-pedantic"
	SHA1LIB_DEFINE = "/DSHA1_USE_SHA1DC" if iswindows else "-DSHA1_USE_SHA1DC"			SHA1LIB_DEFINE = "/DSHA1_USE_SHA1DC" if iswindows else "-DSHA1_USE_SHA1DC"
	SHA1_LIBRARY = "sha1detectcoll"			SHA1_LIBRARY = "sha1detectcoll"
	NOOPTIMIZATION = "/Od" if iswindows else "-O0"			NOOPTIMIZATION = "/Od" if iswindows else "-O0"
	OPTIMIZATION = "" if iswindows else "-O2"			OPTIMIZATION = "" if iswindows else "-O2"
	PRODUCEDEBUGSYMBOLS = "/DEBUG:FULL" if iswindows else "-g"			PRODUCEDEBUGSYMBOLS = "/DEBUG:FULL" if iswindows else "-g"

				if 'USERUST' in os.environ:
				USERUST = int(os.environ['USERUST'])
				else:
				import subprocess
				USERUST = False
				try:
				cargo_version = subprocess.check_output(
				['cargo', '--version']).split()[1]
				quarkUnsubmitted Not Done Maybe this should be a hard error so people won't accidentally miss dependencies. quark: Maybe this should be a hard error so people won't accidentally miss dependencies.
				mbthomasAuthorUnsubmitted Not Done I don't want to break users who simply don't have a rust toolchain installed from using the rest of fb-hgext. If `USERUST` is `1` then it's a hard failure, but if `USERUST` is not set then you just don't get rust-based code if you don't have a rust toolchain. We can change this later once rust toolchains are more commonplace and necessary. mbthomas: I don't want to break users who simply don't have a rust toolchain installed from using the…
				durhamUnsubmitted Not Done We could at least print a warning to stderr, so it's more debugable in build outputs. durham: We could at least print a warning to stderr, so it's more debugable in build outputs.
				except Exception:
				sys.stderr.write(
				"not compiling Rust extensions: cargo is not available\n")
				else:
				required_cargo_version = '0.21'
				if (LooseVersion(cargo_version) >=
				LooseVersion(required_cargo_version)):
				USERUST = True
				else:
				sys.stderr.write(
				"not compiling Rust extensions: cargo is too old " +
				"(found %s, need %s or higher)\n"
				% (cargo_version, required_cargo_version))

	# whether to use Cython to recompile .pyx to .c/.cpp at build time.			# whether to use Cython to recompile .pyx to .c/.cpp at build time.
	# if False, fallback to .c/.cpp in the repo and .pyx files are ignored.			# if False, fallback to .c/.cpp in the repo and .pyx files are ignored.
	# if True, re-compile .c/.cpp from .pyx files, require cython at build time.			# if True, re-compile .c/.cpp from .pyx files, require cython at build time.
	if 'USECYTHON' in os.environ:			if 'USECYTHON' in os.environ:
	USECYTHON = int(os.environ['USECYTHON'])			USECYTHON = int(os.environ['USECYTHON'])
	else:			else:
	try:			try:
	import Cython			import Cython
	self.announce(summary, level=distutils.log.INFO)			self.announce(summary, level=distutils.log.INFO)

	def _rglob(self, patten):			def _rglob(self, patten):
	# recursive glob			# recursive glob
	for dirname, dirs, files in os.walk('.'):			for dirname, dirs, files in os.walk('.'):
	for name in fnmatch.filter(files, patten):			for name in fnmatch.filter(files, patten):
	yield os.path.join(dirname, name)			yield os.path.join(dirname, name)

				rust_ext_modules = []
				if USERUST:
				rust_ext_modules.extend([
				RustExtension('treedirstate',
				package='hgext3rd.rust',
				manifest='rust/treedirstate/Cargo.toml',
				),
				])

	setup(			setup(
	name='fbhgext',			name='fbhgext',
	version='1.0',			version='1.0',
	author='Facebook Source Control Team',			author='Facebook Source Control Team',
	maintainer='Facebook Source Control Team',			maintainer='Facebook Source Control Team',
	maintainer_email='sourcecontrol-dev@fb.com',			maintainer_email='sourcecontrol-dev@fb.com',
	url='https://bitbucket.org/facebook/hg-experimental/',			url='https://bitbucket.org/facebook/hg-experimental/',
	description='Facebook mercurial extensions',			description='Facebook mercurial extensions',
	long_description="",			long_description="",
	keywords='facebook fb hg mercurial shallow remote filelog',			keywords='facebook fb hg mercurial shallow remote filelog',
	license='GPLv2+',			license='GPLv2+',
	packages=packages,			packages=packages,
	install_requires=requires,			install_requires=requires,
	py_modules=py_modules,			py_modules=py_modules,
	ext_modules = ext_modules,			ext_modules=ext_modules,
	libraries=libraries,			libraries=libraries,
				rust_ext_modules=rust_ext_modules,
	cmdclass={			cmdclass={
	'clean_ext': CleanExtCommand,			'clean_ext': CleanExtCommand,
				'build_rust_ext': BuildRustExt,
	}			}
	)			)

treedirstate/Makefile

This file was deleted.

	NAME := rusttreedirstate

	# Cargo's depfiles use full paths, so we must include the full path
	# to the rust library.
	DIR := $(realpath $(join $(dir $(firstword $(MAKEFILE_LIST))),../rust/treedirstate))

	MODE ?= release

	ifeq ($(MODE),release)
	CARGO_ARGS += --release
	endif

	ifeq ($(shell uname),Darwin)
	LIBSUFFIX := .dylib
	else
	LIBSUFFIX := .so
	endif

	RUSTLIB := $(DIR)/target/$(MODE)/lib$(NAME)$(LIBSUFFIX)
	RUSTDEP := $(DIR)/target/$(MODE)/lib$(NAME).d

	.PHONY: all
	all: $(NAME).so

	$(NAME).so: $(RUSTLIB)
	cp $< $@

	$(RUSTLIB):
	cd $(DIR) && cargo build $(CARGO_ARGS)

	.PHONY: test
	test:
	cd $(DIR) && cargo test

	.PHONY: clean
	clean:
	$(RM) $(NAME).so
	cd $(DIR) && cargo clean

	# This pattern rule for Rust source files forces a rebuild if any source file
	# is deleted.
	$(DIR)/src/%.rs: ;

	-include $(RUSTDEP)

treedirstate/init.py

This file was moved to hgext3rd/treedirstate.py.

		Path
V		{ ← rust/treedirstate}/.cargo/config
M		Makefile (3 lines)
A	M	hgext3rd/rust/__init__.py (6 lines)
V		{hgext3rd/treedirstate.py ← treedirstate/__init__.py} (4 lines)
V		{rust/treedirstate → }/.cargo/config
M		rust/treedirstate/Cargo.lock (26 lines)
M		rust/treedirstate/Cargo.toml (4 lines)
M		rust/treedirstate/src/lib.rs (1 line)
M		rust/treedirstate/src/python.rs (14 lines)
M		setup.py (36 lines)
D	M	treedirstate/Makefile (44 lines)
V		{treedirstate/__init__.py → hgext3rd/treedirstate.py}

This is an archive of the discontinued Mercurial Phabricator instance.

setup: build treedirstate and rusttreedirstate packages
ClosedPublic

Details

Diff Detail

Event Timeline

Revision Contents
Changeset List

Diff 3941

.cargo/config

Makefile

hgext3rd/rust/init.py

hgext3rd/treedirstate.py

rust/treedirstate/.cargo/config

rust/treedirstate/Cargo.lock

rust/treedirstate/Cargo.toml

rust/treedirstate/src/lib.rs

rust/treedirstate/src/python.rs

setup.py

treedirstate/Makefile

treedirstate/init.py

This is an archive of the discontinued Mercurial Phabricator instance.

setup: build treedirstate and rusttreedirstate packagesClosedPublic

Details

Diff Detail

Event Timeline

Revision ContentsChangeset List

Diff 3941

.cargo/config

Makefile

hgext3rd/rust/__init__.py

hgext3rd/treedirstate.py

rust/treedirstate/.cargo/config

rust/treedirstate/Cargo.lock

rust/treedirstate/Cargo.toml

rust/treedirstate/src/lib.rs

rust/treedirstate/src/python.rs

setup.py

treedirstate/Makefile

treedirstate/__init__.py

setup: build treedirstate and rusttreedirstate packages
ClosedPublic

Revision Contents
Changeset List

hgext3rd/rust/init.py

treedirstate/init.py