nodemap: use an explicit "Block" object in the reference implementation
ClosedPublic

Diff	ID	Description	Created	Lint	Unit
Base		Base
Diff 1	19179		Jan 13 2020, 10:07 AM	★	★
Diff 2	19296		Jan 15 2020, 9:54 AM	★	★
Diff 3	19758		Jan 31 2020, 10:28 AM	★	★
Diff 4	19787		Jan 31 2020, 5:58 PM	★	★
Diff 5	19831		Feb 2 2020, 3:18 AM	★	★
Diff 6	19890		Feb 4 2020, 7:24 PM	★	★
Diff 7	20110	rHG7762a295fd4ded00e7f2c20c29a56bfef63c18c5	Jan 15 2020, 9:48 AM	★	★

Status	Author	Revision
Closed	marmoute	D7894 nodemap: introduce an option to use mmap to read the nodemap mapping
Closed	marmoute	D7893 nodemap: update the index with the newly written data (when appropriate)
Closed	marmoute	D7892 nodemap: never read more than the expected data amount
Closed	marmoute	D7891 nodemap: write new data from the expected current data length
Closed	marmoute	D7890 nodemap: double check the source docket when doing incremental update
Closed	marmoute	D7889 nodemap: track the total and unused amount of data in the rawdata file
Closed	marmoute	D7888 nodemap: track the maximum revision tracked in the nodemap
Closed	marmoute	D7887 nodemap: add a flag to dump the details of the docket
Closed	marmoute	D7886 nodemap: introduce append-only incremental update of the persisten data
Closed	marmoute	D7885 nodemap: keep track of the docket for loaded data
Closed	marmoute	D7884 nodemap: introduce an explicit class/object for the docket
Closed	marmoute	D7883 nodemap: keep track of the ondisk id of nodemap blocks
Closed	marmoute	D7847 nodemap: provide the on disk data to indexes who support it
Closed	marmoute	D7846 nodemap: all check that revision and nodes match in the nodemap
Closed	marmoute	D7845 nodemap: add basic checking of the on disk nodemap content
Closed	marmoute	D7844 nodemap: code to parse the persistent binary nodemap data
Closed	marmoute	D7843 nodemap: move the iteratio inside the Block object
Closed	marmoute	D7842 nodemap: use an explicit "Block" object in the reference implementation
Closed	marmoute	D7841 nodemap: add a optional `nodemap_add_full` method on indexes
Closed	marmoute	D7840 nodemap: add a (python) index class for persistent nodemap testing
Closed	marmoute	D7839 nodemap: delete older raw data file when creating a new ones
Closed	marmoute	D7838 nodemap: use an intermediate "docket" file to carry small metadata
Closed	marmoute	D7837 nodemap: only use persistent nodemap for non-inlined revlog
Closed	marmoute	D7836 nodemap: add a function to read the data from disk
Closed	marmoute	D7835 nodemap: write nodemap data on disk
Closed	marmoute	D7834 nodemap: have some python code writing a nodemap in persistent binary form

Diff 20110

mercurial/revlogutils/nodemap.py

	return -(rev + REV_OFFSET)			return -(rev + REV_OFFSET)


	def _to_int(hex_digit):			def _to_int(hex_digit):
	"""turn an hexadecimal digit into a proper integer"""			"""turn an hexadecimal digit into a proper integer"""
	return int(hex_digit, 16)			return int(hex_digit, 16)


				class Block(dict):
				"""represent a block of the Trie

				contains up to 16 entry indexed from 0 to 15"""


				durin42Unsubmitted Not Done nit: you could have omitted this superfluous pass statement durin42: nit: you could have omitted this superfluous pass statement
	def _build_trie(index):			def _build_trie(index):
	"""build a nodemap trie			"""build a nodemap trie

	The nodemap stores revision number for each unique prefix.			The nodemap stores revision number for each unique prefix.

	Each block is a dictionary with keys in `[0, 15]`. Values are either			Each block is a dictionary with keys in `[0, 15]`. Values are either
	another block or a revision number.			another block or a revision number.
	"""			"""
	root = {}			root = Block()
	for rev in range(len(index)):			for rev in range(len(index)):
	hex = nodemod.hex(index[rev][7])			hex = nodemod.hex(index[rev][7])
	_insert_into_block(index, 0, root, rev, hex)			_insert_into_block(index, 0, root, rev, hex)
	return root			return root


	def _insert_into_block(index, level, block, current_rev, current_hex):			def _insert_into_block(index, level, block, current_rev, current_hex):
	"""insert a new revision in a block			"""insert a new revision in a block
	elif isinstance(entry, dict):			elif isinstance(entry, dict):
	# need to recurse to an underlying block			# need to recurse to an underlying block
	_insert_into_block(index, level + 1, entry, current_rev, current_hex)			_insert_into_block(index, level + 1, entry, current_rev, current_hex)
	else:			else:
	# collision with a previously unique prefix, inserting new			# collision with a previously unique prefix, inserting new
	# vertices to fit both entry.			# vertices to fit both entry.
	other_hex = nodemod.hex(index[entry][7])			other_hex = nodemod.hex(index[entry][7])
	other_rev = entry			other_rev = entry
	new = {}			new = Block()
	block[hex_digit] = new			block[hex_digit] = new
	_insert_into_block(index, level + 1, new, other_rev, other_hex)			_insert_into_block(index, level + 1, new, other_rev, other_hex)
	_insert_into_block(index, level + 1, new, current_rev, current_hex)			_insert_into_block(index, level + 1, new, current_rev, current_hex)


	def _persist_trie(root):			def _persist_trie(root):
	"""turn a nodemap trie into persistent binary data			"""turn a nodemap trie into persistent binary data

This is an archive of the discontinued Mercurial Phabricator instance.

nodemap: use an explicit "Block" object in the reference implementationClosedPublic

Details

Diff Detail

Event Timeline

Revision ContentsChangeset List

Diff 20110

mercurial/revlogutils/nodemap.py

nodemap: use an explicit "Block" object in the reference implementation
ClosedPublic

Revision Contents
Changeset List