nodemap: keep track of the ondisk id of nodemap blocks
ClosedPublic

Diff	ID	Description	Created	Lint	Unit
Base		Base
Diff 1	19301		Jan 15 2020, 9:55 AM	★	★
Diff 2	19764		Jan 31 2020, 10:29 AM	★	★
Diff 3	19792		Jan 31 2020, 5:59 PM	★	★
Diff 4	19836		Feb 2 2020, 3:18 AM	★	★
Diff 5	19895		Feb 4 2020, 7:25 PM	★	★
Diff 6	20116	rHGf0862ee1a31e81c48585bd93396de90a67836be1	Jan 15 2020, 9:49 AM	★	★

Status	Author	Revision
Closed	marmoute	D7894 nodemap: introduce an option to use mmap to read the nodemap mapping
Closed	marmoute	D7893 nodemap: update the index with the newly written data (when appropriate)
Closed	marmoute	D7892 nodemap: never read more than the expected data amount
Closed	marmoute	D7891 nodemap: write new data from the expected current data length
Closed	marmoute	D7890 nodemap: double check the source docket when doing incremental update
Closed	marmoute	D7889 nodemap: track the total and unused amount of data in the rawdata file
Closed	marmoute	D7888 nodemap: track the maximum revision tracked in the nodemap
Closed	marmoute	D7887 nodemap: add a flag to dump the details of the docket
Closed	marmoute	D7886 nodemap: introduce append-only incremental update of the persisten data
Closed	marmoute	D7885 nodemap: keep track of the docket for loaded data
Closed	marmoute	D7884 nodemap: introduce an explicit class/object for the docket
Closed	marmoute	D7883 nodemap: keep track of the ondisk id of nodemap blocks
Closed	marmoute	D7847 nodemap: provide the on disk data to indexes who support it
Closed	marmoute	D7846 nodemap: all check that revision and nodes match in the nodemap
Closed	marmoute	D7845 nodemap: add basic checking of the on disk nodemap content
Closed	marmoute	D7844 nodemap: code to parse the persistent binary nodemap data
Closed	marmoute	D7843 nodemap: move the iteratio inside the Block object
Closed	marmoute	D7842 nodemap: use an explicit "Block" object in the reference implementation
Closed	marmoute	D7841 nodemap: add a optional `nodemap_add_full` method on indexes
Closed	marmoute	D7840 nodemap: add a (python) index class for persistent nodemap testing
Closed	marmoute	D7839 nodemap: delete older raw data file when creating a new ones
Closed	marmoute	D7838 nodemap: use an intermediate "docket" file to carry small metadata
Closed	marmoute	D7837 nodemap: only use persistent nodemap for non-inlined revlog
Closed	marmoute	D7836 nodemap: add a function to read the data from disk
Closed	marmoute	D7835 nodemap: write nodemap data on disk
Closed	marmoute	D7834 nodemap: have some python code writing a nodemap in persistent binary form

Diff 20116

mercurial/revlogutils/nodemap.py

	return int(hex_digit, 16)			return int(hex_digit, 16)


	class Block(dict):			class Block(dict):
	"""represent a block of the Trie			"""represent a block of the Trie

	contains up to 16 entry indexed from 0 to 15"""			contains up to 16 entry indexed from 0 to 15"""

				def __init__(self):
				super(Block, self).__init__()
				# If this block exist on disk, here is its ID
				self.ondisk_id = None

	def __iter__(self):			def __iter__(self):
	return iter(self.get(i) for i in range(16))			return iter(self.get(i) for i in range(16))


	def _build_trie(index):			def _build_trie(index):
	"""build a nodemap trie			"""build a nodemap trie

	The nodemap stores revision number for each unique prefix.			The nodemap stores revision number for each unique prefix.
	msg = "nodemap data size is not a multiple of block size (%d): %d"			msg = "nodemap data size is not a multiple of block size (%d): %d"
	raise error.Abort(msg % (S_BLOCK.size, len(data)))			raise error.Abort(msg % (S_BLOCK.size, len(data)))
	if not data:			if not data:
	return Block()			return Block()
	block_map = {}			block_map = {}
	new_blocks = []			new_blocks = []
	for i in range(0, len(data), S_BLOCK.size):			for i in range(0, len(data), S_BLOCK.size):
	block = Block()			block = Block()
	ondisk_id = len(block_map)			block.ondisk_id = len(block_map)
	block_map[ondisk_id] = block			block_map[block.ondisk_id] = block
	block_data = data[i : i + S_BLOCK.size]			block_data = data[i : i + S_BLOCK.size]
	values = S_BLOCK.unpack(block_data)			values = S_BLOCK.unpack(block_data)
	new_blocks.append((block, values))			new_blocks.append((block, values))
	for b, values in new_blocks:			for b, values in new_blocks:
	for idx, v in enumerate(values):			for idx, v in enumerate(values):
	if v == NO_ENTRY:			if v == NO_ENTRY:
	continue			continue
	elif v >= 0:			elif v >= 0:

This is an archive of the discontinued Mercurial Phabricator instance.

nodemap: keep track of the ondisk id of nodemap blocksClosedPublic

Details

Diff Detail

Event Timeline

Revision ContentsChangeset List

Diff 20116

mercurial/revlogutils/nodemap.py

nodemap: keep track of the ondisk id of nodemap blocks
ClosedPublic

Revision Contents
Changeset List