mercurial/changegroup.py
1000–1001	A better thing would be for the `strip` case to properly setup the bundle call to make sure the sidedata are removed. The current code seems okay for now, but adding a clear comment about how we should do that ideally would be great.
mercurial/utils/storageutil.py
515	This will need some doc about what is the semantic of this function.

mercurial/changegroup.py
1000–1001	But we don't want to remove the sidedata, we want to leave them there. When restoring a strip bundle, we want the sidedata to be the same as before, so we might as well not drop them only to re-generate them.

mercurial/changegroup.py
1000–1001	My point is that we should (eventually) have the proper way to declare the wanted sidedata when we generate a bundle, and the higher level strip code will set this properly before genering the backup bundle.

Diff	ID	Description	Created	Lint	Unit
Base		Base
Diff 1	25674		Feb 19 2021, 6:16 AM	★	★
Diff 2	25996		Mar 1 2021, 11:52 AM	★	★
Diff 3	26092		Mar 4 2021, 10:18 AM	★	★
Diff 4	26223		Mar 10 2021, 2:12 PM	★	★
Diff 5	26257		Mar 12 2021, 6:34 AM	★	★
Diff 6	26334		Mar 15 2021, 6:24 AM	★	★
Diff 7	26381	rHG45f0d52976983e5fb7434c0d4630bbed0c34401b	Feb 19 2021, 5:15 AM	★	★

Commit	Parents	Author	Summary	Date
22a7cf9125fa	b9ef7b64691c	Raphaël Gomès		Feb 19 2021, 5:15 AM

Status	Author	Revision
Closed	Alphare	D10106 requirements: also add a generaldelta constant
Closed	Alphare	D10105 requirements: add constant for revlog v1 requirement
Closed	Alphare	D10024 error: add `hint` attribute to `SidedataHashError`
Closed	Alphare	D10023 changegroup: use the local variable instead of reaching through self
Closed	Alphare	D10216 configitems: add TODOs blocking the move out of experimental for revlogv2
Closed	Alphare	D10032 sidedata-exchange: rewrite sidedata on-the-fly whenever possible
Closed	Alphare	D10031 revlog-index: add `replace_sidedata_info` method
Closed	Alphare	D10030 revlogv2: temporarily forbid inline revlogs
Closed	Alphare	D10029 changegroupv4: add sidedata helpers
Closed	Alphare	D10151 revlog: add attribute on revlogs that specifies its kind
Closed	Alphare	D10028 sidedata-exchange: add `wanted_sidedata` and `sidedata_computers` to repos
Closed	Alphare	D10027 delta: add sidedata field to revision delta
Closed	Alphare	D10026 changegroup: add v4 changegroup for revlog v2 exchange
Closed	Alphare	D10025 revlogv2: don't assume that the sidedata of the last rev is right after data
Closed	Alphare	D9993 sidedata: move to new sidedata storage in revlogv2
Closed	Alphare	D9846 cext: add support for revlogv2
Closed	Alphare	D9845 bitmanipulation: add utils to read/write bigendian 64bit integers
Closed	Alphare	D10113 format: remove sidedata format variant
Closed	Alphare	D9844 revlogv2: allow upgrading to v2
Closed	Alphare	D9843 revlog: introduce v2 format
Closed	Alphare	D10109 requirements: also add a fncache constant
Closed	Alphare	D10108 requirements: also add a store constant
Closed	Alphare	D10107 requirements: also add a dotencode constant
Closed	Alphare	D10104 pure-parsers: document index class constants

Diff 26257

hgext/sqlitestore.py


	def emitrevisions(			def emitrevisions(
	self,			self,
	nodes,			nodes,
	nodesorder=None,			nodesorder=None,
	revisiondata=False,			revisiondata=False,
	assumehaveparentrevisions=False,			assumehaveparentrevisions=False,
	deltamode=repository.CG_DELTAMODE_STD,			deltamode=repository.CG_DELTAMODE_STD,
				sidedata_helpers=None,
	):			):
	if nodesorder not in (b'nodes', b'storage', b'linear', None):			if nodesorder not in (b'nodes', b'storage', b'linear', None):
	raise error.ProgrammingError(			raise error.ProgrammingError(
	b'unhandled value for nodesorder: %s' % nodesorder			b'unhandled value for nodesorder: %s' % nodesorder
	)			)

	nodes = [n for n in nodes if n != nullid]			nodes = [n for n in nodes if n != nullid]

	self,			self,
	nodes,			nodes,
	nodesorder,			nodesorder,
	sqliterevisiondelta,			sqliterevisiondelta,
	deltaparentfn=deltabases.__getitem__,			deltaparentfn=deltabases.__getitem__,
	revisiondata=revisiondata,			revisiondata=revisiondata,
	assumehaveparentrevisions=assumehaveparentrevisions,			assumehaveparentrevisions=assumehaveparentrevisions,
	deltamode=deltamode,			deltamode=deltamode,
				sidedata_helpers=sidedata_helpers,
	):			):

	yield delta			yield delta

	# End of ifiledata interface.			# End of ifiledata interface.

	# Start of ifilemutation interface.			# Start of ifilemutation interface.

mercurial/changegroup.py

	def apply(			def apply(
	self,			self,
	repo,			repo,
	tr,			tr,
	srctype,			srctype,
	url,			url,
	targetphase=phases.draft,			targetphase=phases.draft,
	expectedtotal=None,			expectedtotal=None,
				sidedata_categories=None,
	):			):
	"""Add the changegroup returned by source.read() to this repo.			"""Add the changegroup returned by source.read() to this repo.
	srctype is a string like 'push', 'pull', or 'unbundle'. url is			srctype is a string like 'push', 'pull', or 'unbundle'. url is
	the URL of the repo where this changegroup is coming from.			the URL of the repo where this changegroup is coming from.

	Return an integer summarizing the change to this repo:			Return an integer summarizing the change to this repo:
	- nothing changed or no source: 0			- nothing changed or no source: 0
	- more heads than before: 1+added heads (2..n)			- more heads than before: 1+added heads (2..n)
	- fewer heads than before: -1-removed heads (-2..-n)			- fewer heads than before: -1-removed heads (-2..-n)
	- number of heads stays the same: 1			- number of heads stays the same: 1

				`sidedata_categories` is an optional set of the remote's sidedata wanted
				categories.
	"""			"""
	repo = repo.unfiltered()			repo = repo.unfiltered()

				# Only useful if we're adding sidedata categories. If both peers have
				# the same categories, then we simply don't do anything.
				if self.version == b'04' and srctype == b'pull':
				sidedata_helpers = get_sidedata_helpers(
				repo,
				sidedata_categories or set(),
				pull=True,
				)
				else:
				sidedata_helpers = None

	def csmap(x):			def csmap(x):
	repo.ui.debug(b"add changeset %s\n" % short(x))			repo.ui.debug(b"add changeset %s\n" % short(x))
	return len(cl)			return len(cl)

	def revmap(x):			def revmap(x):
	return cl.rev(x)			return cl.rev(x)

	try:			try:
	ischangelog,			ischangelog,
	lookup,			lookup,
	forcedeltaparentprev,			forcedeltaparentprev,
	topic=None,			topic=None,
	ellipses=False,			ellipses=False,
	clrevtolocalrev=None,			clrevtolocalrev=None,
	fullclnodes=None,			fullclnodes=None,
	precomputedellipsis=None,			precomputedellipsis=None,
				sidedata_helpers=None,
	):			):
	"""Calculate deltas for a set of revisions.			"""Calculate deltas for a set of revisions.

	Is a generator of ``revisiondelta`` instances.			Is a generator of ``revisiondelta`` instances.

	If topic is not None, progress detail will be generated using this			If topic is not None, progress detail will be generated using this
	topic name (e.g. changesets, manifests, etc).			topic name (e.g. changesets, manifests, etc).

				See `storageutil.emitrevisions` for the doc on `sidedata_helpers`.
	"""			"""
	if not nodes:			if not nodes:
	return			return

	cl = repo.changelog			cl = repo.changelog

	if ischangelog:			if ischangelog:
	# `hg log` shows changesets in storage order. To preserve order			# `hg log` shows changesets in storage order. To preserve order
	deltamode = repository.CG_DELTAMODE_FULL			deltamode = repository.CG_DELTAMODE_FULL

	revisions = store.emitrevisions(			revisions = store.emitrevisions(
	nodes,			nodes,
	nodesorder=nodesorder,			nodesorder=nodesorder,
	revisiondata=True,			revisiondata=True,
	assumehaveparentrevisions=not ellipses,			assumehaveparentrevisions=not ellipses,
	deltamode=deltamode,			deltamode=deltamode,
				sidedata_helpers=sidedata_helpers,
	)			)

	for i, revision in enumerate(revisions):			for i, revision in enumerate(revisions):
	if progress:			if progress:
	progress.update(i + 1)			progress.update(i + 1)

	if ellipses:			if ellipses:
	linknode = linknodes[revision.node]			linknode = linknodes[revision.node]
	"""			"""

	repo = self._repo			repo = self._repo
	cl = repo.changelog			cl = repo.changelog

	self._verbosenote(_(b'uncompressed size of bundle content:\n'))			self._verbosenote(_(b'uncompressed size of bundle content:\n'))
	size = 0			size = 0

				sidedata_helpers = None
				if self.version == b'04':
				remote_sidedata = self._remote_sidedata
				if source == b'strip':
				# We're our own remote when stripping, get the no-op helpers
				# TODO a better approach would be for the strip bundle to
				marmouteUnsubmitted Not Done A better thing would be for the `strip` case to properly setup the bundle call to make sure the sidedata are removed. The current code seems okay for now, but adding a clear comment about how we should do that ideally would be great. marmoute: A better thing would be for the `strip` case to properly setup the bundle call to make sure the…
				AlphareAuthorUnsubmitted Done But we don't want to remove the sidedata, we want to leave them there. When restoring a strip bundle, we want the sidedata to be the same as before, so we might as well not drop them only to re-generate them. Alphare: But we don't want to remove the sidedata, we want to leave them there. When restoring a strip…
				marmouteUnsubmitted Done My point is that we should (eventually) have the proper way to declare the wanted sidedata when we generate a bundle, and the higher level strip code will set this properly before genering the backup bundle. marmoute: My point is that we should (eventually) have the proper way to declare the wanted sidedata when…
				# correctly advertise its sidedata categories directly.
				remote_sidedata = repo._wanted_sidedata
				sidedata_helpers = get_sidedata_helpers(repo, remote_sidedata)

	clstate, deltas = self._generatechangelog(			clstate, deltas = self._generatechangelog(
	cl, clnodes, generate=changelog			cl,
				clnodes,
				generate=changelog,
				sidedata_helpers=sidedata_helpers,
	)			)
	for delta in deltas:			for delta in deltas:
	for chunk in _revisiondeltatochunks(delta, self._builddeltaheader):			for chunk in _revisiondeltatochunks(delta, self._builddeltaheader):
	size += len(chunk)			size += len(chunk)
	yield chunk			yield chunk

	close = closechunk()			close = closechunk()
	size += len(close)			size += len(close)
	it = self.generatemanifests(			it = self.generatemanifests(
	commonrevs,			commonrevs,
	clrevorder,			clrevorder,
	fastpathlinkrev,			fastpathlinkrev,
	manifests,			manifests,
	fnodes,			fnodes,
	source,			source,
	clstate[b'clrevtomanifestrev'],			clstate[b'clrevtomanifestrev'],
				sidedata_helpers=sidedata_helpers,
	)			)

	for tree, deltas in it:			for tree, deltas in it:
	if tree:			if tree:
	assert self.version in (b'03', b'04')			assert self.version in (b'03', b'04')
	chunk = _fileheader(tree)			chunk = _fileheader(tree)
	size += len(chunk)			size += len(chunk)
	yield chunk			yield chunk
	it = self.generatefiles(			it = self.generatefiles(
	changedfiles,			changedfiles,
	commonrevs,			commonrevs,
	source,			source,
	mfdicts,			mfdicts,
	fastpathlinkrev,			fastpathlinkrev,
	fnodes,			fnodes,
	clrevs,			clrevs,
				sidedata_helpers=sidedata_helpers,
	)			)

	for path, deltas in it:			for path, deltas in it:
	h = _fileheader(path)			h = _fileheader(path)
	size = len(h)			size = len(h)
	yield h			yield h

	for delta in deltas:			for delta in deltas:
	chunks = _revisiondeltatochunks(delta, self._builddeltaheader)			chunks = _revisiondeltatochunks(delta, self._builddeltaheader)
	for chunk in chunks:			for chunk in chunks:
	size += len(chunk)			size += len(chunk)
	yield chunk			yield chunk

	close = closechunk()			close = closechunk()
	size += len(close)			size += len(close)
	yield close			yield close

	self._verbosenote(_(b'%8.i %s\n') % (size, path))			self._verbosenote(_(b'%8.i %s\n') % (size, path))

	yield closechunk()			yield closechunk()

	if clnodes:			if clnodes:
	repo.hook(b'outgoing', node=hex(clnodes[0]), source=source)			repo.hook(b'outgoing', node=hex(clnodes[0]), source=source)

	def _generatechangelog(self, cl, nodes, generate=True):			def _generatechangelog(
				self, cl, nodes, generate=True, sidedata_helpers=None
				):
	"""Generate data for changelog chunks.			"""Generate data for changelog chunks.

	Returns a 2-tuple of a dict containing state and an iterable of			Returns a 2-tuple of a dict containing state and an iterable of
	byte chunks. The state will not be fully populated until the			byte chunks. The state will not be fully populated until the
	chunk stream has been fully consumed.			chunk stream has been fully consumed.

	if generate is False, the state will be fully populated and no chunk			if generate is False, the state will be fully populated and no chunk
	stream will be yielded			stream will be yielded

				See `storageutil.emitrevisions` for the doc on `sidedata_helpers`.
	"""			"""
	clrevorder = {}			clrevorder = {}
	manifests = {}			manifests = {}
	mfl = self._repo.manifestlog			mfl = self._repo.manifestlog
	changedfiles = set()			changedfiles = set()
	clrevtomanifestrev = {}			clrevtomanifestrev = {}

	state = {			state = {
	True,			True,
	lookupcl,			lookupcl,
	self._forcedeltaparentprev,			self._forcedeltaparentprev,
	ellipses=self._ellipses,			ellipses=self._ellipses,
	topic=_(b'changesets'),			topic=_(b'changesets'),
	clrevtolocalrev={},			clrevtolocalrev={},
	fullclnodes=self._fullclnodes,			fullclnodes=self._fullclnodes,
	precomputedellipsis=self._precomputedellipsis,			precomputedellipsis=self._precomputedellipsis,
				sidedata_helpers=sidedata_helpers,
	)			)

	return state, gen			return state, gen

	def generatemanifests(			def generatemanifests(
	self,			self,
	commonrevs,			commonrevs,
	clrevorder,			clrevorder,
	fastpathlinkrev,			fastpathlinkrev,
	manifests,			manifests,
	fnodes,			fnodes,
	source,			source,
	clrevtolocalrev,			clrevtolocalrev,
				sidedata_helpers=None,
	):			):
	"""Returns an iterator of changegroup chunks containing manifests.			"""Returns an iterator of changegroup chunks containing manifests.

	`source` is unused here, but is used by extensions like remotefilelog to			`source` is unused here, but is used by extensions like remotefilelog to
	change what is sent based in pulls vs pushes, etc.			change what is sent based in pulls vs pushes, etc.

				See `storageutil.emitrevisions` for the doc on `sidedata_helpers`.
	"""			"""
	repo = self._repo			repo = self._repo
	mfl = repo.manifestlog			mfl = repo.manifestlog
	tmfnodes = {b'': manifests}			tmfnodes = {b'': manifests}

	# Callback for the manifest, used to collect linkrevs for filelog			# Callback for the manifest, used to collect linkrevs for filelog
	# revisions.			# revisions.
	# Returns the linkrev node (collected in lookupcl).			# Returns the linkrev node (collected in lookupcl).
	False,			False,
	lookupfn,			lookupfn,
	self._forcedeltaparentprev,			self._forcedeltaparentprev,
	ellipses=self._ellipses,			ellipses=self._ellipses,
	topic=_(b'manifests'),			topic=_(b'manifests'),
	clrevtolocalrev=clrevtolocalrev,			clrevtolocalrev=clrevtolocalrev,
	fullclnodes=self._fullclnodes,			fullclnodes=self._fullclnodes,
	precomputedellipsis=self._precomputedellipsis,			precomputedellipsis=self._precomputedellipsis,
				sidedata_helpers=sidedata_helpers,
	)			)

	if not self._oldmatcher.visitdir(store.tree[:-1]):			if not self._oldmatcher.visitdir(store.tree[:-1]):
	yield tree, deltas			yield tree, deltas
	else:			else:
	# 'deltas' is a generator and we need to consume it even if			# 'deltas' is a generator and we need to consume it even if
	# we are not going to send it because a side-effect is that			# we are not going to send it because a side-effect is that
	# it updates tmdnodes (via lookupfn)			# it updates tmdnodes (via lookupfn)
	self,			self,
	changedfiles,			changedfiles,
	commonrevs,			commonrevs,
	source,			source,
	mfdicts,			mfdicts,
	fastpathlinkrev,			fastpathlinkrev,
	fnodes,			fnodes,
	clrevs,			clrevs,
				sidedata_helpers=None,
	):			):
	changedfiles = [			changedfiles = [
	f			f
	for f in changedfiles			for f in changedfiles
	if self._matcher(f) and not self._oldmatcher(f)			if self._matcher(f) and not self._oldmatcher(f)
	]			]

	if not fastpathlinkrev:			if not fastpathlinkrev:
	filenodes,			filenodes,
	False,			False,
	lookupfilelog,			lookupfilelog,
	self._forcedeltaparentprev,			self._forcedeltaparentprev,
	ellipses=self._ellipses,			ellipses=self._ellipses,
	clrevtolocalrev=clrevtolocalrev,			clrevtolocalrev=clrevtolocalrev,
	fullclnodes=self._fullclnodes,			fullclnodes=self._fullclnodes,
	precomputedellipsis=self._precomputedellipsis,			precomputedellipsis=self._precomputedellipsis,
				sidedata_helpers=sidedata_helpers,
	)			)

	yield fname, deltas			yield fname, deltas

	progress.complete()			progress.complete()


	def _makecg1packer(			def _makecg1packer(
	fl.rev(n)			fl.rev(n)
	except error.LookupError:			except error.LookupError:
	raise error.Abort(			raise error.Abort(
	_(b'missing file data for %s:%s - run hg verify')			_(b'missing file data for %s:%s - run hg verify')
	% (f, hex(n))			% (f, hex(n))
	)			)

	return revisions, files			return revisions, files


				def get_sidedata_helpers(repo, remote_sd_categories, pull=False):
				# Computers for computing sidedata on-the-fly
				sd_computers = collections.defaultdict(list)
				# Computers for categories to remove from sidedata
				sd_removers = collections.defaultdict(list)

				to_generate = remote_sd_categories - repo._wanted_sidedata
				to_remove = repo._wanted_sidedata - remote_sd_categories
				if pull:
				to_generate, to_remove = to_remove, to_generate

				for revlog_kind, computers in repo._sidedata_computers.items():
				for category, computer in computers.items():
				if category in to_generate:
				sd_computers[revlog_kind].append(computer)
				if category in to_remove:
				sd_removers[revlog_kind].append(computer)

				sidedata_helpers = (repo, sd_computers, sd_removers)
				return sidedata_helpers

mercurial/filelog.py


	def emitrevisions(			def emitrevisions(
	self,			self,
	nodes,			nodes,
	nodesorder=None,			nodesorder=None,
	revisiondata=False,			revisiondata=False,
	assumehaveparentrevisions=False,			assumehaveparentrevisions=False,
	deltamode=repository.CG_DELTAMODE_STD,			deltamode=repository.CG_DELTAMODE_STD,
				sidedata_helpers=None,
	):			):
	return self._revlog.emitrevisions(			return self._revlog.emitrevisions(
	nodes,			nodes,
	nodesorder=nodesorder,			nodesorder=nodesorder,
	revisiondata=revisiondata,			revisiondata=revisiondata,
	assumehaveparentrevisions=assumehaveparentrevisions,			assumehaveparentrevisions=assumehaveparentrevisions,
	deltamode=deltamode,			deltamode=deltamode,
				sidedata_helpers=sidedata_helpers,
	)			)

	def addrevision(			def addrevision(
	self,			self,
	revisiondata,			revisiondata,
	transaction,			transaction,
	linkrev,			linkrev,
	p1,			p1,

mercurial/manifest.py


	def emitrevisions(			def emitrevisions(
	self,			self,
	nodes,			nodes,
	nodesorder=None,			nodesorder=None,
	revisiondata=False,			revisiondata=False,
	assumehaveparentrevisions=False,			assumehaveparentrevisions=False,
	deltamode=repository.CG_DELTAMODE_STD,			deltamode=repository.CG_DELTAMODE_STD,
				sidedata_helpers=None,
	):			):
	return self._revlog.emitrevisions(			return self._revlog.emitrevisions(
	nodes,			nodes,
	nodesorder=nodesorder,			nodesorder=nodesorder,
	revisiondata=revisiondata,			revisiondata=revisiondata,
	assumehaveparentrevisions=assumehaveparentrevisions,			assumehaveparentrevisions=assumehaveparentrevisions,
	deltamode=deltamode,			deltamode=deltamode,
				sidedata_helpers=sidedata_helpers,
	)			)

	def addgroup(			def addgroup(
	self,			self,
	deltas,			deltas,
	linkmapper,			linkmapper,
	transaction,			transaction,
	alwayscache=False,			alwayscache=False,

mercurial/revlog.py


	def emitrevisions(			def emitrevisions(
	self,			self,
	nodes,			nodes,
	nodesorder=None,			nodesorder=None,
	revisiondata=False,			revisiondata=False,
	assumehaveparentrevisions=False,			assumehaveparentrevisions=False,
	deltamode=repository.CG_DELTAMODE_STD,			deltamode=repository.CG_DELTAMODE_STD,
				sidedata_helpers=None,
	):			):
	if nodesorder not in (b'nodes', b'storage', b'linear', None):			if nodesorder not in (b'nodes', b'storage', b'linear', None):
	raise error.ProgrammingError(			raise error.ProgrammingError(
	b'unhandled value for nodesorder: %s' % nodesorder			b'unhandled value for nodesorder: %s' % nodesorder
	)			)

	if nodesorder is None and not self._generaldelta:			if nodesorder is None and not self._generaldelta:
	nodesorder = b'storage'			nodesorder = b'storage'
	deltaparentfn=self.deltaparent,			deltaparentfn=self.deltaparent,
	candeltafn=self.candelta,			candeltafn=self.candelta,
	rawsizefn=self.rawsize,			rawsizefn=self.rawsize,
	revdifffn=self.revdiff,			revdifffn=self.revdiff,
	flagsfn=self.flags,			flagsfn=self.flags,
	deltamode=deltamode,			deltamode=deltamode,
	revisiondata=revisiondata,			revisiondata=revisiondata,
	assumehaveparentrevisions=assumehaveparentrevisions,			assumehaveparentrevisions=assumehaveparentrevisions,
				sidedata_helpers=sidedata_helpers,
	)			)

	DELTAREUSEALWAYS = b'always'			DELTAREUSEALWAYS = b'always'
	DELTAREUSESAMEREVS = b'samerevs'			DELTAREUSESAMEREVS = b'samerevs'
	DELTAREUSENEVER = b'never'			DELTAREUSENEVER = b'never'

	DELTAREUSEFULLADD = b'fulladd'			DELTAREUSEFULLADD = b'fulladd'

mercurial/utils/storageutil.py

	)			)
	from .. import (			from .. import (
	dagop,			dagop,
	error,			error,
	mdiff,			mdiff,
	pycompat,			pycompat,
	)			)
	from ..interfaces import repository			from ..interfaces import repository
				from ..revlogutils import sidedata as sidedatamod
	from ..utils import hashutil			from ..utils import hashutil

	_nullhash = hashutil.sha1(nullid)			_nullhash = hashutil.sha1(nullid)


	def hashrevisionsha1(text, p1, p2):			def hashrevisionsha1(text, p1, p2):
	"""Compute the SHA-1 for revision data and its parents.			"""Compute the SHA-1 for revision data and its parents.

	deltaparentfn=None,			deltaparentfn=None,
	candeltafn=None,			candeltafn=None,
	rawsizefn=None,			rawsizefn=None,
	revdifffn=None,			revdifffn=None,
	flagsfn=None,			flagsfn=None,
	deltamode=repository.CG_DELTAMODE_STD,			deltamode=repository.CG_DELTAMODE_STD,
	revisiondata=False,			revisiondata=False,
	assumehaveparentrevisions=False,			assumehaveparentrevisions=False,
				sidedata_helpers=None,
	):			):
	"""Generic implementation of ifiledata.emitrevisions().			"""Generic implementation of ifiledata.emitrevisions().

	Emitting revision data is subtly complex. This function attempts to			Emitting revision data is subtly complex. This function attempts to
	encapsulate all the logic for doing so in a backend-agnostic way.			encapsulate all the logic for doing so in a backend-agnostic way.

	``store``			``store``
	Object conforming to ``ifilestorage`` interface.			Object conforming to ``ifilestorage`` interface.
	* CG_DELTAMODE_PREV - only delta against "prev",			* CG_DELTAMODE_PREV - only delta against "prev",
	* CG_DELTAMODE_FULL - only issue full snapshot.			* CG_DELTAMODE_FULL - only issue full snapshot.

	Whether to send fulltext revisions instead of deltas, if allowed.			Whether to send fulltext revisions instead of deltas, if allowed.

	``nodesorder``			``nodesorder``
	``revisiondata``			``revisiondata``
	``assumehaveparentrevisions``			``assumehaveparentrevisions``
				``sidedata_helpers`` (optional)
				If not None, means that sidedata should be included.
				A dictionary of revlog type to tuples of `(repo, computers, removers)`:
				* `repo` is used as an argument for computers
				* `computers` is a list of `(category, (keys, computer)` that
				compute the missing sidedata categories that were asked:
				* `category` is the sidedata category
				* `keys` are the sidedata keys to be affected
				* `computer` is the function `(repo, store, rev, sidedata)` that
				returns a new sidedata dict.
				* `removers` will remove the keys corresponding to the categories
				that are present, but not needed.
				If both `computers` and `removers` are empty, sidedata are simply not
				transformed.
				Revlog types are `changelog`, `manifest` or `filelog`.
	"""			"""

	fnode = store.node			fnode = store.node
	frev = store.rev			frev = store.rev

	if nodesorder == b'nodes':			if nodesorder == b'nodes':
	revs = [frev(n) for n in nodes]			revs = [frev(n) for n in nodes]
	elif nodesorder == b'linear':			elif nodesorder == b'linear':
	delta = revdifffn(baserev, rev)			delta = revdifffn(baserev, rev)
	else:			else:
	delta = mdiff.textdiff(			delta = mdiff.textdiff(
	store.rawdata(baserev), store.rawdata(rev)			store.rawdata(baserev), store.rawdata(rev)
	)			)

	available.add(rev)			available.add(rev)

				sidedata = None
				if sidedata_helpers:
				sidedata = store.sidedata(rev)
				sidedata = run_sidedata_helpers(
				store=store,
				sidedata_helpers=sidedata_helpers,
				sidedata=sidedata,
				rev=rev,
				)
				sidedata = sidedatamod.serialize_sidedata(sidedata)

	yield resultcls(			yield resultcls(
	node=node,			node=node,
	p1node=fnode(p1rev),			p1node=fnode(p1rev),
	p2node=fnode(p2rev),			p2node=fnode(p2rev),
	basenode=fnode(baserev),			basenode=fnode(baserev),
	flags=flagsfn(rev) if flagsfn else 0,			flags=flagsfn(rev) if flagsfn else 0,
	baserevisionsize=baserevisionsize,			baserevisionsize=baserevisionsize,
	revision=revision,			revision=revision,
	delta=delta,			delta=delta,
	sidedata=sidedata,			sidedata=sidedata,
	)			)

	prevrev = rev			prevrev = rev


				def run_sidedata_helpers(store, sidedata_helpers, sidedata, rev):
				marmouteUnsubmitted Done This will need some doc about what is the semantic of this function. marmoute: This will need some doc about what is the semantic of this function.
				"""Returns the sidedata for the given revision after running through
				the given helpers.
				- `store`: the revlog this applies to (changelog, manifest, or filelog
				instance)
				- `sidedata_helpers`: see `storageutil.emitrevisions`
				- `sidedata`: previous sidedata at the given rev, if any
				- `rev`: affected rev of `store`
				"""
				repo, sd_computers, sd_removers = sidedata_helpers
				kind = store.revlog_kind
				for _keys, sd_computer in sd_computers.get(kind, []):
				sidedata = sd_computer(repo, store, rev, sidedata)
				for keys, _computer in sd_removers.get(kind, []):
				for key in keys:
				sidedata.pop(key, None)
				return sidedata


	def deltaiscensored(delta, baserev, baselenfn):			def deltaiscensored(delta, baserev, baselenfn):
	"""Determine if a delta represents censored revision data.			"""Determine if a delta represents censored revision data.

	``baserev`` is the base revision this delta is encoded against.			``baserev`` is the base revision this delta is encoded against.
	``baselenfn`` is a callable receiving a revision number that resolves the			``baselenfn`` is a callable receiving a revision number that resolves the
	length of the revision fulltext.			length of the revision fulltext.

	Returns a bool indicating if the result of the delta represents a censored			Returns a bool indicating if the result of the delta represents a censored

tests/simplestorerepo.py


	def emitrevisions(			def emitrevisions(
	self,			self,
	nodes,			nodes,
	nodesorder=None,			nodesorder=None,
	revisiondata=False,			revisiondata=False,
	assumehaveparentrevisions=False,			assumehaveparentrevisions=False,
	deltamode=repository.CG_DELTAMODE_STD,			deltamode=repository.CG_DELTAMODE_STD,
				sidedata_helpers=None,
	):			):
	# TODO this will probably break on some ordering options.			# TODO this will probably break on some ordering options.
	nodes = [n for n in nodes if n != nullid]			nodes = [n for n in nodes if n != nullid]
	if not nodes:			if not nodes:
	return			return
	for delta in storageutil.emitrevisions(			for delta in storageutil.emitrevisions(
	self,			self,
	nodes,			nodes,
	nodesorder,			nodesorder,
	simplestorerevisiondelta,			simplestorerevisiondelta,
	revisiondata=revisiondata,			revisiondata=revisiondata,
	assumehaveparentrevisions=assumehaveparentrevisions,			assumehaveparentrevisions=assumehaveparentrevisions,
	deltamode=deltamode,			deltamode=deltamode,
				sidedata_helpers=sidedata_helpers,
	):			):
	yield delta			yield delta

	def add(self, text, meta, transaction, linkrev, p1, p2):			def add(self, text, meta, transaction, linkrev, p1, p2):
	if meta or text.startswith(b'\1\n'):			if meta or text.startswith(b'\1\n'):
	text = storageutil.packmeta(meta, text)			text = storageutil.packmeta(meta, text)

	return self.addrevision(text, transaction, linkrev, p1, p2)			return self.addrevision(text, transaction, linkrev, p1, p2)

			Path	Packages
M			hgext/sqlitestore.py (2 lines)
M			mercurial/changegroup.py (71 lines)
M			mercurial/filelog.py (2 lines)
M			mercurial/manifest.py (2 lines)
M			mercurial/revlog.py (2 lines)
M			mercurial/utils/storageutil.py (47 lines)
M			tests/simplestorerepo.py (2 lines)

This is an archive of the discontinued Mercurial Phabricator instance.

changegroupv4: add sidedata helpers
ClosedPublic

Details

Diff Detail

Event Timeline

Revision Contents
Changeset List

Diff 26257

hgext/sqlitestore.py

mercurial/changegroup.py

mercurial/filelog.py

mercurial/manifest.py

mercurial/revlog.py

mercurial/utils/storageutil.py

tests/simplestorerepo.py

This is an archive of the discontinued Mercurial Phabricator instance.

changegroupv4: add sidedata helpersClosedPublic

Details

Diff Detail

Event Timeline

Revision ContentsChangeset List

Diff 26257

hgext/sqlitestore.py

mercurial/changegroup.py

mercurial/filelog.py

mercurial/manifest.py

mercurial/revlog.py

mercurial/utils/storageutil.py

tests/simplestorerepo.py

changegroupv4: add sidedata helpers
ClosedPublic

Revision Contents
Changeset List