This is an archive of the discontinued Mercurial Phabricator instance.

Differential D4787

narrow: pass 'narrow_widen' as source while generating changegroup
ClosedPublic

Authored by pulkit on Sep 28 2018, 4:58 PM.

Details

Reviewers
durin42
martinvonz
Group Reviewers
hg-reviewers
Commits
rHGfa2395659828: narrow: pass 'narrow_widen' as source while generating changegroup
Summary

Extensions inspect the source parameter and add some custom logic on the basis
of value of that parameter. The parameter is also passed to hooks. So let's pass
the value as 'narrow_widen' to differentiate widening from pull.

Diff Detail

Repository
rHG Mercurial
Lint
Automatic diff as part of commit; lint not applicable.
Unit
Automatic diff as part of commit; unit tests not applicable.

Event Timeline

pulkit created this revision.Sep 28 2018, 4:58 PM
martinvonz added a subscriber: martinvonz.Oct 1 2018, 2:42 AM

Although this is called as a part of exchange.pull(), we don't want other
extensions to wrap this pull call because it's not a normal pull and it's
widening.

I don't follow this. How does this prevent extensions from wrapping the pull? Do you mean that extensions that inspect the source parameter should not treat it like a normal pull? The source parameter also seems to be passed to hooks, btw (it probably makes sense to differentiate widening from pulling there too).

pulkit added a comment.Oct 1 2018, 10:59 AM
In D4787#72712, @martinvonz wrote:

Although this is called as a part of exchange.pull(), we don't want other
extensions to wrap this pull call because it's not a normal pull and it's
widening.

I don't follow this. How does this prevent extensions from wrapping the pull? Do you mean that extensions that inspect the source parameter should not treat it like a normal pull? The source parameter also seems to be passed to hooks, btw (it probably makes sense to differentiate widening from pulling there too).

Yes, I looked into the code where source parameter is used and it says the extension requires it. I think wrapping pull sounds to generic, I will improve the commit description. Thanks!

pulkit edited the summary of this revision. (Show Details)Oct 1 2018, 12:01 PM
pulkit updated this revision to Diff 11560.Oct 2 2018, 10:19 AM
martinvonz accepted this revision.Oct 2 2018, 12:12 PM
This revision is now accepted and ready to land.Oct 2 2018, 12:12 PM
Closed by commit rHGfa2395659828: narrow: pass 'narrow_widen' as source while generating changegroup (authored by pulkit). · Explain WhyOct 2 2018, 12:55 PM
This revision was automatically updated to reflect the committed changes.

Revision Contents
Changeset List

Diff 11567

hgext/narrow/narrowbundle2.py

_MFHEADERSIZE = struct.calcsize(_ELIDEDMFHEADER) _MFHEADERSIZE = struct.calcsize(_ELIDEDMFHEADER)
# When advertising capabilities, always include narrow clone support. # When advertising capabilities, always include narrow clone support.
def getrepocaps_narrow(orig, repo, **kwargs): def getrepocaps_narrow(orig, repo, **kwargs):
caps = orig(repo, **kwargs) caps = orig(repo, **kwargs)
caps[NARROWCAP] = ['v0'] caps[NARROWCAP] = ['v0']
return caps return caps
def widen_bundle(repo, diffmatcher, common, known, cgversion, source, ellipses): def widen_bundle(repo, diffmatcher, common, known, cgversion, ellipses):
"""generates changegroup for widening a narrow clone """generates changegroup for widening a narrow clone
repo is the localrepository instance repo is the localrepository instance
diffmatcher is a differencemacther of '(newincludes, newexcludes) - diffmatcher is a differencemacther of '(newincludes, newexcludes) -
(oldincludes, oldexcludes)' (oldincludes, oldexcludes)'
common is set of common revs between server and client common is set of common revs between server and client
known is a set of revs known on the client side (used in ellipses) known is a set of revs known on the client side (used in ellipses)
cgversion is the changegroup version to send cgversion is the changegroup version to send
source is the command which called this codepath
ellipses is boolean value telling whether to send ellipses data or not ellipses is boolean value telling whether to send ellipses data or not
returns changegroup data of the changegroup built or return None if there returns changegroup data of the changegroup built or return None if there
are no common revs are no common revs
""" """
# XXX: This patch will start sending bundle2 after couple of patches when # XXX: This patch will start sending bundle2 after couple of patches when
# called from the wireprotocol command # called from the wireprotocol command
common = repo.revs("::%ln", common) common = repo.revs("::%ln", common)
commonnodes = set() commonnodes = set()
cl = repo.changelog cl = repo.changelog
for c in common: for c in common:
commonnodes.add(cl.node(c)) commonnodes.add(cl.node(c))
if commonnodes: if commonnodes:
# XXX: we should only send the filelogs (and treemanifest). user # XXX: we should only send the filelogs (and treemanifest). user
# already has the changelog and manifest # already has the changelog and manifest
packer = changegroup.getbundler(cgversion, repo, packer = changegroup.getbundler(cgversion, repo,
filematcher=diffmatcher, filematcher=diffmatcher,
fullnodes=commonnodes) fullnodes=commonnodes)
cgdata = packer.generate(set([nullid]), list(commonnodes), False, cgdata = packer.generate(set([nullid]), list(commonnodes), False,
source, changelog=False) 'narrow_widen', changelog=False)
return cgdata return cgdata
return None return None
def getbundlechangegrouppart_widen(bundler, repo, source, bundlecaps=None, def getbundlechangegrouppart_widen(bundler, repo, source, bundlecaps=None,
b2caps=None, heads=None, common=None, b2caps=None, heads=None, common=None,
**kwargs): **kwargs):
oldinclude = sorted(filter(bool, kwargs.get(r'oldincludepats', []))) oldinclude = sorted(filter(bool, kwargs.get(r'oldincludepats', [])))
oldexclude = sorted(filter(bool, kwargs.get(r'oldexcludepats', []))) oldexclude = sorted(filter(bool, kwargs.get(r'oldexcludepats', [])))
oldmatch = narrowspec.match(repo.root, include=oldinclude, oldmatch = narrowspec.match(repo.root, include=oldinclude,
exclude=oldexclude) exclude=oldexclude)
diffmatch = matchmod.differencematcher(newmatch, oldmatch) diffmatch = matchmod.differencematcher(newmatch, oldmatch)
common = set(common or [nullid]) common = set(common or [nullid])
if (oldinclude != include or oldexclude != exclude): if (oldinclude != include or oldexclude != exclude):
cgdata = widen_bundle(repo, diffmatch, common, [], version, cgdata = widen_bundle(repo, diffmatch, common, [], version, False)
source, False)
if cgdata is not None: if cgdata is not None:
part = bundler.newpart('changegroup', data=cgdata) part = bundler.newpart('changegroup', data=cgdata)
part.addparam('version', version) part.addparam('version', version)
if 'treemanifest' in repo.requirements: if 'treemanifest' in repo.requirements:
part.addparam('treemanifest', '1') part.addparam('treemanifest', '1')
# Serve a changegroup for a client with a narrow clone. # Serve a changegroup for a client with a narrow clone.
def getbundlechangegrouppart_narrow(bundler, repo, source, def getbundlechangegrouppart_narrow(bundler, repo, source,
repo, set(), common, known, newmatch) repo, set(), common, known, newmatch)
if newvisit: if newvisit:
packer = changegroup.getbundler(version, repo, packer = changegroup.getbundler(version, repo,
filematcher=newmatch, filematcher=newmatch,
ellipses=True, ellipses=True,
shallow=depth is not None, shallow=depth is not None,
ellipsisroots=newellipsis, ellipsisroots=newellipsis,
fullnodes=newfull) fullnodes=newfull)
cgdata = packer.generate(common, newvisit, False, source) cgdata = packer.generate(common, newvisit, False, 'narrow_widen')
part = bundler.newpart('changegroup', data=cgdata) part = bundler.newpart('changegroup', data=cgdata)
part.addparam('version', version) part.addparam('version', version)
if 'treemanifest' in repo.requirements: if 'treemanifest' in repo.requirements:
part.addparam('treemanifest', '1') part.addparam('treemanifest', '1')
visitnodes, relevant_nodes, ellipsisroots = exchange._computeellipsis( visitnodes, relevant_nodes, ellipsisroots = exchange._computeellipsis(
repo, common, heads, set(), newmatch, depth=depth) repo, common, heads, set(), newmatch, depth=depth)
repo.ui.debug('Found %d relevant revs\n' % len(relevant_nodes)) repo.ui.debug('Found %d relevant revs\n' % len(relevant_nodes))
if visitnodes: if visitnodes:
packer = changegroup.getbundler(version, repo, packer = changegroup.getbundler(version, repo,
filematcher=newmatch, filematcher=newmatch,
ellipses=True, ellipses=True,
shallow=depth is not None, shallow=depth is not None,
ellipsisroots=ellipsisroots, ellipsisroots=ellipsisroots,
fullnodes=relevant_nodes) fullnodes=relevant_nodes)
cgdata = packer.generate(common, visitnodes, False, source) cgdata = packer.generate(common, visitnodes, False, 'narrow_widen')
part = bundler.newpart('changegroup', data=cgdata) part = bundler.newpart('changegroup', data=cgdata)
part.addparam('version', version) part.addparam('version', version)
if 'treemanifest' in repo.requirements: if 'treemanifest' in repo.requirements:
part.addparam('treemanifest', '1') part.addparam('treemanifest', '1')
@bundle2.parthandler(_SPECPART, (_SPECPART_INCLUDE, _SPECPART_EXCLUDE)) @bundle2.parthandler(_SPECPART, (_SPECPART_INCLUDE, _SPECPART_EXCLUDE))
def _handlechangespec_2(op, inpart): def _handlechangespec_2(op, inpart):