( )⚙ D4639 narrow: don't send the changelog information when widening without ellipses

This is an archive of the discontinued Mercurial Phabricator instance.

Differential D4639

narrow: don't send the changelog information when widening without ellipses
ClosedPublic

Authored by pulkit on Sep 18 2018, 6:43 AM.

Details

Reviewers
durin42
Group Reviewers
hg-reviewers
Commits
rHGf1844a10ee19: narrow: don't send the changelog information when widening without ellipses
Summary

When we widen anon-ellipses narrow copy, the server sends the changelog
information of all the changesets. The code was copied from ellipses case and in
ellipses cases, it's required to send the new changelog data.

But in non-ellipses cases, we don't need to send the changelog data as we will
have all the changesets locally.

Before this patch, there was a overhead of ~8-10 mins on each widening call
because of all the changelog information being pulled and being applied. After
this patch, we no more pull the changelog information. So this patch can save ~5
mins on Mozilla repo on each widening and more on repos which have more
changesets.

When we apply an empty changelog from changegroup, there is a devel-warn. This
patch kind of hacks to silence that devel-warn.

Diff Detail

Repository
rHG Mercurial
Lint
Lint Skipped
Unit
Unit Tests Skipped

Event Timeline

pulkit created this revision.Sep 18 2018, 6:43 AM
Closed by commit rHGf1844a10ee19: narrow: don't send the changelog information when widening without ellipses (authored by pulkit). · Explain WhySep 18 2018, 1:55 PM
This revision was automatically updated to reflect the committed changes.

Revision Contents
Changeset List

Diff 11138

hgext/narrow/narrowbundle2.py

commonnodes.add(cl.node(c)) commonnodes.add(cl.node(c))
if commonnodes: if commonnodes:
# XXX: we should only send the filelogs (and treemanifest). user # XXX: we should only send the filelogs (and treemanifest). user
# already has the changelog and manifest # already has the changelog and manifest
packer = changegroup.getbundler(version, repo, packer = changegroup.getbundler(version, repo,
filematcher=diffmatch, filematcher=diffmatch,
fullnodes=commonnodes) fullnodes=commonnodes)
cgdata = packer.generate(set([nullid]), list(commonnodes), False, cgdata = packer.generate(set([nullid]), list(commonnodes), False,
source) source, changelog=False)
part = bundler.newpart('changegroup', data=cgdata) part = bundler.newpart('changegroup', data=cgdata)
part.addparam('version', version) part.addparam('version', version)
if 'treemanifest' in repo.requirements: if 'treemanifest' in repo.requirements:
part.addparam('treemanifest', '1') part.addparam('treemanifest', '1')
# Serve a changegroup for a client with a narrow clone. # Serve a changegroup for a client with a narrow clone.
def getbundlechangegrouppart_narrow(bundler, repo, source, def getbundlechangegrouppart_narrow(bundler, repo, source,

hgext/narrow/narrowcommands.py

wrappedextraprepare = extensions.wrappedfunction(exchange, wrappedextraprepare = extensions.wrappedfunction(exchange,
'_pullbundle2extraprepare', pullbundle2extraprepare_widen) '_pullbundle2extraprepare', pullbundle2extraprepare_widen)
# define a function that narrowbundle2 can call after creating the # define a function that narrowbundle2 can call after creating the
# backup bundle, but before applying the bundle from the server # backup bundle, but before applying the bundle from the server
def setnewnarrowpats(): def setnewnarrowpats():
repo.setnarrowpats(newincludes, newexcludes) repo.setnarrowpats(newincludes, newexcludes)
repo.setnewnarrowpats = setnewnarrowpats repo.setnewnarrowpats = setnewnarrowpats
# silence the devel-warning of applying an empty changegroup
overrides = {('devel', 'all-warnings'): False}
with ui.uninterruptable(): with ui.uninterruptable():
ds = repo.dirstate ds = repo.dirstate
p1, p2 = ds.p1(), ds.p2() p1, p2 = ds.p1(), ds.p2()
with ds.parentchange(): with ds.parentchange():
ds.setparents(node.nullid, node.nullid) ds.setparents(node.nullid, node.nullid)
common = commoninc[0] common = commoninc[0]
with wrappedextraprepare: with wrappedextraprepare, repo.ui.configoverride(overrides, 'widen'):
exchange.pull(repo, remote, heads=common) exchange.pull(repo, remote, heads=common)
with ds.parentchange(): with ds.parentchange():
ds.setparents(p1, p2) ds.setparents(p1, p2)
repo.setnewnarrowpats() repo.setnewnarrowpats()
actions = {k: [] for k in 'a am f g cd dc r dm dg m e k p pr'.split()} actions = {k: [] for k in 'a am f g cd dc r dm dg m e k p pr'.split()}
addgaction = actions['g'].append addgaction = actions['g'].append

tests/test-narrow-widen-no-ellipsis.t

sending batch command sending batch command
searching for changes searching for changes
all local heads known remotely all local heads known remotely
no changes found no changes found
sending getbundle command sending getbundle command
bundle2-input-bundle: with-transaction bundle2-input-bundle: with-transaction
bundle2-input-part: "changegroup" (params: * mandatory) supported (glob) bundle2-input-part: "changegroup" (params: * mandatory) supported (glob)
adding changesets adding changesets
add changeset * (glob)
add changeset * (glob)
add changeset * (glob)
adding manifests adding manifests
adding widest/ revisions (tree !) adding widest/ revisions (tree !)
adding file changes adding file changes
adding widest/f revisions (tree !) adding widest/f revisions (tree !)
added 0 changesets with 1 changes to 1 files added 0 changesets with 1 changes to 1 files
bundle2-input-part: total payload size * (glob) bundle2-input-part: total payload size * (glob)
bundle2-input-part: "listkeys" (params: 1 mandatory) supported bundle2-input-part: "listkeys" (params: 1 mandatory) supported
bundle2-input-part: "phase-heads" supported bundle2-input-part: "phase-heads" supported