This is an archive of the discontinued Mercurial Phabricator instance.

Differential D4583

shallowbundle: dump this revision prefetching logic
ClosedPublic

Authored by durin42 on Sep 13 2018, 6:09 PM.
Tags
None
Subscribers
quark

Details

Reviewers
quark
Group Reviewers
Restricted Project
Commits
rFBHGX12c6ed203897: shallowbundle: dump this revision prefetching logic
Summary

junw and I talked about this and we're both reasonably convinced this
is defending against a rare-in-practice case that could only result in
slowness, rather than incorrectness: missing cached blobs of
already-public changes.

Done as a separate follow-up change to give us something easy to back
out if it turns out we were wrong. :)

Diff Detail

Repository
rFBHGX Facebook Mercurial Extensions
Lint
Automatic diff as part of commit; lint not applicable.
Unit
Automatic diff as part of commit; unit tests not applicable.

Event Timeline

durin42 created this revision.Sep 13 2018, 6:09 PM
Herald added a reviewer: Restricted Project. · View Herald TranscriptSep 13 2018, 6:09 PM
quark accepted this revision.Sep 13 2018, 6:12 PM
quark added a subscriber: quark.

Pushed. Thanks!

This revision is now accepted and ready to land.Sep 13 2018, 6:12 PM
Closed by commit rFBHGX12c6ed203897: shallowbundle: dump this revision prefetching logic (authored by durin42). · Explain WhySep 13 2018, 6:12 PM
This revision was automatically updated to reflect the committed changes.

Revision Contents
Changeset List

PathPackages
Mremotefilelog/shallowbundle.py (44 lines)

Diff 11032

remotefilelog/shallowbundle.py

if repo._cgfilespos: if repo._cgfilespos:
raise error.Abort("cannot pull from full bundles", raise error.Abort("cannot pull from full bundles",
hint="use `hg unbundle` instead") hint="use `hg unbundle` instead")
return [] return []
filestosend = self.shouldaddfilegroups(source) filestosend = self.shouldaddfilegroups(source)
if filestosend == NoFiles: if filestosend == NoFiles:
changedfiles = list([f for f in changedfiles changedfiles = list([f for f in changedfiles
if not repo.shallowmatch(f)]) if not repo.shallowmatch(f)])
else:
def linknodes(store, fname):
# TODO(augie): this logic is almost certainly
# incorrect in some subtle way. See the
# similarly-named closure in
# mercurial.changegroup.cgpacker.generatefiles.
return {}
files = []
# Prefetch the revisions being bundled
for i, fname in enumerate(sorted(changedfiles)):
filerevlog = repo.file(fname)
linkrevnodes = linknodes(filerevlog, fname)
# Normally we'd prune the linkrevnodes first,
# but that would perform the server fetches one by one.
for fnode, cnode in list(linkrevnodes.iteritems()):
# Adjust linknodes so remote file revisions aren't sent
if filestosend == LocalFiles:
localkey = fileserverclient.getlocalkey(fname,
hex(fnode))
localpath = repo.sjoin(os.path.join("data",
localkey))
if (not os.path.exists(localpath)
and repo.shallowmatch(fname)):
del linkrevnodes[fnode]
else:
files.append((fname, hex(fnode)))
else:
files.append((fname, hex(fnode)))
repo.fileservice.prefetch(files)
# Prefetch the revisions that are going to be diffed against
prevfiles = []
for fname, fnode in files:
if repo.shallowmatch(fname):
fnode = bin(fnode)
filerevlog = repo.file(fname)
ancestormap = filerevlog.ancestormap(fnode)
p1, p2, linknode, copyfrom = ancestormap[fnode]
if p1 != nullid:
prevfiles.append((copyfrom or fname, hex(p1)))
repo.fileservice.prefetch(prevfiles)
return super(shallowcg1packer, self).generatefiles( return super(shallowcg1packer, self).generatefiles(
changedfiles, *args) changedfiles, *args)
def shouldaddfilegroups(self, source): def shouldaddfilegroups(self, source):
repo = self._repo repo = self._repo
if not requirement in repo.requirements: if not requirement in repo.requirements:
return AllFiles return AllFiles