This is an archive of the discontinued Mercurial Phabricator instance.

Differential D4272

changegroup: change topics during generation
ClosedPublic

Authored by indygreg on Aug 13 2018, 12:55 PM.

Details

Reviewers
None
Group Reviewers
hg-reviewers
Commits
rHG0617a700ef7b: changegroup: change topics during generation
Summary

Changegroup generation and consumption currently uses different sets of
topics. Generation uses "bundling" and consumption uses the name of the
thing being consumed.

This commit makes the topic and unit names consistent on both ends.

The reason I chose to change the generation side is because when
"bundling" is used for 3 different stages, it is unclear which stage
of changegroup generation we're in. By making the topics "changesets,"
"manifests," and "files," one can get a better feel for how far along
in the bundling process we are.

Diff Detail

Repository
rHG Mercurial
Lint
Automatic diff as part of commit; lint not applicable.
Unit
Automatic diff as part of commit; unit tests not applicable.

Event Timeline

indygreg created this revision.Aug 13 2018, 12:55 PM
Closed by commit rHG0617a700ef7b: changegroup: change topics during generation (authored by indygreg). · Explain WhyAug 22 2018, 12:37 PM
This revision was automatically updated to reflect the committed changes.
martinvonz added a subscriber: martinvonz.Aug 28 2018, 2:01 PM

I'm not sure I buy this. It seemed pretty clear before too (it was just on the right side instead of the left side), but I guess that depends on whether your progress bar is configured to show units.

Revision Contents
Changeset List

PathPackages
Mmercurial/changegroup.py (16 lines)
Mtests/test-bundle.t (14 lines)
Mtests/test-bundle2-format.t (22 lines)

Diff 10519

mercurial/changegroup.py

p2node=p2node, p2node=p2node,
linknode=linknode, linknode=linknode,
basenode=nullid, basenode=nullid,
ellipsis=True, ellipsis=True,
) )
def deltagroup(repo, store, nodes, ischangelog, lookup, forcedeltaparentprev, def deltagroup(repo, store, nodes, ischangelog, lookup, forcedeltaparentprev,
allowreorder, allowreorder,
units=None, topic=None,
ellipses=False, clrevtolocalrev=None, fullclnodes=None, ellipses=False, clrevtolocalrev=None, fullclnodes=None,
precomputedellipsis=None): precomputedellipsis=None):
"""Calculate deltas for a set of revisions. """Calculate deltas for a set of revisions.
Is a generator of ``revisiondelta`` instances. Is a generator of ``revisiondelta`` instances.
If units is not None, progress detail will be generated, units specifies If topic is not None, progress detail will be generated using this
the type of revlog that is touched (changelog, manifest, etc.). topic name (e.g. changesets, manifests, etc).
""" """
if not nodes: if not nodes:
return return
# We perform two passes over the revisions whose data we will emit. # We perform two passes over the revisions whose data we will emit.
# #
# In the first pass, we obtain information about the deltas that will # In the first pass, we obtain information about the deltas that will
# be generated. This involves computing linknodes and adjusting the # be generated. This involves computing linknodes and adjusting the
p2node=p2node, p2node=p2node,
linknode=linknode, linknode=linknode,
basenode=store.node(prev) if forcedeltaparentprev else None, basenode=store.node(prev) if forcedeltaparentprev else None,
)) ))
# We expect the first pass to be fast, so we only engage the progress # We expect the first pass to be fast, so we only engage the progress
# meter for constructing the revision deltas. # meter for constructing the revision deltas.
progress = None progress = None
if units is not None: if topic is not None:
progress = repo.ui.makeprogress(_('bundling'), unit=units, progress = repo.ui.makeprogress(topic, unit=_('chunks'),
total=len(requests)) total=len(requests))
for i, delta in enumerate(store.emitrevisiondeltas(requests)): for i, delta in enumerate(store.emitrevisiondeltas(requests)):
if progress: if progress:
progress.update(i + 1) progress.update(i + 1)
yield delta yield delta
} }
gen = deltagroup( gen = deltagroup(
self._repo, cl, nodes, True, lookupcl, self._repo, cl, nodes, True, lookupcl,
self._forcedeltaparentprev, self._forcedeltaparentprev,
# Reorder settings are currently ignored for changelog. # Reorder settings are currently ignored for changelog.
True, True,
ellipses=self._ellipses, ellipses=self._ellipses,
units=_('changesets'), topic=_('changesets'),
clrevtolocalrev={}, clrevtolocalrev={},
fullclnodes=self._fullclnodes, fullclnodes=self._fullclnodes,
precomputedellipsis=self._precomputedellipsis) precomputedellipsis=self._precomputedellipsis)
return state, gen return state, gen
def generatemanifests(self, commonrevs, clrevorder, fastpathlinkrev, def generatemanifests(self, commonrevs, clrevorder, fastpathlinkrev,
manifests, fnodes, source, clrevtolocalrev): manifests, fnodes, source, clrevtolocalrev):
continue continue
lookupfn = makelookupmflinknode(tree, nodes) lookupfn = makelookupmflinknode(tree, nodes)
deltas = deltagroup( deltas = deltagroup(
self._repo, store, prunednodes, False, lookupfn, self._repo, store, prunednodes, False, lookupfn,
self._forcedeltaparentprev, self._reorder, self._forcedeltaparentprev, self._reorder,
ellipses=self._ellipses, ellipses=self._ellipses,
units=_('manifests'), topic=_('manifests'),
clrevtolocalrev=clrevtolocalrev, clrevtolocalrev=clrevtolocalrev,
fullclnodes=self._fullclnodes, fullclnodes=self._fullclnodes,
precomputedellipsis=self._precomputedellipsis) precomputedellipsis=self._precomputedellipsis)
yield tree, deltas yield tree, deltas
# The 'source' parameter is useful for extensions # The 'source' parameter is useful for extensions
def generatefiles(self, changedfiles, commonrevs, source, def generatefiles(self, changedfiles, commonrevs, source,
links[fnode] = min(links[fnode], lr, key=clrev) links[fnode] = min(links[fnode], lr, key=clrev)
elif fnode: elif fnode:
links[fnode] = lr links[fnode] = lr
return links return links
else: else:
linknodes = normallinknodes linknodes = normallinknodes
repo = self._repo repo = self._repo
progress = repo.ui.makeprogress(_('bundling'), unit=_('files'), progress = repo.ui.makeprogress(_('files'), unit=_('files'),
total=len(changedfiles)) total=len(changedfiles))
for i, fname in enumerate(sorted(changedfiles)): for i, fname in enumerate(sorted(changedfiles)):
filerevlog = repo.file(fname) filerevlog = repo.file(fname)
if not filerevlog: if not filerevlog:
raise error.Abort(_("empty or missing file data for %s") % raise error.Abort(_("empty or missing file data for %s") %
fname) fname)
clrevtolocalrev.clear() clrevtolocalrev.clear()

tests/test-bundle.t

searching for changes searching for changes
all remote heads known locally all remote heads known locally
2 changesets found 2 changesets found
list of changesets: list of changesets:
1a38c1b849e8b70c756d2d80b0b9a3ac0b7ea11a 1a38c1b849e8b70c756d2d80b0b9a3ac0b7ea11a
057f4db07f61970e1c11e83be79e9d08adc4dc31 057f4db07f61970e1c11e83be79e9d08adc4dc31
bundle2-output-bundle: "HG20", (1 params) 2 parts total bundle2-output-bundle: "HG20", (1 params) 2 parts total
bundle2-output-part: "changegroup" (params: 1 mandatory 1 advisory) streamed payload bundle2-output-part: "changegroup" (params: 1 mandatory 1 advisory) streamed payload
bundling: 1/2 changesets (50.00%) changesets: 1/2 chunks (50.00%)
bundling: 2/2 changesets (100.00%) changesets: 2/2 chunks (100.00%)
bundling: 1/2 manifests (50.00%) manifests: 1/2 chunks (50.00%)
bundling: 2/2 manifests (100.00%) manifests: 2/2 chunks (100.00%)
bundling: b 1/3 files (33.33%) files: b 1/3 files (33.33%)
bundling: b1 2/3 files (66.67%) files: b1 2/3 files (66.67%)
bundling: x 3/3 files (100.00%) files: x 3/3 files (100.00%)
bundle2-output-part: "cache:rev-branch-cache" (advisory) streamed payload bundle2-output-part: "cache:rev-branch-cache" (advisory) streamed payload
#if repobundlerepo #if repobundlerepo
== Test for issue3441 == Test for issue3441
$ hg clone -q -r0 . part2 $ hg clone -q -r0 . part2
$ hg -q -R part2 pull bundle.hg $ hg -q -R part2 pull bundle.hg
$ hg -R part2 verify $ hg -R part2 verify

tests/test-bundle2-format.t

bundle2-output-bundle: "HG20", 1 parts total bundle2-output-bundle: "HG20", 1 parts total
bundle2-output: start emission of HG20 stream bundle2-output: start emission of HG20 stream
bundle2-output: bundle parameter: bundle2-output: bundle parameter:
bundle2-output: start of parts bundle2-output: start of parts
bundle2-output: bundle part: "changegroup" bundle2-output: bundle part: "changegroup"
bundle2-output-part: "changegroup" (advisory) streamed payload bundle2-output-part: "changegroup" (advisory) streamed payload
bundle2-output: part 0: "changegroup" bundle2-output: part 0: "changegroup"
bundle2-output: header chunk size: 18 bundle2-output: header chunk size: 18
bundling: 1/4 changesets (25.00%) changesets: 1/4 chunks (25.00%)
bundling: 2/4 changesets (50.00%) changesets: 2/4 chunks (50.00%)
bundling: 3/4 changesets (75.00%) changesets: 3/4 chunks (75.00%)
bundling: 4/4 changesets (100.00%) changesets: 4/4 chunks (100.00%)
bundling: 1/4 manifests (25.00%) manifests: 1/4 chunks (25.00%)
bundling: 2/4 manifests (50.00%) manifests: 2/4 chunks (50.00%)
bundling: 3/4 manifests (75.00%) manifests: 3/4 chunks (75.00%)
bundling: 4/4 manifests (100.00%) manifests: 4/4 chunks (100.00%)
bundling: D 1/3 files (33.33%) files: D 1/3 files (33.33%)
bundling: E 2/3 files (66.67%) files: E 2/3 files (66.67%)
bundling: H 3/3 files (100.00%) files: H 3/3 files (100.00%)
bundle2-output: payload chunk size: 1555 bundle2-output: payload chunk size: 1555
bundle2-output: closing payload chunk bundle2-output: closing payload chunk
bundle2-output: end of bundle bundle2-output: end of bundle
$ f --hexdump ../rev.hg2 $ f --hexdump ../rev.hg2
../rev.hg2: ../rev.hg2:
0000: 48 47 32 30 00 00 00 00 00 00 00 12 0b 63 68 61 |HG20.........cha| 0000: 48 47 32 30 00 00 00 00 00 00 00 12 0b 63 68 61 |HG20.........cha|
0010: 6e 67 65 67 72 6f 75 70 00 00 00 00 00 00 00 00 |ngegroup........| 0010: 6e 67 65 67 72 6f 75 70 00 00 00 00 00 00 00 00 |ngegroup........|