This is an archive of the discontinued Mercurial Phabricator instance.

Differential D4721

changegroup: remove reordering control (BC)
ClosedPublic

Authored by indygreg on Sep 24 2018, 1:15 PM.

Details

Reviewers
None
Group Reviewers
hg-reviewers
Commits
rHGdb5501d93bcf: changegroup: remove reordering control (BC)
Summary

This logic - including the experimental bundle.reorder option -
was originally added in a8e3931e3fb5 in 2011 and then later ported
to changegroup.py.

The intent of this option and associated logic is to control
the ordering of revisions in deltagroups in changegroups. At the
time it was implemented, only changegroup version 1 existed
and generaldelta revlogs were just coming into the world. Changegroup
version 1 requires that deltas be made against the last revision
sent over the wire. Used with generaldelta, this created an
impedance mismatch of sorts and resulted in changegroup producers
spending a lot of time recomputing deltas.

Revision reordering was introduced so outgoing revisions would be
sent in "generaldelta order" and producers would be able to
reuse internal deltas from storage.

Later on, we introduced changegroup version 2. It supported denoting
which revision a delta was against. So we no longer needed to
sort outgoing revisions to ensure optimal delta generation from the
producer. So, subsequent changegroup versions disabled reordering.

We also later made the changelog not store deltas by default. And
we also made the changelog send out deltas in storage order. Why we
do this for changelog, I'm not sure. Maybe we want to preserve revision
order across clones? It doesn't really matter for this commit.

Fast forward to 2018. We want to abstract storage backends. And having
changegroup code require knowledge about how deltas are stored
internally interferes with that goal.

This commit removes reordering control from changegroup generation.
After this commit, the reordering behavior is:

  • The changelog is always sent out in storage order (no behavior change).
  • Non-changelog generaldelta revlogs are reordered to always be in DAG topological order (previously, generaldelta revlogs would be emitted in storage order for version 2 and 3 changegroups).
  • Non-changelog non-generaldelta revlogs are sent in storage order (no behavior change).
  • There exists no config option to override behavior.

The big difference here is that generaldelta revlogs now *always* have
their revisions sorted in DAG order before going out over the wire. This
behavior was previously only done for changegroup version 1. Version 2
and version 3 changegroups disabled reordering because the interchange
format supported encoding arbitrary delta parents, so reordering wasn't
strictly necessary.

I can think of a few significant implications for this change.

Because changegroup receivers will now see non-changelog revisions
in DAG order instead of storage order, the internal storage order of
manifests and files may differ substantially between producer and
consumer. I don't think this matters that much, since the storage
order of manifests and files is largely hidden from users. Only
the storage order of changelog matters (because hg log shows the
changelog in storage order). I don't think there should be any
controversy here.

The reordering of revisions has implications for changegroup producers.
Previously, generaldelta revlogs would be emitted in storage order.
And in the common case, the internally-stored delta could effectively
be copied from disk into the deltagroup delta. This meant that emitting
delta groups for generaldelta revlogs would be mostly linear read I/O.
This is desirable for performance. With us now reordering generaldelta
revlog revisions in DAG order, the read operations may use more random
I/O instead of sequential I/O. This could result in performance
loss. But with the prevalence of SSDs and fast random I/O, I'm not
too worried. (Note: the optimal emission order for revlogs is actually
delta encoding order. But the changegroup code wasn't doing that before
or after this change. We could potentially implement that in a later
commit.)

Changegroups in DAG order will have implications for receivers.
Previously, receiving storage order might mean seeing a number of
interleaved branches. This would mean long delta chains, sparse
I/O, and possibly more fulltext revisions instead of deltas, blowing
up storage storage. (This is the same set of problems that sparse
revlogs aims to address.) With the producer now sending revisions in DAG
order, the receiver also stores revisions in DAG order. That means
revisions for the same DAG branch are all grouped together. And this
should yield better storage outcomes. In other words, sending the
reordered changegroup allows the receiver to have better storage
order and for the producer to not propagate its (possibly sub-optimal)
internal storage order.

On the mozilla-unified repository, this change influences bundle
generation:

$ hg bundle -t none-v2 -a
before: time: real 355.680 secs (user 256.790+0.000 sys 16.820+0.000)
after: time: real 382.950 secs (user 281.700+0.000 sys 17.690+0.000)

before: 7,150,228,967 bytes (uncompressed)
after: 7,041,556,273 bytes (uncompressed)

before: 1,669,063,234 bytes (zstd l=3)
after: 1,628,598,830 bytes (zstd l=3)

$ hg unbundle
before: time: real 511.910 secs (user 466.750+0.000 sys 32.680+0.000)
after: time: real 487.790 secs (user 443.940+0.000 sys 30.840+0.000)

00manifest.d size:
source: 274,924,292 bytes
before: 304,741,626 bytes
after: 245,252,087 bytes

.hg/store total file size:
source: 2,649,133,490
before: 2,680,888,130
after: 2,627,875,673

We see the bundle size drop. That's probably because if a revlog
internally isn't storing a delta, it will choose to delta against
the last emitted revision. And on repos with interleaved branches
(like mozilla-unified), the previous revision could be an unrelated
branch and therefore be a large delta. But with this patch, the
previous revision is likely p1 or p2 and a delta should be small.

We also see the manifest size drop by ~50 MB. It's worth noting that
the manifest actually *increased* in size by ~25 MB in the old
strategy and decreased ~25 MB from its source in the new strategy.
Again, my explanation for this is that the DAG ordering in the
changegroup is resulting in better grouping of revisions in the
receiver, which results in more compact delta chains and higher
storage efficiency.

Unbundle time also dropped. I suspect this is due to the revlog having
to work less to compute deltas since the incoming deltas are more
optimal. i.e. the receiver spends less time resolving fulltext
revisions as incoming deltas bounce around between DAG branches and
delta chains.

We also see bundle generation time increase. This is not desirable.
However, the regression is only significant on the original repository:
if we generate a bundle from the repository created from the new,
always reordered bundles, we're close to baseline (if not at it with
expected noise):

$ hg bundle -t none-v2 -a
before (original): time: real 355.680 secs (user 256.790+0.000 sys 16.820+0.000)
after (original): time: real 382.950 secs (user 281.700+0.000 sys 17.690+0.000)
after (new repo): time: real 362.280 secs (user 260.300+0.000 sys 17.700+0.000)

This regression is a bit worrying because it will impact serving
canonical repositories (that don't have optimal internal storage
unless they are reordered - possibly as part of running
hg debugupgraderepo). However, this regression will only be
noticed by very large changegroups. And I'm guessing/hoping that
any repository that large is using clonebundles to mitigate server
load.

Again, sending DAG order isn't the optimal send order for servers:
sending in storage-delta order is. But in order to enable
storage-optimal send order, we'll need a storage API that handles
sorting. Future commits will introduce such an API.

Diff Detail

Repository
rHG Mercurial
Lint
Lint Skipped
Unit
Unit Tests Skipped

Event Timeline

indygreg created this revision.Sep 24 2018, 1:15 PM
indygreg added a subscriber: lothiraldan.Sep 24 2018, 2:10 PM

@lothiraldan: you are likely interested in this change because it is in the same space as sparse revlogs.

Closed by commit rHGdb5501d93bcf: changegroup: remove reordering control (BC) (authored by indygreg). · Explain WhySep 26 2018, 1:25 PM
This revision was automatically updated to reflect the committed changes.

Revision Contents
Changeset List

Diff 11329

mercurial/changegroup.py

phases, phases,
pycompat, pycompat,
repository, repository,
util, util,
) )
from .utils import ( from .utils import (
interfaceutil, interfaceutil,
stringutil,
) )
_CHANGEGROUPV1_DELTA_HEADER = struct.Struct("20s20s20s20s") _CHANGEGROUPV1_DELTA_HEADER = struct.Struct("20s20s20s20s")
_CHANGEGROUPV2_DELTA_HEADER = struct.Struct("20s20s20s20s20s") _CHANGEGROUPV2_DELTA_HEADER = struct.Struct("20s20s20s20s20s")
_CHANGEGROUPV3_DELTA_HEADER = struct.Struct(">20s20s20s20s20sH") _CHANGEGROUPV3_DELTA_HEADER = struct.Struct(">20s20s20s20s20sH")
LFS_REQUIREMENT = 'lfs' LFS_REQUIREMENT = 'lfs'
meta = headerfn(delta) meta = headerfn(delta)
yield chunkheader(len(meta) + len(prefix) + len(data)) yield chunkheader(len(meta) + len(prefix) + len(data))
yield meta yield meta
if prefix: if prefix:
yield prefix yield prefix
yield data yield data
def _sortnodesnormal(store, nodes, reorder): def _sortnodesnormal(store, nodes):
"""Sort nodes for changegroup generation and turn into revnums.""" """Sort nodes for changegroup generation and turn into revnums."""
# for generaldelta revlogs, we linearize the revs; this will both be # for generaldelta revlogs, we linearize the revs; this will both be
# much quicker and generate a much smaller bundle # much quicker and generate a much smaller bundle
if (store._generaldelta and reorder is None) or reorder: if store._generaldelta:
revs = set(store.rev(n) for n in nodes) revs = set(store.rev(n) for n in nodes)
return dagop.linearize(revs, store.parentrevs) return dagop.linearize(revs, store.parentrevs)
else: else:
return sorted([store.rev(n) for n in nodes]) return sorted([store.rev(n) for n in nodes])
def _sortnodesellipsis(store, nodes, cl, lookup): def _sortnodesellipsis(store, nodes, cl, lookup):
"""Sort nodes for changegroup generation and turn into revnums.""" """Sort nodes for changegroup generation and turn into revnums."""
# Ellipses serving mode. # Ellipses serving mode.
p1node=p1node, p1node=p1node,
p2node=p2node, p2node=p2node,
linknode=linknode, linknode=linknode,
basenode=nullid, basenode=nullid,
ellipsis=True, ellipsis=True,
) )
def deltagroup(repo, store, nodes, ischangelog, lookup, forcedeltaparentprev, def deltagroup(repo, store, nodes, ischangelog, lookup, forcedeltaparentprev,
allowreorder,
topic=None, topic=None,
ellipses=False, clrevtolocalrev=None, fullclnodes=None, ellipses=False, clrevtolocalrev=None, fullclnodes=None,
precomputedellipsis=None): precomputedellipsis=None):
"""Calculate deltas for a set of revisions. """Calculate deltas for a set of revisions.
Is a generator of ``revisiondelta`` instances. Is a generator of ``revisiondelta`` instances.
If topic is not None, progress detail will be generated using this If topic is not None, progress detail will be generated using this
# Changelog doesn't benefit from reordering revisions. So send # Changelog doesn't benefit from reordering revisions. So send
# out revisions in store order. # out revisions in store order.
# TODO the API would be cleaner if this were controlled by the # TODO the API would be cleaner if this were controlled by the
# store producing the deltas. # store producing the deltas.
revs = sorted(cl.rev(n) for n in nodes) revs = sorted(cl.rev(n) for n in nodes)
elif ellipses: elif ellipses:
revs = _sortnodesellipsis(store, nodes, cl, lookup) revs = _sortnodesellipsis(store, nodes, cl, lookup)
else: else:
revs = _sortnodesnormal(store, nodes, allowreorder) revs = _sortnodesnormal(store, nodes)
# In the first pass, collect info about the deltas we'll be # In the first pass, collect info about the deltas we'll be
# generating. # generating.
requests = [] requests = []
# Add the parent of the first rev. # Add the parent of the first rev.
revs.insert(0, store.parentrevs(revs[0])[0]) revs.insert(0, store.parentrevs(revs[0])[0])
progress.update(i + 1) progress.update(i + 1)
yield delta yield delta
if progress: if progress:
progress.complete() progress.complete()
class cgpacker(object): class cgpacker(object):
def __init__(self, repo, filematcher, version, allowreorder, def __init__(self, repo, filematcher, version,
builddeltaheader, manifestsend, builddeltaheader, manifestsend,
forcedeltaparentprev=False, forcedeltaparentprev=False,
bundlecaps=None, ellipses=False, bundlecaps=None, ellipses=False,
shallow=False, ellipsisroots=None, fullnodes=None): shallow=False, ellipsisroots=None, fullnodes=None):
"""Given a source repo, construct a bundler. """Given a source repo, construct a bundler.
filematcher is a matcher that matches on files to include in the filematcher is a matcher that matches on files to include in the
changegroup. Used to facilitate sparse changegroups. changegroup. Used to facilitate sparse changegroups.
allowreorder controls whether reordering of revisions is allowed.
This value is used when ``bundle.reorder`` is ``auto`` or isn't
set.
forcedeltaparentprev indicates whether delta parents must be against forcedeltaparentprev indicates whether delta parents must be against
the previous revision in a delta group. This should only be used for the previous revision in a delta group. This should only be used for
compatibility with changegroup version 1. compatibility with changegroup version 1.
builddeltaheader is a callable that constructs the header for a group builddeltaheader is a callable that constructs the header for a group
delta. delta.
manifestsend is a chunk to send after manifests have been fully emitted. manifestsend is a chunk to send after manifests have been fully emitted.
bundlecaps = set() bundlecaps = set()
self._bundlecaps = bundlecaps self._bundlecaps = bundlecaps
self._isshallow = shallow self._isshallow = shallow
self._fullclnodes = fullnodes self._fullclnodes = fullnodes
# Maps ellipsis revs to their roots at the changelog level. # Maps ellipsis revs to their roots at the changelog level.
self._precomputedellipsis = ellipsisroots self._precomputedellipsis = ellipsisroots
# experimental config: bundle.reorder
reorder = repo.ui.config('bundle', 'reorder')
if reorder == 'auto':
self._reorder = allowreorder
else:
self._reorder = stringutil.parsebool(reorder)
self._repo = repo self._repo = repo
if self._repo.ui.verbose and not self._repo.ui.debugflag: if self._repo.ui.verbose and not self._repo.ui.debugflag:
self._verbosenote = self._repo.ui.note self._verbosenote = self._repo.ui.note
else: else:
self._verbosenote = lambda s: None self._verbosenote = lambda s: None
def generate(self, commonrevs, clnodes, fastpathlinkrev, source, def generate(self, commonrevs, clnodes, fastpathlinkrev, source,
manifests = clstate['manifests'] manifests = clstate['manifests']
changedfiles = clstate['changedfiles'] changedfiles = clstate['changedfiles']
# We need to make sure that the linkrev in the changegroup refers to # We need to make sure that the linkrev in the changegroup refers to
# the first changeset that introduced the manifest or file revision. # the first changeset that introduced the manifest or file revision.
# The fastpath is usually safer than the slowpath, because the filelogs # The fastpath is usually safer than the slowpath, because the filelogs
# are walked in revlog order. # are walked in revlog order.
# #
# When taking the slowpath with reorder=None and the manifest revlog # When taking the slowpath when the manifest revlog uses generaldelta,
# uses generaldelta, the manifest may be walked in the "wrong" order. # the manifest may be walked in the "wrong" order. Without 'clrevorder',
# Without 'clrevorder', we would get an incorrect linkrev (see fix in # we would get an incorrect linkrev (see fix in cc0ff93d0c0c).
# cc0ff93d0c0c).
# #
# When taking the fastpath, we are only vulnerable to reordering # When taking the fastpath, we are only vulnerable to reordering
# of the changelog itself. The changelog never uses generaldelta, so # of the changelog itself. The changelog never uses generaldelta and is
# it is only reordered when reorder=True. To handle this case, we # never reordered. To handle this case, we simply take the slowpath,
# simply take the slowpath, which already has the 'clrevorder' logic. # which already has the 'clrevorder' logic. This was also fixed in
# This was also fixed in cc0ff93d0c0c. # cc0ff93d0c0c.
fastpathlinkrev = fastpathlinkrev and not self._reorder
# Treemanifests don't work correctly with fastpathlinkrev # Treemanifests don't work correctly with fastpathlinkrev
# either, because we don't discover which directory nodes to # either, because we don't discover which directory nodes to
# send along with files. This could probably be fixed. # send along with files. This could probably be fixed.
fastpathlinkrev = fastpathlinkrev and ( fastpathlinkrev = fastpathlinkrev and (
'treemanifest' not in repo.requirements) 'treemanifest' not in repo.requirements)
fnodes = {} # needed file nodes fnodes = {} # needed file nodes
'manifests': manifests, 'manifests': manifests,
'changedfiles': changedfiles, 'changedfiles': changedfiles,
'clrevtomanifestrev': clrevtomanifestrev, 'clrevtomanifestrev': clrevtomanifestrev,
} }
gen = deltagroup( gen = deltagroup(
self._repo, cl, nodes, True, lookupcl, self._repo, cl, nodes, True, lookupcl,
self._forcedeltaparentprev, self._forcedeltaparentprev,
# Reorder settings are currently ignored for changelog.
True,
ellipses=self._ellipses, ellipses=self._ellipses,
topic=_('changesets'), topic=_('changesets'),
clrevtolocalrev={}, clrevtolocalrev={},
fullclnodes=self._fullclnodes, fullclnodes=self._fullclnodes,
precomputedellipsis=self._precomputedellipsis) precomputedellipsis=self._precomputedellipsis)
return state, gen return state, gen
prunednodes = self._prunemanifests(store, nodes, commonrevs) prunednodes = self._prunemanifests(store, nodes, commonrevs)
if tree and not prunednodes: if tree and not prunednodes:
continue continue
lookupfn = makelookupmflinknode(tree, nodes) lookupfn = makelookupmflinknode(tree, nodes)
deltas = deltagroup( deltas = deltagroup(
self._repo, store, prunednodes, False, lookupfn, self._repo, store, prunednodes, False, lookupfn,
self._forcedeltaparentprev, self._reorder, self._forcedeltaparentprev,
ellipses=self._ellipses, ellipses=self._ellipses,
topic=_('manifests'), topic=_('manifests'),
clrevtolocalrev=clrevtolocalrev, clrevtolocalrev=clrevtolocalrev,
fullclnodes=self._fullclnodes, fullclnodes=self._fullclnodes,
precomputedellipsis=self._precomputedellipsis) precomputedellipsis=self._precomputedellipsis)
yield tree, deltas yield tree, deltas
if not filenodes: if not filenodes:
continue continue
progress.update(i + 1, item=fname) progress.update(i + 1, item=fname)
deltas = deltagroup( deltas = deltagroup(
self._repo, filerevlog, filenodes, False, lookupfilelog, self._repo, filerevlog, filenodes, False, lookupfilelog,
self._forcedeltaparentprev, self._reorder, self._forcedeltaparentprev,
ellipses=self._ellipses, ellipses=self._ellipses,
clrevtolocalrev=clrevtolocalrev, clrevtolocalrev=clrevtolocalrev,
fullclnodes=self._fullclnodes, fullclnodes=self._fullclnodes,
precomputedellipsis=self._precomputedellipsis) precomputedellipsis=self._precomputedellipsis)
yield fname, deltas yield fname, deltas
progress.complete() progress.complete()
def _makecg1packer(repo, filematcher, bundlecaps, ellipses=False, def _makecg1packer(repo, filematcher, bundlecaps, ellipses=False,
shallow=False, ellipsisroots=None, fullnodes=None): shallow=False, ellipsisroots=None, fullnodes=None):
builddeltaheader = lambda d: _CHANGEGROUPV1_DELTA_HEADER.pack( builddeltaheader = lambda d: _CHANGEGROUPV1_DELTA_HEADER.pack(
d.node, d.p1node, d.p2node, d.linknode) d.node, d.p1node, d.p2node, d.linknode)
return cgpacker(repo, filematcher, b'01', return cgpacker(repo, filematcher, b'01',
allowreorder=None,
builddeltaheader=builddeltaheader, builddeltaheader=builddeltaheader,
manifestsend=b'', manifestsend=b'',
forcedeltaparentprev=True, forcedeltaparentprev=True,
bundlecaps=bundlecaps, bundlecaps=bundlecaps,
ellipses=ellipses, ellipses=ellipses,
shallow=shallow, shallow=shallow,
ellipsisroots=ellipsisroots, ellipsisroots=ellipsisroots,
fullnodes=fullnodes) fullnodes=fullnodes)
def _makecg2packer(repo, filematcher, bundlecaps, ellipses=False, def _makecg2packer(repo, filematcher, bundlecaps, ellipses=False,
shallow=False, ellipsisroots=None, fullnodes=None): shallow=False, ellipsisroots=None, fullnodes=None):
builddeltaheader = lambda d: _CHANGEGROUPV2_DELTA_HEADER.pack( builddeltaheader = lambda d: _CHANGEGROUPV2_DELTA_HEADER.pack(
d.node, d.p1node, d.p2node, d.basenode, d.linknode) d.node, d.p1node, d.p2node, d.basenode, d.linknode)
# Since generaldelta is directly supported by cg2, reordering
# generally doesn't help, so we disable it by default (treating
# bundle.reorder=auto just like bundle.reorder=False).
return cgpacker(repo, filematcher, b'02', return cgpacker(repo, filematcher, b'02',
allowreorder=False,
builddeltaheader=builddeltaheader, builddeltaheader=builddeltaheader,
manifestsend=b'', manifestsend=b'',
bundlecaps=bundlecaps, bundlecaps=bundlecaps,
ellipses=ellipses, ellipses=ellipses,
shallow=shallow, shallow=shallow,
ellipsisroots=ellipsisroots, ellipsisroots=ellipsisroots,
fullnodes=fullnodes) fullnodes=fullnodes)
def _makecg3packer(repo, filematcher, bundlecaps, ellipses=False, def _makecg3packer(repo, filematcher, bundlecaps, ellipses=False,
shallow=False, ellipsisroots=None, fullnodes=None): shallow=False, ellipsisroots=None, fullnodes=None):
builddeltaheader = lambda d: _CHANGEGROUPV3_DELTA_HEADER.pack( builddeltaheader = lambda d: _CHANGEGROUPV3_DELTA_HEADER.pack(
d.node, d.p1node, d.p2node, d.basenode, d.linknode, d.flags) d.node, d.p1node, d.p2node, d.basenode, d.linknode, d.flags)
return cgpacker(repo, filematcher, b'03', return cgpacker(repo, filematcher, b'03',
allowreorder=False,
builddeltaheader=builddeltaheader, builddeltaheader=builddeltaheader,
manifestsend=closechunk(), manifestsend=closechunk(),
bundlecaps=bundlecaps, bundlecaps=bundlecaps,
ellipses=ellipses, ellipses=ellipses,
shallow=shallow, shallow=shallow,
ellipsisroots=ellipsisroots, ellipsisroots=ellipsisroots,
fullnodes=fullnodes) fullnodes=fullnodes)

mercurial/configitems.py

# bookmarks.pushing: internal hack for discovery # bookmarks.pushing: internal hack for discovery
coreconfigitem('bookmarks', 'pushing', coreconfigitem('bookmarks', 'pushing',
default=list, default=list,
) )
# bundle.mainreporoot: internal hack for bundlerepo # bundle.mainreporoot: internal hack for bundlerepo
coreconfigitem('bundle', 'mainreporoot', coreconfigitem('bundle', 'mainreporoot',
default='', default='',
) )
# bundle.reorder: experimental config
coreconfigitem('bundle', 'reorder',
default='auto',
)
coreconfigitem('censor', 'policy', coreconfigitem('censor', 'policy',
default='abort', default='abort',
) )
coreconfigitem('chgserver', 'idletimeout', coreconfigitem('chgserver', 'idletimeout',
default=3600, default=3600,
) )
coreconfigitem('chgserver', 'skiphash', coreconfigitem('chgserver', 'skiphash',
default=False, default=False,

tests/test-generaldelta.t

adding manifests adding manifests
adding file changes adding file changes
added 55 changesets with 53 changes to 53 files (+2 heads) added 55 changesets with 53 changes to 53 files (+2 heads)
new changesets 61246295ee1e:c930ac4a5b32 new changesets 61246295ee1e:c930ac4a5b32
updating to branch default updating to branch default
14 files updated, 0 files merged, 0 files removed, 0 files unresolved 14 files updated, 0 files merged, 0 files removed, 0 files unresolved
$ hg -R relax-chain debugdeltachain -m $ hg -R relax-chain debugdeltachain -m
rev chain# chainlen prev delta size rawsize chainsize ratio lindist extradist extraratio rev chain# chainlen prev delta size rawsize chainsize ratio lindist extradist extraratio
0 1 1 -1 base 46 45 46 1.02222 46 0 0.00000 0 1 1 -1 base 47 46 47 1.02174 47 0 0.00000
1 1 2 0 p1 57 90 103 1.14444 103 0 0.00000 1 1 2 0 p1 58 92 105 1.14130 105 0 0.00000
2 1 3 1 p1 57 135 160 1.18519 160 0 0.00000 2 1 3 1 p1 58 138 163 1.18116 163 0 0.00000
3 1 4 2 p1 57 180 217 1.20556 217 0 0.00000 3 1 4 2 p1 58 184 221 1.20109 221 0 0.00000
4 1 5 3 p1 57 225 274 1.21778 274 0 0.00000 4 1 5 3 p1 58 230 279 1.21304 279 0 0.00000
5 1 6 4 p1 57 270 331 1.22593 331 0 0.00000 5 1 6 4 p1 58 276 337 1.22101 337 0 0.00000
6 2 1 -1 base 46 45 46 1.02222 46 0 0.00000 6 1 7 5 p1 58 322 395 1.22671 395 0 0.00000
7 2 2 6 p1 57 90 103 1.14444 103 0 0.00000 7 1 8 6 p1 58 368 453 1.23098 453 0 0.00000
8 2 3 7 p1 57 135 160 1.18519 160 0 0.00000 8 1 9 7 p1 58 414 511 1.23430 511 0 0.00000
9 2 4 8 p1 57 180 217 1.20556 217 0 0.00000 9 1 10 8 p1 58 460 569 1.23696 569 0 0.00000
10 2 5 9 p1 58 226 275 1.21681 275 0 0.00000 10 1 11 9 p1 58 506 627 1.23913 627 0 0.00000
11 2 6 10 p1 58 272 333 1.22426 333 0 0.00000 11 1 12 10 p1 58 552 685 1.24094 685 0 0.00000
12 2 7 11 p1 58 318 391 1.22956 391 0 0.00000 12 1 13 11 p1 58 598 743 1.24247 743 0 0.00000
13 2 8 12 p1 58 364 449 1.23352 449 0 0.00000 13 1 14 12 p1 58 644 801 1.24379 801 0 0.00000
14 2 9 13 p1 58 410 507 1.23659 507 0 0.00000 14 1 15 13 p1 58 690 859 1.24493 859 0 0.00000
15 2 10 14 p1 58 456 565 1.23904 565 0 0.00000 15 1 16 14 p1 58 736 917 1.24592 917 0 0.00000
16 2 11 15 p1 58 502 623 1.24104 623 0 0.00000 16 1 17 15 p1 58 782 975 1.24680 975 0 0.00000
17 2 12 16 p1 58 548 681 1.24270 681 0 0.00000 17 1 18 16 p1 58 828 1033 1.24758 1033 0 0.00000
18 3 1 -1 base 47 46 47 1.02174 47 0 0.00000 18 1 19 17 p1 58 874 1091 1.24828 1091 0 0.00000
19 3 2 18 p1 58 92 105 1.14130 105 0 0.00000 19 1 20 18 p1 58 920 1149 1.24891 1149 0 0.00000
20 3 3 19 p1 58 138 163 1.18116 163 0 0.00000 20 1 21 19 p1 58 966 1207 1.24948 1207 0 0.00000
21 3 4 20 p1 58 184 221 1.20109 221 0 0.00000 21 1 22 20 p1 58 1012 1265 1.25000 1265 0 0.00000
22 3 5 21 p1 58 230 279 1.21304 279 0 0.00000 22 1 23 21 p1 58 1058 1323 1.25047 1323 0 0.00000
23 3 6 22 p1 58 276 337 1.22101 337 0 0.00000 23 1 24 22 p1 58 1104 1381 1.25091 1381 0 0.00000
24 3 7 23 p1 58 322 395 1.22671 395 0 0.00000 24 1 25 23 p1 58 1150 1439 1.25130 1439 0 0.00000
25 3 8 24 p1 58 368 453 1.23098 453 0 0.00000 25 1 26 24 p1 58 1196 1497 1.25167 1497 0 0.00000
26 3 9 25 p1 58 414 511 1.23430 511 0 0.00000 26 1 27 25 p1 58 1242 1555 1.25201 1555 0 0.00000
27 3 10 26 p1 58 460 569 1.23696 569 0 0.00000 27 1 28 26 p1 58 1288 1613 1.25233 1613 0 0.00000
28 3 11 27 p1 58 506 627 1.23913 627 0 0.00000 28 1 29 27 p1 58 1334 1671 1.25262 1671 0 0.00000
29 3 12 28 p1 58 552 685 1.24094 685 0 0.00000 29 1 30 28 p1 58 1380 1729 1.25290 1729 0 0.00000
30 3 13 29 p1 58 598 743 1.24247 743 0 0.00000 30 1 31 29 p1 58 1426 1787 1.25316 1787 0 0.00000
31 3 14 30 p1 58 644 801 1.24379 801 0 0.00000 31 2 1 -1 base 46 45 46 1.02222 46 0 0.00000
32 3 15 31 p1 58 690 859 1.24493 859 0 0.00000 32 2 2 31 p1 57 90 103 1.14444 103 0 0.00000
33 3 16 32 p1 58 736 917 1.24592 917 0 0.00000 33 2 3 32 p1 57 135 160 1.18519 160 0 0.00000
34 3 17 33 p1 58 782 975 1.24680 975 0 0.00000 34 2 4 33 p1 57 180 217 1.20556 217 0 0.00000
35 3 18 34 p1 58 828 1033 1.24758 1033 0 0.00000 35 2 5 34 p1 57 225 274 1.21778 274 0 0.00000
36 3 19 35 p1 58 874 1091 1.24828 1091 0 0.00000 36 2 6 35 p1 57 270 331 1.22593 331 0 0.00000
37 3 20 36 p1 58 920 1149 1.24891 1149 0 0.00000 37 2 7 36 p1 58 316 389 1.23101 389 0 0.00000
38 3 21 37 p1 58 966 1207 1.24948 1207 0 0.00000 38 2 8 37 p1 58 362 447 1.23481 447 0 0.00000
39 3 22 38 p1 58 1012 1265 1.25000 1265 0 0.00000 39 3 1 -1 base 46 45 46 1.02222 46 0 0.00000
40 3 23 39 p1 58 1058 1323 1.25047 1323 0 0.00000 40 3 2 39 p1 57 90 103 1.14444 103 0 0.00000
41 3 24 40 p1 58 1104 1381 1.25091 1381 0 0.00000 41 3 3 40 p1 57 135 160 1.18519 160 0 0.00000
42 3 25 41 p1 58 1150 1439 1.25130 1439 0 0.00000 42 3 4 41 p1 57 180 217 1.20556 217 0 0.00000
43 3 26 42 p1 58 1196 1497 1.25167 1497 0 0.00000 43 3 5 42 p1 58 226 275 1.21681 275 0 0.00000
44 3 27 43 p1 58 1242 1555 1.25201 1555 0 0.00000 44 3 6 43 p1 58 272 333 1.22426 333 0 0.00000
45 3 28 44 p1 58 1288 1613 1.25233 1613 0 0.00000 45 3 7 44 p1 58 318 391 1.22956 391 0 0.00000
46 3 29 45 p1 58 1334 1671 1.25262 1671 0 0.00000 46 3 8 45 p1 58 364 449 1.23352 449 0 0.00000
47 3 30 46 p1 58 1380 1729 1.25290 1729 0 0.00000 47 3 9 46 p1 58 410 507 1.23659 507 0 0.00000
48 3 31 47 p1 58 1426 1787 1.25316 1787 0 0.00000 48 3 10 47 p1 58 456 565 1.23904 565 0 0.00000
49 4 1 -1 base 197 316 197 0.62342 197 0 0.00000 49 3 11 48 p1 58 502 623 1.24104 623 0 0.00000
50 4 2 49 p1 58 362 255 0.70442 255 0 0.00000 50 3 12 49 p1 58 548 681 1.24270 681 0 0.00000
51 2 13 17 p1 58 594 739 1.24411 2781 2042 2.76319 51 3 13 50 p1 58 594 739 1.24411 739 0 0.00000
52 5 1 -1 base 369 640 369 0.57656 369 0 0.00000 52 3 14 51 p1 58 640 797 1.24531 797 0 0.00000
53 6 1 -1 base 0 0 0 0.00000 0 0 0.00000 53 4 1 -1 base 0 0 0 0.00000 0 0 0.00000
54 7 1 -1 base 369 640 369 0.57656 369 0 0.00000 54 5 1 -1 base 369 640 369 0.57656 369 0 0.00000
$ hg clone --pull source-repo --config experimental.maxdeltachainspan=0 noconst-chain --config format.generaldelta=yes $ hg clone --pull source-repo --config experimental.maxdeltachainspan=0 noconst-chain --config format.generaldelta=yes
requesting all changes requesting all changes
adding changesets adding changesets
adding manifests adding manifests
adding file changes adding file changes
added 55 changesets with 53 changes to 53 files (+2 heads) added 55 changesets with 53 changes to 53 files (+2 heads)
new changesets 61246295ee1e:c930ac4a5b32 new changesets 61246295ee1e:c930ac4a5b32
updating to branch default updating to branch default
14 files updated, 0 files merged, 0 files removed, 0 files unresolved 14 files updated, 0 files merged, 0 files removed, 0 files unresolved
$ hg -R noconst-chain debugdeltachain -m $ hg -R noconst-chain debugdeltachain -m
rev chain# chainlen prev delta size rawsize chainsize ratio lindist extradist extraratio rev chain# chainlen prev delta size rawsize chainsize ratio lindist extradist extraratio
0 1 1 -1 base 46 45 46 1.02222 46 0 0.00000 0 1 1 -1 base 47 46 47 1.02174 47 0 0.00000
1 1 2 0 p1 57 90 103 1.14444 103 0 0.00000 1 1 2 0 p1 58 92 105 1.14130 105 0 0.00000
2 1 3 1 p1 57 135 160 1.18519 160 0 0.00000 2 1 3 1 p1 58 138 163 1.18116 163 0 0.00000
3 1 4 2 p1 57 180 217 1.20556 217 0 0.00000 3 1 4 2 p1 58 184 221 1.20109 221 0 0.00000
4 1 5 3 p1 57 225 274 1.21778 274 0 0.00000 4 1 5 3 p1 58 230 279 1.21304 279 0 0.00000
5 1 6 4 p1 57 270 331 1.22593 331 0 0.00000 5 1 6 4 p1 58 276 337 1.22101 337 0 0.00000
6 2 1 -1 base 46 45 46 1.02222 46 0 0.00000 6 1 7 5 p1 58 322 395 1.22671 395 0 0.00000
7 2 2 6 p1 57 90 103 1.14444 103 0 0.00000 7 1 8 6 p1 58 368 453 1.23098 453 0 0.00000
8 2 3 7 p1 57 135 160 1.18519 160 0 0.00000 8 1 9 7 p1 58 414 511 1.23430 511 0 0.00000
9 2 4 8 p1 57 180 217 1.20556 217 0 0.00000 9 1 10 8 p1 58 460 569 1.23696 569 0 0.00000
10 2 5 9 p1 58 226 275 1.21681 275 0 0.00000 10 1 11 9 p1 58 506 627 1.23913 627 0 0.00000
11 2 6 10 p1 58 272 333 1.22426 333 0 0.00000 11 1 12 10 p1 58 552 685 1.24094 685 0 0.00000
12 2 7 11 p1 58 318 391 1.22956 391 0 0.00000 12 1 13 11 p1 58 598 743 1.24247 743 0 0.00000
13 2 8 12 p1 58 364 449 1.23352 449 0 0.00000 13 1 14 12 p1 58 644 801 1.24379 801 0 0.00000
14 2 9 13 p1 58 410 507 1.23659 507 0 0.00000 14 1 15 13 p1 58 690 859 1.24493 859 0 0.00000
15 2 10 14 p1 58 456 565 1.23904 565 0 0.00000 15 1 16 14 p1 58 736 917 1.24592 917 0 0.00000
16 2 11 15 p1 58 502 623 1.24104 623 0 0.00000 16 1 17 15 p1 58 782 975 1.24680 975 0 0.00000
17 2 12 16 p1 58 548 681 1.24270 681 0 0.00000 17 1 18 16 p1 58 828 1033 1.24758 1033 0 0.00000
18 3 1 -1 base 47 46 47 1.02174 47 0 0.00000 18 1 19 17 p1 58 874 1091 1.24828 1091 0 0.00000
19 3 2 18 p1 58 92 105 1.14130 105 0 0.00000 19 1 20 18 p1 58 920 1149 1.24891 1149 0 0.00000
20 3 3 19 p1 58 138 163 1.18116 163 0 0.00000 20 1 21 19 p1 58 966 1207 1.24948 1207 0 0.00000
21 3 4 20 p1 58 184 221 1.20109 221 0 0.00000 21 1 22 20 p1 58 1012 1265 1.25000 1265 0 0.00000
22 3 5 21 p1 58 230 279 1.21304 279 0 0.00000 22 1 23 21 p1 58 1058 1323 1.25047 1323 0 0.00000
23 3 6 22 p1 58 276 337 1.22101 337 0 0.00000 23 1 24 22 p1 58 1104 1381 1.25091 1381 0 0.00000
24 3 7 23 p1 58 322 395 1.22671 395 0 0.00000 24 1 25 23 p1 58 1150 1439 1.25130 1439 0 0.00000
25 3 8 24 p1 58 368 453 1.23098 453 0 0.00000 25 1 26 24 p1 58 1196 1497 1.25167 1497 0 0.00000
26 3 9 25 p1 58 414 511 1.23430 511 0 0.00000 26 1 27 25 p1 58 1242 1555 1.25201 1555 0 0.00000
27 3 10 26 p1 58 460 569 1.23696 569 0 0.00000 27 1 28 26 p1 58 1288 1613 1.25233 1613 0 0.00000
28 3 11 27 p1 58 506 627 1.23913 627 0 0.00000 28 1 29 27 p1 58 1334 1671 1.25262 1671 0 0.00000
29 3 12 28 p1 58 552 685 1.24094 685 0 0.00000 29 1 30 28 p1 58 1380 1729 1.25290 1729 0 0.00000
30 3 13 29 p1 58 598 743 1.24247 743 0 0.00000 30 1 31 29 p1 58 1426 1787 1.25316 1787 0 0.00000
31 3 14 30 p1 58 644 801 1.24379 801 0 0.00000 31 2 1 -1 base 46 45 46 1.02222 46 0 0.00000
32 3 15 31 p1 58 690 859 1.24493 859 0 0.00000 32 2 2 31 p1 57 90 103 1.14444 103 0 0.00000
33 3 16 32 p1 58 736 917 1.24592 917 0 0.00000 33 2 3 32 p1 57 135 160 1.18519 160 0 0.00000
34 3 17 33 p1 58 782 975 1.24680 975 0 0.00000 34 2 4 33 p1 57 180 217 1.20556 217 0 0.00000
35 3 18 34 p1 58 828 1033 1.24758 1033 0 0.00000 35 2 5 34 p1 57 225 274 1.21778 274 0 0.00000
36 3 19 35 p1 58 874 1091 1.24828 1091 0 0.00000 36 2 6 35 p1 57 270 331 1.22593 331 0 0.00000
37 3 20 36 p1 58 920 1149 1.24891 1149 0 0.00000 37 2 7 36 p1 58 316 389 1.23101 389 0 0.00000
38 3 21 37 p1 58 966 1207 1.24948 1207 0 0.00000 38 2 8 37 p1 58 362 447 1.23481 447 0 0.00000
39 3 22 38 p1 58 1012 1265 1.25000 1265 0 0.00000 39 3 1 -1 base 46 45 46 1.02222 46 0 0.00000
40 3 23 39 p1 58 1058 1323 1.25047 1323 0 0.00000 40 3 2 39 p1 57 90 103 1.14444 103 0 0.00000
41 3 24 40 p1 58 1104 1381 1.25091 1381 0 0.00000 41 3 3 40 p1 57 135 160 1.18519 160 0 0.00000
42 3 25 41 p1 58 1150 1439 1.25130 1439 0 0.00000 42 3 4 41 p1 57 180 217 1.20556 217 0 0.00000
43 3 26 42 p1 58 1196 1497 1.25167 1497 0 0.00000 43 3 5 42 p1 58 226 275 1.21681 275 0 0.00000
44 3 27 43 p1 58 1242 1555 1.25201 1555 0 0.00000 44 3 6 43 p1 58 272 333 1.22426 333 0 0.00000
45 3 28 44 p1 58 1288 1613 1.25233 1613 0 0.00000 45 3 7 44 p1 58 318 391 1.22956 391 0 0.00000
46 3 29 45 p1 58 1334 1671 1.25262 1671 0 0.00000 46 3 8 45 p1 58 364 449 1.23352 449 0 0.00000
47 3 30 46 p1 58 1380 1729 1.25290 1729 0 0.00000 47 3 9 46 p1 58 410 507 1.23659 507 0 0.00000
48 3 31 47 p1 58 1426 1787 1.25316 1787 0 0.00000 48 3 10 47 p1 58 456 565 1.23904 565 0 0.00000
49 1 7 5 p1 58 316 389 1.23101 2857 2468 6.34447 49 3 11 48 p1 58 502 623 1.24104 623 0 0.00000
50 1 8 49 p1 58 362 447 1.23481 2915 2468 5.52125 50 3 12 49 p1 58 548 681 1.24270 681 0 0.00000
51 2 13 17 p1 58 594 739 1.24411 2642 1903 2.57510 51 3 13 50 p1 58 594 739 1.24411 739 0 0.00000
52 2 14 51 p1 58 640 797 1.24531 2700 1903 2.38770 52 3 14 51 p1 58 640 797 1.24531 797 0 0.00000
53 4 1 -1 base 0 0 0 0.00000 0 0 0.00000 53 4 1 -1 base 0 0 0 0.00000 0 0 0.00000
54 5 1 -1 base 369 640 369 0.57656 369 0 0.00000 54 5 1 -1 base 369 640 369 0.57656 369 0 0.00000