This is an archive of the discontinued Mercurial Phabricator instance.

Differential D7127

sidedatacopies: only fetch information once for merge
ClosedPublic

Authored by marmoute on Oct 17 2019, 7:45 AM.

Details

Reviewers
None
Group Reviewers
hg-reviewers
Commits
rHG90213d027154: sidedatacopies: only fetch information once for merge
Summary

Before this change, merge would result in reading the data from revlog twice.
With this change, we keep the information in memory until we encounter the other
parent.

When looking at pypy, I see about 1/3 of the changesets with copy information
being merge. Not doing duplicated fetch for them provide a significant speedup.

revision: large amount; added files: large amount; rename small amount; c3b14617fbd7 9ba6ab77fd29
before: ! wall 0.767042 comb 0.760000 user 0.750000 sys 0.010000 (median of 11)
after: ! wall 0.671162 comb 0.670000 user 0.650000 sys 0.020000 (median of 13)
revision: large amount; added files: small amount; rename small amount; c3b14617fbd7 f650a9b140d2
before: ! wall 1.170169 comb 1.170000 user 1.130000 sys 0.040000 (median of 10)
after: ! wall 1.030596 comb 1.040000 user 1.010000 sys 0.030000 (median of 10)
revision: large amount; added files: large amount; rename large amount; 08ea3258278e d9fa043f30c0
before: ! wall 0.209846 comb 0.200000 user 0.200000 sys 0.000000 (median of 46)
after: ! wall 0.170981 comb 0.170000 user 0.170000 sys 0.000000 (median of 56)
revision: small amount; added files: large amount; rename large amount; df6f7a526b60 a83dc6a2d56f
before: ! wall 0.013248 comb 0.010000 user 0.010000 sys 0.000000 (median of 223)
after: ! wall 0.013295 comb 0.020000 user 0.020000 sys 0.000000 (median of 222)
revision: small amount; added files: large amount; rename small amount; 4aa4e1f8e19a 169138063d63
before: ! wall 0.001672 comb 0.000000 user 0.000000 sys 0.000000 (median of 1000)
after: ! wall 0.001666 comb 0.000000 user 0.000000 sys 0.000000 (median of 1000)
revision: small amount; added files: small amount; rename small amount; 4bc173b045a6 964879152e2e
before: ! wall 0.000119 comb 0.000000 user 0.000000 sys 0.000000 (median of 8010)
after: ! wall 0.000119 comb 0.000000 user 0.000000 sys 0.000000 (median of 8007)
revision: medium amount; added files: large amount; rename medium amount; c95f1ced15f2 2c68e87c3efe
before: ! wall 0.168599 comb 0.160000 user 0.160000 sys 0.000000 (median of 58)
after: ! wall 0.133316 comb 0.140000 user 0.140000 sys 0.000000 (median of 73)
revision: medium amount; added files: medium amount; rename small amount; d343da0c55a8 d7746d32bf9d
before: ! wall 0.036052 comb 0.030000 user 0.030000 sys 0.000000 (median of 100)
after: ! wall 0.032558 comb 0.030000 user 0.030000 sys 0.000000 (median of 100)

Diff Detail

Repository
rHG Mercurial
Lint
Lint Skipped
Unit
Unit Tests Skipped

Event Timeline

marmoute created this revision.Oct 17 2019, 7:45 AM
martinvonz added a subscriber: martinvonz.Oct 19 2019, 1:36 PM

Nice work on this series. It's all queued now.

marmoute added a commit: rHG90213d027154: sidedatacopies: only fetch information once for merge.Oct 19 2019, 1:38 PM
This revision was not accepted when it landed; it landed in state Needs Review.
Closed by commit rHG90213d027154: sidedatacopies: only fetch information once for merge (authored by marmoute). · Explain Why
This revision was automatically updated to reflect the committed changes.

Revision Contents
Changeset List

PathPackages
Mmercurial/copies.py (31 lines)

Diff 17326

mercurial/copies.py

""" """
cl = repo.changelog cl = repo.changelog
parents = cl.parentrevs parents = cl.parentrevs
if repo.filecopiesmode == b'changeset-sidedata': if repo.filecopiesmode == b'changeset-sidedata':
changelogrevision = cl.changelogrevision changelogrevision = cl.changelogrevision
flags = cl.flags flags = cl.flags
# A small cache to avoid doing the work twice for merges
#
# In the vast majority of cases, if we ask information for a revision
# about 1 parent, we'll later ask it for the other. So it make sense to
# keep the information around when reaching the first parent of a merge
# and dropping it after it was provided for the second parents.
#
# It exists cases were only one parent of the merge will be walked. It
# happens when the "destination" the copy tracing is descendant from a
# new root, not common with the "source". In that case, we will only walk
# through merge parents that are descendant of changesets common
# between "source" and "destination".
#
# With the current case implementation if such changesets have a copy
# information, we'll keep them in memory until the end of
# _changesetforwardcopies. We don't expect the case to be frequent
# enough to matters.
#
# In addition, it would be possible to reach pathological case, were
# many first parent are met before any second parent is reached. In
# that case the cache could grow. If this even become an issue one can
# safely introduce a maximum cache size. This would trade extra CPU/IO
# time to save memory.
merge_caches = {}
def revinfo(rev): def revinfo(rev):
p1, p2 = parents(rev) p1, p2 = parents(rev)
if flags(rev) & REVIDX_SIDEDATA: if flags(rev) & REVIDX_SIDEDATA:
e = merge_caches.pop(rev, None)
if e is not None:
return e
c = changelogrevision(rev) c = changelogrevision(rev)
p1copies = c.p1copies p1copies = c.p1copies
p2copies = c.p2copies p2copies = c.p2copies
removed = c.filesremoved removed = c.filesremoved
if p1 != node.nullrev and p2 != node.nullrev:
# XXX some case we over cache, IGNORE
merge_caches[rev] = (p1, p2, p1copies, p2copies, removed)
else: else:
p1copies = {} p1copies = {}
p2copies = {} p2copies = {}
removed = () removed = ()
return p1, p2, p1copies, p2copies, removed return p1, p2, p1copies, p2copies, removed
else: else: