( )⚙ D6419 copies: do full filtering at end of _changesetforwardcopies()

This is an archive of the discontinued Mercurial Phabricator instance.

Differential D6419

copies: do full filtering at end of _changesetforwardcopies()
ClosedPublic

Authored by martinvonz on May 21 2019, 8:32 PM.

Details

Reviewers
None
Group Reviewers
hg-reviewers
Commits
rHG4c39c99d9492: copies: do full filtering at end of _changesetforwardcopies()
Summary

As mentioned earlier, pathcopies() is very slow when copies are stored
in the changeset. Most of the cost comes from calling _chain() for
every changeset, which is slow because it needs to read manifests. It
needs to read manifests to be able to filter out copies that are were
created in one commit and then deleted. (It also filters out copies
that were created from a file that didn't exist in the starting
revision, but that's a fixed revision across calls to _chain(), so
it's much cheaper.)

This patch changes from _chainandfilter() to just _chain() in the main
loop in _changesetforwardcopies(). It instead removes copies that have
subsequently been removed by using ctx.filesremoved(). We thus rely on
that to be fast.

It timed this command in mozilla-unified:

hg debugpathcopies FIREFOX_59_0b3_BUILD2 FIREFOX_BETA_59_END

It took 18s before and 1.1s after. It's still faster when copy
information is stored in filelogs: 0.70s. It also still gets slow when
there are merge commits involved, because we read manifests there
too. We'll deal with that later.

Diff Detail

Repository
rHG Mercurial
Lint
Automatic diff as part of commit; lint not applicable.
Unit
Automatic diff as part of commit; unit tests not applicable.

Event Timeline

martinvonz created this revision.May 21 2019, 8:32 PM
martinvonz added a comment.Jun 14 2019, 2:27 AM

Could someone look at the remaining patches in this series? They've been waiting for three weeks or so now.

martinvonz added a commit: rHG4c39c99d9492: copies: do full filtering at end of _changesetforwardcopies().Jun 17 2019, 1:33 PM
This revision was not accepted when it landed; it landed in state Needs Review.
Closed by commit rHG4c39c99d9492: copies: do full filtering at end of _changesetforwardcopies() (authored by martinvonz). · Explain Why
This revision was automatically updated to reflect the committed changes.
marmoute added a subscriber: marmoute.Jun 17 2019, 6:11 PM

For the record. I am planning to make an extra path on that this week (in case nobody else got there first).

Revision Contents
Changeset List

PathPackages
Mmercurial/copies.py (6 lines)

Diff 15543

mercurial/copies.py

if dst not in p1man: if dst not in p1man:
copies[dst] = copies2[dst] copies[dst] = copies2[dst]
else: else:
# Copied on both sides: mark as copy from p1 side # Copied on both sides: mark as copy from p1 side
copies[dst] = copies1[dst] copies[dst] = copies1[dst]
else: else:
copies = copies1 copies = copies1
if r == b.rev(): if r == b.rev():
_filter(a, b, copies)
return copies return copies
for c in children[r]: for c in children[r]:
childctx = repo[c] childctx = repo[c]
if r == childctx.p1().rev(): if r == childctx.p1().rev():
parent = 1 parent = 1
childcopies = childctx.p1copies() childcopies = childctx.p1copies()
else: else:
assert r == childctx.p2().rev() assert r == childctx.p2().rev()
parent = 2 parent = 2
childcopies = childctx.p2copies() childcopies = childctx.p2copies()
if not match.always(): if not match.always():
childcopies = {dst: src for dst, src in childcopies.items() childcopies = {dst: src for dst, src in childcopies.items()
if match(dst)} if match(dst)}
childcopies = _chainandfilter(a, childctx, copies, childcopies) childcopies = _chain(copies, childcopies)
for f in childctx.filesremoved():
if f in childcopies:
del childcopies[f]
heapq.heappush(work, (c, parent, childcopies)) heapq.heappush(work, (c, parent, childcopies))
assert False assert False
def _forwardcopies(a, b, match=None): def _forwardcopies(a, b, match=None):
"""find {dst@b: src@a} copy mapping where a is an ancestor of b""" """find {dst@b: src@a} copy mapping where a is an ancestor of b"""
match = a.repo().narrowmatch(match) match = a.repo().narrowmatch(match)
# check for working copy # check for working copy