This is an archive of the discontinued Mercurial Phabricator instance.

Differential D9503

copies: avoid materializing a full directory map during copy tracing
ClosedPublic

Authored by spectral on Dec 2 2020, 2:43 PM.

Details

Reviewers
pulkit
Group Reviewers
hg-reviewers
Commits
rHGb9588ff9b66a: copies: avoid materializing a full directory map during copy tracing
Summary

Materializing a full copy of every directory in a treemanifest repo can be quite
expensive, even with a narrow matcher. For flat manifest repos, this should be
equivalent - it will still materialize (and cache) a dict of all of the dirs
inside of the manifest object, we just don't get a copy of it.

In a repo I have here, this brings the time for a simple rebase from 11.197s to
4.609s.

Diff Detail

Repository
rHG Mercurial
Branch
default
Lint
No Linters Available
Unit
No Unit Test Coverage

Revision Contents
Changeset List

PathPackages
Mmercurial/copies.py (6 lines)

Diff 23967

mercurial/copies.py

"""Finds moved directories and files that should move with them. """Finds moved directories and files that should move with them.
ctx: the context for one of the sides ctx: the context for one of the sides
copy: files copied on the same side (as ctx) copy: files copied on the same side (as ctx)
fullcopy: files copied on the same side (as ctx), including those that fullcopy: files copied on the same side (as ctx), including those that
merge.manifestmerge() won't care about merge.manifestmerge() won't care about
addedfiles: added files on the other side (compared to ctx) addedfiles: added files on the other side (compared to ctx)
""" """
# generate a directory move map
d = ctx.dirs()
invalid = set() invalid = set()
dirmove = {} dirmove = {}
# examine each file copy for a potential directory move, which is # examine each file copy for a potential directory move, which is
# when all the files in a directory are moved to a new directory # when all the files in a directory are moved to a new directory
for dst, src in pycompat.iteritems(fullcopy): for dst, src in pycompat.iteritems(fullcopy):
dsrc, ddst = pathutil.dirname(src), pathutil.dirname(dst) dsrc, ddst = pathutil.dirname(src), pathutil.dirname(dst)
if dsrc in invalid: if dsrc in invalid:
# already seen to be uninteresting # already seen to be uninteresting
continue continue
elif dsrc in d and ddst in d: elif ctx.hasdir(dsrc) and ctx.hasdir(ddst):
# directory wasn't entirely moved locally # directory wasn't entirely moved locally
invalid.add(dsrc) invalid.add(dsrc)
elif dsrc in dirmove and dirmove[dsrc] != ddst: elif dsrc in dirmove and dirmove[dsrc] != ddst:
# files from the same directory moved to two different places # files from the same directory moved to two different places
invalid.add(dsrc) invalid.add(dsrc)
else: else:
# looks good so far # looks good so far
dirmove[dsrc] = ddst dirmove[dsrc] = ddst
for i in invalid: for i in invalid:
if i in dirmove: if i in dirmove:
del dirmove[i] del dirmove[i]
del d, invalid del invalid
if not dirmove: if not dirmove:
return {}, {} return {}, {}
dirmove = {k + b"/": v + b"/" for k, v in pycompat.iteritems(dirmove)} dirmove = {k + b"/": v + b"/" for k, v in pycompat.iteritems(dirmove)}
for d in dirmove: for d in dirmove:
repo.ui.debug( repo.ui.debug(