This is an archive of the discontinued Mercurial Phabricator instance.

Differential D763

copytrace: add a a new config to limit the number of drafts in heuristics
ClosedPublic

Authored by pulkit on Sep 21 2017, 8:19 AM.

Details

Reviewers
yuja
Group Reviewers
hg-reviewers
Commits
rHG1826d695ad58: copytrace: add a a new config to limit the number of drafts in heuristics
Summary

The heuristics options tries to the default full copytracing algorithm if both
the source and destination branches contains of non-public changesets only. But
this can be slow in cases when we have a lot of drafts.

This patch adds a new config option experimental.copytrace.sourcecommitlimit
which defaults to 100. This value will be the limit of number of drafts from c1
to base. Incase there are more changesets even though they are draft, the
heuristics algorithm will be used.

Diff Detail

Repository
rHG Mercurial
Lint
Automatic diff as part of commit; lint not applicable.
Unit
Automatic diff as part of commit; unit tests not applicable.

Event Timeline

pulkit created this revision.Sep 21 2017, 8:19 AM
pulkit updated this revision to Diff 1967.Sep 21 2017, 12:37 PM
yuja requested changes to this revision.Sep 25 2017, 9:48 AM
yuja added a subscriber: yuja.
yuja added inline comments.
mercurial/copies.py
394

Can you make it use the new config registrar?

397

Perhaps this loop needs sanity check we've added for
_heuristicscopytracing().

BTW, can't we use revset to count the number of commits?

This revision now requires changes to proceed.Sep 25 2017, 9:48 AM
pulkit updated this revision to Diff 2084.Sep 26 2017, 8:49 AM
yuja requested changes to this revision.Sep 26 2017, 9:19 AM
yuja added inline comments.
mercurial/copies.py
393

Oops. repo isn't passed to this function.

Write tests?

This revision now requires changes to proceed.Sep 26 2017, 9:19 AM
pulkit planned changes to this revision.Sep 26 2017, 9:47 AM
pulkit added inline comments.
mercurial/copies.py
393

Ah, my bad. Thanks for catching. Yeah will include tests in the next version.

pulkit updated this revision to Diff 2089.Sep 26 2017, 11:05 AM
Closed by commit rHG1826d695ad58: copytrace: add a a new config to limit the number of drafts in heuristics (authored by pulkit). · Explain WhySep 27 2017, 3:36 AM
This revision was automatically updated to reflect the committed changes.

Revision Contents
Changeset List

Diff 2103

mercurial/configitems.py

default=False, default=False,
) )
coreconfigitem('experimental', 'clientcompressionengines', coreconfigitem('experimental', 'clientcompressionengines',
default=list, default=list,
) )
coreconfigitem('experimental', 'copytrace', coreconfigitem('experimental', 'copytrace',
default='on', default='on',
) )
coreconfigitem('experimental', 'copytrace.sourcecommitlimit',
default=100,
)
coreconfigitem('experimental', 'crecordtest', coreconfigitem('experimental', 'crecordtest',
default=None, default=None,
) )
coreconfigitem('experimental', 'editortmpinhg', coreconfigitem('experimental', 'editortmpinhg',
default=False, default=False,
) )
coreconfigitem('experimental', 'stabilization', coreconfigitem('experimental', 'stabilization',
default=list, default=list,

mercurial/copies.py

# because the logic above is required for a simple copy to be kept across a # because the logic above is required for a simple copy to be kept across a
# rebase. # rebase.
if copytracing == 'off': if copytracing == 'off':
return {}, {}, {}, {}, {} return {}, {}, {}, {}, {}
elif copytracing == 'heuristics': elif copytracing == 'heuristics':
# Do full copytracing if only drafts are involved as that will be fast # Do full copytracing if only drafts are involved as that will be fast
# enough and will also cover the copies which can be missed by # enough and will also cover the copies which can be missed by
# heuristics # heuristics
if _isfullcopytraceable(c1, base): if _isfullcopytraceable(repo, c1, base):
return _fullcopytracing(repo, c1, c2, base) return _fullcopytracing(repo, c1, c2, base)
return _heuristicscopytracing(repo, c1, c2, base) return _heuristicscopytracing(repo, c1, c2, base)
else: else:
return _fullcopytracing(repo, c1, c2, base) return _fullcopytracing(repo, c1, c2, base)
def _isfullcopytraceable(c1, base): def _isfullcopytraceable(repo, c1, base):
""" Checks that if base, source and destination are all draft branches, if """ Checks that if base, source and destination are all draft branches, if
yes let's use the full copytrace algorithm for increased capabilities since yes let's use the full copytrace algorithm for increased capabilities since
it will be fast enough. it will be fast enough.
""" """
if c1.rev() is None:
c1 = c1.p1()
nonpublicphases = set([phases.draft, phases.secret]) nonpublicphases = set([phases.draft, phases.secret])
if (c1.phase() in nonpublicphases) and (base.phase() in nonpublicphases): if (c1.phase() in nonpublicphases) and (base.phase() in nonpublicphases):
return True sourcecommitlimit = repo.ui.configint('experimental',
'copytrace.sourcecommitlimit')
commits = len(repo.revs('%d::%d', base.rev(), c1.rev()))
yujaUnsubmitted
Not Done

Oops. repo isn't passed to this function.

Write tests?

yuja: Oops. `repo` isn't passed to this function. Write tests?
pulkitAuthorUnsubmitted
Not Done

Ah, my bad. Thanks for catching. Yeah will include tests in the next version.

pulkit: Ah, my bad. Thanks for catching. Yeah will include tests in the next version.
return commits < sourcecommitlimit
yujaUnsubmitted
Not Done

Can you make it use the new config registrar?

yuja: Can you make it use the new config registrar?
return False return False
def _fullcopytracing(repo, c1, c2, base): def _fullcopytracing(repo, c1, c2, base):
yujaUnsubmitted
Not Done

Perhaps this loop needs sanity check we've added for
_heuristicscopytracing().

BTW, can't we use revset to count the number of commits?

yuja: Perhaps this loop needs sanity check we've added for _heuristicscopytracing(). BTW, can't we…
""" The full copytracing algorithm which finds all the new files that were """ The full copytracing algorithm which finds all the new files that were
added from merge base up to the top commit and for each file it checks if added from merge base up to the top commit and for each file it checks if
this file was copied from another file. this file was copied from another file.
This is pretty slow when a lot of changesets are involved but will track all This is pretty slow when a lot of changesets are involved but will track all
the copies. the copies.
""" """
# In certain scenarios (e.g. graft, update or rebase), base can be # In certain scenarios (e.g. graft, update or rebase), base can be

tests/test-copytrace-heuristics.t

$ hg rebase -s . -d 1 $ hg rebase -s . -d 1
rebasing 2:e8919e7df8d0 "mv dir1 dir2" (tip) rebasing 2:e8919e7df8d0 "mv dir1 dir2" (tip)
saved backup bundle to $TESTTMP/repo/repo/.hg/strip-backup/e8919e7df8d0-f62fab62-rebase.hg (glob) saved backup bundle to $TESTTMP/repo/repo/.hg/strip-backup/e8919e7df8d0-f62fab62-rebase.hg (glob)
$ ls dir2 $ ls dir2
a a
dummy dummy
$ rm -rf server $ rm -rf server
$ rm -rf repo $ rm -rf repo
Testing the sourcecommitlimit config
$ hg init repo
$ initclient repo
$ cd repo
$ echo a > a
$ hg ci -Aqm "added a"
$ echo "more things" >> a
$ hg ci -qm "added more things to a"
$ hg up 0
1 files updated, 0 files merged, 0 files removed, 0 files unresolved
$ echo b > b
$ hg ci -Aqm "added b"
$ mkdir foo
$ hg mv a foo/bar
$ hg ci -m "Moved a to foo/bar"
$ hg log -G -T 'changeset {node}\n desc {desc}, phase: {phase}\n'
@ changeset b4b0f7880e500b5c364a5f07b4a2b167de7a6fb0
| desc Moved a to foo/bar, phase: draft
o changeset 5f6d8a4bf34ab274ccc9f631c2536964b8a3666d
| desc added b, phase: draft
| o changeset 8b6e13696c38e8445a759516474640c2f8dddef6
|/ desc added more things to a, phase: draft
o changeset 9092f1db7931481f93b37d5c9fbcfc341bcd7318
desc added a, phase: draft
When the sourcecommitlimit is small and we have more drafts, we use heuristics only
$ hg rebase -s 8b6e13696 -d . --config experimental.copytrace.sourcecommitlimit=0
rebasing 1:8b6e13696c38 "added more things to a"
other [source] changed a which local [dest] deleted
use (c)hanged version, leave (d)eleted, or leave (u)nresolved? u
unresolved conflicts (see hg resolve, then hg rebase --continue)
[1]
But when we have "sourcecommitlimit > (no. of drafts from base to c1)", we do
fullcopytracing
$ hg rebase --abort
rebase aborted
$ hg rebase -s 8b6e13696 -d .
rebasing 1:8b6e13696c38 "added more things to a"
merging foo/bar and a to foo/bar
saved backup bundle to $TESTTMP/repo/repo/repo/.hg/strip-backup/8b6e13696c38-fc14ac83-rebase.hg (glob)
$ cd ..
$ rm -rf repo