This is an archive of the discontinued Mercurial Phabricator instance.

Differential D262

test-rebase: add a brute force test
ClosedPublic

Authored by quark on Aug 7 2017, 12:11 PM.

Details

Reviewers
None
Group Reviewers
hg-reviewers
Commits
rHG71b77b61ed60: test-rebase: add a brute force test
Summary

Rebase is becoming more complex and it looks like a good idea to try some
brute force enumeration to cover cases that are hard to find manually.

Using brute force to generate repos in different shapes and enumerating the
rebase source and destination would generate too many cases that takes too
long to compute. This patch limits the "brute force" to only the "rebase
source" part. Repo and destination are still manual.

The added test cases are crafted manually to reveal some behaviors that are
not covered by other tests:

  • "revlog index out of range" crash
  • after rebase, p1 == p2, p2 != null
  • "nothing to merge" abort

In the future we might want to add more tests here. For now I'm more
interested in revealing interesting behaviors in a minified way. I tried
some more complex cases but didn't find other interesting behaviors.

Diff Detail

Repository
rHG Mercurial
Lint
Automatic diff as part of commit; lint not applicable.
Unit
Automatic diff as part of commit; unit tests not applicable.

Event Timeline

quark created this revision.Aug 7 2017, 12:11 PM
lothiraldan added a subscriber: lothiraldan.Aug 7 2017, 12:59 PM

Thanks for the patch, more testing is always good.

I'm wondering if we could use hypothesis (https://github.com/HypothesisWorks/hypothesis-python) for more tests like that. One article I found interesting was: http://hypothesis.works/articles/rule-based-stateful-testing/

quark added a comment.Aug 7 2017, 1:07 PM

While I think hypothesis might be somehow useful. It is to generate random test cases, which is different from the efficient brute force here.

Closed by commit rHG71b77b61ed60: test-rebase: add a brute force test (authored by quark). · Explain WhyAug 8 2017, 4:38 PM
This revision was automatically updated to reflect the committed changes.

Revision Contents
Changeset List

PathPackages
AMtests/bruterebase.py (69 lines)
AMtests/test-rebase-brute-force.t (55 lines)

Diff 652

tests/bruterebase.py

  • This file was added.
# bruterebase.py - brute force rebase testing
#
# Copyright 2017 Facebook, Inc.
#
# This software may be used and distributed according to the terms of the
# GNU General Public License version 2 or any later version.
from __future__ import absolute_import
from mercurial import (
error,
registrar,
revsetlang,
)
from hgext import rebase
cmdtable = {}
command = registrar.command(cmdtable)
@command('debugbruterebase')
def debugbruterebase(ui, repo, source, dest):
"""for every non-empty subset of source, run rebase -r subset -d dest
Print one line summary for each subset. Assume obsstore is enabled.
"""
srevs = list(repo.revs(source))
with repo.wlock(), repo.lock():
repolen = len(repo)
cl = repo.changelog
def getdesc(rev):
result = cl.changelogrevision(rev).description
if rev >= repolen:
result += "'"
return result
for i in xrange(1, 2 ** len(srevs)):
subset = [rev for j, rev in enumerate(srevs) if i & (1 << j) != 0]
spec = revsetlang.formatspec('%ld', subset)
tr = repo.transaction('rebase')
tr.report = lambda x: 0 # hide "transaction abort"
ui.pushbuffer()
try:
rebase.rebase(ui, repo, dest=dest, rev=[spec])
except error.Abort as ex:
summary = 'ABORT: %s' % ex
except Exception as ex:
summary = 'CRASH: %s' % ex
else:
# short summary about new nodes
cl = repo.changelog
descs = []
for rev in xrange(repolen, len(repo)):
desc = '%s:' % getdesc(rev)
for prev in cl.parentrevs(rev):
if prev > -1:
desc += getdesc(prev)
descs.append(desc)
descs.sort()
summary = ' '.join(descs)
ui.popbuffer()
repo.vfs.tryunlink('rebasestate')
subsetdesc = ''.join(getdesc(rev) for rev in subset)
ui.write(('%s: %s\n') % (subsetdesc.rjust(len(srevs)), summary))
tr.abort()

tests/test-rebase-brute-force.t

  • This file was added.
$ cat >> $HGRCPATH <<EOF
> [extensions]
> drawdag=$TESTDIR/drawdag.py
> bruterebase=$TESTDIR/bruterebase.py
> [experimental]
> evolution=createmarkers,allowunstable
> EOF
$ init() {
> N=`expr ${N:-0} + 1`
> cd $TESTTMP && hg init repo$N && cd repo$N
> hg debugdrawdag
> }
Source looks like "N"
$ init <<'EOS'
> C D
> |\|
> A B Z
> EOS
$ hg debugbruterebase 'all()-Z' Z
A: A':Z
B: B':Z
AB: A':Z B':Z
C: ABORT: cannot use revision 3 as base, result would have 3 parents
AC: A':Z C':A'B
BC: B':Z C':B'A
ABC: A':Z B':Z C':A'B'
D: D':Z
AD: A':Z D':Z
BD: B':Z D':B'
ABD: A':Z B':Z D':B'
CD: CRASH: revlog index out of range
ACD: A':Z C':A'A' D':Z
BCD: B':Z C':B'A D':B'
ABCD: A':Z B':Z C':A'B' D':B'
Moving backwards
$ init <<'EOS'
> C
> |\
> A B
> |
> Z
> EOS
$ hg debugbruterebase 'all()-Z' Z
B: B':Z
A:
BA: B':Z
C: ABORT: cannot use revision 3 as base, result would have 3 parents
BC: B':Z C':B'A
AC:
BAC: ABORT: nothing to merge