This is an archive of the discontinued Mercurial Phabricator instance.

Differential D5070

run-tests: print number of tests and parallel process count
ClosedPublic

Authored by indygreg on Oct 13 2018, 6:34 AM.

Details

Reviewers
pulkit
Group Reviewers
hg-reviewers
Commits
rHG1039404c5e1d: run-tests: print number of tests and parallel process count
Summary

This seems like a useful output message to have.

I also sneak in a change to lower the parallel process count if it
is larger than the number of tests, as that makes no sense and output
saying we're running more tests in parallel than there exists tests would
be wonky.

Diff Detail

Repository
rHG Mercurial
Lint
Automatic diff as part of commit; lint not applicable.
Unit
Automatic diff as part of commit; unit tests not applicable.

Event Timeline

indygreg created this revision.Oct 13 2018, 6:34 AM
pulkit accepted this revision.Oct 13 2018, 7:45 AM
Closed by commit rHG1039404c5e1d: run-tests: print number of tests and parallel process count (authored by indygreg). · Explain WhyOct 13 2018, 8:07 AM
This revision was automatically updated to reflect the committed changes.

Revision Contents
Changeset List

PathPackages
Mtests/run-tests.py (7 lines)
Mtests/test-run-tests.t (71 lines)

Diff 12053

tests/run-tests.py

break break
testdescs.pop(0) testdescs.pop(0)
if not testdescs: if not testdescs:
print("running all tests") print("running all tests")
testdescs = orig testdescs = orig
tests = [self._gettest(d, i) for i, d in enumerate(testdescs)] tests = [self._gettest(d, i) for i, d in enumerate(testdescs)]
jobs = min(len(tests), self.options.jobs)
failed = False failed = False
kws = self.options.keywords kws = self.options.keywords
if kws is not None and PYTHON3: if kws is not None and PYTHON3:
kws = kws.encode('utf-8') kws = kws.encode('utf-8')
suite = TestSuite(self._testdir, suite = TestSuite(self._testdir,
jobs=self.options.jobs, jobs=jobs,
whitelist=self.options.whitelisted, whitelist=self.options.whitelisted,
blacklist=self.options.blacklist, blacklist=self.options.blacklist,
retest=self.options.retest, retest=self.options.retest,
keywords=kws, keywords=kws,
loop=self.options.loop, loop=self.options.loop,
runs_per_test=self.options.runs_per_test, runs_per_test=self.options.runs_per_test,
showchannels=self.options.showchannels, showchannels=self.options.showchannels,
tests=tests, loadtest=_reloadtest) tests=tests, loadtest=_reloadtest)
self._installhg() self._installhg()
self._checkhglib("Testing") self._checkhglib("Testing")
else: else:
self._usecorrectpython() self._usecorrectpython()
if self.options.chg: if self.options.chg:
assert self._installdir assert self._installdir
self._installchg() self._installchg()
log('running %d tests using %d parallel processes' % (
len(tests), jobs))
result = runner.run(suite) result = runner.run(suite)
if result.failures: if result.failures:
failed = True failed = True
result.onEnd() result.onEnd()
if self.options.anycoverage: if self.options.anycoverage:

tests/test-run-tests.t

This file tests the behavior of run-tests.py itself. This file tests the behavior of run-tests.py itself.
Avoid interference from actual test env: Avoid interference from actual test env:
$ . "$TESTDIR/helper-runtests.sh" $ . "$TESTDIR/helper-runtests.sh"
Smoke test with install Smoke test with install
============ ============
$ "$PYTHON" $TESTDIR/run-tests.py $HGTEST_RUN_TESTS_PURE -l $ "$PYTHON" $TESTDIR/run-tests.py $HGTEST_RUN_TESTS_PURE -l
running 0 tests using 0 parallel processes
# Ran 0 tests, 0 skipped, 0 failed. # Ran 0 tests, 0 skipped, 0 failed.
Define a helper to avoid the install step Define a helper to avoid the install step
============= =============
$ rt() $ rt()
> { > {
> "$PYTHON" $TESTDIR/run-tests.py --with-hg=`which hg` "$@" > "$PYTHON" $TESTDIR/run-tests.py --with-hg=`which hg` "$@"
> } > }
error paths error paths
#if symlink #if symlink
$ ln -s `which true` hg $ ln -s `which true` hg
$ "$PYTHON" $TESTDIR/run-tests.py --with-hg=./hg $ "$PYTHON" $TESTDIR/run-tests.py --with-hg=./hg
warning: --with-hg should specify an hg script warning: --with-hg should specify an hg script
running 0 tests using 0 parallel processes
# Ran 0 tests, 0 skipped, 0 failed. # Ran 0 tests, 0 skipped, 0 failed.
$ rm hg $ rm hg
#endif #endif
#if execbit #if execbit
$ touch hg $ touch hg
$ "$PYTHON" $TESTDIR/run-tests.py --with-hg=./hg $ "$PYTHON" $TESTDIR/run-tests.py --with-hg=./hg
> return False > return False
> EOF > EOF
an empty test an empty test
======================= =======================
$ touch test-empty.t $ touch test-empty.t
$ rt $ rt
running 1 tests using 1 parallel processes
. .
# Ran 1 tests, 0 skipped, 0 failed. # Ran 1 tests, 0 skipped, 0 failed.
$ rm test-empty.t $ rm test-empty.t
a succesful test a succesful test
======================= =======================
$ cat > test-success.t << EOF $ cat > test-success.t << EOF
> zyx (custom !) > zyx (custom !)
> wvu > wvu
> no_print (no-custom !) > no_print (no-custom !)
> tsr (no-missing !) > tsr (no-missing !)
> missing (missing !) > missing (missing !)
> EOF > EOF
$ rt $ rt
running 1 tests using 1 parallel processes
. .
# Ran 1 tests, 0 skipped, 0 failed. # Ran 1 tests, 0 skipped, 0 failed.
failing test failing test
================== ==================
test churn with globs test churn with globs
$ cat > test-failure.t <<EOF $ cat > test-failure.t <<EOF
> $ echo "bar-baz"; echo "bar-bad"; echo foo > $ echo "bar-baz"; echo "bar-bad"; echo foo
> bar*bad (glob) > bar*bad (glob)
> bar*baz (glob) > bar*baz (glob)
> | fo (re) > | fo (re)
> EOF > EOF
$ rt test-failure.t $ rt test-failure.t
running 1 tests using 1 parallel processes
--- $TESTTMP/test-failure.t --- $TESTTMP/test-failure.t
+++ $TESTTMP/test-failure.t.err +++ $TESTTMP/test-failure.t.err
@@ -1,4 +1,4 @@ @@ -1,4 +1,4 @@
$ echo "bar-baz"; echo "bar-bad"; echo foo $ echo "bar-baz"; echo "bar-bad"; echo foo
+ bar*baz (glob) + bar*baz (glob)
bar*bad (glob) bar*bad (glob)
- bar*baz (glob) - bar*baz (glob)
> key: 2 > key: 2
> value: b > value: b
> key: 3 > key: 3
> value: * (glob) > value: * (glob)
> key: 4 > key: 4
> value: * (glob) > value: * (glob)
> EOF > EOF
$ rt test-failure-globs.t $ rt test-failure-globs.t
running 1 tests using 1 parallel processes
--- $TESTTMP/test-failure-globs.t --- $TESTTMP/test-failure-globs.t
+++ $TESTTMP/test-failure-globs.t.err +++ $TESTTMP/test-failure-globs.t.err
@@ -2,9 +2,9 @@ @@ -2,9 +2,9 @@
context context
context context
key: 1 key: 1
- value: a - value: a
> The "missing" line should stay, even though awol is dropped > The "missing" line should stay, even though awol is dropped
> $ echo 'testing' > $ echo 'testing'
> test.ng (re) (true !) > test.ng (re) (true !)
> foo.ar (?) > foo.ar (?)
> awol > awol
> missing (?) > missing (?)
> EOF > EOF
$ rt test-failure.t $ rt test-failure.t
running 1 tests using 1 parallel processes
--- $TESTTMP/test-failure.t --- $TESTTMP/test-failure.t
+++ $TESTTMP/test-failure.t.err +++ $TESTTMP/test-failure.t.err
@@ -1,11 +1,9 @@ @@ -1,11 +1,9 @@
$ true $ true
- should go away (true !) - should go away (true !)
$ true $ true
should stay (false !) should stay (false !)
> pad pad pad pad............................................................ > pad pad pad pad............................................................
> EOF > EOF
>>> fh = open('test-failure-unicode.t', 'wb') >>> fh = open('test-failure-unicode.t', 'wb')
>>> fh.write(u' $ echo babar\u03b1\n'.encode('utf-8')) and None >>> fh.write(u' $ echo babar\u03b1\n'.encode('utf-8')) and None
>>> fh.write(u' l\u03b5\u03b5t\n'.encode('utf-8')) and None >>> fh.write(u' l\u03b5\u03b5t\n'.encode('utf-8')) and None
$ rt $ rt
running 3 tests using 1 parallel processes
--- $TESTTMP/test-failure.t --- $TESTTMP/test-failure.t
+++ $TESTTMP/test-failure.t.err +++ $TESTTMP/test-failure.t.err
@@ -1,5 +1,5 @@ @@ -1,5 +1,5 @@
$ echo babar $ echo babar
- rataxes - rataxes
+ babar + babar
This is a noop statement so that This is a noop statement so that
Failed test-failure-unicode.t: output changed Failed test-failure-unicode.t: output changed
# Ran 3 tests, 0 skipped, 2 failed. # Ran 3 tests, 0 skipped, 2 failed.
python hash seed: * (glob) python hash seed: * (glob)
[1] [1]
test --outputdir test --outputdir
$ mkdir output $ mkdir output
$ rt --outputdir output $ rt --outputdir output
running 3 tests using 1 parallel processes
--- $TESTTMP/test-failure.t --- $TESTTMP/test-failure.t
+++ $TESTTMP/output/test-failure.t.err +++ $TESTTMP/output/test-failure.t.err
@@ -1,5 +1,5 @@ @@ -1,5 +1,5 @@
$ echo babar $ echo babar
- rataxes - rataxes
+ babar + babar
This is a noop statement so that This is a noop statement so that
. .
.. ..
.testtimes .testtimes
test-failure-unicode.t.err test-failure-unicode.t.err
test-failure.t.err test-failure.t.err
test --xunit support test --xunit support
$ rt --xunit=xunit.xml $ rt --xunit=xunit.xml
running 3 tests using 1 parallel processes
--- $TESTTMP/test-failure.t --- $TESTTMP/test-failure.t
+++ $TESTTMP/test-failure.t.err +++ $TESTTMP/test-failure.t.err
@@ -1,5 +1,5 @@ @@ -1,5 +1,5 @@
$ echo babar $ echo babar
- rataxes - rataxes
+ babar + babar
This is a noop statement so that This is a noop statement so that
</testsuite> </testsuite>
$ rm test-failure-unicode.t $ rm test-failure-unicode.t
test for --retest test for --retest
==================== ====================
$ rt --retest $ rt --retest
running 2 tests using 1 parallel processes
--- $TESTTMP/test-failure.t --- $TESTTMP/test-failure.t
+++ $TESTTMP/test-failure.t.err +++ $TESTTMP/test-failure.t.err
@@ -1,5 +1,5 @@ @@ -1,5 +1,5 @@
$ echo babar $ echo babar
- rataxes - rataxes
+ babar + babar
This is a noop statement so that This is a noop statement so that
this test is still more bytes than success. this test is still more bytes than success.
pad pad pad pad............................................................ pad pad pad pad............................................................
ERROR: test-failure.t output changed ERROR: test-failure.t output changed
! !
Failed test-failure.t: output changed Failed test-failure.t: output changed
# Ran 2 tests, 1 skipped, 1 failed. # Ran 2 tests, 1 skipped, 1 failed.
python hash seed: * (glob) python hash seed: * (glob)
[1] [1]
--retest works with --outputdir --retest works with --outputdir
$ rm -r output $ rm -r output
$ mkdir output $ mkdir output
$ mv test-failure.t.err output $ mv test-failure.t.err output
$ rt --retest --outputdir output $ rt --retest --outputdir output
running 2 tests using 1 parallel processes
--- $TESTTMP/test-failure.t --- $TESTTMP/test-failure.t
+++ $TESTTMP/output/test-failure.t.err +++ $TESTTMP/output/test-failure.t.err
@@ -1,5 +1,5 @@ @@ -1,5 +1,5 @@
$ echo babar $ echo babar
- rataxes - rataxes
+ babar + babar
This is a noop statement so that This is a noop statement so that
this test is still more bytes than success. this test is still more bytes than success.
pad pad pad pad............................................................ pad pad pad pad............................................................
ERROR: test-failure.t output changed ERROR: test-failure.t output changed
! !
Failed test-failure.t: output changed Failed test-failure.t: output changed
# Ran 2 tests, 1 skipped, 1 failed. # Ran 2 tests, 1 skipped, 1 failed.
python hash seed: * (glob) python hash seed: * (glob)
[1] [1]
Selecting Tests To Run Selecting Tests To Run
====================== ======================
successful successful
$ rt test-success.t $ rt test-success.t
running 1 tests using 1 parallel processes
. .
# Ran 1 tests, 0 skipped, 0 failed. # Ran 1 tests, 0 skipped, 0 failed.
success w/ keyword success w/ keyword
$ rt -k xyzzy $ rt -k xyzzy
running 2 tests using 1 parallel processes
. .
# Ran 2 tests, 1 skipped, 0 failed. # Ran 2 tests, 1 skipped, 0 failed.
failed failed
$ rt test-failure.t $ rt test-failure.t
running 1 tests using 1 parallel processes
--- $TESTTMP/test-failure.t --- $TESTTMP/test-failure.t
+++ $TESTTMP/test-failure.t.err +++ $TESTTMP/test-failure.t.err
@@ -1,5 +1,5 @@ @@ -1,5 +1,5 @@
$ echo babar $ echo babar
- rataxes - rataxes
+ babar + babar
This is a noop statement so that This is a noop statement so that
this test is still more bytes than success. this test is still more bytes than success.
pad pad pad pad............................................................ pad pad pad pad............................................................
ERROR: test-failure.t output changed ERROR: test-failure.t output changed
! !
Failed test-failure.t: output changed Failed test-failure.t: output changed
# Ran 1 tests, 0 skipped, 1 failed. # Ran 1 tests, 0 skipped, 1 failed.
python hash seed: * (glob) python hash seed: * (glob)
[1] [1]
failure w/ keyword failure w/ keyword
$ rt -k rataxes $ rt -k rataxes
running 2 tests using 1 parallel processes
--- $TESTTMP/test-failure.t --- $TESTTMP/test-failure.t
+++ $TESTTMP/test-failure.t.err +++ $TESTTMP/test-failure.t.err
@@ -1,5 +1,5 @@ @@ -1,5 +1,5 @@
$ echo babar $ echo babar
- rataxes - rataxes
+ babar + babar
This is a noop statement so that This is a noop statement so that
Verify that when a process fails to start we show a useful message Verify that when a process fails to start we show a useful message
================================================================== ==================================================================
$ cat > test-serve-fail.t <<EOF $ cat > test-serve-fail.t <<EOF
> $ echo 'abort: child process failed to start blah' > $ echo 'abort: child process failed to start blah'
> EOF > EOF
$ rt test-serve-fail.t $ rt test-serve-fail.t
running 1 tests using 1 parallel processes
--- $TESTTMP/test-serve-fail.t --- $TESTTMP/test-serve-fail.t
+++ $TESTTMP/test-serve-fail.t.err +++ $TESTTMP/test-serve-fail.t.err
@@ -1* +1,2 @@ (glob) @@ -1* +1,2 @@ (glob)
$ echo 'abort: child process failed to start blah' $ echo 'abort: child process failed to start blah'
+ abort: child process failed to start blah + abort: child process failed to start blah
ERROR: test-serve-fail.t output changed ERROR: test-serve-fail.t output changed
$ HGRCPATH= hg init inuse $ HGRCPATH= hg init inuse
$ hg serve -R inuse -p $HGPORT -d --pid-file=blocks.pid $ hg serve -R inuse -p $HGPORT -d --pid-file=blocks.pid
$ cat blocks.pid >> $DAEMON_PIDS $ cat blocks.pid >> $DAEMON_PIDS
$ cat > test-serve-inuse.t <<EOF $ cat > test-serve-inuse.t <<EOF
> $ hg serve -R `pwd`/inuse -p \$HGPORT -d --pid-file=hg.pid > $ hg serve -R `pwd`/inuse -p \$HGPORT -d --pid-file=hg.pid
> $ cat hg.pid >> \$DAEMON_PIDS > $ cat hg.pid >> \$DAEMON_PIDS
> EOF > EOF
$ rt test-serve-inuse.t $ rt test-serve-inuse.t
running 1 tests using 1 parallel processes
. .
# Ran 1 tests, 0 skipped, 0 failed. # Ran 1 tests, 0 skipped, 0 failed.
$ rm test-serve-inuse.t $ rm test-serve-inuse.t
$ killdaemons.py $DAEMON_PIDS $ killdaemons.py $DAEMON_PIDS
Running In Debug Mode Running In Debug Mode
====================== ======================
$ rt --debug 2>&1 | grep -v pwd $ rt --debug 2>&1 | grep -v pwd
running 2 tests using 1 parallel processes
+ echo *SALT* 0 0 (glob) + echo *SALT* 0 0 (glob)
*SALT* 0 0 (glob) *SALT* 0 0 (glob)
+ echo babar + echo babar
babar babar
+ echo *SALT* 10 0 (glob) + echo *SALT* 10 0 (glob)
*SALT* 10 0 (glob) *SALT* 10 0 (glob)
*+ echo *SALT* 0 0 (glob) *+ echo *SALT* 0 0 (glob)
*SALT* 0 0 (glob) *SALT* 0 0 (glob)
Parallel runs Parallel runs
============== ==============
(duplicate the failing test to get predictable output) (duplicate the failing test to get predictable output)
$ cp test-failure.t test-failure-copy.t $ cp test-failure.t test-failure-copy.t
$ rt --jobs 2 test-failure*.t -n $ rt --jobs 2 test-failure*.t -n
running 2 tests using 2 parallel processes
!! !!
Failed test-failure*.t: output changed (glob) Failed test-failure*.t: output changed (glob)
Failed test-failure*.t: output changed (glob) Failed test-failure*.t: output changed (glob)
# Ran 2 tests, 0 skipped, 2 failed. # Ran 2 tests, 0 skipped, 2 failed.
python hash seed: * (glob) python hash seed: * (glob)
[1] [1]
failures in parallel with --first should only print one failure failures in parallel with --first should only print one failure
$ rt --jobs 2 --first test-failure*.t $ rt --jobs 2 --first test-failure*.t
running 2 tests using 2 parallel processes
--- $TESTTMP/test-failure*.t (glob) --- $TESTTMP/test-failure*.t (glob)
+++ $TESTTMP/test-failure*.t.err (glob) +++ $TESTTMP/test-failure*.t.err (glob)
@@ -1,5 +1,5 @@ @@ -1,5 +1,5 @@
$ echo babar $ echo babar
- rataxes - rataxes
+ babar + babar
This is a noop statement so that This is a noop statement so that
=============== ===============
(backup the failing test) (backup the failing test)
$ cp test-failure.t backup $ cp test-failure.t backup
Refuse the fix Refuse the fix
$ echo 'n' | rt -i $ echo 'n' | rt -i
running 2 tests using 1 parallel processes
--- $TESTTMP/test-failure.t --- $TESTTMP/test-failure.t
+++ $TESTTMP/test-failure.t.err +++ $TESTTMP/test-failure.t.err
@@ -1,5 +1,5 @@ @@ -1,5 +1,5 @@
$ echo babar $ echo babar
- rataxes - rataxes
+ babar + babar
This is a noop statement so that This is a noop statement so that
pad pad pad pad............................................................ pad pad pad pad............................................................
pad pad pad pad............................................................ pad pad pad pad............................................................
pad pad pad pad............................................................ pad pad pad pad............................................................
pad pad pad pad............................................................ pad pad pad pad............................................................
Interactive with custom view Interactive with custom view
$ echo 'n' | rt -i --view echo $ echo 'n' | rt -i --view echo
running 2 tests using 1 parallel processes
$TESTTMP/test-failure.t $TESTTMP/test-failure.t.err $TESTTMP/test-failure.t $TESTTMP/test-failure.t.err
Accept this change? [n]* (glob) Accept this change? [n]* (glob)
ERROR: test-failure.t output changed ERROR: test-failure.t output changed
!. !.
Failed test-failure.t: output changed Failed test-failure.t: output changed
# Ran 2 tests, 0 skipped, 1 failed. # Ran 2 tests, 0 skipped, 1 failed.
python hash seed: * (glob) python hash seed: * (glob)
[1] [1]
View the fix View the fix
$ echo 'y' | rt --view echo $ echo 'y' | rt --view echo
running 2 tests using 1 parallel processes
$TESTTMP/test-failure.t $TESTTMP/test-failure.t.err $TESTTMP/test-failure.t $TESTTMP/test-failure.t.err
ERROR: test-failure.t output changed ERROR: test-failure.t output changed
!. !.
Failed test-failure.t: output changed Failed test-failure.t: output changed
# Ran 2 tests, 0 skipped, 1 failed. # Ran 2 tests, 0 skipped, 1 failed.
python hash seed: * (glob) python hash seed: * (glob)
[1] [1]
Accept the fix Accept the fix
$ cat >> test-failure.t <<EOF $ cat >> test-failure.t <<EOF
> $ echo 'saved backup bundle to \$TESTTMP/foo.hg' > $ echo 'saved backup bundle to \$TESTTMP/foo.hg'
> saved backup bundle to \$TESTTMP/foo.hg > saved backup bundle to \$TESTTMP/foo.hg
> $ echo 'saved backup bundle to \$TESTTMP/foo.hg' > $ echo 'saved backup bundle to \$TESTTMP/foo.hg'
> saved backup bundle to $TESTTMP\\foo.hg > saved backup bundle to $TESTTMP\\foo.hg
> $ echo 'saved backup bundle to \$TESTTMP/foo.hg' > $ echo 'saved backup bundle to \$TESTTMP/foo.hg'
> saved backup bundle to \$TESTTMP/*.hg (glob) > saved backup bundle to \$TESTTMP/*.hg (glob)
> EOF > EOF
$ echo 'y' | rt -i 2>&1 $ echo 'y' | rt -i 2>&1
running 2 tests using 1 parallel processes
--- $TESTTMP/test-failure.t --- $TESTTMP/test-failure.t
+++ $TESTTMP/test-failure.t.err +++ $TESTTMP/test-failure.t.err
@@ -1,5 +1,5 @@ @@ -1,5 +1,5 @@
$ echo babar $ echo babar
- rataxes - rataxes
+ babar + babar
This is a noop statement so that This is a noop statement so that
$ TESTRACEDIR=`pwd` $ TESTRACEDIR=`pwd`
$ export TESTRACEDIR $ export TESTRACEDIR
$ cat > test-race.t <<EOF $ cat > test-race.t <<EOF
> $ echo 1 > $ echo 1
> $ echo "# a new line" >> $TESTRACEDIR/test-race.t > $ echo "# a new line" >> $TESTRACEDIR/test-race.t
> EOF > EOF
$ rt -i test-race.t $ rt -i test-race.t
running 1 tests using 1 parallel processes
--- $TESTTMP/test-race.t --- $TESTTMP/test-race.t
+++ $TESTTMP/test-race.t.err +++ $TESTTMP/test-race.t.err
@@ -1,2 +1,3 @@ @@ -1,2 +1,3 @@
$ echo 1 $ echo 1
+ 1 + 1
$ echo "# a new line" >> $TESTTMP/test-race.t $ echo "# a new line" >> $TESTTMP/test-race.t
Reference output has changed (run again to prompt changes) Reference output has changed (run again to prompt changes)
> $ echo 2 > $ echo 2
> #endif > #endif
> EOF > EOF
$ cat <<EOF | rt -i test-cases.t 2>&1 $ cat <<EOF | rt -i test-cases.t 2>&1
> y > y
> y > y
> EOF > EOF
running 2 tests using 1 parallel processes
--- $TESTTMP/test-cases.t --- $TESTTMP/test-cases.t
+++ $TESTTMP/test-cases.t#a.err +++ $TESTTMP/test-cases.t#a.err
@@ -1,6 +1,7 @@ @@ -1,6 +1,7 @@
#testcases a b #testcases a b
#if a #if a
$ echo 1 $ echo 1
+ 1 + 1
> #else > #else
> $ NAME=B > $ NAME=B
> #endif > #endif
> $ echo $NAME > $ echo $NAME
> A (a !) > A (a !)
> B (b !) > B (b !)
> EOF > EOF
$ rt test-cases.t $ rt test-cases.t
running 2 tests using 1 parallel processes
.. ..
# Ran 2 tests, 0 skipped, 0 failed. # Ran 2 tests, 0 skipped, 0 failed.
When using multiple dimensions of "#testcases" in .t files When using multiple dimensions of "#testcases" in .t files
$ cat > test-cases.t <<'EOF' $ cat > test-cases.t <<'EOF'
> #testcases a b > #testcases a b
> #testcases c d > #testcases c d
> #if a d > #if a d
> $ echo $TESTCASE > $ echo $TESTCASE
> a#d > a#d
> #endif > #endif
> #if b c > #if b c
> $ echo yes > $ echo yes
> no > no
> #endif > #endif
> EOF > EOF
$ rt test-cases.t $ rt test-cases.t
running 4 tests using 1 parallel processes
.. ..
--- $TESTTMP/test-cases.t --- $TESTTMP/test-cases.t
+++ $TESTTMP/test-cases.t#b#c.err +++ $TESTTMP/test-cases.t#b#c.err
@@ -6,5 +6,5 @@ @@ -6,5 +6,5 @@
#endif #endif
#if b c #if b c
$ echo yes $ echo yes
- no - no
(reinstall) (reinstall)
$ mv backup test-failure.t $ mv backup test-failure.t
No Diff No Diff
=============== ===============
$ rt --nodiff $ rt --nodiff
running 2 tests using 1 parallel processes
!. !.
Failed test-failure.t: output changed Failed test-failure.t: output changed
# Ran 2 tests, 0 skipped, 1 failed. # Ran 2 tests, 0 skipped, 1 failed.
python hash seed: * (glob) python hash seed: * (glob)
[1] [1]
test --tmpdir support test --tmpdir support
$ rt --tmpdir=$TESTTMP/keep test-success.t $ rt --tmpdir=$TESTTMP/keep test-success.t
running 1 tests using 1 parallel processes
Keeping testtmp dir: $TESTTMP/keep/child1/test-success.t Keeping testtmp dir: $TESTTMP/keep/child1/test-success.t
Keeping threadtmp dir: $TESTTMP/keep/child1 Keeping threadtmp dir: $TESTTMP/keep/child1
. .
# Ran 1 tests, 0 skipped, 0 failed. # Ran 1 tests, 0 skipped, 0 failed.
timeouts timeouts
======== ========
$ cat > test-timeout.t <<EOF $ cat > test-timeout.t <<EOF
> $ sleep 2 > $ sleep 2
> $ echo pass > $ echo pass
> pass > pass
> EOF > EOF
> echo '#require slow' > test-slow-timeout.t > echo '#require slow' > test-slow-timeout.t
> cat test-timeout.t >> test-slow-timeout.t > cat test-timeout.t >> test-slow-timeout.t
$ rt --timeout=1 --slowtimeout=3 test-timeout.t test-slow-timeout.t $ rt --timeout=1 --slowtimeout=3 test-timeout.t test-slow-timeout.t
running 2 tests using 1 parallel processes
st st
Skipped test-slow-timeout.t: missing feature: allow slow tests (use --allow-slow-tests) Skipped test-slow-timeout.t: missing feature: allow slow tests (use --allow-slow-tests)
Failed test-timeout.t: timed out Failed test-timeout.t: timed out
# Ran 1 tests, 1 skipped, 1 failed. # Ran 1 tests, 1 skipped, 1 failed.
python hash seed: * (glob) python hash seed: * (glob)
[1] [1]
$ rt --timeout=1 --slowtimeout=3 \ $ rt --timeout=1 --slowtimeout=3 \
> test-timeout.t test-slow-timeout.t --allow-slow-tests > test-timeout.t test-slow-timeout.t --allow-slow-tests
running 2 tests using 1 parallel processes
.t .t
Failed test-timeout.t: timed out Failed test-timeout.t: timed out
# Ran 2 tests, 0 skipped, 1 failed. # Ran 2 tests, 0 skipped, 1 failed.
python hash seed: * (glob) python hash seed: * (glob)
[1] [1]
$ rm test-timeout.t test-slow-timeout.t $ rm test-timeout.t test-slow-timeout.t
test for --time test for --time
================== ==================
$ rt test-success.t --time $ rt test-success.t --time
running 1 tests using 1 parallel processes
. .
# Ran 1 tests, 0 skipped, 0 failed. # Ran 1 tests, 0 skipped, 0 failed.
# Producing time report # Producing time report
start end cuser csys real Test start end cuser csys real Test
\s*[\d\.]{5} \s*[\d\.]{5} \s*[\d\.]{5} \s*[\d\.]{5} \s*[\d\.]{5} test-success.t (re) \s*[\d\.]{5} \s*[\d\.]{5} \s*[\d\.]{5} \s*[\d\.]{5} \s*[\d\.]{5} test-success.t (re)
test for --time with --job enabled test for --time with --job enabled
==================================== ====================================
$ rt test-success.t --time --jobs 2 $ rt test-success.t --time --jobs 2
running 1 tests using 1 parallel processes
. .
# Ran 1 tests, 0 skipped, 0 failed. # Ran 1 tests, 0 skipped, 0 failed.
# Producing time report # Producing time report
start end cuser csys real Test start end cuser csys real Test
\s*[\d\.]{5} \s*[\d\.]{5} \s*[\d\.]{5} \s*[\d\.]{5} \s*[\d\.]{5} test-success.t (re) \s*[\d\.]{5} \s*[\d\.]{5} \s*[\d\.]{5} \s*[\d\.]{5} \s*[\d\.]{5} test-success.t (re)
Skips Skips
================ ================
$ cat > test-skip.t <<EOF $ cat > test-skip.t <<EOF
> $ echo xyzzy > $ echo xyzzy
> #if true > #if true
> #require false > #require false
> #end > #end
> EOF > EOF
$ cat > test-noskip.t <<EOF $ cat > test-noskip.t <<EOF
> #if false > #if false
> #require false > #require false
> #endif > #endif
> EOF > EOF
$ rt --nodiff $ rt --nodiff
running 4 tests using 1 parallel processes
!.s. !.s.
Skipped test-skip.t: missing feature: nail clipper Skipped test-skip.t: missing feature: nail clipper
Failed test-failure.t: output changed Failed test-failure.t: output changed
# Ran 3 tests, 1 skipped, 1 failed. # Ran 3 tests, 1 skipped, 1 failed.
python hash seed: * (glob) python hash seed: * (glob)
[1] [1]
$ rm test-noskip.t $ rm test-noskip.t
$ rt --keyword xyzzy $ rt --keyword xyzzy
running 3 tests using 1 parallel processes
.s .s
Skipped test-skip.t: missing feature: nail clipper Skipped test-skip.t: missing feature: nail clipper
# Ran 2 tests, 2 skipped, 0 failed. # Ran 2 tests, 2 skipped, 0 failed.
Skips with xml Skips with xml
$ rt --keyword xyzzy \ $ rt --keyword xyzzy \
> --xunit=xunit.xml > --xunit=xunit.xml
running 3 tests using 1 parallel processes
.s .s
Skipped test-skip.t: missing feature: nail clipper Skipped test-skip.t: missing feature: nail clipper
# Ran 2 tests, 2 skipped, 0 failed. # Ran 2 tests, 2 skipped, 0 failed.
$ cat xunit.xml $ cat xunit.xml
<?xml version="1.0" encoding="utf-8"?> <?xml version="1.0" encoding="utf-8"?>
<testsuite errors="0" failures="0" name="run-tests" skipped="2" tests="2"> <testsuite errors="0" failures="0" name="run-tests" skipped="2" tests="2">
<testcase name="test-success.t" time="*"/> (glob) <testcase name="test-success.t" time="*"/> (glob)
<testcase name="test-skip.t"> <testcase name="test-skip.t">
<skipped> <skipped>
<![CDATA[missing feature: nail clipper]]> </skipped> <![CDATA[missing feature: nail clipper]]> </skipped>
</testcase> </testcase>
</testsuite> </testsuite>
Missing skips or blacklisted skips don't count as executed: Missing skips or blacklisted skips don't count as executed:
$ echo test-failure.t > blacklist $ echo test-failure.t > blacklist
$ rt --blacklist=blacklist --json\ $ rt --blacklist=blacklist --json\
> test-failure.t test-bogus.t > test-failure.t test-bogus.t
running 2 tests using 1 parallel processes
ss ss
Skipped test-bogus.t: Doesn't exist Skipped test-bogus.t: Doesn't exist
Skipped test-failure.t: blacklisted Skipped test-failure.t: blacklisted
# Ran 0 tests, 2 skipped, 0 failed. # Ran 0 tests, 2 skipped, 0 failed.
$ cat report.json $ cat report.json
testreport ={ testreport ={
"test-bogus.t": { "test-bogus.t": {
"result": "skip" "result": "skip"
}, },
"test-failure.t": { "test-failure.t": {
"result": "skip" "result": "skip"
} }
} (no-eol) } (no-eol)
Whitelist trumps blacklist Whitelist trumps blacklist
$ echo test-failure.t > whitelist $ echo test-failure.t > whitelist
$ rt --blacklist=blacklist --whitelist=whitelist --json\ $ rt --blacklist=blacklist --whitelist=whitelist --json\
> test-failure.t test-bogus.t > test-failure.t test-bogus.t
running 2 tests using 1 parallel processes
s s
--- $TESTTMP/test-failure.t --- $TESTTMP/test-failure.t
+++ $TESTTMP/test-failure.t.err +++ $TESTTMP/test-failure.t.err
@@ -1,5 +1,5 @@ @@ -1,5 +1,5 @@
$ echo babar $ echo babar
- rataxes - rataxes
+ babar + babar
This is a noop statement so that This is a noop statement so that
this test is still more bytes than success. this test is still more bytes than success.
pad pad pad pad............................................................ pad pad pad pad............................................................
ERROR: test-failure.t output changed ERROR: test-failure.t output changed
! !
Skipped test-bogus.t: Doesn't exist Skipped test-bogus.t: Doesn't exist
Failed test-failure.t: output changed Failed test-failure.t: output changed
# Ran 1 tests, 1 skipped, 1 failed. # Ran 1 tests, 1 skipped, 1 failed.
python hash seed: * (glob) python hash seed: * (glob)
[1] [1]
Ensure that --test-list causes only the tests listed in that file to Ensure that --test-list causes only the tests listed in that file to
be executed. be executed.
$ echo test-success.t >> onlytest $ echo test-success.t >> onlytest
$ rt --test-list=onlytest $ rt --test-list=onlytest
running 1 tests using 1 parallel processes
. .
# Ran 1 tests, 0 skipped, 0 failed. # Ran 1 tests, 0 skipped, 0 failed.
$ echo test-bogus.t >> anothertest $ echo test-bogus.t >> anothertest
$ rt --test-list=onlytest --test-list=anothertest $ rt --test-list=onlytest --test-list=anothertest
running 2 tests using 1 parallel processes
s. s.
Skipped test-bogus.t: Doesn't exist Skipped test-bogus.t: Doesn't exist
# Ran 1 tests, 1 skipped, 0 failed. # Ran 1 tests, 1 skipped, 0 failed.
$ rm onlytest anothertest $ rm onlytest anothertest
test for --json test for --json
================== ==================
$ rt --json $ rt --json
running 3 tests using 1 parallel processes
--- $TESTTMP/test-failure.t --- $TESTTMP/test-failure.t
+++ $TESTTMP/test-failure.t.err +++ $TESTTMP/test-failure.t.err
@@ -1,5 +1,5 @@ @@ -1,5 +1,5 @@
$ echo babar $ echo babar
- rataxes - rataxes
+ babar + babar
This is a noop statement so that This is a noop statement so that
} }
} (no-eol) } (no-eol)
--json with --outputdir --json with --outputdir
$ rm report.json $ rm report.json
$ rm -r output $ rm -r output
$ mkdir output $ mkdir output
$ rt --json --outputdir output $ rt --json --outputdir output
running 3 tests using 1 parallel processes
--- $TESTTMP/test-failure.t --- $TESTTMP/test-failure.t
+++ $TESTTMP/output/test-failure.t.err +++ $TESTTMP/output/test-failure.t.err
@@ -1,5 +1,5 @@ @@ -1,5 +1,5 @@
$ echo babar $ echo babar
- rataxes - rataxes
+ babar + babar
This is a noop statement so that This is a noop statement so that
.testtimes .testtimes
report.json report.json
test-failure.t.err test-failure.t.err
Test that failed test accepted through interactive are properly reported: Test that failed test accepted through interactive are properly reported:
$ cp test-failure.t backup $ cp test-failure.t backup
$ echo y | rt --json -i $ echo y | rt --json -i
running 3 tests using 1 parallel processes
--- $TESTTMP/test-failure.t --- $TESTTMP/test-failure.t
+++ $TESTTMP/test-failure.t.err +++ $TESTTMP/test-failure.t.err
@@ -1,5 +1,5 @@ @@ -1,5 +1,5 @@
$ echo babar $ echo babar
- rataxes - rataxes
+ babar + babar
This is a noop statement so that This is a noop statement so that
backslash on end of line with glob matching is handled properly backslash on end of line with glob matching is handled properly
$ cat > test-glob-backslash.t << EOF $ cat > test-glob-backslash.t << EOF
> $ echo 'foo bar \\' > $ echo 'foo bar \\'
> foo * \ (glob) > foo * \ (glob)
> EOF > EOF
$ rt test-glob-backslash.t $ rt test-glob-backslash.t
running 1 tests using 1 parallel processes
. .
# Ran 1 tests, 0 skipped, 0 failed. # Ran 1 tests, 0 skipped, 0 failed.
$ rm -f test-glob-backslash.t $ rm -f test-glob-backslash.t
Test globbing of local IP addresses Test globbing of local IP addresses
$ echo 172.16.18.1 $ echo 172.16.18.1
$LOCALIP (glob) $LOCALIP (glob)
$ echo dead:beef::1 $ echo dead:beef::1
$LOCALIP (glob) $LOCALIP (glob)
Add support for external test formatter Add support for external test formatter
======================================= =======================================
$ CUSTOM_TEST_RESULT=basic_test_result "$PYTHON" $TESTDIR/run-tests.py --with-hg=`which hg` "$@" test-success.t test-failure.t $ CUSTOM_TEST_RESULT=basic_test_result "$PYTHON" $TESTDIR/run-tests.py --with-hg=`which hg` "$@" test-success.t test-failure.t
running 2 tests using 1 parallel processes
# Ran 2 tests, 0 skipped, 0 failed. # Ran 2 tests, 0 skipped, 0 failed.
ON_START! <__main__.TestSuite tests=[<__main__.TTest testMethod=test-failure.t>, <__main__.TTest testMethod=test-success.t>]> ON_START! <__main__.TestSuite tests=[<__main__.TTest testMethod=test-failure.t>, <__main__.TTest testMethod=test-success.t>]>
FAILURE! test-failure.t output changed FAILURE! test-failure.t output changed
SUCCESS! test-success.t SUCCESS! test-success.t
ON_END! ON_END!
Test reusability for third party tools Test reusability for third party tools
====================================== ======================================
$ mkdir "$TESTTMP"/anothertests $ mkdir "$TESTTMP"/anothertests
$ cd "$TESTTMP"/anothertests $ cd "$TESTTMP"/anothertests
test that `run-tests.py` can execute hghave, even if it runs not in test that `run-tests.py` can execute hghave, even if it runs not in
Mercurial source tree. Mercurial source tree.
$ cat > test-hghave.t <<EOF $ cat > test-hghave.t <<EOF
> #require true > #require true
> $ echo foo > $ echo foo
> foo > foo
> EOF > EOF
$ rt test-hghave.t $ rt test-hghave.t
running 1 tests using 1 parallel processes
. .
# Ran 1 tests, 0 skipped, 0 failed. # Ran 1 tests, 0 skipped, 0 failed.
test that RUNTESTDIR refers the directory, in which `run-tests.py` now test that RUNTESTDIR refers the directory, in which `run-tests.py` now
running is placed. running is placed.
$ cat > test-runtestdir.t <<EOF $ cat > test-runtestdir.t <<EOF
> - $TESTDIR, in which test-run-tests.t is placed > - $TESTDIR, in which test-run-tests.t is placed
> previous check failed, that means we found a copy of check-code at whatever > previous check failed, that means we found a copy of check-code at whatever
> RUNTESTSDIR ended up containing, even though it doesn't match TESTDIR. > RUNTESTSDIR ended up containing, even though it doesn't match TESTDIR.
> $ head -n 3 "\$RUNTESTDIR"/../contrib/check-code.py | sed 's@.!.*python@#!USRBINENVPY@' > $ head -n 3 "\$RUNTESTDIR"/../contrib/check-code.py | sed 's@.!.*python@#!USRBINENVPY@'
> #!USRBINENVPY > #!USRBINENVPY
> # > #
> # check-code - a style and portability checker for Mercurial > # check-code - a style and portability checker for Mercurial
> EOF > EOF
$ rt test-runtestdir.t $ rt test-runtestdir.t
running 1 tests using 1 parallel processes
. .
# Ran 1 tests, 0 skipped, 0 failed. # Ran 1 tests, 0 skipped, 0 failed.
#if execbit #if execbit
test that TESTDIR is referred in PATH test that TESTDIR is referred in PATH
$ cat > custom-command.sh <<EOF $ cat > custom-command.sh <<EOF
> #!/bin/sh > #!/bin/sh
> echo "hello world" > echo "hello world"
> EOF > EOF
$ chmod +x custom-command.sh $ chmod +x custom-command.sh
$ cat > test-testdir-path.t <<EOF $ cat > test-testdir-path.t <<EOF
> $ custom-command.sh > $ custom-command.sh
> hello world > hello world
> EOF > EOF
$ rt test-testdir-path.t $ rt test-testdir-path.t
running 1 tests using 1 parallel processes
. .
# Ran 1 tests, 0 skipped, 0 failed. # Ran 1 tests, 0 skipped, 0 failed.
#endif #endif
test support for --allow-slow-tests test support for --allow-slow-tests
$ cat > test-very-slow-test.t <<EOF $ cat > test-very-slow-test.t <<EOF
> #require slow > #require slow
> $ echo pass > $ echo pass
> pass > pass
> EOF > EOF
$ rt test-very-slow-test.t $ rt test-very-slow-test.t
running 1 tests using 1 parallel processes
s s
Skipped test-very-slow-test.t: missing feature: allow slow tests (use --allow-slow-tests) Skipped test-very-slow-test.t: missing feature: allow slow tests (use --allow-slow-tests)
# Ran 0 tests, 1 skipped, 0 failed. # Ran 0 tests, 1 skipped, 0 failed.
$ rt $HGTEST_RUN_TESTS_PURE --allow-slow-tests test-very-slow-test.t $ rt $HGTEST_RUN_TESTS_PURE --allow-slow-tests test-very-slow-test.t
running 1 tests using 1 parallel processes
. .
# Ran 1 tests, 0 skipped, 0 failed. # Ran 1 tests, 0 skipped, 0 failed.
support for running a test outside the current directory support for running a test outside the current directory
$ mkdir nonlocal $ mkdir nonlocal
$ cat > nonlocal/test-is-not-here.t << EOF $ cat > nonlocal/test-is-not-here.t << EOF
> $ echo pass > $ echo pass
> pass > pass
> EOF > EOF
$ rt nonlocal/test-is-not-here.t $ rt nonlocal/test-is-not-here.t
running 1 tests using 1 parallel processes
. .
# Ran 1 tests, 0 skipped, 0 failed. # Ran 1 tests, 0 skipped, 0 failed.
support for automatically discovering test if arg is a folder support for automatically discovering test if arg is a folder
$ mkdir tmp && cd tmp $ mkdir tmp && cd tmp
$ cat > test-uno.t << EOF $ cat > test-uno.t << EOF
> $ echo line > $ echo line
> line > line
> EOF > EOF
$ cp test-uno.t test-dos.t $ cp test-uno.t test-dos.t
$ cd .. $ cd ..
$ cp -R tmp tmpp $ cp -R tmp tmpp
$ cp tmp/test-uno.t test-solo.t $ cp tmp/test-uno.t test-solo.t
$ rt tmp/ test-solo.t tmpp $ rt tmp/ test-solo.t tmpp
running 5 tests using 1 parallel processes
..... .....
# Ran 5 tests, 0 skipped, 0 failed. # Ran 5 tests, 0 skipped, 0 failed.
$ rm -rf tmp tmpp $ rm -rf tmp tmpp
support for running run-tests.py from another directory support for running run-tests.py from another directory
$ mkdir tmp && cd tmp $ mkdir tmp && cd tmp
$ cat > useful-file.sh << EOF $ cat > useful-file.sh << EOF
> important command > important command
> EOF > EOF
$ cat > test-folder.t << EOF $ cat > test-folder.t << EOF
> $ cat \$TESTDIR/useful-file.sh > $ cat \$TESTDIR/useful-file.sh
> important command > important command
> EOF > EOF
$ cat > test-folder-fail.t << EOF $ cat > test-folder-fail.t << EOF
> $ cat \$TESTDIR/useful-file.sh > $ cat \$TESTDIR/useful-file.sh
> important commando > important commando
> EOF > EOF
$ cd .. $ cd ..
$ rt tmp/test-*.t $ rt tmp/test-*.t
running 2 tests using 1 parallel processes
--- $TESTTMP/anothertests/tmp/test-folder-fail.t --- $TESTTMP/anothertests/tmp/test-folder-fail.t
+++ $TESTTMP/anothertests/tmp/test-folder-fail.t.err +++ $TESTTMP/anothertests/tmp/test-folder-fail.t.err
@@ -1,2 +1,2 @@ @@ -1,2 +1,2 @@
$ cat $TESTDIR/useful-file.sh $ cat $TESTDIR/useful-file.sh
- important commando - important commando
+ important command + important command
$ hg add test-bisect.t $ hg add test-bisect.t
$ hg ci -m 'good' $ hg ci -m 'good'
$ cat >> test-bisect.t <<EOF $ cat >> test-bisect.t <<EOF
> $ echo pass > $ echo pass
> fail > fail
> EOF > EOF
$ hg ci -m 'bad' $ hg ci -m 'bad'
$ rt --known-good-rev=0 test-bisect.t $ rt --known-good-rev=0 test-bisect.t
running 1 tests using 1 parallel processes
--- $TESTTMP/anothertests/bisect/test-bisect.t --- $TESTTMP/anothertests/bisect/test-bisect.t
+++ $TESTTMP/anothertests/bisect/test-bisect.t.err +++ $TESTTMP/anothertests/bisect/test-bisect.t.err
@@ -1,4 +1,4 @@ @@ -1,4 +1,4 @@
$ echo pass $ echo pass
pass pass
$ echo pass $ echo pass
- fail - fail
$ cd bisect-dependent $ cd bisect-dependent
$ cat > test-bisect-dependent.t <<EOF $ cat > test-bisect-dependent.t <<EOF
> $ tail -1 \$TESTDIR/../bisect/test-bisect.t > $ tail -1 \$TESTDIR/../bisect/test-bisect.t
> pass > pass
> EOF > EOF
$ hg commit -Am dependent test-bisect-dependent.t $ hg commit -Am dependent test-bisect-dependent.t
$ rt --known-good-rev=0 test-bisect-dependent.t $ rt --known-good-rev=0 test-bisect-dependent.t
running 1 tests using 1 parallel processes
--- $TESTTMP/anothertests/bisect-dependent/test-bisect-dependent.t --- $TESTTMP/anothertests/bisect-dependent/test-bisect-dependent.t
+++ $TESTTMP/anothertests/bisect-dependent/test-bisect-dependent.t.err +++ $TESTTMP/anothertests/bisect-dependent/test-bisect-dependent.t.err
@@ -1,2 +1,2 @@ @@ -1,2 +1,2 @@
$ tail -1 $TESTDIR/../bisect/test-bisect.t $ tail -1 $TESTDIR/../bisect/test-bisect.t
- pass - pass
+ fail + fail
ERROR: test-bisect-dependent.t output changed ERROR: test-bisect-dependent.t output changed
! !
Failed test-bisect-dependent.t: output changed Failed test-bisect-dependent.t: output changed
Failed to identify failure point for test-bisect-dependent.t Failed to identify failure point for test-bisect-dependent.t
# Ran 1 tests, 0 skipped, 1 failed. # Ran 1 tests, 0 skipped, 1 failed.
python hash seed: * (glob) python hash seed: * (glob)
[1] [1]
$ rt --bisect-repo=../test-bisect test-bisect-dependent.t $ rt --bisect-repo=../test-bisect test-bisect-dependent.t
usage: run-tests.py [options] [tests] usage: run-tests.py [options] [tests]
run-tests.py: error: --bisect-repo cannot be used without --known-good-rev run-tests.py: error: --bisect-repo cannot be used without --known-good-rev
[2] [2]
$ rt --known-good-rev=0 --bisect-repo=../bisect test-bisect-dependent.t $ rt --known-good-rev=0 --bisect-repo=../bisect test-bisect-dependent.t
running 1 tests using 1 parallel processes
--- $TESTTMP/anothertests/bisect-dependent/test-bisect-dependent.t --- $TESTTMP/anothertests/bisect-dependent/test-bisect-dependent.t
+++ $TESTTMP/anothertests/bisect-dependent/test-bisect-dependent.t.err +++ $TESTTMP/anothertests/bisect-dependent/test-bisect-dependent.t.err
@@ -1,2 +1,2 @@ @@ -1,2 +1,2 @@
$ tail -1 $TESTDIR/../bisect/test-bisect.t $ tail -1 $TESTDIR/../bisect/test-bisect.t
- pass - pass
+ fail + fail
> EOF > EOF
$ for f in 1 2 3 4 ; do $ for f in 1 2 3 4 ; do
> cat > test-works-$f.t <<EOF > cat > test-works-$f.t <<EOF
> This is test case $f > This is test case $f
> $ sleep 1 > $ sleep 1
> EOF > EOF
> done > done
$ rt -j 2 $ rt -j 2
running 5 tests using 2 parallel processes
.... ....
# Ran 5 tests, 0 skipped, 0 failed. # Ran 5 tests, 0 skipped, 0 failed.
skipped: unknown feature: notarealhghavefeature skipped: unknown feature: notarealhghavefeature
$ cd .. $ cd ..
$ rm -rf broken $ rm -rf broken
Test cases in .t files Test cases in .t files
> #if no-C > #if no-C
> $ [ $V = C ] > $ [ $V = C ]
> [1] > [1]
> #endif > #endif
> $ [ $V = D ] > $ [ $V = D ]
> [1] > [1]
> EOF > EOF
$ rt $ rt
running 3 tests using 1 parallel processes
. .
--- $TESTTMP/anothertests/cases/test-cases-abc.t --- $TESTTMP/anothertests/cases/test-cases-abc.t
+++ $TESTTMP/anothertests/cases/test-cases-abc.t#B.err +++ $TESTTMP/anothertests/cases/test-cases-abc.t#B.err
@@ -7,7 +7,7 @@ @@ -7,7 +7,7 @@
$ V=C $ V=C
#endif #endif
$ echo $V | sed 's/A/C/' $ echo $V | sed 's/A/C/'
- C - C
+ B + B
#if C #if C
$ [ $V = C ] $ [ $V = C ]
#endif #endif
ERROR: test-cases-abc.t#B output changed ERROR: test-cases-abc.t#B output changed
!. !.
Failed test-cases-abc.t#B: output changed Failed test-cases-abc.t#B: output changed
# Ran 3 tests, 0 skipped, 1 failed. # Ran 3 tests, 0 skipped, 1 failed.
python hash seed: * (glob) python hash seed: * (glob)
[1] [1]
--restart works --restart works
$ rt --restart $ rt --restart
running 2 tests using 1 parallel processes
--- $TESTTMP/anothertests/cases/test-cases-abc.t --- $TESTTMP/anothertests/cases/test-cases-abc.t
+++ $TESTTMP/anothertests/cases/test-cases-abc.t#B.err +++ $TESTTMP/anothertests/cases/test-cases-abc.t#B.err
@@ -7,7 +7,7 @@ @@ -7,7 +7,7 @@
$ V=C $ V=C
#endif #endif
$ echo $V | sed 's/A/C/' $ echo $V | sed 's/A/C/'
- C - C
python hash seed: * (glob) python hash seed: * (glob)
[1] [1]
--restart works with outputdir --restart works with outputdir
$ mkdir output $ mkdir output
$ mv test-cases-abc.t#B.err output $ mv test-cases-abc.t#B.err output
$ rt --restart --outputdir output $ rt --restart --outputdir output
running 2 tests using 1 parallel processes
--- $TESTTMP/anothertests/cases/test-cases-abc.t --- $TESTTMP/anothertests/cases/test-cases-abc.t
+++ $TESTTMP/anothertests/cases/output/test-cases-abc.t#B.err +++ $TESTTMP/anothertests/cases/output/test-cases-abc.t#B.err
@@ -7,7 +7,7 @@ @@ -7,7 +7,7 @@
$ V=C $ V=C
#endif #endif
$ echo $V | sed 's/A/C/' $ echo $V | sed 's/A/C/'
- C - C
> In case A > In case A
> #endif > #endif
> #if B > #if B
> $ dostuff > $ dostuff
> In case B > In case B
> #endif > #endif
> EOF > EOF
$ rt test-cases-ab.t $ rt test-cases-ab.t
running 2 tests using 1 parallel processes
.. ..
# Ran 2 tests, 0 skipped, 0 failed. # Ran 2 tests, 0 skipped, 0 failed.
Support running a specific test case Support running a specific test case
$ rt "test-cases-abc.t#B" $ rt "test-cases-abc.t#B"
running 1 tests using 1 parallel processes
--- $TESTTMP/anothertests/cases/test-cases-abc.t --- $TESTTMP/anothertests/cases/test-cases-abc.t
+++ $TESTTMP/anothertests/cases/test-cases-abc.t#B.err +++ $TESTTMP/anothertests/cases/test-cases-abc.t#B.err
@@ -7,7 +7,7 @@ @@ -7,7 +7,7 @@
$ V=C $ V=C
#endif #endif
$ echo $V | sed 's/A/C/' $ echo $V | sed 's/A/C/'
- C - C
+ B + B
#if C #if C
$ [ $V = C ] $ [ $V = C ]
#endif #endif
ERROR: test-cases-abc.t#B output changed ERROR: test-cases-abc.t#B output changed
! !
Failed test-cases-abc.t#B: output changed Failed test-cases-abc.t#B: output changed
# Ran 1 tests, 0 skipped, 1 failed. # Ran 1 tests, 0 skipped, 1 failed.
python hash seed: * (glob) python hash seed: * (glob)
[1] [1]
Support running multiple test cases in the same file Support running multiple test cases in the same file
$ rt test-cases-abc.t#B test-cases-abc.t#C $ rt test-cases-abc.t#B test-cases-abc.t#C
running 2 tests using 1 parallel processes
--- $TESTTMP/anothertests/cases/test-cases-abc.t --- $TESTTMP/anothertests/cases/test-cases-abc.t
+++ $TESTTMP/anothertests/cases/test-cases-abc.t#B.err +++ $TESTTMP/anothertests/cases/test-cases-abc.t#B.err
@@ -7,7 +7,7 @@ @@ -7,7 +7,7 @@
$ V=C $ V=C
#endif #endif
$ echo $V | sed 's/A/C/' $ echo $V | sed 's/A/C/'
- C - C
+ B + B
#if C #if C
$ [ $V = C ] $ [ $V = C ]
#endif #endif
ERROR: test-cases-abc.t#B output changed ERROR: test-cases-abc.t#B output changed
!. !.
Failed test-cases-abc.t#B: output changed Failed test-cases-abc.t#B: output changed
# Ran 2 tests, 0 skipped, 1 failed. # Ran 2 tests, 0 skipped, 1 failed.
python hash seed: * (glob) python hash seed: * (glob)
[1] [1]
Support ignoring invalid test cases Support ignoring invalid test cases
$ rt test-cases-abc.t#B test-cases-abc.t#D $ rt test-cases-abc.t#B test-cases-abc.t#D
running 1 tests using 1 parallel processes
--- $TESTTMP/anothertests/cases/test-cases-abc.t --- $TESTTMP/anothertests/cases/test-cases-abc.t
+++ $TESTTMP/anothertests/cases/test-cases-abc.t#B.err +++ $TESTTMP/anothertests/cases/test-cases-abc.t#B.err
@@ -7,7 +7,7 @@ @@ -7,7 +7,7 @@
$ V=C $ V=C
#endif #endif
$ echo $V | sed 's/A/C/' $ echo $V | sed 's/A/C/'
- C - C
> EOF > EOF
$ cat test-cases-advanced-cases.t $ cat test-cases-advanced-cases.t
#testcases simple case-with-dashes casewith_-.chars #testcases simple case-with-dashes casewith_-.chars
$ echo $TESTCASE $ echo $TESTCASE
simple simple
$ rt test-cases-advanced-cases.t $ rt test-cases-advanced-cases.t
running 3 tests using 1 parallel processes
--- $TESTTMP/anothertests/cases/test-cases-advanced-cases.t --- $TESTTMP/anothertests/cases/test-cases-advanced-cases.t
+++ $TESTTMP/anothertests/cases/test-cases-advanced-cases.t#case-with-dashes.err +++ $TESTTMP/anothertests/cases/test-cases-advanced-cases.t#case-with-dashes.err
@@ -1,3 +1,3 @@ @@ -1,3 +1,3 @@
#testcases simple case-with-dashes casewith_-.chars #testcases simple case-with-dashes casewith_-.chars
$ echo $TESTCASE $ echo $TESTCASE
- simple - simple
+ case-with-dashes + case-with-dashes
!. !.
Failed test-cases-advanced-cases.t#case-with-dashes: output changed Failed test-cases-advanced-cases.t#case-with-dashes: output changed
Failed test-cases-advanced-cases.t#casewith_-.chars: output changed Failed test-cases-advanced-cases.t#casewith_-.chars: output changed
# Ran 3 tests, 0 skipped, 2 failed. # Ran 3 tests, 0 skipped, 2 failed.
python hash seed: * (glob) python hash seed: * (glob)
[1] [1]
$ rt "test-cases-advanced-cases.t#case-with-dashes" $ rt "test-cases-advanced-cases.t#case-with-dashes"
running 1 tests using 1 parallel processes
--- $TESTTMP/anothertests/cases/test-cases-advanced-cases.t --- $TESTTMP/anothertests/cases/test-cases-advanced-cases.t
+++ $TESTTMP/anothertests/cases/test-cases-advanced-cases.t#case-with-dashes.err +++ $TESTTMP/anothertests/cases/test-cases-advanced-cases.t#case-with-dashes.err
@@ -1,3 +1,3 @@ @@ -1,3 +1,3 @@
#testcases simple case-with-dashes casewith_-.chars #testcases simple case-with-dashes casewith_-.chars
$ echo $TESTCASE $ echo $TESTCASE
- simple - simple
+ case-with-dashes + case-with-dashes
ERROR: test-cases-advanced-cases.t#case-with-dashes output changed ERROR: test-cases-advanced-cases.t#case-with-dashes output changed
! !
Failed test-cases-advanced-cases.t#case-with-dashes: output changed Failed test-cases-advanced-cases.t#case-with-dashes: output changed
# Ran 1 tests, 0 skipped, 1 failed. # Ran 1 tests, 0 skipped, 1 failed.
python hash seed: * (glob) python hash seed: * (glob)
[1] [1]
$ rt "test-cases-advanced-cases.t#casewith_-.chars" $ rt "test-cases-advanced-cases.t#casewith_-.chars"
running 1 tests using 1 parallel processes
--- $TESTTMP/anothertests/cases/test-cases-advanced-cases.t --- $TESTTMP/anothertests/cases/test-cases-advanced-cases.t
+++ $TESTTMP/anothertests/cases/test-cases-advanced-cases.t#casewith_-.chars.err +++ $TESTTMP/anothertests/cases/test-cases-advanced-cases.t#casewith_-.chars.err
@@ -1,3 +1,3 @@ @@ -1,3 +1,3 @@
#testcases simple case-with-dashes casewith_-.chars #testcases simple case-with-dashes casewith_-.chars
$ echo $TESTCASE $ echo $TESTCASE
- simple - simple
+ casewith_-.chars + casewith_-.chars
> $ echo bar prior > $ echo bar prior
> bar prior > bar prior
> $ echo lastbar > $ echo lastbar
> last\$YYY$ > last\$YYY$
> $ echo foo-bar foo-baz > $ echo foo-bar foo-baz
> EOF > EOF
$ rt test-substitution.t $ rt test-substitution.t
running 1 tests using 1 parallel processes
--- $TESTTMP/anothertests/cases/test-substitution.t --- $TESTTMP/anothertests/cases/test-substitution.t
+++ $TESTTMP/anothertests/cases/test-substitution.t.err +++ $TESTTMP/anothertests/cases/test-substitution.t.err
@@ -7,3 +7,4 @@ @@ -7,3 +7,4 @@
$ echo lastbar $ echo lastbar
last$YYY$ last$YYY$
$ echo foo-bar foo-baz $ echo foo-bar foo-baz
+ $XXX=bar foo-baz$ + $XXX=bar foo-baz$
ERROR: test-substitution.t output changed ERROR: test-substitution.t output changed
! !
Failed test-substitution.t: output changed Failed test-substitution.t: output changed
# Ran 1 tests, 0 skipped, 1 failed. # Ran 1 tests, 0 skipped, 1 failed.
python hash seed: * (glob) python hash seed: * (glob)
[1] [1]
--extra-config-opt works --extra-config-opt works
$ cat << EOF >> test-config-opt.t $ cat << EOF >> test-config-opt.t
> $ hg init test-config-opt > $ hg init test-config-opt
> $ hg -R test-config-opt purge > $ hg -R test-config-opt purge
> EOF > EOF
$ rt --extra-config-opt extensions.purge= test-config-opt.t $ rt --extra-config-opt extensions.purge= test-config-opt.t
running 1 tests using 1 parallel processes
. .
# Ran 1 tests, 0 skipped, 0 failed. # Ran 1 tests, 0 skipped, 0 failed.