This is an archive of the discontinued Mercurial Phabricator instance.

Differential D1186

tests: de-flake test-run-tests.t's "--jobs=2 --first" test
ClosedPublic

Authored by martinvonz on Oct 18 2017, 7:31 PM.

Details

Reviewers
quark
durin42
Group Reviewers
hg-reviewers
Commits
rHGb63a7d839c33: tests: de-flake test-run-tests.t's "--jobs=2 --first" test
rHGe2efb1377340: tests: de-flake test-run-tests.t's "--jobs=2 --first" test
Summary

Once in a while, test-nothing.t finished before test-failure.t or
test-failure-copy.t (I haven't checked which one actually gets run
first). Since there already are two tests that will fail in the same
way, just run those two instead so the diff will be reproducible and
not timing-dependent.

The test case was added in 9a20f53e436f (run-tests: handle --jobs and
--first gracefully, 2014-10-09), and I have checked that backing that
out results in two failures being printed. Note that the summary may
still include multiple tests even if --first is given, it's just that
the diff is only printed for the first failure.

Diff Detail

Repository
rHG Mercurial
Lint
Automatic diff as part of commit; lint not applicable.
Unit
Automatic diff as part of commit; unit tests not applicable.

Event Timeline

martinvonz created this revision.Oct 18 2017, 7:31 PM
quark accepted this revision.Oct 18 2017, 8:46 PM
durin42 accepted this revision.Oct 19 2017, 10:56 AM
This revision is now accepted and ready to land.Oct 19 2017, 10:56 AM
Closed by commit rHGe2efb1377340: tests: de-flake test-run-tests.t's "--jobs=2 --first" test (authored by martinvonz). · Explain WhyOct 19 2017, 11:11 AM
This revision was automatically updated to reflect the committed changes.

Revision Contents
Changeset List

PathPackages
Mtests/test-run-tests.t (9 lines)

Diff 3032

tests/test-run-tests.t

!! !!
Failed test-failure*.t: output changed (glob) Failed test-failure*.t: output changed (glob)
Failed test-failure*.t: output changed (glob) Failed test-failure*.t: output changed (glob)
# Ran 2 tests, 0 skipped, 2 failed. # Ran 2 tests, 0 skipped, 2 failed.
python hash seed: * (glob) python hash seed: * (glob)
[1] [1]
failures in parallel with --first should only print one failure failures in parallel with --first should only print one failure
>>> f = open('test-nothing.t', 'w') $ rt --jobs 2 --first test-failure*.t
>>> f.write('foo\n' * 1024) and None
>>> f.write(' $ sleep 1') and None
$ rt --jobs 2 --first
--- $TESTTMP/test-failure*.t (glob) --- $TESTTMP/test-failure*.t (glob)
+++ $TESTTMP/test-failure*.t.err (glob) +++ $TESTTMP/test-failure*.t.err (glob)
@@ -1,5 +1,5 @@ @@ -1,5 +1,5 @@
$ echo babar $ echo babar
- rataxes - rataxes
+ babar + babar
This is a noop statement so that This is a noop statement so that
this test is still more bytes than success. this test is still more bytes than success.
pad pad pad pad............................................................ pad pad pad pad............................................................
Failed test-failure*.t: output changed (glob) Failed test-failure*.t: output changed (glob)
Failed test-nothing.t: output changed Failed test-failure*.t: output changed (glob)
# Ran 2 tests, 0 skipped, 2 failed. # Ran 2 tests, 0 skipped, 2 failed.
python hash seed: * (glob) python hash seed: * (glob)
[1] [1]
(delete the duplicated test file) (delete the duplicated test file)
$ rm test-failure-copy.t test-nothing.t $ rm test-failure-copy.t
Interactive run Interactive run
=============== ===============
(backup the failing test) (backup the failing test)
$ cp test-failure.t backup $ cp test-failure.t backup