( )⚙ D3721 run-tests: restrict the test cases allowed characters

This is an archive of the discontinued Mercurial Phabricator instance.

Differential D3721

run-tests: restrict the test cases allowed characters
ClosedPublic

Authored by lothiraldan on Jun 12 2018, 5:05 PM.

Details

Reviewers
mharbison72
durin42
Group Reviewers
hg-reviewers
Commits
rHGb5651ae53127: run-tests: restrict the test cases allowed characters
Summary

Most test cases in core and in extension only use A-Za-z0-9- but a few tests
in extensions also have a . in their test cases names. Also add a _ to
allow all kind of case naming format.

Diff Detail

Repository
rHG Mercurial
Lint
Lint Skipped
Unit
Unit Tests Skipped

Event Timeline

lothiraldan created this revision.Jun 12 2018, 5:05 PM
mharbison72 accepted this revision.Jun 12 2018, 7:54 PM
mharbison72 added a subscriber: mharbison72.

The second hunk in run-test.py doesn't apply for me (it looks like adding b to a string has landed, but wasn't applied near the first hunk). But manually applying it, Windows is happy again. Thanks.

durin42 accepted this revision.Jun 12 2018, 10:13 PM
This revision is now accepted and ready to land.Jun 12 2018, 10:13 PM
Closed by commit rHGb5651ae53127: run-tests: restrict the test cases allowed characters (authored by lothiraldan). · Explain WhyJun 12 2018, 11:42 PM
This revision was automatically updated to reflect the committed changes.

Revision Contents
Changeset List

PathPackages
Mtests/run-tests.py (4 lines)
Mtests/test-run-tests.t (30 lines)

Diff 9030

tests/run-tests.py

styles = { styles = {
skipped: '#e5e5e5', skipped: '#e5e5e5',
skippedname: '#00ffff', skippedname: '#00ffff',
failed: '#7f0000', failed: '#7f0000',
failedname: '#ff0000', failedname: '#ff0000',
} }
class TestRunnerLexer(lexer.RegexLexer): class TestRunnerLexer(lexer.RegexLexer):
testpattern = r'[\w-]+\.(t|py)(#[^\s]+)?' testpattern = r'[\w-]+\.(t|py)(#[a-zA-Z0-9_\-\.]+)?'
tokens = { tokens = {
'root': [ 'root': [
(r'^Skipped', token.Generic.Skipped, 'skipped'), (r'^Skipped', token.Generic.Skipped, 'skipped'),
(r'^Failed ', token.Generic.Failed, 'failed'), (r'^Failed ', token.Generic.Failed, 'failed'),
(r'^ERROR: ', token.Generic.Failed, 'failed'), (r'^ERROR: ', token.Generic.Failed, 'failed'),
], ],
'skipped': [ 'skipped': [
(testpattern, token.Generic.SName), (testpattern, token.Generic.SName),
if os.path.isdir(arg): if os.path.isdir(arg):
if not arg.endswith(b'/'): if not arg.endswith(b'/'):
arg += b'/' arg += b'/'
expanded_args.extend([arg + a for a in os.listdir(arg)]) expanded_args.extend([arg + a for a in os.listdir(arg)])
else: else:
expanded_args.append(arg) expanded_args.append(arg)
args = expanded_args args = expanded_args
testcasepattern = re.compile(r'([\w-]+\.t|py)(#([^\s]+))') testcasepattern = re.compile(r'([\w-]+\.t|py)(#([a-zA-Z0-9_\-\.]+))')
tests = [] tests = []
for t in args: for t in args:
case = None case = None
if not (os.path.basename(t).startswith(b'test-') if not (os.path.basename(t).startswith(b'test-')
and (t.endswith(b'.py') or t.endswith(b'.t'))): and (t.endswith(b'.py') or t.endswith(b'.t'))):
m = testcasepattern.match(t) m = testcasepattern.match(t)

tests/test-run-tests.t

Failed test-cases-abc.t#B: output changed Failed test-cases-abc.t#B: output changed
# Ran 1 tests, 0 skipped, 1 failed. # Ran 1 tests, 0 skipped, 1 failed.
python hash seed: * (glob) python hash seed: * (glob)
[1] [1]
Support running complex test cases names Support running complex test cases names
$ cat > test-cases-advanced-cases.t <<'EOF' $ cat > test-cases-advanced-cases.t <<'EOF'
> #testcases simple case-with-dashes casewith!@#$%^&*()chars > #testcases simple case-with-dashes casewith_-.chars
> $ echo $TESTCASE > $ echo $TESTCASE
> simple > simple
> EOF > EOF
$ cat test-cases-advanced-cases.t $ cat test-cases-advanced-cases.t
#testcases simple case-with-dashes casewith!@#$%^&*()chars #testcases simple case-with-dashes casewith_-.chars
$ echo $TESTCASE $ echo $TESTCASE
simple simple
$ rt test-cases-advanced-cases.t $ rt test-cases-advanced-cases.t
--- $TESTTMP/anothertests/cases/test-cases-advanced-cases.t --- $TESTTMP/anothertests/cases/test-cases-advanced-cases.t
+++ $TESTTMP/anothertests/cases/test-cases-advanced-cases.t.case-with-dashes.err +++ $TESTTMP/anothertests/cases/test-cases-advanced-cases.t.case-with-dashes.err
@@ -1,3 +1,3 @@ @@ -1,3 +1,3 @@
#testcases simple case-with-dashes casewith!@#$%^&*()chars #testcases simple case-with-dashes casewith_-.chars
$ echo $TESTCASE $ echo $TESTCASE
- simple - simple
+ case-with-dashes + case-with-dashes
ERROR: test-cases-advanced-cases.t#case-with-dashes output changed ERROR: test-cases-advanced-cases.t#case-with-dashes output changed
! !
--- $TESTTMP/anothertests/cases/test-cases-advanced-cases.t --- $TESTTMP/anothertests/cases/test-cases-advanced-cases.t
+++ $TESTTMP/anothertests/cases/test-cases-advanced-cases.t.casewith!@#$%^&*()chars.err +++ $TESTTMP/anothertests/cases/test-cases-advanced-cases.t.casewith_-.chars.err
@@ -1,3 +1,3 @@ @@ -1,3 +1,3 @@
#testcases simple case-with-dashes casewith!@#$%^&*()chars #testcases simple case-with-dashes casewith_-.chars
$ echo $TESTCASE $ echo $TESTCASE
- simple - simple
+ casewith!@#$%^&*()chars + casewith_-.chars
ERROR: test-cases-advanced-cases.t#casewith!@#$%^&*()chars output changed ERROR: test-cases-advanced-cases.t#casewith_-.chars output changed
!. !.
Failed test-cases-advanced-cases.t#case-with-dashes: output changed Failed test-cases-advanced-cases.t#case-with-dashes: output changed
Failed test-cases-advanced-cases.t#casewith!@#$%^&*()chars: output changed Failed test-cases-advanced-cases.t#casewith_-.chars: output changed
# Ran 3 tests, 0 skipped, 2 failed. # Ran 3 tests, 0 skipped, 2 failed.
python hash seed: * (glob) python hash seed: * (glob)
[1] [1]
$ rt "test-cases-advanced-cases.t#case-with-dashes" $ rt "test-cases-advanced-cases.t#case-with-dashes"
--- $TESTTMP/anothertests/cases/test-cases-advanced-cases.t --- $TESTTMP/anothertests/cases/test-cases-advanced-cases.t
+++ $TESTTMP/anothertests/cases/test-cases-advanced-cases.t.case-with-dashes.err +++ $TESTTMP/anothertests/cases/test-cases-advanced-cases.t.case-with-dashes.err
@@ -1,3 +1,3 @@ @@ -1,3 +1,3 @@
#testcases simple case-with-dashes casewith!@#$%^&*()chars #testcases simple case-with-dashes casewith_-.chars
$ echo $TESTCASE $ echo $TESTCASE
- simple - simple
+ case-with-dashes + case-with-dashes
ERROR: test-cases-advanced-cases.t#case-with-dashes output changed ERROR: test-cases-advanced-cases.t#case-with-dashes output changed
! !
Failed test-cases-advanced-cases.t#case-with-dashes: output changed Failed test-cases-advanced-cases.t#case-with-dashes: output changed
# Ran 1 tests, 0 skipped, 1 failed. # Ran 1 tests, 0 skipped, 1 failed.
python hash seed: * (glob) python hash seed: * (glob)
[1] [1]
$ rt "test-cases-advanced-cases.t#casewith!@#$%^&*()chars" $ rt "test-cases-advanced-cases.t#casewith_-.chars"
--- $TESTTMP/anothertests/cases/test-cases-advanced-cases.t --- $TESTTMP/anothertests/cases/test-cases-advanced-cases.t
+++ $TESTTMP/anothertests/cases/test-cases-advanced-cases.t.casewith!@#$%^&*()chars.err +++ $TESTTMP/anothertests/cases/test-cases-advanced-cases.t.casewith_-.chars.err
@@ -1,3 +1,3 @@ @@ -1,3 +1,3 @@
#testcases simple case-with-dashes casewith!@#$%^&*()chars #testcases simple case-with-dashes casewith_-.chars
$ echo $TESTCASE $ echo $TESTCASE
- simple - simple
+ casewith!@#$%^&*()chars + casewith_-.chars
ERROR: test-cases-advanced-cases.t#casewith!@#$%^&*()chars output changed ERROR: test-cases-advanced-cases.t#casewith_-.chars output changed
! !
Failed test-cases-advanced-cases.t#casewith!@#$%^&*()chars: output changed Failed test-cases-advanced-cases.t#casewith_-.chars: output changed
# Ran 1 tests, 0 skipped, 1 failed. # Ran 1 tests, 0 skipped, 1 failed.
python hash seed: * (glob) python hash seed: * (glob)
[1] [1]
Test automatic pattern replacement Test automatic pattern replacement
================================== ==================================
$ cat << EOF >> common-pattern.py $ cat << EOF >> common-pattern.py