This is an archive of the discontinued Mercurial Phabricator instance.

Differential D10693

run-tests: fix whitelist/blacklist with directories on Windows
ClosedPublic

Authored by mharbison72 on May 7 2021, 12:01 AM.

Details

Reviewers
Alphare
Group Reviewers
hg-reviewers
Summary

The file name is resolved with os.path.relpath() in the Test constructor,
which yields \ on Windows. That doesn't match the / separator when using
MSYS tools to build the list, and it isn't obvious that this is the problem
because directory separators can mostly be used interchangeably. The
--test-list argument already seems to be properly handled.

Diff Detail

Repository
rHG Mercurial
Branch
stable
Lint
No Linters Available
Unit
No Unit Test Coverage

Event Timeline

mharbison72 created this revision.May 7 2021, 12:01 AM
marmoute added a subscriber: marmoute.May 7 2021, 1:19 AM

Should we fix the handling of that list in run-tests.py instead ?

mharbison72 added a comment.May 7 2021, 10:24 AM
In D10693#162446, @marmoute wrote:

Should we fix the handling of that list in run-tests.py instead ?

Not sure what you're asking- this is run-test.py. I think any sys.argv input is assigned to a Test, and so they get run through os.path.relpath(). This just normalizes the input from files to match.

Alphare accepted this revision.May 17 2021, 3:58 AM
This revision is now accepted and ready to land.May 17 2021, 3:58 AM
mharbison72 added a comment.Jul 9 2021, 11:13 AM

It looks like this never landed.

Alphare added a comment.Jul 9 2021, 11:21 AM

Whoops, sorry!

mharbison72 closed this revision.Jul 9 2021, 11:31 AM

This landed as dff19fe2973c, but lacked the Differential URL in the commit comment somehow. It may have gotten pulled off of the Windows CI work on heptapod.

Revision Contents
Changeset List

PathPackages
Mtests/run-tests.py (3 lines)
Mtests/test-run-tests.t (10 lines)

Diff 27738

tests/run-tests.py

raise raise
if warn: if warn:
print("warning: no such %s file: %s" % (listtype, filename)) print("warning: no such %s file: %s" % (listtype, filename))
continue continue
for line in f.readlines(): for line in f.readlines():
line = line.split(b'#', 1)[0].strip() line = line.split(b'#', 1)[0].strip()
if line: if line:
entries[line] = filename # Ensure path entries are compatible with os.path.relpath()
entries[os.path.normpath(line)] = filename
f.close() f.close()
return entries return entries
def parsettestcases(path): def parsettestcases(path):
"""read a .t test file, return a set of test case names """read a .t test file, return a set of test case names

tests/test-run-tests.t

<testcase name="test-skip.t"> <testcase name="test-skip.t">
<skipped><![CDATA[missing feature: nail clipper]]></skipped> (py38 !) <skipped><![CDATA[missing feature: nail clipper]]></skipped> (py38 !)
<skipped> (no-py38 !) <skipped> (no-py38 !)
<![CDATA[missing feature: nail clipper]]> </skipped> (no-py38 !) <![CDATA[missing feature: nail clipper]]> </skipped> (no-py38 !)
</testcase> </testcase>
</testsuite> </testsuite>
Missing skips or blacklisted skips don't count as executed: Missing skips or blacklisted skips don't count as executed:
$ echo test-failure.t > blacklist $ mkdir tests
$ echo tests/test-failure.t > blacklist
$ cp test-failure.t tests
$ rt --blacklist=blacklist --json\ $ rt --blacklist=blacklist --json\
> test-failure.t test-bogus.t > tests/test-failure.t tests/test-bogus.t
running 2 tests using 1 parallel processes running 2 tests using 1 parallel processes
ss ss
Skipped test-bogus.t: Doesn't exist Skipped test-bogus.t: Doesn't exist
Skipped test-failure.t: blacklisted Skipped test-failure.t: blacklisted
# Ran 0 tests, 2 skipped, 0 failed. # Ran 0 tests, 2 skipped, 0 failed.
$ cat report.json $ cat tests/report.json
testreport ={ testreport ={
"test-bogus.t": { "test-bogus.t": {
"result": "skip" "result": "skip"
}, },
"test-failure.t": { "test-failure.t": {
"result": "skip" "result": "skip"
} }
} (no-eol) } (no-eol)
$ rm -r tests
$ echo test-failure.t > blacklist
Whitelist trumps blacklist Whitelist trumps blacklist
$ echo test-failure.t > whitelist $ echo test-failure.t > whitelist
$ rt --blacklist=blacklist --whitelist=whitelist --json\ $ rt --blacklist=blacklist --whitelist=whitelist --json\
> test-failure.t test-bogus.t > test-failure.t test-bogus.t
running 2 tests using 1 parallel processes running 2 tests using 1 parallel processes
s s
--- $TESTTMP/test-failure.t --- $TESTTMP/test-failure.t