Page MenuHomePhabricator

tests: make run-tests exit non-zero if there are "errors"
ClosedPublic

Authored by spectral on Wed, May 29, 2:26 AM.

Details

Summary

Previously, if there was an error such as a broken .t file that caused
run-tests.py to encounter an exception during parsing, the test would be
considered in an "errored" state, which is separate from "failed".

The check for whether to exit non-zero or not was based entirely on whether
there were any tests in a "failed" state, so if there was only an error,
run-tests would exit with 0. Our test infrastructure would then consider the
test as passing, causing us to have some tests with false negatives that have
gone undetected for a few weeks now.

Diff Detail

Repository
rHG Mercurial
Lint
Automatic diff as part of commit; lint not applicable.
Unit
Automatic diff as part of commit; unit tests not applicable.

Event Timeline

spectral created this revision.Wed, May 29, 2:26 AM
pulkit accepted this revision.Wed, May 29, 8:40 AM
This revision was automatically updated to reflect the committed changes.