This is an archive of the discontinued Mercurial Phabricator instance.

Differential D8270

run-tests: restrict Rust thread pool to 3 threads during tests
ClosedPublic

Authored by Alphare on Mar 10 2020, 11:28 AM.

Details

Reviewers
durin42
marmoute
Group Reviewers
hg-reviewers
Commits
rHGb2e41723f886: run-tests: restrict Rust thread pool to 3 threads during tests

Diff Detail

Repository
rHG Mercurial
Lint
Automatic diff as part of commit; lint not applicable.
Unit
Automatic diff as part of commit; unit tests not applicable.

Event Timeline

Alphare created this revision.Mar 10 2020, 11:28 AM
pulkit added a subscriber: pulkit.Mar 10 2020, 12:33 PM
pulkit added inline comments.
tests/run-tests.py
3002

Should we also take value of -j into account too? If -j 1 is used, then we should not do multi-threading here.

durin42 accepted this revision as: durin42.Mar 10 2020, 4:34 PM
durin42 added a subscriber: durin42.
durin42 added inline comments.
tests/run-tests.py
3002

I don't think so, since part of the goal is to uncover threading bugs.

marmoute added a subscriber: marmoute.Mar 10 2020, 8:22 PM
marmoute added inline comments.
tests/run-tests.py
3002

I don't think we should take -j in account. Even if we run test one at a time, it is important to test the multithreading code path

marmoute accepted this revision.Mar 11 2020, 11:56 AM
Alphare added a commit: rHGb2e41723f886: run-tests: restrict Rust thread pool to 3 threads during tests.Mar 11 2020, 12:22 PM
This revision was not accepted when it landed; it landed in state Needs Review.
Closed by commit rHGb2e41723f886: run-tests: restrict Rust thread pool to 3 threads during tests (authored by Alphare). · Explain Why
This revision was automatically updated to reflect the committed changes.

Revision Contents
Changeset List

PathPackages
Mtests/run-tests.py (6 lines)

Diff 20711

tests/run-tests.py

previoustimes = dict(loadtimes(self._outputdir)) previoustimes = dict(loadtimes(self._outputdir))
sorttests(testdescs, previoustimes, shuffle=self.options.random) sorttests(testdescs, previoustimes, shuffle=self.options.random)
if 'PYTHONHASHSEED' not in os.environ: if 'PYTHONHASHSEED' not in os.environ:
# use a random python hash seed all the time # use a random python hash seed all the time
# we do the randomness ourself to know what seed is used # we do the randomness ourself to know what seed is used
os.environ['PYTHONHASHSEED'] = str(random.getrandbits(32)) os.environ['PYTHONHASHSEED'] = str(random.getrandbits(32))
# Rayon (Rust crate for multi-threading) will use all logical CPU cores
# by default, causing thrashing on high-cpu-count systems.
# Setting its limit to 3 during tests should still let us uncover
# multi-threading bugs while keeping the thrashing reasonable.
os.environ.setdefault("RAYON_NUM_THREADS", "3")
pulkitUnsubmitted
Not Done

Should we also take value of -j into account too? If -j 1 is used, then we should not do multi-threading here.

pulkit: Should we also take value of `-j` into account too? If `-j 1` is used, then we should not do…
marmouteUnsubmitted
Not Done

I don't think we should take -j in account. Even if we run test one at a time, it is important to test the multithreading code path

marmoute: I don't think we should take -j in account. Even if we run test one at a time, it is important…
durin42Unsubmitted
Not Done

I don't think so, since part of the goal is to uncover threading bugs.

durin42: I don't think so, since part of the goal is to uncover threading bugs.
if self.options.tmpdir: if self.options.tmpdir:
self.options.keep_tmpdir = True self.options.keep_tmpdir = True
tmpdir = _sys2bytes(self.options.tmpdir) tmpdir = _sys2bytes(self.options.tmpdir)
if os.path.exists(tmpdir): if os.path.exists(tmpdir):
# Meaning of tmpdir has changed since 1.3: we used to create # Meaning of tmpdir has changed since 1.3: we used to create
# HGTMP inside tmpdir; now HGTMP is tmpdir. So fail if # HGTMP inside tmpdir; now HGTMP is tmpdir. So fail if
# tmpdir already exists. # tmpdir already exists.
print("error: temp dir %r already exists" % tmpdir) print("error: temp dir %r already exists" % tmpdir)