( )⚙ D3962 worker: ability to disable thread unsafe tasks

This is an archive of the discontinued Mercurial Phabricator instance.

worker: ability to disable thread unsafe tasks
ClosedPublic

Authored by indygreg on Jul 17 2018, 8:43 PM.

Details

Summary

The worker on Windows is implemented using a thread pool. If worker
tasks are not thread safe, badness can occur. In addition, if tasks
are executing CPU bound code and holding onto the GIL, there will be
non-substantial overhead in Python context switching between active
threads. This can result in significant slowdowns of tasks.

This commit teaches the code for determining whether to use a worker
to take thread safety into account. Effectively, thread unsafe tasks
don't use the thread-based worker on Windows.

Diff Detail

Repository
rHG Mercurial
Lint
Automatic diff as part of commit; lint not applicable.
Unit
Automatic diff as part of commit; unit tests not applicable.

Event Timeline

indygreg created this revision.Jul 17 2018, 8:43 PM
indygreg edited the summary of this revision. (Show Details)Jul 18 2018, 12:55 PM
indygreg retitled this revision from worker: ability to disable worker for CPU heavy tasks to worker: ability to disable thread unsafe tasks.
indygreg updated this revision to Diff 9623.
lothiraldan accepted this revision.Jul 19 2018, 3:03 AM
yuja added a subscriber: yuja.Jul 19 2018, 8:15 AM

if pycompat.isposix or pycompat.iswindows:

_STARTUP_COST = 0.01

+ # The Windows worker is thread based. If tasks are CPU bound, threads
+ # in the presence of the GIL result in excessive context switching and
+ # this overhead can slow down execution.
+ _DISALLOW_THREAD_UNSAFE = True

Fixed this as _DISALLOW_THREAD_UNSAFE = pycompat.iswindows, and queued.
Thanks.

This revision was automatically updated to reflect the committed changes.