This is an archive of the discontinued Mercurial Phabricator instance.

Differential D8927

posixworker: avoid creating workers that end up getting no work
ClosedPublic

Authored by martinvonz on Aug 13 2020, 1:45 PM.

Details

Reviewers
pulkit
Group Reviewers
hg-reviewers
Commits
rHG26eb62bd0550: posixworker: avoid creating workers that end up getting no work
Summary

If workers (the detected or configured number of CPUs) is greater
than the number of work items, then some of the workers end up getting
0 work items. Let's not create such workers.

Diff Detail

Repository
rHG Mercurial
Lint
Automatic diff as part of commit; lint not applicable.
Unit
Automatic diff as part of commit; unit tests not applicable.

Event Timeline

martinvonz created this revision.Aug 13 2020, 1:45 PM
pulkit accepted this revision.Aug 14 2020, 4:40 AM
This revision is now accepted and ready to land.Aug 14 2020, 4:40 AM

Revision Contents
Changeset List

PathPackages
Mmercurial/worker.py (2 lines)

Diff 22404

mercurial/worker.py

if problem[0]: if problem[0]:
killworkers() killworkers()
oldchldhandler = signal.signal(signal.SIGCHLD, sigchldhandler) oldchldhandler = signal.signal(signal.SIGCHLD, sigchldhandler)
ui.flush() ui.flush()
parentpid = os.getpid() parentpid = os.getpid()
pipes = [] pipes = []
retval = {} retval = {}
for pargs in partition(args, workers): for pargs in partition(args, min(workers, len(args))):
# Every worker gets its own pipe to send results on, so we don't have to # Every worker gets its own pipe to send results on, so we don't have to
# implement atomic writes larger than PIPE_BUF. Each forked process has # implement atomic writes larger than PIPE_BUF. Each forked process has
# its own pipe's descriptors in the local variables, and the parent # its own pipe's descriptors in the local variables, and the parent
# process has the full list of pipe descriptors (and it doesn't really # process has the full list of pipe descriptors (and it doesn't really
# care what order they're in). # care what order they're in).
rfd, wfd = os.pipe() rfd, wfd = os.pipe()
pipes.append((rfd, wfd)) pipes.append((rfd, wfd))
# make sure we use os._exit in all worker code paths. otherwise the # make sure we use os._exit in all worker code paths. otherwise the