( )⚙ D12384 rust-status: cap the number of concurrent threads to 16

This is an archive of the discontinued Mercurial Phabricator instance.

Differential D12384

rust-status: cap the number of concurrent threads to 16
ClosedPublic

Authored by Alphare on Fri, Mar 18, 12:15 PM.

Details

Reviewers
marmoute
Group Reviewers
hg-reviewers
Commits
rHGe2f8ed37201c: rust-status: cap the number of concurrent threads to 16
rHGf8025bfc7bf3: rust-status: cap the number of concurrent threads to 16
Summary

During benchmarking it was determined that the use of more threads is very
advantageous... until we use more than 16. This is most likely due to some
resource contention (thrashing, etc.). Until we have time to figure out and
fix the underlying cause, let's just cap at 16 threads.

Diff Detail

Repository
rHG Mercurial
Branch
stable
Lint
No Linters Available
Unit
No Unit Test Coverage

Event Timeline

Alphare created this revision.Fri, Mar 18, 12:15 PM
marmoute accepted this revision.Fri, Mar 18, 12:32 PM
marmoute added a subscriber: marmoute.

Looks good for stable.

Alphare added a commit: rHGf8025bfc7bf3: rust-status: cap the number of concurrent threads to 16.Sat, Mar 19, 1:10 AM
This revision was not accepted when it landed; it landed in state Needs Review.
Closed by commit rHGf8025bfc7bf3: rust-status: cap the number of concurrent threads to 16 (authored by Alphare). · Explain Why
This revision was automatically updated to reflect the committed changes.

Revision Contents
Changeset List

Diff 32663

rust/hg-core/src/dirstate_tree/status.rs

#[timed] #[timed]
pub fn status<'tree, 'on_disk: 'tree>( pub fn status<'tree, 'on_disk: 'tree>(
dmap: &'tree mut DirstateMap<'on_disk>, dmap: &'tree mut DirstateMap<'on_disk>,
matcher: &(dyn Matcher + Sync), matcher: &(dyn Matcher + Sync),
root_dir: PathBuf, root_dir: PathBuf,
ignore_files: Vec<PathBuf>, ignore_files: Vec<PathBuf>,
options: StatusOptions, options: StatusOptions,
) -> Result<(DirstateStatus<'on_disk>, Vec<PatternFileWarning>), StatusError> { ) -> Result<(DirstateStatus<'on_disk>, Vec<PatternFileWarning>), StatusError> {
// Force the global rayon threadpool to not exceed 16 concurrent threads.
// This is a stop-gap measure until we figure out why using more than 16
// threads makes `status` slower for each additional thread.
// We use `ok()` in case the global threadpool has already been instantiated
// in `rhg` or some other caller.
// TODO find the underlying cause and fix it, then remove this.
rayon::ThreadPoolBuilder::new()
.num_threads(16)
.build_global()
.ok();
let (ignore_fn, warnings, patterns_changed): (IgnoreFnType, _, _) = let (ignore_fn, warnings, patterns_changed): (IgnoreFnType, _, _) =
if options.list_ignored || options.list_unknown { if options.list_ignored || options.list_unknown {
let mut hasher = Sha1::new(); let mut hasher = Sha1::new();
let (ignore_fn, warnings) = get_ignore_function( let (ignore_fn, warnings) = get_ignore_function(
ignore_files, ignore_files,
&root_dir, &root_dir,
&mut |pattern_bytes| hasher.update(pattern_bytes), &mut |pattern_bytes| hasher.update(pattern_bytes),
)?; )?;