This is an archive of the discontinued Mercurial Phabricator instance.

Using workers in lfs prefetch
ClosedPublic

Authored by wlis on Nov 15 2017, 4:13 AM.
Tags
None
Subscribers
None

Details

Reviewers
quark
Group Reviewers
Restricted Project
Commits
rFBHGX67bcf096af13: Using workers in lfs prefetch
Summary

Added workers in lfs.
I had to remove the fine progress tracking because between processes in *nix and threads in windows (diffs will appear soon) the tracking of 1MB progress is quite tricky.
With our network tracking progress per file is way enough to see things moving.

This change gives close to 50% speedup on hg sparse --enable-profile when prefetch is run. My current understanding is that prefetch is ran when profile is enabled for the first time.

Test Plan

Enable profile:
time hg sprase --enable-profile SparseProfiles/TestProfile.sparse
The profile contains 42k files including 9GB of lfs files
On my machine the time improves by 47% while still being dominated by lfs
download time

Diff Detail

Repository
rFBHGX Facebook Mercurial Extensions
Lint
Automatic diff as part of commit; lint not applicable.
Unit
Automatic diff as part of commit; unit tests not applicable.

Event Timeline

wlis created this revision.Nov 15 2017, 4:13 AM
Herald added a reviewer: Restricted Project. · View Herald TranscriptNov 15 2017, 4:13 AM
wlis edited the test plan for this revision. (Show Details)Nov 15 2017, 4:15 AM
quark accepted this revision.Nov 15 2017, 9:05 PM
This revision is now accepted and ready to land.Nov 15 2017, 9:05 PM
This revision was automatically updated to reflect the committed changes.