This is an archive of the discontinued Mercurial Phabricator instance.

Differential D7862

dirstate: move rust fast-path calling code to its own method
ClosedPublic

Authored by Alphare on Jan 14 2020, 12:32 PM.

Details

Reviewers
pulkit
Group Reviewers
hg-reviewers
Commits
rHGf2c350e7371e: dirstate: move rust fast-path calling code to its own method
rHG717f7eb39655: dirstate: move rust fast-path calling code to its own method
Summary

This logic is about to get bigger, this will make it easier to read and not
pollute the main Python logic.

Diff Detail

Repository
rHG Mercurial
Branch
default
Lint
No Linters Available
Unit
No Unit Test Coverage

Event Timeline

Alphare created this revision.Jan 14 2020, 12:32 PM
Alphare added a comment.Jan 14 2020, 12:38 PM

Note to reviewers: this stack is part of the larger series of getting a full Rust dirstate.status which is most of the performance hit in bare hg status. More patches are coming, but I figured I would send the patches as I go to help with review timing.

pulkit accepted this revision.Jan 14 2020, 4:10 PM
This revision is now accepted and ready to land.Jan 14 2020, 4:10 PM
pulkit added a comment.Jan 15 2020, 8:11 AM

Absorbing the following diff to make test-check-format.t happy:

diff --git a/mercurial/dirstate.py b/mercurial/dirstate.py
--- a/mercurial/dirstate.py
+++ b/mercurial/dirstate.py
@@ -1089,9 +1089,7 @@ class dirstate(object):
         # how to read the config file.
         numcpus = self._ui.configint(b"worker", b"numcpus")
         if numcpus is not None:
-            encoding.environ.setdefault(
-                b'RAYON_NUM_THREADS', b'%d' % numcpus
-            )
+            encoding.environ.setdefault(b'RAYON_NUM_THREADS', b'%d' % numcpus)
 
         workers_enabled = self._ui.configbool(b"worker", b"enabled", True)
         if not workers_enabled:
Alphare added a comment.Jan 15 2020, 9:03 AM
In D7862#115784, @pulkit wrote:

Absorbing the following diff to make test-check-format.t happy:

Sorry about that.

Alphare updated this revision to Diff 19314.Jan 15 2020, 4:55 PM

Revision Contents
Changeset List

PathPackages
Mmercurial/dirstate.py (83 lines)

Diff 19314

mercurial/dirstate.py

else: else:
# We may not have walked the full directory tree above, # We may not have walked the full directory tree above,
# so stat and check everything we missed. # so stat and check everything we missed.
iv = iter(visit) iv = iter(visit)
for st in util.statfiles([join(i) for i in visit]): for st in util.statfiles([join(i) for i in visit]):
results[next(iv)] = st results[next(iv)] = st
return results return results
def _rust_status(self, matcher, list_clean):
# Force Rayon (Rust parallelism library) to respect the number of
# workers. This is a temporary workaround until Rust code knows
# how to read the config file.
numcpus = self._ui.configint(b"worker", b"numcpus")
if numcpus is not None:
encoding.environ.setdefault(
b'RAYON_NUM_THREADS', b'%d' % numcpus
)
workers_enabled = self._ui.configbool(b"worker", b"enabled", True)
if not workers_enabled:
encoding.environ[b"RAYON_NUM_THREADS"] = b"1"
(
lookup,
modified,
added,
removed,
deleted,
unknown,
clean,
) = rustmod.status(
self._map._rustmap,
matcher,
self._rootdir,
bool(list_clean),
self._lastnormaltime,
self._checkexec,
)
status = scmutil.status(
modified=modified,
added=added,
removed=removed,
deleted=deleted,
unknown=unknown,
ignored=[],
clean=clean,
)
return (lookup, status)
def status(self, match, subrepos, ignored, clean, unknown): def status(self, match, subrepos, ignored, clean, unknown):
'''Determine the status of the working copy relative to the '''Determine the status of the working copy relative to the
dirstate and return a pair of (unsure, status), where status is of type dirstate and return a pair of (unsure, status), where status is of type
scmutil.status and: scmutil.status and:
unsure: unsure:
files that might have been modified since the dirstate was files that might have been modified since the dirstate was
written, but need to be read to be sure (size is the same written, but need to be read to be sure (size is the same
# Rust has no ignore mechanism yet, so don't use Rust for # Rust has no ignore mechanism yet, so don't use Rust for
# commands that need ignore. # commands that need ignore.
use_rust = False use_rust = False
elif not isinstance(match, allowed_matchers): elif not isinstance(match, allowed_matchers):
# Matchers have yet to be implemented # Matchers have yet to be implemented
use_rust = False use_rust = False
if use_rust: if use_rust:
# Force Rayon (Rust parallelism library) to respect the number of return self._rust_status(match, listclean)
# workers. This is a temporary workaround until Rust code knows
# how to read the config file.
numcpus = self._ui.configint(b"worker", b"numcpus")
if numcpus is not None:
encoding.environ.setdefault(
b'RAYON_NUM_THREADS', b'%d' % numcpus
)
workers_enabled = self._ui.configbool(b"worker", b"enabled", True)
if not workers_enabled:
encoding.environ[b"RAYON_NUM_THREADS"] = b"1"
(
lookup,
modified,
added,
removed,
deleted,
unknown,
clean,
) = rustmod.status(
dmap._rustmap,
match,
self._rootdir,
bool(listclean),
self._lastnormaltime,
self._checkexec,
)
status = scmutil.status(
modified=modified,
added=added,
removed=removed,
deleted=deleted,
unknown=unknown,
ignored=ignored,
clean=clean,
)
return (lookup, status)
def noop(f): def noop(f):
pass pass
dcontains = dmap.__contains__ dcontains = dmap.__contains__
dget = dmap.__getitem__ dget = dmap.__getitem__
ladd = lookup.append # aka "unsure" ladd = lookup.append # aka "unsure"
madd = modified.append madd = modified.append