rust-cpython: using MissingAncestors from Python code
ClosedPublic

Authored by gracinet on Jan 10 2019, 4:56 AM.

Details

Summary

As precedently done with LazyAncestors on cpython.rs, we test for the
presence of the 'rustext' module.

incrementalmissingrevs() has two callers within the Mercurial core:
setdiscovery.partialdiscovery and the only() revset.

This move shows a significant discovery performance improvement
in cases where the baseline is slow: using perfdiscovery on the PyPy
repos, prepared with contrib/discovery-helper <repo> 50 100, we
get averaged medians of 403ms with the Rust version vs 742ms without
(about 45% better).

But there are still indications that performance can be worse in cases
the baseline is fast, possibly due to the conversion from Python to
Rust and back becoming the bottleneck. We could measure this on
mozilla-central in cases were the delta is just a few changesets.
This requires confirmation, but if that's the reason, then an
upcoming partialdiscovery fully in Rust should solve the problem.

Diff Detail

Repository
rHG Mercurial
Lint
Automatic diff as part of commit; lint not applicable.
Unit
Automatic diff as part of commit; unit tests not applicable.
gracinet created this revision.Jan 10 2019, 4:56 AM

+ bases = self._common.bases
+ if callable(bases):
+ # This happens for Rust exported object, and we don't know
+ # at the moment how to make an equivalent of property for them
+ bases = bases()
+

    1. The presence of nullrev will confuse heads(). So filter it out. return set(self._repo.revs('heads(%ld)',
  • self._common.bases - {nullrev})) + (r for r in bases if r != nullrev)))

@lothiraldan Any idea on how to make this less ad-hoc?

If we'll never need bases of set type, maybe we can add baseslist() to both
Python and Rust implementations.

@yuja I talked with @lothiraldan about this today. It turns out that it'd be better to provide a basesheads returning a set directly from the MissingAncestors object (I do have a Rust implementation for heads).

This would make the call from setdiscovery uniform, and our measurements show that this would be a significant performance gain in the Rust version.
Preliminary measurements indeed show a performance increase with this approach over x30 within a perfdiscovery on the PyPy repo for this commonheads() method, which was abnormaly slow in the version you commented upon and ate most of the Rust boost.
It brings the Rust overall improvement for that discovery at 28% faster.

gracinet updated this revision to Diff 13219.Jan 14 2019, 9:14 PM
gracinet edited the summary of this revision. (Show Details)
This revision was automatically updated to reflect the committed changes.