( )⚙ D9813 discovery: add config options to control sample size

This is an archive of the discontinued Mercurial Phabricator instance.

Differential D9813

discovery: add config options to control sample size
ClosedPublic

Authored by marmoute on Jan 17 2021, 5:26 PM.

Details

Reviewers
pulkit
Group Reviewers
hg-reviewers
Commits
rHG0600e8467101: discovery: add config options to control sample size
Summary

This get useful for running analysis of the current algorithm.

Diff Detail

Repository
rHG Mercurial
Branch
default
Lint
No Linters Available
Unit
No Unit Test Coverage

Event Timeline

marmoute created this revision.Jan 17 2021, 5:26 PM
baymax updated this revision to Diff 25095.Jan 17 2021, 6:35 PM

✅ refresh by Heptapod after a successful CI run (🐙 💚)

pulkit accepted this revision.Jan 21 2021, 6:23 AM
This revision is now accepted and ready to land.Jan 21 2021, 6:23 AM
marmoute added a commit: rHG0600e8467101: discovery: add config options to control sample size.Jan 21 2021, 7:24 AM
Closed by commit rHG0600e8467101: discovery: add config options to control sample size (authored by marmoute). · Explain Why
This revision was automatically updated to reflect the committed changes.

Revision Contents
Changeset List

Diff 25095

mercurial/configitems.py

) )
# If discovery.randomize is False, random sampling during discovery are # If discovery.randomize is False, random sampling during discovery are
# deterministic. It is meant for integration tests. # deterministic. It is meant for integration tests.
coreconfigitem( coreconfigitem(
b'devel', b'devel',
b'discovery.randomize', b'discovery.randomize',
default=True, default=True,
) )
# Control the initial size of the discovery sample
coreconfigitem(
b'devel',
b'discovery.sample-size',
default=200,
)
# Control the initial size of the discovery for initial change
coreconfigitem(
b'devel',
b'discovery.sample-size.initial',
default=100,
)
_registerdiffopts(section=b'diff') _registerdiffopts(section=b'diff')
coreconfigitem( coreconfigitem(
b'email', b'email',
b'bcc', b'bcc',
default=None, default=None,
) )
coreconfigitem( coreconfigitem(
b'email', b'email',

mercurial/setdiscovery.py

'discovery', member='PartialDiscovery', default=partialdiscovery 'discovery', member='PartialDiscovery', default=partialdiscovery
) )
def findcommonheads( def findcommonheads(
ui, ui,
local, local,
remote, remote,
initialsamplesize=100,
fullsamplesize=200,
abortwhenunrelated=True, abortwhenunrelated=True,
ancestorsof=None, ancestorsof=None,
audit=None, audit=None,
): ):
"""Return a tuple (common, anyincoming, remoteheads) used to identify """Return a tuple (common, anyincoming, remoteheads) used to identify
missing nodes from or in remote. missing nodes from or in remote.
The audit argument is an optional dictionnary that a caller can pass. it The audit argument is an optional dictionnary that a caller can pass. it
clrev = cl.rev clrev = cl.rev
if ancestorsof is not None: if ancestorsof is not None:
ownheads = [clrev(n) for n in ancestorsof] ownheads = [clrev(n) for n in ancestorsof]
else: else:
ownheads = [rev for rev in cl.headrevs() if rev != nullrev] ownheads = [rev for rev in cl.headrevs() if rev != nullrev]
initial_head_exchange = ui.configbool(b'devel', b'discovery.exchange-heads') initial_head_exchange = ui.configbool(b'devel', b'discovery.exchange-heads')
initialsamplesize = ui.configint(b'devel', b'discovery.sample-size.initial')
fullsamplesize = ui.configint(b'devel', b'discovery.sample-size')
# We also ask remote about all the local heads. That set can be arbitrarily # We also ask remote about all the local heads. That set can be arbitrarily
# large, so we used to limit it size to `initialsamplesize`. We no longer # large, so we used to limit it size to `initialsamplesize`. We no longer
# do as it proved counter productive. The skipped heads could lead to a # do as it proved counter productive. The skipped heads could lead to a
# large "undecided" set, slower to be clarified than if we asked the # large "undecided" set, slower to be clarified than if we asked the
# question for all heads right away. # question for all heads right away.
# #
# We are already fetching all server heads using the `heads` commands, # We are already fetching all server heads using the `heads` commands,
# sending a equivalent number of heads the other way should not have a # sending a equivalent number of heads the other way should not have a

tests/test-setdiscovery.t

adding changesets adding changesets
adding manifests adding manifests
adding file changes adding file changes
added 304 changesets with 0 changes to 0 files added 304 changesets with 0 changes to 0 files
new changesets 1ea73414a91b:513314ca8b3a new changesets 1ea73414a91b:513314ca8b3a
updating to branch b updating to branch b
0 files updated, 0 files merged, 0 files removed, 0 files unresolved 0 files updated, 0 files merged, 0 files removed, 0 files unresolved
$ hg -R a debugdiscovery b --debug --verbose --config progress.debug=true --config devel.discovery.randomize=false $ hg -R a debugdiscovery b --debug --verbose --config progress.debug=true --config devel.discovery.randomize=false --config devel.discovery.sample-size.initial=50
comparing with b comparing with b
query 1; heads query 1; heads
searching for changes searching for changes
taking quick initial sample taking quick initial sample
searching: 2 queries searching: 2 queries
query 2; still undecided: 1080, sample size is: 100 query 2; still undecided: 1080, sample size is: 50
sampling from both directions sampling from both directions
searching: 3 queries searching: 3 queries
query 3; still undecided: 980, sample size is: 200 query 3; still undecided: 1030, sample size is: 200
sampling from both directions sampling from both directions
searching: 4 queries searching: 4 queries
query 4; still undecided: 497, sample size is: 210 query 4; still undecided: 547, sample size is: 210
sampling from both directions sampling from both directions
searching: 5 queries searching: 5 queries
query 5; still undecided: 285, sample size is: 220 query 5; still undecided: 336, sample size is: 220
sampling from both directions sampling from both directions
searching: 6 queries searching: 6 queries
query 6; still undecided: 63, sample size is: 63 query 6; still undecided: 114, sample size is: 114
6 total queries in *.????s (glob) 6 total queries in *.????s (glob)
elapsed time: * seconds (glob) elapsed time: * seconds (glob)
round-trips: 6 round-trips: 6
heads summary: heads summary:
total common heads: 1 total common heads: 1
also local heads: 0 also local heads: 0
also remote heads: 0 also remote heads: 0
both: 0 both: 0
roots: 260 roots: 260
first undecided set: 1340 first undecided set: 1340
heads: 260 heads: 260
roots: 1 roots: 1
common: 300 common: 300
missing: 1040 missing: 1040
common heads: 3ee37d65064a common heads: 3ee37d65064a
$ hg -R a debugdiscovery b --debug --config devel.discovery.exchange-heads=false --config devel.discovery.randomize=false --config devel.discovery.grow-sample.rate=1.01 $ hg -R a debugdiscovery b --debug --config devel.discovery.exchange-heads=false --config devel.discovery.randomize=false --config devel.discovery.grow-sample.rate=1.20 --config devel.discovery.sample-size=50
comparing with b comparing with b
searching for changes searching for changes
sampling from both directions sampling from both directions
query 1; still undecided: 1340, sample size is: 200 query 1; still undecided: 1340, sample size is: 50
sampling from both directions sampling from both directions
query 2; still undecided: 795, sample size is: 202 query 2; still undecided: 995, sample size is: 60
sampling from both directions sampling from both directions
query 3; still undecided: 525, sample size is: 204 query 3; still undecided: 913, sample size is: 72
sampling from both directions sampling from both directions
query 4; still undecided: 252, sample size is: 206 query 4; still undecided: 816, sample size is: 204
sampling from both directions sampling from both directions
query 5; still undecided: 44, sample size is: 44 query 5; still undecided: 612, sample size is: 153
5 total queries in *s (glob) sampling from both directions
query 6; still undecided: 456, sample size is: 123
sampling from both directions
query 7; still undecided: 332, sample size is: 147
sampling from both directions
query 8; still undecided: 184, sample size is: 176
sampling from both directions
query 9; still undecided: 8, sample size is: 8
9 total queries in *s (glob)
elapsed time: * seconds (glob) elapsed time: * seconds (glob)
round-trips: 5 round-trips: 9
heads summary: heads summary:
total common heads: 1 total common heads: 1
also local heads: 0 also local heads: 0
also remote heads: 0 also remote heads: 0
both: 0 both: 0
local heads: 260 local heads: 260
common: 0 common: 0
missing: 260 missing: 260