This is an archive of the discontinued Mercurial Phabricator instance.

Differential D9813

discovery: add config options to control sample size
ClosedPublic

Authored by marmoute on Jan 17 2021, 5:26 PM.

Details

Reviewers
pulkit
Group Reviewers
hg-reviewers
Commits
rHG0600e8467101: discovery: add config options to control sample size
Summary

This get useful for running analysis of the current algorithm.

Diff Detail

Repository
rHG Mercurial
Lint
Automatic diff as part of commit; lint not applicable.
Unit
Automatic diff as part of commit; unit tests not applicable.

Event Timeline

marmoute created this revision.Jan 17 2021, 5:26 PM
baymax updated this revision to Diff 25095.Jan 17 2021, 6:35 PM

✅ refresh by Heptapod after a successful CI run (🐙 💚)

pulkit accepted this revision.Jan 21 2021, 6:23 AM
This revision is now accepted and ready to land.Jan 21 2021, 6:23 AM
marmoute added a commit: rHG0600e8467101: discovery: add config options to control sample size.Jan 21 2021, 7:24 AM
Closed by commit rHG0600e8467101: discovery: add config options to control sample size (authored by marmoute). · Explain Why
This revision was automatically updated to reflect the committed changes.

Revision Contents
Changeset List

Diff 25190

mercurial/configitems.py

) )
# If discovery.randomize is False, random sampling during discovery are # If discovery.randomize is False, random sampling during discovery are
# deterministic. It is meant for integration tests. # deterministic. It is meant for integration tests.
coreconfigitem( coreconfigitem(
b'devel', b'devel',
b'discovery.randomize', b'discovery.randomize',
default=True, default=True,
) )
# Control the initial size of the discovery sample
coreconfigitem(
b'devel',
b'discovery.sample-size',
default=200,
)
# Control the initial size of the discovery for initial change
coreconfigitem(
b'devel',
b'discovery.sample-size.initial',
default=100,
)
_registerdiffopts(section=b'diff') _registerdiffopts(section=b'diff')
coreconfigitem( coreconfigitem(
b'email', b'email',
b'bcc', b'bcc',
default=None, default=None,
) )
coreconfigitem( coreconfigitem(
b'email', b'email',

mercurial/setdiscovery.py

'discovery', member='PartialDiscovery', default=partialdiscovery 'discovery', member='PartialDiscovery', default=partialdiscovery
) )
def findcommonheads( def findcommonheads(
ui, ui,
local, local,
remote, remote,
initialsamplesize=100,
fullsamplesize=200,
abortwhenunrelated=True, abortwhenunrelated=True,
ancestorsof=None, ancestorsof=None,
audit=None, audit=None,
): ):
"""Return a tuple (common, anyincoming, remoteheads) used to identify """Return a tuple (common, anyincoming, remoteheads) used to identify
missing nodes from or in remote. missing nodes from or in remote.
The audit argument is an optional dictionnary that a caller can pass. it The audit argument is an optional dictionnary that a caller can pass. it
clrev = cl.rev clrev = cl.rev
if ancestorsof is not None: if ancestorsof is not None:
ownheads = [clrev(n) for n in ancestorsof] ownheads = [clrev(n) for n in ancestorsof]
else: else:
ownheads = [rev for rev in cl.headrevs() if rev != nullrev] ownheads = [rev for rev in cl.headrevs() if rev != nullrev]
initial_head_exchange = ui.configbool(b'devel', b'discovery.exchange-heads') initial_head_exchange = ui.configbool(b'devel', b'discovery.exchange-heads')
initialsamplesize = ui.configint(b'devel', b'discovery.sample-size.initial')
fullsamplesize = ui.configint(b'devel', b'discovery.sample-size')
# We also ask remote about all the local heads. That set can be arbitrarily # We also ask remote about all the local heads. That set can be arbitrarily
# large, so we used to limit it size to `initialsamplesize`. We no longer # large, so we used to limit it size to `initialsamplesize`. We no longer
# do as it proved counter productive. The skipped heads could lead to a # do as it proved counter productive. The skipped heads could lead to a
# large "undecided" set, slower to be clarified than if we asked the # large "undecided" set, slower to be clarified than if we asked the
# question for all heads right away. # question for all heads right away.
# #
# We are already fetching all server heads using the `heads` commands, # We are already fetching all server heads using the `heads` commands,
# sending a equivalent number of heads the other way should not have a # sending a equivalent number of heads the other way should not have a

tests/test-setdiscovery.t

adding changesets adding changesets
adding manifests adding manifests
adding file changes adding file changes
added 304 changesets with 0 changes to 0 files added 304 changesets with 0 changes to 0 files
new changesets 1ea73414a91b:513314ca8b3a new changesets 1ea73414a91b:513314ca8b3a
updating to branch b updating to branch b
0 files updated, 0 files merged, 0 files removed, 0 files unresolved 0 files updated, 0 files merged, 0 files removed, 0 files unresolved
$ hg -R a debugdiscovery b --debug --verbose --config progress.debug=true --config devel.discovery.randomize=false $ hg -R a debugdiscovery b --debug --verbose --config progress.debug=true --config devel.discovery.randomize=false --config devel.discovery.sample-size.initial=50
comparing with b comparing with b
query 1; heads query 1; heads
searching for changes searching for changes
taking quick initial sample taking quick initial sample
searching: 2 queries searching: 2 queries
query 2; still undecided: 1080, sample size is: 100 query 2; still undecided: 1080, sample size is: 50
sampling from both directions sampling from both directions
searching: 3 queries searching: 3 queries
query 3; still undecided: 980, sample size is: 200 query 3; still undecided: 1030, sample size is: 200
sampling from both directions sampling from both directions
searching: 4 queries searching: 4 queries
query 4; still undecided: 497, sample size is: 210 query 4; still undecided: 547, sample size is: 210
sampling from both directions sampling from both directions
searching: 5 queries searching: 5 queries
query 5; still undecided: 285, sample size is: 220 query 5; still undecided: 336, sample size is: 220
sampling from both directions sampling from both directions
searching: 6 queries searching: 6 queries
query 6; still undecided: 63, sample size is: 63 query 6; still undecided: 114, sample size is: 114
6 total queries in *.????s (glob) 6 total queries in *.????s (glob)
elapsed time: * seconds (glob) elapsed time: * seconds (glob)
round-trips: 6 round-trips: 6
heads summary: heads summary:
total common heads: 1 total common heads: 1
also local heads: 0 also local heads: 0
also remote heads: 0 also remote heads: 0
both: 0 both: 0
roots: 260 roots: 260
first undecided set: 1340 first undecided set: 1340
heads: 260 heads: 260
roots: 1 roots: 1
common: 300 common: 300
missing: 1040 missing: 1040
common heads: 3ee37d65064a common heads: 3ee37d65064a
$ hg -R a debugdiscovery b --debug --config devel.discovery.exchange-heads=false --config devel.discovery.randomize=false --config devel.discovery.grow-sample.rate=1.01 $ hg -R a debugdiscovery b --debug --config devel.discovery.exchange-heads=false --config devel.discovery.randomize=false --config devel.discovery.grow-sample.rate=1.20 --config devel.discovery.sample-size=50
comparing with b comparing with b
searching for changes searching for changes
sampling from both directions sampling from both directions
query 1; still undecided: 1340, sample size is: 200 query 1; still undecided: 1340, sample size is: 50
sampling from both directions sampling from both directions
query 2; still undecided: 795, sample size is: 202 query 2; still undecided: 995, sample size is: 60
sampling from both directions sampling from both directions
query 3; still undecided: 525, sample size is: 204 query 3; still undecided: 913, sample size is: 72
sampling from both directions sampling from both directions
query 4; still undecided: 252, sample size is: 206 query 4; still undecided: 816, sample size is: 204
sampling from both directions sampling from both directions
query 5; still undecided: 44, sample size is: 44 query 5; still undecided: 612, sample size is: 153
5 total queries in *s (glob) sampling from both directions
query 6; still undecided: 456, sample size is: 123
sampling from both directions
query 7; still undecided: 332, sample size is: 147
sampling from both directions
query 8; still undecided: 184, sample size is: 176
sampling from both directions
query 9; still undecided: 8, sample size is: 8
9 total queries in *s (glob)
elapsed time: * seconds (glob) elapsed time: * seconds (glob)
round-trips: 5 round-trips: 9
heads summary: heads summary:
total common heads: 1 total common heads: 1
also local heads: 0 also local heads: 0
also remote heads: 0 also remote heads: 0
both: 0 both: 0
local heads: 260 local heads: 260
common: 0 common: 0
missing: 260 missing: 260