This is an archive of the discontinued Mercurial Phabricator instance.

Differential D6427

discovery: new devel.discovery.randomize option
ClosedPublic

Authored by gracinet on May 22 2019, 12:50 PM.

Details

Reviewers
kevincox
Group Reviewers
hg-reviewers
Commits
rHG334c1ea57136: discovery: new devel.discovery.randomize option
Summary

By default, this is True, but setting it to False is a uniform
way to kill all randomness in integration tests such as test-setdiscovery.t

By "uniform" we mean that it can be passed to implementations in other
languages, for which the monkey-patching of random.sample would be
irrelevant.

In the above mentioned test file, we use it right away,
replacing the adhoc extension that had the same purpose, and to derandomize a
case with many round-trips, that we'll need to behave uniformly in the Rust
version.

Diff Detail

Repository
rHG Mercurial
Lint
Automatic diff as part of commit; lint not applicable.
Unit
Automatic diff as part of commit; unit tests not applicable.

Event Timeline

gracinet created this revision.May 22 2019, 12:50 PM
gracinet added a subscriber: kevincox.May 22 2019, 12:56 PM

@kevincox this used to be D6346
I had to fix conflict and adapt the test due to growing sample size having landed since then

kevincox accepted this revision.Jun 10 2019, 8:39 AM
gracinet updated this revision to Diff 15470.Jun 12 2019, 2:17 PM
Alphare updated this revision to Diff 15937.Jul 17 2019, 12:30 PM
Alphare updated this revision to Diff 16015.Jul 23 2019, 5:15 AM
gracinet added a commit: rHG334c1ea57136: discovery: new devel.discovery.randomize option.Aug 14 2019, 4:16 PM
This revision was not accepted when it landed; it landed in state Needs Review.
Closed by commit rHG334c1ea57136: discovery: new devel.discovery.randomize option (authored by gracinet). · Explain Why
This revision was automatically updated to reflect the committed changes.

Revision Contents
Changeset List

Diff 16190

mercurial/configitems.py

default=False, default=False,
) )
coreconfigitem('devel', 'debug.extensions', coreconfigitem('devel', 'debug.extensions',
default=False, default=False,
) )
coreconfigitem('devel', 'debug.peer-request', coreconfigitem('devel', 'debug.peer-request',
default=False, default=False,
) )
coreconfigitem('devel', 'discovery.randomize',
default=True,
)
_registerdiffopts(section='diff') _registerdiffopts(section='diff')
coreconfigitem('email', 'bcc', coreconfigitem('email', 'bcc',
default=None, default=None,
) )
coreconfigitem('email', 'cc', coreconfigitem('email', 'cc',
default=None, default=None,
) )
coreconfigitem('email', 'charsets', coreconfigitem('email', 'charsets',

mercurial/setdiscovery.py

if len(sample) == len(ownheads) and all(yesno): if len(sample) == len(ownheads) and all(yesno):
ui.note(_("all local heads known remotely\n")) ui.note(_("all local heads known remotely\n"))
ownheadhashes = [clnode(r) for r in ownheads] ownheadhashes = [clnode(r) for r in ownheads]
return ownheadhashes, True, srvheadhashes return ownheadhashes, True, srvheadhashes
# full blown discovery # full blown discovery
disco = partialdiscovery(local, ownheads, remote.limitedarguments) randomize = ui.configbool('devel', 'discovery.randomize')
disco = partialdiscovery(local, ownheads, remote.limitedarguments,
randomize=randomize)
# treat remote heads (and maybe own heads) as a first implicit sample # treat remote heads (and maybe own heads) as a first implicit sample
# response # response
disco.addcommons(knownsrvheads) disco.addcommons(knownsrvheads)
disco.addinfo(zip(sample, yesno)) disco.addinfo(zip(sample, yesno))
full = False full = False
progress = ui.makeprogress(_('searching'), unit=_('queries')) progress = ui.makeprogress(_('searching'), unit=_('queries'))
while not disco.iscomplete(): while not disco.iscomplete():

tests/test-setdiscovery.t

adding changesets adding changesets
adding manifests adding manifests
adding file changes adding file changes
added 304 changesets with 0 changes to 0 files added 304 changesets with 0 changes to 0 files
new changesets 1ea73414a91b:513314ca8b3a new changesets 1ea73414a91b:513314ca8b3a
updating to branch b updating to branch b
0 files updated, 0 files merged, 0 files removed, 0 files unresolved 0 files updated, 0 files merged, 0 files removed, 0 files unresolved
$ hg -R a debugdiscovery b --debug --verbose --config progress.debug=true $ hg -R a debugdiscovery b --debug --verbose --config progress.debug=true --config devel.discovery.randomize=false
comparing with b comparing with b
query 1; heads query 1; heads
searching for changes searching for changes
taking quick initial sample taking quick initial sample
searching: 2 queries searching: 2 queries
query 2; still undecided: 1080, sample size is: 100 query 2; still undecided: 1080, sample size is: 100
sampling from both directions sampling from both directions
searching: 3 queries searching: 3 queries
query 3; still undecided: 980, sample size is: 200 query 3; still undecided: 980, sample size is: 200
sampling from both directions sampling from both directions
searching: 4 queries searching: 4 queries
query 4; still undecided: 435, sample size is: 210 (no-py3 !) query 4; still undecided: 497, sample size is: 210
query 4; still undecided: 430, sample size is: 210 (py3 !)
sampling from both directions sampling from both directions
searching: 5 queries searching: 5 queries
query 5; still undecided: 185, sample size is: 185 (no-py3 !) query 5; still undecided: 285, sample size is: 220
query 5; still undecided: 187, sample size is: 187 (py3 !) sampling from both directions
5 total queries in *.????s (glob) searching: 6 queries
query 6; still undecided: 63, sample size is: 63
6 total queries in *.????s (glob)
elapsed time: * seconds (glob) elapsed time: * seconds (glob)
heads summary: heads summary:
total common heads: 1 total common heads: 1
also local heads: 0 also local heads: 0
also remote heads: 0 also remote heads: 0
both: 0 both: 0
local heads: 260 local heads: 260
common: 0 common: 0
comparing with r2 comparing with r2
searching for changes searching for changes
101 102 103 104 105 106 107 108 109 110 (no-eol) 101 102 103 104 105 106 107 108 109 110 (no-eol)
The case where all the 'initialsamplesize' samples already were common would The case where all the 'initialsamplesize' samples already were common would
give 'all remote heads known locally' without checking the remaining heads - give 'all remote heads known locally' without checking the remaining heads -
fixed in 86c35b7ae300: fixed in 86c35b7ae300:
$ cat >> $TESTTMP/unrandomsample.py << EOF
> import random
> def sample(population, k):
> return sorted(population)[:k]
> random.sample = sample
> EOF
$ cat >> r1/.hg/hgrc << EOF $ cat >> r1/.hg/hgrc << EOF
> [extensions] > [devel]
> unrandomsample = $TESTTMP/unrandomsample.py > discovery.randomize = False
> EOF > EOF
$ hg -R r1 outgoing r2 -T'{rev} ' --config extensions.blackbox= \ $ hg -R r1 outgoing r2 -T'{rev} ' --config extensions.blackbox= \
> --config blackbox.track='command commandfinish discovery' > --config blackbox.track='command commandfinish discovery'
comparing with r2 comparing with r2
searching for changes searching for changes
101 102 103 104 105 106 107 108 109 110 (no-eol) 101 102 103 104 105 106 107 108 109 110 (no-eol)
$ hg -R r1 --config extensions.blackbox= blackbox --config blackbox.track= $ hg -R r1 --config extensions.blackbox= blackbox --config blackbox.track=
* @5d0b986a083e0d91f116de4691e2aaa54d5bbec0 (*)> serve --cmdserver chgunix * (glob) (chg !) * @5d0b986a083e0d91f116de4691e2aaa54d5bbec0 (*)> serve --cmdserver chgunix * (glob) (chg !)
* @5d0b986a083e0d91f116de4691e2aaa54d5bbec0 (*)> -R r1 outgoing r2 *-T{rev} * --config *extensions.blackbox=* (glob) * @5d0b986a083e0d91f116de4691e2aaa54d5bbec0 (*)> -R r1 outgoing r2 *-T{rev} * --config *extensions.blackbox=* (glob)
* @5d0b986a083e0d91f116de4691e2aaa54d5bbec0 (*)> found 101 common and 1 unknown server heads, 1 roundtrips in *.????s (glob) * @5d0b986a083e0d91f116de4691e2aaa54d5bbec0 (*)> found 101 common and 1 unknown server heads, 1 roundtrips in *.????s (glob)
* @5d0b986a083e0d91f116de4691e2aaa54d5bbec0 (*)> -R r1 outgoing r2 *-T{rev} * --config *extensions.blackbox=* exited 0 after *.?? seconds (glob) * @5d0b986a083e0d91f116de4691e2aaa54d5bbec0 (*)> -R r1 outgoing r2 *-T{rev} * --config *extensions.blackbox=* exited 0 after *.?? seconds (glob)
$ cd .. $ cd ..