This is an archive of the discontinued Mercurial Phabricator instance.

Differential D1927

bundle2: move version of stream clone into part name
ClosedPublic

Authored by indygreg on Jan 20 2018, 7:44 PM.

Details

Reviewers
durin42
Group Reviewers
hg-reviewers
Commits
rHGb116a66bcc44: bundle2: move version of stream clone into part name
Summary

I don't like having version numbers as part parameters. It means
that parts can theoretically vary wildly in their generation and
processing semantics. I think that a named part should have consistent
behavior over time. In other words, if you need to introduce new
functionality or behavior, that should be expressed by inventing
a new bundle2 part, not adding functionality to an existing part.

This commit applies this advice to the just-introduced stream clone
via bundle2 feature.

The "version" part parameter is removed. The name of the bundle2 part
is now "stream2" instead of "stream" with "version=v2".

Diff Detail

Repository
rHG Mercurial
Lint
Automatic diff as part of commit; lint not applicable.
Unit
Automatic diff as part of commit; unit tests not applicable.

Event Timeline

indygreg created this revision.Jan 20 2018, 7:44 PM
lothiraldan added a subscriber: lothiraldan.Jan 22 2018, 11:20 AM

Having a part named stream1 that used a V2 version of the streaming protocol and transporting a V2 bundle format seems confusing, I'm voting for a stream2 name.

indygreg edited the summary of this revision. (Show Details)Jan 22 2018, 3:32 PM
indygreg updated this revision to Diff 4985.
durin42 accepted this revision.Jan 22 2018, 4:09 PM
This revision is now accepted and ready to land.Jan 22 2018, 4:09 PM
Closed by commit rHGb116a66bcc44: bundle2: move version of stream clone into part name (authored by indygreg). · Explain WhyJan 22 2018, 4:22 PM
This revision was automatically updated to reflect the committed changes.

Revision Contents
Changeset List

Diff 4994

mercurial/bundle2.py

for key, value in part.advisoryparams: for key, value in part.advisoryparams:
key = key.upper() key = key.upper()
# We want pushed variables to have USERVAR_ prepended so we know # We want pushed variables to have USERVAR_ prepended so we know
# they came from the --pushvar flag. # they came from the --pushvar flag.
key = "USERVAR_" + key key = "USERVAR_" + key
hookargs[key] = value hookargs[key] = value
op.addhookargs(hookargs) op.addhookargs(hookargs)
@parthandler('stream', ('requirements', 'filecount', 'bytecount', 'version')) @parthandler('stream2', ('requirements', 'filecount', 'bytecount'))
def handlestreambundle(op, part): def handlestreamv2bundle(op, part):
version = part.params['version']
if version != 'v2':
raise error.Abort(_('unknown stream bundle version %s') % version)
requirements = part.params['requirements'].split() requirements = part.params['requirements'].split()
filecount = int(part.params['filecount']) filecount = int(part.params['filecount'])
bytecount = int(part.params['bytecount']) bytecount = int(part.params['bytecount'])
repo = op.repo repo = op.repo
if len(repo): if len(repo):
msg = _('cannot apply stream clone to non empty repository') msg = _('cannot apply stream clone to non empty repository')
raise error.Abort(msg) raise error.Abort(msg)

mercurial/exchange.py

func = getbundle2partsmapping[name] func = getbundle2partsmapping[name]
func(bundler, repo, source, bundlecaps=bundlecaps, b2caps=b2caps, func(bundler, repo, source, bundlecaps=bundlecaps, b2caps=b2caps,
**pycompat.strkwargs(kwargs)) **pycompat.strkwargs(kwargs))
info['prefercompressed'] = bundler.prefercompressed info['prefercompressed'] = bundler.prefercompressed
return info, bundler.getchunks() return info, bundler.getchunks()
@getbundle2partsgenerator('stream') @getbundle2partsgenerator('stream2')
def _getbundlestream(bundler, repo, source, bundlecaps=None, def _getbundlestream2(bundler, repo, source, bundlecaps=None,
b2caps=None, heads=None, common=None, **kwargs): b2caps=None, heads=None, common=None, **kwargs):
if not kwargs.get('stream', False): if not kwargs.get('stream', False):
return return
# Stream clones don't compress well. And compression undermines a # Stream clones don't compress well. And compression undermines a
# goal of stream clones, which is to be fast. Communicate the desire # goal of stream clones, which is to be fast. Communicate the desire
# to avoid compression to consumers of the bundle. # to avoid compression to consumers of the bundle.
bundler.prefercompressed = False bundler.prefercompressed = False
filecount, bytecount, it = streamclone.generatev2(repo) filecount, bytecount, it = streamclone.generatev2(repo)
requirements = ' '.join(sorted(repo.requirements)) requirements = ' '.join(sorted(repo.requirements))
part = bundler.newpart('stream', data=it) part = bundler.newpart('stream2', data=it)
part.addparam('bytecount', '%d' % bytecount, mandatory=True) part.addparam('bytecount', '%d' % bytecount, mandatory=True)
part.addparam('filecount', '%d' % filecount, mandatory=True) part.addparam('filecount', '%d' % filecount, mandatory=True)
part.addparam('requirements', requirements, mandatory=True) part.addparam('requirements', requirements, mandatory=True)
part.addparam('version', 'v2', mandatory=True)
@getbundle2partsgenerator('changegroup') @getbundle2partsgenerator('changegroup')
def _getbundlechangegrouppart(bundler, repo, source, bundlecaps=None, def _getbundlechangegrouppart(bundler, repo, source, bundlecaps=None,
b2caps=None, heads=None, common=None, **kwargs): b2caps=None, heads=None, common=None, **kwargs):
"""add a changegroup part to the requested bundle""" """add a changegroup part to the requested bundle"""
cgstream = None cgstream = None
if kwargs.get(r'cg', True): if kwargs.get(r'cg', True):
# build changegroup bundle here. # build changegroup bundle here.

tests/test-clone-uncompressed.t

getbundle requests with stream=1 are uncompressed getbundle requests with stream=1 are uncompressed
$ get-with-headers.py $LOCALIP:$HGPORT '?cmd=getbundle' content-type --bodyfile body --hgproto '0.1 0.2 comp=zlib,none' --requestheader "x-hgarg-1=bundlecaps=HG20%2Cbundle2%3DHG20%250Abookmarks%250Achangegroup%253D01%252C02%250Adigests%253Dmd5%252Csha1%252Csha512%250Aerror%253Dabort%252Cunsupportedcontent%252Cpushraced%252Cpushkey%250Ahgtagsfnodes%250Alistkeys%250Aphases%253Dheads%250Apushkey%250Aremote-changegroup%253Dhttp%252Chttps&cg=0&common=0000000000000000000000000000000000000000&heads=c17445101a72edac06facd130d14808dfbd5c7c2&stream=1" $ get-with-headers.py $LOCALIP:$HGPORT '?cmd=getbundle' content-type --bodyfile body --hgproto '0.1 0.2 comp=zlib,none' --requestheader "x-hgarg-1=bundlecaps=HG20%2Cbundle2%3DHG20%250Abookmarks%250Achangegroup%253D01%252C02%250Adigests%253Dmd5%252Csha1%252Csha512%250Aerror%253Dabort%252Cunsupportedcontent%252Cpushraced%252Cpushkey%250Ahgtagsfnodes%250Alistkeys%250Aphases%253Dheads%250Apushkey%250Aremote-changegroup%253Dhttp%252Chttps&cg=0&common=0000000000000000000000000000000000000000&heads=c17445101a72edac06facd130d14808dfbd5c7c2&stream=1"
200 Script output follows 200 Script output follows
content-type: application/mercurial-0.2 content-type: application/mercurial-0.2
$ f --size --hex --bytes 256 body $ f --size --hex --bytes 256 body
body: size=112328 body: size=112318
0000: 04 6e 6f 6e 65 48 47 32 30 00 00 00 00 00 00 00 |.noneHG20.......| 0000: 04 6e 6f 6e 65 48 47 32 30 00 00 00 00 00 00 00 |.noneHG20.......|
0010: 72 06 53 54 52 45 41 4d 00 00 00 00 04 00 09 05 |r.STREAM........| 0010: 68 07 53 54 52 45 41 4d 32 00 00 00 00 03 00 09 |h.STREAM2.......|
0020: 09 04 0c 2d 07 02 62 79 74 65 63 6f 75 6e 74 39 |...-..bytecount9| 0020: 05 09 04 0c 2d 62 79 74 65 63 6f 75 6e 74 39 38 |....-bytecount98|
0030: 38 37 35 38 66 69 6c 65 63 6f 75 6e 74 31 30 33 |8758filecount103| 0030: 37 35 38 66 69 6c 65 63 6f 75 6e 74 31 30 33 30 |758filecount1030|
0040: 30 72 65 71 75 69 72 65 6d 65 6e 74 73 64 6f 74 |0requirementsdot| 0040: 72 65 71 75 69 72 65 6d 65 6e 74 73 64 6f 74 65 |requirementsdote|
0050: 65 6e 63 6f 64 65 20 66 6e 63 61 63 68 65 20 67 |encode fncache g| 0050: 6e 63 6f 64 65 20 66 6e 63 61 63 68 65 20 67 65 |ncode fncache ge|
0060: 65 6e 65 72 61 6c 64 65 6c 74 61 20 72 65 76 6c |eneraldelta revl| 0060: 6e 65 72 61 6c 64 65 6c 74 61 20 72 65 76 6c 6f |neraldelta revlo|
0070: 6f 67 76 31 20 73 74 6f 72 65 76 65 72 73 69 6f |ogv1 storeversio| 0070: 67 76 31 20 73 74 6f 72 65 00 00 10 00 73 08 42 |gv1 store....s.B|
0080: 6e 76 32 00 00 10 00 73 08 42 64 61 74 61 2f 30 |nv2....s.Bdata/0| 0080: 64 61 74 61 2f 30 2e 69 00 03 00 01 00 00 00 00 |data/0.i........|
0090: 2e 69 00 03 00 01 00 00 00 00 00 00 00 02 00 00 |.i..............| 0090: 00 00 00 02 00 00 00 01 00 00 00 00 00 00 00 01 |................|
00a0: 00 01 00 00 00 00 00 00 00 01 ff ff ff ff ff ff |................| 00a0: ff ff ff ff ff ff ff ff 80 29 63 a0 49 d3 23 87 |.........)c.I.#.|
00b0: ff ff 80 29 63 a0 49 d3 23 87 bf ce fe 56 67 92 |...)c.I.#....Vg.| 00b0: bf ce fe 56 67 92 67 2c 69 d1 ec 39 00 00 00 00 |...Vg.g,i..9....|
00c0: 67 2c 69 d1 ec 39 00 00 00 00 00 00 00 00 00 00 |g,i..9..........| 00c0: 00 00 00 00 00 00 00 00 75 30 73 08 42 64 61 74 |........u0s.Bdat|
00d0: 00 00 75 30 73 08 42 64 61 74 61 2f 31 2e 69 00 |..u0s.Bdata/1.i.| 00d0: 61 2f 31 2e 69 00 03 00 01 00 00 00 00 00 00 00 |a/1.i...........|
00e0: 03 00 01 00 00 00 00 00 00 00 02 00 00 00 01 00 |................| 00e0: 02 00 00 00 01 00 00 00 00 00 00 00 01 ff ff ff |................|
00f0: 00 00 00 00 00 00 01 ff ff ff ff ff ff ff ff f9 |................| 00f0: ff ff ff ff ff f9 76 da 1d 0d f2 25 6c de 08 db |......v....%l...|
--uncompressed is an alias to --stream --uncompressed is an alias to --stream
#if stream-legacy #if stream-legacy
$ hg clone --uncompressed -U http://localhost:$HGPORT clone1-uncompressed $ hg clone --uncompressed -U http://localhost:$HGPORT clone1-uncompressed
streaming all changes streaming all changes
1027 files to transfer, 96.3 KB of data 1027 files to transfer, 96.3 KB of data
transferred 96.3 KB in * seconds (*/sec) (glob) transferred 96.3 KB in * seconds (*/sec) (glob)
$ hg --debug --config worker.backgroundclose=true --config worker.backgroundcloseminfilecount=1 clone --stream -U http://localhost:$HGPORT clone-background | grep -v adding $ hg --debug --config worker.backgroundclose=true --config worker.backgroundcloseminfilecount=1 clone --stream -U http://localhost:$HGPORT clone-background | grep -v adding
using http://localhost:$HGPORT/ using http://localhost:$HGPORT/
sending capabilities command sending capabilities command
query 1; heads query 1; heads
sending batch command sending batch command
streaming all changes streaming all changes
sending getbundle command sending getbundle command
bundle2-input-bundle: with-transaction bundle2-input-bundle: with-transaction
bundle2-input-part: "stream" (params: 4 mandatory) supported bundle2-input-part: "stream2" (params: 3 mandatory) supported
applying stream bundle applying stream bundle
1030 files to transfer, 96.4 KB of data 1030 files to transfer, 96.4 KB of data
starting 4 threads for background file closing starting 4 threads for background file closing
starting 4 threads for background file closing starting 4 threads for background file closing
transferred 96.4 KB in * seconds (* */sec) (glob) transferred 96.4 KB in * seconds (* */sec) (glob)
bundle2-input-part: total payload size 112077 bundle2-input-part: total payload size 112077
bundle2-input-part: "listkeys" (params: 1 mandatory) supported bundle2-input-part: "listkeys" (params: 1 mandatory) supported
bundle2-input-bundle: 1 parts total bundle2-input-bundle: 1 parts total