This is an archive of the discontinued Mercurial Phabricator instance.

Differential D3532

wireprotov2: define and implement "rawstorefile" command
AbandonedPublic

Authored by indygreg on May 11 2018, 6:35 PM.

Details

Reviewers
None
Group Reviewers
hg-reviewers
Summary

stream_out - the previous command for sending raw revlog files -
was not carried forward to protocol version 2.

This commit introduces a minimal viable replacement for stream_out
in wire protocol version 2.

The new command allows obtaining "raw store files" - essentially
files from Mercurial's store as they exist on disk.

The command currently only allows obtaining the changelog or the
changelog plus root manifestlog. This is the feature set required
to support partial clones where only files data is partial.

We'll probably want to implement support for retrieving changelog
and manifest data via dedicated commands in order to facilitate
partial clone. And if we do decide to keep a command for streaming
"raw" files, we'll want to support tree manifests. This command
is very much a minimum viable implementation. I foresee things
changing substantially. None of wire protocol version 2 is
covered by BC yet. So hopefully the barrier to entry is low.

Diff Detail

Repository
rHG Mercurial
Lint
Lint Skipped
Unit
Unit Tests Skipped

Event Timeline

indygreg created this revision.May 11 2018, 6:35 PM
martinvonz added a subscriber: martinvonz.May 11 2018, 8:21 PM
martinvonz added inline comments.
mercurial/wireprotov2server.py
548–549

I understand that you don't want to lock the repo for the entire operation, but I assume that also means that the result may fail hg verify? If changelog is always sent before the manifest (is it?), then you might have some orphan entries in the manifest if it had been written while the changelog was read.

554

tree manifest support might be as easy as also iterating over data files, stripping leading 'meta/' and trailing '00manifest.[id]' and testing against repo.narrowmatch().visitdir()

joerg.sonnenberger added a subscriber: joerg.sonnenberger.May 12 2018, 1:35 PM

Similar to Martin's question, I would like to allow streaming clones without any locks. For that to work, one party needs to know how to truncate additional undesired data. That can be fully done by the client or the client could send a size or revision hint to the server.

indygreg added a comment.May 12 2018, 2:13 PM

In order to support streaming clone without any locks *and* for the result of that clone to pass hg verify with no warnings about unreferenced revisions (assuming the server was clean to begin with), I believe we would need to scan the changelog for all referenced manifest nodes and then find the end offset of the last node in the manifest. We would then send the manifest up to that offset.

Then for filelogs, we would need to do something similar for every file.

This doesn't scale.

I see the following solutions to this problem:

  1. Obtaining a lock, determining file sizes, and only streaming files up to their known sizes. (This is the current solution.)
  2. Do not obtain a lock, send all manifest and filelog data. This potentially results the client receiving extra manifest and file revisions if the server was in the middle of a transaction when obtaining data. Also, there are race conditions involving a rollback that we'd need to worry about.
  3. Tracking the offsets of all files up to the last transaction so they can be obtained with a lock.

Anyway, I'm not a super big fan of *stream clones*. Their existence is a glorified hack to make clones faster. Their existence is a massive layering violation because it makes the server's storage implementation the client's. This *can* be useful. But I'd rather focus on making normal [partial] clones fast.

I'm considering dropping this command from the series and implementing commands to obtain changeset and manifest data. Then we could implement partial clones without *stream clones* and avoid this debate :)

lothiraldan added a subscriber: lothiraldan.May 14 2018, 4:56 AM

First, thank you for your work on the new wire protocol.

We used to send cache file also in streaming clone with the V1 wire protocol. Do you think we would use the new rawstorefile command for this purpose? If so, the command name might be confusing as cache files reside outside the store directory.

You were suggesting dropping support for stream clones. No matter its design, do you have ideas how to be as fast as stream clones? Stream clones make cloning huge repositories hours faster than traditional clones.

indygreg added a comment.May 14 2018, 12:01 PM

I'm not yet sure what will be done with stream clones. There's a good chance the existing approach more or less gets carried forward. I do concede that it is pretty optimal and we'll have a hard time reproducing its performance.

durin42 added a subscriber: durin42.May 31 2018, 3:39 PM

I'm...not thrilled by the abstraction leak in this, but as long as it's strictly temporary on the path to saner partial clones I can live with it.

mercurial/help/internals/wireprotocol.txt
1935

Having this here makes it infeasible to generate the revlog files on the fly, which means non-traditional storage backends won't ever be able to implement this endpoint efficiently.

Do we care? Should we note that this method is going to go away in all likelihood?

mercurial/wireprotov2server.py
548–549

By grabbing the size along with the list of files inside the lock, that won't happen. :)

indygreg abandoned this revision.Aug 21 2018, 4:23 PM

I'll be taking a different approach to partial clone and the wire protocol in an upcoming series. These patches may get revived someday. But not now.

Revision Contents
Changeset List

Diff 8643

mercurial/help/internals/wireprotocol.txt

(bytestring) The pushkey key to set. (bytestring) The pushkey key to set.
old old
(bytestring) Old value for this key. (bytestring) Old value for this key.
new new
(bytestring) New value for this key. (bytestring) New value for this key.
TODO consider using binary to represent nodes is certain pushkey namespaces. TODO consider using binary to represent nodes is certain pushkey namespaces.
TODO better define response type and meaning. TODO better define response type and meaning.
rawstorefile
------------
Obtain raw files from the server.
This command is used to quickly obtain files in the repository store
from the remote with minimal additional processing.
The command receives the following arguments:
what
(bytestring) Type of query to perform. Can be ``changelog`` or
``changelog+rootmanifestlog``.
TODO need a mechanism to consider treemanifests
The response begins with a CBOR map containing the following keys:
filecount
(unsigned integer) Number of files being included in the payload.
totalsize
durin42Unsubmitted
Not Done

Having this here makes it infeasible to generate the revlog files on the fly, which means non-traditional storage backends won't ever be able to implement this endpoint efficiently.

Do we care? Should we note that this method is going to go away in all likelihood?

durin42: Having this here makes it infeasible to generate the revlog files on the fly, which means non…
(unsigned integer) Total size of all transferred file data, in bytes.
Following this map are N file segments. Each file segment consists of
a CBOR map followed by a CBOR bytestring, possibly of indefinite length.
Each CBOR map contains the following bytestring keys:
name
(bytestring) Path of file being transferred. Path is the raw store
path and can be any sequence of bytes that can be tracked in a
Mercurial manifest.
size
(unsigned integer) Size of file data. This will be the final written
file size. The total size of the data that follows the CBOR map
will be greater due to encoding overhead of CBOR.

mercurial/wireprotov2server.py

error, error,
pycompat, pycompat,
streamclone, streamclone,
util, util,
wireprotoframing, wireprotoframing,
wireprototypes, wireprototypes,
) )
from .utils import ( from .utils import (
cborutil,
interfaceutil, interfaceutil,
) )
FRAMINGTYPE = b'application/mercurial-exp-framing-0005' FRAMINGTYPE = b'application/mercurial-exp-framing-0005'
HTTP_WIREPROTO_V2 = wireprototypes.HTTP_WIREPROTO_V2 HTTP_WIREPROTO_V2 = wireprototypes.HTTP_WIREPROTO_V2
COMMANDS = wireprototypes.commanddict() COMMANDS = wireprototypes.commanddict()
def pushkeyv2(repo, proto, namespace, key, old, new): def pushkeyv2(repo, proto, namespace, key, old, new):
# TODO handle ui output redirection # TODO handle ui output redirection
r = repo.pushkey(encoding.tolocal(namespace), r = repo.pushkey(encoding.tolocal(namespace),
encoding.tolocal(key), encoding.tolocal(key),
encoding.tolocal(old), encoding.tolocal(old),
encoding.tolocal(new)) encoding.tolocal(new))
return wireprototypes.cborresponse(r) return wireprototypes.cborresponse(r)
@wireprotocommand('rawstorefile',
args={
'what': b'changelog',
},
permission='pull')
def rawstorefile(repo, proto, what=None):
if what not in ('changelog', 'changelog+rootmanifestlog'):
return wireprototypes.v2errorresponse(
'illegal value for "what" argument (%s): must be "changelog" '
'or "changelog+rootmanifestlog"', (what,))
with repo.lock():
topfiles = list(repo.store.topfiles())
martinvonzUnsubmitted
Not Done

I understand that you don't want to lock the repo for the entire operation, but I assume that also means that the result may fail hg verify? If changelog is always sent before the manifest (is it?), then you might have some orphan entries in the manifest if it had been written while the changelog was read.

martinvonz: I understand that you don't want to lock the repo for the entire operation, but I assume that…
durin42Unsubmitted
Not Done

By grabbing the size along with the list of files inside the lock, that won't happen. :)

durin42: By grabbing the size along with the list of files inside the lock, that won't happen. :)
sendfiles = []
totalsize = 0
for name, encodedname, size in topfiles:
martinvonzUnsubmitted
Not Done

tree manifest support might be as easy as also iterating over data files, stripping leading 'meta/' and trailing '00manifest.[id]' and testing against repo.narrowmatch().visitdir()

martinvonz: tree manifest support might be as easy as also iterating over data files, stripping leading…
if what.startswith(b'changelog') and name.startswith(b'00changelog'):
pass
elif (what == b'changelog+rootmanifestlog' and
name.startswith(b'00manifest')):
pass
else:
continue
sendfiles.append((name, encodedname, size))
totalsize += size
def senddata():
chunks = cborutil.streamencodemap({
b'filecount': len(sendfiles),
b'totalsize': totalsize,
})
for chunk in chunks:
yield chunk
for name, encodedname, size in sendfiles:
chunks = cborutil.streamencodemap({
b'name': encodedname,
b'size': size,
})
for chunk in chunks:
yield chunk
with repo.svfs(name, 'rb', auditpath=False) as fh:
if size <= 65536:
chunks = [fh.read(size)]
else:
chunks = util.filechunkiter(fh, limit=size)
for chunk in cborutil.streamencodebytestringfromiter(chunks):
yield chunk
# We assume we're serving revlogs and that they don't compress well.
return wireprototypes.v2streamingresponse(senddata(), compressible=False)

tests/test-http-protocol.t

s> \r\n s> \r\n
s> makefile('rb', None) s> makefile('rb', None)
s> HTTP/1.1 200 OK\r\n s> HTTP/1.1 200 OK\r\n
s> Server: testing stub value\r\n s> Server: testing stub value\r\n
s> Date: $HTTP_DATE$\r\n s> Date: $HTTP_DATE$\r\n
s> Content-Type: application/mercurial-cbor\r\n s> Content-Type: application/mercurial-cbor\r\n
s> Content-Length: *\r\n (glob) s> Content-Length: *\r\n (glob)
s> \r\n s> \r\n
s> \xa3Dapis\xa1Pexp-http-v2-0001\xa4Hcommands\xa7Eheads\xa2Dargs\xa1Jpubliconly\xf4Kpermissions\x81DpullEknown\xa2Dargs\xa1Enodes\x81HdeadbeefKpermissions\x81DpullFlookup\xa2Dargs\xa1CkeyCfooKpermissions\x81DpullGpushkey\xa2Dargs\xa4CkeyCkeyCnewCnewColdColdInamespaceBnsKpermissions\x81DpushHlistkeys\xa2Dargs\xa1InamespaceBnsKpermissions\x81DpullIbranchmap\xa2Dargs\xa0Kpermissions\x81DpullLcapabilities\xa2Dargs\xa0Kpermissions\x81DpullKcompression\x81\xa1DnameDzlibNrawrepoformats\x82LgeneraldeltaHrevlogv1Qframingmediatypes\x81X&application/mercurial-exp-framing-0005GapibaseDapi/Nv1capabilitiesY\x01\xc5batch branchmap $USUAL_BUNDLE2_CAPS_SERVER$ changegroupsubset compression=$BUNDLE2_COMPRESSIONS$ getbundle httpheader=1024 httpmediatype=0.1rx,0.1tx,0.2tx known lookup pushkey streamreqs=generaldelta,revlogv1 unbundle=HG10GZ,HG10BZ,HG10UN unbundlehash s> \xa3Dapis\xa1Pexp-http-v2-0001\xa4Hcommands\xa8Eheads\xa2Dargs\xa1Jpubliconly\xf4Kpermissions\x81DpullEknown\xa2Dargs\xa1Enodes\x81HdeadbeefKpermissions\x81DpullFlookup\xa2Dargs\xa1CkeyCfooKpermissions\x81DpullGpushkey\xa2Dargs\xa4CkeyCkeyCnewCnewColdColdInamespaceBnsKpermissions\x81DpushHlistkeys\xa2Dargs\xa1InamespaceBnsKpermissions\x81DpullIbranchmap\xa2Dargs\xa0Kpermissions\x81DpullLcapabilities\xa2Dargs\xa0Kpermissions\x81DpullLrawstorefile\xa2Dargs\xa1DwhatIchangelogKpermissions\x81DpullKcompression\x81\xa1DnameDzlibNrawrepoformats\x82LgeneraldeltaHrevlogv1Qframingmediatypes\x81X&application/mercurial-exp-framing-0005GapibaseDapi/Nv1capabilitiesY\x01\xc5batch branchmap $USUAL_BUNDLE2_CAPS_SERVER$ changegroupsubset compression=$BUNDLE2_COMPRESSIONS$ getbundle httpheader=1024 httpmediatype=0.1rx,0.1tx,0.2tx known lookup pushkey streamreqs=generaldelta,revlogv1 unbundle=HG10GZ,HG10BZ,HG10UN unbundlehash
sending heads command sending heads command
s> POST /api/exp-http-v2-0001/ro/heads HTTP/1.1\r\n s> POST /api/exp-http-v2-0001/ro/heads HTTP/1.1\r\n
s> Accept-Encoding: identity\r\n s> Accept-Encoding: identity\r\n
s> accept: application/mercurial-exp-framing-0005\r\n s> accept: application/mercurial-exp-framing-0005\r\n
s> content-type: application/mercurial-exp-framing-0005\r\n s> content-type: application/mercurial-exp-framing-0005\r\n
s> content-length: 20\r\n s> content-length: 20\r\n
s> host: $LOCALIP:$HGPORT\r\n (glob) s> host: $LOCALIP:$HGPORT\r\n (glob)
s> user-agent: Mercurial debugwireproto\r\n s> user-agent: Mercurial debugwireproto\r\n

tests/test-wireproto-command-capabilities.t

s> \r\n s> \r\n
s> makefile('rb', None) s> makefile('rb', None)
s> HTTP/1.1 200 OK\r\n s> HTTP/1.1 200 OK\r\n
s> Server: testing stub value\r\n s> Server: testing stub value\r\n
s> Date: $HTTP_DATE$\r\n s> Date: $HTTP_DATE$\r\n
s> Content-Type: application/mercurial-cbor\r\n s> Content-Type: application/mercurial-cbor\r\n
s> Content-Length: *\r\n (glob) s> Content-Length: *\r\n (glob)
s> \r\n s> \r\n
s> \xa3Dapis\xa1Pexp-http-v2-0001\xa4Hcommands\xa7Eheads\xa2Dargs\xa1Jpubliconly\xf4Kpermissions\x81DpullEknown\xa2Dargs\xa1Enodes\x81HdeadbeefKpermissions\x81DpullFlookup\xa2Dargs\xa1CkeyCfooKpermissions\x81DpullGpushkey\xa2Dargs\xa4CkeyCkeyCnewCnewColdColdInamespaceBnsKpermissions\x81DpushHlistkeys\xa2Dargs\xa1InamespaceBnsKpermissions\x81DpullIbranchmap\xa2Dargs\xa0Kpermissions\x81DpullLcapabilities\xa2Dargs\xa0Kpermissions\x81DpullKcompression\x81\xa1DnameDzlibNrawrepoformats\x82LgeneraldeltaHrevlogv1Qframingmediatypes\x81X&application/mercurial-exp-framing-0005GapibaseDapi/Nv1capabilitiesY\x01\xc5batch branchmap $USUAL_BUNDLE2_CAPS_SERVER$ changegroupsubset compression=$BUNDLE2_COMPRESSIONS$ getbundle httpheader=1024 httpmediatype=0.1rx,0.1tx,0.2tx known lookup pushkey streamreqs=generaldelta,revlogv1 unbundle=HG10GZ,HG10BZ,HG10UN unbundlehash s> \xa3Dapis\xa1Pexp-http-v2-0001\xa4Hcommands\xa8Eheads\xa2Dargs\xa1Jpubliconly\xf4Kpermissions\x81DpullEknown\xa2Dargs\xa1Enodes\x81HdeadbeefKpermissions\x81DpullFlookup\xa2Dargs\xa1CkeyCfooKpermissions\x81DpullGpushkey\xa2Dargs\xa4CkeyCkeyCnewCnewColdColdInamespaceBnsKpermissions\x81DpushHlistkeys\xa2Dargs\xa1InamespaceBnsKpermissions\x81DpullIbranchmap\xa2Dargs\xa0Kpermissions\x81DpullLcapabilities\xa2Dargs\xa0Kpermissions\x81DpullLrawstorefile\xa2Dargs\xa1DwhatIchangelogKpermissions\x81DpullKcompression\x81\xa1DnameDzlibNrawrepoformats\x82LgeneraldeltaHrevlogv1Qframingmediatypes\x81X&application/mercurial-exp-framing-0005GapibaseDapi/Nv1capabilitiesY\x01\xc5batch branchmap $USUAL_BUNDLE2_CAPS_SERVER$ changegroupsubset compression=$BUNDLE2_COMPRESSIONS$ getbundle httpheader=1024 httpmediatype=0.1rx,0.1tx,0.2tx known lookup pushkey streamreqs=generaldelta,revlogv1 unbundle=HG10GZ,HG10BZ,HG10UN unbundlehash
cbor> {b'apibase': b'api/', b'apis': {b'exp-http-v2-0001': {b'commands': {b'branchmap': {b'args': {}, b'permissions': [b'pull']}, b'capabilities': {b'args': {}, b'permissions': [b'pull']}, b'heads': {b'args': {b'publiconly': False}, b'permissions': [b'pull']}, b'known': {b'args': {b'nodes': [b'deadbeef']}, b'permissions': [b'pull']}, b'listkeys': {b'args': {b'namespace': b'ns'}, b'permissions': [b'pull']}, b'lookup': {b'args': {b'key': b'foo'}, b'permissions': [b'pull']}, b'pushkey': {b'args': {b'key': b'key', b'namespace': b'ns', b'new': b'new', b'old': b'old'}, b'permissions': [b'push']}}, b'compression': [{b'name': b'zlib'}], b'framingmediatypes': [b'application/mercurial-exp-framing-0005'], b'rawrepoformats': [b'generaldelta', b'revlogv1']}}, b'v1capabilities': b'batch branchmap $USUAL_BUNDLE2_CAPS_SERVER$ changegroupsubset compression=$BUNDLE2_COMPRESSIONS$ getbundle httpheader=1024 httpmediatype=0.1rx,0.1tx,0.2tx known lookup pushkey streamreqs=generaldelta,revlogv1 unbundle=HG10GZ,HG10BZ,HG10UN unbundlehash'} cbor> {b'apibase': b'api/', b'apis': {b'exp-http-v2-0001': {b'commands': {b'branchmap': {b'args': {}, b'permissions': [b'pull']}, b'capabilities': {b'args': {}, b'permissions': [b'pull']}, b'heads': {b'args': {b'publiconly': False}, b'permissions': [b'pull']}, b'known': {b'args': {b'nodes': [b'deadbeef']}, b'permissions': [b'pull']}, b'listkeys': {b'args': {b'namespace': b'ns'}, b'permissions': [b'pull']}, b'lookup': {b'args': {b'key': b'foo'}, b'permissions': [b'pull']}, b'pushkey': {b'args': {b'key': b'key', b'namespace': b'ns', b'new': b'new', b'old': b'old'}, b'permissions': [b'push']}, b'rawstorefile': {b'args': {b'what': b'changelog'}, b'permissions': [b'pull']}}, b'compression': [{b'name': b'zlib'}], b'framingmediatypes': [b'application/mercurial-exp-framing-0005'], b'rawrepoformats': [b'generaldelta', b'revlogv1']}}, b'v1capabilities': b'batch branchmap $USUAL_BUNDLE2_CAPS_SERVER$ changegroupsubset compression=$BUNDLE2_COMPRESSIONS$ getbundle httpheader=1024 httpmediatype=0.1rx,0.1tx,0.2tx known lookup pushkey streamreqs=generaldelta,revlogv1 unbundle=HG10GZ,HG10BZ,HG10UN unbundlehash'}
capabilities command returns expected info capabilities command returns expected info
$ sendhttpv2peerhandshake << EOF $ sendhttpv2peerhandshake << EOF
> command capabilities > command capabilities
> EOF > EOF
creating http peer for wire protocol version 2 creating http peer for wire protocol version 2
s> GET /?cmd=capabilities HTTP/1.1\r\n s> GET /?cmd=capabilities HTTP/1.1\r\n
s> Accept-Encoding: identity\r\n s> Accept-Encoding: identity\r\n
s> vary: X-HgProto-1,X-HgUpgrade-1\r\n s> vary: X-HgProto-1,X-HgUpgrade-1\r\n
s> x-hgproto-1: cbor\r\n s> x-hgproto-1: cbor\r\n
s> x-hgupgrade-1: exp-http-v2-0001\r\n s> x-hgupgrade-1: exp-http-v2-0001\r\n
s> accept: application/mercurial-0.1\r\n s> accept: application/mercurial-0.1\r\n
s> host: $LOCALIP:$HGPORT\r\n (glob) s> host: $LOCALIP:$HGPORT\r\n (glob)
s> user-agent: Mercurial debugwireproto\r\n s> user-agent: Mercurial debugwireproto\r\n
s> \r\n s> \r\n
s> makefile('rb', None) s> makefile('rb', None)
s> HTTP/1.1 200 OK\r\n s> HTTP/1.1 200 OK\r\n
s> Server: testing stub value\r\n s> Server: testing stub value\r\n
s> Date: $HTTP_DATE$\r\n s> Date: $HTTP_DATE$\r\n
s> Content-Type: application/mercurial-cbor\r\n s> Content-Type: application/mercurial-cbor\r\n
s> Content-Length: *\r\n (glob) s> Content-Length: *\r\n (glob)
s> \r\n s> \r\n
s> \xa3Dapis\xa1Pexp-http-v2-0001\xa4Hcommands\xa7Eheads\xa2Dargs\xa1Jpubliconly\xf4Kpermissions\x81DpullEknown\xa2Dargs\xa1Enodes\x81HdeadbeefKpermissions\x81DpullFlookup\xa2Dargs\xa1CkeyCfooKpermissions\x81DpullGpushkey\xa2Dargs\xa4CkeyCkeyCnewCnewColdColdInamespaceBnsKpermissions\x81DpushHlistkeys\xa2Dargs\xa1InamespaceBnsKpermissions\x81DpullIbranchmap\xa2Dargs\xa0Kpermissions\x81DpullLcapabilities\xa2Dargs\xa0Kpermissions\x81DpullKcompression\x81\xa1DnameDzlibNrawrepoformats\x82LgeneraldeltaHrevlogv1Qframingmediatypes\x81X&application/mercurial-exp-framing-0005GapibaseDapi/Nv1capabilitiesY\x01\xc5batch branchmap $USUAL_BUNDLE2_CAPS_SERVER$ changegroupsubset compression=$BUNDLE2_COMPRESSIONS$ getbundle httpheader=1024 httpmediatype=0.1rx,0.1tx,0.2tx known lookup pushkey streamreqs=generaldelta,revlogv1 unbundle=HG10GZ,HG10BZ,HG10UN unbundlehash s> \xa3Dapis\xa1Pexp-http-v2-0001\xa4Hcommands\xa8Eheads\xa2Dargs\xa1Jpubliconly\xf4Kpermissions\x81DpullEknown\xa2Dargs\xa1Enodes\x81HdeadbeefKpermissions\x81DpullFlookup\xa2Dargs\xa1CkeyCfooKpermissions\x81DpullGpushkey\xa2Dargs\xa4CkeyCkeyCnewCnewColdColdInamespaceBnsKpermissions\x81DpushHlistkeys\xa2Dargs\xa1InamespaceBnsKpermissions\x81DpullIbranchmap\xa2Dargs\xa0Kpermissions\x81DpullLcapabilities\xa2Dargs\xa0Kpermissions\x81DpullLrawstorefile\xa2Dargs\xa1DwhatIchangelogKpermissions\x81DpullKcompression\x81\xa1DnameDzlibNrawrepoformats\x82LgeneraldeltaHrevlogv1Qframingmediatypes\x81X&application/mercurial-exp-framing-0005GapibaseDapi/Nv1capabilitiesY\x01\xc5batch branchmap $USUAL_BUNDLE2_CAPS_SERVER$ changegroupsubset compression=$BUNDLE2_COMPRESSIONS$ getbundle httpheader=1024 httpmediatype=0.1rx,0.1tx,0.2tx known lookup pushkey streamreqs=generaldelta,revlogv1 unbundle=HG10GZ,HG10BZ,HG10UN unbundlehash
sending capabilities command sending capabilities command
s> POST /api/exp-http-v2-0001/ro/capabilities HTTP/1.1\r\n s> POST /api/exp-http-v2-0001/ro/capabilities HTTP/1.1\r\n
s> Accept-Encoding: identity\r\n s> Accept-Encoding: identity\r\n
s> *\r\n (glob) s> *\r\n (glob)
s> content-type: application/mercurial-exp-framing-0005\r\n s> content-type: application/mercurial-exp-framing-0005\r\n
s> content-length: 27\r\n s> content-length: 27\r\n
s> host: $LOCALIP:$HGPORT\r\n (glob) s> host: $LOCALIP:$HGPORT\r\n (glob)
s> user-agent: Mercurial debugwireproto\r\n s> user-agent: Mercurial debugwireproto\r\n
s> \r\n s> \r\n
s> \x13\x00\x00\x01\x00\x01\x01\x11\xa1DnameLcapabilities s> \x13\x00\x00\x01\x00\x01\x01\x11\xa1DnameLcapabilities
s> makefile('rb', None) s> makefile('rb', None)
s> HTTP/1.1 200 OK\r\n s> HTTP/1.1 200 OK\r\n
s> Server: testing stub value\r\n s> Server: testing stub value\r\n
s> Date: $HTTP_DATE$\r\n s> Date: $HTTP_DATE$\r\n
s> Content-Type: application/mercurial-exp-framing-0005\r\n s> Content-Type: application/mercurial-exp-framing-0005\r\n
s> Transfer-Encoding: chunked\r\n s> Transfer-Encoding: chunked\r\n
s> \r\n s> \r\n
s> 1d7\r\n s> 20c\r\n
s> \xcf\x01\x00\x01\x00\x02\x012 s> \x04\x02\x00\x01\x00\x02\x012
s> \xa1FstatusBok\xa4Hcommands\xa7Eheads\xa2Dargs\xa1Jpubliconly\xf4Kpermissions\x81DpullEknown\xa2Dargs\xa1Enodes\x81HdeadbeefKpermissions\x81DpullFlookup\xa2Dargs\xa1CkeyCfooKpermissions\x81DpullGpushkey\xa2Dargs\xa4CkeyCkeyCnewCnewColdColdInamespaceBnsKpermissions\x81DpushHlistkeys\xa2Dargs\xa1InamespaceBnsKpermissions\x81DpullIbranchmap\xa2Dargs\xa0Kpermissions\x81DpullLcapabilities\xa2Dargs\xa0Kpermissions\x81DpullKcompression\x81\xa1DnameDzlibNrawrepoformats\x82LgeneraldeltaHrevlogv1Qframingmediatypes\x81X&application/mercurial-exp-framing-0005 s> \xa1FstatusBok\xa4Hcommands\xa8Eheads\xa2Dargs\xa1Jpubliconly\xf4Kpermissions\x81DpullEknown\xa2Dargs\xa1Enodes\x81HdeadbeefKpermissions\x81DpullFlookup\xa2Dargs\xa1CkeyCfooKpermissions\x81DpullGpushkey\xa2Dargs\xa4CkeyCkeyCnewCnewColdColdInamespaceBnsKpermissions\x81DpushHlistkeys\xa2Dargs\xa1InamespaceBnsKpermissions\x81DpullIbranchmap\xa2Dargs\xa0Kpermissions\x81DpullLcapabilities\xa2Dargs\xa0Kpermissions\x81DpullLrawstorefile\xa2Dargs\xa1DwhatIchangelogKpermissions\x81DpullKcompression\x81\xa1DnameDzlibNrawrepoformats\x82LgeneraldeltaHrevlogv1Qframingmediatypes\x81X&application/mercurial-exp-framing-0005
s> \r\n s> \r\n
received frame(size=463; request=1; stream=2; streamflags=stream-begin; type=command-response; flags=eos) received frame(size=516; request=1; stream=2; streamflags=stream-begin; type=command-response; flags=eos)
s> 0\r\n s> 0\r\n
s> \r\n s> \r\n
response: [{b'status': b'ok'}, {b'commands': {b'branchmap': {b'args': {}, b'permissions': [b'pull']}, b'capabilities': {b'args': {}, b'permissions': [b'pull']}, b'heads': {b'args': {b'publiconly': False}, b'permissions': [b'pull']}, b'known': {b'args': {b'nodes': [b'deadbeef']}, b'permissions': [b'pull']}, b'listkeys': {b'args': {b'namespace': b'ns'}, b'permissions': [b'pull']}, b'lookup': {b'args': {b'key': b'foo'}, b'permissions': [b'pull']}, b'pushkey': {b'args': {b'key': b'key', b'namespace': b'ns', b'new': b'new', b'old': b'old'}, b'permissions': [b'push']}}, b'compression': [{b'name': b'zlib'}], b'framingmediatypes': [b'application/mercurial-exp-framing-0005'], b'rawrepoformats': [b'generaldelta', b'revlogv1']}] response: [{b'status': b'ok'}, {b'commands': {b'branchmap': {b'args': {}, b'permissions': [b'pull']}, b'capabilities': {b'args': {}, b'permissions': [b'pull']}, b'heads': {b'args': {b'publiconly': False}, b'permissions': [b'pull']}, b'known': {b'args': {b'nodes': [b'deadbeef']}, b'permissions': [b'pull']}, b'listkeys': {b'args': {b'namespace': b'ns'}, b'permissions': [b'pull']}, b'lookup': {b'args': {b'key': b'foo'}, b'permissions': [b'pull']}, b'pushkey': {b'args': {b'key': b'key', b'namespace': b'ns', b'new': b'new', b'old': b'old'}, b'permissions': [b'push']}, b'rawstorefile': {b'args': {b'what': b'changelog'}, b'permissions': [b'pull']}}, b'compression': [{b'name': b'zlib'}], b'framingmediatypes': [b'application/mercurial-exp-framing-0005'], b'rawrepoformats': [b'generaldelta', b'revlogv1']}]
$ cat error.log $ cat error.log

tests/test-wireproto-command-rawstorefile.t

  • This file was added.
$ . $TESTDIR/wireprotohelpers.sh
$ hg init server
$ enablehttpv2 server
$ cd server
$ echo a0 > a
$ echo b0 > b
$ echo c0 > c
$ mkdir dir0
$ echo d0 > dir0/d
$ echo e1 > dir0/e
$ hg -q commit -A -m initial
$ echo a1 > a
$ hg commit -m 'commit 2'
$ echo a2 > a
$ hg commit -m 'commit 3'
$ hg up -r 0
1 files updated, 0 files merged, 0 files removed, 0 files unresolved
$ echo a1-branches > a
$ hg commit -m 'commit 4 (new head)'
created new head
$ hg up 3
0 files updated, 0 files merged, 0 files removed, 0 files unresolved
$ hg mv c c-moved
$ hg commit -m 'commit 5 (moved c)'
$ cd ..
$ hg -R server serve -p $HGPORT -d --pid-file hg.pid -E error.log
$ cat hg.pid > $DAEMON_PIDS
Tests for the "rawstorefile" command
Invalid "what" value results in error
$ sendhttpv2peer << EOF
> command rawstorefile
> what badvalue
> EOF
creating http peer for wire protocol version 2
sending rawstorefile command
s> POST /api/exp-http-v2-0001/ro/rawstorefile HTTP/1.1\r\n
s> Accept-Encoding: identity\r\n
s> accept: application/mercurial-exp-framing-0005\r\n
s> content-type: application/mercurial-exp-framing-0005\r\n
s> content-length: 47\r\n
s> host: $LOCALIP:$HGPORT\r\n (glob)
s> user-agent: Mercurial debugwireproto\r\n
s> \r\n
s> \'\x00\x00\x01\x00\x01\x01\x11\xa2Dargs\xa1DwhatHbadvalueDnameLrawstorefile
s> makefile('rb', None)
s> HTTP/1.1 200 OK\r\n
s> Server: testing stub value\r\n
s> Date: $HTTP_DATE$\r\n
s> Content-Type: application/mercurial-exp-framing-0005\r\n
s> Transfer-Encoding: chunked\r\n
s> \r\n
s> 90\r\n
s> \x88\x00\x00\x01\x00\x02\x012
s> \xa2Eerror\xa2Dargs\x81HbadvalueGmessageXZillegal value for "what" argument (%s): must be "changelog" or "changelog+rootmanifestlog"FstatusEerror
s> \r\n
received frame(size=136; request=1; stream=2; streamflags=stream-begin; type=command-response; flags=eos)
s> 0\r\n
s> \r\n
response: [{b'error': {b'args': [b'badvalue'], b'message': b'illegal value for "what" argument (%s): must be "changelog" or "changelog+rootmanifestlog"'}, b'status': b'error'}]
Obtaining just the changelog works
$ sendhttpv2peer << EOF
> command rawstorefile
> what changelog
> EOF
creating http peer for wire protocol version 2
sending rawstorefile command
s> POST /api/exp-http-v2-0001/ro/rawstorefile HTTP/1.1\r\n
s> Accept-Encoding: identity\r\n
s> accept: application/mercurial-exp-framing-0005\r\n
s> content-type: application/mercurial-exp-framing-0005\r\n
s> content-length: 48\r\n
s> host: $LOCALIP:$HGPORT\r\n (glob)
s> user-agent: Mercurial debugwireproto\r\n
s> \r\n
s> (\x00\x00\x01\x00\x01\x01\x11\xa2Dargs\xa1DwhatIchangelogDnameLrawstorefile
s> makefile('rb', None)
s> HTTP/1.1 200 OK\r\n
s> Server: testing stub value\r\n
s> Date: $HTTP_DATE$\r\n
s> Content-Type: application/mercurial-exp-framing-0005\r\n
s> Transfer-Encoding: chunked\r\n
s> \r\n
s> 13\r\n
s> \x0b\x00\x00\x01\x00\x02\x011
s> \xa1FstatusBok
s> \r\n
received frame(size=11; request=1; stream=2; streamflags=stream-begin; type=command-response; flags=continuation)
s> 2e4\r\n
s> \xdc\x02\x00\x01\x00\x02\x000
s> \xa2Ifilecount\x01Itotalsize\x19\x02\xa2\xa2DnameM00changelog.iDsize\x19\x02\xa2_Y\x02\xa2\x00\x01\x00\x01\x00\x00\x00\x00\x00\x00\x00M\x00\x00\x00N\x00\x00\x00\x00\x00\x00\x00\x00\xff\xff\xff\xff\xff\xff\xff\xff\xfd\x8dO6\xd3@\xdb\x7f\xa4Y\x10\x078k\xac\x0c\xc1\xb2#\xd0\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00x\x9c%\xc5\xbb\r
s> \x800\x0c\x05\xc0\xfeM\xc1\x0686!\xf18\xce\xc7\x92%D\x01\xd9_\x14\\s*f\xa7sME[\xa1,\xb3\xa9\xf8Q\x9d\x93\x0b[\xe6\xd4U\xbaQ\xc1\x9a\xef\x02m\x04CC\xc7\x88\x87\xf6\xf17\x81\xb8c\x85]\x1f+i\x15B\x00\x00\x00\x00\x00M\x00\x00\x00\x00\x00>\x00\x00\x00=\x00\x00\x00\x01\x00\x00\x00\x01\x00\x00\x00\x00\xff\xff\xff\xff\x04\x03\xe9K\x8c\xe4e\xa1\xf6\x99;\xb1\xe4`{\xbe\x17`V\xa7\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00u356327300696aeacd7285a8755cea8447b237070\n
s> test\n
s> 0 0\n
s> a\n
s> \n
s> commit 2\x00\x00\x00\x00\x00\x8b\x00\x00\x00\x00\x00>\x00\x00\x00=\x00\x00\x00\x02\x00\x00\x00\x02\x00\x00\x00\x01\xff\xff\xff\xff\x0e\xb5\x06\x83\x84\x08E\xf3\xa7R\x8a}\xf3+l\xe2\x1c1J\x19\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00u9ff26517c62e7379b7e07457817a1c9602aad8d6\n
s> test\n
s> 0 0\n
s> a\n
s> \n
s> commit 3\x00\x00\x00\x00\x00\xc9\x00\x00\x00\x00\x00I\x00\x00\x00H\x00\x00\x00\x03\x00\x00\x00\x03\x00\x00\x00\x00\xff\xff\xff\xff\x15_\xcd\xfdBs\xeb\xeb\\.T\xd1\xc8q\x1f\xfd\xc2\x993\xab\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00u642dcaababdeb2f602127859e230e7859ca538b3\n
s> test\n
s> 0 0\n
s> a\n
s> \n
s> commit 4 (new head)\x00\x00\x00\x00\x01\x12\x00\x00\x00\x00\x00P\x00\x00\x00O\x00\x00\x00\x04\x00\x00\x00\x04\x00\x00\x00\x03\xff\xff\xff\xff\x9eE\xd8%Y&G)\xb4r\xc5\x167\xde\xdc\xf0\x99>\xb0\x8a\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00u19e0c69382fe33ca7026a1e2dfe59524b27a2b8f\n
s> test\n
s> 0 0\n
s> c\n
s> c-moved\n
s> \n
s> commit 5 (moved c)\xff
s> \r\n
received frame(size=732; request=1; stream=2; streamflags=; type=command-response; flags=)
s> 8\r\n
s> \x00\x00\x00\x01\x00\x02\x002
s> \r\n
s> 0\r\n
s> \r\n
received frame(size=0; request=1; stream=2; streamflags=; type=command-response; flags=eos)
response: [{b'status': b'ok'}, {b'filecount': 1, b'totalsize': 674}, {b'name': b'00changelog.i', b'size': 674}, bytearray['\x00\x01\x00\x01\x00\x00\x00\x00\x00\x00\x00M\x00\x00\x00N\x00\x00\x00\x00\x00\x00\x00\x00\xff\xff\xff\xff\xff\xff\xff\xff\xfd\x8dO6\xd3@\xdb\x7f\xa4Y\x10\x078k\xac\x0c\xc1\xb2#\xd0\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00x\x9c%\xc5\xbb\r\x800\x0c\x05\xc0\xfeM\xc1\x0686!\xf18\xce\xc7\x92%D\x01\xd9_\x14\\s*f\xa7sME[\xa1,\xb3\xa9\xf8Q\x9d\x93\x0b[\xe6\xd4U\xbaQ\xc1\x9a\xef\x02m\x04CC\xc7\x88\x87\xf6\xf17\x81\xb8c\x85]\x1f+i\x15B\x00\x00\x00\x00\x00M\x00\x00\x00\x00\x00>\x00\x00\x00=\x00\x00\x00\x01\x00\x00\x00\x01\x00\x00\x00\x00\xff\xff\xff\xff\x04\x03\xe9K\x8c\xe4e\xa1\xf6\x99;\xb1\xe4`{\xbe\x17`V\xa7\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00u356327300696aeacd7285a8755cea8447b237070\ntest\n0 0\na\n\ncommit 2\x00\x00\x00\x00\x00\x8b\x00\x00\x00\x00\x00>\x00\x00\x00=\x00\x00\x00\x02\x00\x00\x00\x02\x00\x00\x00\x01\xff\xff\xff\xff\x0e\xb5\x06\x83\x84\x08E\xf3\xa7R\x8a}\xf3+l\xe2\x1c1J\x19\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00u9ff26517c62e7379b7e07457817a1c9602aad8d6\ntest\n0 0\na\n\ncommit 3\x00\x00\x00\x00\x00\xc9\x00\x00\x00\x00\x00I\x00\x00\x00H\x00\x00\x00\x03\x00\x00\x00\x03\x00\x00\x00\x00\xff\xff\xff\xff\x15_\xcd\xfdBs\xeb\xeb\\.T\xd1\xc8q\x1f\xfd\xc2\x993\xab\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00u642dcaababdeb2f602127859e230e7859ca538b3\ntest\n0 0\na\n\ncommit 4 (new head)\x00\x00\x00\x00\x01\x12\x00\x00\x00\x00\x00P\x00\x00\x00O\x00\x00\x00\x04\x00\x00\x00\x04\x00\x00\x00\x03\xff\xff\xff\xff\x9eE\xd8%Y&G)\xb4r\xc5\x167\xde\xdc\xf0\x99>\xb0\x8a\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00u19e0c69382fe33ca7026a1e2dfe59524b27a2b8f\ntest\n0 0\nc\nc-moved\n\ncommit 5 (moved c)']]
Obtaining changelog and the manifest works
$ sendhttpv2peer << EOF
> command rawstorefile
> what changelog+rootmanifestlog
> EOF
creating http peer for wire protocol version 2
sending rawstorefile command
s> POST /api/exp-http-v2-0001/ro/rawstorefile HTTP/1.1\r\n
s> Accept-Encoding: identity\r\n
s> accept: application/mercurial-exp-framing-0005\r\n
s> content-type: application/mercurial-exp-framing-0005\r\n
s> content-length: 65\r\n
s> host: $LOCALIP:$HGPORT\r\n (glob)
s> user-agent: Mercurial debugwireproto\r\n
s> \r\n
s> 9\x00\x00\x01\x00\x01\x01\x11\xa2Dargs\xa1DwhatX\x19changelog+rootmanifestlogDnameLrawstorefile
s> makefile('rb', None)
s> HTTP/1.1 200 OK\r\n
s> Server: testing stub value\r\n
s> Date: $HTTP_DATE$\r\n
s> Content-Type: application/mercurial-exp-framing-0005\r\n
s> Transfer-Encoding: chunked\r\n
s> \r\n
s> 13\r\n
s> \x0b\x00\x00\x01\x00\x02\x011
s> \xa1FstatusBok
s> \r\n
received frame(size=11; request=1; stream=2; streamflags=stream-begin; type=command-response; flags=continuation)
s> 5be\r\n
s> \xb6\x05\x00\x01\x00\x02\x000
s> \xa2Ifilecount\x02Itotalsize\x19\x05\\\xa2DnameL00manifest.iDsize\x19\x02\xba_Y\x02\xba\x00\x03\x00\x01\x00\x00\x00\x00\x00\x00\x00\x98\x00\x00\x00\xe1\x00\x00\x00\x00\x00\x00\x00\x00\xff\xff\xff\xff\xff\xff\xff\xff\x93\xaao(\x17\x9bpS\xeb\x93\xf4\x8f!\xf3*R\x1c\x93\xca\x07\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00x\x9c%\xcd1\x8e\x041\x08\x05Q\xc7}\x99\x01\xfc\xc1\xf6q\xc0\x804\xe9\xde?\xd8\x96&\x7f\xaa\xf2!\x81\n
s> Z\x93O\xb4\xd3Z\x0eri\xc2\xe1iz\xbc\x9a\xd2!\xe9O\x8c\xcd\xa7DwNv-62\x93\xd3\xaf\x8a8$\xce!\\\x05a{\xee8\x0c@\'\xcc\xe9:\xbd.\x12\xebVhv\xd1*$n\x8b\x12?\xf9\xfd\xa3O\x0e\x9d[\xc8\xf2\x9e\xc5\xa5\xc2\n
s> J\xdb\x98\x1e]\x96\xefjJ[B\xec\xe7k\xec\x15\xcau\xde\xb2Z\x91w\xc3H\xfd\xf2\xcd\xd5\xda\x9b\x80;9=\x9e\x7f\xb8\xcc:\xbf\x00\x00\x00\x00\x00\x98\x00\x00\x00\x00\x007\x00\x00\x00\xe1\x00\x00\x00\x00\x00\x00\x00\x01\x00\x00\x00\x00\xff\xff\xff\xff5c\'0\x06\x96\xae\xac\xd7(Z\x87U\xce\xa8D{#pp\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00+\x00\x00\x00+a\x009a38122997b3ac97be2a9aa2e556838341fdf2cc\n
s> \x00\x00\x00\x00\x00\xcf\x00\x00\x00\x00\x007\x00\x00\x00\xe1\x00\x00\x00\x01\x00\x00\x00\x02\x00\x00\x00\x01\xff\xff\xff\xff\x9f\xf2e\x17\xc6.sy\xb7\xe0tW\x81z\x1c\x96\x02\xaa\xd8\xd6\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00+\x00\x00\x00+a\x00c2a205c8b2ade24af26062e53cd5bc3801d660da\n
s> \x00\x00\x00\x00\x01\x06\x00\x00\x00\x00\x007\x00\x00\x00\xe1\x00\x00\x00\x00\x00\x00\x00\x03\x00\x00\x00\x00\xff\xff\xff\xffd-\xca\xab\xab\xde\xb2\xf6\x02\x12xY\xe20\xe7\x85\x9c\xa58\xb3\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00+\x00\x00\x00+a\x0036b617abfb829883cdd60767d165ea454dd81726\n
s> \x00\x00\x00\x00\x01=\x00\x00\x00\x00\x00=\x00\x00\x00\xe7\x00\x00\x00\x03\x00\x00\x00\x04\x00\x00\x00\x03\xff\xff\xff\xff\x19\xe0\xc6\x93\x82\xfe3\xcap&\xa1\xe2\xdf\xe5\x95$\xb2z+\x8f\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00V\x00\x00\x00\x81\x00\x00\x001c-moved\x00de51c302cf94dafe23574832ce72086e984acdde\n
s> \xff\xa2DnameM00changelog.iDsize\x19\x02\xa2_Y\x02\xa2\x00\x01\x00\x01\x00\x00\x00\x00\x00\x00\x00M\x00\x00\x00N\x00\x00\x00\x00\x00\x00\x00\x00\xff\xff\xff\xff\xff\xff\xff\xff\xfd\x8dO6\xd3@\xdb\x7f\xa4Y\x10\x078k\xac\x0c\xc1\xb2#\xd0\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00x\x9c%\xc5\xbb\r
s> \x800\x0c\x05\xc0\xfeM\xc1\x0686!\xf18\xce\xc7\x92%D\x01\xd9_\x14\\s*f\xa7sME[\xa1,\xb3\xa9\xf8Q\x9d\x93\x0b[\xe6\xd4U\xbaQ\xc1\x9a\xef\x02m\x04CC\xc7\x88\x87\xf6\xf17\x81\xb8c\x85]\x1f+i\x15B\x00\x00\x00\x00\x00M\x00\x00\x00\x00\x00>\x00\x00\x00=\x00\x00\x00\x01\x00\x00\x00\x01\x00\x00\x00\x00\xff\xff\xff\xff\x04\x03\xe9K\x8c\xe4e\xa1\xf6\x99;\xb1\xe4`{\xbe\x17`V\xa7\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00u356327300696aeacd7285a8755cea8447b237070\n
s> test\n
s> 0 0\n
s> a\n
s> \n
s> commit 2\x00\x00\x00\x00\x00\x8b\x00\x00\x00\x00\x00>\x00\x00\x00=\x00\x00\x00\x02\x00\x00\x00\x02\x00\x00\x00\x01\xff\xff\xff\xff\x0e\xb5\x06\x83\x84\x08E\xf3\xa7R\x8a}\xf3+l\xe2\x1c1J\x19\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00u9ff26517c62e7379b7e07457817a1c9602aad8d6\n
s> test\n
s> 0 0\n
s> a\n
s> \n
s> commit 3\x00\x00\x00\x00\x00\xc9\x00\x00\x00\x00\x00I\x00\x00\x00H\x00\x00\x00\x03\x00\x00\x00\x03\x00\x00\x00\x00\xff\xff\xff\xff\x15_\xcd\xfdBs\xeb\xeb\\.T\xd1\xc8q\x1f\xfd\xc2\x993\xab\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00u642dcaababdeb2f602127859e230e7859ca538b3\n
s> test\n
s> 0 0\n
s> a\n
s> \n
s> commit 4 (new head)\x00\x00\x00\x00\x01\x12\x00\x00\x00\x00\x00P\x00\x00\x00O\x00\x00\x00\x04\x00\x00\x00\x04\x00\x00\x00\x03\xff\xff\xff\xff\x9eE\xd8%Y&G)\xb4r\xc5\x167\xde\xdc\xf0\x99>\xb0\x8a\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00u19e0c69382fe33ca7026a1e2dfe59524b27a2b8f\n
s> test\n
s> 0 0\n
s> c\n
s> c-moved\n
s> \n
s> commit 5 (moved c)\xff
s> \r\n
received frame(size=1462; request=1; stream=2; streamflags=; type=command-response; flags=)
s> 8\r\n
s> \x00\x00\x00\x01\x00\x02\x002
s> \r\n
s> 0\r\n
s> \r\n
received frame(size=0; request=1; stream=2; streamflags=; type=command-response; flags=eos)
response: [{b'status': b'ok'}, {b'filecount': 2, b'totalsize': 1372}, {b'name': b'00manifest.i', b'size': 698}, bytearray['\x00\x03\x00\x01\x00\x00\x00\x00\x00\x00\x00\x98\x00\x00\x00\xe1\x00\x00\x00\x00\x00\x00\x00\x00\xff\xff\xff\xff\xff\xff\xff\xff\x93\xaao(\x17\x9bpS\xeb\x93\xf4\x8f!\xf3*R\x1c\x93\xca\x07\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00x\x9c%\xcd1\x8e\x041\x08\x05Q\xc7}\x99\x01\xfc\xc1\xf6q\xc0\x804\xe9\xde?\xd8\x96&\x7f\xaa\xf2!\x81\nZ\x93O\xb4\xd3Z\x0eri\xc2\xe1iz\xbc\x9a\xd2!\xe9O\x8c\xcd\xa7DwNv-62\x93\xd3\xaf\x8a8$\xce!\\\x05a{\xee8\x0c@\'\xcc\xe9:\xbd.\x12\xebVhv\xd1*$n\x8b\x12?\xf9\xfd\xa3O\x0e\x9d[\xc8\xf2\x9e\xc5\xa5\xc2\nJ\xdb\x98\x1e]\x96\xefjJ[B\xec\xe7k\xec\x15\xcau\xde\xb2Z\x91w\xc3H\xfd\xf2\xcd\xd5\xda\x9b\x80;9=\x9e\x7f\xb8\xcc:\xbf\x00\x00\x00\x00\x00\x98\x00\x00\x00\x00\x007\x00\x00\x00\xe1\x00\x00\x00\x00\x00\x00\x00\x01\x00\x00\x00\x00\xff\xff\xff\xff5c\'0\x06\x96\xae\xac\xd7(Z\x87U\xce\xa8D{#pp\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00+\x00\x00\x00+a\x009a38122997b3ac97be2a9aa2e556838341fdf2cc\n\x00\x00\x00\x00\x00\xcf\x00\x00\x00\x00\x007\x00\x00\x00\xe1\x00\x00\x00\x01\x00\x00\x00\x02\x00\x00\x00\x01\xff\xff\xff\xff\x9f\xf2e\x17\xc6.sy\xb7\xe0tW\x81z\x1c\x96\x02\xaa\xd8\xd6\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00+\x00\x00\x00+a\x00c2a205c8b2ade24af26062e53cd5bc3801d660da\n\x00\x00\x00\x00\x01\x06\x00\x00\x00\x00\x007\x00\x00\x00\xe1\x00\x00\x00\x00\x00\x00\x00\x03\x00\x00\x00\x00\xff\xff\xff\xffd-\xca\xab\xab\xde\xb2\xf6\x02\x12xY\xe20\xe7\x85\x9c\xa58\xb3\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00+\x00\x00\x00+a\x0036b617abfb829883cdd60767d165ea454dd81726\n\x00\x00\x00\x00\x01=\x00\x00\x00\x00\x00=\x00\x00\x00\xe7\x00\x00\x00\x03\x00\x00\x00\x04\x00\x00\x00\x03\xff\xff\xff\xff\x19\xe0\xc6\x93\x82\xfe3\xcap&\xa1\xe2\xdf\xe5\x95$\xb2z+\x8f\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00V\x00\x00\x00\x81\x00\x00\x001c-moved\x00de51c302cf94dafe23574832ce72086e984acdde\n'], {b'name': b'00changelog.i', b'size': 674}, bytearray['\x00\x01\x00\x01\x00\x00\x00\x00\x00\x00\x00M\x00\x00\x00N\x00\x00\x00\x00\x00\x00\x00\x00\xff\xff\xff\xff\xff\xff\xff\xff\xfd\x8dO6\xd3@\xdb\x7f\xa4Y\x10\x078k\xac\x0c\xc1\xb2#\xd0\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00x\x9c%\xc5\xbb\r\x800\x0c\x05\xc0\xfeM\xc1\x0686!\xf18\xce\xc7\x92%D\x01\xd9_\x14\\s*f\xa7sME[\xa1,\xb3\xa9\xf8Q\x9d\x93\x0b[\xe6\xd4U\xbaQ\xc1\x9a\xef\x02m\x04CC\xc7\x88\x87\xf6\xf17\x81\xb8c\x85]\x1f+i\x15B\x00\x00\x00\x00\x00M\x00\x00\x00\x00\x00>\x00\x00\x00=\x00\x00\x00\x01\x00\x00\x00\x01\x00\x00\x00\x00\xff\xff\xff\xff\x04\x03\xe9K\x8c\xe4e\xa1\xf6\x99;\xb1\xe4`{\xbe\x17`V\xa7\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00u356327300696aeacd7285a8755cea8447b237070\ntest\n0 0\na\n\ncommit 2\x00\x00\x00\x00\x00\x8b\x00\x00\x00\x00\x00>\x00\x00\x00=\x00\x00\x00\x02\x00\x00\x00\x02\x00\x00\x00\x01\xff\xff\xff\xff\x0e\xb5\x06\x83\x84\x08E\xf3\xa7R\x8a}\xf3+l\xe2\x1c1J\x19\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00u9ff26517c62e7379b7e07457817a1c9602aad8d6\ntest\n0 0\na\n\ncommit 3\x00\x00\x00\x00\x00\xc9\x00\x00\x00\x00\x00I\x00\x00\x00H\x00\x00\x00\x03\x00\x00\x00\x03\x00\x00\x00\x00\xff\xff\xff\xff\x15_\xcd\xfdBs\xeb\xeb\\.T\xd1\xc8q\x1f\xfd\xc2\x993\xab\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00u642dcaababdeb2f602127859e230e7859ca538b3\ntest\n0 0\na\n\ncommit 4 (new head)\x00\x00\x00\x00\x01\x12\x00\x00\x00\x00\x00P\x00\x00\x00O\x00\x00\x00\x04\x00\x00\x00\x04\x00\x00\x00\x03\xff\xff\xff\xff\x9eE\xd8%Y&G)\xb4r\xc5\x167\xde\xdc\xf0\x99>\xb0\x8a\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00u19e0c69382fe33ca7026a1e2dfe59524b27a2b8f\ntest\n0 0\nc\nc-moved\n\ncommit 5 (moved c)']]
$ cat error.log