( )⚙ D4472 wireprotov2: implement commands as a generator of objects

This is an archive of the discontinued Mercurial Phabricator instance.

Differential D4472

wireprotov2: implement commands as a generator of objects
ClosedPublic

Authored by indygreg on Sep 4 2018, 2:28 PM.

Details

Reviewers
None
Group Reviewers
hg-reviewers
Commits
rHG07b58266bce3: wireprotov2: implement commands as a generator of objects
Summary

Previously, wire protocol version 2 inherited version 1's model of
having separate types to represent the results of different wire
protocol commands.

As I implemented more powerful commands in future commits, I found
I was using a common pattern of returning a special type to hold a
generator. This meant the command function required a closure to
do most of the work. That made logic flow more difficult to follow.
I also noticed that many commands were effectively a sequence of
objects to be CBOR encoded.

I think it makes sense to define version 2 commands as generators.
This way, commands can simply emit the data structures they wish to
send to the client. This eliminates the need for a closure in
command functions and removes encoding from the bodies of commands.

As part of this commit, the handling of response objects has been
moved into the serverreactor class. This puts the reactor in the
driver's seat with regards to CBOR encoding and error handling.
Having error handling in the function that emits frames is
particularly important because exceptions in that function can lead
to things getting in a bad state: I'm fairly certain that uncaught
exceptions in the frame generator were causing deadlocks.

I also introduced a dedicated error type for explicit error reporting
in command handlers. This will be used in subsequent commits.

There's still a bit of work to be done here, especially around
formalizing the error handling "protocol." I've added yet another
TODO to track this so we don't forget.

Test output changed because we're using generators and no longer know
we are at the end of the data until we hit the end of the generator.
This means we can't emit the end-of-stream flag until we've exhausted
the generator. Hence the introduction of 0-sized end-of-stream frames.

Diff Detail

Repository
rHG Mercurial
Lint
Automatic diff as part of commit; lint not applicable.
Unit
Automatic diff as part of commit; unit tests not applicable.

Event Timeline

indygreg created this revision.Sep 4 2018, 2:28 PM
indygreg updated this revision to Diff 10791.Sep 5 2018, 12:20 PM
Closed by commit rHG07b58266bce3: wireprotov2: implement commands as a generator of objects (authored by indygreg). · Explain WhySep 12 2018, 10:43 AM
This revision was automatically updated to reflect the committed changes.

Revision Contents
Changeset List

Diff 10933

mercurial/error.py

__bytes__ = _tobytes __bytes__ = _tobytes
class PeerTransportError(Abort): class PeerTransportError(Abort):
"""Transport-level I/O error when communicating with a peer repo.""" """Transport-level I/O error when communicating with a peer repo."""
class InMemoryMergeConflictsError(Exception): class InMemoryMergeConflictsError(Exception):
"""Exception raised when merge conflicts arose during an in-memory merge.""" """Exception raised when merge conflicts arose during an in-memory merge."""
__bytes__ = _tobytes __bytes__ = _tobytes
class WireprotoCommandError(Exception):
"""Represents an error during execution of a wire protocol command.
Should only be thrown by wire protocol version 2 commands.
The error is a formatter string and an optional iterable of arguments.
"""
def __init__(self, message, args=None):
self.message = message
self.messageargs = args

mercurial/help/internals/wireprotocolrpc.txt

error (optional) error (optional)
A map containing information about an encountered error. The map has the A map containing information about an encountered error. The map has the
following keys: following keys:
message message
(array of maps) A message describing the error. The message uses the (array of maps) A message describing the error. The message uses the
same format as those in the ``Human Output Side-Channel`` frame. same format as those in the ``Human Output Side-Channel`` frame.
TODO formalize when error frames can be seen and how errors can be
recognized midway through a command response.

mercurial/wireprotoframing.py

flags=flags, flags=flags,
payload=chunk) payload=chunk)
if done: if done:
break break
def createbytesresponseframesfromgen(stream, requestid, gen, def createbytesresponseframesfromgen(stream, requestid, gen,
maxframesize=DEFAULT_MAX_FRAME_SIZE): maxframesize=DEFAULT_MAX_FRAME_SIZE):
overall = b''.join(cborutil.streamencode({b'status': b'ok'})) """Generator of frames from a generator of byte chunks.
yield stream.makeframe(requestid=requestid,
typeid=FRAME_TYPE_COMMAND_RESPONSE,
flags=FLAG_COMMAND_RESPONSE_CONTINUATION,
payload=overall)
This assumes that another frame will follow whatever this emits. i.e.
this always emits the continuation flag and never emits the end-of-stream
flag.
"""
cb = util.chunkbuffer(gen) cb = util.chunkbuffer(gen)
flags = FLAG_COMMAND_RESPONSE_CONTINUATION
flags = 0
while True: while True:
chunk = cb.read(maxframesize) chunk = cb.read(maxframesize)
if not chunk: if not chunk:
break break
yield stream.makeframe(requestid=requestid, yield stream.makeframe(requestid=requestid,
typeid=FRAME_TYPE_COMMAND_RESPONSE, typeid=FRAME_TYPE_COMMAND_RESPONSE,
flags=flags, flags=flags,
payload=chunk) payload=chunk)
flags |= FLAG_COMMAND_RESPONSE_CONTINUATION flags |= FLAG_COMMAND_RESPONSE_CONTINUATION
flags ^= FLAG_COMMAND_RESPONSE_CONTINUATION def createcommandresponseokframe(stream, requestid):
flags |= FLAG_COMMAND_RESPONSE_EOS overall = b''.join(cborutil.streamencode({b'status': b'ok'}))
yield stream.makeframe(requestid=requestid,
return stream.makeframe(requestid=requestid,
typeid=FRAME_TYPE_COMMAND_RESPONSE, typeid=FRAME_TYPE_COMMAND_RESPONSE,
flags=flags, flags=FLAG_COMMAND_RESPONSE_CONTINUATION,
payload=overall)
def createcommandresponseeosframe(stream, requestid):
"""Create an empty payload frame representing command end-of-stream."""
return stream.makeframe(requestid=requestid,
typeid=FRAME_TYPE_COMMAND_RESPONSE,
flags=FLAG_COMMAND_RESPONSE_EOS,
payload=b'') payload=b'')
def createcommanderrorresponse(stream, requestid, message, args=None): def createcommanderrorresponse(stream, requestid, message, args=None):
# TODO should this be using a list of {'msg': ..., 'args': {}} so atom # TODO should this be using a list of {'msg': ..., 'args': {}} so atom
# formatting works consistently? # formatting works consistently?
m = { m = {
b'status': b'error', b'status': b'error',
b'error': { b'error': {
b'message': message, b'message': message,
if self._deferoutput: if self._deferoutput:
self._bufferedframegens.append(result) self._bufferedframegens.append(result)
return 'noop', {} return 'noop', {}
else: else:
return 'sendframes', { return 'sendframes', {
'framegen': result, 'framegen': result,
} }
def oncommandresponsereadygen(self, stream, requestid, gen): def oncommandresponsereadyobjects(self, stream, requestid, objs):
"""Signal that a bytes response is ready, with data as a generator.""" """Signal that objects are ready to be sent to the client.
``objs`` is an iterable of objects (typically a generator) that will
be encoded via CBOR and added to frames, which will be sent to the
client.
"""
ensureserverstream(stream) ensureserverstream(stream)
# We need to take care over exception handling. Uncaught exceptions
# when generating frames could lead to premature end of the frame
# stream and the possibility of the server or client process getting
# in a bad state.
#
# Keep in mind that if ``objs`` is a generator, advancing it could
# raise exceptions that originated in e.g. wire protocol command
# functions. That is why we differentiate between exceptions raised
# when iterating versus other exceptions that occur.
#
# In all cases, when the function finishes, the request is fully
# handled and no new frames for it should be seen.
def sendframes(): def sendframes():
for frame in createbytesresponseframesfromgen(stream, requestid, emitted = False
gen): while True:
try:
o = next(objs)
except StopIteration:
if emitted:
yield createcommandresponseeosframe(stream, requestid)
break
except error.WireprotoCommandError as e:
for frame in createcommanderrorresponse(
stream, requestid, e.message, e.messageargs):
yield frame yield frame
break
except Exception as e:
for frame in createerrorframe(stream, requestid,
'%s' % e,
errtype='server'):
yield frame
break
try:
if not emitted:
yield createcommandresponseokframe(stream, requestid)
emitted = True
# TODO buffer chunks so emitted frame payloads can be
# larger.
for frame in createbytesresponseframesfromgen(
stream, requestid, cborutil.streamencode(o)):
yield frame
except Exception as e:
for frame in createerrorframe(stream, requestid,
'%s' % e,
errtype='server'):
yield frame
break
self._activecommands.remove(requestid) self._activecommands.remove(requestid)
return self._handlesendframes(sendframes()) return self._handlesendframes(sendframes())
def oninputeof(self): def oninputeof(self):
"""Signals that end of input has been received. """Signals that end of input has been received.

mercurial/wireprototypes.py

Accepts a generator containing chunks of data to be sent to the client. Accepts a generator containing chunks of data to be sent to the client.
Like ``streamres``, but sends an uncompressed data for "version 1" clients Like ``streamres``, but sends an uncompressed data for "version 1" clients
using the application/mercurial-0.1 media type. using the application/mercurial-0.1 media type.
""" """
def __init__(self, gen=None): def __init__(self, gen=None):
self.gen = gen self.gen = gen
class cborresponse(object):
"""Encode the response value as CBOR."""
def __init__(self, v):
self.value = v
class v2errorresponse(object):
"""Represents a command error for version 2 transports."""
def __init__(self, message, args=None):
self.message = message
self.args = args
class v2streamingresponse(object):
"""A response whose data is supplied by a generator.
The generator can either consist of data structures to CBOR
encode or a stream of already-encoded bytes.
"""
def __init__(self, gen, compressible=True):
self.gen = gen
self.compressible = compressible
# list of nodes encoding / decoding # list of nodes encoding / decoding
def decodelist(l, sep=' '): def decodelist(l, sep=' '):
if l: if l:
return [bin(v) for v in l.split(sep)] return [bin(v) for v in l.split(sep)]
return [] return []
def encodelist(l, sep=' '): def encodelist(l, sep=' '):
try: try:

mercurial/wireprotov2peer.py

if action == 'error': if action == 'error':
e = error.RepoError(meta['message']) e = error.RepoError(meta['message'])
if frame.requestid in self._futures: if frame.requestid in self._futures:
self._futures[frame.requestid].set_exception(e) self._futures[frame.requestid].set_exception(e)
else: else:
raise e raise e
return
if frame.requestid not in self._requests: if frame.requestid not in self._requests:
raise error.ProgrammingError( raise error.ProgrammingError(
'received frame for unknown request; this is either a bug in ' 'received frame for unknown request; this is either a bug in '
'the clientreactor not screening for this or this instance was ' 'the clientreactor not screening for this or this instance was '
'never told about this request: %r' % frame) 'never told about this request: %r' % frame)
response = self._responses[frame.requestid] response = self._responses[frame.requestid]

mercurial/wireprotov2server.py

error, error,
pycompat, pycompat,
streamclone, streamclone,
util, util,
wireprotoframing, wireprotoframing,
wireprototypes, wireprototypes,
) )
from .utils import ( from .utils import (
cborutil,
interfaceutil, interfaceutil,
) )
FRAMINGTYPE = b'application/mercurial-exp-framing-0005' FRAMINGTYPE = b'application/mercurial-exp-framing-0005'
HTTP_WIREPROTO_V2 = wireprototypes.HTTP_WIREPROTO_V2 HTTP_WIREPROTO_V2 = wireprototypes.HTTP_WIREPROTO_V2
COMMANDS = wireprototypes.commanddict() COMMANDS = wireprototypes.commanddict()
if reqcommand != command['command']: if reqcommand != command['command']:
# TODO define proper error mechanism # TODO define proper error mechanism
res.status = b'200 OK' res.status = b'200 OK'
res.headers[b'Content-Type'] = b'text/plain' res.headers[b'Content-Type'] = b'text/plain'
res.setbodybytes(_('command in frame must match command in URL')) res.setbodybytes(_('command in frame must match command in URL'))
return True return True
rsp = dispatch(repo, proto, command['command'])
res.status = b'200 OK' res.status = b'200 OK'
res.headers[b'Content-Type'] = FRAMINGTYPE res.headers[b'Content-Type'] = FRAMINGTYPE
# TODO consider adding a type to represent an iterable of values to try:
# be CBOR encoded. objs = dispatch(repo, proto, command['command'])
if isinstance(rsp, wireprototypes.cborresponse):
# TODO consider calling oncommandresponsereadygen(). action, meta = reactor.oncommandresponsereadyobjects(
encoded = b''.join(cborutil.streamencode(rsp.value)) outstream, command['requestid'], objs)
action, meta = reactor.oncommandresponseready(outstream,
command['requestid'], except Exception as e:
encoded)
elif isinstance(rsp, wireprototypes.v2streamingresponse):
action, meta = reactor.oncommandresponsereadygen(outstream,
command['requestid'],
rsp.gen)
elif isinstance(rsp, wireprototypes.v2errorresponse):
action, meta = reactor.oncommanderror(outstream,
command['requestid'],
rsp.message,
rsp.args)
else:
action, meta = reactor.onservererror( action, meta = reactor.onservererror(
_('unhandled response type from wire proto command')) outstream, command['requestid'],
_('exception when invoking command: %s') % e)
if action == 'sendframes': if action == 'sendframes':
res.setbodygen(meta['framegen']) res.setbodygen(meta['framegen'])
return True return True
elif action == 'noop': elif action == 'noop':
return False return False
else: else:
raise error.ProgrammingError('unhandled event from reactor: %s' % raise error.ProgrammingError('unhandled event from reactor: %s' %
``args`` is a dict of argument names to example values. ``args`` is a dict of argument names to example values.
``permission`` defines the permission type needed to run this command. ``permission`` defines the permission type needed to run this command.
Can be ``push`` or ``pull``. These roughly map to read-write and read-only, Can be ``push`` or ``pull``. These roughly map to read-write and read-only,
respectively. Default is to assume command requires ``push`` permissions respectively. Default is to assume command requires ``push`` permissions
because otherwise commands not declaring their permissions could modify because otherwise commands not declaring their permissions could modify
a repository that is supposed to be read-only. a repository that is supposed to be read-only.
Wire protocol commands are generators of objects to be serialized and
sent to the client.
If a command raises an uncaught exception, this will be translated into
a command error.
""" """
transports = {k for k, v in wireprototypes.TRANSPORTS.items() transports = {k for k, v in wireprototypes.TRANSPORTS.items()
if v['version'] == 2} if v['version'] == 2}
if permission not in ('push', 'pull'): if permission not in ('push', 'pull'):
raise error.ProgrammingError('invalid wire protocol permission; ' raise error.ProgrammingError('invalid wire protocol permission; '
'got %s; expected "push" or "pull"' % 'got %s; expected "push" or "pull"' %
permission) permission)
func, args=args, transports=transports, permission=permission) func, args=args, transports=transports, permission=permission)
return func return func
return register return register
@wireprotocommand('branchmap', permission='pull') @wireprotocommand('branchmap', permission='pull')
def branchmapv2(repo, proto): def branchmapv2(repo, proto):
branchmap = {encoding.fromlocal(k): v yield {encoding.fromlocal(k): v
for k, v in repo.branchmap().iteritems()} for k, v in repo.branchmap().iteritems()}
return wireprototypes.cborresponse(branchmap)
@wireprotocommand('capabilities', permission='pull') @wireprotocommand('capabilities', permission='pull')
def capabilitiesv2(repo, proto): def capabilitiesv2(repo, proto):
caps = _capabilitiesv2(repo, proto) yield _capabilitiesv2(repo, proto)
return wireprototypes.cborresponse(caps)
@wireprotocommand('heads', @wireprotocommand('heads',
args={ args={
'publiconly': False, 'publiconly': False,
}, },
permission='pull') permission='pull')
def headsv2(repo, proto, publiconly=False): def headsv2(repo, proto, publiconly=False):
if publiconly: if publiconly:
repo = repo.filtered('immutable') repo = repo.filtered('immutable')
return wireprototypes.cborresponse(repo.heads()) yield repo.heads()
@wireprotocommand('known', @wireprotocommand('known',
args={ args={
'nodes': [b'deadbeef'], 'nodes': [b'deadbeef'],
}, },
permission='pull') permission='pull')
def knownv2(repo, proto, nodes=None): def knownv2(repo, proto, nodes=None):
nodes = nodes or [] nodes = nodes or []
result = b''.join(b'1' if n else b'0' for n in repo.known(nodes)) result = b''.join(b'1' if n else b'0' for n in repo.known(nodes))
return wireprototypes.cborresponse(result) yield result
@wireprotocommand('listkeys', @wireprotocommand('listkeys',
args={ args={
'namespace': b'ns', 'namespace': b'ns',
}, },
permission='pull') permission='pull')
def listkeysv2(repo, proto, namespace=None): def listkeysv2(repo, proto, namespace=None):
keys = repo.listkeys(encoding.tolocal(namespace)) keys = repo.listkeys(encoding.tolocal(namespace))
keys = {encoding.fromlocal(k): encoding.fromlocal(v) keys = {encoding.fromlocal(k): encoding.fromlocal(v)
for k, v in keys.iteritems()} for k, v in keys.iteritems()}
return wireprototypes.cborresponse(keys) yield keys
@wireprotocommand('lookup', @wireprotocommand('lookup',
args={ args={
'key': b'foo', 'key': b'foo',
}, },
permission='pull') permission='pull')
def lookupv2(repo, proto, key): def lookupv2(repo, proto, key):
key = encoding.tolocal(key) key = encoding.tolocal(key)
# TODO handle exception. # TODO handle exception.
node = repo.lookup(key) node = repo.lookup(key)
return wireprototypes.cborresponse(node) yield node
@wireprotocommand('pushkey', @wireprotocommand('pushkey',
args={ args={
'namespace': b'ns', 'namespace': b'ns',
'key': b'key', 'key': b'key',
'old': b'old', 'old': b'old',
'new': b'new', 'new': b'new',
}, },
permission='push') permission='push')
def pushkeyv2(repo, proto, namespace, key, old, new): def pushkeyv2(repo, proto, namespace, key, old, new):
# TODO handle ui output redirection # TODO handle ui output redirection
r = repo.pushkey(encoding.tolocal(namespace), yield repo.pushkey(encoding.tolocal(namespace),
encoding.tolocal(key), encoding.tolocal(key),
encoding.tolocal(old), encoding.tolocal(old),
encoding.tolocal(new)) encoding.tolocal(new))
return wireprototypes.cborresponse(r)

tests/test-http-api-httpv2.t

s> \x15\x00\x00\x01\x00\x01\x01\x11\xa1DnameNcustomreadonly s> \x15\x00\x00\x01\x00\x01\x01\x11\xa1DnameNcustomreadonly
s> makefile('rb', None) s> makefile('rb', None)
s> HTTP/1.1 200 OK\r\n s> HTTP/1.1 200 OK\r\n
s> Server: testing stub value\r\n s> Server: testing stub value\r\n
s> Date: $HTTP_DATE$\r\n s> Date: $HTTP_DATE$\r\n
s> Content-Type: application/mercurial-exp-framing-0005\r\n s> Content-Type: application/mercurial-exp-framing-0005\r\n
s> Transfer-Encoding: chunked\r\n s> Transfer-Encoding: chunked\r\n
s> \r\n s> \r\n
s> 32\r\n s> 13\r\n
s> *\x00\x00\x01\x00\x02\x012\xa1FstatusBokX\x1dcustomreadonly bytes response s> \x0b\x00\x00\x01\x00\x02\x011\xa1FstatusBok
s> \r\n
s> 27\r\n
s> \x1f\x00\x00\x01\x00\x02\x001X\x1dcustomreadonly bytes response
s> \r\n
s> 8\r\n
s> \x00\x00\x00\x01\x00\x02\x002
s> \r\n s> \r\n
s> 0\r\n s> 0\r\n
s> \r\n s> \r\n
$ sendhttpv2peer << EOF $ sendhttpv2peer << EOF
> command customreadonly > command customreadonly
> EOF > EOF
creating http peer for wire protocol version 2 creating http peer for wire protocol version 2
s> \x15\x00\x00\x01\x00\x01\x01\x11\xa1DnameNcustomreadonly s> \x15\x00\x00\x01\x00\x01\x01\x11\xa1DnameNcustomreadonly
s> makefile('rb', None) s> makefile('rb', None)
s> HTTP/1.1 200 OK\r\n s> HTTP/1.1 200 OK\r\n
s> Server: testing stub value\r\n s> Server: testing stub value\r\n
s> Date: $HTTP_DATE$\r\n s> Date: $HTTP_DATE$\r\n
s> Content-Type: application/mercurial-exp-framing-0005\r\n s> Content-Type: application/mercurial-exp-framing-0005\r\n
s> Transfer-Encoding: chunked\r\n s> Transfer-Encoding: chunked\r\n
s> \r\n s> \r\n
s> 32\r\n s> 13\r\n
s> *\x00\x00\x01\x00\x02\x012 s> \x0b\x00\x00\x01\x00\x02\x011
s> \xa1FstatusBokX\x1dcustomreadonly bytes response s> \xa1FstatusBok
s> \r\n
received frame(size=11; request=1; stream=2; streamflags=stream-begin; type=command-response; flags=continuation)
s> 27\r\n
s> \x1f\x00\x00\x01\x00\x02\x001
s> X\x1dcustomreadonly bytes response
s> \r\n
received frame(size=31; request=1; stream=2; streamflags=; type=command-response; flags=continuation)
s> 8\r\n
s> \x00\x00\x00\x01\x00\x02\x002
s> \r\n s> \r\n
received frame(size=42; request=1; stream=2; streamflags=stream-begin; type=command-response; flags=eos)
s> 0\r\n s> 0\r\n
s> \r\n s> \r\n
received frame(size=0; request=1; stream=2; streamflags=; type=command-response; flags=eos)
response: [ response: [
{ {
b'status': b'ok' b'status': b'ok'
}, },
b'customreadonly bytes response' b'customreadonly bytes response'
] ]
Request to read-write command fails because server is read-only by default Request to read-write command fails because server is read-only by default
s> \x15\x00\x00\x01\x00\x01\x01\x11\xa1DnameNcustomreadonly s> \x15\x00\x00\x01\x00\x01\x01\x11\xa1DnameNcustomreadonly
s> makefile('rb', None) s> makefile('rb', None)
s> HTTP/1.1 200 OK\r\n s> HTTP/1.1 200 OK\r\n
s> Server: testing stub value\r\n s> Server: testing stub value\r\n
s> Date: $HTTP_DATE$\r\n s> Date: $HTTP_DATE$\r\n
s> Content-Type: application/mercurial-exp-framing-0005\r\n s> Content-Type: application/mercurial-exp-framing-0005\r\n
s> Transfer-Encoding: chunked\r\n s> Transfer-Encoding: chunked\r\n
s> \r\n s> \r\n
s> 32\r\n s> 13\r\n
s> *\x00\x00\x01\x00\x02\x012\xa1FstatusBokX\x1dcustomreadonly bytes response s> \x0b\x00\x00\x01\x00\x02\x011\xa1FstatusBok
s> \r\n
s> 27\r\n
s> \x1f\x00\x00\x01\x00\x02\x001X\x1dcustomreadonly bytes response
s> \r\n
s> 8\r\n
s> \x00\x00\x00\x01\x00\x02\x002
s> \r\n s> \r\n
s> 0\r\n s> 0\r\n
s> \r\n s> \r\n
Authorized request for unknown command is rejected Authorized request for unknown command is rejected
$ sendhttpraw << EOF $ sendhttpraw << EOF
> httprequest POST api/$HTTPV2/rw/badcommand > httprequest POST api/$HTTPV2/rw/badcommand
s> \x15\x00\x00\x01\x00\x01\x01\x11\xa1DnameNcustomreadonly s> \x15\x00\x00\x01\x00\x01\x01\x11\xa1DnameNcustomreadonly
s> makefile('rb', None) s> makefile('rb', None)
s> HTTP/1.1 200 OK\r\n s> HTTP/1.1 200 OK\r\n
s> Server: testing stub value\r\n s> Server: testing stub value\r\n
s> Date: $HTTP_DATE$\r\n s> Date: $HTTP_DATE$\r\n
s> Content-Type: application/mercurial-exp-framing-0005\r\n s> Content-Type: application/mercurial-exp-framing-0005\r\n
s> Transfer-Encoding: chunked\r\n s> Transfer-Encoding: chunked\r\n
s> \r\n s> \r\n
s> 32\r\n s> 13\r\n
s> *\x00\x00\x01\x00\x02\x012\xa1FstatusBokX\x1dcustomreadonly bytes response s> \x0b\x00\x00\x01\x00\x02\x011\xa1FstatusBok
s> \r\n
s> 27\r\n
s> \x1f\x00\x00\x01\x00\x02\x001X\x1dcustomreadonly bytes response
s> \r\n
s> 8\r\n
s> \x00\x00\x00\x01\x00\x02\x002
s> \r\n s> \r\n
s> 0\r\n s> 0\r\n
s> \r\n s> \r\n
Multiple requests to "multirequest" URL are allowed Multiple requests to "multirequest" URL are allowed
$ sendhttpraw << EOF $ sendhttpraw << EOF
> httprequest POST api/$HTTPV2/ro/multirequest > httprequest POST api/$HTTPV2/ro/multirequest
s> \x15\x00\x00\x01\x00\x01\x01\x11\xa1DnameNcustomreadonly\x15\x00\x00\x03\x00\x01\x00\x11\xa1DnameNcustomreadonly s> \x15\x00\x00\x01\x00\x01\x01\x11\xa1DnameNcustomreadonly\x15\x00\x00\x03\x00\x01\x00\x11\xa1DnameNcustomreadonly
s> makefile('rb', None) s> makefile('rb', None)
s> HTTP/1.1 200 OK\r\n s> HTTP/1.1 200 OK\r\n
s> Server: testing stub value\r\n s> Server: testing stub value\r\n
s> Date: $HTTP_DATE$\r\n s> Date: $HTTP_DATE$\r\n
s> Content-Type: application/mercurial-exp-framing-0005\r\n s> Content-Type: application/mercurial-exp-framing-0005\r\n
s> Transfer-Encoding: chunked\r\n s> Transfer-Encoding: chunked\r\n
s> \r\n s> \r\n
s> 32\r\n s> 13\r\n
s> *\x00\x00\x01\x00\x02\x012\xa1FstatusBokX\x1dcustomreadonly bytes response s> \x0b\x00\x00\x01\x00\x02\x011\xa1FstatusBok
s> \r\n
s> 27\r\n
s> \x1f\x00\x00\x01\x00\x02\x001X\x1dcustomreadonly bytes response
s> \r\n s> \r\n
s> 32\r\n s> 8\r\n
s> *\x00\x00\x03\x00\x02\x002\xa1FstatusBokX\x1dcustomreadonly bytes response s> \x00\x00\x00\x01\x00\x02\x002
s> \r\n
s> 13\r\n
s> \x0b\x00\x00\x03\x00\x02\x001\xa1FstatusBok
s> \r\n
s> 27\r\n
s> \x1f\x00\x00\x03\x00\x02\x001X\x1dcustomreadonly bytes response
s> \r\n
s> 8\r\n
s> \x00\x00\x00\x03\x00\x02\x002
s> \r\n s> \r\n
s> 0\r\n s> 0\r\n
s> \r\n s> \r\n
Interleaved requests to "multirequest" are processed Interleaved requests to "multirequest" are processed
$ sendhttpraw << EOF $ sendhttpraw << EOF
> httprequest POST api/$HTTPV2/ro/multirequest > httprequest POST api/$HTTPV2/ro/multirequest
s> \x11\x00\x00\x01\x00\x01\x01\x15\xa2Dargs\xa1Inamespace\x11\x00\x00\x03\x00\x01\x00\x15\xa2Dargs\xa1Inamespace\x19\x00\x00\x03\x00\x01\x00\x12JnamespacesDnameHlistkeys\x18\x00\x00\x01\x00\x01\x00\x12IbookmarksDnameHlistkeys s> \x11\x00\x00\x01\x00\x01\x01\x15\xa2Dargs\xa1Inamespace\x11\x00\x00\x03\x00\x01\x00\x15\xa2Dargs\xa1Inamespace\x19\x00\x00\x03\x00\x01\x00\x12JnamespacesDnameHlistkeys\x18\x00\x00\x01\x00\x01\x00\x12IbookmarksDnameHlistkeys
s> makefile('rb', None) s> makefile('rb', None)
s> HTTP/1.1 200 OK\r\n s> HTTP/1.1 200 OK\r\n
s> Server: testing stub value\r\n s> Server: testing stub value\r\n
s> Date: $HTTP_DATE$\r\n s> Date: $HTTP_DATE$\r\n
s> Content-Type: application/mercurial-exp-framing-0005\r\n s> Content-Type: application/mercurial-exp-framing-0005\r\n
s> Transfer-Encoding: chunked\r\n s> Transfer-Encoding: chunked\r\n
s> \r\n s> \r\n
s> 33\r\n s> 13\r\n
s> +\x00\x00\x03\x00\x02\x012\xa1FstatusBok\xa3Ibookmarks@Jnamespaces@Fphases@ s> \x0b\x00\x00\x03\x00\x02\x011\xa1FstatusBok
s> \r\n
s> 28\r\n
s> \x00\x00\x03\x00\x02\x001\xa3Ibookmarks@Jnamespaces@Fphases@
s> \r\n
s> 8\r\n
s> \x00\x00\x00\x03\x00\x02\x002
s> \r\n
s> 13\r\n
s> \x0b\x00\x00\x01\x00\x02\x001\xa1FstatusBok
s> \r\n
s> 9\r\n
s> \x01\x00\x00\x01\x00\x02\x001\xa0
s> \r\n s> \r\n
s> 14\r\n s> 8\r\n
s> \x0c\x00\x00\x01\x00\x02\x002\xa1FstatusBok\xa0 s> \x00\x00\x00\x01\x00\x02\x002
s> \r\n s> \r\n
s> 0\r\n s> 0\r\n
s> \r\n s> \r\n
Restart server to disable read-write access Restart server to disable read-write access
$ killdaemons.py $ killdaemons.py
$ cat > server/.hg/hgrc << EOF $ cat > server/.hg/hgrc << EOF

tests/test-http-protocol.t

s> \x0c\x00\x00\x01\x00\x01\x01\x11\xa1DnameEheads s> \x0c\x00\x00\x01\x00\x01\x01\x11\xa1DnameEheads
s> makefile('rb', None) s> makefile('rb', None)
s> HTTP/1.1 200 OK\r\n s> HTTP/1.1 200 OK\r\n
s> Server: testing stub value\r\n s> Server: testing stub value\r\n
s> Date: $HTTP_DATE$\r\n s> Date: $HTTP_DATE$\r\n
s> Content-Type: application/mercurial-exp-framing-0005\r\n s> Content-Type: application/mercurial-exp-framing-0005\r\n
s> Transfer-Encoding: chunked\r\n s> Transfer-Encoding: chunked\r\n
s> \r\n s> \r\n
s> 29\r\n s> 13\r\n
s> !\x00\x00\x01\x00\x02\x012 s> \x0b\x00\x00\x01\x00\x02\x011
s> \xa1FstatusBok\x81T\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00 s> \xa1FstatusBok
s> \r\n
received frame(size=11; request=1; stream=2; streamflags=stream-begin; type=command-response; flags=continuation)
s> 1e\r\n
s> \x16\x00\x00\x01\x00\x02\x001
s> \x81T\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00
s> \r\n
received frame(size=22; request=1; stream=2; streamflags=; type=command-response; flags=continuation)
s> 8\r\n
s> \x00\x00\x00\x01\x00\x02\x002
s> \r\n s> \r\n
received frame(size=33; request=1; stream=2; streamflags=stream-begin; type=command-response; flags=eos)
s> 0\r\n s> 0\r\n
s> \r\n s> \r\n
received frame(size=0; request=1; stream=2; streamflags=; type=command-response; flags=eos)
response: [ response: [
b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00' b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00'
] ]
$ killdaemons.py $ killdaemons.py
HTTP client follows HTTP redirect on handshake to new repo HTTP client follows HTTP redirect on handshake to new repo

tests/test-wireproto-command-branchmap.t

s> \x10\x00\x00\x01\x00\x01\x01\x11\xa1DnameIbranchmap s> \x10\x00\x00\x01\x00\x01\x01\x11\xa1DnameIbranchmap
s> makefile('rb', None) s> makefile('rb', None)
s> HTTP/1.1 200 OK\r\n s> HTTP/1.1 200 OK\r\n
s> Server: testing stub value\r\n s> Server: testing stub value\r\n
s> Date: $HTTP_DATE$\r\n s> Date: $HTTP_DATE$\r\n
s> Content-Type: application/mercurial-exp-framing-0005\r\n s> Content-Type: application/mercurial-exp-framing-0005\r\n
s> Transfer-Encoding: chunked\r\n s> Transfer-Encoding: chunked\r\n
s> \r\n s> \r\n
s> 83\r\n s> 13\r\n
s> {\x00\x00\x01\x00\x02\x012 s> \x0b\x00\x00\x01\x00\x02\x011
s> \xa1FstatusBok\xa3Gbranch1\x81T\xb5\xfa\xac\xdf\xd2c7h\xcb1R3l\xc0\x953\x81&f\x88Gbranch2\x81T"Aa\xc7X\x9a\xa4\x8f\xa8:H\xfe\xff^\x95\xb5j\xe3\'\xfcGdefault\x82T&\x80Z\xba\x1e`\n s> \xa1FstatusBok
s> \r\n
received frame(size=11; request=1; stream=2; streamflags=stream-begin; type=command-response; flags=continuation)
s> 78\r\n
s> p\x00\x00\x01\x00\x02\x001
s> \xa3Gbranch1\x81T\xb5\xfa\xac\xdf\xd2c7h\xcb1R3l\xc0\x953\x81&f\x88Gbranch2\x81T"Aa\xc7X\x9a\xa4\x8f\xa8:H\xfe\xff^\x95\xb5j\xe3\'\xfcGdefault\x82T&\x80Z\xba\x1e`\n
s> \x82\xe96a\x14\x9f#\x13\x86j"\x1a{T\xbe\x0e\xf7<\x17\xad\xe3\xfc\x89\xdcAp\x1e\xb9\xfc:\x91\xb5\x82\x82 s> \x82\xe96a\x14\x9f#\x13\x86j"\x1a{T\xbe\x0e\xf7<\x17\xad\xe3\xfc\x89\xdcAp\x1e\xb9\xfc:\x91\xb5\x82\x82
s> \r\n s> \r\n
received frame(size=123; request=1; stream=2; streamflags=stream-begin; type=command-response; flags=eos) received frame(size=112; request=1; stream=2; streamflags=; type=command-response; flags=continuation)
s> 8\r\n
s> \x00\x00\x00\x01\x00\x02\x002
s> \r\n
s> 0\r\n s> 0\r\n
s> \r\n s> \r\n
received frame(size=0; request=1; stream=2; streamflags=; type=command-response; flags=eos)
response: { response: {
b'branch1': [ b'branch1': [
b'\xb5\xfa\xac\xdf\xd2c7h\xcb1R3l\xc0\x953\x81&f\x88' b'\xb5\xfa\xac\xdf\xd2c7h\xcb1R3l\xc0\x953\x81&f\x88'
], ],
b'branch2': [ b'branch2': [
b'"Aa\xc7X\x9a\xa4\x8f\xa8:H\xfe\xff^\x95\xb5j\xe3\'\xfc' b'"Aa\xc7X\x9a\xa4\x8f\xa8:H\xfe\xff^\x95\xb5j\xe3\'\xfc'
], ],
b'default': [ b'default': [
b'&\x80Z\xba\x1e`\n\x82\xe96a\x14\x9f#\x13\x86j"\x1a{', b'&\x80Z\xba\x1e`\n\x82\xe96a\x14\x9f#\x13\x86j"\x1a{',
b'\xbe\x0e\xf7<\x17\xad\xe3\xfc\x89\xdcAp\x1e\xb9\xfc:\x91\xb5\x82\x82' b'\xbe\x0e\xf7<\x17\xad\xe3\xfc\x89\xdcAp\x1e\xb9\xfc:\x91\xb5\x82\x82'
] ]
} }
$ cat error.log $ cat error.log

tests/test-wireproto-command-capabilities.t

s> \x13\x00\x00\x01\x00\x01\x01\x11\xa1DnameLcapabilities s> \x13\x00\x00\x01\x00\x01\x01\x11\xa1DnameLcapabilities
s> makefile('rb', None) s> makefile('rb', None)
s> HTTP/1.1 200 OK\r\n s> HTTP/1.1 200 OK\r\n
s> Server: testing stub value\r\n s> Server: testing stub value\r\n
s> Date: $HTTP_DATE$\r\n s> Date: $HTTP_DATE$\r\n
s> Content-Type: application/mercurial-exp-framing-0005\r\n s> Content-Type: application/mercurial-exp-framing-0005\r\n
s> Transfer-Encoding: chunked\r\n s> Transfer-Encoding: chunked\r\n
s> \r\n s> \r\n
s> 1d7\r\n s> 13\r\n
s> \xcf\x01\x00\x01\x00\x02\x012 s> \x0b\x00\x00\x01\x00\x02\x011
s> \xa1FstatusBok\xa4Hcommands\xa7Ibranchmap\xa2Dargs\xa0Kpermissions\x81DpullLcapabilities\xa2Dargs\xa0Kpermissions\x81DpullEheads\xa2Dargs\xa1Jpubliconly\xf4Kpermissions\x81DpullEknown\xa2Dargs\xa1Enodes\x81HdeadbeefKpermissions\x81DpullHlistkeys\xa2Dargs\xa1InamespaceBnsKpermissions\x81DpullFlookup\xa2Dargs\xa1CkeyCfooKpermissions\x81DpullGpushkey\xa2Dargs\xa4CkeyCkeyInamespaceBnsCnewCnewColdColdKpermissions\x81DpushKcompression\x81\xa1DnameDzlibQframingmediatypes\x81X&application/mercurial-exp-framing-0005Nrawrepoformats\x82LgeneraldeltaHrevlogv1 s> \xa1FstatusBok
s> \r\n
received frame(size=11; request=1; stream=2; streamflags=stream-begin; type=command-response; flags=continuation)
s> 1cc\r\n
s> \xc4\x01\x00\x01\x00\x02\x001
s> \xa4Hcommands\xa7Ibranchmap\xa2Dargs\xa0Kpermissions\x81DpullLcapabilities\xa2Dargs\xa0Kpermissions\x81DpullEheads\xa2Dargs\xa1Jpubliconly\xf4Kpermissions\x81DpullEknown\xa2Dargs\xa1Enodes\x81HdeadbeefKpermissions\x81DpullHlistkeys\xa2Dargs\xa1InamespaceBnsKpermissions\x81DpullFlookup\xa2Dargs\xa1CkeyCfooKpermissions\x81DpullGpushkey\xa2Dargs\xa4CkeyCkeyInamespaceBnsCnewCnewColdColdKpermissions\x81DpushKcompression\x81\xa1DnameDzlibQframingmediatypes\x81X&application/mercurial-exp-framing-0005Nrawrepoformats\x82LgeneraldeltaHrevlogv1
s> \r\n
received frame(size=452; request=1; stream=2; streamflags=; type=command-response; flags=continuation)
s> 8\r\n
s> \x00\x00\x00\x01\x00\x02\x002
s> \r\n s> \r\n
received frame(size=463; request=1; stream=2; streamflags=stream-begin; type=command-response; flags=eos)
s> 0\r\n s> 0\r\n
s> \r\n s> \r\n
received frame(size=0; request=1; stream=2; streamflags=; type=command-response; flags=eos)
response: [ response: [
{ {
b'status': b'ok' b'status': b'ok'
}, },
{ {
b'commands': { b'commands': {
b'branchmap': { b'branchmap': {
b'args': {}, b'args': {},

tests/test-wireproto-command-heads.t

s> \x0c\x00\x00\x01\x00\x01\x01\x11\xa1DnameEheads s> \x0c\x00\x00\x01\x00\x01\x01\x11\xa1DnameEheads
s> makefile('rb', None) s> makefile('rb', None)
s> HTTP/1.1 200 OK\r\n s> HTTP/1.1 200 OK\r\n
s> Server: testing stub value\r\n s> Server: testing stub value\r\n
s> Date: $HTTP_DATE$\r\n s> Date: $HTTP_DATE$\r\n
s> Content-Type: application/mercurial-exp-framing-0005\r\n s> Content-Type: application/mercurial-exp-framing-0005\r\n
s> Transfer-Encoding: chunked\r\n s> Transfer-Encoding: chunked\r\n
s> \r\n s> \r\n
s> 53\r\n s> 13\r\n
s> K\x00\x00\x01\x00\x02\x012 s> \x0b\x00\x00\x01\x00\x02\x011
s> \xa1FstatusBok\x83T\x1dok\x91\xd4J\xab\xa6\xd5\xe5\x80\xbc0\xa9\x94\x850\xdb\xe0\x0bT\xaeI.6\xb0\xc83\x9f\xfa\xf3(\xd0\x0b\x85\xb4R]\xe1\x16^T)Dm-\xc5A\x9c_\x97Dz\x8b\xc0b\xe4\xcc2\x8b\xf2A s> \xa1FstatusBok
s> \r\n
received frame(size=11; request=1; stream=2; streamflags=stream-begin; type=command-response; flags=continuation)
s> 48\r\n
s> @\x00\x00\x01\x00\x02\x001
s> \x83T\x1dok\x91\xd4J\xab\xa6\xd5\xe5\x80\xbc0\xa9\x94\x850\xdb\xe0\x0bT\xaeI.6\xb0\xc83\x9f\xfa\xf3(\xd0\x0b\x85\xb4R]\xe1\x16^T)Dm-\xc5A\x9c_\x97Dz\x8b\xc0b\xe4\xcc2\x8b\xf2A
s> \r\n
received frame(size=64; request=1; stream=2; streamflags=; type=command-response; flags=continuation)
s> 8\r\n
s> \x00\x00\x00\x01\x00\x02\x002
s> \r\n s> \r\n
received frame(size=75; request=1; stream=2; streamflags=stream-begin; type=command-response; flags=eos)
s> 0\r\n s> 0\r\n
s> \r\n s> \r\n
received frame(size=0; request=1; stream=2; streamflags=; type=command-response; flags=eos)
response: [ response: [
b'\x1dok\x91\xd4J\xab\xa6\xd5\xe5\x80\xbc0\xa9\x94\x850\xdb\xe0\x0b', b'\x1dok\x91\xd4J\xab\xa6\xd5\xe5\x80\xbc0\xa9\x94\x850\xdb\xe0\x0b',
b'\xaeI.6\xb0\xc83\x9f\xfa\xf3(\xd0\x0b\x85\xb4R]\xe1\x16^', b'\xaeI.6\xb0\xc83\x9f\xfa\xf3(\xd0\x0b\x85\xb4R]\xe1\x16^',
b')Dm-\xc5A\x9c_\x97Dz\x8b\xc0b\xe4\xcc2\x8b\xf2A' b')Dm-\xc5A\x9c_\x97Dz\x8b\xc0b\xe4\xcc2\x8b\xf2A'
] ]
Requesting just the public heads works Requesting just the public heads works
s> \x1f\x00\x00\x01\x00\x01\x01\x11\xa2Dargs\xa1JpubliconlyA1DnameEheads s> \x1f\x00\x00\x01\x00\x01\x01\x11\xa2Dargs\xa1JpubliconlyA1DnameEheads
s> makefile('rb', None) s> makefile('rb', None)
s> HTTP/1.1 200 OK\r\n s> HTTP/1.1 200 OK\r\n
s> Server: testing stub value\r\n s> Server: testing stub value\r\n
s> Date: $HTTP_DATE$\r\n s> Date: $HTTP_DATE$\r\n
s> Content-Type: application/mercurial-exp-framing-0005\r\n s> Content-Type: application/mercurial-exp-framing-0005\r\n
s> Transfer-Encoding: chunked\r\n s> Transfer-Encoding: chunked\r\n
s> \r\n s> \r\n
s> 29\r\n s> 13\r\n
s> !\x00\x00\x01\x00\x02\x012 s> \x0b\x00\x00\x01\x00\x02\x011
s> \xa1FstatusBok\x81Tx\xd2\xdc\xa46\xb2\xf5\xb1\x88\xac&~)\xb8\x1e\x07&m8\xfc s> \xa1FstatusBok
s> \r\n
received frame(size=11; request=1; stream=2; streamflags=stream-begin; type=command-response; flags=continuation)
s> 1e\r\n
s> \x16\x00\x00\x01\x00\x02\x001
s> \x81Tx\xd2\xdc\xa46\xb2\xf5\xb1\x88\xac&~)\xb8\x1e\x07&m8\xfc
s> \r\n
received frame(size=22; request=1; stream=2; streamflags=; type=command-response; flags=continuation)
s> 8\r\n
s> \x00\x00\x00\x01\x00\x02\x002
s> \r\n s> \r\n
received frame(size=33; request=1; stream=2; streamflags=stream-begin; type=command-response; flags=eos)
s> 0\r\n s> 0\r\n
s> \r\n s> \r\n
received frame(size=0; request=1; stream=2; streamflags=; type=command-response; flags=eos)
response: [ response: [
b'x\xd2\xdc\xa46\xb2\xf5\xb1\x88\xac&~)\xb8\x1e\x07&m8\xfc' b'x\xd2\xdc\xa46\xb2\xf5\xb1\x88\xac&~)\xb8\x1e\x07&m8\xfc'
] ]
$ cat error.log $ cat error.log

tests/test-wireproto-command-known.t

s> \x0c\x00\x00\x01\x00\x01\x01\x11\xa1DnameEknown s> \x0c\x00\x00\x01\x00\x01\x01\x11\xa1DnameEknown
s> makefile('rb', None) s> makefile('rb', None)
s> HTTP/1.1 200 OK\r\n s> HTTP/1.1 200 OK\r\n
s> Server: testing stub value\r\n s> Server: testing stub value\r\n
s> Date: $HTTP_DATE$\r\n s> Date: $HTTP_DATE$\r\n
s> Content-Type: application/mercurial-exp-framing-0005\r\n s> Content-Type: application/mercurial-exp-framing-0005\r\n
s> Transfer-Encoding: chunked\r\n s> Transfer-Encoding: chunked\r\n
s> \r\n s> \r\n
s> 14\r\n s> 13\r\n
s> \x0c\x00\x00\x01\x00\x02\x012 s> \x0b\x00\x00\x01\x00\x02\x011
s> \xa1FstatusBok@ s> \xa1FstatusBok
s> \r\n
received frame(size=11; request=1; stream=2; streamflags=stream-begin; type=command-response; flags=continuation)
s> 9\r\n
s> \x01\x00\x00\x01\x00\x02\x001
s> @
s> \r\n
received frame(size=1; request=1; stream=2; streamflags=; type=command-response; flags=continuation)
s> 8\r\n
s> \x00\x00\x00\x01\x00\x02\x002
s> \r\n s> \r\n
received frame(size=12; request=1; stream=2; streamflags=stream-begin; type=command-response; flags=eos)
s> 0\r\n s> 0\r\n
s> \r\n s> \r\n
received frame(size=0; request=1; stream=2; streamflags=; type=command-response; flags=eos)
response: [] response: []
Single known node works Single known node works
$ sendhttpv2peer << EOF $ sendhttpv2peer << EOF
> command known > command known
> nodes eval:[b'\x42\x6b\xad\xa5\xc6\x75\x98\xca\x65\x03\x6d\x57\xd9\xe4\xb6\x4b\x0c\x1c\xe7\xa0'] > nodes eval:[b'\x42\x6b\xad\xa5\xc6\x75\x98\xca\x65\x03\x6d\x57\xd9\xe4\xb6\x4b\x0c\x1c\xe7\xa0']
> EOF > EOF
s> .\x00\x00\x01\x00\x01\x01\x11\xa2Dargs\xa1Enodes\x81TBk\xad\xa5\xc6u\x98\xcae\x03mW\xd9\xe4\xb6K\x0c\x1c\xe7\xa0DnameEknown s> .\x00\x00\x01\x00\x01\x01\x11\xa2Dargs\xa1Enodes\x81TBk\xad\xa5\xc6u\x98\xcae\x03mW\xd9\xe4\xb6K\x0c\x1c\xe7\xa0DnameEknown
s> makefile('rb', None) s> makefile('rb', None)
s> HTTP/1.1 200 OK\r\n s> HTTP/1.1 200 OK\r\n
s> Server: testing stub value\r\n s> Server: testing stub value\r\n
s> Date: $HTTP_DATE$\r\n s> Date: $HTTP_DATE$\r\n
s> Content-Type: application/mercurial-exp-framing-0005\r\n s> Content-Type: application/mercurial-exp-framing-0005\r\n
s> Transfer-Encoding: chunked\r\n s> Transfer-Encoding: chunked\r\n
s> \r\n s> \r\n
s> 15\r\n s> 13\r\n
s> \r\x00\x00\x01\x00\x02\x012 s> \x0b\x00\x00\x01\x00\x02\x011
s> \xa1FstatusBokA1 s> \xa1FstatusBok
s> \r\n
received frame(size=11; request=1; stream=2; streamflags=stream-begin; type=command-response; flags=continuation)
s> a\r\n
s> \x02\x00\x00\x01\x00\x02\x001
s> A1
s> \r\n
received frame(size=2; request=1; stream=2; streamflags=; type=command-response; flags=continuation)
s> 8\r\n
s> \x00\x00\x00\x01\x00\x02\x002
s> \r\n s> \r\n
received frame(size=13; request=1; stream=2; streamflags=stream-begin; type=command-response; flags=eos)
s> 0\r\n s> 0\r\n
s> \r\n s> \r\n
received frame(size=0; request=1; stream=2; streamflags=; type=command-response; flags=eos)
response: [ response: [
True True
] ]
Multiple nodes works Multiple nodes works
$ sendhttpv2peer << EOF $ sendhttpv2peer << EOF
> command known > command known
s> X\x00\x00\x01\x00\x01\x01\x11\xa2Dargs\xa1Enodes\x83TBk\xad\xa5\xc6u\x98\xcae\x03mW\xd9\xe4\xb6K\x0c\x1c\xe7\xa0T00000000000000000000T\x11$x\x96)a\x14q$\xed\xd45I\xae\xdd\x1a3^D\xbfDnameEknown s> X\x00\x00\x01\x00\x01\x01\x11\xa2Dargs\xa1Enodes\x83TBk\xad\xa5\xc6u\x98\xcae\x03mW\xd9\xe4\xb6K\x0c\x1c\xe7\xa0T00000000000000000000T\x11$x\x96)a\x14q$\xed\xd45I\xae\xdd\x1a3^D\xbfDnameEknown
s> makefile('rb', None) s> makefile('rb', None)
s> HTTP/1.1 200 OK\r\n s> HTTP/1.1 200 OK\r\n
s> Server: testing stub value\r\n s> Server: testing stub value\r\n
s> Date: $HTTP_DATE$\r\n s> Date: $HTTP_DATE$\r\n
s> Content-Type: application/mercurial-exp-framing-0005\r\n s> Content-Type: application/mercurial-exp-framing-0005\r\n
s> Transfer-Encoding: chunked\r\n s> Transfer-Encoding: chunked\r\n
s> \r\n s> \r\n
s> 17\r\n s> 13\r\n
s> \x0f\x00\x00\x01\x00\x02\x012 s> \x0b\x00\x00\x01\x00\x02\x011
s> \xa1FstatusBokC101 s> \xa1FstatusBok
s> \r\n
received frame(size=11; request=1; stream=2; streamflags=stream-begin; type=command-response; flags=continuation)
s> c\r\n
s> \x04\x00\x00\x01\x00\x02\x001
s> C101
s> \r\n
received frame(size=4; request=1; stream=2; streamflags=; type=command-response; flags=continuation)
s> 8\r\n
s> \x00\x00\x00\x01\x00\x02\x002
s> \r\n s> \r\n
received frame(size=15; request=1; stream=2; streamflags=stream-begin; type=command-response; flags=eos)
s> 0\r\n s> 0\r\n
s> \r\n s> \r\n
received frame(size=0; request=1; stream=2; streamflags=; type=command-response; flags=eos)
response: [ response: [
True, True,
False, False,
True True
] ]
$ cat error.log $ cat error.log

tests/test-wireproto-command-listkeys.t

s> *\x00\x00\x01\x00\x01\x01\x11\xa2Dargs\xa1InamespaceJnamespacesDnameHlistkeys s> *\x00\x00\x01\x00\x01\x01\x11\xa2Dargs\xa1InamespaceJnamespacesDnameHlistkeys
s> makefile('rb', None) s> makefile('rb', None)
s> HTTP/1.1 200 OK\r\n s> HTTP/1.1 200 OK\r\n
s> Server: testing stub value\r\n s> Server: testing stub value\r\n
s> Date: $HTTP_DATE$\r\n s> Date: $HTTP_DATE$\r\n
s> Content-Type: application/mercurial-exp-framing-0005\r\n s> Content-Type: application/mercurial-exp-framing-0005\r\n
s> Transfer-Encoding: chunked\r\n s> Transfer-Encoding: chunked\r\n
s> \r\n s> \r\n
s> 33\r\n s> 13\r\n
s> +\x00\x00\x01\x00\x02\x012 s> \x0b\x00\x00\x01\x00\x02\x011
s> \xa1FstatusBok\xa3Ibookmarks@Jnamespaces@Fphases@ s> \xa1FstatusBok
s> \r\n
received frame(size=11; request=1; stream=2; streamflags=stream-begin; type=command-response; flags=continuation)
s> 28\r\n
s> \x00\x00\x01\x00\x02\x001
s> \xa3Ibookmarks@Jnamespaces@Fphases@
s> \r\n
received frame(size=32; request=1; stream=2; streamflags=; type=command-response; flags=continuation)
s> 8\r\n
s> \x00\x00\x00\x01\x00\x02\x002
s> \r\n s> \r\n
received frame(size=43; request=1; stream=2; streamflags=stream-begin; type=command-response; flags=eos)
s> 0\r\n s> 0\r\n
s> \r\n s> \r\n
received frame(size=0; request=1; stream=2; streamflags=; type=command-response; flags=eos)
response: { response: {
b'bookmarks': b'', b'bookmarks': b'',
b'namespaces': b'', b'namespaces': b'',
b'phases': b'' b'phases': b''
} }
Request for phases works Request for phases works
s> &\x00\x00\x01\x00\x01\x01\x11\xa2Dargs\xa1InamespaceFphasesDnameHlistkeys s> &\x00\x00\x01\x00\x01\x01\x11\xa2Dargs\xa1InamespaceFphasesDnameHlistkeys
s> makefile('rb', None) s> makefile('rb', None)
s> HTTP/1.1 200 OK\r\n s> HTTP/1.1 200 OK\r\n
s> Server: testing stub value\r\n s> Server: testing stub value\r\n
s> Date: $HTTP_DATE$\r\n s> Date: $HTTP_DATE$\r\n
s> Content-Type: application/mercurial-exp-framing-0005\r\n s> Content-Type: application/mercurial-exp-framing-0005\r\n
s> Transfer-Encoding: chunked\r\n s> Transfer-Encoding: chunked\r\n
s> \r\n s> \r\n
s> 50\r\n s> 13\r\n
s> H\x00\x00\x01\x00\x02\x012 s> \x0b\x00\x00\x01\x00\x02\x011
s> \xa1FstatusBok\xa2X(be0ef73c17ade3fc89dc41701eb9fc3a91b58282A1JpublishingDTrue s> \xa1FstatusBok
s> \r\n
received frame(size=11; request=1; stream=2; streamflags=stream-begin; type=command-response; flags=continuation)
s> 45\r\n
s> =\x00\x00\x01\x00\x02\x001
s> \xa2X(be0ef73c17ade3fc89dc41701eb9fc3a91b58282A1JpublishingDTrue
s> \r\n
received frame(size=61; request=1; stream=2; streamflags=; type=command-response; flags=continuation)
s> 8\r\n
s> \x00\x00\x00\x01\x00\x02\x002
s> \r\n s> \r\n
received frame(size=72; request=1; stream=2; streamflags=stream-begin; type=command-response; flags=eos)
s> 0\r\n s> 0\r\n
s> \r\n s> \r\n
received frame(size=0; request=1; stream=2; streamflags=; type=command-response; flags=eos)
response: { response: {
b'be0ef73c17ade3fc89dc41701eb9fc3a91b58282': b'1', b'be0ef73c17ade3fc89dc41701eb9fc3a91b58282': b'1',
b'publishing': b'True' b'publishing': b'True'
} }
Request for bookmarks works Request for bookmarks works
$ sendhttpv2peer << EOF $ sendhttpv2peer << EOF
s> )\x00\x00\x01\x00\x01\x01\x11\xa2Dargs\xa1InamespaceIbookmarksDnameHlistkeys s> )\x00\x00\x01\x00\x01\x01\x11\xa2Dargs\xa1InamespaceIbookmarksDnameHlistkeys
s> makefile('rb', None) s> makefile('rb', None)
s> HTTP/1.1 200 OK\r\n s> HTTP/1.1 200 OK\r\n
s> Server: testing stub value\r\n s> Server: testing stub value\r\n
s> Date: $HTTP_DATE$\r\n s> Date: $HTTP_DATE$\r\n
s> Content-Type: application/mercurial-exp-framing-0005\r\n s> Content-Type: application/mercurial-exp-framing-0005\r\n
s> Transfer-Encoding: chunked\r\n s> Transfer-Encoding: chunked\r\n
s> \r\n s> \r\n
s> 40\r\n s> 13\r\n
s> 8\x00\x00\x01\x00\x02\x012 s> \x0b\x00\x00\x01\x00\x02\x011
s> \xa1FstatusBok\xa1A@X(26805aba1e600a82e93661149f2313866a221a7b s> \xa1FstatusBok
s> \r\n
received frame(size=11; request=1; stream=2; streamflags=stream-begin; type=command-response; flags=continuation)
s> 35\r\n
s> -\x00\x00\x01\x00\x02\x001
s> \xa1A@X(26805aba1e600a82e93661149f2313866a221a7b
s> \r\n
received frame(size=45; request=1; stream=2; streamflags=; type=command-response; flags=continuation)
s> 8\r\n
s> \x00\x00\x00\x01\x00\x02\x002
s> \r\n s> \r\n
received frame(size=56; request=1; stream=2; streamflags=stream-begin; type=command-response; flags=eos)
s> 0\r\n s> 0\r\n
s> \r\n s> \r\n
received frame(size=0; request=1; stream=2; streamflags=; type=command-response; flags=eos)
response: { response: {
b'@': b'26805aba1e600a82e93661149f2313866a221a7b' b'@': b'26805aba1e600a82e93661149f2313866a221a7b'
} }
$ cat error.log $ cat error.log

tests/test-wireproto-command-lookup.t

s> A\x00\x00\x01\x00\x01\x01\x11\xa2Dargs\xa1CkeyX(426bada5c67598ca65036d57d9e4b64b0c1ce7a0DnameFlookup s> A\x00\x00\x01\x00\x01\x01\x11\xa2Dargs\xa1CkeyX(426bada5c67598ca65036d57d9e4b64b0c1ce7a0DnameFlookup
s> makefile('rb', None) s> makefile('rb', None)
s> HTTP/1.1 200 OK\r\n s> HTTP/1.1 200 OK\r\n
s> Server: testing stub value\r\n s> Server: testing stub value\r\n
s> Date: $HTTP_DATE$\r\n s> Date: $HTTP_DATE$\r\n
s> Content-Type: application/mercurial-exp-framing-0005\r\n s> Content-Type: application/mercurial-exp-framing-0005\r\n
s> Transfer-Encoding: chunked\r\n s> Transfer-Encoding: chunked\r\n
s> \r\n s> \r\n
s> 28\r\n s> 13\r\n
s> \x00\x00\x01\x00\x02\x012 s> \x0b\x00\x00\x01\x00\x02\x011
s> \xa1FstatusBokTBk\xad\xa5\xc6u\x98\xcae\x03mW\xd9\xe4\xb6K\x0c\x1c\xe7\xa0 s> \xa1FstatusBok
s> \r\n
received frame(size=11; request=1; stream=2; streamflags=stream-begin; type=command-response; flags=continuation)
s> 1d\r\n
s> \x15\x00\x00\x01\x00\x02\x001
s> TBk\xad\xa5\xc6u\x98\xcae\x03mW\xd9\xe4\xb6K\x0c\x1c\xe7\xa0
s> \r\n
received frame(size=21; request=1; stream=2; streamflags=; type=command-response; flags=continuation)
s> 8\r\n
s> \x00\x00\x00\x01\x00\x02\x002
s> \r\n s> \r\n
received frame(size=32; request=1; stream=2; streamflags=stream-begin; type=command-response; flags=eos)
s> 0\r\n s> 0\r\n
s> \r\n s> \r\n
received frame(size=0; request=1; stream=2; streamflags=; type=command-response; flags=eos)
response: b'Bk\xad\xa5\xc6u\x98\xcae\x03mW\xd9\xe4\xb6K\x0c\x1c\xe7\xa0' response: b'Bk\xad\xa5\xc6u\x98\xcae\x03mW\xd9\xe4\xb6K\x0c\x1c\xe7\xa0'
$ cat error.log $ cat error.log

tests/test-wireproto-command-pushkey.t

s> a\x00\x00\x01\x00\x01\x01\x11\xa2Dargs\xa4CkeyA@InamespaceIbookmarksCnewX(426bada5c67598ca65036d57d9e4b64b0c1ce7a0Cold@DnameGpushkey s> a\x00\x00\x01\x00\x01\x01\x11\xa2Dargs\xa4CkeyA@InamespaceIbookmarksCnewX(426bada5c67598ca65036d57d9e4b64b0c1ce7a0Cold@DnameGpushkey
s> makefile('rb', None) s> makefile('rb', None)
s> HTTP/1.1 200 OK\r\n s> HTTP/1.1 200 OK\r\n
s> Server: testing stub value\r\n s> Server: testing stub value\r\n
s> Date: $HTTP_DATE$\r\n s> Date: $HTTP_DATE$\r\n
s> Content-Type: application/mercurial-exp-framing-0005\r\n s> Content-Type: application/mercurial-exp-framing-0005\r\n
s> Transfer-Encoding: chunked\r\n s> Transfer-Encoding: chunked\r\n
s> \r\n s> \r\n
s> 14\r\n s> 13\r\n
s> \x0c\x00\x00\x01\x00\x02\x012 s> \x0b\x00\x00\x01\x00\x02\x011
s> \xa1FstatusBok\xf5 s> \xa1FstatusBok
s> \r\n
received frame(size=11; request=1; stream=2; streamflags=stream-begin; type=command-response; flags=continuation)
s> 9\r\n
s> \x01\x00\x00\x01\x00\x02\x001
s> \xf5
s> \r\n
received frame(size=1; request=1; stream=2; streamflags=; type=command-response; flags=continuation)
s> 8\r\n
s> \x00\x00\x00\x01\x00\x02\x002
s> \r\n s> \r\n
received frame(size=12; request=1; stream=2; streamflags=stream-begin; type=command-response; flags=eos)
s> 0\r\n s> 0\r\n
s> \r\n s> \r\n
received frame(size=0; request=1; stream=2; streamflags=; type=command-response; flags=eos)
response: True response: True
$ sendhttpv2peer << EOF $ sendhttpv2peer << EOF
> command listkeys > command listkeys
> namespace bookmarks > namespace bookmarks
> EOF > EOF
creating http peer for wire protocol version 2 creating http peer for wire protocol version 2
sending listkeys command sending listkeys command
s> POST /api/exp-http-v2-0001/ro/listkeys HTTP/1.1\r\n s> POST /api/exp-http-v2-0001/ro/listkeys HTTP/1.1\r\n
s> Accept-Encoding: identity\r\n s> Accept-Encoding: identity\r\n
s> accept: application/mercurial-exp-framing-0005\r\n s> accept: application/mercurial-exp-framing-0005\r\n
s> content-type: application/mercurial-exp-framing-0005\r\n s> content-type: application/mercurial-exp-framing-0005\r\n
s> content-length: 49\r\n s> content-length: 49\r\n
s> host: $LOCALIP:$HGPORT\r\n (glob) s> host: $LOCALIP:$HGPORT\r\n (glob)
s> user-agent: Mercurial debugwireproto\r\n s> user-agent: Mercurial debugwireproto\r\n
s> \r\n s> \r\n
s> )\x00\x00\x01\x00\x01\x01\x11\xa2Dargs\xa1InamespaceIbookmarksDnameHlistkeys s> )\x00\x00\x01\x00\x01\x01\x11\xa2Dargs\xa1InamespaceIbookmarksDnameHlistkeys
s> makefile('rb', None) s> makefile('rb', None)
s> HTTP/1.1 200 OK\r\n s> HTTP/1.1 200 OK\r\n
s> Server: testing stub value\r\n s> Server: testing stub value\r\n
s> Date: $HTTP_DATE$\r\n s> Date: $HTTP_DATE$\r\n
s> Content-Type: application/mercurial-exp-framing-0005\r\n s> Content-Type: application/mercurial-exp-framing-0005\r\n
s> Transfer-Encoding: chunked\r\n s> Transfer-Encoding: chunked\r\n
s> \r\n s> \r\n
s> 40\r\n s> 13\r\n
s> 8\x00\x00\x01\x00\x02\x012 s> \x0b\x00\x00\x01\x00\x02\x011
s> \xa1FstatusBok\xa1A@X(426bada5c67598ca65036d57d9e4b64b0c1ce7a0 s> \xa1FstatusBok
s> \r\n
received frame(size=11; request=1; stream=2; streamflags=stream-begin; type=command-response; flags=continuation)
s> 35\r\n
s> -\x00\x00\x01\x00\x02\x001
s> \xa1A@X(426bada5c67598ca65036d57d9e4b64b0c1ce7a0
s> \r\n
received frame(size=45; request=1; stream=2; streamflags=; type=command-response; flags=continuation)
s> 8\r\n
s> \x00\x00\x00\x01\x00\x02\x002
s> \r\n s> \r\n
received frame(size=56; request=1; stream=2; streamflags=stream-begin; type=command-response; flags=eos)
s> 0\r\n s> 0\r\n
s> \r\n s> \r\n
received frame(size=0; request=1; stream=2; streamflags=; type=command-response; flags=eos)
response: { response: {
b'@': b'426bada5c67598ca65036d57d9e4b64b0c1ce7a0' b'@': b'426bada5c67598ca65036d57d9e4b64b0c1ce7a0'
} }
$ cat error.log $ cat error.log

tests/wireprotohelpers.sh

) )
@wireprotov1server.wireprotocommand(b'customreadonly', permission=b'pull') @wireprotov1server.wireprotocommand(b'customreadonly', permission=b'pull')
def customreadonlyv1(repo, proto): def customreadonlyv1(repo, proto):
return wireprototypes.bytesresponse(b'customreadonly bytes response') return wireprototypes.bytesresponse(b'customreadonly bytes response')
@wireprotov2server.wireprotocommand(b'customreadonly', permission=b'pull') @wireprotov2server.wireprotocommand(b'customreadonly', permission=b'pull')
def customreadonlyv2(repo, proto): def customreadonlyv2(repo, proto):
return wireprototypes.cborresponse(b'customreadonly bytes response') yield b'customreadonly bytes response'
@wireprotov1server.wireprotocommand(b'customreadwrite', permission=b'push') @wireprotov1server.wireprotocommand(b'customreadwrite', permission=b'push')
def customreadwrite(repo, proto): def customreadwrite(repo, proto):
return wireprototypes.bytesresponse(b'customreadwrite bytes response') return wireprototypes.bytesresponse(b'customreadwrite bytes response')
@wireprotov2server.wireprotocommand(b'customreadwrite', permission=b'push') @wireprotov2server.wireprotocommand(b'customreadwrite', permission=b'push')
def customreadwritev2(repo, proto): def customreadwritev2(repo, proto):
return wireprototypes.cborresponse(b'customreadwrite bytes response') yield b'customreadwrite bytes response'
EOF EOF
cat >> $HGRCPATH << EOF cat >> $HGRCPATH << EOF
[extensions] [extensions]
drawdag = $TESTDIR/drawdag.py drawdag = $TESTDIR/drawdag.py
EOF EOF
enabledummycommands() { enabledummycommands() {