This is an archive of the discontinued Mercurial Phabricator instance.

Differential D2951

wireproto: use CBOR for command requests
ClosedPublic

Authored by indygreg on Mar 26 2018, 5:35 PM.

Details

Reviewers
durin42
Group Reviewers
hg-reviewers
Commits
rHG3d0e2cd86e05: wireproto: use CBOR for command requests
Summary

Now that we're using CBOR in the new wire protocol, let's convert
command requests to it.

Before I wrote this patch and was even thinking about CBOR, I was
thinking about how commands should be issued and came to the
conclusion that we didn't need separate frames to represent the
command name from its arguments. I already had a partially
completed patch prepared to merge the frames.

But with CBOR, it makes the implementation a bit simpler because
we don't need to roll our own serialization.

The changes here are a bit invasive. I tried to split this into
multiple commits to make it easier to review. But it was just too
hard.

  • "command name" and "command argument" frames have been collapsed into a "command request" frame.
  • The flags for this new frame are totally different.
  • Frame processing has been overhauled to reflect the new order of things.
  • Test fallout was significant. A handful of tests were removed.

Altogether, I think the new code is simpler. We don't have
complicated state around receiving commands. We're either receiving
command request frames or command data frames. We /could/
potentially collapse command data frames into command request
frames. Although I'd have to think a bit more about this before
I do it.

Diff Detail

Repository
rHG Mercurial
Lint
Automatic diff as part of commit; lint not applicable.
Unit
Automatic diff as part of commit; unit tests not applicable.

Event Timeline

indygreg created this revision.Mar 26 2018, 5:35 PM
durin42 accepted this revision.Apr 3 2018, 11:45 AM
This revision is now accepted and ready to land.Apr 3 2018, 11:45 AM
Closed by commit rHG3d0e2cd86e05: wireproto: use CBOR for command requests (authored by indygreg). · Explain WhyApr 3 2018, 12:50 PM
This revision was automatically updated to reflect the committed changes.

Revision Contents
Changeset List

Diff 7553

mercurial/help/internals/wireprotocol.txt

The sections below define the frame types and their behavior. The sections below define the frame types and their behavior.
Command Request (``0x01``) Command Request (``0x01``)
-------------------------- --------------------------
This frame contains a request to run a command. This frame contains a request to run a command.
The name of the command to run constitutes the entirety of the frame The payload consists of a CBOR map defining the command request. The
payload. bytestring keys of that map are:
name
Name of the command that should be executed (bytestring).
args
Map of bytestring keys to various value types containing the named
arguments to this command.
Each command defines its own set of argument names and their expected
types.
This frame type MUST ONLY be sent from clients to servers: it is illegal This frame type MUST ONLY be sent from clients to servers: it is illegal
for a server to send this frame to a client. for a server to send this frame to a client.
The following flag values are defined for this type: The following flag values are defined for this type:
0x01 0x01
End of command data. When set, the client will not send any command New command request. When set, this frame represents the beginning
arguments or additional command data. When set, the command has been of a new request to run a command. The ``Request ID`` attached to this
fully issued and the server has the full context to process the command. frame MUST NOT be active.
The next frame issued by the client is not part of this command.
0x02 0x02
Command argument frames expected. When set, the client will send Command request continuation. When set, this frame is a continuation
*Command Argument* frames containing command argument data. from a previous command request frame for its ``Request ID``. This
flag is set when the CBOR data for a command request does not fit
in a single frame.
0x04 0x04
Command data frames expected. When set, the client will send Additional frames expected. When set, the command request didn't fit
*Command Data* frames containing a raw stream of data for this into a single frame and additional CBOR data follows in a subsequent
command. frame.
0x08
The ``0x01`` flag is mutually exclusive with both the ``0x02`` and ``0x04`` Command data frames expected. When set, command data frames are
flags. expected to follow the final command request frame for this request.
Command Argument (``0x02``)
---------------------------
This frame contains a named argument for a command.
The frame type MUST ONLY be sent from clients to servers: it is illegal
for a server to send this frame to a client.
The payload consists of:
* A 16-bit little endian integer denoting the length of the ``0x01`` MUST be set on the initial command request frame for a
argument name. ``Request ID``.
* A 16-bit little endian integer denoting the length of the
argument value.
* N bytes of ASCII data containing the argument name.
* N bytes of binary data containing the argument value.
The payload MUST hold the entirety of the 32-bit header and the
argument name. The argument value MAY span multiple frames. If this
occurs, the appropriate frame flag should be set to indicate this.
The following flag values are defined for this type: ``0x01`` or ``0x02`` MUST be set to indicate this frame's role in
a series of command request frames.
0x01 If command data frames are to be sent, ``0x10`` MUST be set on ALL
Argument data continuation. When set, the data for this argument did command request frames.
not fit in a single frame and the next frame will contain additional
argument data.
0x02
End of arguments data. When set, the client will not send any more
command arguments for the command this frame is associated with.
The next frame issued by the client will be command data or
belong to a separate request.
Command Data (``0x03``) Command Data (``0x03``)
----------------------- -----------------------
This frame contains raw data for a command. This frame contains raw data for a command.
Most commands can be executed by specifying arguments. However, Most commands can be executed by specifying arguments. However,
arguments have an upper bound to their length. For commands that arguments have an upper bound to their length. For commands that
TBD TBD
Issuing Commands Issuing Commands
---------------- ----------------
A client can request that a remote run a command by sending it A client can request that a remote run a command by sending it
frames defining that command. This logical stream is composed of frames defining that command. This logical stream is composed of
1 ``Command Request`` frame, 0 or more ``Command Argument`` frames, 1 or more ``Command Request`` frames and and 0 or more ``Command Data``
and 0 or more ``Command Data`` frames. frames.
All frames composing a single command request MUST be associated with All frames composing a single command request MUST be associated with
the same ``Request ID``. the same ``Request ID``.
Clients MAY send additional command requests without waiting on the Clients MAY send additional command requests without waiting on the
response to a previous command request. If they do so, they MUST ensure response to a previous command request. If they do so, they MUST ensure
that the ``Request ID`` field of outbound frames does not conflict that the ``Request ID`` field of outbound frames does not conflict
with that of an active ``Request ID`` whose response has not yet been with that of an active ``Request ID`` whose response has not yet been
fully received. fully received.
Servers MAY respond to commands in a different order than they were Servers MAY respond to commands in a different order than they were
sent over the wire. Clients MUST be prepared to deal with this. Servers sent over the wire. Clients MUST be prepared to deal with this. Servers
also MAY start executing commands in a different order than they were also MAY start executing commands in a different order than they were
received, or MAY execute multiple commands concurrently. received, or MAY execute multiple commands concurrently.
If there is a dependency between commands or a race condition between If there is a dependency between commands or a race condition between
commands executing (e.g. a read-only command that depends on the results commands executing (e.g. a read-only command that depends on the results
of a command that mutates the repository), then clients MUST NOT send of a command that mutates the repository), then clients MUST NOT send
frames issuing a command until a response to all dependent commands has frames issuing a command until a response to all dependent commands has
been received. been received.
TODO think about whether we should express dependencies between commands TODO think about whether we should express dependencies between commands
to avoid roundtrip latency. to avoid roundtrip latency.
Argument frames are the recommended mechanism for transferring fixed A command is defined by a command name, 0 or more command arguments,
sets of parameters to a command. Data frames are appropriate for and optional command data.
transferring variable data. A similar comparison would be to HTTP:
argument frames are headers and the message body is data frames. Arguments are the recommended mechanism for transferring fixed sets of
parameters to a command. Data is appropriate for transferring variable
data. Thinking in terms of HTTP, arguments would be headers and data
would be the message body.
It is recommended for servers to delay the dispatch of a command It is recommended for servers to delay the dispatch of a command
until all argument frames for that command have been received. Servers until all argument have been received. Servers MAY impose limits on the
MAY impose limits on the maximum argument size. maximum argument size.
TODO define failure mechanism. TODO define failure mechanism.
Servers MAY dispatch to commands immediately once argument data Servers MAY dispatch to commands immediately once argument data
is available or delay until command data is received in full. is available or delay until command data is received in full.
Capabilities Capabilities
============ ============

mercurial/wireprotoframing.py

STREAM_FLAG_ENCODING_APPLIED = 0x04 STREAM_FLAG_ENCODING_APPLIED = 0x04
STREAM_FLAGS = { STREAM_FLAGS = {
b'stream-begin': STREAM_FLAG_BEGIN_STREAM, b'stream-begin': STREAM_FLAG_BEGIN_STREAM,
b'stream-end': STREAM_FLAG_END_STREAM, b'stream-end': STREAM_FLAG_END_STREAM,
b'encoded': STREAM_FLAG_ENCODING_APPLIED, b'encoded': STREAM_FLAG_ENCODING_APPLIED,
} }
FRAME_TYPE_COMMAND_NAME = 0x01 FRAME_TYPE_COMMAND_REQUEST = 0x01
FRAME_TYPE_COMMAND_ARGUMENT = 0x02
FRAME_TYPE_COMMAND_DATA = 0x03 FRAME_TYPE_COMMAND_DATA = 0x03
FRAME_TYPE_BYTES_RESPONSE = 0x04 FRAME_TYPE_BYTES_RESPONSE = 0x04
FRAME_TYPE_ERROR_RESPONSE = 0x05 FRAME_TYPE_ERROR_RESPONSE = 0x05
FRAME_TYPE_TEXT_OUTPUT = 0x06 FRAME_TYPE_TEXT_OUTPUT = 0x06
FRAME_TYPE_PROGRESS = 0x07 FRAME_TYPE_PROGRESS = 0x07
FRAME_TYPE_STREAM_SETTINGS = 0x08 FRAME_TYPE_STREAM_SETTINGS = 0x08
FRAME_TYPES = { FRAME_TYPES = {
b'command-name': FRAME_TYPE_COMMAND_NAME, b'command-request': FRAME_TYPE_COMMAND_REQUEST,
b'command-argument': FRAME_TYPE_COMMAND_ARGUMENT,
b'command-data': FRAME_TYPE_COMMAND_DATA, b'command-data': FRAME_TYPE_COMMAND_DATA,
b'bytes-response': FRAME_TYPE_BYTES_RESPONSE, b'bytes-response': FRAME_TYPE_BYTES_RESPONSE,
b'error-response': FRAME_TYPE_ERROR_RESPONSE, b'error-response': FRAME_TYPE_ERROR_RESPONSE,
b'text-output': FRAME_TYPE_TEXT_OUTPUT, b'text-output': FRAME_TYPE_TEXT_OUTPUT,
b'progress': FRAME_TYPE_PROGRESS, b'progress': FRAME_TYPE_PROGRESS,
b'stream-settings': FRAME_TYPE_STREAM_SETTINGS, b'stream-settings': FRAME_TYPE_STREAM_SETTINGS,
} }
FLAG_COMMAND_NAME_EOS = 0x01 FLAG_COMMAND_REQUEST_NEW = 0x01
FLAG_COMMAND_NAME_HAVE_ARGS = 0x02 FLAG_COMMAND_REQUEST_CONTINUATION = 0x02
FLAG_COMMAND_NAME_HAVE_DATA = 0x04 FLAG_COMMAND_REQUEST_MORE_FRAMES = 0x04
FLAG_COMMAND_REQUEST_EXPECT_DATA = 0x08
FLAGS_COMMAND = {
b'eos': FLAG_COMMAND_NAME_EOS, FLAGS_COMMAND_REQUEST = {
b'have-args': FLAG_COMMAND_NAME_HAVE_ARGS, b'new': FLAG_COMMAND_REQUEST_NEW,
b'have-data': FLAG_COMMAND_NAME_HAVE_DATA, b'continuation': FLAG_COMMAND_REQUEST_CONTINUATION,
} b'more': FLAG_COMMAND_REQUEST_MORE_FRAMES,
b'have-data': FLAG_COMMAND_REQUEST_EXPECT_DATA,
FLAG_COMMAND_ARGUMENT_CONTINUATION = 0x01
FLAG_COMMAND_ARGUMENT_EOA = 0x02
FLAGS_COMMAND_ARGUMENT = {
b'continuation': FLAG_COMMAND_ARGUMENT_CONTINUATION,
b'eoa': FLAG_COMMAND_ARGUMENT_EOA,
} }
FLAG_COMMAND_DATA_CONTINUATION = 0x01 FLAG_COMMAND_DATA_CONTINUATION = 0x01
FLAG_COMMAND_DATA_EOS = 0x02 FLAG_COMMAND_DATA_EOS = 0x02
FLAGS_COMMAND_DATA = { FLAGS_COMMAND_DATA = {
b'continuation': FLAG_COMMAND_DATA_CONTINUATION, b'continuation': FLAG_COMMAND_DATA_CONTINUATION,
b'eos': FLAG_COMMAND_DATA_EOS, b'eos': FLAG_COMMAND_DATA_EOS,
FLAGS_ERROR_RESPONSE = { FLAGS_ERROR_RESPONSE = {
b'protocol': FLAG_ERROR_RESPONSE_PROTOCOL, b'protocol': FLAG_ERROR_RESPONSE_PROTOCOL,
b'application': FLAG_ERROR_RESPONSE_APPLICATION, b'application': FLAG_ERROR_RESPONSE_APPLICATION,
} }
# Maps frame types to their available flags. # Maps frame types to their available flags.
FRAME_TYPE_FLAGS = { FRAME_TYPE_FLAGS = {
FRAME_TYPE_COMMAND_NAME: FLAGS_COMMAND, FRAME_TYPE_COMMAND_REQUEST: FLAGS_COMMAND_REQUEST,
FRAME_TYPE_COMMAND_ARGUMENT: FLAGS_COMMAND_ARGUMENT,
FRAME_TYPE_COMMAND_DATA: FLAGS_COMMAND_DATA, FRAME_TYPE_COMMAND_DATA: FLAGS_COMMAND_DATA,
FRAME_TYPE_BYTES_RESPONSE: FLAGS_BYTES_RESPONSE, FRAME_TYPE_BYTES_RESPONSE: FLAGS_BYTES_RESPONSE,
FRAME_TYPE_ERROR_RESPONSE: FLAGS_ERROR_RESPONSE, FRAME_TYPE_ERROR_RESPONSE: FLAGS_ERROR_RESPONSE,
FRAME_TYPE_TEXT_OUTPUT: {}, FRAME_TYPE_TEXT_OUTPUT: {},
FRAME_TYPE_PROGRESS: {}, FRAME_TYPE_PROGRESS: {},
FRAME_TYPE_STREAM_SETTINGS: {}, FRAME_TYPE_STREAM_SETTINGS: {},
} }
ARGUMENT_FRAME_HEADER = struct.Struct(r'<HH') ARGUMENT_RECORD_HEADER = struct.Struct(r'<HH')
@attr.s(slots=True) @attr.s(slots=True)
class frameheader(object): class frameheader(object):
"""Represents the data in a frame header.""" """Represents the data in a frame header."""
length = attr.ib() length = attr.ib()
requestid = attr.ib() requestid = attr.ib()
streamid = attr.ib() streamid = attr.ib()
payload = fh.read(h.length) payload = fh.read(h.length)
if len(payload) != h.length: if len(payload) != h.length:
raise error.Abort(_('frame length error: expected %d; got %d') % raise error.Abort(_('frame length error: expected %d; got %d') %
(h.length, len(payload))) (h.length, len(payload)))
return frame(h.requestid, h.streamid, h.streamflags, h.typeid, h.flags, return frame(h.requestid, h.streamid, h.streamflags, h.typeid, h.flags,
payload) payload)
def createcommandframes(stream, requestid, cmd, args, datafh=None): def createcommandframes(stream, requestid, cmd, args, datafh=None,
maxframesize=DEFAULT_MAX_FRAME_SIZE):
"""Create frames necessary to transmit a request to run a command. """Create frames necessary to transmit a request to run a command.
This is a generator of bytearrays. Each item represents a frame This is a generator of bytearrays. Each item represents a frame
ready to be sent over the wire to a peer. ready to be sent over the wire to a peer.
""" """
flags = 0 data = {b'name': cmd}
if args: if args:
flags |= FLAG_COMMAND_NAME_HAVE_ARGS data[b'args'] = args
if datafh:
flags |= FLAG_COMMAND_NAME_HAVE_DATA
if not flags: data = cbor.dumps(data, canonical=True)
flags |= FLAG_COMMAND_NAME_EOS
yield stream.makeframe(requestid=requestid, typeid=FRAME_TYPE_COMMAND_NAME, offset = 0
flags=flags, payload=cmd)
for i, k in enumerate(sorted(args)): while True:
v = args[k] flags = 0
last = i == len(args) - 1
# TODO handle splitting of argument values across frames. # Must set new or continuation flag.
payload = bytearray(ARGUMENT_FRAME_HEADER.size + len(k) + len(v)) if not offset:
offset = 0 flags |= FLAG_COMMAND_REQUEST_NEW
ARGUMENT_FRAME_HEADER.pack_into(payload, offset, len(k), len(v)) else:
offset += ARGUMENT_FRAME_HEADER.size flags |= FLAG_COMMAND_REQUEST_CONTINUATION
payload[offset:offset + len(k)] = k
offset += len(k) # Data frames is set on all frames.
payload[offset:offset + len(v)] = v if datafh:
flags |= FLAG_COMMAND_REQUEST_EXPECT_DATA
payload = data[offset:offset + maxframesize]
offset += len(payload)
if len(payload) == maxframesize and offset < len(data):
flags |= FLAG_COMMAND_REQUEST_MORE_FRAMES
flags = FLAG_COMMAND_ARGUMENT_EOA if last else 0
yield stream.makeframe(requestid=requestid, yield stream.makeframe(requestid=requestid,
typeid=FRAME_TYPE_COMMAND_ARGUMENT, typeid=FRAME_TYPE_COMMAND_REQUEST,
flags=flags, flags=flags,
payload=payload) payload=payload)
if not (flags & FLAG_COMMAND_REQUEST_MORE_FRAMES):
break
if datafh: if datafh:
while True: while True:
data = datafh.read(DEFAULT_MAX_FRAME_SIZE) data = datafh.read(DEFAULT_MAX_FRAME_SIZE)
done = False done = False
if len(data) == DEFAULT_MAX_FRAME_SIZE: if len(data) == DEFAULT_MAX_FRAME_SIZE:
flags = FLAG_COMMAND_DATA_CONTINUATION flags = FLAG_COMMAND_DATA_CONTINUATION
else: else:
def _makeerrorresult(self, msg): def _makeerrorresult(self, msg):
return 'error', { return 'error', {
'message': msg, 'message': msg,
} }
def _makeruncommandresult(self, requestid): def _makeruncommandresult(self, requestid):
entry = self._receivingcommands[requestid] entry = self._receivingcommands[requestid]
if not entry['requestdone']:
self._state = 'errored'
raise error.ProgrammingError('should not be called without '
'requestdone set')
del self._receivingcommands[requestid] del self._receivingcommands[requestid]
if self._receivingcommands: if self._receivingcommands:
self._state = 'command-receiving' self._state = 'command-receiving'
else: else:
self._state = 'idle' self._state = 'idle'
# Decode the payloads as CBOR.
entry['payload'].seek(0)
request = cbor.load(entry['payload'])
if b'name' not in request:
self._state = 'errored'
return self._makeerrorresult(
_('command request missing "name" field'))
if b'args' not in request:
request[b'args'] = {}
assert requestid not in self._activecommands assert requestid not in self._activecommands
self._activecommands.add(requestid) self._activecommands.add(requestid)
return 'runcommand', { return 'runcommand', {
'requestid': requestid, 'requestid': requestid,
'command': entry['command'], 'command': request[b'name'],
'args': entry['args'], 'args': request[b'args'],
'data': entry['data'].getvalue() if entry['data'] else None, 'data': entry['data'].getvalue() if entry['data'] else None,
} }
def _makewantframeresult(self): def _makewantframeresult(self):
return 'wantframe', { return 'wantframe', {
'state': self._state, 'state': self._state,
} }
def _validatecommandrequestframe(self, frame):
new = frame.flags & FLAG_COMMAND_REQUEST_NEW
continuation = frame.flags & FLAG_COMMAND_REQUEST_CONTINUATION
if new and continuation:
self._state = 'errored'
return self._makeerrorresult(
_('received command request frame with both new and '
'continuation flags set'))
if not new and not continuation:
self._state = 'errored'
return self._makeerrorresult(
_('received command request frame with neither new nor '
'continuation flags set'))
def _onframeidle(self, frame): def _onframeidle(self, frame):
# The only frame type that should be received in this state is a # The only frame type that should be received in this state is a
# command request. # command request.
if frame.typeid != FRAME_TYPE_COMMAND_NAME: if frame.typeid != FRAME_TYPE_COMMAND_REQUEST:
self._state = 'errored' self._state = 'errored'
return self._makeerrorresult( return self._makeerrorresult(
_('expected command frame; got %d') % frame.typeid) _('expected command request frame; got %d') % frame.typeid)
res = self._validatecommandrequestframe(frame)
if res:
return res
if frame.requestid in self._receivingcommands: if frame.requestid in self._receivingcommands:
self._state = 'errored' self._state = 'errored'
return self._makeerrorresult( return self._makeerrorresult(
_('request with ID %d already received') % frame.requestid) _('request with ID %d already received') % frame.requestid)
if frame.requestid in self._activecommands: if frame.requestid in self._activecommands:
self._state = 'errored' self._state = 'errored'
return self._makeerrorresult(( return self._makeerrorresult(
_('request with ID %d is already active') % frame.requestid)) _('request with ID %d is already active') % frame.requestid)
new = frame.flags & FLAG_COMMAND_REQUEST_NEW
moreframes = frame.flags & FLAG_COMMAND_REQUEST_MORE_FRAMES
expectingdata = frame.flags & FLAG_COMMAND_REQUEST_EXPECT_DATA
if not new:
self._state = 'errored'
return self._makeerrorresult(
_('received command request frame without new flag set'))
expectingargs = bool(frame.flags & FLAG_COMMAND_NAME_HAVE_ARGS) payload = util.bytesio()
expectingdata = bool(frame.flags & FLAG_COMMAND_NAME_HAVE_DATA) payload.write(frame.payload)
self._receivingcommands[frame.requestid] = { self._receivingcommands[frame.requestid] = {
'command': frame.payload, 'payload': payload,
'args': {},
'data': None, 'data': None,
'expectingargs': expectingargs, 'requestdone': not moreframes,
'expectingdata': expectingdata, 'expectingdata': bool(expectingdata),
} }
if frame.flags & FLAG_COMMAND_NAME_EOS: # This is the final frame for this request. Dispatch it.
if not moreframes and not expectingdata:
return self._makeruncommandresult(frame.requestid) return self._makeruncommandresult(frame.requestid)
if expectingargs or expectingdata: assert moreframes or expectingdata
self._state = 'command-receiving' self._state = 'command-receiving'
return self._makewantframeresult() return self._makewantframeresult()
else:
self._state = 'errored'
return self._makeerrorresult(_('missing frame flags on '
'command frame'))
def _onframecommandreceiving(self, frame): def _onframecommandreceiving(self, frame):
# It could be a new command request. Process it as such. if frame.typeid == FRAME_TYPE_COMMAND_REQUEST:
if frame.typeid == FRAME_TYPE_COMMAND_NAME: # Process new command requests as such.
if frame.flags & FLAG_COMMAND_REQUEST_NEW:
return self._onframeidle(frame) return self._onframeidle(frame)
res = self._validatecommandrequestframe(frame)
if res:
return res
# All other frames should be related to a command that is currently # All other frames should be related to a command that is currently
# receiving but is not active. # receiving but is not active.
if frame.requestid in self._activecommands: if frame.requestid in self._activecommands:
self._state = 'errored' self._state = 'errored'
return self._makeerrorresult( return self._makeerrorresult(
_('received frame for request that is still active: %d') % _('received frame for request that is still active: %d') %
frame.requestid) frame.requestid)
if frame.requestid not in self._receivingcommands: if frame.requestid not in self._receivingcommands:
self._state = 'errored' self._state = 'errored'
return self._makeerrorresult( return self._makeerrorresult(
_('received frame for request that is not receiving: %d') % _('received frame for request that is not receiving: %d') %
frame.requestid) frame.requestid)
entry = self._receivingcommands[frame.requestid] entry = self._receivingcommands[frame.requestid]
if frame.typeid == FRAME_TYPE_COMMAND_ARGUMENT: if frame.typeid == FRAME_TYPE_COMMAND_REQUEST:
if not entry['expectingargs']: moreframes = frame.flags & FLAG_COMMAND_REQUEST_MORE_FRAMES
expectingdata = bool(frame.flags & FLAG_COMMAND_REQUEST_EXPECT_DATA)
if entry['requestdone']:
self._state = 'errored' self._state = 'errored'
return self._makeerrorresult(_( return self._makeerrorresult(
'received command argument frame for request that is not ' _('received command request frame when request frames '
'expecting arguments: %d') % frame.requestid) 'were supposedly done'))
return self._handlecommandargsframe(frame, entry) if expectingdata != entry['expectingdata']:
self._state = 'errored'
return self._makeerrorresult(
_('mismatch between expect data flag and previous frame'))
entry['payload'].write(frame.payload)
if not moreframes:
entry['requestdone'] = True
if not moreframes and not expectingdata:
return self._makeruncommandresult(frame.requestid)
return self._makewantframeresult()
elif frame.typeid == FRAME_TYPE_COMMAND_DATA: elif frame.typeid == FRAME_TYPE_COMMAND_DATA:
if not entry['expectingdata']: if not entry['expectingdata']:
self._state = 'errored' self._state = 'errored'
return self._makeerrorresult(_( return self._makeerrorresult(_(
'received command data frame for request that is not ' 'received command data frame for request that is not '
'expecting data: %d') % frame.requestid) 'expecting data: %d') % frame.requestid)
if entry['data'] is None: if entry['data'] is None:
entry['data'] = util.bytesio() entry['data'] = util.bytesio()
return self._handlecommanddataframe(frame, entry) return self._handlecommanddataframe(frame, entry)
def _handlecommandargsframe(self, frame, entry):
# The frame and state of command should have already been validated.
assert frame.typeid == FRAME_TYPE_COMMAND_ARGUMENT
offset = 0
namesize, valuesize = ARGUMENT_FRAME_HEADER.unpack_from(frame.payload)
offset += ARGUMENT_FRAME_HEADER.size
# The argument name MUST fit inside the frame.
argname = bytes(frame.payload[offset:offset + namesize])
offset += namesize
if len(argname) != namesize:
self._state = 'errored'
return self._makeerrorresult(_('malformed argument frame: '
'partial argument name'))
argvalue = bytes(frame.payload[offset:])
# Argument value spans multiple frames. Record our active state
# and wait for the next frame.
if frame.flags & FLAG_COMMAND_ARGUMENT_CONTINUATION:
raise error.ProgrammingError('not yet implemented')
# Common case: the argument value is completely contained in this
# frame.
if len(argvalue) != valuesize:
self._state = 'errored'
return self._makeerrorresult(_('malformed argument frame: '
'partial argument value'))
entry['args'][argname] = argvalue
if frame.flags & FLAG_COMMAND_ARGUMENT_EOA:
if entry['expectingdata']:
# TODO signal request to run a command once we don't
# buffer data frames.
return self._makewantframeresult()
else:
return self._makeruncommandresult(frame.requestid)
else: else:
return self._makewantframeresult() self._state = 'errored'
return self._makeerrorresult(_(
'received unexpected frame type: %d') % frame.typeid)
def _handlecommanddataframe(self, frame, entry): def _handlecommanddataframe(self, frame, entry):
assert frame.typeid == FRAME_TYPE_COMMAND_DATA assert frame.typeid == FRAME_TYPE_COMMAND_DATA
# TODO support streaming data instead of buffering it. # TODO support streaming data instead of buffering it.
entry['data'].write(frame.payload) entry['data'].write(frame.payload)
if frame.flags & FLAG_COMMAND_DATA_CONTINUATION: if frame.flags & FLAG_COMMAND_DATA_CONTINUATION:

mercurial/wireprotoserver.py

urlerr = util.urlerr urlerr = util.urlerr
urlreq = util.urlreq urlreq = util.urlreq
HTTP_OK = 200 HTTP_OK = 200
HGTYPE = 'application/mercurial-0.1' HGTYPE = 'application/mercurial-0.1'
HGTYPE2 = 'application/mercurial-0.2' HGTYPE2 = 'application/mercurial-0.2'
HGERRTYPE = 'application/hg-error' HGERRTYPE = 'application/hg-error'
FRAMINGTYPE = b'application/mercurial-exp-framing-0002' FRAMINGTYPE = b'application/mercurial-exp-framing-0003'
HTTPV2 = wireprototypes.HTTPV2 HTTPV2 = wireprototypes.HTTPV2
SSHV1 = wireprototypes.SSHV1 SSHV1 = wireprototypes.SSHV1
SSHV2 = wireprototypes.SSHV2 SSHV2 = wireprototypes.SSHV2
def decodevaluefromheaders(req, headerprefix): def decodevaluefromheaders(req, headerprefix):
"""Decode a long value from multiple HTTP request headers. """Decode a long value from multiple HTTP request headers.

tests/test-http-api-httpv2.t

$ HTTPV2=exp-http-v2-0001 $ HTTPV2=exp-http-v2-0001
$ MEDIATYPE=application/mercurial-exp-framing-0002 $ MEDIATYPE=application/mercurial-exp-framing-0003
$ send() { $ send() {
> hg --verbose debugwireproto --peer raw http://$LOCALIP:$HGPORT/ > hg --verbose debugwireproto --peer raw http://$LOCALIP:$HGPORT/
> } > }
$ cat > dummycommands.py << EOF $ cat > dummycommands.py << EOF
> from mercurial import wireprototypes, wireproto > from mercurial import wireprototypes, wireproto
> @wireproto.wireprotocommand('customreadonly', permission='pull') > @wireproto.wireprotocommand('customreadonly', permission='pull')
s> \r\n s> \r\n
s> makefile('rb', None) s> makefile('rb', None)
s> HTTP/1.1 406 Not Acceptable\r\n s> HTTP/1.1 406 Not Acceptable\r\n
s> Server: testing stub value\r\n s> Server: testing stub value\r\n
s> Date: $HTTP_DATE$\r\n s> Date: $HTTP_DATE$\r\n
s> Content-Type: text/plain\r\n s> Content-Type: text/plain\r\n
s> Content-Length: 85\r\n s> Content-Length: 85\r\n
s> \r\n s> \r\n
s> client MUST specify Accept header with value: application/mercurial-exp-framing-0002\n s> client MUST specify Accept header with value: application/mercurial-exp-framing-0003\n
Bad Accept header results in 406 Bad Accept header results in 406
$ send << EOF $ send << EOF
> httprequest POST api/$HTTPV2/ro/customreadonly > httprequest POST api/$HTTPV2/ro/customreadonly
> accept: invalid > accept: invalid
> user-agent: test > user-agent: test
> EOF > EOF
using raw connection to peer using raw connection to peer
s> POST /api/exp-http-v2-0001/ro/customreadonly HTTP/1.1\r\n s> POST /api/exp-http-v2-0001/ro/customreadonly HTTP/1.1\r\n
s> Accept-Encoding: identity\r\n s> Accept-Encoding: identity\r\n
s> accept: invalid\r\n s> accept: invalid\r\n
s> user-agent: test\r\n s> user-agent: test\r\n
s> host: $LOCALIP:$HGPORT\r\n (glob) s> host: $LOCALIP:$HGPORT\r\n (glob)
s> \r\n s> \r\n
s> makefile('rb', None) s> makefile('rb', None)
s> HTTP/1.1 406 Not Acceptable\r\n s> HTTP/1.1 406 Not Acceptable\r\n
s> Server: testing stub value\r\n s> Server: testing stub value\r\n
s> Date: $HTTP_DATE$\r\n s> Date: $HTTP_DATE$\r\n
s> Content-Type: text/plain\r\n s> Content-Type: text/plain\r\n
s> Content-Length: 85\r\n s> Content-Length: 85\r\n
s> \r\n s> \r\n
s> client MUST specify Accept header with value: application/mercurial-exp-framing-0002\n s> client MUST specify Accept header with value: application/mercurial-exp-framing-0003\n
Bad Content-Type header results in 415 Bad Content-Type header results in 415
$ send << EOF $ send << EOF
> httprequest POST api/$HTTPV2/ro/customreadonly > httprequest POST api/$HTTPV2/ro/customreadonly
> accept: $MEDIATYPE > accept: $MEDIATYPE
> user-agent: test > user-agent: test
> content-type: badmedia > content-type: badmedia
> EOF > EOF
using raw connection to peer using raw connection to peer
s> POST /api/exp-http-v2-0001/ro/customreadonly HTTP/1.1\r\n s> POST /api/exp-http-v2-0001/ro/customreadonly HTTP/1.1\r\n
s> Accept-Encoding: identity\r\n s> Accept-Encoding: identity\r\n
s> accept: application/mercurial-exp-framing-0002\r\n s> accept: application/mercurial-exp-framing-0003\r\n
s> content-type: badmedia\r\n s> content-type: badmedia\r\n
s> user-agent: test\r\n s> user-agent: test\r\n
s> host: $LOCALIP:$HGPORT\r\n (glob) s> host: $LOCALIP:$HGPORT\r\n (glob)
s> \r\n s> \r\n
s> makefile('rb', None) s> makefile('rb', None)
s> HTTP/1.1 415 Unsupported Media Type\r\n s> HTTP/1.1 415 Unsupported Media Type\r\n
s> Server: testing stub value\r\n s> Server: testing stub value\r\n
s> Date: $HTTP_DATE$\r\n s> Date: $HTTP_DATE$\r\n
s> Content-Type: text/plain\r\n s> Content-Type: text/plain\r\n
s> Content-Length: 88\r\n s> Content-Length: 88\r\n
s> \r\n s> \r\n
s> client MUST send Content-Type header with value: application/mercurial-exp-framing-0002\n s> client MUST send Content-Type header with value: application/mercurial-exp-framing-0003\n
Request to read-only command works out of the box Request to read-only command works out of the box
$ send << EOF $ send << EOF
> httprequest POST api/$HTTPV2/ro/customreadonly > httprequest POST api/$HTTPV2/ro/customreadonly
> accept: $MEDIATYPE > accept: $MEDIATYPE
> content-type: $MEDIATYPE > content-type: $MEDIATYPE
> user-agent: test > user-agent: test
> frame 1 1 stream-begin command-name eos customreadonly > frame 1 1 stream-begin command-request new cbor:{b'name': b'customreadonly'}
> EOF > EOF
using raw connection to peer using raw connection to peer
s> POST /api/exp-http-v2-0001/ro/customreadonly HTTP/1.1\r\n s> POST /api/exp-http-v2-0001/ro/customreadonly HTTP/1.1\r\n
s> Accept-Encoding: identity\r\n s> Accept-Encoding: identity\r\n
s> accept: application/mercurial-exp-framing-0002\r\n
s> content-type: application/mercurial-exp-framing-0002\r\n
s> user-agent: test\r\n
s> *\r\n (glob) s> *\r\n (glob)
s> content-type: application/mercurial-exp-framing-0003\r\n
s> user-agent: test\r\n
s> content-length: 29\r\n
s> host: $LOCALIP:$HGPORT\r\n (glob) s> host: $LOCALIP:$HGPORT\r\n (glob)
s> \r\n s> \r\n
s> \x0e\x00\x00\x01\x00\x01\x01\x11customreadonly s> \x15\x00\x00\x01\x00\x01\x01\x11\xa1DnameNcustomreadonly
s> makefile('rb', None) s> makefile('rb', None)
s> HTTP/1.1 200 OK\r\n s> HTTP/1.1 200 OK\r\n
s> Server: testing stub value\r\n s> Server: testing stub value\r\n
s> Date: $HTTP_DATE$\r\n s> Date: $HTTP_DATE$\r\n
s> Content-Type: application/mercurial-exp-framing-0002\r\n s> Content-Type: application/mercurial-exp-framing-0003\r\n
s> Transfer-Encoding: chunked\r\n s> Transfer-Encoding: chunked\r\n
s> \r\n s> \r\n
s> 25\r\n s> 25\r\n
s> \x1d\x00\x00\x01\x00\x02\x01Bcustomreadonly bytes response s> \x1d\x00\x00\x01\x00\x02\x01Bcustomreadonly bytes response
s> \r\n s> \r\n
s> 0\r\n s> 0\r\n
s> \r\n s> \r\n
Authorized request for valid read-write command works Authorized request for valid read-write command works
$ send << EOF $ send << EOF
> httprequest POST api/$HTTPV2/rw/customreadonly > httprequest POST api/$HTTPV2/rw/customreadonly
> user-agent: test > user-agent: test
> accept: $MEDIATYPE > accept: $MEDIATYPE
> content-type: $MEDIATYPE > content-type: $MEDIATYPE
> frame 1 1 stream-begin command-name eos customreadonly > frame 1 1 stream-begin command-request new cbor:{b'name': b'customreadonly'}
> EOF > EOF
using raw connection to peer using raw connection to peer
s> POST /api/exp-http-v2-0001/rw/customreadonly HTTP/1.1\r\n s> POST /api/exp-http-v2-0001/rw/customreadonly HTTP/1.1\r\n
s> Accept-Encoding: identity\r\n s> Accept-Encoding: identity\r\n
s> accept: application/mercurial-exp-framing-0002\r\n s> accept: application/mercurial-exp-framing-0003\r\n
s> content-type: application/mercurial-exp-framing-0002\r\n s> content-type: application/mercurial-exp-framing-0003\r\n
s> user-agent: test\r\n s> user-agent: test\r\n
s> content-length: 22\r\n s> content-length: 29\r\n
s> host: $LOCALIP:$HGPORT\r\n (glob) s> host: $LOCALIP:$HGPORT\r\n (glob)
s> \r\n s> \r\n
s> \x0e\x00\x00\x01\x00\x01\x01\x11customreadonly s> \x15\x00\x00\x01\x00\x01\x01\x11\xa1DnameNcustomreadonly
s> makefile('rb', None) s> makefile('rb', None)
s> HTTP/1.1 200 OK\r\n s> HTTP/1.1 200 OK\r\n
s> Server: testing stub value\r\n s> Server: testing stub value\r\n
s> Date: $HTTP_DATE$\r\n s> Date: $HTTP_DATE$\r\n
s> Content-Type: application/mercurial-exp-framing-0002\r\n s> Content-Type: application/mercurial-exp-framing-0003\r\n
s> Transfer-Encoding: chunked\r\n s> Transfer-Encoding: chunked\r\n
s> \r\n s> \r\n
s> 25\r\n s> 25\r\n
s> \x1d\x00\x00\x01\x00\x02\x01Bcustomreadonly bytes response s> \x1d\x00\x00\x01\x00\x02\x01Bcustomreadonly bytes response
s> \r\n s> \r\n
s> 0\r\n s> 0\r\n
s> \r\n s> \r\n
Authorized request for unknown command is rejected Authorized request for unknown command is rejected
$ send << EOF $ send << EOF
> httprequest POST api/$HTTPV2/rw/badcommand > httprequest POST api/$HTTPV2/rw/badcommand
> user-agent: test > user-agent: test
> accept: $MEDIATYPE > accept: $MEDIATYPE
> EOF > EOF
using raw connection to peer using raw connection to peer
s> POST /api/exp-http-v2-0001/rw/badcommand HTTP/1.1\r\n s> POST /api/exp-http-v2-0001/rw/badcommand HTTP/1.1\r\n
s> Accept-Encoding: identity\r\n s> Accept-Encoding: identity\r\n
s> accept: application/mercurial-exp-framing-0002\r\n s> accept: application/mercurial-exp-framing-0003\r\n
s> user-agent: test\r\n s> user-agent: test\r\n
s> host: $LOCALIP:$HGPORT\r\n (glob) s> host: $LOCALIP:$HGPORT\r\n (glob)
s> \r\n s> \r\n
s> makefile('rb', None) s> makefile('rb', None)
s> HTTP/1.1 404 Not Found\r\n s> HTTP/1.1 404 Not Found\r\n
s> Server: testing stub value\r\n s> Server: testing stub value\r\n
s> Date: $HTTP_DATE$\r\n s> Date: $HTTP_DATE$\r\n
s> Content-Type: text/plain\r\n s> Content-Type: text/plain\r\n
Command frames can be reflected via debugreflect Command frames can be reflected via debugreflect
$ send << EOF $ send << EOF
> httprequest POST api/$HTTPV2/ro/debugreflect > httprequest POST api/$HTTPV2/ro/debugreflect
> accept: $MEDIATYPE > accept: $MEDIATYPE
> content-type: $MEDIATYPE > content-type: $MEDIATYPE
> user-agent: test > user-agent: test
> frame 1 1 stream-begin command-name have-args command1 > frame 1 1 stream-begin command-request new cbor:{b'name': b'command1', b'args': {b'foo': b'val1', b'bar1': b'val'}}
> frame 1 1 0 command-argument 0 \x03\x00\x04\x00fooval1
> frame 1 1 0 command-argument eoa \x04\x00\x03\x00bar1val
> EOF > EOF
using raw connection to peer using raw connection to peer
s> POST /api/exp-http-v2-0001/ro/debugreflect HTTP/1.1\r\n s> POST /api/exp-http-v2-0001/ro/debugreflect HTTP/1.1\r\n
s> Accept-Encoding: identity\r\n s> Accept-Encoding: identity\r\n
s> accept: application/mercurial-exp-framing-0002\r\n s> accept: application/mercurial-exp-framing-0003\r\n
s> content-type: application/mercurial-exp-framing-0002\r\n s> content-type: application/mercurial-exp-framing-0003\r\n
s> user-agent: test\r\n s> user-agent: test\r\n
s> content-length: 54\r\n s> content-length: 47\r\n
s> host: $LOCALIP:$HGPORT\r\n (glob) s> host: $LOCALIP:$HGPORT\r\n (glob)
s> \r\n s> \r\n
s> \x08\x00\x00\x01\x00\x01\x01\x12command1\x0b\x00\x00\x01\x00\x01\x00 \x03\x00\x04\x00fooval1\x0b\x00\x00\x01\x00\x01\x00"\x04\x00\x03\x00bar1val s> '\x00\x00\x01\x00\x01\x01\x11\xa2Dargs\xa2CfooDval1Dbar1CvalDnameHcommand1
s> makefile('rb', None) s> makefile('rb', None)
s> HTTP/1.1 200 OK\r\n s> HTTP/1.1 200 OK\r\n
s> Server: testing stub value\r\n s> Server: testing stub value\r\n
s> Date: $HTTP_DATE$\r\n s> Date: $HTTP_DATE$\r\n
s> Content-Type: text/plain\r\n s> Content-Type: text/plain\r\n
s> Content-Length: 322\r\n s> Content-Length: 205\r\n
s> \r\n s> \r\n
s> received: 1 2 1 command1\n s> received: 1 1 1 \xa2Dargs\xa2CfooDval1Dbar1CvalDnameHcommand1\n
s> ["wantframe", {"state": "command-receiving"}]\n
s> received: 2 0 1 \x03\x00\x04\x00fooval1\n
s> ["wantframe", {"state": "command-receiving"}]\n
s> received: 2 2 1 \x04\x00\x03\x00bar1val\n
s> ["runcommand", {"args": {"bar1": "val", "foo": "val1"}, "command": "command1", "data": null, "requestid": 1}]\n s> ["runcommand", {"args": {"bar1": "val", "foo": "val1"}, "command": "command1", "data": null, "requestid": 1}]\n
s> received: <no frame>\n s> received: <no frame>\n
s> {"action": "noop"} s> {"action": "noop"}
Multiple requests to regular command URL are not allowed Multiple requests to regular command URL are not allowed
$ send << EOF $ send << EOF
> httprequest POST api/$HTTPV2/ro/customreadonly > httprequest POST api/$HTTPV2/ro/customreadonly
> accept: $MEDIATYPE > accept: $MEDIATYPE
> content-type: $MEDIATYPE > content-type: $MEDIATYPE
> user-agent: test > user-agent: test
> frame 1 1 stream-begin command-name eos customreadonly > frame 1 1 stream-begin command-request new cbor:{b'name': b'customreadonly'}
> frame 3 1 0 command-name eos customreadonly
> EOF > EOF
using raw connection to peer using raw connection to peer
s> POST /api/exp-http-v2-0001/ro/customreadonly HTTP/1.1\r\n s> POST /api/exp-http-v2-0001/ro/customreadonly HTTP/1.1\r\n
s> Accept-Encoding: identity\r\n s> Accept-Encoding: identity\r\n
s> accept: application/mercurial-exp-framing-0002\r\n s> accept: application/mercurial-exp-framing-0003\r\n
s> content-type: application/mercurial-exp-framing-0002\r\n s> content-type: application/mercurial-exp-framing-0003\r\n
s> user-agent: test\r\n s> user-agent: test\r\n
s> content-length: 44\r\n s> content-length: 29\r\n
s> host: $LOCALIP:$HGPORT\r\n (glob) s> host: $LOCALIP:$HGPORT\r\n (glob)
s> \r\n s> \r\n
s> \x0e\x00\x00\x01\x00\x01\x01\x11customreadonly\x0e\x00\x00\x03\x00\x01\x00\x11customreadonly s> \x15\x00\x00\x01\x00\x01\x01\x11\xa1DnameNcustomreadonly
s> makefile('rb', None) s> makefile('rb', None)
s> HTTP/1.1 200 OK\r\n s> HTTP/1.1 200 OK\r\n
s> Server: testing stub value\r\n s> Server: testing stub value\r\n
s> Date: $HTTP_DATE$\r\n s> Date: $HTTP_DATE$\r\n
s> Content-Type: text/plain\r\n s> Content-Type: application/mercurial-exp-framing-0003\r\n
s> Content-Length: 46\r\n s> Transfer-Encoding: chunked\r\n
s> \r\n
s> 25\r\n
s> \x1d\x00\x00\x01\x00\x02\x01Bcustomreadonly bytes response
s> \r\n
s> 0\r\n
s> \r\n s> \r\n
s> multiple commands cannot be issued to this URL
Multiple requests to "multirequest" URL are allowed Multiple requests to "multirequest" URL are allowed
$ send << EOF $ send << EOF
> httprequest POST api/$HTTPV2/ro/multirequest > httprequest POST api/$HTTPV2/ro/multirequest
> accept: $MEDIATYPE > accept: $MEDIATYPE
> content-type: $MEDIATYPE > content-type: $MEDIATYPE
> user-agent: test > user-agent: test
> frame 1 1 stream-begin command-name eos customreadonly > frame 1 1 stream-begin command-request new cbor:{b'name': b'customreadonly'}
> frame 3 1 0 command-name eos customreadonly > frame 3 1 0 command-request new cbor:{b'name': b'customreadonly'}
> EOF > EOF
using raw connection to peer using raw connection to peer
s> POST /api/exp-http-v2-0001/ro/multirequest HTTP/1.1\r\n s> POST /api/exp-http-v2-0001/ro/multirequest HTTP/1.1\r\n
s> Accept-Encoding: identity\r\n s> Accept-Encoding: identity\r\n
s> accept: application/mercurial-exp-framing-0002\r\n
s> content-type: application/mercurial-exp-framing-0002\r\n
s> user-agent: test\r\n
s> *\r\n (glob) s> *\r\n (glob)
s> *\r\n (glob)
s> user-agent: test\r\n
s> content-length: 58\r\n
s> host: $LOCALIP:$HGPORT\r\n (glob) s> host: $LOCALIP:$HGPORT\r\n (glob)
s> \r\n s> \r\n
s> \x0e\x00\x00\x01\x00\x01\x01\x11customreadonly\x0e\x00\x00\x03\x00\x01\x00\x11customreadonly s> \x15\x00\x00\x01\x00\x01\x01\x11\xa1DnameNcustomreadonly\x15\x00\x00\x03\x00\x01\x00\x11\xa1DnameNcustomreadonly
s> makefile('rb', None) s> makefile('rb', None)
s> HTTP/1.1 200 OK\r\n s> HTTP/1.1 200 OK\r\n
s> Server: testing stub value\r\n s> Server: testing stub value\r\n
s> Date: $HTTP_DATE$\r\n s> Date: $HTTP_DATE$\r\n
s> Content-Type: application/mercurial-exp-framing-0002\r\n s> Content-Type: application/mercurial-exp-framing-0003\r\n
s> Transfer-Encoding: chunked\r\n s> Transfer-Encoding: chunked\r\n
s> \r\n s> \r\n
s> *\r\n (glob) s> 25\r\n
s> \x1d\x00\x00\x01\x00\x02\x01Bcustomreadonly bytes response s> \x1d\x00\x00\x01\x00\x02\x01Bcustomreadonly bytes response
s> \r\n s> \r\n
s> 25\r\n s> 25\r\n
s> \x1d\x00\x00\x03\x00\x02\x00Bcustomreadonly bytes response s> \x1d\x00\x00\x03\x00\x02\x00Bcustomreadonly bytes response
s> \r\n s> \r\n
s> 0\r\n s> 0\r\n
s> \r\n s> \r\n
Interleaved requests to "multirequest" are processed Interleaved requests to "multirequest" are processed
$ send << EOF $ send << EOF
> httprequest POST api/$HTTPV2/ro/multirequest > httprequest POST api/$HTTPV2/ro/multirequest
> accept: $MEDIATYPE > accept: $MEDIATYPE
> content-type: $MEDIATYPE > content-type: $MEDIATYPE
> user-agent: test > user-agent: test
> frame 1 1 stream-begin command-name have-args listkeys > frame 1 1 stream-begin command-request new|more \xa2Dargs\xa1Inamespace
> frame 3 1 0 command-name have-args listkeys > frame 3 1 0 command-request new|more \xa2Dargs\xa1Inamespace
> frame 3 1 0 command-argument eoa \x09\x00\x09\x00namespacebookmarks > frame 3 1 0 command-request continuation JnamespacesDnameHlistkeys
> frame 1 1 0 command-argument eoa \x09\x00\x0a\x00namespacenamespaces > frame 1 1 0 command-request continuation IbookmarksDnameHlistkeys
> EOF > EOF
using raw connection to peer using raw connection to peer
s> POST /api/exp-http-v2-0001/ro/multirequest HTTP/1.1\r\n s> POST /api/exp-http-v2-0001/ro/multirequest HTTP/1.1\r\n
s> Accept-Encoding: identity\r\n s> Accept-Encoding: identity\r\n
s> accept: application/mercurial-exp-framing-0002\r\n s> accept: application/mercurial-exp-framing-0003\r\n
s> content-type: application/mercurial-exp-framing-0002\r\n s> content-type: application/mercurial-exp-framing-0003\r\n
s> user-agent: test\r\n s> user-agent: test\r\n
s> content-length: 93\r\n s> content-length: 115\r\n
s> host: $LOCALIP:$HGPORT\r\n (glob) s> host: $LOCALIP:$HGPORT\r\n (glob)
s> \r\n s> \r\n
s> \x08\x00\x00\x01\x00\x01\x01\x12listkeys\x08\x00\x00\x03\x00\x01\x00\x12listkeys\x16\x00\x00\x03\x00\x01\x00" \x00 \x00namespacebookmarks\x17\x00\x00\x01\x00\x01\x00" \x00\n s> \x11\x00\x00\x01\x00\x01\x01\x15\xa2Dargs\xa1Inamespace\x11\x00\x00\x03\x00\x01\x00\x15\xa2Dargs\xa1Inamespace\x19\x00\x00\x03\x00\x01\x00\x12JnamespacesDnameHlistkeys\x18\x00\x00\x01\x00\x01\x00\x12IbookmarksDnameHlistkeys
s> \x00namespacenamespaces
s> makefile('rb', None) s> makefile('rb', None)
s> HTTP/1.1 200 OK\r\n s> HTTP/1.1 200 OK\r\n
s> Server: testing stub value\r\n s> Server: testing stub value\r\n
s> Date: $HTTP_DATE$\r\n s> Date: $HTTP_DATE$\r\n
s> Content-Type: application/mercurial-exp-framing-0002\r\n s> Content-Type: application/mercurial-exp-framing-0003\r\n
s> Transfer-Encoding: chunked\r\n s> Transfer-Encoding: chunked\r\n
s> \r\n s> \r\n
s> 8\r\n
s> \x00\x00\x00\x03\x00\x02\x01B
s> \r\n
s> 26\r\n s> 26\r\n
s> \x1e\x00\x00\x01\x00\x02\x00Bbookmarks \n s> \x1e\x00\x00\x03\x00\x02\x01Bbookmarks \n
s> namespaces \n s> namespaces \n
s> phases s> phases
s> \r\n s> \r\n
s> 8\r\n
s> \x00\x00\x00\x01\x00\x02\x00B
s> \r\n
s> 0\r\n s> 0\r\n
s> \r\n s> \r\n
Restart server to disable read-write access Restart server to disable read-write access
$ killdaemons.py $ killdaemons.py
$ cat > server/.hg/hgrc << EOF $ cat > server/.hg/hgrc << EOF
> [experimental] > [experimental]
Attempting to run a read-write command via multirequest on read-only URL is not allowed Attempting to run a read-write command via multirequest on read-only URL is not allowed
$ send << EOF $ send << EOF
> httprequest POST api/$HTTPV2/ro/multirequest > httprequest POST api/$HTTPV2/ro/multirequest
> accept: $MEDIATYPE > accept: $MEDIATYPE
> content-type: $MEDIATYPE > content-type: $MEDIATYPE
> user-agent: test > user-agent: test
> frame 1 1 stream-begin command-name eos unbundle > frame 1 1 stream-begin command-request new cbor:{b'name': b'unbundle'}
> EOF > EOF
using raw connection to peer using raw connection to peer
s> POST /api/exp-http-v2-0001/ro/multirequest HTTP/1.1\r\n s> POST /api/exp-http-v2-0001/ro/multirequest HTTP/1.1\r\n
s> Accept-Encoding: identity\r\n s> Accept-Encoding: identity\r\n
s> accept: application/mercurial-exp-framing-0002\r\n s> accept: application/mercurial-exp-framing-0003\r\n
s> content-type: application/mercurial-exp-framing-0002\r\n s> content-type: application/mercurial-exp-framing-0003\r\n
s> user-agent: test\r\n s> user-agent: test\r\n
s> content-length: 16\r\n s> content-length: 23\r\n
s> host: $LOCALIP:$HGPORT\r\n (glob) s> host: $LOCALIP:$HGPORT\r\n (glob)
s> \r\n s> \r\n
s> \x08\x00\x00\x01\x00\x01\x01\x11unbundle s> \x0f\x00\x00\x01\x00\x01\x01\x11\xa1DnameHunbundle
s> makefile('rb', None) s> makefile('rb', None)
s> HTTP/1.1 403 Forbidden\r\n s> HTTP/1.1 403 Forbidden\r\n
s> Server: testing stub value\r\n s> Server: testing stub value\r\n
s> Date: $HTTP_DATE$\r\n s> Date: $HTTP_DATE$\r\n
s> Content-Type: text/plain\r\n s> Content-Type: text/plain\r\n
s> Content-Length: 53\r\n s> Content-Length: 53\r\n
s> \r\n s> \r\n
s> insufficient permissions to execute command: unbundle s> insufficient permissions to execute command: unbundle
$ cat error.log $ cat error.log

tests/test-wireproto-serverreactor.py

from __future__ import absolute_import, print_function from __future__ import absolute_import, print_function
import unittest import unittest
from mercurial.thirdparty import (
cbor,
)
from mercurial import ( from mercurial import (
util, util,
wireprotoframing as framing, wireprotoframing as framing,
) )
ffs = framing.makeframefromhumanstring ffs = framing.makeframefromhumanstring
def makereactor(deferoutput=False): def makereactor(deferoutput=False):
class FrameTests(unittest.TestCase): class FrameTests(unittest.TestCase):
def testdataexactframesize(self): def testdataexactframesize(self):
data = util.bytesio(b'x' * framing.DEFAULT_MAX_FRAME_SIZE) data = util.bytesio(b'x' * framing.DEFAULT_MAX_FRAME_SIZE)
stream = framing.stream(1) stream = framing.stream(1)
frames = list(framing.createcommandframes(stream, 1, b'command', frames = list(framing.createcommandframes(stream, 1, b'command',
{}, data)) {}, data))
self.assertEqual(frames, [ self.assertEqual(frames, [
ffs(b'1 1 stream-begin command-name have-data command'), ffs(b'1 1 stream-begin command-request new|have-data '
b"cbor:{b'name': b'command'}"),
ffs(b'1 1 0 command-data continuation %s' % data.getvalue()), ffs(b'1 1 0 command-data continuation %s' % data.getvalue()),
ffs(b'1 1 0 command-data eos ') ffs(b'1 1 0 command-data eos ')
]) ])
def testdatamultipleframes(self): def testdatamultipleframes(self):
data = util.bytesio(b'x' * (framing.DEFAULT_MAX_FRAME_SIZE + 1)) data = util.bytesio(b'x' * (framing.DEFAULT_MAX_FRAME_SIZE + 1))
stream = framing.stream(1) stream = framing.stream(1)
frames = list(framing.createcommandframes(stream, 1, b'command', {}, frames = list(framing.createcommandframes(stream, 1, b'command', {},
data)) data))
self.assertEqual(frames, [ self.assertEqual(frames, [
ffs(b'1 1 stream-begin command-name have-data command'), ffs(b'1 1 stream-begin command-request new|have-data '
b"cbor:{b'name': b'command'}"),
ffs(b'1 1 0 command-data continuation %s' % ( ffs(b'1 1 0 command-data continuation %s' % (
b'x' * framing.DEFAULT_MAX_FRAME_SIZE)), b'x' * framing.DEFAULT_MAX_FRAME_SIZE)),
ffs(b'1 1 0 command-data eos x'), ffs(b'1 1 0 command-data eos x'),
]) ])
def testargsanddata(self): def testargsanddata(self):
data = util.bytesio(b'x' * 100) data = util.bytesio(b'x' * 100)
stream = framing.stream(1) stream = framing.stream(1)
frames = list(framing.createcommandframes(stream, 1, b'command', { frames = list(framing.createcommandframes(stream, 1, b'command', {
b'key1': b'key1value', b'key1': b'key1value',
b'key2': b'key2value', b'key2': b'key2value',
b'key3': b'key3value', b'key3': b'key3value',
}, data)) }, data))
self.assertEqual(frames, [ self.assertEqual(frames, [
ffs(b'1 1 stream-begin command-name have-args|have-data command'), ffs(b'1 1 stream-begin command-request new|have-data '
ffs(br'1 1 0 command-argument 0 \x04\x00\x09\x00key1key1value'), b"cbor:{b'name': b'command', b'args': {b'key1': b'key1value', "
ffs(br'1 1 0 command-argument 0 \x04\x00\x09\x00key2key2value'), b"b'key2': b'key2value', b'key3': b'key3value'}}"),
ffs(br'1 1 0 command-argument eoa \x04\x00\x09\x00key3key3value'),
ffs(b'1 1 0 command-data eos %s' % data.getvalue()), ffs(b'1 1 0 command-data eos %s' % data.getvalue()),
]) ])
def testtextoutputexcessiveargs(self): def testtextoutputexcessiveargs(self):
"""At most 255 formatting arguments are allowed.""" """At most 255 formatting arguments are allowed."""
with self.assertRaisesRegexp(ValueError, with self.assertRaisesRegexp(ValueError,
'cannot use more than 255 formatting'): 'cannot use more than 255 formatting'):
args = [b'x' for i in range(256)] args = [b'x' for i in range(256)]
result = reactor.oninputeof() result = reactor.oninputeof()
self.assertaction(result, 'noop') self.assertaction(result, 'noop')
def test1argument(self): def test1argument(self):
reactor = makereactor() reactor = makereactor()
stream = framing.stream(1) stream = framing.stream(1)
results = list(sendcommandframes(reactor, stream, 41, b'mycommand', results = list(sendcommandframes(reactor, stream, 41, b'mycommand',
{b'foo': b'bar'})) {b'foo': b'bar'}))
self.assertEqual(len(results), 2) self.assertEqual(len(results), 1)
self.assertaction(results[0], 'wantframe') self.assertaction(results[0], 'runcommand')
self.assertaction(results[1], 'runcommand') self.assertEqual(results[0][1], {
self.assertEqual(results[1][1], {
'requestid': 41, 'requestid': 41,
'command': b'mycommand', 'command': b'mycommand',
'args': {b'foo': b'bar'}, 'args': {b'foo': b'bar'},
'data': None, 'data': None,
}) })
def testmultiarguments(self): def testmultiarguments(self):
reactor = makereactor() reactor = makereactor()
stream = framing.stream(1) stream = framing.stream(1)
results = list(sendcommandframes(reactor, stream, 1, b'mycommand', results = list(sendcommandframes(reactor, stream, 1, b'mycommand',
{b'foo': b'bar', b'biz': b'baz'})) {b'foo': b'bar', b'biz': b'baz'}))
self.assertEqual(len(results), 3) self.assertEqual(len(results), 1)
self.assertaction(results[0], 'wantframe') self.assertaction(results[0], 'runcommand')
self.assertaction(results[1], 'wantframe') self.assertEqual(results[0][1], {
self.assertaction(results[2], 'runcommand')
self.assertEqual(results[2][1], {
'requestid': 1, 'requestid': 1,
'command': b'mycommand', 'command': b'mycommand',
'args': {b'foo': b'bar', b'biz': b'baz'}, 'args': {b'foo': b'bar', b'biz': b'baz'},
'data': None, 'data': None,
}) })
def testsimplecommanddata(self): def testsimplecommanddata(self):
reactor = makereactor() reactor = makereactor()
stream = framing.stream(1) stream = framing.stream(1)
results = list(sendcommandframes(reactor, stream, 1, b'mycommand', {}, results = list(sendcommandframes(reactor, stream, 1, b'mycommand', {},
util.bytesio(b'data!'))) util.bytesio(b'data!')))
self.assertEqual(len(results), 2) self.assertEqual(len(results), 2)
self.assertaction(results[0], 'wantframe') self.assertaction(results[0], 'wantframe')
self.assertaction(results[1], 'runcommand') self.assertaction(results[1], 'runcommand')
self.assertEqual(results[1][1], { self.assertEqual(results[1][1], {
'requestid': 1, 'requestid': 1,
'command': b'mycommand', 'command': b'mycommand',
'args': {}, 'args': {},
'data': b'data!', 'data': b'data!',
}) })
def testmultipledataframes(self): def testmultipledataframes(self):
frames = [ frames = [
ffs(b'1 1 stream-begin command-name have-data mycommand'), ffs(b'1 1 stream-begin command-request new|have-data '
b"cbor:{b'name': b'mycommand'}"),
ffs(b'1 1 0 command-data continuation data1'), ffs(b'1 1 0 command-data continuation data1'),
ffs(b'1 1 0 command-data continuation data2'), ffs(b'1 1 0 command-data continuation data2'),
ffs(b'1 1 0 command-data eos data3'), ffs(b'1 1 0 command-data eos data3'),
] ]
reactor = makereactor() reactor = makereactor()
results = list(sendframes(reactor, frames)) results = list(sendframes(reactor, frames))
self.assertEqual(len(results), 4) self.assertEqual(len(results), 4)
for i in range(3): for i in range(3):
self.assertaction(results[i], 'wantframe') self.assertaction(results[i], 'wantframe')
self.assertaction(results[3], 'runcommand') self.assertaction(results[3], 'runcommand')
self.assertEqual(results[3][1], { self.assertEqual(results[3][1], {
'requestid': 1, 'requestid': 1,
'command': b'mycommand', 'command': b'mycommand',
'args': {}, 'args': {},
'data': b'data1data2data3', 'data': b'data1data2data3',
}) })
def testargumentanddata(self): def testargumentanddata(self):
frames = [ frames = [
ffs(b'1 1 stream-begin command-name have-args|have-data command'), ffs(b'1 1 stream-begin command-request new|have-data '
ffs(br'1 1 0 command-argument 0 \x03\x00\x03\x00keyval'), b"cbor:{b'name': b'command', b'args': {b'key': b'val',"
ffs(br'1 1 0 command-argument eoa \x03\x00\x03\x00foobar'), b"b'foo': b'bar'}}"),
ffs(b'1 1 0 command-data continuation value1'), ffs(b'1 1 0 command-data continuation value1'),
ffs(b'1 1 0 command-data eos value2'), ffs(b'1 1 0 command-data eos value2'),
] ]
reactor = makereactor() reactor = makereactor()
results = list(sendframes(reactor, frames)) results = list(sendframes(reactor, frames))
self.assertaction(results[-1], 'runcommand') self.assertaction(results[-1], 'runcommand')
self.assertEqual(results[-1][1], { self.assertEqual(results[-1][1], {
'requestid': 1, 'requestid': 1,
'command': b'command', 'command': b'command',
'args': { 'args': {
b'key': b'val', b'key': b'val',
b'foo': b'bar', b'foo': b'bar',
}, },
'data': b'value1value2', 'data': b'value1value2',
}) })
def testunexpectedcommandargument(self): def testnewandcontinuation(self):
"""Command argument frame when not running a command is an error.""" result = self._sendsingleframe(makereactor(),
result = self._sendsingleframe( ffs(b'1 1 stream-begin command-request new|continuation '))
makereactor(), ffs(b'1 1 stream-begin command-argument 0 ignored'))
self.assertaction(result, 'error') self.assertaction(result, 'error')
self.assertEqual(result[1], { self.assertEqual(result[1], {
'message': b'expected command frame; got 2', 'message': b'received command request frame with both new and '
b'continuation flags set',
}) })
def testunexpectedcommandargumentreceiving(self): def testneithernewnorcontinuation(self):
"""Same as above but the command is receiving.""" result = self._sendsingleframe(makereactor(),
results = list(sendframes(makereactor(), [ ffs(b'1 1 stream-begin command-request 0 '))
ffs(b'1 1 stream-begin command-name have-data command'), self.assertaction(result, 'error')
ffs(b'1 1 0 command-argument eoa ignored'), self.assertEqual(result[1], {
])) 'message': b'received command request frame with neither new nor '
b'continuation flags set',
self.assertaction(results[1], 'error')
self.assertEqual(results[1][1], {
'message': b'received command argument frame for request that is '
b'not expecting arguments: 1',
}) })
def testunexpectedcommanddata(self): def testunexpectedcommanddata(self):
"""Command argument frame when not running a command is an error.""" """Command data frame when not running a command is an error."""
result = self._sendsingleframe( result = self._sendsingleframe(makereactor(),
makereactor(), ffs(b'1 1 stream-begin command-data 0 ignored')) ffs(b'1 1 stream-begin command-data 0 ignored'))
self.assertaction(result, 'error') self.assertaction(result, 'error')
self.assertEqual(result[1], { self.assertEqual(result[1], {
'message': b'expected command frame; got 3', 'message': b'expected command request frame; got 3',
}) })
def testunexpectedcommanddatareceiving(self): def testunexpectedcommanddatareceiving(self):
"""Same as above except the command is receiving.""" """Same as above except the command is receiving."""
results = list(sendframes(makereactor(), [ results = list(sendframes(makereactor(), [
ffs(b'1 1 stream-begin command-name have-args command'), ffs(b'1 1 stream-begin command-request new|more '
b"cbor:{b'name': b'ignored'}"),
ffs(b'1 1 0 command-data eos ignored'), ffs(b'1 1 0 command-data eos ignored'),
])) ]))
self.assertaction(results[0], 'wantframe')
self.assertaction(results[1], 'error') self.assertaction(results[1], 'error')
self.assertEqual(results[1][1], { self.assertEqual(results[1][1], {
'message': b'received command data frame for request that is not ' 'message': b'received command data frame for request that is not '
b'expecting data: 1', b'expecting data: 1',
}) })
def testmissingcommandframeflags(self):
"""Command name frame must have flags set."""
result = self._sendsingleframe(
makereactor(), ffs(b'1 1 stream-begin command-name 0 command'))
self.assertaction(result, 'error')
self.assertEqual(result[1], {
'message': b'missing frame flags on command frame',
})
def testconflictingrequestidallowed(self): def testconflictingrequestidallowed(self):
"""Multiple fully serviced commands with same request ID is allowed.""" """Multiple fully serviced commands with same request ID is allowed."""
reactor = makereactor() reactor = makereactor()
results = [] results = []
outstream = reactor.makeoutputstream() outstream = reactor.makeoutputstream()
results.append(self._sendsingleframe( results.append(self._sendsingleframe(
reactor, ffs(b'1 1 stream-begin command-name eos command'))) reactor, ffs(b'1 1 stream-begin command-request new '
b"cbor:{b'name': b'command'}")))
result = reactor.onbytesresponseready(outstream, 1, b'response1') result = reactor.onbytesresponseready(outstream, 1, b'response1')
self.assertaction(result, 'sendframes') self.assertaction(result, 'sendframes')
list(result[1]['framegen']) list(result[1]['framegen'])
results.append(self._sendsingleframe( results.append(self._sendsingleframe(
reactor, ffs(b'1 1 0 command-name eos command'))) reactor, ffs(b'1 1 stream-begin command-request new '
b"cbor:{b'name': b'command'}")))
result = reactor.onbytesresponseready(outstream, 1, b'response2') result = reactor.onbytesresponseready(outstream, 1, b'response2')
self.assertaction(result, 'sendframes') self.assertaction(result, 'sendframes')
list(result[1]['framegen']) list(result[1]['framegen'])
results.append(self._sendsingleframe( results.append(self._sendsingleframe(
reactor, ffs(b'1 1 0 command-name eos command'))) reactor, ffs(b'1 1 stream-begin command-request new '
b"cbor:{b'name': b'command'}")))
result = reactor.onbytesresponseready(outstream, 1, b'response3') result = reactor.onbytesresponseready(outstream, 1, b'response3')
self.assertaction(result, 'sendframes') self.assertaction(result, 'sendframes')
list(result[1]['framegen']) list(result[1]['framegen'])
for i in range(3): for i in range(3):
self.assertaction(results[i], 'runcommand') self.assertaction(results[i], 'runcommand')
self.assertEqual(results[i][1], { self.assertEqual(results[i][1], {
'requestid': 1, 'requestid': 1,
'command': b'command', 'command': b'command',
'args': {}, 'args': {},
'data': None, 'data': None,
}) })
def testconflictingrequestid(self): def testconflictingrequestid(self):
"""Request ID for new command matching in-flight command is illegal.""" """Request ID for new command matching in-flight command is illegal."""
results = list(sendframes(makereactor(), [ results = list(sendframes(makereactor(), [
ffs(b'1 1 stream-begin command-name have-args command'), ffs(b'1 1 stream-begin command-request new|more '
ffs(b'1 1 0 command-name eos command'), b"cbor:{b'name': b'command'}"),
ffs(b'1 1 0 command-request new '
b"cbor:{b'name': b'command1'}"),
])) ]))
self.assertaction(results[0], 'wantframe') self.assertaction(results[0], 'wantframe')
self.assertaction(results[1], 'error') self.assertaction(results[1], 'error')
self.assertEqual(results[1][1], { self.assertEqual(results[1][1], {
'message': b'request with ID 1 already received', 'message': b'request with ID 1 already received',
}) })
def testinterleavedcommands(self): def testinterleavedcommands(self):
cbor1 = cbor.dumps({
b'name': b'command1',
b'args': {
b'foo': b'bar',
b'key1': b'val',
}
}, canonical=True)
cbor3 = cbor.dumps({
b'name': b'command3',
b'args': {
b'biz': b'baz',
b'key': b'val',
},
}, canonical=True)
results = list(sendframes(makereactor(), [ results = list(sendframes(makereactor(), [
ffs(b'1 1 stream-begin command-name have-args command1'), ffs(b'1 1 stream-begin command-request new|more %s' % cbor1[0:6]),
ffs(b'3 1 0 command-name have-args command3'), ffs(b'3 1 0 command-request new|more %s' % cbor3[0:10]),
ffs(br'1 1 0 command-argument 0 \x03\x00\x03\x00foobar'), ffs(b'1 1 0 command-request continuation|more %s' % cbor1[6:9]),
ffs(br'3 1 0 command-argument 0 \x03\x00\x03\x00bizbaz'), ffs(b'3 1 0 command-request continuation|more %s' % cbor3[10:13]),
ffs(br'3 1 0 command-argument eoa \x03\x00\x03\x00keyval'), ffs(b'3 1 0 command-request continuation %s' % cbor3[13:]),
ffs(br'1 1 0 command-argument eoa \x04\x00\x03\x00key1val'), ffs(b'1 1 0 command-request continuation %s' % cbor1[9:]),
])) ]))
self.assertEqual([t[0] for t in results], [ self.assertEqual([t[0] for t in results], [
'wantframe', 'wantframe',
'wantframe', 'wantframe',
'wantframe', 'wantframe',
'wantframe', 'wantframe',
'runcommand', 'runcommand',
'runcommand', 'runcommand',
]) ])
self.assertEqual(results[4][1], { self.assertEqual(results[4][1], {
'requestid': 3, 'requestid': 3,
'command': 'command3', 'command': 'command3',
'args': {b'biz': b'baz', b'key': b'val'}, 'args': {b'biz': b'baz', b'key': b'val'},
'data': None, 'data': None,
}) })
self.assertEqual(results[5][1], { self.assertEqual(results[5][1], {
'requestid': 1, 'requestid': 1,
'command': 'command1', 'command': 'command1',
'args': {b'foo': b'bar', b'key1': b'val'}, 'args': {b'foo': b'bar', b'key1': b'val'},
'data': None, 'data': None,
}) })
def testmissingargumentframe(self):
# This test attempts to test behavior when reactor has an incomplete
# command request waiting on argument data. But it doesn't handle that
# scenario yet. So this test does nothing of value.
frames = [
ffs(b'1 1 stream-begin command-name have-args command'),
]
results = list(sendframes(makereactor(), frames))
self.assertaction(results[0], 'wantframe')
def testincompleteargumentname(self):
"""Argument frame with incomplete name."""
frames = [
ffs(b'1 1 stream-begin command-name have-args command1'),
ffs(br'1 1 0 command-argument eoa \x04\x00\xde\xadfoo'),
]
results = list(sendframes(makereactor(), frames))
self.assertEqual(len(results), 2)
self.assertaction(results[0], 'wantframe')
self.assertaction(results[1], 'error')
self.assertEqual(results[1][1], {
'message': b'malformed argument frame: partial argument name',
})
def testincompleteargumentvalue(self):
"""Argument frame with incomplete value."""
frames = [
ffs(b'1 1 stream-begin command-name have-args command'),
ffs(br'1 1 0 command-argument eoa \x03\x00\xaa\xaafoopartialvalue'),
]
results = list(sendframes(makereactor(), frames))
self.assertEqual(len(results), 2)
self.assertaction(results[0], 'wantframe')
self.assertaction(results[1], 'error')
self.assertEqual(results[1][1], {
'message': b'malformed argument frame: partial argument value',
})
def testmissingcommanddataframe(self): def testmissingcommanddataframe(self):
# The reactor doesn't currently handle partially received commands. # The reactor doesn't currently handle partially received commands.
# So this test is failing to do anything with request 1. # So this test is failing to do anything with request 1.
frames = [ frames = [
ffs(b'1 1 stream-begin command-name have-data command1'), ffs(b'1 1 stream-begin command-request new|have-data '
ffs(b'3 1 0 command-name eos command2'), b"cbor:{b'name': b'command1'}"),
ffs(b'3 1 0 command-request new '
b"cbor:{b'name': b'command2'}"),
] ]
results = list(sendframes(makereactor(), frames)) results = list(sendframes(makereactor(), frames))
self.assertEqual(len(results), 2) self.assertEqual(len(results), 2)
self.assertaction(results[0], 'wantframe') self.assertaction(results[0], 'wantframe')
self.assertaction(results[1], 'runcommand') self.assertaction(results[1], 'runcommand')
def testmissingcommanddataframeflags(self): def testmissingcommanddataframeflags(self):
frames = [ frames = [
ffs(b'1 1 stream-begin command-name have-data command1'), ffs(b'1 1 stream-begin command-request new|have-data '
b"cbor:{b'name': b'command1'}"),
ffs(b'1 1 0 command-data 0 data'), ffs(b'1 1 0 command-data 0 data'),
] ]
results = list(sendframes(makereactor(), frames)) results = list(sendframes(makereactor(), frames))
self.assertEqual(len(results), 2) self.assertEqual(len(results), 2)
self.assertaction(results[0], 'wantframe') self.assertaction(results[0], 'wantframe')
self.assertaction(results[1], 'error') self.assertaction(results[1], 'error')
self.assertEqual(results[1][1], { self.assertEqual(results[1][1], {
'message': b'command data frame without flags', 'message': b'command data frame without flags',
}) })
def testframefornonreceivingrequest(self): def testframefornonreceivingrequest(self):
"""Receiving a frame for a command that is not receiving is illegal.""" """Receiving a frame for a command that is not receiving is illegal."""
results = list(sendframes(makereactor(), [ results = list(sendframes(makereactor(), [
ffs(b'1 1 stream-begin command-name eos command1'), ffs(b'1 1 stream-begin command-request new '
ffs(b'3 1 0 command-name have-data command3'), b"cbor:{b'name': b'command1'}"),
ffs(b'5 1 0 command-argument eoa ignored'), ffs(b'3 1 0 command-request new|have-data '
b"cbor:{b'name': b'command3'}"),
ffs(b'5 1 0 command-data eos ignored'),
])) ]))
self.assertaction(results[2], 'error') self.assertaction(results[2], 'error')
self.assertEqual(results[2][1], { self.assertEqual(results[2][1], {
'message': b'received frame for request that is not receiving: 5', 'message': b'received frame for request that is not receiving: 5',
}) })
def testsimpleresponse(self): def testsimpleresponse(self):
"""Bytes response to command sends result frames.""" """Bytes response to command sends result frames."""
# perspective of the reactor, the command is still active. # perspective of the reactor, the command is still active.
results = list(sendcommandframes(reactor, instream, 1, b'command1', {})) results = list(sendcommandframes(reactor, instream, 1, b'command1', {}))
self.assertaction(results[0], 'error') self.assertaction(results[0], 'error')
self.assertEqual(results[0][1], { self.assertEqual(results[0][1], {
'message': b'request with ID 1 is already active', 'message': b'request with ID 1 is already active',
}) })
def testduplicaterequestargumentframe(self):
"""Variant on above except we sent an argument frame instead of name."""
reactor = makereactor()
stream = framing.stream(1)
list(sendcommandframes(reactor, stream, 1, b'command', {}))
results = list(sendframes(reactor, [
ffs(b'3 1 stream-begin command-name have-args command'),
ffs(b'1 1 0 command-argument 0 ignored'),
]))
self.assertaction(results[0], 'wantframe')
self.assertaction(results[1], 'error')
self.assertEqual(results[1][1], {
'message': 'received frame for request that is still active: 1',
})
def testduplicaterequestaftersend(self): def testduplicaterequestaftersend(self):
"""We can use a duplicate request ID after we've sent the response.""" """We can use a duplicate request ID after we've sent the response."""
reactor = makereactor() reactor = makereactor()
instream = framing.stream(1) instream = framing.stream(1)
list(sendcommandframes(reactor, instream, 1, b'command1', {})) list(sendcommandframes(reactor, instream, 1, b'command1', {}))
outstream = reactor.makeoutputstream() outstream = reactor.makeoutputstream()
res = reactor.onbytesresponseready(outstream, 1, b'response') res = reactor.onbytesresponseready(outstream, 1, b'response')
list(res[1]['framegen']) list(res[1]['framegen'])
results = list(sendcommandframes(reactor, instream, 1, b'command1', {})) results = list(sendcommandframes(reactor, instream, 1, b'command1', {}))
self.assertaction(results[0], 'runcommand') self.assertaction(results[0], 'runcommand')
if __name__ == '__main__': if __name__ == '__main__':
import silenttestrunner import silenttestrunner
silenttestrunner.main(__name__) silenttestrunner.main(__name__)