( )⚙ D2857 wireproto: implement basic command dispatching for HTTPv2

This is an archive of the discontinued Mercurial Phabricator instance.

Differential D2857

wireproto: implement basic command dispatching for HTTPv2
ClosedPublic

Authored by indygreg on Mar 14 2018, 12:01 PM.

Details

Reviewers
durin42
Group Reviewers
hg-reviewers
Commits
rHGe7a012b60d6e: wireproto: implement basic command dispatching for HTTPv2
Summary

Now that we can ingest frames and decode them to requests to run
commands, we are able to actually run those commands. So this
commit starts to implement that.

There are numerous shortcomings. We can't operate on commands
with "*" arguments. We can only emit bytesresponse results. We
don't yet issue a response in the unified framing protocol.
But it's a start.

Diff Detail

Repository
rHG Mercurial
Lint
Lint Skipped
Unit
Unit Tests Skipped

Event Timeline

indygreg created this revision.Mar 14 2018, 12:01 PM
indygreg updated this revision to Diff 7033.Mar 14 2018, 12:43 PM
indygreg updated this revision to Diff 7050.Mar 14 2018, 9:10 PM
indygreg updated this revision to Diff 7142.Mar 19 2018, 7:59 PM
durin42 accepted this revision.Mar 21 2018, 9:32 PM
This revision is now accepted and ready to land.Mar 21 2018, 9:32 PM
Closed by commit rHGe7a012b60d6e: wireproto: implement basic command dispatching for HTTPv2 (authored by indygreg). · Explain WhyMar 21 2018, 9:40 PM
This revision was automatically updated to reflect the committed changes.

Revision Contents
Changeset List

Diff 7031

mercurial/wireprotoserver.py

res.status = b'415 Unsupported Media Type' res.status = b'415 Unsupported Media Type'
# TODO we should send a response with appropriate media type, # TODO we should send a response with appropriate media type,
# since client does Accept it. # since client does Accept it.
res.headers[b'Content-Type'] = b'text/plain' res.headers[b'Content-Type'] = b'text/plain'
res.setbodybytes(_('client MUST send Content-Type header with ' res.setbodybytes(_('client MUST send Content-Type header with '
'value: %s\n') % FRAMINGTYPE) 'value: %s\n') % FRAMINGTYPE)
return return
# We don't do anything meaningful yet. _processhttpv2request(ui, repo, req, res, permission, command, proto)
res.status = b'200 OK'
res.headers[b'Content-Type'] = b'text/plain'
res.setbodybytes(b'/'.join(urlparts) + b'\n')
def _processhttpv2reflectrequest(ui, repo, req, res): def _processhttpv2reflectrequest(ui, repo, req, res):
"""Reads unified frame protocol request and dumps out state to client. """Reads unified frame protocol request and dumps out state to client.
This special endpoint can be used to help debug the wire protocol. This special endpoint can be used to help debug the wire protocol.
Instead of routing the request through the normal dispatch mechanism, Instead of routing the request through the normal dispatch mechanism,
we instead read all frames, decode them, and feed them into our state we instead read all frames, decode them, and feed them into our state
r = reactor.onframerecv(frametype, frameflags, payload) r = reactor.onframerecv(frametype, frameflags, payload)
states.append(json.dumps(r, sort_keys=True, separators=(', ', ': '))) states.append(json.dumps(r, sort_keys=True, separators=(', ', ': ')))
res.status = b'200 OK' res.status = b'200 OK'
res.headers[b'Content-Type'] = b'text/plain' res.headers[b'Content-Type'] = b'text/plain'
res.setbodybytes(b'\n'.join(states)) res.setbodybytes(b'\n'.join(states))
def _processhttpv2request(ui, repo, req, res, authedperm, reqcommand, proto):
"""Post-validation handler for HTTPv2 requests.
Called when the HTTP request contains unified frame-based protocol
frames for evaluation.
"""
reactor = wireprotoframing.serverreactor(ui, repo)
seencommand = False
while True:
frame = wireprotoframing.readframe(req.bodyfh)
if not frame:
break
state = reactor.onframerecv(*frame)
if not state:
raise error.Programming('did not receive valid reply from '
'server reactor')
action = state['action']
if action == 'wantframe':
# Need more data before we can do anything.
continue
elif action == 'runcommand':
# We currently only support running a single command per
# HTTP request.
if seencommand:
# TODO define proper error mechanism.
res.status = b'200 OK'
res.headers[b'Content-Type'] = b'text/plain'
res.setbodybytes(_('support for multiple commands per request '
'not yet implemented'))
return
_httpv2runcommand(ui, repo, req, res, authedperm, reqcommand,
reactor, state)
elif action == 'error':
# TODO define proper error mechanism.
res.status = b'200 OK'
res.headers[b'Content-Type'] = b'text/plain'
res.setbodybytes(state['message'] + b'\n')
return
else:
raise error.Programming('unhandled action from frame processor: %s'
% action)
def _httpv2runcommand(ui, repo, req, res, authedperm, reqcommand, reactor,
command):
"""Dispatch a wire protocol command made from HTTPv2 requests.
The authenticated permission (``authedperm``) along with the original
command from the URL (``reqcommand``) are passed in.
"""
# We already validated that the session has permissions to perform the
# actions in ``authedperm``. In the unified frame protocol, the canonical
# command to run is expressed in a frame. However, the URL also requested
# to run a specific command. We need to be careful that the command we
# run doesn't have permissions requirements greater than what was granted
# by ``authedperm``.
#
# For now, this is no big deal, as we only allow a single command per
# request and that command must match the command in the URL. But when
# things change, we need to watch out...
if reqcommand != command['command']:
# TODO define proper error mechanism
res.status = b'200 OK'
res.headers[b'Content-Type'] = b'text/plain'
res.setbodybytes(_('command in frame must match command in URL'))
return
# TODO once we get rid of the command==URL restriction, we'll need to
# revalidate command validity and auth here. checkperm,
# wireproto.commands.commandavailable(), etc.
proto = httpv2protocolhandler(req, ui, args=command['args'])
assert wireproto.commands.commandavailable(command['command'], proto)
wirecommand = wireproto.commands[command['command']]
assert authedperm in (b'ro', b'rw')
assert wirecommand.permission in ('push', 'pull')
# We already checked this as part of the URL==command check, but
# permissions are important, so do it again.
if authedperm == b'ro':
assert wirecommand.permission == 'pull'
elif authedperm == b'rw':
# We are allowed to access read-only commands under the rw URL.
assert wirecommand.permission in ('push', 'pull')
rsp = wireproto.dispatch(repo, proto, command['command'])
# TODO use proper response format.
res.status = b'200 OK'
res.headers[b'Content-Type'] = b'text/plain'
if isinstance(rsp, wireprototypes.bytesresponse):
res.setbodybytes(rsp.data)
else:
res.setbodybytes(b'unhandled response type from wire proto '
'command')
# Maps API name to metadata so custom API can be registered. # Maps API name to metadata so custom API can be registered.
API_HANDLERS = { API_HANDLERS = {
HTTPV2: { HTTPV2: {
'config': ('experimental', 'web.api.http-v2'), 'config': ('experimental', 'web.api.http-v2'),
'handler': _handlehttpv2request, 'handler': _handlehttpv2request,
}, },
} }
class httpv2protocolhandler(wireprototypes.baseprotocolhandler): class httpv2protocolhandler(wireprototypes.baseprotocolhandler):
def __init__(self, req, ui): def __init__(self, req, ui, args=None):
self._req = req self._req = req
self._ui = ui self._ui = ui
self._args = args
@property @property
def name(self): def name(self):
return HTTPV2 return HTTPV2
def getargs(self, args): def getargs(self, args):
raise NotImplementedError data = {}
for k in args.split():
if k == '*':
raise NotImplementedError('do not support * args')
else:
data[k] = self._args[k]
return [data[k] for k in args.split()]
def forwardpayload(self, fp): def forwardpayload(self, fp):
raise NotImplementedError raise NotImplementedError
@contextlib.contextmanager @contextlib.contextmanager
def mayberedirectstdio(self): def mayberedirectstdio(self):
raise NotImplementedError raise NotImplementedError
def client(self): def client(self):
raise NotImplementedError raise NotImplementedError
def addcapabilities(self, repo, caps): def addcapabilities(self, repo, caps):
raise NotImplementedError return caps
def checkperm(self, perm): def checkperm(self, perm):
raise NotImplementedError raise NotImplementedError
def _httpresponsetype(ui, req, prefer_uncompressed): def _httpresponsetype(ui, req, prefer_uncompressed):
"""Determine the appropriate response type and compression settings. """Determine the appropriate response type and compression settings.
Returns a tuple of (mediatype, compengine, engineopts). Returns a tuple of (mediatype, compengine, engineopts).

tests/test-http-api-httpv2.t

s> host: $LOCALIP:$HGPORT\r\n (glob) s> host: $LOCALIP:$HGPORT\r\n (glob)
s> \r\n s> \r\n
s> \x0c\x00\x00\x11capabilities s> \x0c\x00\x00\x11capabilities
s> makefile('rb', None) s> makefile('rb', None)
s> HTTP/1.1 200 OK\r\n s> HTTP/1.1 200 OK\r\n
s> Server: testing stub value\r\n s> Server: testing stub value\r\n
s> Date: $HTTP_DATE$\r\n s> Date: $HTTP_DATE$\r\n
s> Content-Type: text/plain\r\n s> Content-Type: text/plain\r\n
s> Content-Length: 16\r\n s> Content-Length: *\r\n (glob)
s> \r\n s> \r\n
s> ro/capabilities\n s> lookup branchmap pushkey known getbundle unbundlehash streamreqs=generaldelta,revlogv1 $USUAL_BUNDLE2_CAPS_SERVER$ unbundle=HG10GZ,HG10BZ,HG10UN
Request to read-write command fails because server is read-only by default Request to read-write command fails because server is read-only by default
GET to read-write request yields 405 GET to read-write request yields 405
$ send << EOF $ send << EOF
> httprequest GET api/$HTTPV2/rw/capabilities > httprequest GET api/$HTTPV2/rw/capabilities
> user-agent: test > user-agent: test
s> host: $LOCALIP:$HGPORT\r\n (glob) s> host: $LOCALIP:$HGPORT\r\n (glob)
s> \r\n s> \r\n
s> \x0c\x00\x00\x11capabilities s> \x0c\x00\x00\x11capabilities
s> makefile('rb', None) s> makefile('rb', None)
s> HTTP/1.1 200 OK\r\n s> HTTP/1.1 200 OK\r\n
s> Server: testing stub value\r\n s> Server: testing stub value\r\n
s> Date: $HTTP_DATE$\r\n s> Date: $HTTP_DATE$\r\n
s> Content-Type: text/plain\r\n s> Content-Type: text/plain\r\n
s> Content-Length: 16\r\n s> Content-Length: *\r\n (glob)
s> \r\n s> \r\n
s> rw/capabilities\n s> lookup branchmap pushkey known getbundle unbundlehash streamreqs=generaldelta,revlogv1 $USUAL_BUNDLE2_CAPS_SERVER$ unbundle=HG10GZ,HG10BZ,HG10UN
Authorized request for unknown command is rejected Authorized request for unknown command is rejected
$ send << EOF $ send << EOF
> httprequest POST api/$HTTPV2/rw/badcommand > httprequest POST api/$HTTPV2/rw/badcommand
> user-agent: test > user-agent: test
> accept: $MEDIATYPE > accept: $MEDIATYPE
> EOF > EOF