This is an archive of the discontinued Mercurial Phabricator instance.

Differential D2841

debugcommands: support sending HTTP requests with debugwireproto
ClosedPublic

Authored by indygreg on Mar 13 2018, 3:11 PM.

Details

Reviewers
durin42
Group Reviewers
hg-reviewers
Commits
rHGb6a7070e7663: debugcommands: support sending HTTP requests with debugwireproto
Summary

We implement an action that can issue an HTTP request. We can define
headers via arguments and specify a file to use for the HTTP request
body.

The request uses the HTTP peer's opener, which is already configured
for auth, etc. This is both good and bad. Good in that we get some
nice behavior out of the box. Bad in that some HTTP request headers
are added automatically.

Diff Detail

Repository
rHG Mercurial
Lint
Automatic diff as part of commit; lint not applicable.
Unit
Automatic diff as part of commit; unit tests not applicable.

Event Timeline

indygreg created this revision.Mar 13 2018, 3:11 PM
durin42 accepted this revision.Mar 19 2018, 9:36 PM
This revision is now accepted and ready to land.Mar 19 2018, 9:36 PM
Closed by commit rHGb6a7070e7663: debugcommands: support sending HTTP requests with debugwireproto (authored by indygreg). · Explain WhyMar 19 2018, 9:40 PM
This revision was automatically updated to reflect the committed changes.

Revision Contents
Changeset List

PathPackages
Mmercurial/debugcommands.py (56 lines)
Mtests/test-http-protocol.t (35 lines)

Diff 7159

mercurial/debugcommands.py

# debugcommands.py - command processing for debug* commands # debugcommands.py - command processing for debug* commands
# #
# Copyright 2005-2016 Matt Mackall <mpm@selenic.com> # Copyright 2005-2016 Matt Mackall <mpm@selenic.com>
# #
# This software may be used and distributed according to the terms of the # This software may be used and distributed according to the terms of the
# GNU General Public License version 2 or any later version. # GNU General Public License version 2 or any later version.
from __future__ import absolute_import from __future__ import absolute_import
import codecs import codecs
import collections import collections
import difflib import difflib
import errno import errno
import operator import operator
import os import os
import random import random
import re
import socket import socket
import ssl import ssl
import stat import stat
import string import string
import subprocess import subprocess
import sys import sys
import tempfile import tempfile
import time import time
batchsubmit batchsubmit
----------- -----------
Submit previously queued ``command`` blocks as a batch request. Submit previously queued ``command`` blocks as a batch request.
This action MUST be paired with a ``batchbegin`` action. This action MUST be paired with a ``batchbegin`` action.
httprequest <method> <path>
---------------------------
(HTTP peer only)
Send an HTTP request to the peer.
The HTTP request line follows the ``httprequest`` action. e.g. ``GET /foo``.
Arguments of the form ``<key>: <value>`` are interpreted as HTTP request
headers to add to the request. e.g. ``Accept: foo``.
The following arguments are special:
``BODYFILE``
The content of the file defined as the value to this argument will be
transferred verbatim as the HTTP request body.
close close
----- -----
Close the connection to the server. Close the connection to the server.
flush flush
----- -----
ui.write(_('(waiting for commands on stdin)\n')) ui.write(_('(waiting for commands on stdin)\n'))
blocks = list(_parsewirelangblocks(ui.fin)) blocks = list(_parsewirelangblocks(ui.fin))
proc = None proc = None
stdin = None stdin = None
stdout = None stdout = None
stderr = None stderr = None
opener = None
if opts['localssh']: if opts['localssh']:
# We start the SSH server in its own process so there is process # We start the SSH server in its own process so there is process
# separation. This prevents a whole class of potential bugs around # separation. This prevents a whole class of potential bugs around
# shared state from interfering with server operation. # shared state from interfering with server operation.
args = util.hgcmd() + [ args = util.hgcmd() + [
'-R', repo.root, '-R', repo.root,
'debugserve', '--sshstdio', 'debugserve', '--sshstdio',
# difficult to normalize requests into function calls. It is easier # difficult to normalize requests into function calls. It is easier
# to bypass this layer and normalize to commands + args. # to bypass this layer and normalize to commands + args.
ui.status(_('sending batch with %d sub-commands\n') % ui.status(_('sending batch with %d sub-commands\n') %
len(batchedcommands)) len(batchedcommands))
for i, chunk in enumerate(peer._submitbatch(batchedcommands)): for i, chunk in enumerate(peer._submitbatch(batchedcommands)):
ui.status(_('response #%d: %s\n') % (i, util.escapedata(chunk))) ui.status(_('response #%d: %s\n') % (i, util.escapedata(chunk)))
batchedcommands = None batchedcommands = None
elif action.startswith('httprequest '):
if not opener:
raise error.Abort(_('cannot use httprequest without an HTTP '
'peer'))
request = action.split(' ', 2)
if len(request) != 3:
raise error.Abort(_('invalid httprequest: expected format is '
'"httprequest <method> <path>'))
method, httppath = request[1:]
headers = {}
body = None
for line in lines:
line = line.lstrip()
m = re.match(b'^([a-zA-Z0-9_-]+): (.*)$', line)
if m:
headers[m.group(1)] = m.group(2)
continue
if line.startswith(b'BODYFILE '):
with open(line.split(b' ', 1), 'rb') as fh:
body = fh.read()
else:
raise error.Abort(_('unknown argument to httprequest: %s') %
line)
url = path + httppath
req = urlmod.urlreq.request(pycompat.strurl(url), body, headers)
try:
opener.open(req).read()
except util.urlerr.urlerror as e:
e.read()
elif action == 'close': elif action == 'close':
peer.close() peer.close()
elif action == 'readavailable': elif action == 'readavailable':
if not stdout or not stderr: if not stdout or not stderr:
raise error.Abort(_('readavailable not available on this peer')) raise error.Abort(_('readavailable not available on this peer'))
stdin.close() stdin.close()
stdout.read() stdout.read()

tests/test-http-protocol.t

s> readline() -> 2: s> readline() -> 2:
s> \r\n s> \r\n
s> read(30) -> 30: s> read(30) -> 30:
s> bookmarks \n s> bookmarks \n
s> namespaces \n s> namespaces \n
s> phases s> phases
response: bookmarks \nnamespaces \nphases response: bookmarks \nnamespaces \nphases
Same thing, but with "httprequest" command
$ hg --verbose debugwireproto --peer raw http://$LOCALIP:$HGPORT << EOF
> httprequest GET ?cmd=listkeys
> accept: application/mercurial-0.1
> user-agent: mercurial/proto-1.0 (Mercurial 42)
> x-hgarg-1: namespace=namespaces
> EOF
using raw connection to peer
s> sendall(*, 0): (glob)
s> GET /?cmd=listkeys HTTP/1.1\r\n
s> Accept-Encoding: identity\r\n
s> accept: application/mercurial-0.1\r\n
s> user-agent: mercurial/proto-1.0 (Mercurial 42)\r\n (glob)
s> x-hgarg-1: namespace=namespaces\r\n
s> host: $LOCALIP:$HGPORT\r\n (glob)
s> \r\n
s> makefile('rb', None)
s> readline() -> 36:
s> HTTP/1.1 200 Script output follows\r\n
s> readline() -> 28:
s> Server: testing stub value\r\n
s> readline() -> *: (glob)
s> Date: $HTTP_DATE$\r\n
s> readline() -> 41:
s> Content-Type: application/mercurial-0.1\r\n
s> readline() -> 20:
s> Content-Length: 30\r\n
s> readline() -> 2:
s> \r\n
s> read(30) -> 30:
s> bookmarks \n
s> namespaces \n
s> phases
$ killdaemons.py $ killdaemons.py