This is an archive of the discontinued Mercurial Phabricator instance.

Differential D2722

url: add HTTP handler that uses a proxied socket
ClosedPublic

Authored by indygreg on Mar 8 2018, 12:31 AM.

Details

Reviewers
durin42
Group Reviewers
hg-reviewers
Commits
rHG02221d6fb041: url: add HTTP handler that uses a proxied socket
Summary

Now that we have a socket proxy that can log I/O, we need to teach
mechanisms that open URLs how to use it.

We invent a custom HTTP handler class that knows how to proxy
sockets as soon as they are opened. We teach the high-level
opener() to accept logging arguments so a logging HTTP handler
can be constructed.

We don't yet support intercepting HTTPS sockets because I
don't want to go down that rabbit hole.

For the record, the urllib API is crazy and it took way too long
to figure out at what levels I needed to plug in to modify
the socket.

  1. no-check-commit because we must name http_open that way

Diff Detail

Repository
rHG Mercurial
Lint
Automatic diff as part of commit; lint not applicable.
Unit
Automatic diff as part of commit; unit tests not applicable.

Event Timeline

indygreg created this revision.Mar 8 2018, 12:31 AM
• Jeffreydoomb added a subscriber: • Jeffreydoomb.Mar 8 2018, 5:41 AM
This comment was removed by admin.
• Michaelexics added a subscriber: • Michaelexics.Mar 8 2018, 8:38 AM
This comment was removed by admin.
admin added a subscriber: admin.Mar 8 2018, 3:45 PM
indygreg edited the summary of this revision. (Show Details)Mar 12 2018, 8:07 PM
durin42 accepted this revision.Mar 19 2018, 9:36 PM
This revision is now accepted and ready to land.Mar 19 2018, 9:36 PM
Closed by commit rHG02221d6fb041: url: add HTTP handler that uses a proxied socket (authored by indygreg). · Explain WhyMar 19 2018, 9:40 PM
This revision was automatically updated to reflect the committed changes.

Revision Contents
Changeset List

PathPackages
Mmercurial/url.py (54 lines)

Diff 7157

mercurial/url.py

class httphandler(keepalive.HTTPHandler): class httphandler(keepalive.HTTPHandler):
def http_open(self, req): def http_open(self, req):
return self.do_open(httpconnection, req) return self.do_open(httpconnection, req)
def _start_transaction(self, h, req): def _start_transaction(self, h, req):
_generic_start_transaction(self, h, req) _generic_start_transaction(self, h, req)
return keepalive.HTTPHandler._start_transaction(self, h, req) return keepalive.HTTPHandler._start_transaction(self, h, req)
class logginghttpconnection(keepalive.HTTPConnection):
def __init__(self, createconn, *args, **kwargs):
keepalive.HTTPConnection.__init__(self, *args, **kwargs)
self._create_connection = createconn
class logginghttphandler(httphandler):
"""HTTP handler that logs socket I/O."""
def __init__(self, logfh, name, observeropts):
super(logginghttphandler, self).__init__()
self._logfh = logfh
self._logname = name
self._observeropts = observeropts
# do_open() calls the passed class to instantiate an HTTPConnection. We
# pass in a callable method that creates a custom HTTPConnection instance
# whose callback to create the socket knows how to proxy the socket.
def http_open(self, req):
return self.do_open(self._makeconnection, req)
def _makeconnection(self, *args, **kwargs):
def createconnection(*args, **kwargs):
sock = socket.create_connection(*args, **kwargs)
return util.makeloggingsocket(self._logfh, sock, self._logname,
**self._observeropts)
return logginghttpconnection(createconnection, *args, **kwargs)
if has_https: if has_https:
class httpsconnection(httplib.HTTPConnection): class httpsconnection(httplib.HTTPConnection):
response_class = keepalive.HTTPResponse response_class = keepalive.HTTPResponse
default_port = httplib.HTTPS_PORT default_port = httplib.HTTPS_PORT
# must be able to send big bundle as stream. # must be able to send big bundle as stream.
send = _gen_sendfile(keepalive.safesend) send = _gen_sendfile(keepalive.safesend)
getresponse = keepalive.wrapgetresponse(httplib.HTTPConnection) getresponse = keepalive.wrapgetresponse(httplib.HTTPConnection)
def https_request(self, request): def https_request(self, request):
if self.cookiejar: if self.cookiejar:
self.cookiejar.add_cookie_header(request) self.cookiejar.add_cookie_header(request)
return request return request
handlerfuncs = [] handlerfuncs = []
def opener(ui, authinfo=None, useragent=None): def opener(ui, authinfo=None, useragent=None, loggingfh=None,
loggingname=b's', loggingopts=None):
''' '''
construct an opener suitable for urllib2 construct an opener suitable for urllib2
authinfo will be added to the password manager authinfo will be added to the password manager
The opener can be configured to log socket events if the various
``logging*`` arguments are specified.
``loggingfh`` denotes a file object to log events to.
``loggingname`` denotes the name of the to print when logging.
``loggingopts`` is a dict of keyword arguments to pass to the constructed
``util.socketobserver`` instance.
''' '''
handlers = [httphandler()] handlers = []
if loggingfh:
handlers.append(logginghttphandler(loggingfh, loggingname,
loggingopts or {}))
# We don't yet support HTTPS when logging I/O. If we attempt to open
# an HTTPS URL, we'll likely fail due to unknown protocol.
else:
handlers.append(httphandler())
if has_https: if has_https:
handlers.append(httpshandler(ui)) handlers.append(httpshandler(ui))
handlers.append(proxyhandler(ui)) handlers.append(proxyhandler(ui))
passmgr = passwordmgr(ui, ui.httppasswordmgrdb) passmgr = passwordmgr(ui, ui.httppasswordmgrdb)
if authinfo is not None: if authinfo is not None:
realm, uris, user, passwd = authinfo realm, uris, user, passwd = authinfo
saveduser, savedpass = passmgr.find_stored_password(uris[0]) saveduser, savedpass = passmgr.find_stored_password(uris[0])
if user != saveduser or passwd: if user != saveduser or passwd: