This is an archive of the discontinued Mercurial Phabricator instance.

Differential D2204

wireprotoserver: handle SSH protocol version 2 upgrade requests
ClosedPublic

Authored by indygreg on Feb 12 2018, 7:46 PM.

Details

Reviewers
lothiraldan
durin42
Group Reviewers
hg-reviewers
Commits
rHG464bedc0fdb4: wireprotoserver: handle SSH protocol version 2 upgrade requests
rHG2c11f8d5554b: wireprotoserver: handle SSH protocol version 2 upgrade requests
Summary

This commit teaches the SSH server to recognize the "upgrade"
request line that clients send when they wish to switch the
channel to version 2 of the SSH protocol.

Servers don't honor upgrade requests unless an experimental config
option is set.

Since the built-in server now supports upgrade requests, our test
server to test the handshake has been deleted. Existing tests
use the built-in server and their output doesn't change.

The upgrade is handled in our state machine. The end result is a bit
wonky, as the server transitions back to version 1 state immediately
after upgrading. But this will change as soon as version 2 has an
actual protocol that differs from version 1.

Tests demonstrating that the new server is a bit more strict about
the upgrade handshake have been added.

Diff Detail

Repository
rHG Mercurial
Lint
Automatic diff as part of commit; lint not applicable.
Unit
Automatic diff as part of commit; unit tests not applicable.

Event Timeline

indygreg created this revision.Feb 12 2018, 7:46 PM
lothiraldan accepted this revision.Feb 13 2018, 4:03 AM
lothiraldan added a subscriber: lothiraldan.

LGTM after a first glance

durin42 accepted this revision.Feb 14 2018, 8:49 PM
This revision is now accepted and ready to land.Feb 14 2018, 8:49 PM
Closed by commit rHG2c11f8d5554b: wireprotoserver: handle SSH protocol version 2 upgrade requests (authored by indygreg). · Explain WhyFeb 14 2018, 9:04 PM
This revision was automatically updated to reflect the committed changes.

Revision Contents
Changeset List

Diff 5738

mercurial/configitems.py

default=False, default=False,
) )
coreconfigitem('experimental', 'revlogv2', coreconfigitem('experimental', 'revlogv2',
default=None, default=None,
) )
coreconfigitem('experimental', 'single-head-per-branch', coreconfigitem('experimental', 'single-head-per-branch',
default=False, default=False,
) )
coreconfigitem('experimental', 'sshserver.support-v2',
default=False,
)
coreconfigitem('experimental', 'spacemovesdown', coreconfigitem('experimental', 'spacemovesdown',
default=False, default=False,
) )
coreconfigitem('experimental', 'sparse-read', coreconfigitem('experimental', 'sparse-read',
default=False, default=False,
) )
coreconfigitem('experimental', 'sparse-read.density-threshold', coreconfigitem('experimental', 'sparse-read.density-threshold',
default=0.25, default=0.25,

mercurial/wireprotoserver.py

@contextlib.contextmanager @contextlib.contextmanager
def mayberedirectstdio(self): def mayberedirectstdio(self):
yield None yield None
def client(self): def client(self):
client = encoding.environ.get('SSH_CLIENT', '').split(' ', 1)[0] client = encoding.environ.get('SSH_CLIENT', '').split(' ', 1)[0]
return 'remote:ssh:' + client return 'remote:ssh:' + client
class sshv2protocolhandler(sshv1protocolhandler):
"""Protocol handler for version 2 of the SSH protocol."""
def _runsshserver(ui, repo, fin, fout): def _runsshserver(ui, repo, fin, fout):
# This function operates like a state machine of sorts. The following
# states are defined:
#
# protov1-serving
# Server is in protocol version 1 serving mode. Commands arrive on
# new lines. These commands are processed in this state, one command
# after the other.
#
# protov2-serving
# Server is in protocol version 2 serving mode.
#
# upgrade-initial
# The server is going to process an upgrade request.
#
# upgrade-v2-filter-legacy-handshake
# The protocol is being upgraded to version 2. The server is expecting
# the legacy handshake from version 1.
#
# upgrade-v2-finish
# The upgrade to version 2 of the protocol is imminent.
#
# shutdown
# The server is shutting down, possibly in reaction to a client event.
#
# And here are their transitions:
#
# protov1-serving -> shutdown
# When server receives an empty request or encounters another
# error.
#
# protov1-serving -> upgrade-initial
# An upgrade request line was seen.
#
# upgrade-initial -> upgrade-v2-filter-legacy-handshake
# Upgrade to version 2 in progress. Server is expecting to
# process a legacy handshake.
#
# upgrade-v2-filter-legacy-handshake -> shutdown
# Client did not fulfill upgrade handshake requirements.
#
# upgrade-v2-filter-legacy-handshake -> upgrade-v2-finish
# Client fulfilled version 2 upgrade requirements. Finishing that
# upgrade.
#
# upgrade-v2-finish -> protov2-serving
# Protocol upgrade to version 2 complete. Server can now speak protocol
# version 2.
#
# protov2-serving -> protov1-serving
# Ths happens by default since protocol version 2 is the same as
# version 1 except for the handshake.
state = 'protov1-serving' state = 'protov1-serving'
proto = sshv1protocolhandler(ui, fin, fout) proto = sshv1protocolhandler(ui, fin, fout)
protoswitched = False
while True: while True:
if state == 'protov1-serving': if state == 'protov1-serving':
# Commands are issued on new lines. # Commands are issued on new lines.
request = fin.readline()[:-1] request = fin.readline()[:-1]
# Empty lines signal to terminate the connection. # Empty lines signal to terminate the connection.
if not request: if not request:
state = 'shutdown' state = 'shutdown'
continue continue
# It looks like a protocol upgrade request. Transition state to
# handle it.
if request.startswith(b'upgrade '):
if protoswitched:
_sshv1respondooberror(fout, ui.ferr,
b'cannot upgrade protocols multiple '
b'times')
state = 'shutdown'
continue
state = 'upgrade-initial'
continue
available = wireproto.commands.commandavailable(request, proto) available = wireproto.commands.commandavailable(request, proto)
# This command isn't available. Send an empty response and go # This command isn't available. Send an empty response and go
# back to waiting for a new command. # back to waiting for a new command.
if not available: if not available:
_sshv1respondbytes(fout, b'') _sshv1respondbytes(fout, b'')
continue continue
elif isinstance(rsp, wireprototypes.pusherr): elif isinstance(rsp, wireprototypes.pusherr):
_sshv1respondbytes(fout, rsp.res) _sshv1respondbytes(fout, rsp.res)
elif isinstance(rsp, wireprototypes.ooberror): elif isinstance(rsp, wireprototypes.ooberror):
_sshv1respondooberror(fout, ui.ferr, rsp.message) _sshv1respondooberror(fout, ui.ferr, rsp.message)
else: else:
raise error.ProgrammingError('unhandled response type from ' raise error.ProgrammingError('unhandled response type from '
'wire protocol command: %s' % rsp) 'wire protocol command: %s' % rsp)
# For now, protocol version 2 serving just goes back to version 1.
elif state == 'protov2-serving':
state = 'protov1-serving'
continue
elif state == 'upgrade-initial':
# We should never transition into this state if we've switched
# protocols.
assert not protoswitched
assert proto.name == SSHV1
# Expected: upgrade <token> <capabilities>
# If we get something else, the request is malformed. It could be
# from a future client that has altered the upgrade line content.
# We treat this as an unknown command.
try:
token, caps = request.split(b' ')[1:]
except ValueError:
_sshv1respondbytes(fout, b'')
state = 'protov1-serving'
continue
# Send empty response if we don't support upgrading protocols.
if not ui.configbool('experimental', 'sshserver.support-v2'):
_sshv1respondbytes(fout, b'')
state = 'protov1-serving'
continue
try:
caps = urlreq.parseqs(caps)
except ValueError:
_sshv1respondbytes(fout, b'')
state = 'protov1-serving'
continue
# We don't see an upgrade request to protocol version 2. Ignore
# the upgrade request.
wantedprotos = caps.get(b'proto', [b''])[0]
if SSHV2 not in wantedprotos:
_sshv1respondbytes(fout, b'')
state = 'protov1-serving'
continue
# It looks like we can honor this upgrade request to protocol 2.
# Filter the rest of the handshake protocol request lines.
state = 'upgrade-v2-filter-legacy-handshake'
continue
elif state == 'upgrade-v2-filter-legacy-handshake':
# Client should have sent legacy handshake after an ``upgrade``
# request. Expected lines:
#
# hello
# between
# pairs 81
# 0000...-0000...
ok = True
for line in (b'hello', b'between', b'pairs 81'):
request = fin.readline()[:-1]
if request != line:
_sshv1respondooberror(fout, ui.ferr,
b'malformed handshake protocol: '
b'missing %s' % line)
ok = False
state = 'shutdown'
break
if not ok:
continue
request = fin.read(81)
if request != b'%s-%s' % (b'0' * 40, b'0' * 40):
_sshv1respondooberror(fout, ui.ferr,
b'malformed handshake protocol: '
b'missing between argument value')
state = 'shutdown'
continue
state = 'upgrade-v2-finish'
continue
elif state == 'upgrade-v2-finish':
# Send the upgrade response.
fout.write(b'upgraded %s %s\n' % (token, SSHV2))
servercaps = wireproto.capabilities(repo, proto)
rsp = b'capabilities: %s' % servercaps.data
fout.write(b'%d\n%s\n' % (len(rsp), rsp))
fout.flush()
proto = sshv2protocolhandler(ui, fin, fout)
protoswitched = True
state = 'protov2-serving'
continue
elif state == 'shutdown': elif state == 'shutdown':
break break
else: else:
raise error.ProgrammingError('unhandled ssh server state: %s' % raise error.ProgrammingError('unhandled ssh server state: %s' %
state) state)
class sshserver(object): class sshserver(object):

tests/sshprotoext.py

assert l == b'between\n' assert l == b'between\n'
proto = wireprotoserver.sshv1protocolhandler(self._ui, self._fin, proto = wireprotoserver.sshv1protocolhandler(self._ui, self._fin,
self._fout) self._fout)
rsp = wireproto.dispatch(self._repo, proto, b'between') rsp = wireproto.dispatch(self._repo, proto, b'between')
wireprotoserver._sshv1respondbytes(self._fout, rsp.data) wireprotoserver._sshv1respondbytes(self._fout, rsp.data)
super(prehelloserver, self).serve_forever() super(prehelloserver, self).serve_forever()
class upgradev2server(wireprotoserver.sshserver):
"""Tests behavior for clients that issue upgrade to version 2."""
def serve_forever(self):
name = wireprotoserver.SSHV2
l = self._fin.readline()
assert l.startswith(b'upgrade ')
token, caps = l[:-1].split(b' ')[1:]
assert caps == b'proto=%s' % name
# Filter hello and between requests.
l = self._fin.readline()
assert l == b'hello\n'
l = self._fin.readline()
assert l == b'between\n'
l = self._fin.readline()
assert l == b'pairs 81\n'
self._fin.read(81)
# Send the upgrade response.
proto = wireprotoserver.sshv1protocolhandler(self._ui, self._fin,
self._fout)
self._fout.write(b'upgraded %s %s\n' % (token, name))
servercaps = wireproto.capabilities(self._repo, proto)
rsp = b'capabilities: %s' % servercaps.data
self._fout.write(b'%d\n' % len(rsp))
self._fout.write(rsp)
self._fout.write(b'\n')
self._fout.flush()
super(upgradev2server, self).serve_forever()
def performhandshake(orig, ui, stdin, stdout, stderr): def performhandshake(orig, ui, stdin, stdout, stderr):
"""Wrapped version of sshpeer._performhandshake to send extra commands.""" """Wrapped version of sshpeer._performhandshake to send extra commands."""
mode = ui.config(b'sshpeer', b'handshake-mode') mode = ui.config(b'sshpeer', b'handshake-mode')
if mode == b'pre-no-args': if mode == b'pre-no-args':
ui.debug(b'sending no-args command\n') ui.debug(b'sending no-args command\n')
stdin.write(b'no-args\n') stdin.write(b'no-args\n')
stdin.flush() stdin.flush()
return orig(ui, stdin, stdout, stderr) return orig(ui, stdin, stdout, stderr)
# has to be invoked with a certain form for security reasons and # has to be invoked with a certain form for security reasons and
# `dummyssh` can't just add `--config` flags to the command line. # `dummyssh` can't just add `--config` flags to the command line.
servermode = ui.environ.get(b'SSHSERVERMODE') servermode = ui.environ.get(b'SSHSERVERMODE')
if servermode == b'banner': if servermode == b'banner':
wireprotoserver.sshserver = bannerserver wireprotoserver.sshserver = bannerserver
elif servermode == b'no-hello': elif servermode == b'no-hello':
wireprotoserver.sshserver = prehelloserver wireprotoserver.sshserver = prehelloserver
elif servermode == b'upgradev2':
wireprotoserver.sshserver = upgradev2server
elif servermode: elif servermode:
raise error.ProgrammingError(b'unknown server mode: %s' % servermode) raise error.ProgrammingError(b'unknown server mode: %s' % servermode)
peermode = ui.config(b'sshpeer', b'mode') peermode = ui.config(b'sshpeer', b'mode')
if peermode == b'extra-handshake-commands': if peermode == b'extra-handshake-commands':
extensions.wrapfunction(sshpeer, '_performhandshake', performhandshake) extensions.wrapfunction(sshpeer, '_performhandshake', performhandshake)
elif peermode: elif peermode:
raise error.ProgrammingError(b'unknown peer mode: %s' % peermode) raise error.ProgrammingError(b'unknown peer mode: %s' % peermode)

tests/test-ssh-proto.t

remote: 0 remote: 0
remote: 384 remote: 384
remote: capabilities: lookup changegroupsubset branchmap pushkey known getbundle unbundlehash batch streamreqs=generaldelta,revlogv1 $USUAL_BUNDLE2_CAPS_SERVER$ unbundle=HG10GZ,HG10BZ,HG10UN remote: capabilities: lookup changegroupsubset branchmap pushkey known getbundle unbundlehash batch streamreqs=generaldelta,revlogv1 $USUAL_BUNDLE2_CAPS_SERVER$ unbundle=HG10GZ,HG10BZ,HG10UN
remote: 1 remote: 1
url: ssh://user@dummy/server url: ssh://user@dummy/server
local: no local: no
pushable: yes pushable: yes
Enable version 2 support on server. We need to do this in hgrc because we can't
use --config with `hg serve --stdio`.
$ cat >> server/.hg/hgrc << EOF
> [experimental]
> sshserver.support-v2 = true
> EOF
Send an upgrade request to a server that supports upgrade Send an upgrade request to a server that supports upgrade
$ SSHSERVERMODE=upgradev2 hg -R server serve --stdio << EOF $ hg -R server serve --stdio << EOF
> upgrade this-is-some-token proto=exp-ssh-v2-0001 > upgrade this-is-some-token proto=exp-ssh-v2-0001
> hello > hello
> between > between
> pairs 81 > pairs 81
> 0000000000000000000000000000000000000000-0000000000000000000000000000000000000000 > 0000000000000000000000000000000000000000-0000000000000000000000000000000000000000
> EOF > EOF
upgraded this-is-some-token exp-ssh-v2-0001 upgraded this-is-some-token exp-ssh-v2-0001
383 383
capabilities: lookup changegroupsubset branchmap pushkey known getbundle unbundlehash batch streamreqs=generaldelta,revlogv1 $USUAL_BUNDLE2_CAPS_SERVER$ unbundle=HG10GZ,HG10BZ,HG10UN capabilities: lookup changegroupsubset branchmap pushkey known getbundle unbundlehash batch streamreqs=generaldelta,revlogv1 $USUAL_BUNDLE2_CAPS_SERVER$ unbundle=HG10GZ,HG10BZ,HG10UN
$ SSHSERVERMODE=upgradev2 hg --config experimental.sshpeer.advertise-v2=true --debug debugpeer ssh://user@dummy/server $ hg --config experimental.sshpeer.advertise-v2=true --debug debugpeer ssh://user@dummy/server
running * "*/tests/dummyssh" 'user@dummy' 'hg -R server serve --stdio' (glob) (no-windows !) running * "*/tests/dummyssh" 'user@dummy' 'hg -R server serve --stdio' (glob) (no-windows !)
running * "*\tests/dummyssh" "user@dummy" "hg -R server serve --stdio" (glob) (windows !) running * "*\tests/dummyssh" "user@dummy" "hg -R server serve --stdio" (glob) (windows !)
sending upgrade request: * proto=exp-ssh-v2-0001 (glob) sending upgrade request: * proto=exp-ssh-v2-0001 (glob)
devel-peer-request: hello devel-peer-request: hello
sending hello command sending hello command
devel-peer-request: between devel-peer-request: between
devel-peer-request: pairs: 81 bytes devel-peer-request: pairs: 81 bytes
sending between command sending between command
protocol upgraded to exp-ssh-v2-0001 protocol upgraded to exp-ssh-v2-0001
url: ssh://user@dummy/server url: ssh://user@dummy/server
local: no local: no
pushable: yes pushable: yes
Verify the peer has capabilities Verify the peer has capabilities
$ SSHSERVERMODE=upgradev2 hg --config experimental.sshpeer.advertise-v2=true --debug debugcapabilities ssh://user@dummy/server $ hg --config experimental.sshpeer.advertise-v2=true --debug debugcapabilities ssh://user@dummy/server
running * "*/tests/dummyssh" 'user@dummy' 'hg -R server serve --stdio' (glob) (no-windows !) running * "*/tests/dummyssh" 'user@dummy' 'hg -R server serve --stdio' (glob) (no-windows !)
running * "*\tests/dummyssh" "user@dummy" "hg -R server serve --stdio" (glob) (windows !) running * "*\tests/dummyssh" "user@dummy" "hg -R server serve --stdio" (glob) (windows !)
sending upgrade request: * proto=exp-ssh-v2-0001 (glob) sending upgrade request: * proto=exp-ssh-v2-0001 (glob)
devel-peer-request: hello devel-peer-request: hello
sending hello command sending hello command
devel-peer-request: between devel-peer-request: between
devel-peer-request: pairs: 81 bytes devel-peer-request: pairs: 81 bytes
sending between command sending between command
hgtagsfnodes hgtagsfnodes
listkeys listkeys
phases phases
heads heads
pushkey pushkey
remote-changegroup remote-changegroup
http http
https https
Command after upgrade to version 2 is processed
$ hg -R server serve --stdio << EOF
> upgrade this-is-some-token proto=exp-ssh-v2-0001
> hello
> between
> pairs 81
> 0000000000000000000000000000000000000000-0000000000000000000000000000000000000000hello
> EOF
upgraded this-is-some-token exp-ssh-v2-0001
383
capabilities: lookup changegroupsubset branchmap pushkey known getbundle unbundlehash batch streamreqs=generaldelta,revlogv1 $USUAL_BUNDLE2_CAPS_SERVER$ unbundle=HG10GZ,HG10BZ,HG10UN
384
capabilities: lookup changegroupsubset branchmap pushkey known getbundle unbundlehash batch streamreqs=generaldelta,revlogv1 $USUAL_BUNDLE2_CAPS_SERVER$ unbundle=HG10GZ,HG10BZ,HG10UN
Multiple upgrades is not allowed
$ hg -R server serve --stdio << EOF
> upgrade this-is-some-token proto=exp-ssh-v2-0001
> hello
> between
> pairs 81
> 0000000000000000000000000000000000000000-0000000000000000000000000000000000000000upgrade another-token proto=irrelevant
> hello
> EOF
upgraded this-is-some-token exp-ssh-v2-0001
383
capabilities: lookup changegroupsubset branchmap pushkey known getbundle unbundlehash batch streamreqs=generaldelta,revlogv1 $USUAL_BUNDLE2_CAPS_SERVER$ unbundle=HG10GZ,HG10BZ,HG10UN
cannot upgrade protocols multiple times
-
Malformed upgrade request line (not exactly 3 space delimited tokens)
$ hg -R server serve --stdio << EOF
> upgrade
> EOF
0
$ hg -R server serve --stdio << EOF
> upgrade token
> EOF
0
$ hg -R server serve --stdio << EOF
> upgrade token foo=bar extra-token
> EOF
0
Upgrade request to unsupported protocol is ignored
$ hg -R server serve --stdio << EOF
> upgrade this-is-some-token proto=unknown1,unknown2
> hello
> between
> pairs 81
> 0000000000000000000000000000000000000000-0000000000000000000000000000000000000000
> EOF
0
384
capabilities: lookup changegroupsubset branchmap pushkey known getbundle unbundlehash batch streamreqs=generaldelta,revlogv1 $USUAL_BUNDLE2_CAPS_SERVER$ unbundle=HG10GZ,HG10BZ,HG10UN
1
Upgrade request must be followed by hello + between
$ hg -R server serve --stdio << EOF
> upgrade token proto=exp-ssh-v2-0001
> invalid
> EOF
malformed handshake protocol: missing hello
-
$ hg -R server serve --stdio << EOF
> upgrade token proto=exp-ssh-v2-0001
> hello
> invalid
> EOF
malformed handshake protocol: missing between
-
$ hg -R server serve --stdio << EOF
> upgrade token proto=exp-ssh-v2-0001
> hello
> between
> invalid
> EOF
malformed handshake protocol: missing pairs 81
-