This is an archive of the discontinued Mercurial Phabricator instance.

Differential D3231

wireproto: move version 2 command handlers to wireprotov2server
ClosedPublic

Authored by indygreg on Apr 10 2018, 9:28 PM.

Download Raw Diff

Details

Reviewers

durin42

Group Reviewers

hg-reviewers

Commits

rHG3a2367e6c6f2: wireproto: move version 2 command handlers to wireprotov2server

Summary

This is relatively straightforward.

As part of this, we introduced a local @wireprotocommand that
wraps the main one and defines a v2 only policy by default.

Because the hacky HTTPv2 peer isn't using capabilities response
yet, we had to move some code around to force import of
wireprotov2server so commands are registered. This is super
hacky. But this code will go away once the HTTPv2 peer is using
the capabilities response to derive permissions.

Diff Detail

Repository

rHG Mercurial

Lint

Automatic diff as part of commit; lint not applicable.

Unit

Automatic diff as part of commit; unit tests not applicable.

Event Timeline

indygreg created this revision.Apr 10 2018, 9:28 PM

Herald added a reviewer: hg-reviewers. · View Herald TranscriptApr 10 2018, 9:28 PM

Herald added a subscriber: mercurial-devel. · View Herald Transcript

indygreg added a child revision: D3232: httppeer: move requestbuilder defaults into makepeer() argument.Apr 10 2018, 9:28 PM

durin42 accepted this revision.Apr 11 2018, 12:37 PM

This revision is now accepted and ready to land.Apr 11 2018, 12:37 PM

Closed by commit rHG3a2367e6c6f2: wireproto: move version 2 command handlers to wireprotov2server (authored by indygreg). · Explain WhyApr 11 2018, 1:00 PM

This revision was automatically updated to reflect the committed changes.

Revision Contents
Changeset List

			Path	Packages
M			mercurial/httppeer.py (15 lines)
M			mercurial/wireproto.py (110 lines)
M			mercurial/wireprotov2server.py (110 lines)

Status	Author	Revision
Closed	indygreg	D3243 httppeer: support protocol upgrade
Closed	indygreg	D3242 wireproto: define and implement HTTP handshake to upgrade protocol
Closed	indygreg	D3241 httppeer: only advertise partial-pull if capabilities are known
Closed	indygreg	D3240 httppeer: always add x-hg* headers to Vary header
Closed	indygreg	D3239 httppeer: don't accept very old media types (BC)
Closed	indygreg	D3238 httppeer: allow opener to be passed to makepeer()
Closed	indygreg	D3237 httppeer: perform capabilities request in makepeer()
Closed	indygreg	D3236 httppeer: extract common response handling into own function
Closed	indygreg	D3235 httppeer: move error handling and response wrapping into sendrequest
Closed	indygreg	D3234 httppeer: extract code for creating a request into own function
Closed	indygreg	D3233 httppeer: extract code for performing an HTTP request
Closed	indygreg	D3232 httppeer: move requestbuilder defaults into makepeer() argument
Closed	indygreg	D3231 wireproto: move version 2 command handlers to wireprotov2server
Closed	indygreg	D3230 wireproto: extract HTTP version 2 code to own module
Closed	indygreg	D3224 wireproto: client reactor support for receiving frames
Closed	indygreg	D3223 wireproto: introduce a reactor for client-side state
Closed	indygreg	D3222 tests: extract wire protocol framing tests to own file
Closed	indygreg	D3208 wireproto: disallow commands handlers for multiple transport versions
Closed	indygreg	D3207 wireproto: make @wireprotocommand version 1 only by default
Closed	indygreg	D3206 wireproto: only expose "getbundle" and "unbundle" to v1 transports
Closed	indygreg	D3205 wireproto: port lookup to wire protocol v2
Closed	indygreg	D3204 wireproto: port pushkey command to wire protocol version 2
Closed	indygreg	D3203 wireproto: only expose "clonebundles" to version 1 transports
Closed	indygreg	D3202 wireproto: define and expose types of wire command arguments
Closed	indygreg	D3201 wireproto: only expose "stream_out" to version 1 transports
Closed	indygreg	D3200 wireproto: implement capabilities for wire protocol v2

Diff 7992

mercurial/httppeer.py


	def close(self):			def close(self):
	pass			pass

	# TODO require to be part of a batched primitive, use futures.			# TODO require to be part of a batched primitive, use futures.
	def _call(self, name, **args):			def _call(self, name, **args):
	"""Call a wire protocol command with arguments."""			"""Call a wire protocol command with arguments."""

				# Having this early has a side-effect of importing wireprotov2server,
				# which has the side-effect of ensuring commands are registered.

				# TODO modify user-agent to reflect v2.
				headers = {
				r'Accept': wireprotov2server.FRAMINGTYPE,
				r'Content-Type': wireprotov2server.FRAMINGTYPE,
				}

	# TODO permissions should come from capabilities results.			# TODO permissions should come from capabilities results.
	permission = wireproto.commandsv2[name].permission			permission = wireproto.commandsv2[name].permission
	if permission not in ('push', 'pull'):			if permission not in ('push', 'pull'):
	raise error.ProgrammingError('unknown permission type: %s' %			raise error.ProgrammingError('unknown permission type: %s' %
	permission)			permission)

	permission = {			permission = {
	'push': 'rw',			'push': 'rw',
	'pull': 'ro',			'pull': 'ro',
	}[permission]			}[permission]

	url = '%s/api/%s/%s/%s' % (self.url, wireprotov2server.HTTPV2,			url = '%s/api/%s/%s/%s' % (self.url, wireprotov2server.HTTPV2,
	permission, name)			permission, name)

	# TODO modify user-agent to reflect v2.
	headers = {
	r'Accept': wireprotov2server.FRAMINGTYPE,
	r'Content-Type': wireprotov2server.FRAMINGTYPE,
	}

	# TODO this should be part of a generic peer for the frame-based			# TODO this should be part of a generic peer for the frame-based
	# protocol.			# protocol.
	reactor = wireprotoframing.clientreactor(hasmultiplesend=False,			reactor = wireprotoframing.clientreactor(hasmultiplesend=False,
	buffersends=True)			buffersends=True)

	request, action, meta = reactor.callcommand(name, args)			request, action, meta = reactor.callcommand(name, args)
	assert action == 'noop'			assert action == 'noop'

mercurial/wireproto.py

	if exc.hint is not None:			if exc.hint is not None:
	advargs.append(('hint', exc.hint))			advargs.append(('hint', exc.hint))
	bundler.addpart(bundle2.bundlepart('error:abort',			bundler.addpart(bundle2.bundlepart('error:abort',
	manargs, advargs))			manargs, advargs))
	except error.PushRaced as exc:			except error.PushRaced as exc:
	bundler.newpart('error:pushraced',			bundler.newpart('error:pushraced',
	[('message', stringutil.forcebytestr(exc))])			[('message', stringutil.forcebytestr(exc))])
	return wireprototypes.streamreslegacy(gen=bundler.getchunks())			return wireprototypes.streamreslegacy(gen=bundler.getchunks())

	# Wire protocol version 2 commands only past this point.

	def _capabilitiesv2(repo, proto):
	"""Obtain the set of capabilities for version 2 transports.

	These capabilities are distinct from the capabilities for version 1
	transports.
	"""
	compression = []
	for engine in supportedcompengines(repo.ui, util.SERVERROLE):
	compression.append({
	b'name': engine.wireprotosupport().name,
	})

	caps = {
	'commands': {},
	'compression': compression,
	}

	for command, entry in commandsv2.items():
	caps['commands'][command] = {
	'args': entry.args,
	'permissions': [entry.permission],
	}

	return proto.addcapabilities(repo, caps)

	@wireprotocommand('branchmap', permission='pull',
	transportpolicy=POLICY_V2_ONLY)
	def branchmapv2(repo, proto):
	branchmap = {encoding.fromlocal(k): v
	for k, v in repo.branchmap().iteritems()}

	return wireprototypes.cborresponse(branchmap)

	@wireprotocommand('capabilities', permission='pull',
	transportpolicy=POLICY_V2_ONLY)
	def capabilitiesv2(repo, proto):
	caps = _capabilitiesv2(repo, proto)

	return wireprototypes.cborresponse(caps)

	@wireprotocommand('heads',
	args={
	'publiconly': False,
	},
	permission='pull',
	transportpolicy=POLICY_V2_ONLY)
	def headsv2(repo, proto, publiconly=False):
	if publiconly:
	repo = repo.filtered('immutable')

	return wireprototypes.cborresponse(repo.heads())

	@wireprotocommand('known',
	args={
	'nodes': [b'deadbeef'],
	},
	permission='pull',
	transportpolicy=POLICY_V2_ONLY)
	def knownv2(repo, proto, nodes=None):
	nodes = nodes or []
	result = b''.join(b'1' if n else b'0' for n in repo.known(nodes))
	return wireprototypes.cborresponse(result)

	@wireprotocommand('listkeys',
	args={
	'namespace': b'ns',
	},
	permission='pull',
	transportpolicy=POLICY_V2_ONLY)
	def listkeysv2(repo, proto, namespace=None):
	keys = repo.listkeys(encoding.tolocal(namespace))
	keys = {encoding.fromlocal(k): encoding.fromlocal(v)
	for k, v in keys.iteritems()}

	return wireprototypes.cborresponse(keys)

	@wireprotocommand('lookup',
	args={
	'key': b'foo',
	},
	permission='pull',
	transportpolicy=POLICY_V2_ONLY)
	def lookupv2(repo, proto, key):
	key = encoding.tolocal(key)

	# TODO handle exception.
	node = repo.lookup(key)

	return wireprototypes.cborresponse(node)

	@wireprotocommand('pushkey',
	args={
	'namespace': b'ns',
	'key': b'key',
	'old': b'old',
	'new': b'new',
	},
	permission='push',
	transportpolicy=POLICY_V2_ONLY)
	def pushkeyv2(repo, proto, namespace, key, old, new):
	# TODO handle ui output redirection
	r = repo.pushkey(encoding.tolocal(namespace),
	encoding.tolocal(key),
	encoding.tolocal(old),
	encoding.tolocal(new))

	return wireprototypes.cborresponse(r)

mercurial/wireprotov2server.py

	from .i18n import _			from .i18n import _
	from .thirdparty import (			from .thirdparty import (
	cbor,			cbor,
	)			)
	from .thirdparty.zope import (			from .thirdparty.zope import (
	interface as zi,			interface as zi,
	)			)
	from . import (			from . import (
				encoding,
	error,			error,
	pycompat,			pycompat,
				util,
	wireproto,			wireproto,
	wireprotoframing,			wireprotoframing,
	wireprototypes,			wireprototypes,
	)			)

	FRAMINGTYPE = b'application/mercurial-exp-framing-0003'			FRAMINGTYPE = b'application/mercurial-exp-framing-0003'

	HTTPV2 = wireprototypes.HTTPV2			HTTPV2 = wireprototypes.HTTPV2
	def client(self):			def client(self):
	raise NotImplementedError			raise NotImplementedError

	def addcapabilities(self, repo, caps):			def addcapabilities(self, repo, caps):
	return caps			return caps

	def checkperm(self, perm):			def checkperm(self, perm):
	raise NotImplementedError			raise NotImplementedError

				def _capabilitiesv2(repo, proto):
				"""Obtain the set of capabilities for version 2 transports.

				These capabilities are distinct from the capabilities for version 1
				transports.
				"""
				compression = []
				for engine in wireproto.supportedcompengines(repo.ui, util.SERVERROLE):
				compression.append({
				b'name': engine.wireprotosupport().name,
				})

				caps = {
				'commands': {},
				'compression': compression,
				}

				for command, entry in wireproto.commandsv2.items():
				caps['commands'][command] = {
				'args': entry.args,
				'permissions': [entry.permission],
				}

				return proto.addcapabilities(repo, caps)

				def wireprotocommand(args, *kwargs):
				def register(func):
				return wireproto.wireprotocommand(
				args, transportpolicy=wireproto.POLICY_V2_ONLY, *kwargs)(func)

				return register

				@wireprotocommand('branchmap', permission='pull')
				def branchmapv2(repo, proto):
				branchmap = {encoding.fromlocal(k): v
				for k, v in repo.branchmap().iteritems()}

				return wireprototypes.cborresponse(branchmap)

				@wireprotocommand('capabilities', permission='pull')
				def capabilitiesv2(repo, proto):
				caps = _capabilitiesv2(repo, proto)

				return wireprototypes.cborresponse(caps)

				@wireprotocommand('heads',
				args={
				'publiconly': False,
				},
				permission='pull')
				def headsv2(repo, proto, publiconly=False):
				if publiconly:
				repo = repo.filtered('immutable')

				return wireprototypes.cborresponse(repo.heads())

				@wireprotocommand('known',
				args={
				'nodes': [b'deadbeef'],
				},
				permission='pull')
				def knownv2(repo, proto, nodes=None):
				nodes = nodes or []
				result = b''.join(b'1' if n else b'0' for n in repo.known(nodes))
				return wireprototypes.cborresponse(result)

				@wireprotocommand('listkeys',
				args={
				'namespace': b'ns',
				},
				permission='pull')
				def listkeysv2(repo, proto, namespace=None):
				keys = repo.listkeys(encoding.tolocal(namespace))
				keys = {encoding.fromlocal(k): encoding.fromlocal(v)
				for k, v in keys.iteritems()}

				return wireprototypes.cborresponse(keys)

				@wireprotocommand('lookup',
				args={
				'key': b'foo',
				},
				permission='pull')
				def lookupv2(repo, proto, key):
				key = encoding.tolocal(key)

				# TODO handle exception.
				node = repo.lookup(key)

				return wireprototypes.cborresponse(node)

				@wireprotocommand('pushkey',
				args={
				'namespace': b'ns',
				'key': b'key',
				'old': b'old',
				'new': b'new',
				},
				permission='push')
				def pushkeyv2(repo, proto, namespace, key, old, new):
				# TODO handle ui output redirection
				r = repo.pushkey(encoding.tolocal(namespace),
				encoding.tolocal(key),
				encoding.tolocal(old),
				encoding.tolocal(new))

				return wireprototypes.cborresponse(r)

Diff	ID	Description	Created	Lint	Unit
Base		Base
Diff 1	7949		Apr 10 2018, 9:28 PM	★	★
Diff 2	7992	rHG3a2367e6c6f2106ca60f3dd04cd4c02b288fe90e	Apr 9 2018, 10:35 PM	★	★