This is an archive of the discontinued Mercurial Phabricator instance.

Differential D5052

mercurial: strip function return type annotations (RFC)
AbandonedPublic

Authored by indygreg on Oct 13 2018, 3:29 AM.

Details

Reviewers
baymax
Group Reviewers
hg-reviewers
Summary

THIS IS SUPER HACKY. DO NOT EVEN THINK ABOUT LANDING YET.

Now that we have a custom module importer on Python 2 and that importer
is using Python 3.7's tokenizer, it is possible for us to analyze the
source code token stream and transform it to valid Python 2.7 source
before it is compiled into a code object.

This commit teaches the token stream transformer to recognize the tokens
for a function return type annotation and to strip those tokens from the
token stream. As far as Python 2's code compiler is concerned, the original
source didn't contain a type annotation!

To prove it works, we define a type annotation in the dispatch module.
Python 2 is able to import the module just fine.

This feature means we could start adding type annotations throughout the
code base while still remaining Python 2 compatible.

But why stop at type annotations? We could also rewrite the token stream
so we can leverage other Python 3 only features. This includes "yield from"
super() without arguments, class definitions without "(object)", etc. We
could also "invert" our source transformer strategy and make the code base
native Python 3 and source transform to remain compatible with Python 2.

Diff Detail

Repository
rHG Mercurial
Lint
Lint Skipped
Unit
Unit Tests Skipped

Event Timeline

indygreg created this revision.Oct 13 2018, 3:29 AM
baymax requested changes to this revision.Jan 24 2020, 12:32 PM

There seems to have been no activities on this Diff for the past 3 Months.

By policy, we are automatically moving it out of the need-review state.

Please, move it back to need-review without hesitation if this diff should still be discussed.

:baymax:need-review-idle:

This revision now requires changes to proceed.Jan 24 2020, 12:32 PM
indygreg abandoned this revision.Jan 24 2020, 11:52 PM

Revision Contents
Changeset List

PathPackages
Mmercurial/__init__.py (13 lines)
Mmercurial/dispatch.py (2 lines)

Diff 12008

mercurial/__init__.py

import binascii import binascii
import errno import errno
import io import io
import imp import imp
import marshal import marshal
import os import os
import struct import struct
from hgdemandimport import py3tokenize as tokenize from hgdemandimport import py3tokenize as tokenize
from hgdemandimport import py3token as token
rootpath = os.path.normpath(os.path.join(os.path.dirname(__file__), '..')) rootpath = os.path.normpath(os.path.join(os.path.dirname(__file__), '..'))
def replacetokens(tokens, fullname): def replacetokens(tokens, fullname):
for i, t in enumerate(tokens): i = 0
while i < len(tokens):
t = tokens[i]
# Function return type annotation.
if t.type == token.OP and t.string == '->':
t2 = tokens[i + 1]
i += 2
continue
yield t yield t
i += 1
# Header to add to bytecode files. This MUST be changed when # Header to add to bytecode files. This MUST be changed when
# ``replacetokens`` or any mechanism that changes semantics of module # ``replacetokens`` or any mechanism that changes semantics of module
# loading is changed. Otherwise cached bytecode may get loaded without # loading is changed. Otherwise cached bytecode may get loaded without
# the new transformation mechanisms applied. # the new transformation mechanisms applied.
BYTECODEHEADER = b'HG\x00\x01' BYTECODEHEADER = b'HG\x00\x01'
def loadbytecode(path): def loadbytecode(path):

mercurial/dispatch.py

write(_("hg: parse error: %s\n") % inst.args[0]) write(_("hg: parse error: %s\n") % inst.args[0])
_reportsimilar(write, similar) _reportsimilar(write, similar)
if inst.hint: if inst.hint:
write(_("(%s)\n") % inst.hint) write(_("(%s)\n") % inst.hint)
def _formatargs(args): def _formatargs(args):
return ' '.join(procutil.shellquote(a) for a in args) return ' '.join(procutil.shellquote(a) for a in args)
def dispatch(req): def dispatch(req) -> int:
"""run the command specified in req.args; returns an integer status code""" """run the command specified in req.args; returns an integer status code"""
with tracing.log('dispatch.dispatch'): with tracing.log('dispatch.dispatch'):
if req.ferr: if req.ferr:
ferr = req.ferr ferr = req.ferr
elif req.ui: elif req.ui:
ferr = req.ui.ferr ferr = req.ui.ferr
else: else:
ferr = procutil.stderr ferr = procutil.stderr