This is an archive of the discontinued Mercurial Phabricator instance.

Differential D6050

revsetlang: use sysbytes() instead of blind encode()
ClosedPublic

Authored by indygreg on Mar 2 2019, 4:26 PM.

Details

Reviewers
None
Group Reviewers
hg-reviewers
Commits
rHGddb174511f1b: revsetlang: use sysbytes() instead of blind encode()
Summary

Otherwise we will call str.encode() on Python 2, which is wrong.

sysbytes() does encode('utf-8) on Python 3. But the source is
guaranteed ASCII, so it shouldn't matter.

With this change, hg now runs with HGUNICODEPEDANTRY=1 set.
However, several tests are failing.

Diff Detail

Repository
rHG Mercurial
Lint
Automatic diff as part of commit; lint not applicable.
Unit
Automatic diff as part of commit; unit tests not applicable.

Event Timeline

indygreg created this revision.Mar 2 2019, 4:26 PM
Closed by commit rHGddb174511f1b: revsetlang: use sysbytes() instead of blind encode() (authored by indygreg). · Explain WhyMar 3 2019, 4:34 AM
This revision was automatically updated to reflect the committed changes.

Revision Contents
Changeset List

PathPackages
Mmercurial/revsetlang.py (4 lines)

Diff 14315

mercurial/revsetlang.py

symbols = {} symbols = {}
_quoteletters = {'"', "'"} _quoteletters = {'"', "'"}
_simpleopletters = set(pycompat.iterbytestr("()[]#:=,-|&+!~^%")) _simpleopletters = set(pycompat.iterbytestr("()[]#:=,-|&+!~^%"))
# default set of valid characters for the initial letter of symbols # default set of valid characters for the initial letter of symbols
_syminitletters = set(pycompat.iterbytestr( _syminitletters = set(pycompat.iterbytestr(
string.ascii_letters.encode('ascii') + pycompat.sysbytes(string.ascii_letters) +
string.digits.encode('ascii') + pycompat.sysbytes(string.digits) +
'._@')) | set(map(pycompat.bytechr, pycompat.xrange(128, 256))) '._@')) | set(map(pycompat.bytechr, pycompat.xrange(128, 256)))
# default set of valid characters for non-initial letters of symbols # default set of valid characters for non-initial letters of symbols
_symletters = _syminitletters | set(pycompat.iterbytestr('-/')) _symletters = _syminitletters | set(pycompat.iterbytestr('-/'))
def tokenize(program, lookup=None, syminitletters=None, symletters=None): def tokenize(program, lookup=None, syminitletters=None, symletters=None):
''' '''
Parse a revset statement into a stream of tokens Parse a revset statement into a stream of tokens