This is an archive of the discontinued Mercurial Phabricator instance.

Differential D7020

pycompat: implement a shlexquote that properly handles bytes
AbandonedPublic

Authored by indygreg on Oct 8 2019, 12:42 AM.

Details

Reviewers
None
Group Reviewers
hg-reviewers
Summary

2cc453284d5 introduced this function call to for mail.py. This broke Python
3 because shlex.quote() expects str, not bytes. In order to unbust it,
we need to normalize bytes to str then go back to bytes to appease the
caller.

This is a bit ugly. But I don't see any other obvious solution.

Diff Detail

Repository
rHG Mercurial
Lint
Lint Skipped
Unit
Unit Tests Skipped

Event Timeline

indygreg created this revision.Oct 8 2019, 12:42 AM
marmoute added a subscriber: marmoute.Oct 8 2019, 6:33 AM
marmoute added inline comments.
mercurial/pycompat.py
343

It might be a silly questions, but why are we going through latin-1 instead of utf-8 or any other crazy encoding ?

indygreg added inline comments.Oct 8 2019, 11:05 AM
mercurial/pycompat.py
343

Because the underlying string is likely raw bytes and latin-1 will preserve those code points. i.e. we don't care what the encoding is: just that we can pass str to shlex.quote.

indygreg abandoned this revision.Oct 9 2019, 11:15 PM

Yuya's patch fixed the test failure and is a better solution.

Revision Contents
Changeset List

PathPackages
Mmercurial/pycompat.py (6 lines)

Diff 16957

mercurial/pycompat.py

""" """
Takes bytes argument, convert it to str i.e. unicodes, pass that into Takes bytes argument, convert it to str i.e. unicodes, pass that into
shlex.split(), convert the returned value to bytes and return that for shlex.split(), convert the returned value to bytes and return that for
Python 3 compatibility as shelx.split() don't accept bytes on Python 3. Python 3 compatibility as shelx.split() don't accept bytes on Python 3.
""" """
ret = shlex.split(s.decode('latin-1'), comments, posix) ret = shlex.split(s.decode('latin-1'), comments, posix)
return [a.encode('latin-1') for a in ret] return [a.encode('latin-1') for a in ret]
shlexquote = shlex.quote def shlexquote(s):
s = s.decode('latin-1')
s = shlex.quote(s)
return s.encode('latin-1')
marmouteUnsubmitted
Not Done

It might be a silly questions, but why are we going through latin-1 instead of utf-8 or any other crazy encoding ?

marmoute: It might be a silly questions, but why are we going through `latin-1` instead of `utf-8` or any…
indygregAuthorUnsubmitted
Done

Because the underlying string is likely raw bytes and latin-1 will preserve those code points. i.e. we don't care what the encoding is: just that we can pass str to shlex.quote.

indygreg: Because the underlying string is likely raw bytes and `latin-1` will preserve those code points.
iteritems = lambda x: x.items() iteritems = lambda x: x.items()
itervalues = lambda x: x.values() itervalues = lambda x: x.values()
else: else:
import cStringIO import cStringIO
import pipes import pipes
xrange = xrange xrange = xrange