This is an archive of the discontinued Mercurial Phabricator instance.

Differential D3841

stringutil: add a new function to do minimal regex escaping
ClosedPublic

Authored by durin42 on Jun 26 2018, 11:22 AM.

Details

Reviewers
pulkit
Group Reviewers
hg-reviewers
Commits
rHG96f65bdf0bf4: stringutil: add a new function to do minimal regex escaping
Summary

Per https://bugs.python.org/issue29995, re.escape() used to
over-escape regular expression strings, but in Python 3.7 that's been
fixed, which also improved the performance of re.escape(). Since it's
both an output change for us *and* a perfomance win, let's just
effectively backport the new behavior to hg on all Python versions.

Diff Detail

Repository
rHG Mercurial
Lint
Automatic diff as part of commit; lint not applicable.
Unit
Automatic diff as part of commit; unit tests not applicable.

Event Timeline

durin42 created this revision.Jun 26 2018, 11:22 AM
pulkit accepted this revision.Jun 26 2018, 1:26 PM
Closed by commit rHG96f65bdf0bf4: stringutil: add a new function to do minimal regex escaping (authored by durin42). · Explain WhyJun 26 2018, 1:51 PM
This revision was automatically updated to reflect the committed changes.
yuja added a subscriber: yuja.Jun 27 2018, 9:47 AM

+# regex special chars pulled from https://bugs.python.org/issue29995
+# which was part of Python 3.7.
+_respecial = pycompat.bytestr(b'()[]{}?*+-|^$\\.# \t\n\r\v\f')
+_regexescapemap = {ord(i): (b'\\' + i).decode('latin1') for i in _respecial}

The Py3.7 version also includes '&' and '~'.

https://github.com/python/cpython/blob/v3.7.0rc1/Lib/re.py#L248

durin42 added a comment.Jun 27 2018, 10:21 AM
In D3841#60110, @yuja wrote:

+# regex special chars pulled from https://bugs.python.org/issue29995
+# which was part of Python 3.7.
+_respecial = pycompat.bytestr(b'()[]{}?*+-|^$\\.# \t\n\r\v\f')
+_regexescapemap = {ord(i): (b'\\' + i).decode('latin1') for i in _respecial}

The Py3.7 version also includes '&' and '~'.
https://github.com/python/cpython/blob/v3.7.0rc1/Lib/re.py#L248

Nice catch, mailed D3850 as a follow-up.

Revision Contents
Changeset List

PathPackages
Mmercurial/utils/stringutil.py (19 lines)

Diff 9312

mercurial/utils/stringutil.py

from ..thirdparty import attr from ..thirdparty import attr
from .. import ( from .. import (
encoding, encoding,
error, error,
pycompat, pycompat,
) )
# regex special chars pulled from https://bugs.python.org/issue29995
# which was part of Python 3.7.
_respecial = pycompat.bytestr(b'()[]{}?*+-|^$\\.# \t\n\r\v\f')
_regexescapemap = {ord(i): (b'\\' + i).decode('latin1') for i in _respecial}
def reescape(pat):
"""Drop-in replacement for re.escape."""
# NOTE: it is intentional that this works on unicodes and not
# bytes, as it's only possible to do the escaping with
# unicode.translate, not bytes.translate. Sigh.
wantuni = True
if isinstance(pat, bytes):
wantuni = False
pat = pat.decode('latin1')
pat = pat.translate(_regexescapemap)
if wantuni:
return pat
return pat.encode('latin1')
def pprint(o, bprefix=False): def pprint(o, bprefix=False):
"""Pretty print an object.""" """Pretty print an object."""
if isinstance(o, bytes): if isinstance(o, bytes):
if bprefix: if bprefix:
return "b'%s'" % escapestr(o) return "b'%s'" % escapestr(o)
return "'%s'" % escapestr(o) return "'%s'" % escapestr(o)
elif isinstance(o, bytearray): elif isinstance(o, bytearray):
# codecs.escape_encode() can't handle bytearray, so escapestr fails # codecs.escape_encode() can't handle bytearray, so escapestr fails