This is an archive of the discontinued Mercurial Phabricator instance.

Differential D3841

stringutil: add a new function to do minimal regex escaping
ClosedPublic

Authored by durin42 on Jun 26 2018, 11:22 AM.

Details

Reviewers
pulkit
Group Reviewers
hg-reviewers
Commits
rHG96f65bdf0bf4: stringutil: add a new function to do minimal regex escaping
Summary

Per https://bugs.python.org/issue29995, re.escape() used to
over-escape regular expression strings, but in Python 3.7 that's been
fixed, which also improved the performance of re.escape(). Since it's
both an output change for us *and* a perfomance win, let's just
effectively backport the new behavior to hg on all Python versions.

Diff Detail

Repository
rHG Mercurial
Lint
Lint Skipped
Unit
Unit Tests Skipped

Event Timeline

durin42 created this revision.Jun 26 2018, 11:22 AM
pulkit accepted this revision.Jun 26 2018, 1:26 PM
Closed by commit rHG96f65bdf0bf4: stringutil: add a new function to do minimal regex escaping (authored by durin42). · Explain WhyJun 26 2018, 1:51 PM
This revision was automatically updated to reflect the committed changes.
yuja added a subscriber: yuja.Jun 27 2018, 9:47 AM

+# regex special chars pulled from https://bugs.python.org/issue29995
+# which was part of Python 3.7.
+_respecial = pycompat.bytestr(b'()[]{}?*+-|^$\\.# \t\n\r\v\f')
+_regexescapemap = {ord(i): (b'\\' + i).decode('latin1') for i in _respecial}

The Py3.7 version also includes '&' and '~'.

https://github.com/python/cpython/blob/v3.7.0rc1/Lib/re.py#L248

durin42 added a comment.Jun 27 2018, 10:21 AM
In D3841#60110, @yuja wrote:

+# regex special chars pulled from https://bugs.python.org/issue29995
+# which was part of Python 3.7.
+_respecial = pycompat.bytestr(b'()[]{}?*+-|^$\\.# \t\n\r\v\f')
+_regexescapemap = {ord(i): (b'\\' + i).decode('latin1') for i in _respecial}

The Py3.7 version also includes '&' and '~'.
https://github.com/python/cpython/blob/v3.7.0rc1/Lib/re.py#L248

Nice catch, mailed D3850 as a follow-up.

Revision Contents
Changeset List

PathPackages
Mmercurial/utils/stringutil.py (19 lines)

Diff 9308

mercurial/utils/stringutil.py

from ..thirdparty import attr from ..thirdparty import attr
from .. import ( from .. import (
encoding, encoding,
error, error,
pycompat, pycompat,
) )
# regex special chars pulled from https://bugs.python.org/issue29995
# which was part of Python 3.7.
_respecial = pycompat.bytestr(b'()[]{}?*+-|^$\\.# \t\n\r\v\f')
_regexescapemap = {ord(i): (b'\\' + i).decode('latin1') for i in _respecial}
def reescape(pat):
"""Drop-in replacement for re.escape."""
# NOTE: it is intentional that this works on unicodes and not
# bytes, as it's only possible to do the escaping with
# unicode.translate, not bytes.translate. Sigh.
wantuni = True
if isinstance(pat, bytes):
wantuni = False
pat = pat.decode('latin1')
pat = pat.translate(_regexescapemap)
if wantuni:
return pat
return pat.encode('latin1')
def pprint(o, bprefix=False): def pprint(o, bprefix=False):
"""Pretty print an object.""" """Pretty print an object."""
if isinstance(o, bytes): if isinstance(o, bytes):
if bprefix: if bprefix:
return "b'%s'" % escapestr(o) return "b'%s'" % escapestr(o)
return "'%s'" % escapestr(o) return "'%s'" % escapestr(o)
elif isinstance(o, bytearray): elif isinstance(o, bytearray):
# codecs.escape_encode() can't handle bytearray, so escapestr fails # codecs.escape_encode() can't handle bytearray, so escapestr fails