This is an archive of the discontinued Mercurial Phabricator instance.

Differential D5288

tests: make test-alias.t pass with re2
ClosedPublic

Authored by valentin.gatienbaron on Nov 19 2018, 1:40 PM.

Details

Reviewers
None
Group Reviewers
hg-reviewers
Commits
rHG557d3509da55: tests: make test-alias.t pass with re2
Summary

Locally, these "non-ASCII character in alias" errors don't show up,
though I get them when the alias is defined at the command line rather
than in an hgrc.
The brokenness comes from the fact that hgrcs are parsed with regexes,
and re/re2 differ in this way:

$ python -c 'import re; print(re.compile("(.*)").match("aaa\xc0bbbb").groups())'
('aaa\xc0bbbb',)
$ python -c 'import re2; print(re2.compile("(.*)").match("aaa\xc0bbbb").groups())'
('aaa',)

Apparently re2 stops when it encounters invalid utf8 (which I suppose makes sense
given that '.' matches what appears to be a codepoint rather than a byte). This is
presumably a bug in hg, but not very important, so just change the test to stick
to valid utf8.

Diff Detail

Repository
rHG Mercurial
Lint
Lint Skipped
Unit
Unit Tests Skipped

Event Timeline

yuja added a subscriber: yuja.Nov 20 2018, 7:22 AM

Queued, thanks.

Closed by commit rHG557d3509da55: tests: make test-alias.t pass with re2 (authored by valentin.gatienbaron). · Explain WhyNov 20 2018, 7:26 AM
This revision was automatically updated to reflect the committed changes.
valentin.gatienbaron added a comment.Nov 20 2018, 11:42 PM

Thanks. FWIW, re2 options allow to choose an encoding which is latin1 or utf8 (https://github.com/google/re2/blob/master/re2/re2.h#L609). Presumably latin1 means that "." matches a byte, which would seem more compatible with re, but python bindings don't provide the ability to choose this encoding.

yuja added a comment.Nov 21 2018, 9:22 AM
Thanks. FWIW, re2 options allow to choose an encoding which is latin1 or utf8 (https://github.com/google/re2/blob/master/re2/re2.h#L609). Presumably latin1 means that "." matches a byte, which would seem more compatible with re, but python bindings don't provide the ability to choose this encoding.

Indeed. We could pass in a fat unicode array (i.e. each byte as latin-1 char)
to re2, but that would sacrifice the performance.

Thanks for the info.

Revision Contents
Changeset List

PathPackages
Mtests/test-alias.t (4 lines)

Diff 12565

tests/test-alias.t

(some details hidden, use --verbose to show complete help) (some details hidden, use --verbose to show complete help)
invalid character in user-specified help invalid character in user-specified help
>>> with open('.hg/hgrc', 'ab') as f: >>> with open('.hg/hgrc', 'ab') as f:
... f.write(b'[alias]\n' ... f.write(b'[alias]\n'
... b'invaliddoc = log\n' ... b'invaliddoc = log\n'
... b'invaliddoc:doc = \xc0\n' ... b'invaliddoc:doc = \xc3\xa9\n'
... b'invalidhelp = log\n' ... b'invalidhelp = log\n'
... b'invalidhelp:help = \xc0\n') and None ... b'invalidhelp:help = \xc3\xa9\n') and None
$ hg help invaliddoc $ hg help invaliddoc
non-ASCII character in alias definition 'invaliddoc:doc' non-ASCII character in alias definition 'invaliddoc:doc'
$ hg help invalidhelp $ hg help invalidhelp
non-ASCII character in alias definition 'invalidhelp:help' non-ASCII character in alias definition 'invalidhelp:help'
$ hg invaliddoc $ hg invaliddoc
abort: non-ASCII character in alias definition 'invaliddoc:doc' abort: non-ASCII character in alias definition 'invaliddoc:doc'
[255] [255]
$ hg invalidhelp $ hg invalidhelp