( )⚙ D5025 py3: make test-contrib-perf.t work on python 3

This is an archive of the discontinued Mercurial Phabricator instance.

Differential D5025

py3: make test-contrib-perf.t work on python 3
ClosedPublic

Authored by pulkit on Oct 12 2018, 9:58 PM.

Details

Reviewers
None
Group Reviewers
hg-reviewers
Commits
rHGb456b2e0ad9f: py3: make test-contrib-perf.t work on python 3
Summary

Diff Detail

Repository
rHG Mercurial
Lint
Automatic diff as part of commit; lint not applicable.
Unit
Automatic diff as part of commit; unit tests not applicable.

Event Timeline

pulkit created this revision.Oct 12 2018, 9:58 PM
yuja added a subscriber: yuja.Oct 13 2018, 12:39 AM
  • a/contrib/perf.py

+++ b/contrib/perf.py
@@ -75,14 +75,16 @@

_byteskwargs = pycompat.byteskwargs  # since 4.1 (or fbc3f73dc802)
_sysstr = pycompat.sysstr         # since 4.0 (or 2219f4f82ede)
_xrange = pycompat.xrange         # since 4.8 (or 7eba8f83129b)

+ fsencode = pycompat.fsencode # since 3.9 (or f4a5e0e86a7e)

if pycompat.ispy3:
    _maxint = sys.maxsize  # per py3 docs for replacing maxint
else:
    _maxint = sys.maxint

except (ImportError, AttributeError):

import inspect
getargspec = inspect.getargspec
_byteskwargs = identity

+ fsencode = fsencode # no py3 support

Fixed invalid reference in flight.

def perfstartup(ui, repo, **opts):

opts = _byteskwargs(opts)
timer, fm = gettimer(ui, opts)
  • cmd = sys.argv[0]

+ cmd = fsencode(sys.argv[0])

Applying fsencode() on sys.argv is probably wrong on Windows, but it's perf.py,
I don't care.

Closed by commit rHGb456b2e0ad9f: py3: make test-contrib-perf.t work on python 3 (authored by pulkit). · Explain WhyOct 13 2018, 1:04 AM
This revision was automatically updated to reflect the committed changes.
pulkit added a comment.Oct 13 2018, 4:45 AM

def perfstartup(ui, repo, **opts):

opts = _byteskwargs(opts)
timer, fm = gettimer(ui, opts)
  • cmd = sys.argv[0]

+ cmd = fsencode(sys.argv[0])

Applying fsencode() on sys.argv is probably wrong on Windows, but it's perf.py,
I don't care.

What's the right way to do it on windows?

yuja added a comment.Oct 13 2018, 5:53 AM
>> - cmd = sys.argv[0] +    cmd = fsencode(sys.argv[0])
> 
> Applying fsencode() on sys.argv is probably wrong on Windows, but it's perf.py,
>  I don't care.
What's the right way to do it on windows?

I think encoding.strtolocal() is close, but I have no idea what's the best
way to get char *argv[] back from wchar_t *argv[].

mharbison72 added a subscriber: mharbison72.Oct 15 2018, 11:05 PM
In D5025#75556, @yuja wrote:

def perfstartup(ui, repo, **opts):

opts = _byteskwargs(opts)
timer, fm = gettimer(ui, opts)
  • cmd = sys.argv[0]

+ cmd = fsencode(sys.argv[0])

Applying fsencode() on sys.argv is probably wrong on Windows, but it's perf.py,
I don't care.

Sadly, it doesn't work on Windows. But since os.system() is called with an r-string, bytes wouldn't be right anyway. When I simply formatted in sys.argv[0], I got this:

--- c:/Users/Matt/projects/hg_py3/tests/test-contrib-perf.t
+++ c:/Users/Matt/projects/hg_py3/tests/test-contrib-perf.t.err
@@ -184,6 +184,7 @@
   $ hg perfrevrange
   $ hg perfrevset 'all()'
   $ hg perfstartup
+  (null): can't open file '<unprintable file name>': [Errno 2] $ENOENT$
   $ hg perfstatus
   $ hg perftags
   $ hg perftemplating

This will be good to figure out because there are ~400 other instances of this error, mostly around launching hooks. What I've found to work (both here and for the hooks) is to make sure that python3 is launched:

os.system(r"py -3 %s version -q > NUL" % sys.argv[0])

It's perfectly capable of running r'notepad', so this must be an hg.exe/python.exe issue. This probably gets messy (at least with the hooks) because *.py files are associated with python2, and python2 is in $PATH (but not python3- both of which are named python.exe). py -3 will also pay attention to the shbang line, all of which are coded to python2. The only other thing of note that I see is that wrapper.exe implements main() and not wmain(), but I can't believe that py3 is incapable of launching things built without Unicode support.

mharbison72 added a comment.Oct 16 2018, 12:10 AM
In D5025#76443, @mharbison72 wrote:
In D5025#75556, @yuja wrote:

def perfstartup(ui, repo, **opts):

opts = _byteskwargs(opts)
timer, fm = gettimer(ui, opts)
  • cmd = sys.argv[0]

+ cmd = fsencode(sys.argv[0])

Applying fsencode() on sys.argv is probably wrong on Windows, but it's perf.py,
I don't care.

Sadly, it doesn't work on Windows. But since os.system() is called with an r-string, bytes wouldn't be right anyway. When I simply formatted in sys.argv[0], I got this:

--- c:/Users/Matt/projects/hg_py3/tests/test-contrib-perf.t
+++ c:/Users/Matt/projects/hg_py3/tests/test-contrib-perf.t.err
@@ -184,6 +184,7 @@
   $ hg perfrevrange
   $ hg perfrevset 'all()'
   $ hg perfstartup
+  (null): can't open file '<unprintable file name>': [Errno 2] $ENOENT$
   $ hg perfstatus
   $ hg perftags
   $ hg perftemplating

This will be good to figure out because there are ~400 other instances of this error, mostly around launching hooks.

Nevermind. Looking closer at exewrapper.c, the signature of Py_Main() changed. Is there any reason not to convert that file to the usual TCHAR type, and #define UNICODE when building for py3?

Revision Contents
Changeset List

PathPackages
Mcontrib/perf.py (15 lines)
Mcontrib/python3-whitelist (1 line)

Diff 11980

contrib/perf.py

return a return a
try: try:
from mercurial import pycompat from mercurial import pycompat
getargspec = pycompat.getargspec # added to module after 4.5 getargspec = pycompat.getargspec # added to module after 4.5
_byteskwargs = pycompat.byteskwargs # since 4.1 (or fbc3f73dc802) _byteskwargs = pycompat.byteskwargs # since 4.1 (or fbc3f73dc802)
_sysstr = pycompat.sysstr # since 4.0 (or 2219f4f82ede) _sysstr = pycompat.sysstr # since 4.0 (or 2219f4f82ede)
_xrange = pycompat.xrange # since 4.8 (or 7eba8f83129b) _xrange = pycompat.xrange # since 4.8 (or 7eba8f83129b)
fsencode = pycompat.fsencode # since 3.9 (or f4a5e0e86a7e)
if pycompat.ispy3: if pycompat.ispy3:
_maxint = sys.maxsize # per py3 docs for replacing maxint _maxint = sys.maxsize # per py3 docs for replacing maxint
else: else:
_maxint = sys.maxint _maxint = sys.maxint
except (ImportError, AttributeError): except (ImportError, AttributeError):
import inspect import inspect
getargspec = inspect.getargspec getargspec = inspect.getargspec
_byteskwargs = identity _byteskwargs = identity
fsencode = identity # no py3 support
_maxint = sys.maxint # no py3 support _maxint = sys.maxint # no py3 support
_sysstr = lambda x: x # no py3 support _sysstr = lambda x: x # no py3 support
_xrange = xrange _xrange = xrange
try: try:
# 4.7+ # 4.7+
queue = pycompat.queue.Queue queue = pycompat.queue.Queue
except (AttributeError, ImportError): except (AttributeError, ImportError):
cl.rev(n) cl.rev(n)
timer(d) timer(d)
fm.end() fm.end()
@command(b'perfstartup', formatteropts) @command(b'perfstartup', formatteropts)
def perfstartup(ui, repo, **opts): def perfstartup(ui, repo, **opts):
opts = _byteskwargs(opts) opts = _byteskwargs(opts)
timer, fm = gettimer(ui, opts) timer, fm = gettimer(ui, opts)
cmd = sys.argv[0] cmd = fsencode(sys.argv[0])
def d(): def d():
if os.name != r'nt': if os.name != r'nt':
os.system(b"HGRCPATH= %s version -q > /dev/null" % cmd) os.system(b"HGRCPATH= %s version -q > /dev/null" % cmd)
else: else:
os.environ[r'HGRCPATH'] = r' ' os.environ[r'HGRCPATH'] = r' '
os.system(r"%s version -q > NUL" % cmd) os.system(r"%s version -q > NUL" % cmd)
timer(d) timer(d)
fm.end() fm.end()
fm.end() fm.end()
@command(b'perfdiffwd', formatteropts) @command(b'perfdiffwd', formatteropts)
def perfdiffwd(ui, repo, **opts): def perfdiffwd(ui, repo, **opts):
"""Profile diff of working directory changes""" """Profile diff of working directory changes"""
opts = _byteskwargs(opts) opts = _byteskwargs(opts)
timer, fm = gettimer(ui, opts) timer, fm = gettimer(ui, opts)
options = { options = {
b'w': b'ignore_all_space', 'w': 'ignore_all_space',
b'b': b'ignore_space_change', 'b': 'ignore_space_change',
b'B': b'ignore_blank_lines', 'B': 'ignore_blank_lines',
} }
for diffopt in (b'', b'w', b'b', b'B', b'wB'): for diffopt in ('', 'w', 'b', 'B', 'wB'):
opts = dict((options[c], b'1') for c in diffopt) opts = dict((options[c], b'1') for c in diffopt)
def d(): def d():
ui.pushbuffer() ui.pushbuffer()
commands.diff(ui, repo, **opts) commands.diff(ui, repo, **opts)
ui.popbuffer() ui.popbuffer()
diffopt = diffopt.encode('ascii')
title = b'diffopts: %s' % (diffopt and (b'-' + diffopt) or b'none') title = b'diffopts: %s' % (diffopt and (b'-' + diffopt) or b'none')
timer(d, title) timer(d, title)
fm.end() fm.end()
@command(b'perfrevlogindex', revlogopts + formatteropts, @command(b'perfrevlogindex', revlogopts + formatteropts,
b'-c|-m|FILE') b'-c|-m|FILE')
def perfrevlogindex(ui, repo, file_=None, **opts): def perfrevlogindex(ui, repo, file_=None, **opts):
"""Benchmark operations against a revlog index. """Benchmark operations against a revlog index.
if not cache: if not cache:
r.clearcaches() r.clearcaches()
r.revision(node) r.revision(node)
chain = r._deltachain(rev)[0] chain = r._deltachain(rev)[0]
data = segmentforrevs(chain[0], chain[-1])[1] data = segmentforrevs(chain[0], chain[-1])[1]
rawchunks = getrawchunks(data, chain) rawchunks = getrawchunks(data, chain)
bins = r._chunks(chain) bins = r._chunks(chain)
text = str(bins[0]) text = bytes(bins[0])
bins = bins[1:] bins = bins[1:]
text = mdiff.patches(text, bins) text = mdiff.patches(text, bins)
benches = [ benches = [
(lambda: dorevision(), b'full'), (lambda: dorevision(), b'full'),
(lambda: dodeltachain(rev), b'deltachain'), (lambda: dodeltachain(rev), b'deltachain'),
(lambda: doread(chain), b'read'), (lambda: doread(chain), b'read'),
(lambda: dorawchunks(data, chain), b'rawchunks'), (lambda: dorawchunks(data, chain), b'rawchunks'),

contrib/python3-whitelist

test-config-env.py test-config-env.py
test-config.t test-config.t
test-conflict.t test-conflict.t
test-confused-revert.t test-confused-revert.t
test-context.py test-context.py
test-contrib-check-code.t test-contrib-check-code.t
test-contrib-check-commit.t test-contrib-check-commit.t
test-contrib-dumprevlog.t test-contrib-dumprevlog.t
test-contrib-perf.t
test-contrib-testparseutil.t test-contrib-testparseutil.t
test-convert-authormap.t test-convert-authormap.t
test-convert-clonebranches.t test-convert-clonebranches.t
test-convert-cvs-branch.t test-convert-cvs-branch.t
test-convert-cvs-detectmerge.t test-convert-cvs-detectmerge.t
test-convert-cvs-synthetic.t test-convert-cvs-synthetic.t
test-convert-cvs.t test-convert-cvs.t
test-convert-cvsnt-mergepoints.t test-convert-cvsnt-mergepoints.t