This is an archive of the discontinued Mercurial Phabricator instance.

Differential D3419

interfaceutil: module to stub out zope.interface
ClosedPublic

Authored by indygreg on Apr 22 2018, 3:59 PM.

Details

Reviewers
durin42
Group Reviewers
hg-reviewers
Commits
rHG856f381ad74b: interfaceutil: module to stub out zope.interface
Summary

The startup time of hg increased during the 4.6 development cycle. A
cause of that was importing more modules and doing more work at module
import time.

The import of zope.interface and the declaring of various interfaces
is partially responsible for the startup time regression.

Our current usage of zope.interface doesn't do much at run time: we are
merely declaring interfaces and stating that certain types implement
various interfaces. Core Mercurial is not (yet) using of any of
zope.interface features that actually require that interface plumbing be
defined. The only place we actually need the interface metadata is in
test-check-interfaces.py.

This commit establishes a new interfaceutil module. It exposes the subset
of the zope.interface API that we currently use. By default, the APIs
no-op. But if an environment variable is set, we export the real
zope.interface APIs.

Existing importers of zope.interface have been converted to use the new
module. test-check-interfaces.py has been updated to define the
environment variable so the real zope.interface is used.

The net effect of this change is we stop importing 9 zope.interface.*
modules and we no longer perform interface bookkeeping when registering
interfaces.

On my i7-6700K on Linux, a shell loop that runs hg log -r . 300 times
on a repo with 1 commit shows a significant CPU time improvement
(average of 4 runs):

4.5: 14.814s
before: 19.028s
after: 16.945s

And with run-tests.py -j10 (single run):

4.5: ~3100s (~51.7m)
before: ~4450s (~74.2m)
after: ~3980s (~66.3m)

So this claws back about half of the regressions in 4.6.

Diff Detail

Repository
rHG Mercurial
Lint
Automatic diff as part of commit; lint not applicable.
Unit
Automatic diff as part of commit; unit tests not applicable.

Event Timeline

indygreg created this revision.Apr 22 2018, 3:59 PM
durin42 accepted this revision.Apr 23 2018, 4:27 PM
This revision is now accepted and ready to land.Apr 23 2018, 4:27 PM
Closed by commit rHG856f381ad74b: interfaceutil: module to stub out zope.interface (authored by indygreg). · Explain WhyApr 23 2018, 4:28 PM
This revision was automatically updated to reflect the committed changes.
lothiraldan added a subscriber: lothiraldan.Apr 30 2018, 11:52 AM

We also detected the regression in 4.6rc0 (http://perf.octobus.net/#basic_commands.DiscoveryTimeSuite.time_debugdiscovery?branch=stable) and can confirm locally that half the regression is fixed on stable (e82b137a8b4e).

Do you have any idea where the second half of regressions come from?

indygreg added a comment.Apr 30 2018, 12:10 PM

I'm not sure where the 2nd half of the start-up time regressions came from. I had a hacky test where I printed sys.modules keys after command execution. It resulted in the following:

46,68d45
< email
< email.Charset
< email.Encoders
< email.Errors
< email.FeedParser
< email.Generator
< email.Header
< email.Iterators
< email.MIMEAudio
< email.MIMEBase
< email.MIMEImage
< email.MIMEMessage
< email.MIMEMultipart
< email.MIMENonMultipart
< email.MIMEText
< email.Message
< email.Parser
< email.Utils
< email.base64MIME
< email.email
< email.mime
< email.quopriMIME
< email.sys
89a67
> inspect
101d78
< mercurial.cext.diffhelpers
124a102
> mercurial.logcmdutil
127a106
> mercurial.narrowspec
152a132
> mercurial.templatefuncs
153a134,135
> mercurial.templater
> mercurial.templateutil
160a143,152
> mercurial.thirdparty.concurrent
> mercurial.thirdparty.zope
> mercurial.thirdparty.zope.interface
> mercurial.thirdparty.zope.interface._compat
> mercurial.thirdparty.zope.interface._zope_interface_coptimizations
> mercurial.thirdparty.zope.interface.declarations
> mercurial.thirdparty.zope.interface.exceptions
> mercurial.thirdparty.zope.interface.interface
> mercurial.thirdparty.zope.interface.interfaces
> mercurial.thirdparty.zope.interface.ro
164a157,160
> mercurial.utils
> mercurial.utils.dateutil
> mercurial.utils.procutil
> mercurial.utils.stringutil

I'm not convinced the total number of modules is responsible: I think it has more to do with the volume of code being imported. But I could be wrong. These things are a bit challenging to profile.

And for reference, all modules that were imported in 4.6rc after running (I think it was hg log -r .):

Queue
UserDict
__builtin__
__future__
__main__
_abcoll
_codecs
_collections
_curses
_functools
_hashlib
_heapq
_locale
_socket
_sre
_ssl
_struct
_sysconfigdata
_warnings
_weakref
_weakrefset
abc
bdb
binascii
cStringIO
cmd
codecs
collections
contextlib
copy
copy_reg
curses
curses._curses
curses.curses
curses.wrapper
datetime
encodings
encodings.__builtin__
encodings.aliases
encodings.ascii
encodings.codecs
encodings.encodings
encodings.utf_8
errno
exceptions
fcntl
functools
gc
genericpath
getopt
gettext
grp
hashlib
heapq
hgdemandimport
hgdemandimport.demandimportpy2
imp
inspect
itertools
keyword
linecache
locale
mercurial
mercurial.__modulepolicy__
mercurial.bookmarks
mercurial.branchmap
mercurial.cext
mercurial.cext.base85
mercurial.cext.bdiff
mercurial.cext.mpatch
mercurial.cext.osutil
mercurial.cext.parsers
mercurial.changelog
mercurial.cmdutil
mercurial.color
mercurial.commands
mercurial.config
mercurial.configitems
mercurial.context
mercurial.debugcommands
mercurial.dirstate
mercurial.dispatch
mercurial.encoding
mercurial.error
mercurial.extensions
mercurial.fancyopts
mercurial.formatter
mercurial.hg
mercurial.hook
mercurial.i18n
mercurial.localrepo
mercurial.lock
mercurial.logcmdutil
mercurial.match
mercurial.mdiff
mercurial.namespaces
mercurial.narrowspec
mercurial.node
mercurial.obsolete
mercurial.obsutil
mercurial.parser
mercurial.patch
mercurial.pathutil
mercurial.phases
mercurial.policy
mercurial.posix
mercurial.profiling
mercurial.pure
mercurial.pycompat
mercurial.rcutil
mercurial.registrar
mercurial.repository
mercurial.repoview
mercurial.revlog
mercurial.revset
mercurial.revsetlang
mercurial.scmposix
mercurial.scmutil
mercurial.smartset
mercurial.store
mercurial.tags
mercurial.templatefilters
mercurial.templatefuncs
mercurial.templatekw
mercurial.templater
mercurial.templateutil
mercurial.thirdparty
mercurial.thirdparty.attr
mercurial.thirdparty.attr._compat
mercurial.thirdparty.attr._config
mercurial.thirdparty.attr._funcs
mercurial.thirdparty.attr._make
mercurial.thirdparty.attr.exceptions
mercurial.thirdparty.concurrent
mercurial.thirdparty.zope
mercurial.thirdparty.zope.interface
mercurial.thirdparty.zope.interface._compat
mercurial.thirdparty.zope.interface._zope_interface_coptimizations
mercurial.thirdparty.zope.interface.declarations
mercurial.thirdparty.zope.interface.exceptions
mercurial.thirdparty.zope.interface.interface
mercurial.thirdparty.zope.interface.interfaces
mercurial.thirdparty.zope.interface.ro
mercurial.txnutil
mercurial.ui
mercurial.urllibcompat
mercurial.util
mercurial.utils
mercurial.utils.dateutil
mercurial.utils.procutil
mercurial.utils.stringutil
mercurial.vfs
mercurial.zstd
operator
os
os.path
pdb
platform
posix
posixpath
pwd
re
repr
shlex
signal
site
socket
sre_compile
sre_constants
sre_parse
stat
string
strop
struct
sys
sysconfig
thread
time
traceback
types
urllib
urlparse
warnings
weakref
zipimport
zlib

I didn't record whether these are actually imported or our module proxy stub used by the custom module importer. Sorry.

I do have some patches to improve efficiency of the importer. I don't feel comfortable sending them to stable unless we commit to doing another 4.6 RC since they are API breaking. And since release is delayed by a week, maybe that is justified. @durin42?

durin42 added a comment.Apr 30 2018, 12:39 PM
In D3419#54662, @indygreg wrote:

I do have some patches to improve efficiency of the importer. I don't feel comfortable sending them to stable unless we commit to doing another 4.6 RC since they are API breaking. And since release is delayed by a week, maybe that is justified. @durin42?

I'd rather we reopen default and start doing normal development work there, and keep the 4.6 release as just rc1+security patches.

lothiraldan added a comment.May 2 2018, 9:53 AM

We saw a ~15ms regression hit on our performance benchmark for the incoming test between 4.5.3 and 4.6rc0. This regression is ~30ms when the test is over ssh.

We are still running tests against 4.6rc1 because it was available late on https://www.mercurial-scm.org/repo/hg/

Revision Contents
Changeset List

Diff 8423

mercurial/filelog.py

# filelog.py - file history class for mercurial # filelog.py - file history class for mercurial
# #
# Copyright 2005-2007 Matt Mackall <mpm@selenic.com> # Copyright 2005-2007 Matt Mackall <mpm@selenic.com>
# #
# This software may be used and distributed according to the terms of the # This software may be used and distributed according to the terms of the
# GNU General Public License version 2 or any later version. # GNU General Public License version 2 or any later version.
from __future__ import absolute_import from __future__ import absolute_import
from .thirdparty.zope import (
interface as zi,
)
from . import ( from . import (
error, error,
repository, repository,
revlog, revlog,
) )
from .utils import (
interfaceutil,
)
@zi.implementer(repository.ifilestorage) @interfaceutil.implementer(repository.ifilestorage)
class filelog(object): class filelog(object):
def __init__(self, opener, path): def __init__(self, opener, path):
self._revlog = revlog.revlog(opener, self._revlog = revlog.revlog(opener,
'/'.join(('data', path + '.i')), '/'.join(('data', path + '.i')),
censorable=True) censorable=True)
# full name of the user visible file, relative to the repository root # full name of the user visible file, relative to the repository root
self.filename = path self.filename = path
self.index = self._revlog.index self.index = self._revlog.index

mercurial/httppeer.py

import struct import struct
import tempfile import tempfile
import weakref import weakref
from .i18n import _ from .i18n import _
from .thirdparty import ( from .thirdparty import (
cbor, cbor,
) )
from .thirdparty.zope import (
interface as zi,
)
from . import ( from . import (
bundle2, bundle2,
error, error,
httpconnection, httpconnection,
pycompat, pycompat,
repository, repository,
statichttprepo, statichttprepo,
url as urlmod, url as urlmod,
util, util,
wireprotoframing, wireprotoframing,
wireprototypes, wireprototypes,
wireprotov1peer, wireprotov1peer,
wireprotov2peer, wireprotov2peer,
wireprotov2server, wireprotov2server,
) )
from .utils import (
interfaceutil,
)
httplib = util.httplib httplib = util.httplib
urlerr = util.urlerr urlerr = util.urlerr
urlreq = util.urlreq urlreq = util.urlreq
def encodevalueinheaders(value, header, limit): def encodevalueinheaders(value, header, limit):
"""Encode a string value into multiple HTTP headers. """Encode a string value into multiple HTTP headers.
return pycompat.futures.Future.result(self, timeout) return pycompat.futures.Future.result(self, timeout)
self._peerexecutor.sendcommands() self._peerexecutor.sendcommands()
# sendcommands() will restore the original __class__ and self.result # sendcommands() will restore the original __class__ and self.result
# will resolve to Future.result. # will resolve to Future.result.
return self.result(timeout) return self.result(timeout)
@zi.implementer(repository.ipeercommandexecutor) @interfaceutil.implementer(repository.ipeercommandexecutor)
class httpv2executor(object): class httpv2executor(object):
def __init__(self, ui, opener, requestbuilder, apiurl, descriptor): def __init__(self, ui, opener, requestbuilder, apiurl, descriptor):
self._ui = ui self._ui = ui
self._opener = opener self._opener = opener
self._requestbuilder = requestbuilder self._requestbuilder = requestbuilder
self._apiurl = apiurl self._apiurl = apiurl
self._descriptor = descriptor self._descriptor = descriptor
self._sent = False self._sent = False
def _handleresponse(self, handler, resp): def _handleresponse(self, handler, resp):
# Called in a thread to read the response. # Called in a thread to read the response.
while handler.readframe(resp): while handler.readframe(resp):
pass pass
# TODO implement interface for version 2 peers # TODO implement interface for version 2 peers
@zi.implementer(repository.ipeerconnection, repository.ipeercapabilities, @interfaceutil.implementer(repository.ipeerconnection,
repository.ipeercapabilities,
repository.ipeerrequests) repository.ipeerrequests)
class httpv2peer(object): class httpv2peer(object):
def __init__(self, ui, repourl, apipath, opener, requestbuilder, def __init__(self, ui, repourl, apipath, opener, requestbuilder,
apidescriptor): apidescriptor):
self.ui = ui self.ui = ui
if repourl.endswith('/'): if repourl.endswith('/'):
repourl = repourl[:-1] repourl = repourl[:-1]

mercurial/localrepo.py

import weakref import weakref
from .i18n import _ from .i18n import _
from .node import ( from .node import (
hex, hex,
nullid, nullid,
short, short,
) )
from .thirdparty.zope import (
interface as zi,
)
from . import ( from . import (
bookmarks, bookmarks,
branchmap, branchmap,
bundle2, bundle2,
changegroup, changegroup,
changelog, changelog,
color, color,
context, context,
subrepoutil, subrepoutil,
tags as tagsmod, tags as tagsmod,
transaction, transaction,
txnutil, txnutil,
util, util,
vfs as vfsmod, vfs as vfsmod,
) )
from .utils import ( from .utils import (
interfaceutil,
procutil, procutil,
stringutil, stringutil,
) )
release = lockmod.release release = lockmod.release
urlerr = util.urlerr urlerr = util.urlerr
urlreq = util.urlreq urlreq = util.urlreq
def wrapper(repo, *args, **kwargs): def wrapper(repo, *args, **kwargs):
return orig(repo.unfiltered(), *args, **kwargs) return orig(repo.unfiltered(), *args, **kwargs)
return wrapper return wrapper
moderncaps = {'lookup', 'branchmap', 'pushkey', 'known', 'getbundle', moderncaps = {'lookup', 'branchmap', 'pushkey', 'known', 'getbundle',
'unbundle'} 'unbundle'}
legacycaps = moderncaps.union({'changegroupsubset'}) legacycaps = moderncaps.union({'changegroupsubset'})
@zi.implementer(repository.ipeercommandexecutor) @interfaceutil.implementer(repository.ipeercommandexecutor)
class localcommandexecutor(object): class localcommandexecutor(object):
def __init__(self, peer): def __init__(self, peer):
self._peer = peer self._peer = peer
self._sent = False self._sent = False
self._closed = False self._closed = False
def __enter__(self): def __enter__(self):
return self return self
return f return f
def sendcommands(self): def sendcommands(self):
self._sent = True self._sent = True
def close(self): def close(self):
self._closed = True self._closed = True
@zi.implementer(repository.ipeercommands) @interfaceutil.implementer(repository.ipeercommands)
class localpeer(repository.peer): class localpeer(repository.peer):
'''peer for a local repo; reflects only the most recent API''' '''peer for a local repo; reflects only the most recent API'''
def __init__(self, repo, caps=None): def __init__(self, repo, caps=None):
super(localpeer, self).__init__() super(localpeer, self).__init__()
if caps is None: if caps is None:
caps = moderncaps.copy() caps = moderncaps.copy()
# Begin of peer interface. # Begin of peer interface.
def commandexecutor(self): def commandexecutor(self):
return localcommandexecutor(self) return localcommandexecutor(self)
# End of peer interface. # End of peer interface.
@zi.implementer(repository.ipeerlegacycommands) @interfaceutil.implementer(repository.ipeerlegacycommands)
class locallegacypeer(localpeer): class locallegacypeer(localpeer):
'''peer extension which implements legacy methods too; used for tests with '''peer extension which implements legacy methods too; used for tests with
restricted capabilities''' restricted capabilities'''
def __init__(self, repo): def __init__(self, repo):
super(locallegacypeer, self).__init__(repo, caps=legacycaps) super(locallegacypeer, self).__init__(repo, caps=legacycaps)
# Begin of baselegacywirecommands interface. # Begin of baselegacywirecommands interface.
# the ability to load repositories with custom requirements. Only # the ability to load repositories with custom requirements. Only
# functions defined in loaded extensions are called. # functions defined in loaded extensions are called.
# #
# The function receives a set of requirement strings that the repository # The function receives a set of requirement strings that the repository
# is capable of opening. Functions will typically add elements to the # is capable of opening. Functions will typically add elements to the
# set to reflect that the extension knows how to handle that requirements. # set to reflect that the extension knows how to handle that requirements.
featuresetupfuncs = set() featuresetupfuncs = set()
@zi.implementer(repository.completelocalrepository) @interfaceutil.implementer(repository.completelocalrepository)
class localrepository(object): class localrepository(object):
# obsolete experimental requirements: # obsolete experimental requirements:
# - manifestv2: An experimental new manifest format that allowed # - manifestv2: An experimental new manifest format that allowed
# for stem compression of long paths. Experiment ended up not # for stem compression of long paths. Experiment ended up not
# being successful (repository sizes went up due to worse delta # being successful (repository sizes went up due to worse delta
# chains), and the code was deleted in 4.6. # chains), and the code was deleted in 4.6.
supportedformats = { supportedformats = {

mercurial/repository.py

# repository.py - Interfaces and base classes for repositories and peers. # repository.py - Interfaces and base classes for repositories and peers.
# #
# Copyright 2017 Gregory Szorc <gregory.szorc@gmail.com> # Copyright 2017 Gregory Szorc <gregory.szorc@gmail.com>
# #
# This software may be used and distributed according to the terms of the # This software may be used and distributed according to the terms of the
# GNU General Public License version 2 or any later version. # GNU General Public License version 2 or any later version.
from __future__ import absolute_import from __future__ import absolute_import
from .i18n import _ from .i18n import _
from .thirdparty.zope import (
interface as zi,
)
from . import ( from . import (
error, error,
) )
from .utils import (
interfaceutil,
)
class ipeerconnection(zi.Interface): class ipeerconnection(interfaceutil.Interface):
"""Represents a "connection" to a repository. """Represents a "connection" to a repository.
This is the base interface for representing a connection to a repository. This is the base interface for representing a connection to a repository.
It holds basic properties and methods applicable to all peer types. It holds basic properties and methods applicable to all peer types.
This is not a complete interface definition and should not be used This is not a complete interface definition and should not be used
outside of this module. outside of this module.
""" """
ui = zi.Attribute("""ui.ui instance""") ui = interfaceutil.Attribute("""ui.ui instance""")
def url(): def url():
"""Returns a URL string representing this peer. """Returns a URL string representing this peer.
Currently, implementations expose the raw URL used to construct the Currently, implementations expose the raw URL used to construct the
instance. It may contain credentials as part of the URL. The instance. It may contain credentials as part of the URL. The
expectations of the value aren't well-defined and this could lead to expectations of the value aren't well-defined and this could lead to
data leakage. data leakage.
def close(): def close():
"""Close the connection to this peer. """Close the connection to this peer.
This is called when the peer will no longer be used. Resources This is called when the peer will no longer be used. Resources
associated with the peer should be cleaned up. associated with the peer should be cleaned up.
""" """
class ipeercapabilities(zi.Interface): class ipeercapabilities(interfaceutil.Interface):
"""Peer sub-interface related to capabilities.""" """Peer sub-interface related to capabilities."""
def capable(name): def capable(name):
"""Determine support for a named capability. """Determine support for a named capability.
Returns ``False`` if capability not supported. Returns ``False`` if capability not supported.
Returns ``True`` if boolean capability is supported. Returns a string Returns ``True`` if boolean capability is supported. Returns a string
if capability support is non-boolean. if capability support is non-boolean.
Capability strings may or may not map to wire protocol capabilities. Capability strings may or may not map to wire protocol capabilities.
""" """
def requirecap(name, purpose): def requirecap(name, purpose):
"""Require a capability to be present. """Require a capability to be present.
Raises a ``CapabilityError`` if the capability isn't present. Raises a ``CapabilityError`` if the capability isn't present.
""" """
class ipeercommands(zi.Interface): class ipeercommands(interfaceutil.Interface):
"""Client-side interface for communicating over the wire protocol. """Client-side interface for communicating over the wire protocol.
This interface is used as a gateway to the Mercurial wire protocol. This interface is used as a gateway to the Mercurial wire protocol.
methods commonly call wire protocol commands of the same name. methods commonly call wire protocol commands of the same name.
""" """
def branchmap(): def branchmap():
"""Obtain heads in named branches. """Obtain heads in named branches.
def unbundle(bundle, heads, url): def unbundle(bundle, heads, url):
"""Transfer repository data to the peer. """Transfer repository data to the peer.
This is how the bulk of data during a push is transferred. This is how the bulk of data during a push is transferred.
Returns the integer number of heads added to the peer. Returns the integer number of heads added to the peer.
""" """
class ipeerlegacycommands(zi.Interface): class ipeerlegacycommands(interfaceutil.Interface):
"""Interface for implementing support for legacy wire protocol commands. """Interface for implementing support for legacy wire protocol commands.
Wire protocol commands transition to legacy status when they are no longer Wire protocol commands transition to legacy status when they are no longer
used by modern clients. To facilitate identifying which commands are used by modern clients. To facilitate identifying which commands are
legacy, the interfaces are split. legacy, the interfaces are split.
""" """
def between(pairs): def between(pairs):
""" """
def changegroup(nodes, source): def changegroup(nodes, source):
"""Obtain a changegroup with data for descendants of specified nodes.""" """Obtain a changegroup with data for descendants of specified nodes."""
def changegroupsubset(bases, heads, source): def changegroupsubset(bases, heads, source):
pass pass
class ipeercommandexecutor(zi.Interface): class ipeercommandexecutor(interfaceutil.Interface):
"""Represents a mechanism to execute remote commands. """Represents a mechanism to execute remote commands.
This is the primary interface for requesting that wire protocol commands This is the primary interface for requesting that wire protocol commands
be executed. Instances of this interface are active in a context manager be executed. Instances of this interface are active in a context manager
and have a well-defined lifetime. When the context manager exits, all and have a well-defined lifetime. When the context manager exits, all
outstanding requests are waited on. outstanding requests are waited on.
""" """
to finish. to finish.
When used as a context manager, this method is called when exiting the When used as a context manager, this method is called when exiting the
context manager. context manager.
This method may call ``sendcommands()`` if there are buffered commands. This method may call ``sendcommands()`` if there are buffered commands.
""" """
class ipeerrequests(zi.Interface): class ipeerrequests(interfaceutil.Interface):
"""Interface for executing commands on a peer.""" """Interface for executing commands on a peer."""
def commandexecutor(): def commandexecutor():
"""A context manager that resolves to an ipeercommandexecutor. """A context manager that resolves to an ipeercommandexecutor.
The object this resolves to can be used to issue command requests The object this resolves to can be used to issue command requests
to the peer. to the peer.
""" """
class ipeerbase(ipeerconnection, ipeercapabilities, ipeerrequests): class ipeerbase(ipeerconnection, ipeercapabilities, ipeerrequests):
"""Unified interface for peer repositories. """Unified interface for peer repositories.
All peer instances must conform to this interface. All peer instances must conform to this interface.
""" """
@zi.implementer(ipeerbase) @interfaceutil.implementer(ipeerbase)
class peer(object): class peer(object):
"""Base class for peer repositories.""" """Base class for peer repositories."""
def capable(self, name): def capable(self, name):
caps = self.capabilities() caps = self.capabilities()
if name in caps: if name in caps:
return True return True
name = '%s=' % name name = '%s=' % name
for cap in caps: for cap in caps:
if cap.startswith(name): if cap.startswith(name):
return cap[len(name):] return cap[len(name):]
return False return False
def requirecap(self, name, purpose): def requirecap(self, name, purpose):
if self.capable(name): if self.capable(name):
return return
raise error.CapabilityError( raise error.CapabilityError(
_('cannot %s; remote repository does not support the %r ' _('cannot %s; remote repository does not support the %r '
'capability') % (purpose, name)) 'capability') % (purpose, name))
class ifilerevisionssequence(zi.Interface): class ifilerevisionssequence(interfaceutil.Interface):
"""Contains index data for all revisions of a file. """Contains index data for all revisions of a file.
Types implementing this behave like lists of tuples. The index Types implementing this behave like lists of tuples. The index
in the list corresponds to the revision number. The values contain in the list corresponds to the revision number. The values contain
index metadata. index metadata.
The *null* revision (revision number -1) is always the last item The *null* revision (revision number -1) is always the last item
in the index. in the index.
""" """
def __contains__(rev): def __contains__(rev):
"""Whether a revision number exists.""" """Whether a revision number exists."""
def insert(self, i, entry): def insert(self, i, entry):
"""Add an item to the index at specific revision.""" """Add an item to the index at specific revision."""
class ifileindex(zi.Interface): class ifileindex(interfaceutil.Interface):
"""Storage interface for index data of a single file. """Storage interface for index data of a single file.
File storage data is divided into index metadata and data storage. File storage data is divided into index metadata and data storage.
This interface defines the index portion of the interface. This interface defines the index portion of the interface.
The index logically consists of: The index logically consists of:
* A mapping between revision numbers and nodes. * A mapping between revision numbers and nodes.
* DAG data (storing and querying the relationship between nodes). * DAG data (storing and querying the relationship between nodes).
* Metadata to facilitate storage. * Metadata to facilitate storage.
""" """
index = zi.Attribute( index = interfaceutil.Attribute(
"""An ``ifilerevisionssequence`` instance.""") """An ``ifilerevisionssequence`` instance.""")
def __len__(): def __len__():
"""Obtain the number of revisions stored for this file.""" """Obtain the number of revisions stored for this file."""
def __iter__(): def __iter__():
"""Iterate over revision numbers for this file.""" """Iterate over revision numbers for this file."""
""" """
def deltaparent(rev): def deltaparent(rev):
""""Return the revision that is a suitable parent to delta against.""" """"Return the revision that is a suitable parent to delta against."""
def candelta(baserev, rev): def candelta(baserev, rev):
""""Whether a delta can be generated between two revisions.""" """"Whether a delta can be generated between two revisions."""
class ifiledata(zi.Interface): class ifiledata(interfaceutil.Interface):
"""Storage interface for data storage of a specific file. """Storage interface for data storage of a specific file.
This complements ``ifileindex`` and provides an interface for accessing This complements ``ifileindex`` and provides an interface for accessing
data for a tracked file. data for a tracked file.
""" """
def rawsize(rev): def rawsize(rev):
"""The size of the fulltext data for a revision as stored.""" """The size of the fulltext data for a revision as stored."""
"""Obtain a delta between two revision numbers. """Obtain a delta between two revision numbers.
Operates on raw data in the store (``revision(node, raw=True)``). Operates on raw data in the store (``revision(node, raw=True)``).
The returned data is the result of ``bdiff.bdiff`` on the raw The returned data is the result of ``bdiff.bdiff`` on the raw
revision data. revision data.
""" """
class ifilemutation(zi.Interface): class ifilemutation(interfaceutil.Interface):
"""Storage interface for mutation events of a tracked file.""" """Storage interface for mutation events of a tracked file."""
def add(filedata, meta, transaction, linkrev, p1, p2): def add(filedata, meta, transaction, linkrev, p1, p2):
"""Add a new revision to the store. """Add a new revision to the store.
Takes file data, dictionary of metadata, a transaction, linkrev, Takes file data, dictionary of metadata, a transaction, linkrev,
and parent nodes. and parent nodes.
TODO this is highly revlog centric and should be abstracted into a TODO this is highly revlog centric and should be abstracted into a
higher-level deletion API. higher-level deletion API.
""" """
class ifilestorage(ifileindex, ifiledata, ifilemutation): class ifilestorage(ifileindex, ifiledata, ifilemutation):
"""Complete storage interface for a single tracked file.""" """Complete storage interface for a single tracked file."""
version = zi.Attribute( version = interfaceutil.Attribute(
"""Version number of storage. """Version number of storage.
TODO this feels revlog centric and could likely be removed. TODO this feels revlog centric and could likely be removed.
""") """)
storedeltachains = zi.Attribute( storedeltachains = interfaceutil.Attribute(
"""Whether the store stores deltas. """Whether the store stores deltas.
TODO deltachains are revlog centric. This can probably removed TODO deltachains are revlog centric. This can probably removed
once there are better abstractions for obtaining/writing once there are better abstractions for obtaining/writing
data. data.
""") """)
_generaldelta = zi.Attribute( _generaldelta = interfaceutil.Attribute(
"""Whether deltas can be against any parent revision. """Whether deltas can be against any parent revision.
TODO this is used by changegroup code and it could probably be TODO this is used by changegroup code and it could probably be
folded into another API. folded into another API.
""") """)
def files(): def files():
"""Obtain paths that are backing storage for this file. """Obtain paths that are backing storage for this file.
TODO this is used heavily by verify code and there should probably TODO this is used heavily by verify code and there should probably
be a better API for that. be a better API for that.
""" """
def checksize(): def checksize():
"""Obtain the expected sizes of backing files. """Obtain the expected sizes of backing files.
TODO this is used by verify and it should not be part of the interface. TODO this is used by verify and it should not be part of the interface.
""" """
class completelocalrepository(zi.Interface): class completelocalrepository(interfaceutil.Interface):
"""Monolithic interface for local repositories. """Monolithic interface for local repositories.
This currently captures the reality of things - not how things should be. This currently captures the reality of things - not how things should be.
""" """
supportedformats = zi.Attribute( supportedformats = interfaceutil.Attribute(
"""Set of requirements that apply to stream clone. """Set of requirements that apply to stream clone.
This is actually a class attribute and is shared among all instances. This is actually a class attribute and is shared among all instances.
""") """)
openerreqs = zi.Attribute( openerreqs = interfaceutil.Attribute(
"""Set of requirements that are passed to the opener. """Set of requirements that are passed to the opener.
This is actually a class attribute and is shared among all instances. This is actually a class attribute and is shared among all instances.
""") """)
supported = zi.Attribute( supported = interfaceutil.Attribute(
"""Set of requirements that this repo is capable of opening.""") """Set of requirements that this repo is capable of opening.""")
requirements = zi.Attribute( requirements = interfaceutil.Attribute(
"""Set of requirements this repo uses.""") """Set of requirements this repo uses.""")
filtername = zi.Attribute( filtername = interfaceutil.Attribute(
"""Name of the repoview that is active on this repo.""") """Name of the repoview that is active on this repo.""")
wvfs = zi.Attribute( wvfs = interfaceutil.Attribute(
"""VFS used to access the working directory.""") """VFS used to access the working directory.""")
vfs = zi.Attribute( vfs = interfaceutil.Attribute(
"""VFS rooted at the .hg directory. """VFS rooted at the .hg directory.
Used to access repository data not in the store. Used to access repository data not in the store.
""") """)
svfs = zi.Attribute( svfs = interfaceutil.Attribute(
"""VFS rooted at the store. """VFS rooted at the store.
Used to access repository data in the store. Typically .hg/store. Used to access repository data in the store. Typically .hg/store.
But can point elsewhere if the store is shared. But can point elsewhere if the store is shared.
""") """)
root = zi.Attribute( root = interfaceutil.Attribute(
"""Path to the root of the working directory.""") """Path to the root of the working directory.""")
path = zi.Attribute( path = interfaceutil.Attribute(
"""Path to the .hg directory.""") """Path to the .hg directory.""")
origroot = zi.Attribute( origroot = interfaceutil.Attribute(
"""The filesystem path that was used to construct the repo.""") """The filesystem path that was used to construct the repo.""")
auditor = zi.Attribute( auditor = interfaceutil.Attribute(
"""A pathauditor for the working directory. """A pathauditor for the working directory.
This checks if a path refers to a nested repository. This checks if a path refers to a nested repository.
Operates on the filesystem. Operates on the filesystem.
""") """)
nofsauditor = zi.Attribute( nofsauditor = interfaceutil.Attribute(
"""A pathauditor for the working directory. """A pathauditor for the working directory.
This is like ``auditor`` except it doesn't do filesystem checks. This is like ``auditor`` except it doesn't do filesystem checks.
""") """)
baseui = zi.Attribute( baseui = interfaceutil.Attribute(
"""Original ui instance passed into constructor.""") """Original ui instance passed into constructor.""")
ui = zi.Attribute( ui = interfaceutil.Attribute(
"""Main ui instance for this instance.""") """Main ui instance for this instance.""")
sharedpath = zi.Attribute( sharedpath = interfaceutil.Attribute(
"""Path to the .hg directory of the repo this repo was shared from.""") """Path to the .hg directory of the repo this repo was shared from.""")
store = zi.Attribute( store = interfaceutil.Attribute(
"""A store instance.""") """A store instance.""")
spath = zi.Attribute( spath = interfaceutil.Attribute(
"""Path to the store.""") """Path to the store.""")
sjoin = zi.Attribute( sjoin = interfaceutil.Attribute(
"""Alias to self.store.join.""") """Alias to self.store.join.""")
cachevfs = zi.Attribute( cachevfs = interfaceutil.Attribute(
"""A VFS used to access the cache directory. """A VFS used to access the cache directory.
Typically .hg/cache. Typically .hg/cache.
""") """)
filteredrevcache = zi.Attribute( filteredrevcache = interfaceutil.Attribute(
"""Holds sets of revisions to be filtered.""") """Holds sets of revisions to be filtered.""")
names = zi.Attribute( names = interfaceutil.Attribute(
"""A ``namespaces`` instance.""") """A ``namespaces`` instance.""")
def close(): def close():
"""Close the handle on this repository.""" """Close the handle on this repository."""
def peer(): def peer():
"""Obtain an object conforming to the ``peer`` interface.""" """Obtain an object conforming to the ``peer`` interface."""
def unfiltered(): def unfiltered():
"""Obtain an unfiltered/raw view of this repo.""" """Obtain an unfiltered/raw view of this repo."""
def filtered(name, visibilityexceptions=None): def filtered(name, visibilityexceptions=None):
"""Obtain a named view of this repository.""" """Obtain a named view of this repository."""
obsstore = zi.Attribute( obsstore = interfaceutil.Attribute(
"""A store of obsolescence data.""") """A store of obsolescence data.""")
changelog = zi.Attribute( changelog = interfaceutil.Attribute(
"""A handle on the changelog revlog.""") """A handle on the changelog revlog.""")
manifestlog = zi.Attribute( manifestlog = interfaceutil.Attribute(
"""A handle on the root manifest revlog.""") """A handle on the root manifest revlog.""")
dirstate = zi.Attribute( dirstate = interfaceutil.Attribute(
"""Working directory state.""") """Working directory state.""")
narrowpats = zi.Attribute( narrowpats = interfaceutil.Attribute(
"""Matcher patterns for this repository's narrowspec.""") """Matcher patterns for this repository's narrowspec.""")
def narrowmatch(): def narrowmatch():
"""Obtain a matcher for the narrowspec.""" """Obtain a matcher for the narrowspec."""
def setnarrowpats(newincludes, newexcludes): def setnarrowpats(newincludes, newexcludes):
"""Define the narrowspec for this repository.""" """Define the narrowspec for this repository."""
pass pass
def between(pairs): def between(pairs):
pass pass
def checkpush(pushop): def checkpush(pushop):
pass pass
prepushoutgoinghooks = zi.Attribute( prepushoutgoinghooks = interfaceutil.Attribute(
"""util.hooks instance.""") """util.hooks instance.""")
def pushkey(namespace, key, old, new): def pushkey(namespace, key, old, new):
pass pass
def listkeys(namespace): def listkeys(namespace):
pass pass
def debugwireargs(one, two, three=None, four=None, five=None): def debugwireargs(one, two, three=None, four=None, five=None):
pass pass
def savecommitmessage(text): def savecommitmessage(text):
pass pass

mercurial/utils/interfaceutil.py

  • This file was added.
# interfaceutil.py - Utilities for declaring interfaces.
#
# Copyright 2018 Gregory Szorc <gregory.szorc@gmail.com>
#
# This software may be used and distributed according to the terms of the
# GNU General Public License version 2 or any later version.
# zope.interface imposes a run-time cost due to module import overhead and
# bookkeeping for declaring interfaces. So, we use stubs for various
# zope.interface primitives unless instructed otherwise.
from __future__ import absolute_import
from .. import (
encoding,
)
if encoding.environ.get('HGREALINTERFACES'):
from ..thirdparty.zope import (
interface as zi,
)
Attribute = zi.Attribute
Interface = zi.Interface
implementer = zi.implementer
else:
class Attribute(object):
def __init__(self, __name__, __doc__=''):
pass
class Interface(object):
def __init__(self, name, bases=(), attrs=None, __doc__=None,
__module__=None):
pass
def implementer(*ifaces):
def wrapper(cls):
return cls
return wrapper

mercurial/wireprotoserver.py

import struct import struct
import sys import sys
import threading import threading
from .i18n import _ from .i18n import _
from .thirdparty import ( from .thirdparty import (
cbor, cbor,
) )
from .thirdparty.zope import (
interface as zi,
)
from . import ( from . import (
encoding, encoding,
error, error,
hook, hook,
pycompat, pycompat,
util, util,
wireprototypes, wireprototypes,
wireprotov1server, wireprotov1server,
wireprotov2server, wireprotov2server,
) )
from .utils import ( from .utils import (
interfaceutil,
procutil, procutil,
) )
stringio = util.stringio stringio = util.stringio
urlerr = util.urlerr urlerr = util.urlerr
urlreq = util.urlreq urlreq = util.urlreq
v = req.headers.get(b'%s-%d' % (headerprefix, i)) v = req.headers.get(b'%s-%d' % (headerprefix, i))
if v is None: if v is None:
break break
chunks.append(pycompat.bytesurl(v)) chunks.append(pycompat.bytesurl(v))
i += 1 i += 1
return ''.join(chunks) return ''.join(chunks)
@zi.implementer(wireprototypes.baseprotocolhandler) @interfaceutil.implementer(wireprototypes.baseprotocolhandler)
class httpv1protocolhandler(object): class httpv1protocolhandler(object):
def __init__(self, req, ui, checkperm): def __init__(self, req, ui, checkperm):
self._req = req self._req = req
self._ui = ui self._ui = ui
self._checkperm = checkperm self._checkperm = checkperm
self._protocaps = None self._protocaps = None
@property @property
fout.flush() fout.flush()
def _sshv1respondooberror(fout, ferr, rsp): def _sshv1respondooberror(fout, ferr, rsp):
ferr.write(b'%s\n-\n' % rsp) ferr.write(b'%s\n-\n' % rsp)
ferr.flush() ferr.flush()
fout.write(b'\n') fout.write(b'\n')
fout.flush() fout.flush()
@zi.implementer(wireprototypes.baseprotocolhandler) @interfaceutil.implementer(wireprototypes.baseprotocolhandler)
class sshv1protocolhandler(object): class sshv1protocolhandler(object):
"""Handler for requests services via version 1 of SSH protocol.""" """Handler for requests services via version 1 of SSH protocol."""
def __init__(self, ui, fin, fout): def __init__(self, ui, fin, fout):
self._ui = ui self._ui = ui
self._fin = fin self._fin = fin
self._fout = fout self._fout = fout
self._protocaps = set() self._protocaps = set()

mercurial/wireprototypes.py

# Copyright 2018 Gregory Szorc <gregory.szorc@gmail.com> # Copyright 2018 Gregory Szorc <gregory.szorc@gmail.com>
# #
# This software may be used and distributed according to the terms of the # This software may be used and distributed according to the terms of the
# GNU General Public License version 2 or any later version. # GNU General Public License version 2 or any later version.
from __future__ import absolute_import from __future__ import absolute_import
from .node import ( from .node import (
bin, bin,
hex, hex,
) )
from .thirdparty.zope import (
interface as zi,
)
from .i18n import _ from .i18n import _
from . import ( from . import (
error, error,
util, util,
) )
from .utils import (
interfaceutil,
)
# Names of the SSH protocol implementations. # Names of the SSH protocol implementations.
SSHV1 = 'ssh-v1' SSHV1 = 'ssh-v1'
# These are advertised over the wire. Increment the counters at the end # These are advertised over the wire. Increment the counters at the end
# to reflect BC breakages. # to reflect BC breakages.
SSHV2 = 'exp-ssh-v2-0001' SSHV2 = 'exp-ssh-v2-0001'
HTTP_WIREPROTO_V2 = 'exp-http-v2-0001' HTTP_WIREPROTO_V2 = 'exp-http-v2-0001'
'phases': 'boolean', 'phases': 'boolean',
'bundlecaps': 'scsv', 'bundlecaps': 'scsv',
'listkeys': 'csv', 'listkeys': 'csv',
'cg': 'boolean', 'cg': 'boolean',
'cbattempted': 'boolean', 'cbattempted': 'boolean',
'stream': 'boolean', 'stream': 'boolean',
} }
class baseprotocolhandler(zi.Interface): class baseprotocolhandler(interfaceutil.Interface):
"""Abstract base class for wire protocol handlers. """Abstract base class for wire protocol handlers.
A wire protocol handler serves as an interface between protocol command A wire protocol handler serves as an interface between protocol command
handlers and the wire protocol transport layer. Protocol handlers provide handlers and the wire protocol transport layer. Protocol handlers provide
methods to read command arguments, redirect stdio for the duration of methods to read command arguments, redirect stdio for the duration of
the request, handle response types, etc. the request, handle response types, etc.
""" """
name = zi.Attribute( name = interfaceutil.Attribute(
"""The name of the protocol implementation. """The name of the protocol implementation.
Used for uniquely identifying the transport type. Used for uniquely identifying the transport type.
""") """)
def getargs(args): def getargs(args):
"""return the value for arguments in <args> """return the value for arguments in <args>

mercurial/wireprotov1peer.py

import hashlib import hashlib
import sys import sys
import weakref import weakref
from .i18n import _ from .i18n import _
from .node import ( from .node import (
bin, bin,
) )
from .thirdparty.zope import (
interface as zi,
)
from . import ( from . import (
bundle2, bundle2,
changegroup as changegroupmod, changegroup as changegroupmod,
encoding, encoding,
error, error,
pushkey as pushkeymod, pushkey as pushkeymod,
pycompat, pycompat,
repository, repository,
util, util,
wireprototypes, wireprototypes,
) )
from .utils import (
interfaceutil,
)
urlreq = util.urlreq urlreq = util.urlreq
def batchable(f): def batchable(f):
'''annotation for batchable methods '''annotation for batchable methods
Such methods must implement a coroutine as follows: Such methods must implement a coroutine as follows:
self._peerexecutor.sendcommands() self._peerexecutor.sendcommands()
# This looks like it will infinitely recurse. However, # This looks like it will infinitely recurse. However,
# sendcommands() should modify __class__. This call serves as a check # sendcommands() should modify __class__. This call serves as a check
# on that. # on that.
return self.result(timeout) return self.result(timeout)
@zi.implementer(repository.ipeercommandexecutor) @interfaceutil.implementer(repository.ipeercommandexecutor)
class peerexecutor(object): class peerexecutor(object):
def __init__(self, peer): def __init__(self, peer):
self._peer = peer self._peer = peer
self._sent = False self._sent = False
self._closed = False self._closed = False
self._calls = [] self._calls = []
self._futures = weakref.WeakSet() self._futures = weakref.WeakSet()
self._responseexecutor = None self._responseexecutor = None
# And ask the coroutine to decode that value. # And ask the coroutine to decode that value.
try: try:
result = next(batchable) result = next(batchable)
except Exception: except Exception:
pycompat.future_set_exception_info(f, sys.exc_info()[1:]) pycompat.future_set_exception_info(f, sys.exc_info()[1:])
else: else:
f.set_result(result) f.set_result(result)
@zi.implementer(repository.ipeercommands, repository.ipeerlegacycommands) @interfaceutil.implementer(repository.ipeercommands,
repository.ipeerlegacycommands)
class wirepeer(repository.peer): class wirepeer(repository.peer):
"""Client-side interface for communicating with a peer repository. """Client-side interface for communicating with a peer repository.
Methods commonly call wire protocol commands of the same name. Methods commonly call wire protocol commands of the same name.
See also httppeer.py and sshpeer.py for protocol-specific See also httppeer.py and sshpeer.py for protocol-specific
implementations of this interface. implementations of this interface.
""" """

mercurial/wireprotov2server.py

# Copyright 21 May 2005 - (c) 2005 Jake Edge <jake@edge2.net> # Copyright 21 May 2005 - (c) 2005 Jake Edge <jake@edge2.net>
# Copyright 2005-2007 Matt Mackall <mpm@selenic.com> # Copyright 2005-2007 Matt Mackall <mpm@selenic.com>
# #
# This software may be used and distributed according to the terms of the # This software may be used and distributed according to the terms of the
# GNU General Public License version 2 or any later version. # GNU General Public License version 2 or any later version.
from __future__ import absolute_import from __future__ import absolute_import
import contextlib import contextlib
from .i18n import _ from .i18n import _
from .thirdparty import ( from .thirdparty import (
cbor, cbor,
) )
from .thirdparty.zope import (
interface as zi,
)
from . import ( from . import (
encoding, encoding,
error, error,
pycompat, pycompat,
streamclone, streamclone,
util, util,
wireprotoframing, wireprotoframing,
wireprototypes, wireprototypes,
) )
from .utils import (
interfaceutil,
)
FRAMINGTYPE = b'application/mercurial-exp-framing-0005' FRAMINGTYPE = b'application/mercurial-exp-framing-0005'
HTTP_WIREPROTO_V2 = wireprototypes.HTTP_WIREPROTO_V2 HTTP_WIREPROTO_V2 = wireprototypes.HTTP_WIREPROTO_V2
COMMANDS = wireprototypes.commanddict() COMMANDS = wireprototypes.commanddict()
def handlehttpv2request(rctx, req, res, checkperm, urlparts): def handlehttpv2request(rctx, req, res, checkperm, urlparts):
def dispatch(repo, proto, command): def dispatch(repo, proto, command):
repo = getdispatchrepo(repo, proto, command) repo = getdispatchrepo(repo, proto, command)
func, spec = COMMANDS[command] func, spec = COMMANDS[command]
args = proto.getargs(spec) args = proto.getargs(spec)
return func(repo, proto, **args) return func(repo, proto, **args)
@zi.implementer(wireprototypes.baseprotocolhandler) @interfaceutil.implementer(wireprototypes.baseprotocolhandler)
class httpv2protocolhandler(object): class httpv2protocolhandler(object):
def __init__(self, req, ui, args=None): def __init__(self, req, ui, args=None):
self._req = req self._req = req
self._ui = ui self._ui = ui
self._args = args self._args = args
@property @property
def name(self): def name(self):

tests/test-check-interfaces.py

# Test that certain objects conform to well-defined interfaces. # Test that certain objects conform to well-defined interfaces.
from __future__ import absolute_import, print_function from __future__ import absolute_import, print_function
from mercurial import encoding
encoding.environ[b'HGREALINTERFACES'] = b'1'
import os import os
from mercurial.thirdparty.zope import ( from mercurial.thirdparty.zope import (
interface as zi, interface as zi,
) )
from mercurial.thirdparty.zope.interface import ( from mercurial.thirdparty.zope.interface import (
verify as ziverify, verify as ziverify,
) )

tests/test-check-module-imports.t

> -X contrib/hgweb.fcgi \ > -X contrib/hgweb.fcgi \
> -X contrib/python-zstandard/ \ > -X contrib/python-zstandard/ \
> -X contrib/win32/hgwebdir_wsgi.py \ > -X contrib/win32/hgwebdir_wsgi.py \
> -X doc/gendoc.py \ > -X doc/gendoc.py \
> -X doc/hgmanpage.py \ > -X doc/hgmanpage.py \
> -X i18n/posplit \ > -X i18n/posplit \
> -X mercurial/thirdparty \ > -X mercurial/thirdparty \
> -X tests/hypothesishelpers.py \ > -X tests/hypothesishelpers.py \
> -X tests/test-check-interfaces.py \
> -X tests/test-commit-interactive.t \ > -X tests/test-commit-interactive.t \
> -X tests/test-contrib-check-code.t \ > -X tests/test-contrib-check-code.t \
> -X tests/test-demandimport.py \ > -X tests/test-demandimport.py \
> -X tests/test-extension.t \ > -X tests/test-extension.t \
> -X tests/test-hghave.t \ > -X tests/test-hghave.t \
> -X tests/test-hgweb-auth.py \ > -X tests/test-hgweb-auth.py \
> -X tests/test-hgweb-no-path-info.t \ > -X tests/test-hgweb-no-path-info.t \
> -X tests/test-hgweb-no-request-uri.t \ > -X tests/test-hgweb-no-request-uri.t \
> -X tests/test-hgweb-non-interactive.t \ > -X tests/test-hgweb-non-interactive.t \
> -X tests/test-hook.t \ > -X tests/test-hook.t \
> -X tests/test-import.t \ > -X tests/test-import.t \
> -X tests/test-imports-checker.t \ > -X tests/test-imports-checker.t \
> -X tests/test-lock.py \ > -X tests/test-lock.py \
> -X tests/test-verify-repo-operations.py \ > -X tests/test-verify-repo-operations.py \
> | sed 's-\\-/-g' | $PYTHON "$import_checker" - > | sed 's-\\-/-g' | $PYTHON "$import_checker" -