This is an archive of the discontinued Mercurial Phabricator instance.

Differential D155

fastmanifest: make usecache prevent cache creation
ClosedPublic

Authored by durham on Jul 20 2017, 8:15 AM.
Tags
None
Subscribers
None

Details

Reviewers
quark
Group Reviewers
Restricted Project
Commits
rFBHGX296920e70bc6: fastmanifest: make usecache prevent cache creation
Summary

Previously fastmanifest.usecache=False would only prevent the reading of the
fastmanifest cache, but the cache was still created. This created excess IO on
services that frequently recreated the repository.

Test Plan

Added a test

Diff Detail

Repository
rFBHGX Facebook Mercurial Extensions
Lint
Automatic diff as part of commit; lint not applicable.
Unit
Automatic diff as part of commit; unit tests not applicable.

Event Timeline

durham created this revision.Jul 20 2017, 8:15 AM
quark accepted this revision.Jul 21 2017, 1:27 AM
This revision is now accepted and ready to land.Jul 21 2017, 1:27 AM
Closed by commit rFBHGX296920e70bc6: fastmanifest: make usecache prevent cache creation (authored by durham). · Explain WhyJul 31 2017, 4:55 PM
This revision was automatically updated to reflect the committed changes.

Revision Contents
Changeset List

PathPackages
Mfastmanifest/__init__.py (66 lines)
Mtests/test-fastmanifest.t (13 lines)

Diff 461

fastmanifest/__init__.py

# Cut off date, revisions older than the cutoff won't be cached, default is # Cut off date, revisions older than the cutoff won't be cached, default is
# 60 days. -1 means no limit. # 60 days. -1 means no limit.
cachecutoffdays = 60 cachecutoffdays = 60
# List of relevant remotenames whose manifest is to be included in the cache. # List of relevant remotenames whose manifest is to be included in the cache.
# The list is comma or space separated # The list is comma or space separated
relevantremotenames = master relevantremotenames = master
# Enables the creation and use of fast cache manifests (defaults to True)
usecache=False
# Enables the use of treemanifests (defaults to False)
usetree=True
Description: Description:
`manifestaccesslogger` logs manifest accessed to a logfile specified with `manifestaccesslogger` logs manifest accessed to a logfile specified with
the option fastmanifest.logfile the option fastmanifest.logfile
`fastmanifesttocache` is a revset of relevant manifests to cache `fastmanifesttocache` is a revset of relevant manifests to cache
`hybridmanifest` is a proxy class for flat and cached manifest that loads `hybridmanifest` is a proxy class for flat and cached manifest that loads
return FastManifestExtension.uiproxy return FastManifestExtension.uiproxy
@staticmethod @staticmethod
def set_ui(ui): def set_ui(ui):
FastManifestExtension.uiproxy._updateui(ui) FastManifestExtension.uiproxy._updateui(ui)
@staticmethod @staticmethod
def setup(): def setup():
logger = debug.manifestaccesslogger(FastManifestExtension.get_ui()) ui = FastManifestExtension.get_ui()
logger = debug.manifestaccesslogger(ui)
extensions.wrapfunction(manifest.manifestrevlog, 'rev', logger.revwrap) extensions.wrapfunction(manifest.manifestrevlog, 'rev', logger.revwrap)
factory = manifestfactory(FastManifestExtension.get_ui()) factory = manifestfactory(ui)
extensions.wrapfunction( extensions.wrapfunction(
manifest.manifestlog, '__getitem__', factory.newgetitem) manifest.manifestlog, '__getitem__', factory.newgetitem)
extensions.wrapfunction( extensions.wrapfunction(
manifest.manifestlog, 'get', factory.newgetdirmanifestctx) manifest.manifestlog, 'get', factory.newgetdirmanifestctx)
extensions.wrapfunction(manifest.manifestrevlog, 'add', factory.add) extensions.wrapfunction(manifest.manifestrevlog, 'add', factory.add)
if ui.configbool('fastmanifest', 'usecache', True):
revsetmod.symbols['fastmanifesttocache'] = ( revsetmod.symbols['fastmanifesttocache'] = (
cachemanager.fastmanifesttocache cachemanager.fastmanifesttocache
) )
revsetmod.safesymbols.add('fastmanifesttocache') revsetmod.safesymbols.add('fastmanifesttocache')
revsetmod.symbols['fastmanifestcached'] = ( revsetmod.symbols['fastmanifestcached'] = (
cachemanager.fastmanifestcached cachemanager.fastmanifestcached
) )
revsetmod.safesymbols.add('fastmanifestcached') revsetmod.safesymbols.add('fastmanifestcached')
# Trigger to enable caching of relevant manifests # Trigger to enable caching of relevant manifests
extensions.wrapfunction(bookmarks.bmstore, '_write', extensions.wrapfunction(bookmarks.bmstore, '_write',
cachemanager.triggers.onbookmarkchange) cachemanager.triggers.onbookmarkchange)
extensions.wrapfunction(localrepo.localrepository, 'commitctx', extensions.wrapfunction(localrepo.localrepository, 'commitctx',
cachemanager.triggers.oncommit) cachemanager.triggers.oncommit)
try: try:
remotenames = extensions.find('remotenames') remotenames = extensions.find('remotenames')
except KeyError: except KeyError:
pass pass
else: else:
if remotenames: if remotenames:
extensions.wrapfunction( extensions.wrapfunction(
remotenames, remotenames,
'saveremotenames', 'saveremotenames',
cachemanager.triggers.onremotenameschange) cachemanager.triggers.onremotenameschange)
extensions.wrapfunction( extensions.wrapfunction(
dispatch, 'runcommand', dispatch, 'runcommand',
FastManifestExtension._logonexit) cachemanager.triggers.runcommandtrigger)
extensions.wrapfunction( extensions.wrapfunction(
dispatch, 'runcommand', dispatch, 'runcommand',
cachemanager.triggers.runcommandtrigger) FastManifestExtension._logonexit)
def extsetup(ui): def extsetup(ui):
# always update the ui object. this is probably a bogus ui object, but we # always update the ui object. this is probably a bogus ui object, but we
# don't want to have a backing ui object of None. # don't want to have a backing ui object of None.
FastManifestExtension.set_ui(ui) FastManifestExtension.set_ui(ui)
FastManifestExtension.setup() FastManifestExtension.setup()

tests/test-fastmanifest.t

$ mkdir cachetesting $ mkdir cachetesting
$ cd cachetesting $ cd cachetesting
$ hg init $ hg init
$ cat >> .hg/hgrc << EOF $ cat >> .hg/hgrc << EOF
> [extensions] > [extensions]
> fastmanifest=$TESTDIR/../fastmanifest > fastmanifest=$TESTDIR/../fastmanifest
> [fastmanifest] > [fastmanifest]
> cachecutoffdays=-1 > cachecutoffdays=-1
> debugmetrics=True
> randomorder=False > randomorder=False
> usecache=False
> EOF
Test usecache=False prevents cache creation
$ mkcommit 0
$ ls .hg/store/manifestcache
$ hg strip -q -r . --config extensions.strip=
$ cat >> .hg/hgrc <<EOF
> [fastmanifest]
> debugmetrics=True
> usecache=True
> EOF > EOF
Here we expect a cache hit of 100% because we are based of revision -1 that Here we expect a cache hit of 100% because we are based of revision -1 that
is cached is cached
$ mkcommit a $ mkcommit a
[FM-METRICS] Begin metrics [FM-METRICS] Begin metrics
[FM-METRICS] kind: cachehitratio, kwargs: [('cachehitratio', -1)] [FM-METRICS] kind: cachehitratio, kwargs: [('cachehitratio', -1)]
[FM-METRICS] kind: diffcachehitratio, kwargs: [('diffcachehitratio', -1)] [FM-METRICS] kind: diffcachehitratio, kwargs: [('diffcachehitratio', -1)]