This is an archive of the discontinued Mercurial Phabricator instance.

Differential D8417

nodemap: gate the feature behind a new requirement
ClosedPublic

Authored by marmoute on Apr 14 2020, 11:48 AM.

Details

Reviewers
Alphare
Group Reviewers
hg-reviewers
Commits
rHGb81486b609a3: nodemap: gate the feature behind a new requirement
Summary

Now that the feature is working smoothly, a question was still open, should we
gate the feature behind a new requirement or just treat it as a cache to be
warmed by those who can and ignored by other.

The advantage of using the cache approach is a transparent upgrade/downgrade
story, making the feature easier to move to. However having out of date cache
can come with a significant performance hit for process who expect an up to
date cache but found none. In this case the file needs to be stored under
.hg/cache.

The "requirement" approach guarantee that the persistent nodemap is up to date.
However, it comes with a less flexible activation story since an explicite
upgrade is required. In this case the file can be stored in .hg/store.

This wiki page is relevant to this questions:

https://www.mercurial-scm.org/wiki/ComputedIndexPlan

So which one should we take? Another element came into plan, the persistent
nodemap use the add method of the transaction, it is used to keep track of a
file content before a transaction in case we need to rollback it back. It turns
out that the transaction.add API does not support file stored anywhere than
.hg/store. Making it support file stored elsewhere is possible, require a
change in on disk transaction format. Updating on disk file requires…
introducing a new requirements.

As a result, we pick the second option "gating the persistent nodemap behind a
new requirements".

Diff Detail

Repository
rHG Mercurial
Lint
Automatic diff as part of commit; lint not applicable.
Unit
Automatic diff as part of commit; unit tests not applicable.

Event Timeline

marmoute created this revision.Apr 14 2020, 11:48 AM
marmoute updated this revision to Diff 21228.Apr 27 2020, 1:12 PM
Alphare accepted this revision.May 7 2020, 9:09 AM
marmoute added a commit: rHGb81486b609a3: nodemap: gate the feature behind a new requirement.May 7 2020, 11:34 AM
This revision was not accepted when it landed; it landed in state Needs Review.
Closed by commit rHGb81486b609a3: nodemap: gate the feature behind a new requirement (authored by marmoute). · Explain Why
This revision was automatically updated to reflect the committed changes.

Revision Contents
Changeset List

Diff 21293

mercurial/localrepo.py

# A repository with the sidedataflag requirement will allow to store extra # A repository with the sidedataflag requirement will allow to store extra
# information for revision without altering their original hashes. # information for revision without altering their original hashes.
SIDEDATA_REQUIREMENT = b'exp-sidedata-flag' SIDEDATA_REQUIREMENT = b'exp-sidedata-flag'
# A repository with the the copies-sidedata-changeset requirement will store # A repository with the the copies-sidedata-changeset requirement will store
# copies related information in changeset's sidedata. # copies related information in changeset's sidedata.
COPIESSDC_REQUIREMENT = b'exp-copies-sidedata-changeset' COPIESSDC_REQUIREMENT = b'exp-copies-sidedata-changeset'
# The repository use persistent nodemap for the changelog and the manifest.
NODEMAP_REQUIREMENT = b'persistent-nodemap'
# Functions receiving (ui, features) that extensions can register to impact # Functions receiving (ui, features) that extensions can register to impact
# the ability to load repositories with custom requirements. Only # the ability to load repositories with custom requirements. Only
# functions defined in loaded extensions are called. # functions defined in loaded extensions are called.
# #
# The function receives a set of requirement strings that the repository # The function receives a set of requirement strings that the repository
# is capable of opening. Functions will typically add elements to the # is capable of opening. Functions will typically add elements to the
# set to reflect that the extension knows how to handle that requirements. # set to reflect that the extension knows how to handle that requirements.
featuresetupfuncs = set() featuresetupfuncs = set()
msg = _(b'invalid value for `storage.revlog.zstd.level` config: %d') msg = _(b'invalid value for `storage.revlog.zstd.level` config: %d')
raise error.Abort(msg % options[b'zstd.level']) raise error.Abort(msg % options[b'zstd.level'])
if repository.NARROW_REQUIREMENT in requirements: if repository.NARROW_REQUIREMENT in requirements:
options[b'enableellipsis'] = True options[b'enableellipsis'] = True
if ui.configbool(b'experimental', b'rust.index'): if ui.configbool(b'experimental', b'rust.index'):
options[b'rust.index'] = True options[b'rust.index'] = True
if ui.configbool(b'experimental', b'exp-persistent-nodemap'): if NODEMAP_REQUIREMENT in requirements:
options[b'exp-persistent-nodemap'] = True options[b'exp-persistent-nodemap'] = True
if ui.configbool(b'experimental', b'exp-persistent-nodemap.mmap'): if ui.configbool(b'experimental', b'exp-persistent-nodemap.mmap'):
options[b'exp-persistent-nodemap.mmap'] = True options[b'exp-persistent-nodemap.mmap'] = True
epnm = ui.config(b'experimental', b'exp-persistent-nodemap.mode') epnm = ui.config(b'experimental', b'exp-persistent-nodemap.mode')
options[b'exp-persistent-nodemap.mode'] = epnm options[b'exp-persistent-nodemap.mode'] = epnm
if ui.configbool(b'devel', b'persistent-nodemap'): if ui.configbool(b'devel', b'persistent-nodemap'):
options[b'devel-force-nodemap'] = True options[b'devel-force-nodemap'] = True
supportedformats = { supportedformats = {
b'revlogv1', b'revlogv1',
b'generaldelta', b'generaldelta',
b'treemanifest', b'treemanifest',
COPIESSDC_REQUIREMENT, COPIESSDC_REQUIREMENT,
REVLOGV2_REQUIREMENT, REVLOGV2_REQUIREMENT,
SIDEDATA_REQUIREMENT, SIDEDATA_REQUIREMENT,
SPARSEREVLOG_REQUIREMENT, SPARSEREVLOG_REQUIREMENT,
NODEMAP_REQUIREMENT,
bookmarks.BOOKMARKS_IN_STORE_REQUIREMENT, bookmarks.BOOKMARKS_IN_STORE_REQUIREMENT,
} }
_basesupported = supportedformats | { _basesupported = supportedformats | {
b'store', b'store',
b'fncache', b'fncache',
b'shared', b'shared',
b'relshared', b'relshared',
b'dotencode', b'dotencode',
requirements.add(repository.NARROW_REQUIREMENT) requirements.add(repository.NARROW_REQUIREMENT)
if createopts.get(b'lfs'): if createopts.get(b'lfs'):
requirements.add(b'lfs') requirements.add(b'lfs')
if ui.configbool(b'format', b'bookmarks-in-store'): if ui.configbool(b'format', b'bookmarks-in-store'):
requirements.add(bookmarks.BOOKMARKS_IN_STORE_REQUIREMENT) requirements.add(bookmarks.BOOKMARKS_IN_STORE_REQUIREMENT)
if ui.configbool(b'experimental', b'exp-persistent-nodemap'):
requirements.add(NODEMAP_REQUIREMENT)
return requirements return requirements
def filterknowncreateopts(ui, createopts): def filterknowncreateopts(ui, createopts):
"""Filters a dict of repo creation options against options that are known. """Filters a dict of repo creation options against options that are known.
Receives a dict of repo creation options and returns a dict of those Receives a dict of repo creation options and returns a dict of those
options that we don't know how to handle. options that we don't know how to handle.

tests/test-persistent-nodemap.t

=================================== ===================================
Test the persistent on-disk nodemap Test the persistent on-disk nodemap
=================================== ===================================
$ hg init test-repo $ cat << EOF >> $HGRCPATH
$ cd test-repo
$ cat << EOF >> .hg/hgrc
> [experimental] > [experimental]
> exp-persistent-nodemap=yes > exp-persistent-nodemap=yes
> [devel] > [devel]
> persistent-nodemap=yes > persistent-nodemap=yes
> EOF > EOF
$ hg init test-repo
$ cd test-repo
$ hg debugbuilddag .+5000 --new-file --config "experimental.exp-persistent-nodemap.mode=warn" $ hg debugbuilddag .+5000 --new-file --config "experimental.exp-persistent-nodemap.mode=warn"
persistent nodemap in strict mode without efficient method (no-rust no-pure !) persistent nodemap in strict mode without efficient method (no-rust no-pure !)
persistent nodemap in strict mode without efficient method (no-rust no-pure !) persistent nodemap in strict mode without efficient method (no-rust no-pure !)
$ hg debugnodemap --metadata $ hg debugnodemap --metadata
uid: ???????????????? (glob) uid: ???????????????? (glob)
tip-rev: 5000 tip-rev: 5000
tip-node: 6b02b8c7b96654c25e86ba69eda198d7e6ad8b3c tip-node: 6b02b8c7b96654c25e86ba69eda198d7e6ad8b3c
data-length: 121088 data-length: 121088