This is an archive of the discontinued Mercurial Phabricator instance.

Differential D11870

upgrade: issue a message when a revlog type has to be upgraded
ClosedPublic

Authored by marmoute on Dec 7 2021, 3:53 AM.

Details

Reviewers
Alphare
Group Reviewers
hg-reviewers
Commits
rHG8405c1bffacf: upgrade: issue a message when a revlog type has to be upgraded
Summary

This is more explicite and prepare for a smoother transition to smarter picking
of the revlog we will process.

Diff Detail

Repository
rHG Mercurial
Branch
default
Lint
No Linters Available
Unit
No Unit Test Coverage

Event Timeline

marmoute created this revision.Dec 7 2021, 3:53 AM
marmoute updated this revision to Diff 31340.Dec 7 2021, 4:13 AM
Alphare requested changes to this revision.Dec 10 2021, 4:55 AM
Alphare added a subscriber: Alphare.
Alphare added inline comments.
mercurial/upgrade.py
114

Should be all filelogs since this is user-facing

127

The UI uses "manifest", I think "manifestlog" will be confusing for some users. Also, shouldn't it be "all manifests"?

This revision now requires changes to proceed.Dec 10 2021, 4:55 AM
marmoute added inline comments.Dec 10 2021, 5:16 AM
mercurial/upgrade.py
114

I prefer to use a single word approach for this (to the user) so that it is clearer that this is monolithic thing.

127

We are processing revlogs here, I find manifestlog clearer. Keep in mind that this is a debug command and the user here is likely some kind of admin.

We can make it all-manifestlogs as tree manifest exists.

marmoute updated this revision to Diff 31409.Dec 10 2021, 6:24 AM
marmoute added a commit: rHG8405c1bffacf: upgrade: issue a message when a revlog type has to be upgraded.Dec 10 2021, 10:47 AM
This revision was not accepted when it landed; it landed in state Needs Review.
Closed by commit rHG8405c1bffacf: upgrade: issue a message when a revlog type has to be upgraded (authored by marmoute). · Explain Why
This revision was automatically updated to reflect the committed changes.

Revision Contents
Changeset List

PathPackages
Mmercurial/upgrade.py (28 lines)
Mtests/test-upgrade-repo.t (22 lines)

Diff 31409

mercurial/upgrade.py

removedreqs = repo.requirements - newreqs removedreqs = repo.requirements - newreqs
addedreqs = newreqs - repo.requirements addedreqs = newreqs - repo.requirements
# check if we need to touch revlog and if so, which ones # check if we need to touch revlog and if so, which ones
touched_revlogs = set() touched_revlogs = set()
overwrite_msg = _(b'warning: ignoring %14s, as upgrade is changing: %s\n') overwrite_msg = _(b'warning: ignoring %14s, as upgrade is changing: %s\n')
msg_issued = False select_msg = _(b'note: selecting %s for processing to change: %s\n')
msg_issued = 0
FL = upgrade_engine.UPGRADE_FILELOGS FL = upgrade_engine.UPGRADE_FILELOGS
MN = upgrade_engine.UPGRADE_MANIFEST MN = upgrade_engine.UPGRADE_MANIFEST
CL = upgrade_engine.UPGRADE_CHANGELOG CL = upgrade_engine.UPGRADE_CHANGELOG
for action in sorted(up_actions + removed_actions, key=lambda a: a.name): for action in sorted(up_actions + removed_actions, key=lambda a: a.name):
# optimisation does not "requires anything, they just needs it. # optimisation does not "requires anything, they just needs it.
if action.type != upgrade_actions.FORMAT_VARIANT: if action.type != upgrade_actions.FORMAT_VARIANT:
continue continue
if action.touches_filelogs and FL not in touched_revlogs: if action.touches_filelogs and FL not in touched_revlogs:
if FL in specified_revlogs: if FL in specified_revlogs:
if not specified_revlogs[FL]: if not specified_revlogs[FL]:
msg = overwrite_msg % (b'--no-filelogs', action.name) msg = overwrite_msg % (b'--no-filelogs', action.name)
ui.warn(msg) ui.warn(msg)
msg_issued = True msg_issued = 2
else:
msg = select_msg % (b'all-filelogs', action.name)
AlphareUnsubmitted
Not Done

Should be all filelogs since this is user-facing

Alphare: Should be `all filelogs` since this is user-facing
marmouteAuthorUnsubmitted
Done

I prefer to use a single word approach for this (to the user) so that it is clearer that this is monolithic thing.

marmoute: I prefer to use a single word approach for this (to the user) so that it is clearer that this…
ui.status(msg)
if not ui.quiet:
msg_issued = 1
touched_revlogs.add(FL) touched_revlogs.add(FL)
if action.touches_manifests and MN not in touched_revlogs: if action.touches_manifests and MN not in touched_revlogs:
if MN in specified_revlogs: if MN in specified_revlogs:
if not specified_revlogs[MN]: if not specified_revlogs[MN]:
msg = overwrite_msg % (b'--no-manifest', action.name) msg = overwrite_msg % (b'--no-manifest', action.name)
ui.warn(msg) ui.warn(msg)
msg_issued = True msg_issued = 2
else:
msg = select_msg % (b'all-manifestlog', action.name)
AlphareUnsubmitted
Not Done

The UI uses "manifest", I think "manifestlog" will be confusing for some users. Also, shouldn't it be "all manifests"?

Alphare: The UI uses "manifest", I think "manifestlog" will be confusing for some users. Also, shouldn't…
marmouteAuthorUnsubmitted
Done

We are processing revlogs here, I find manifestlog clearer. Keep in mind that this is a debug command and the user here is likely some kind of admin.

We can make it all-manifestlogs as tree manifest exists.

marmoute: We are processing revlogs here, I find manifestlog clearer. Keep in mind that this is a debug…
ui.status(msg)
if not ui.quiet:
msg_issued = 1
touched_revlogs.add(MN) touched_revlogs.add(MN)
if action.touches_changelog and CL not in touched_revlogs: if action.touches_changelog and CL not in touched_revlogs:
if CL in specified_revlogs: if CL in specified_revlogs:
if not specified_revlogs[CL]: if not specified_revlogs[CL]:
msg = overwrite_msg % (b'--no-changelog', action.name) msg = overwrite_msg % (b'--no-changelog', action.name)
ui.warn(msg) ui.warn(msg)
msg_issued = True msg_issued = True
else:
msg = select_msg % (b'changelog', action.name)
ui.status(msg)
if not ui.quiet:
msg_issued = 1
touched_revlogs.add(CL) touched_revlogs.add(CL)
if msg_issued: if msg_issued >= 2:
ui.warn((b"\n")) ui.warn((b"\n"))
elif msg_issued >= 1:
ui.status((b"\n"))
revlogs = set(upgrade_engine.UPGRADE_ALL_REVLOGS) revlogs = set(upgrade_engine.UPGRADE_ALL_REVLOGS)
if specified_revlogs: if specified_revlogs:
# we have some limitation on revlogs to be recloned # we have some limitation on revlogs to be recloned
if any(specified_revlogs.values()): if any(specified_revlogs.values()):
revlogs = set() revlogs = set()
for upgrade, enabled in specified_revlogs.items(): for upgrade, enabled in specified_revlogs.items():
if enabled: if enabled:

tests/test-upgrade-repo.t

[formatvariant.name.uptodate|copies-sdc: ][formatvariant.repo.uptodate| no][formatvariant.config.default| no][formatvariant.default| no] [formatvariant.name.uptodate|copies-sdc: ][formatvariant.repo.uptodate| no][formatvariant.config.default| no][formatvariant.default| no]
[formatvariant.name.uptodate|revlog-v2: ][formatvariant.repo.uptodate| no][formatvariant.config.default| no][formatvariant.default| no] [formatvariant.name.uptodate|revlog-v2: ][formatvariant.repo.uptodate| no][formatvariant.config.default| no][formatvariant.default| no]
[formatvariant.name.uptodate|changelog-v2: ][formatvariant.repo.uptodate| no][formatvariant.config.default| no][formatvariant.default| no] [formatvariant.name.uptodate|changelog-v2: ][formatvariant.repo.uptodate| no][formatvariant.config.default| no][formatvariant.default| no]
[formatvariant.name.uptodate|plain-cl-delta: ][formatvariant.repo.uptodate| yes][formatvariant.config.default| yes][formatvariant.default| yes] [formatvariant.name.uptodate|plain-cl-delta: ][formatvariant.repo.uptodate| yes][formatvariant.config.default| yes][formatvariant.default| yes]
[formatvariant.name.uptodate|compression: ][formatvariant.repo.uptodate| zlib][formatvariant.config.default| zlib][formatvariant.default| zlib] (no-zstd !) [formatvariant.name.uptodate|compression: ][formatvariant.repo.uptodate| zlib][formatvariant.config.default| zlib][formatvariant.default| zlib] (no-zstd !)
[formatvariant.name.mismatchdefault|compression: ][formatvariant.repo.mismatchdefault| zlib][formatvariant.config.special| zlib][formatvariant.default| zstd] (zstd !) [formatvariant.name.mismatchdefault|compression: ][formatvariant.repo.mismatchdefault| zlib][formatvariant.config.special| zlib][formatvariant.default| zstd] (zstd !)
[formatvariant.name.uptodate|compression-level: ][formatvariant.repo.uptodate| default][formatvariant.config.default| default][formatvariant.default| default] [formatvariant.name.uptodate|compression-level: ][formatvariant.repo.uptodate| default][formatvariant.config.default| default][formatvariant.default| default]
$ hg debugupgraderepo $ hg debugupgraderepo
note: selecting all-filelogs for processing to change: dotencode
note: selecting all-manifestlog for processing to change: dotencode
note: selecting changelog for processing to change: dotencode
repository lacks features recommended by current config options: repository lacks features recommended by current config options:
fncache fncache
long and reserved filenames may not work correctly; repository performance is sub-optimal long and reserved filenames may not work correctly; repository performance is sub-optimal
dotencode dotencode
storage of filenames beginning with a period or space may not work correctly storage of filenames beginning with a period or space may not work correctly
processed revlogs: processed revlogs:
- all-filelogs - all-filelogs
- changelog - changelog
- manifest - manifest
$ hg --config format.dotencode=false debugupgraderepo $ hg --config format.dotencode=false debugupgraderepo
note: selecting all-filelogs for processing to change: fncache
note: selecting all-manifestlog for processing to change: fncache
note: selecting changelog for processing to change: fncache
repository lacks features recommended by current config options: repository lacks features recommended by current config options:
fncache fncache
long and reserved filenames may not work correctly; repository performance is sub-optimal long and reserved filenames may not work correctly; repository performance is sub-optimal
generaldelta generaldelta
deltas within internal storage are unable to choose optimal revisions; repository is larger and slower than it could be; interaction with other repositories may require extra network and CPU resources, making "hg push" and "hg pull" slower deltas within internal storage are unable to choose optimal revisions; repository is larger and slower than it could be; interaction with other repositories may require extra network and CPU resources, making "hg push" and "hg pull" slower
$ ls -d .hg/store/data/* $ ls -d .hg/store/data/*
.hg/store/data/_foo_bar_directory.d.hg .hg/store/data/_foo_bar_directory.d.hg
.hg/store/data/f0.i .hg/store/data/f0.i
.hg/store/data/f2.d .hg/store/data/f2.d
.hg/store/data/f2.i .hg/store/data/f2.i
$ hg debugupgraderepo --run --config format.sparse-revlog=false $ hg debugupgraderepo --run --config format.sparse-revlog=false
note: selecting all-filelogs for processing to change: generaldelta
note: selecting all-manifestlog for processing to change: generaldelta
note: selecting changelog for processing to change: generaldelta
upgrade will perform the following actions: upgrade will perform the following actions:
requirements requirements
preserved: dotencode, fncache, revlogv1, store (no-rust !) preserved: dotencode, fncache, revlogv1, store (no-rust !)
preserved: dotencode, fncache, persistent-nodemap, revlogv1, store (rust !) preserved: dotencode, fncache, persistent-nodemap, revlogv1, store (rust !)
added: generaldelta added: generaldelta
generaldelta generaldelta
undo.backup.fncache undo.backup.fncache
undo.backupfiles undo.backupfiles
undo.phaseroots undo.phaseroots
unless --no-backup is passed unless --no-backup is passed
$ rm -rf .hg/upgradebackup.*/ $ rm -rf .hg/upgradebackup.*/
$ hg debugupgraderepo --run --no-backup $ hg debugupgraderepo --run --no-backup
note: selecting all-filelogs for processing to change: sparserevlog
note: selecting all-manifestlog for processing to change: sparserevlog
note: selecting changelog for processing to change: sparserevlog
upgrade will perform the following actions: upgrade will perform the following actions:
requirements requirements
preserved: dotencode, fncache, generaldelta, revlogv1, store (no-rust !) preserved: dotencode, fncache, generaldelta, revlogv1, store (no-rust !)
preserved: dotencode, fncache, generaldelta, persistent-nodemap, revlogv1, store (rust !) preserved: dotencode, fncache, generaldelta, persistent-nodemap, revlogv1, store (rust !)
added: sparserevlog added: sparserevlog
sparserevlog sparserevlog
optimisations: re-delta-parent optimisations: re-delta-parent
processed revlogs: processed revlogs:
- all-filelogs - all-filelogs
- changelog - changelog
- manifest - manifest
$ hg debugupgrade --optimize re-delta-parent --run --manifest --no-backup --debug --traceback $ hg debugupgrade --optimize re-delta-parent --run --manifest --no-backup --debug --traceback
note: selecting all-filelogs for processing to change: sparserevlog
note: selecting changelog for processing to change: sparserevlog
ignoring revlogs selection flags, format requirements change: sparserevlog ignoring revlogs selection flags, format requirements change: sparserevlog
upgrade will perform the following actions: upgrade will perform the following actions:
requirements requirements
preserved: dotencode, fncache, generaldelta, revlogv1, store (no-rust !) preserved: dotencode, fncache, generaldelta, revlogv1, store (no-rust !)
preserved: dotencode, fncache, generaldelta, persistent-nodemap, revlogv1, store (rust !) preserved: dotencode, fncache, generaldelta, persistent-nodemap, revlogv1, store (rust !)
removed: sparserevlog removed: sparserevlog
crosschecking files in changesets and manifests crosschecking files in changesets and manifests
checking files checking files
checked 3 changesets with 3 changes to 3 files checked 3 changesets with 3 changes to 3 files
Check you can't skip revlog clone during important format upgrade Check you can't skip revlog clone during important format upgrade
$ echo "sparse-revlog=yes" >> .hg/hgrc $ echo "sparse-revlog=yes" >> .hg/hgrc
$ hg debugupgrade --optimize re-delta-parent --run --manifest --no-backup --debug --traceback $ hg debugupgrade --optimize re-delta-parent --run --manifest --no-backup --debug --traceback
note: selecting all-filelogs for processing to change: sparserevlog
note: selecting changelog for processing to change: sparserevlog
ignoring revlogs selection flags, format requirements change: sparserevlog ignoring revlogs selection flags, format requirements change: sparserevlog
upgrade will perform the following actions: upgrade will perform the following actions:
requirements requirements
preserved: dotencode, fncache, generaldelta, revlogv1, store (no-rust !) preserved: dotencode, fncache, generaldelta, revlogv1, store (no-rust !)
preserved: dotencode, fncache, generaldelta, persistent-nodemap, revlogv1, store (rust !) preserved: dotencode, fncache, generaldelta, persistent-nodemap, revlogv1, store (rust !)
added: sparserevlog added: sparserevlog