( )⚙ D6887 upgrade: detect the side-data format variants

This is an archive of the discontinued Mercurial Phabricator instance.

Differential D6887

upgrade: detect the side-data format variants
ClosedPublic

Authored by marmoute on Sep 27 2019, 12:16 PM.

Details

Reviewers
durin42
Group Reviewers
hg-reviewers
Commits
rHGe16ca9fde7aa: upgrade: detect the side-data format variants
Summary

Note that for now we cannot upgrade/downgrade to it.

Diff Detail

Repository
rHG Mercurial
Lint
Automatic diff as part of commit; lint not applicable.
Unit
Automatic diff as part of commit; unit tests not applicable.

Event Timeline

marmoute created this revision.Sep 27 2019, 12:16 PM
durin42 added inline comments.Sep 30 2019, 4:40 PM
tests/test-lfs-serve.t
136

how about:

"Allows storage of extra data alongside a revision."

marmoute added inline comments.Sep 30 2019, 6:18 PM
tests/test-lfs-serve.t
136

This looks good to me. Updating shortly.

marmoute updated this revision to Diff 16713.Sep 30 2019, 6:31 PM
marmoute updated this revision to Diff 16736.Oct 1 2019, 12:06 PM
marmoute added a commit: rHGe16ca9fde7aa: upgrade: detect the side-data format variants.Oct 1 2019, 2:22 PM
This revision was not accepted when it landed; it landed in state Needs Review.
Closed by commit rHGe16ca9fde7aa: upgrade: detect the side-data format variants (authored by marmoute). · Explain Why
This revision was automatically updated to reflect the committed changes.

Revision Contents
Changeset List

PathPackages
Mmercurial/upgrade.py (13 lines)
Mtests/test-lfs-serve.t (3 lines)
AMtests/test-sidedata.t (65 lines)
Mtests/test-upgrade-repo.t (74 lines)

Diff 16759

mercurial/upgrade.py

) )
upgrademessage = _('Revlog supports delta chain with more unused data ' upgrademessage = _('Revlog supports delta chain with more unused data '
'between payload. These gaps will be skipped at read ' 'between payload. These gaps will be skipped at read '
'time. This allows for better delta chains, making a ' 'time. This allows for better delta chains, making a '
'better compression and faster exchange with server.') 'better compression and faster exchange with server.')
@registerformatvariant @registerformatvariant
class sidedata(requirementformatvariant):
name = 'sidedata'
_requirement = localrepo.SIDEDATA_REQUIREMENT
default = False
description = _('Allows storage of extra data alongside a revision, '
'unlocking various caching options.')
upgrademessage = _('Allows storage of extra data alongside a revision.')
@registerformatvariant
class removecldeltachain(formatvariant): class removecldeltachain(formatvariant):
name = 'plain-cl-delta' name = 'plain-cl-delta'
default = True default = True
description = _('changelog storage is using deltas instead of ' description = _('changelog storage is using deltas instead of '
'raw entries; changelog reading and any ' 'raw entries; changelog reading and any '
'operation relying on changelog data are slower ' 'operation relying on changelog data are slower '

tests/test-lfs-serve.t

@@ -0,0 +1,1 @@ @@ -0,0 +1,1 @@
+non-lfs +non-lfs
*** runcommand debugupgraderepo -q --run *** runcommand debugupgraderepo -q --run
upgrade will perform the following actions: upgrade will perform the following actions:
requirements requirements
preserved: dotencode, fncache, generaldelta, revlogv1, sparserevlog, store preserved: dotencode, fncache, generaldelta, revlogv1, sparserevlog, store
sidedata
Allows storage of extra data alongside a revision.
durin42Unsubmitted
Not Done

how about:

"Allows storage of extra data alongside a revision."

durin42: how about: "Allows storage of extra data alongside a revision."
marmouteAuthorUnsubmitted
Done

This looks good to me. Updating shortly.

marmoute: This looks good to me. Updating shortly.
beginning upgrade... beginning upgrade...
repository locked and read-only repository locked and read-only
creating temporary repository to stage migrated data: * (glob) creating temporary repository to stage migrated data: * (glob)
(it is safe to interrupt this process any time before data migration completes) (it is safe to interrupt this process any time before data migration completes)
migrating 3 total revisions (1 in filelogs, 1 in manifests, 1 in changelog) migrating 3 total revisions (1 in filelogs, 1 in manifests, 1 in changelog)
migrating 324 bytes in store; 129 bytes tracked data migrating 324 bytes in store; 129 bytes tracked data
migrating 1 filelogs containing 1 revisions (73 bytes in store; 8 bytes tracked data) migrating 1 filelogs containing 1 revisions (73 bytes in store; 8 bytes tracked data)
finished migrating 1 filelog revisions across 1 filelogs; change in size: 0 bytes finished migrating 1 filelog revisions across 1 filelogs; change in size: 0 bytes

tests/test-sidedata.t

  • This file was added.
==========================================================
Test file dedicated to checking side-data related behavior
==========================================================
Check upgrade behavior
======================
Right now, sidedata has not upgrade support
Check that we cannot upgrade to sidedata
----------------------------------------
$ hg init up-no-side-data --config format.use-side-data=no
$ hg debugformat -v -R up-no-side-data
format-variant repo config default
fncache: yes yes yes
dotencode: yes yes yes
generaldelta: yes yes yes
sparserevlog: yes yes yes
sidedata: no no no
plain-cl-delta: yes yes yes
compression: zlib zlib zlib
compression-level: default default default
$ hg debugformat -v -R up-no-side-data --config format.use-side-data=yes
format-variant repo config default
fncache: yes yes yes
dotencode: yes yes yes
generaldelta: yes yes yes
sparserevlog: yes yes yes
sidedata: no yes no
plain-cl-delta: yes yes yes
compression: zlib zlib zlib
compression-level: default default default
$ hg debugupgraderepo -R up-no-side-data --config format.use-side-data=yes
abort: cannot upgrade repository; do not support adding requirement: exp-sidedata-flag
[255]
Check that we cannot upgrade to sidedata
----------------------------------------
$ hg init up-side-data --config format.use-side-data=yes
$ hg debugformat -v -R up-side-data
format-variant repo config default
fncache: yes yes yes
dotencode: yes yes yes
generaldelta: yes yes yes
sparserevlog: yes yes yes
sidedata: yes no no
plain-cl-delta: yes yes yes
compression: zlib zlib zlib
compression-level: default default default
$ hg debugformat -v -R up-side-data --config format.use-side-data=no
format-variant repo config default
fncache: yes yes yes
dotencode: yes yes yes
generaldelta: yes yes yes
sparserevlog: yes yes yes
sidedata: yes no no
plain-cl-delta: yes yes yes
compression: zlib zlib zlib
compression-level: default default default
$ hg debugupgraderepo -R up-side-data --config format.use-side-data=no
abort: cannot upgrade repository; requirement would be removed: exp-sidedata-flag
[255]

tests/test-upgrade-repo.t

$ hg init empty $ hg init empty
$ cd empty $ cd empty
$ hg debugformat $ hg debugformat
format-variant repo format-variant repo
fncache: yes fncache: yes
dotencode: yes dotencode: yes
generaldelta: yes generaldelta: yes
sparserevlog: yes sparserevlog: yes
sidedata: no
plain-cl-delta: yes plain-cl-delta: yes
compression: zlib compression: zlib
compression-level: default compression-level: default
$ hg debugformat --verbose $ hg debugformat --verbose
format-variant repo config default format-variant repo config default
fncache: yes yes yes fncache: yes yes yes
dotencode: yes yes yes dotencode: yes yes yes
generaldelta: yes yes yes generaldelta: yes yes yes
sparserevlog: yes yes yes sparserevlog: yes yes yes
sidedata: no no no
plain-cl-delta: yes yes yes plain-cl-delta: yes yes yes
compression: zlib zlib zlib compression: zlib zlib zlib
compression-level: default default default compression-level: default default default
$ hg debugformat --verbose --config format.usefncache=no $ hg debugformat --verbose --config format.usefncache=no
format-variant repo config default format-variant repo config default
fncache: yes no yes fncache: yes no yes
dotencode: yes no yes dotencode: yes no yes
generaldelta: yes yes yes generaldelta: yes yes yes
sparserevlog: yes yes yes sparserevlog: yes yes yes
sidedata: no no no
plain-cl-delta: yes yes yes plain-cl-delta: yes yes yes
compression: zlib zlib zlib compression: zlib zlib zlib
compression-level: default default default compression-level: default default default
$ hg debugformat --verbose --config format.usefncache=no --color=debug $ hg debugformat --verbose --config format.usefncache=no --color=debug
format-variant repo config default format-variant repo config default
[formatvariant.name.mismatchconfig|fncache: ][formatvariant.repo.mismatchconfig| yes][formatvariant.config.special| no][formatvariant.default| yes] [formatvariant.name.mismatchconfig|fncache: ][formatvariant.repo.mismatchconfig| yes][formatvariant.config.special| no][formatvariant.default| yes]
[formatvariant.name.mismatchconfig|dotencode: ][formatvariant.repo.mismatchconfig| yes][formatvariant.config.special| no][formatvariant.default| yes] [formatvariant.name.mismatchconfig|dotencode: ][formatvariant.repo.mismatchconfig| yes][formatvariant.config.special| no][formatvariant.default| yes]
[formatvariant.name.uptodate|generaldelta: ][formatvariant.repo.uptodate| yes][formatvariant.config.default| yes][formatvariant.default| yes] [formatvariant.name.uptodate|generaldelta: ][formatvariant.repo.uptodate| yes][formatvariant.config.default| yes][formatvariant.default| yes]
[formatvariant.name.uptodate|sparserevlog: ][formatvariant.repo.uptodate| yes][formatvariant.config.default| yes][formatvariant.default| yes] [formatvariant.name.uptodate|sparserevlog: ][formatvariant.repo.uptodate| yes][formatvariant.config.default| yes][formatvariant.default| yes]
[formatvariant.name.uptodate|sidedata: ][formatvariant.repo.uptodate| no][formatvariant.config.default| no][formatvariant.default| no]
[formatvariant.name.uptodate|plain-cl-delta: ][formatvariant.repo.uptodate| yes][formatvariant.config.default| yes][formatvariant.default| yes] [formatvariant.name.uptodate|plain-cl-delta: ][formatvariant.repo.uptodate| yes][formatvariant.config.default| yes][formatvariant.default| yes]
[formatvariant.name.uptodate|compression: ][formatvariant.repo.uptodate| zlib][formatvariant.config.default| zlib][formatvariant.default| zlib] [formatvariant.name.uptodate|compression: ][formatvariant.repo.uptodate| zlib][formatvariant.config.default| zlib][formatvariant.default| zlib]
[formatvariant.name.uptodate|compression-level:][formatvariant.repo.uptodate| default][formatvariant.config.default| default][formatvariant.default| default] [formatvariant.name.uptodate|compression-level:][formatvariant.repo.uptodate| default][formatvariant.config.default| default][formatvariant.default| default]
$ hg debugformat -Tjson $ hg debugformat -Tjson
[ [
{ {
"config": true, "config": true,
"default": true, "default": true,
}, },
{ {
"config": true, "config": true,
"default": true, "default": true,
"name": "sparserevlog", "name": "sparserevlog",
"repo": true "repo": true
}, },
{ {
"config": false,
"default": false,
"name": "sidedata",
"repo": false
},
{
"config": true, "config": true,
"default": true, "default": true,
"name": "plain-cl-delta", "name": "plain-cl-delta",
"repo": true "repo": true
}, },
{ {
"config": "zlib", "config": "zlib",
"default": "zlib", "default": "zlib",
] ]
$ hg debugupgraderepo $ hg debugupgraderepo
(no feature deficiencies found in existing repository) (no feature deficiencies found in existing repository)
performing an upgrade with "--run" will make the following changes: performing an upgrade with "--run" will make the following changes:
requirements requirements
preserved: dotencode, fncache, generaldelta, revlogv1, sparserevlog, store preserved: dotencode, fncache, generaldelta, revlogv1, sparserevlog, store
sidedata
Allows storage of extra data alongside a revision.
additional optimizations are available by specifying "--optimize <name>": additional optimizations are available by specifying "--optimize <name>":
re-delta-parent re-delta-parent
deltas within internal storage will be recalculated to choose an optimal base revision where this was not already done; the size of the repository may shrink and various operations may become faster; the first time this optimization is performed could slow down upgrade execution considerably; subsequent invocations should not run noticeably slower deltas within internal storage will be recalculated to choose an optimal base revision where this was not already done; the size of the repository may shrink and various operations may become faster; the first time this optimization is performed could slow down upgrade execution considerably; subsequent invocations should not run noticeably slower
re-delta-multibase re-delta-multibase
deltas within internal storage will be recalculated against multiple base revision and the smallest difference will be used; the size of the repository may shrink significantly when there are many merges; this optimization will slow down execution in proportion to the number of merges in the repository and the amount of files in the repository; this slow down should not be significant unless there are tens of thousands of files and thousands of merges deltas within internal storage will be recalculated against multiple base revision and the smallest difference will be used; the size of the repository may shrink significantly when there are many merges; this optimization will slow down execution in proportion to the number of merges in the repository and the amount of files in the repository; this slow down should not be significant unless there are tens of thousands of files and thousands of merges
re-delta-all re-delta-all
deltas within internal storage will always be recalculated without reusing prior deltas; this will likely make execution run several times slower; this optimization is typically not needed deltas within internal storage will always be recalculated without reusing prior deltas; this will likely make execution run several times slower; this optimization is typically not needed
re-delta-fulladd re-delta-fulladd
every revision will be re-added as if it was new content. It will go through the full storage mechanism giving extensions a chance to process it (eg. lfs). This is similar to "re-delta-all" but even slower since more logic is involved. every revision will be re-added as if it was new content. It will go through the full storage mechanism giving extensions a chance to process it (eg. lfs). This is similar to "re-delta-all" but even slower since more logic is involved.
--optimize can be used to add optimizations --optimize can be used to add optimizations
$ hg debugupgrade --optimize redeltaparent $ hg debugupgrade --optimize redeltaparent
(no feature deficiencies found in existing repository) (no feature deficiencies found in existing repository)
performing an upgrade with "--run" will make the following changes: performing an upgrade with "--run" will make the following changes:
requirements requirements
preserved: dotencode, fncache, generaldelta, revlogv1, sparserevlog, store preserved: dotencode, fncache, generaldelta, revlogv1, sparserevlog, store
sidedata
Allows storage of extra data alongside a revision.
re-delta-parent re-delta-parent
deltas within internal storage will choose a new base revision if needed deltas within internal storage will choose a new base revision if needed
additional optimizations are available by specifying "--optimize <name>": additional optimizations are available by specifying "--optimize <name>":
re-delta-multibase re-delta-multibase
deltas within internal storage will be recalculated against multiple base revision and the smallest difference will be used; the size of the repository may shrink significantly when there are many merges; this optimization will slow down execution in proportion to the number of merges in the repository and the amount of files in the repository; this slow down should not be significant unless there are tens of thousands of files and thousands of merges deltas within internal storage will be recalculated against multiple base revision and the smallest difference will be used; the size of the repository may shrink significantly when there are many merges; this optimization will slow down execution in proportion to the number of merges in the repository and the amount of files in the repository; this slow down should not be significant unless there are tens of thousands of files and thousands of merges
re-delta-all re-delta-all
deltas within internal storage will always be recalculated without reusing prior deltas; this will likely make execution run several times slower; this optimization is typically not needed deltas within internal storage will always be recalculated without reusing prior deltas; this will likely make execution run several times slower; this optimization is typically not needed
re-delta-fulladd re-delta-fulladd
every revision will be re-added as if it was new content. It will go through the full storage mechanism giving extensions a chance to process it (eg. lfs). This is similar to "re-delta-all" but even slower since more logic is involved. every revision will be re-added as if it was new content. It will go through the full storage mechanism giving extensions a chance to process it (eg. lfs). This is similar to "re-delta-all" but even slower since more logic is involved.
modern form of the option modern form of the option
$ hg debugupgrade --optimize re-delta-parent $ hg debugupgrade --optimize re-delta-parent
(no feature deficiencies found in existing repository) (no feature deficiencies found in existing repository)
performing an upgrade with "--run" will make the following changes: performing an upgrade with "--run" will make the following changes:
requirements requirements
preserved: dotencode, fncache, generaldelta, revlogv1, sparserevlog, store preserved: dotencode, fncache, generaldelta, revlogv1, sparserevlog, store
sidedata
Allows storage of extra data alongside a revision.
re-delta-parent re-delta-parent
deltas within internal storage will choose a new base revision if needed deltas within internal storage will choose a new base revision if needed
additional optimizations are available by specifying "--optimize <name>": additional optimizations are available by specifying "--optimize <name>":
re-delta-multibase re-delta-multibase
deltas within internal storage will be recalculated against multiple base revision and the smallest difference will be used; the size of the repository may shrink significantly when there are many merges; this optimization will slow down execution in proportion to the number of merges in the repository and the amount of files in the repository; this slow down should not be significant unless there are tens of thousands of files and thousands of merges deltas within internal storage will be recalculated against multiple base revision and the smallest difference will be used; the size of the repository may shrink significantly when there are many merges; this optimization will slow down execution in proportion to the number of merges in the repository and the amount of files in the repository; this slow down should not be significant unless there are tens of thousands of files and thousands of merges
> EOF > EOF
$ hg debugformat $ hg debugformat
format-variant repo format-variant repo
fncache: no fncache: no
dotencode: no dotencode: no
generaldelta: no generaldelta: no
sparserevlog: no sparserevlog: no
sidedata: no
plain-cl-delta: yes plain-cl-delta: yes
compression: zlib compression: zlib
compression-level: default compression-level: default
$ hg debugformat --verbose $ hg debugformat --verbose
format-variant repo config default format-variant repo config default
fncache: no yes yes fncache: no yes yes
dotencode: no yes yes dotencode: no yes yes
generaldelta: no yes yes generaldelta: no yes yes
sparserevlog: no yes yes sparserevlog: no yes yes
sidedata: no no no
plain-cl-delta: yes yes yes plain-cl-delta: yes yes yes
compression: zlib zlib zlib compression: zlib zlib zlib
compression-level: default default default compression-level: default default default
$ hg debugformat --verbose --config format.usegeneraldelta=no $ hg debugformat --verbose --config format.usegeneraldelta=no
format-variant repo config default format-variant repo config default
fncache: no yes yes fncache: no yes yes
dotencode: no yes yes dotencode: no yes yes
generaldelta: no no yes generaldelta: no no yes
sparserevlog: no no yes sparserevlog: no no yes
sidedata: no no no
plain-cl-delta: yes yes yes plain-cl-delta: yes yes yes
compression: zlib zlib zlib compression: zlib zlib zlib
compression-level: default default default compression-level: default default default
$ hg debugformat --verbose --config format.usegeneraldelta=no --color=debug $ hg debugformat --verbose --config format.usegeneraldelta=no --color=debug
format-variant repo config default format-variant repo config default
[formatvariant.name.mismatchconfig|fncache: ][formatvariant.repo.mismatchconfig| no][formatvariant.config.default| yes][formatvariant.default| yes] [formatvariant.name.mismatchconfig|fncache: ][formatvariant.repo.mismatchconfig| no][formatvariant.config.default| yes][formatvariant.default| yes]
[formatvariant.name.mismatchconfig|dotencode: ][formatvariant.repo.mismatchconfig| no][formatvariant.config.default| yes][formatvariant.default| yes] [formatvariant.name.mismatchconfig|dotencode: ][formatvariant.repo.mismatchconfig| no][formatvariant.config.default| yes][formatvariant.default| yes]
[formatvariant.name.mismatchdefault|generaldelta: ][formatvariant.repo.mismatchdefault| no][formatvariant.config.special| no][formatvariant.default| yes] [formatvariant.name.mismatchdefault|generaldelta: ][formatvariant.repo.mismatchdefault| no][formatvariant.config.special| no][formatvariant.default| yes]
[formatvariant.name.mismatchdefault|sparserevlog: ][formatvariant.repo.mismatchdefault| no][formatvariant.config.special| no][formatvariant.default| yes] [formatvariant.name.mismatchdefault|sparserevlog: ][formatvariant.repo.mismatchdefault| no][formatvariant.config.special| no][formatvariant.default| yes]
[formatvariant.name.uptodate|sidedata: ][formatvariant.repo.uptodate| no][formatvariant.config.default| no][formatvariant.default| no]
[formatvariant.name.uptodate|plain-cl-delta: ][formatvariant.repo.uptodate| yes][formatvariant.config.default| yes][formatvariant.default| yes] [formatvariant.name.uptodate|plain-cl-delta: ][formatvariant.repo.uptodate| yes][formatvariant.config.default| yes][formatvariant.default| yes]
[formatvariant.name.uptodate|compression: ][formatvariant.repo.uptodate| zlib][formatvariant.config.default| zlib][formatvariant.default| zlib] [formatvariant.name.uptodate|compression: ][formatvariant.repo.uptodate| zlib][formatvariant.config.default| zlib][formatvariant.default| zlib]
[formatvariant.name.uptodate|compression-level:][formatvariant.repo.uptodate| default][formatvariant.config.default| default][formatvariant.default| default] [formatvariant.name.uptodate|compression-level:][formatvariant.repo.uptodate| default][formatvariant.config.default| default][formatvariant.default| default]
$ hg debugupgraderepo $ hg debugupgraderepo
repository lacks features recommended by current config options: repository lacks features recommended by current config options:
fncache fncache
long and reserved filenames may not work correctly; repository performance is sub-optimal long and reserved filenames may not work correctly; repository performance is sub-optimal
repository will be better able to store files beginning with a space or period repository will be better able to store files beginning with a space or period
generaldelta generaldelta
repository storage will be able to create optimal deltas; new repository data will be smaller and read times should decrease; interacting with other repositories using this storage model should require less network and CPU resources, making "hg push" and "hg pull" faster repository storage will be able to create optimal deltas; new repository data will be smaller and read times should decrease; interacting with other repositories using this storage model should require less network and CPU resources, making "hg push" and "hg pull" faster
sparserevlog sparserevlog
Revlog supports delta chain with more unused data between payload. These gaps will be skipped at read time. This allows for better delta chains, making a better compression and faster exchange with server. Revlog supports delta chain with more unused data between payload. These gaps will be skipped at read time. This allows for better delta chains, making a better compression and faster exchange with server.
sidedata
Allows storage of extra data alongside a revision.
additional optimizations are available by specifying "--optimize <name>": additional optimizations are available by specifying "--optimize <name>":
re-delta-parent re-delta-parent
deltas within internal storage will be recalculated to choose an optimal base revision where this was not already done; the size of the repository may shrink and various operations may become faster; the first time this optimization is performed could slow down upgrade execution considerably; subsequent invocations should not run noticeably slower deltas within internal storage will be recalculated to choose an optimal base revision where this was not already done; the size of the repository may shrink and various operations may become faster; the first time this optimization is performed could slow down upgrade execution considerably; subsequent invocations should not run noticeably slower
re-delta-multibase re-delta-multibase
deltas within internal storage will be recalculated against multiple base revision and the smallest difference will be used; the size of the repository may shrink significantly when there are many merges; this optimization will slow down execution in proportion to the number of merges in the repository and the amount of files in the repository; this slow down should not be significant unless there are tens of thousands of files and thousands of merges deltas within internal storage will be recalculated against multiple base revision and the smallest difference will be used; the size of the repository may shrink significantly when there are many merges; this optimization will slow down execution in proportion to the number of merges in the repository and the amount of files in the repository; this slow down should not be significant unless there are tens of thousands of files and thousands of merges
repository will be more resilient to storing certain paths and performance of certain operations should be improved repository will be more resilient to storing certain paths and performance of certain operations should be improved
generaldelta generaldelta
repository storage will be able to create optimal deltas; new repository data will be smaller and read times should decrease; interacting with other repositories using this storage model should require less network and CPU resources, making "hg push" and "hg pull" faster repository storage will be able to create optimal deltas; new repository data will be smaller and read times should decrease; interacting with other repositories using this storage model should require less network and CPU resources, making "hg push" and "hg pull" faster
sparserevlog sparserevlog
Revlog supports delta chain with more unused data between payload. These gaps will be skipped at read time. This allows for better delta chains, making a better compression and faster exchange with server. Revlog supports delta chain with more unused data between payload. These gaps will be skipped at read time. This allows for better delta chains, making a better compression and faster exchange with server.
sidedata
Allows storage of extra data alongside a revision.
additional optimizations are available by specifying "--optimize <name>": additional optimizations are available by specifying "--optimize <name>":
re-delta-parent re-delta-parent
deltas within internal storage will be recalculated to choose an optimal base revision where this was not already done; the size of the repository may shrink and various operations may become faster; the first time this optimization is performed could slow down upgrade execution considerably; subsequent invocations should not run noticeably slower deltas within internal storage will be recalculated to choose an optimal base revision where this was not already done; the size of the repository may shrink and various operations may become faster; the first time this optimization is performed could slow down upgrade execution considerably; subsequent invocations should not run noticeably slower
re-delta-multibase re-delta-multibase
deltas within internal storage will be recalculated against multiple base revision and the smallest difference will be used; the size of the repository may shrink significantly when there are many merges; this optimization will slow down execution in proportion to the number of merges in the repository and the amount of files in the repository; this slow down should not be significant unless there are tens of thousands of files and thousands of merges deltas within internal storage will be recalculated against multiple base revision and the smallest difference will be used; the size of the repository may shrink significantly when there are many merges; this optimization will slow down execution in proportion to the number of merges in the repository and the amount of files in the repository; this slow down should not be significant unless there are tens of thousands of files and thousands of merges
$ hg init modern $ hg init modern
$ hg -R modern debugupgraderepo --run $ hg -R modern debugupgraderepo --run
upgrade will perform the following actions: upgrade will perform the following actions:
requirements requirements
preserved: dotencode, fncache, generaldelta, revlogv1, sparserevlog, store preserved: dotencode, fncache, generaldelta, revlogv1, sparserevlog, store
sidedata
Allows storage of extra data alongside a revision.
beginning upgrade... beginning upgrade...
repository locked and read-only repository locked and read-only
creating temporary repository to stage migrated data: $TESTTMP/modern/.hg/upgrade.* (glob) creating temporary repository to stage migrated data: $TESTTMP/modern/.hg/upgrade.* (glob)
(it is safe to interrupt this process any time before data migration completes) (it is safe to interrupt this process any time before data migration completes)
data fully migrated to temporary repository data fully migrated to temporary repository
marking source repository as being upgraded; clients will be unable to read from repository marking source repository as being upgraded; clients will be unable to read from repository
starting in-place swap of repository data starting in-place swap of repository data
replaced files will be backed up at $TESTTMP/modern/.hg/upgradebackup.* (glob) replaced files will be backed up at $TESTTMP/modern/.hg/upgradebackup.* (glob)
requirements requirements
preserved: dotencode, fncache, revlogv1, store preserved: dotencode, fncache, revlogv1, store
added: generaldelta added: generaldelta
generaldelta generaldelta
repository storage will be able to create optimal deltas; new repository data will be smaller and read times should decrease; interacting with other repositories using this storage model should require less network and CPU resources, making "hg push" and "hg pull" faster repository storage will be able to create optimal deltas; new repository data will be smaller and read times should decrease; interacting with other repositories using this storage model should require less network and CPU resources, making "hg push" and "hg pull" faster
sidedata
Allows storage of extra data alongside a revision.
beginning upgrade... beginning upgrade...
repository locked and read-only repository locked and read-only
creating temporary repository to stage migrated data: $TESTTMP/upgradegd/.hg/upgrade.* (glob) creating temporary repository to stage migrated data: $TESTTMP/upgradegd/.hg/upgrade.* (glob)
(it is safe to interrupt this process any time before data migration completes) (it is safe to interrupt this process any time before data migration completes)
migrating 9 total revisions (3 in filelogs, 3 in manifests, 3 in changelog) migrating 9 total revisions (3 in filelogs, 3 in manifests, 3 in changelog)
migrating 519 KB in store; 1.05 MB tracked data migrating 519 KB in store; 1.05 MB tracked data
migrating 3 filelogs containing 3 revisions (518 KB in store; 1.05 MB tracked data) migrating 3 filelogs containing 3 revisions (518 KB in store; 1.05 MB tracked data)
finished migrating 3 filelog revisions across 3 filelogs; change in size: 0 bytes finished migrating 3 filelog revisions across 3 filelogs; change in size: 0 bytes
requirements requirements
preserved: dotencode, fncache, generaldelta, revlogv1, store preserved: dotencode, fncache, generaldelta, revlogv1, store
added: sparserevlog added: sparserevlog
sparserevlog sparserevlog
Revlog supports delta chain with more unused data between payload. These gaps will be skipped at read time. This allows for better delta chains, making a better compression and faster exchange with server. Revlog supports delta chain with more unused data between payload. These gaps will be skipped at read time. This allows for better delta chains, making a better compression and faster exchange with server.
sidedata
Allows storage of extra data alongside a revision.
beginning upgrade... beginning upgrade...
repository locked and read-only repository locked and read-only
creating temporary repository to stage migrated data: $TESTTMP/upgradegd/.hg/upgrade.* (glob) creating temporary repository to stage migrated data: $TESTTMP/upgradegd/.hg/upgrade.* (glob)
(it is safe to interrupt this process any time before data migration completes) (it is safe to interrupt this process any time before data migration completes)
migrating 9 total revisions (3 in filelogs, 3 in manifests, 3 in changelog) migrating 9 total revisions (3 in filelogs, 3 in manifests, 3 in changelog)
migrating 519 KB in store; 1.05 MB tracked data migrating 519 KB in store; 1.05 MB tracked data
migrating 3 filelogs containing 3 revisions (518 KB in store; 1.05 MB tracked data) migrating 3 filelogs containing 3 revisions (518 KB in store; 1.05 MB tracked data)
finished migrating 3 filelog revisions across 3 filelogs; change in size: 0 bytes finished migrating 3 filelog revisions across 3 filelogs; change in size: 0 bytes
We can restrict optimization to some revlog: We can restrict optimization to some revlog:
$ hg debugupgrade --optimize re-delta-parent --run --manifest --no-backup --debug --traceback $ hg debugupgrade --optimize re-delta-parent --run --manifest --no-backup --debug --traceback
upgrade will perform the following actions: upgrade will perform the following actions:
requirements requirements
preserved: dotencode, fncache, generaldelta, revlogv1, sparserevlog, store preserved: dotencode, fncache, generaldelta, revlogv1, sparserevlog, store
sidedata
Allows storage of extra data alongside a revision.
re-delta-parent re-delta-parent
deltas within internal storage will choose a new base revision if needed deltas within internal storage will choose a new base revision if needed
beginning upgrade... beginning upgrade...
repository locked and read-only repository locked and read-only
creating temporary repository to stage migrated data: $TESTTMP/upgradegd/.hg/upgrade.* (glob) creating temporary repository to stage migrated data: $TESTTMP/upgradegd/.hg/upgrade.* (glob)
(it is safe to interrupt this process any time before data migration completes) (it is safe to interrupt this process any time before data migration completes)
migrating 9 total revisions (3 in filelogs, 3 in manifests, 3 in changelog) migrating 9 total revisions (3 in filelogs, 3 in manifests, 3 in changelog)
Check we can select negatively Check we can select negatively
$ hg debugupgrade --optimize re-delta-parent --run --no-manifest --no-backup --debug --traceback $ hg debugupgrade --optimize re-delta-parent --run --no-manifest --no-backup --debug --traceback
upgrade will perform the following actions: upgrade will perform the following actions:
requirements requirements
preserved: dotencode, fncache, generaldelta, revlogv1, sparserevlog, store preserved: dotencode, fncache, generaldelta, revlogv1, sparserevlog, store
sidedata
Allows storage of extra data alongside a revision.
re-delta-parent re-delta-parent
deltas within internal storage will choose a new base revision if needed deltas within internal storage will choose a new base revision if needed
beginning upgrade... beginning upgrade...
repository locked and read-only repository locked and read-only
creating temporary repository to stage migrated data: $TESTTMP/upgradegd/.hg/upgrade.* (glob) creating temporary repository to stage migrated data: $TESTTMP/upgradegd/.hg/upgrade.* (glob)
(it is safe to interrupt this process any time before data migration completes) (it is safe to interrupt this process any time before data migration completes)
migrating 9 total revisions (3 in filelogs, 3 in manifests, 3 in changelog) migrating 9 total revisions (3 in filelogs, 3 in manifests, 3 in changelog)
Check that we can select changelog only Check that we can select changelog only
$ hg debugupgrade --optimize re-delta-parent --run --changelog --no-backup --debug --traceback $ hg debugupgrade --optimize re-delta-parent --run --changelog --no-backup --debug --traceback
upgrade will perform the following actions: upgrade will perform the following actions:
requirements requirements
preserved: dotencode, fncache, generaldelta, revlogv1, sparserevlog, store preserved: dotencode, fncache, generaldelta, revlogv1, sparserevlog, store
sidedata
Allows storage of extra data alongside a revision.
re-delta-parent re-delta-parent
deltas within internal storage will choose a new base revision if needed deltas within internal storage will choose a new base revision if needed
beginning upgrade... beginning upgrade...
repository locked and read-only repository locked and read-only
creating temporary repository to stage migrated data: $TESTTMP/upgradegd/.hg/upgrade.* (glob) creating temporary repository to stage migrated data: $TESTTMP/upgradegd/.hg/upgrade.* (glob)
(it is safe to interrupt this process any time before data migration completes) (it is safe to interrupt this process any time before data migration completes)
migrating 9 total revisions (3 in filelogs, 3 in manifests, 3 in changelog) migrating 9 total revisions (3 in filelogs, 3 in manifests, 3 in changelog)
Check that we can select filelog only Check that we can select filelog only
$ hg debugupgrade --optimize re-delta-parent --run --no-changelog --no-manifest --no-backup --debug --traceback $ hg debugupgrade --optimize re-delta-parent --run --no-changelog --no-manifest --no-backup --debug --traceback
upgrade will perform the following actions: upgrade will perform the following actions:
requirements requirements
preserved: dotencode, fncache, generaldelta, revlogv1, sparserevlog, store preserved: dotencode, fncache, generaldelta, revlogv1, sparserevlog, store
sidedata
Allows storage of extra data alongside a revision.
re-delta-parent re-delta-parent
deltas within internal storage will choose a new base revision if needed deltas within internal storage will choose a new base revision if needed
beginning upgrade... beginning upgrade...
repository locked and read-only repository locked and read-only
creating temporary repository to stage migrated data: $TESTTMP/upgradegd/.hg/upgrade.* (glob) creating temporary repository to stage migrated data: $TESTTMP/upgradegd/.hg/upgrade.* (glob)
(it is safe to interrupt this process any time before data migration completes) (it is safe to interrupt this process any time before data migration completes)
migrating 9 total revisions (3 in filelogs, 3 in manifests, 3 in changelog) migrating 9 total revisions (3 in filelogs, 3 in manifests, 3 in changelog)
$ hg debugupgrade --optimize re-delta-parent --run --manifest --no-backup --debug --traceback $ hg debugupgrade --optimize re-delta-parent --run --manifest --no-backup --debug --traceback
ignoring revlogs selection flags, format requirements change: sparserevlog ignoring revlogs selection flags, format requirements change: sparserevlog
upgrade will perform the following actions: upgrade will perform the following actions:
requirements requirements
preserved: dotencode, fncache, generaldelta, revlogv1, store preserved: dotencode, fncache, generaldelta, revlogv1, store
removed: sparserevlog removed: sparserevlog
sidedata
Allows storage of extra data alongside a revision.
re-delta-parent re-delta-parent
deltas within internal storage will choose a new base revision if needed deltas within internal storage will choose a new base revision if needed
beginning upgrade... beginning upgrade...
repository locked and read-only repository locked and read-only
creating temporary repository to stage migrated data: $TESTTMP/upgradegd/.hg/upgrade.* (glob) creating temporary repository to stage migrated data: $TESTTMP/upgradegd/.hg/upgrade.* (glob)
(it is safe to interrupt this process any time before data migration completes) (it is safe to interrupt this process any time before data migration completes)
migrating 9 total revisions (3 in filelogs, 3 in manifests, 3 in changelog) migrating 9 total revisions (3 in filelogs, 3 in manifests, 3 in changelog)
requirements requirements
preserved: dotencode, fncache, generaldelta, revlogv1, store preserved: dotencode, fncache, generaldelta, revlogv1, store
added: sparserevlog added: sparserevlog
sparserevlog sparserevlog
Revlog supports delta chain with more unused data between payload. These gaps will be skipped at read time. This allows for better delta chains, making a better compression and faster exchange with server. Revlog supports delta chain with more unused data between payload. These gaps will be skipped at read time. This allows for better delta chains, making a better compression and faster exchange with server.
sidedata
Allows storage of extra data alongside a revision.
re-delta-parent re-delta-parent
deltas within internal storage will choose a new base revision if needed deltas within internal storage will choose a new base revision if needed
beginning upgrade... beginning upgrade...
repository locked and read-only repository locked and read-only
creating temporary repository to stage migrated data: $TESTTMP/upgradegd/.hg/upgrade.* (glob) creating temporary repository to stage migrated data: $TESTTMP/upgradegd/.hg/upgrade.* (glob)
(it is safe to interrupt this process any time before data migration completes) (it is safe to interrupt this process any time before data migration completes)
migrating 9 total revisions (3 in filelogs, 3 in manifests, 3 in changelog) migrating 9 total revisions (3 in filelogs, 3 in manifests, 3 in changelog)
$ touch .hg/store/.XX_special_filename $ touch .hg/store/.XX_special_filename
$ hg debugupgraderepo --run $ hg debugupgraderepo --run
upgrade will perform the following actions: upgrade will perform the following actions:
requirements requirements
preserved: dotencode, fncache, generaldelta, revlogv1, sparserevlog, store preserved: dotencode, fncache, generaldelta, revlogv1, sparserevlog, store
sidedata
Allows storage of extra data alongside a revision.
beginning upgrade... beginning upgrade...
repository locked and read-only repository locked and read-only
creating temporary repository to stage migrated data: $TESTTMP/store-filenames/.hg/upgrade.* (glob) creating temporary repository to stage migrated data: $TESTTMP/store-filenames/.hg/upgrade.* (glob)
(it is safe to interrupt this process any time before data migration completes) (it is safe to interrupt this process any time before data migration completes)
migrating 3 total revisions (1 in filelogs, 1 in manifests, 1 in changelog) migrating 3 total revisions (1 in filelogs, 1 in manifests, 1 in changelog)
migrating 301 bytes in store; 107 bytes tracked data migrating 301 bytes in store; 107 bytes tracked data
migrating 1 filelogs containing 1 revisions (64 bytes in store; 0 bytes tracked data) migrating 1 filelogs containing 1 revisions (64 bytes in store; 0 bytes tracked data)
finished migrating 1 filelog revisions across 1 filelogs; change in size: 0 bytes finished migrating 1 filelog revisions across 1 filelogs; change in size: 0 bytes
copy of old repository backed up at $TESTTMP/store-filenames/.hg/upgradebackup.* (glob) copy of old repository backed up at $TESTTMP/store-filenames/.hg/upgradebackup.* (glob)
the old repository will not be deleted; remove it to free up disk space once the upgraded repository is verified the old repository will not be deleted; remove it to free up disk space once the upgraded repository is verified
$ hg debugupgraderepo --run --optimize redeltafulladd $ hg debugupgraderepo --run --optimize redeltafulladd
upgrade will perform the following actions: upgrade will perform the following actions:
requirements requirements
preserved: dotencode, fncache, generaldelta, revlogv1, sparserevlog, store preserved: dotencode, fncache, generaldelta, revlogv1, sparserevlog, store
sidedata
Allows storage of extra data alongside a revision.
re-delta-fulladd re-delta-fulladd
each revision will be added as new content to the internal storage; this will likely drastically slow down execution time, but some extensions might need it each revision will be added as new content to the internal storage; this will likely drastically slow down execution time, but some extensions might need it
beginning upgrade... beginning upgrade...
repository locked and read-only repository locked and read-only
creating temporary repository to stage migrated data: $TESTTMP/store-filenames/.hg/upgrade.* (glob) creating temporary repository to stage migrated data: $TESTTMP/store-filenames/.hg/upgrade.* (glob)
(it is safe to interrupt this process any time before data migration completes) (it is safe to interrupt this process any time before data migration completes)
migrating 3 total revisions (1 in filelogs, 1 in manifests, 1 in changelog) migrating 3 total revisions (1 in filelogs, 1 in manifests, 1 in changelog)
store store
$ hg debugupgraderepo --run $ hg debugupgraderepo --run
upgrade will perform the following actions: upgrade will perform the following actions:
requirements requirements
preserved: dotencode, fncache, generaldelta, largefiles, revlogv1, sparserevlog, store preserved: dotencode, fncache, generaldelta, largefiles, revlogv1, sparserevlog, store
sidedata
Allows storage of extra data alongside a revision.
beginning upgrade... beginning upgrade...
repository locked and read-only repository locked and read-only
creating temporary repository to stage migrated data: $TESTTMP/largefilesrepo/.hg/upgrade.* (glob) creating temporary repository to stage migrated data: $TESTTMP/largefilesrepo/.hg/upgrade.* (glob)
(it is safe to interrupt this process any time before data migration completes) (it is safe to interrupt this process any time before data migration completes)
migrating 3 total revisions (1 in filelogs, 1 in manifests, 1 in changelog) migrating 3 total revisions (1 in filelogs, 1 in manifests, 1 in changelog)
migrating 355 bytes in store; 160 bytes tracked data migrating 355 bytes in store; 160 bytes tracked data
migrating 1 filelogs containing 1 revisions (106 bytes in store; 41 bytes tracked data) migrating 1 filelogs containing 1 revisions (106 bytes in store; 41 bytes tracked data)
finished migrating 1 filelog revisions across 1 filelogs; change in size: 0 bytes finished migrating 1 filelog revisions across 1 filelogs; change in size: 0 bytes
.hg/store/lfs/objects/d0/beab232adff5ba365880366ad30b1edb85c4c5372442b5d2fe27adc96d653f .hg/store/lfs/objects/d0/beab232adff5ba365880366ad30b1edb85c4c5372442b5d2fe27adc96d653f
$ hg debugupgraderepo --run $ hg debugupgraderepo --run
upgrade will perform the following actions: upgrade will perform the following actions:
requirements requirements
preserved: dotencode, fncache, generaldelta, largefiles, lfs, revlogv1, sparserevlog, store preserved: dotencode, fncache, generaldelta, largefiles, lfs, revlogv1, sparserevlog, store
sidedata
Allows storage of extra data alongside a revision.
beginning upgrade... beginning upgrade...
repository locked and read-only repository locked and read-only
creating temporary repository to stage migrated data: $TESTTMP/largefilesrepo/.hg/upgrade.* (glob) creating temporary repository to stage migrated data: $TESTTMP/largefilesrepo/.hg/upgrade.* (glob)
(it is safe to interrupt this process any time before data migration completes) (it is safe to interrupt this process any time before data migration completes)
migrating 6 total revisions (2 in filelogs, 2 in manifests, 2 in changelog) migrating 6 total revisions (2 in filelogs, 2 in manifests, 2 in changelog)
migrating 801 bytes in store; 467 bytes tracked data migrating 801 bytes in store; 467 bytes tracked data
migrating 2 filelogs containing 2 revisions (296 bytes in store; 182 bytes tracked data) migrating 2 filelogs containing 2 revisions (296 bytes in store; 182 bytes tracked data)
finished migrating 2 filelog revisions across 2 filelogs; change in size: 0 bytes finished migrating 2 filelog revisions across 2 filelogs; change in size: 0 bytes
2 1 2 0 other 30 200 107 0.53500 128 21 0.19626 128 128 0.83594 1 2 1 2 0 other 30 200 107 0.53500 128 21 0.19626 128 128 0.83594 1
$ hg debugupgraderepo --run --optimize redeltaall $ hg debugupgraderepo --run --optimize redeltaall
upgrade will perform the following actions: upgrade will perform the following actions:
requirements requirements
preserved: dotencode, fncache, generaldelta, revlogv1, sparserevlog, store preserved: dotencode, fncache, generaldelta, revlogv1, sparserevlog, store
sidedata
Allows storage of extra data alongside a revision.
re-delta-all re-delta-all
deltas within internal storage will be fully recomputed; this will likely drastically slow down execution time deltas within internal storage will be fully recomputed; this will likely drastically slow down execution time
beginning upgrade... beginning upgrade...
repository locked and read-only repository locked and read-only
creating temporary repository to stage migrated data: $TESTTMP/localconfig/.hg/upgrade.* (glob) creating temporary repository to stage migrated data: $TESTTMP/localconfig/.hg/upgrade.* (glob)
(it is safe to interrupt this process any time before data migration completes) (it is safe to interrupt this process any time before data migration completes)
migrating 9 total revisions (3 in filelogs, 3 in manifests, 3 in changelog) migrating 9 total revisions (3 in filelogs, 3 in manifests, 3 in changelog)
$ hg --config format.revlog-compression=zstd debugupgraderepo --run --no-backup >/dev/null $ hg --config format.revlog-compression=zstd debugupgraderepo --run --no-backup >/dev/null
$ hg debugformat -v $ hg debugformat -v
format-variant repo config default format-variant repo config default
fncache: yes yes yes fncache: yes yes yes
dotencode: yes yes yes dotencode: yes yes yes
generaldelta: yes yes yes generaldelta: yes yes yes
sparserevlog: yes yes yes sparserevlog: yes yes yes
sidedata: no no no
plain-cl-delta: yes yes yes plain-cl-delta: yes yes yes
compression: zstd zlib zlib compression: zstd zlib zlib
compression-level: default default default compression-level: default default default
$ cat .hg/requires $ cat .hg/requires
dotencode dotencode
fncache fncache
generaldelta generaldelta
revlog-compression-zstd revlog-compression-zstd
revlogv1 revlogv1
sparserevlog sparserevlog
store store
downgrade downgrade
$ hg debugupgraderepo --run --no-backup > /dev/null $ hg debugupgraderepo --run --no-backup > /dev/null
$ hg debugformat -v $ hg debugformat -v
format-variant repo config default format-variant repo config default
fncache: yes yes yes fncache: yes yes yes
dotencode: yes yes yes dotencode: yes yes yes
generaldelta: yes yes yes generaldelta: yes yes yes
sparserevlog: yes yes yes sparserevlog: yes yes yes
sidedata: no no no
plain-cl-delta: yes yes yes plain-cl-delta: yes yes yes
compression: zlib zlib zlib compression: zlib zlib zlib
compression-level: default default default compression-level: default default default
$ cat .hg/requires $ cat .hg/requires
dotencode dotencode
fncache fncache
generaldelta generaldelta
revlogv1 revlogv1
sparserevlog sparserevlog
store store
upgrade from hgrc upgrade from hgrc
$ cat >> .hg/hgrc << EOF $ cat >> .hg/hgrc << EOF
> [format] > [format]
> revlog-compression=zstd > revlog-compression=zstd
> EOF > EOF
$ hg debugupgraderepo --run --no-backup > /dev/null $ hg debugupgraderepo --run --no-backup > /dev/null
$ hg debugformat -v $ hg debugformat -v
format-variant repo config default format-variant repo config default
fncache: yes yes yes fncache: yes yes yes
dotencode: yes yes yes dotencode: yes yes yes
generaldelta: yes yes yes generaldelta: yes yes yes
sparserevlog: yes yes yes sparserevlog: yes yes yes
sidedata: no no no
plain-cl-delta: yes yes yes plain-cl-delta: yes yes yes
compression: zstd zstd zlib compression: zstd zstd zlib
compression-level: default default default compression-level: default default default
$ cat .hg/requires $ cat .hg/requires
dotencode dotencode
fncache fncache
generaldelta generaldelta
revlog-compression-zstd revlog-compression-zstd
revlogv1 revlogv1
sparserevlog sparserevlog
store store
$ cd .. $ cd ..
#endif #endif