This is an archive of the discontinued Mercurial Phabricator instance.

Differential D8148

revlog-compression: update the config to be a list
ClosedPublic

Authored by marmoute on Feb 25 2020, 5:15 AM.

Download Raw Diff

Details

Reviewers

mharbison72

Group Reviewers

hg-reviewers

Commits

rHGf0027a3dd7cb: revlog-compression: update the config to be a list
rHG54b9238c27be: revlog-compression: update the config to be a list

Summary

format.revlog-compression is now a list of engine, the first supported one is to
be used. Doing this have several benefits:

this is fully backward compatible, config using a single entry will be read

as a single item list, not changing any behavior.

This open the way to use zstd by default without impacting platform were it

is not available. This will be done in a later changesets.

Using zstd provide a significant performance boost explained in : bb271ec2fbfb.
However zstd is not available in some cases, A notable example is the --pure
version of Mercurial which doesn't come with zstd support.

Diff Detail

Repository

rHG Mercurial

Lint

Automatic diff as part of commit; lint not applicable.

Unit

Automatic diff as part of commit; unit tests not applicable.

Event Timeline

marmoute created this revision.Feb 25 2020, 5:15 AM

Herald added a reviewer: hg-reviewers. · View Herald TranscriptFeb 25 2020, 5:15 AM

Herald added a subscriber: mercurial-devel. · View Herald Transcript

Looks good to me, can you add an entry to relnotes/next?

mharbison72 requested changes to this revision.Feb 25 2020, 12:45 PM

mharbison72 added a subscriber: mharbison72.

mharbison72 added inline comments.

mercurial/helptext/config.txt
891–892	s/value/values/
892	s/wille/will/
tests/test-repo-compengines.t
25	Nit: It might help readability (especially with a list) if the engines were delimited with quotes or parenthesis or something. It's mostly obvious here, but not if it's in a config file. (And actually, I missed that "unknown" was used above, and thought it was an awkward sentence at first.)

This revision now requires changes to proceed.Feb 25 2020, 12:45 PM

marmoute marked an inline comment as done.Feb 25 2020, 6:18 PM

marmoute updated this revision to Diff 20303.

LGTM, thanks for the update.

In D8148#121295, @pulkit wrote:

Looks good to me, can you add an entry to relnotes/next?

still missing the release notes entry.

marmoute added a commit: rHG54b9238c27be: revlog-compression: update the config to be a list.Feb 26 2020, 10:32 AM

This revision was not accepted when it landed; it landed in state Needs Review.

Closed by commit rHG54b9238c27be: revlog-compression: update the config to be a list (authored by marmoute). · Explain Why

This revision was automatically updated to reflect the committed changes.

marmoute added a commit: rHGf0027a3dd7cb: revlog-compression: update the config to be a list.Feb 26 2020, 11:52 AM

Really glad for this online alarmsinwindows10 amazing post forever

Herald added a subscriber: mercurial-patches. · View Herald TranscriptOct 21 2021, 4:04 AM

Revision Contents
Changeset List

			Path	Packages
M			mercurial/configitems.py (2 lines)
M			mercurial/helptext/config.txt (3 lines)
M			mercurial/localrepo.py (13 lines)
M			mercurial/upgrade.py (9 lines)
M			tests/test-repo-compengines.t (7 lines)

Diff	ID	Description	Created	Lint	Unit
Base		Base
Diff 1	20294		Feb 25 2020, 5:14 AM	★	★
Diff 2	20303		Feb 25 2020, 6:18 PM	★	★
Diff 3	20307	rHG54b9238c27bee7a4cd316d57e2ba9a78c53f0910	Feb 19 2020, 11:30 AM	★	★

Diff 20307

mercurial/configitems.py

	b'format', b'obsstore-version', default=None,			b'format', b'obsstore-version', default=None,
	)			)
	coreconfigitem(			coreconfigitem(
	b'format', b'sparse-revlog', default=True,			b'format', b'sparse-revlog', default=True,
	)			)
	coreconfigitem(			coreconfigitem(
	b'format',			b'format',
	b'revlog-compression',			b'revlog-compression',
	default=b'zlib',			default=lambda: [b'zlib'],
	alias=[(b'experimental', b'format.compression')],			alias=[(b'experimental', b'format.compression')],
	)			)
	coreconfigitem(			coreconfigitem(
	b'format', b'usefncache', default=True,			b'format', b'usefncache', default=True,
	)			)
	coreconfigitem(			coreconfigitem(
	b'format', b'usegeneraldelta', default=True,			b'format', b'usegeneraldelta', default=True,
	)			)

mercurial/helptext/config.txt

	Repositories with this on-disk format require Mercurial version 4.7			Repositories with this on-disk format require Mercurial version 4.7

	Enabled by default.			Enabled by default.

	``revlog-compression``			``revlog-compression``
	Compression algorithm used by revlog. Supported values are `zlib` and			Compression algorithm used by revlog. Supported values are `zlib` and
	`zstd`. The `zlib` engine is the historical default of Mercurial. `zstd` is			`zstd`. The `zlib` engine is the historical default of Mercurial. `zstd` is
	a newer format that is usually a net win over `zlib`, operating faster at			a newer format that is usually a net win over `zlib`, operating faster at
	better compression rates. Use `zstd` to reduce CPU usage.			better compression rates. Use `zstd` to reduce CPU usage. Multiple values
				can be specified, the first available one will be used.
				mharbison72Unsubmitted Not Done s/value/values/ mharbison72: s/value/values/
				mharbison72Unsubmitted Not Done s/wille/will/ mharbison72: s/wille/will/

	On some systems, the Mercurial installation may lack `zstd` support.			On some systems, the Mercurial installation may lack `zstd` support.

	Default is `zlib`.			Default is `zlib`.

	``bookmarks-in-store``			``bookmarks-in-store``
	Store bookmarks in .hg/store/. This means that bookmarks are shared when			Store bookmarks in .hg/store/. This means that bookmarks are shared when
	using `hg share` regardless of the `-B` option.			using `hg share` regardless of the `-B` option.

mercurial/localrepo.py

	requirements = {b'revlogv1'}			requirements = {b'revlogv1'}
	if ui.configbool(b'format', b'usestore'):			if ui.configbool(b'format', b'usestore'):
	requirements.add(b'store')			requirements.add(b'store')
	if ui.configbool(b'format', b'usefncache'):			if ui.configbool(b'format', b'usefncache'):
	requirements.add(b'fncache')			requirements.add(b'fncache')
	if ui.configbool(b'format', b'dotencode'):			if ui.configbool(b'format', b'dotencode'):
	requirements.add(b'dotencode')			requirements.add(b'dotencode')

	compengine = ui.config(b'format', b'revlog-compression')			compengines = ui.configlist(b'format', b'revlog-compression')
	if compengine not in util.compengines:			for compengine in compengines:
				if compengine in util.compengines:
				break
				else:
	raise error.Abort(			raise error.Abort(
	_(			_(
	b'compression engine %s defined by '			b'compression engines %s defined by '
	b'format.revlog-compression not available'			b'format.revlog-compression not available'
	)			)
	% compengine,			% b', '.join(b'"%s"' % e for e in compengines),
	hint=_(			hint=_(
	b'run "hg debuginstall" to list available '			b'run "hg debuginstall" to list available '
	b'compression engines'			b'compression engines'
	),			),
	)			)

	# zlib is the historical default and doesn't need an explicit requirement.			# zlib is the historical default and doesn't need an explicit requirement.
	elif compengine == b'zstd':			if compengine == b'zstd':
	requirements.add(b'revlog-compression-zstd')			requirements.add(b'revlog-compression-zstd')
	elif compengine != b'zlib':			elif compengine != b'zlib':
	requirements.add(b'exp-compression-%s' % compengine)			requirements.add(b'exp-compression-%s' % compengine)

	if scmutil.gdinitconfig(ui):			if scmutil.gdinitconfig(ui):
	requirements.add(b'generaldelta')			requirements.add(b'generaldelta')
	if ui.configbool(b'format', b'sparse-revlog'):			if ui.configbool(b'format', b'sparse-revlog'):
	requirements.add(SPARSEREVLOG_REQUIREMENT)			requirements.add(SPARSEREVLOG_REQUIREMENT)

mercurial/upgrade.py

	for req in repo.requirements:			for req in repo.requirements:
	prefix = req.startswith			prefix = req.startswith
	if prefix(b'revlog-compression-') or prefix(b'exp-compression-'):			if prefix(b'revlog-compression-') or prefix(b'exp-compression-'):
	compression = req.split(b'-', 2)[2]			compression = req.split(b'-', 2)[2]
	return compression			return compression

	@classmethod			@classmethod
	def fromconfig(cls, repo):			def fromconfig(cls, repo):
	return repo.ui.config(b'format', b'revlog-compression')			compengines = repo.ui.configlist(b'format', b'revlog-compression')
				# return the first valid value as the selection code would do
				for comp in compengines:
				if comp in util.compengines:
				return comp

				# no valide compression found lets display it all for clarity
				return b','.join(compengines)


	@registerformatvariant			@registerformatvariant
	class compressionlevel(formatvariant):			class compressionlevel(formatvariant):
	name = b'compression-level'			name = b'compression-level'
	default = b'default'			default = b'default'

	description = _(b'compression level')			description = _(b'compression level')

tests/test-repo-compengines.t

	0x78 (x) : 1 (100.00%)			0x78 (x) : 1 (100.00%)
	0x78 (x) : 110 (100.00%)			0x78 (x) : 110 (100.00%)

	$ cd ..			$ cd ..

	Unknown compression engine to format.compression aborts			Unknown compression engine to format.compression aborts

	$ hg --config format.revlog-compression=unknown init unknown			$ hg --config format.revlog-compression=unknown init unknown
	abort: compression engine unknown defined by format.revlog-compression not available			abort: compression engines "unknown" defined by format.revlog-compression not available
				mharbison72Unsubmitted Done Nit: It might help readability (especially with a list) if the engines were delimited with quotes or parenthesis or something. It's mostly obvious here, but not if it's in a config file. (And actually, I missed that "unknown" was used above, and thought it was an awkward sentence at first.) mharbison72: Nit: It might help readability (especially with a list) if the engines were delimited with…
	(run "hg debuginstall" to list available compression engines)			(run "hg debuginstall" to list available compression engines)
	[255]			[255]

				unknown compression engine in a list with known one works fine

				$ hg --config format.revlog-compression=zlib,unknown init zlib-before-unknow
				$ hg --config format.revlog-compression=unknown,zlib init unknown-before-zlib

	A requirement specifying an unknown compression engine results in bail			A requirement specifying an unknown compression engine results in bail

	$ hg init unknownrequirement			$ hg init unknownrequirement
	$ cd unknownrequirement			$ cd unknownrequirement
	$ echo exp-compression-unknown >> .hg/requires			$ echo exp-compression-unknown >> .hg/requires
	$ hg log			$ hg log
	abort: repository requires features unknown to this Mercurial: exp-compression-unknown!			abort: repository requires features unknown to this Mercurial: exp-compression-unknown!
	(see https://mercurial-scm.org/wiki/MissingRequirement for more information)			(see https://mercurial-scm.org/wiki/MissingRequirement for more information)