This is an archive of the discontinued Mercurial Phabricator instance.

Differential D8148

revlog-compression: update the config to be a list
ClosedPublic

Authored by marmoute on Feb 25 2020, 5:15 AM.

Download Raw Diff

Details

Reviewers

mharbison72

Group Reviewers

hg-reviewers

Commits

rHGf0027a3dd7cb: revlog-compression: update the config to be a list
rHG54b9238c27be: revlog-compression: update the config to be a list

Summary

format.revlog-compression is now a list of engine, the first supported one is to
be used. Doing this have several benefits:

this is fully backward compatible, config using a single entry will be read

as a single item list, not changing any behavior.

This open the way to use zstd by default without impacting platform were it

is not available. This will be done in a later changesets.

Using zstd provide a significant performance boost explained in : bb271ec2fbfb.
However zstd is not available in some cases, A notable example is the --pure
version of Mercurial which doesn't come with zstd support.

Diff Detail

Repository

rHG Mercurial

Branch

default

Lint

No Linters Available

Unit

No Unit Test Coverage

Event Timeline

marmoute created this revision.Feb 25 2020, 5:15 AM

Herald added a reviewer: hg-reviewers. · View Herald TranscriptFeb 25 2020, 5:15 AM

Herald added a subscriber: mercurial-devel. · View Herald Transcript

Looks good to me, can you add an entry to relnotes/next?

mharbison72 requested changes to this revision.Feb 25 2020, 12:45 PM

mharbison72 added a subscriber: mharbison72.

mharbison72 added inline comments.

mercurial/helptext/config.txt
891–892	s/value/values/
892	s/wille/will/
tests/test-repo-compengines.t
25	Nit: It might help readability (especially with a list) if the engines were delimited with quotes or parenthesis or something. It's mostly obvious here, but not if it's in a config file. (And actually, I missed that "unknown" was used above, and thought it was an awkward sentence at first.)

This revision now requires changes to proceed.Feb 25 2020, 12:45 PM

marmoute marked an inline comment as done.Feb 25 2020, 6:18 PM

marmoute updated this revision to Diff 20303.

LGTM, thanks for the update.

In D8148#121295, @pulkit wrote:

Looks good to me, can you add an entry to relnotes/next?

still missing the release notes entry.

marmoute added a commit: rHG54b9238c27be: revlog-compression: update the config to be a list.Feb 26 2020, 10:32 AM

This revision was not accepted when it landed; it landed in state Needs Review.

Closed by commit rHG54b9238c27be: revlog-compression: update the config to be a list (authored by marmoute). · Explain Why

This revision was automatically updated to reflect the committed changes.

marmoute added a commit: rHGf0027a3dd7cb: revlog-compression: update the config to be a list.Feb 26 2020, 11:52 AM

Really glad for this online alarmsinwindows10 amazing post forever

Herald added a subscriber: mercurial-patches. · View Herald TranscriptOct 21 2021, 4:04 AM

Revision Contents
Changeset List

			Path	Packages
M			mercurial/configitems.py (2 lines)
M			mercurial/helptext/config.txt (3 lines)
M			mercurial/localrepo.py (13 lines)
M			mercurial/upgrade.py (9 lines)
M			tests/test-repo-compengines.t (7 lines)

Diff	ID	Description	Created	Lint	Unit
Base		Base
Diff 1	20294		Feb 25 2020, 5:14 AM	★	★
Diff 2	20303		Feb 25 2020, 6:18 PM	★	★
Diff 3	20307	rHG54b9238c27bee7a4cd316d57e2ba9a78c53f0910	Feb 19 2020, 11:30 AM	★	★

Commit	Parents	Author	Summary	Date
ac887de593a4	c089a0947f3e	Pierre-Yves David		Feb 19 2020, 11:30 AM

Diff 20303

mercurial/configitems.py

	b'format', b'obsstore-version', default=None,			b'format', b'obsstore-version', default=None,
	)			)
	coreconfigitem(			coreconfigitem(
	b'format', b'sparse-revlog', default=True,			b'format', b'sparse-revlog', default=True,
	)			)
	coreconfigitem(			coreconfigitem(
	b'format',			b'format',
	b'revlog-compression',			b'revlog-compression',
	default=b'zlib',			default=lambda: [b'zlib'],
	alias=[(b'experimental', b'format.compression')],			alias=[(b'experimental', b'format.compression')],
	)			)
	coreconfigitem(			coreconfigitem(
	b'format', b'usefncache', default=True,			b'format', b'usefncache', default=True,
	)			)
	coreconfigitem(			coreconfigitem(
	b'format', b'usegeneraldelta', default=True,			b'format', b'usegeneraldelta', default=True,
	)			)

mercurial/helptext/config.txt

	Repositories with this on-disk format require Mercurial version 4.7			Repositories with this on-disk format require Mercurial version 4.7

	Enabled by default.			Enabled by default.

	``revlog-compression``			``revlog-compression``
	Compression algorithm used by revlog. Supported values are `zlib` and			Compression algorithm used by revlog. Supported values are `zlib` and
	`zstd`. The `zlib` engine is the historical default of Mercurial. `zstd` is			`zstd`. The `zlib` engine is the historical default of Mercurial. `zstd` is
	a newer format that is usually a net win over `zlib`, operating faster at			a newer format that is usually a net win over `zlib`, operating faster at
	better compression rates. Use `zstd` to reduce CPU usage.			better compression rates. Use `zstd` to reduce CPU usage. Multiple values
				can be specified, the first available one will be used.
				mharbison72Unsubmitted Not Done s/value/values/ mharbison72: s/value/values/
				mharbison72Unsubmitted Not Done s/wille/will/ mharbison72: s/wille/will/

	On some systems, the Mercurial installation may lack `zstd` support.			On some systems, the Mercurial installation may lack `zstd` support.

	Default is `zlib`.			Default is `zlib`.

	``bookmarks-in-store``			``bookmarks-in-store``
	Store bookmarks in .hg/store/. This means that bookmarks are shared when			Store bookmarks in .hg/store/. This means that bookmarks are shared when
	using `hg share` regardless of the `-B` option.			using `hg share` regardless of the `-B` option.

mercurial/localrepo.py

	requirements = {b'revlogv1'}			requirements = {b'revlogv1'}
	if ui.configbool(b'format', b'usestore'):			if ui.configbool(b'format', b'usestore'):
	requirements.add(b'store')			requirements.add(b'store')
	if ui.configbool(b'format', b'usefncache'):			if ui.configbool(b'format', b'usefncache'):
	requirements.add(b'fncache')			requirements.add(b'fncache')
	if ui.configbool(b'format', b'dotencode'):			if ui.configbool(b'format', b'dotencode'):
	requirements.add(b'dotencode')			requirements.add(b'dotencode')

	compengine = ui.config(b'format', b'revlog-compression')			compengines = ui.configlist(b'format', b'revlog-compression')
	if compengine not in util.compengines:			for compengine in compengines:
				if compengine in util.compengines:
				break
				else:
	raise error.Abort(			raise error.Abort(
	_(			_(
	b'compression engine %s defined by '			b'compression engines %s defined by '
	b'format.revlog-compression not available'			b'format.revlog-compression not available'
	)			)
	% compengine,			% b', '.join(b'"%s"' % e for e in compengines),
	hint=_(			hint=_(
	b'run "hg debuginstall" to list available '			b'run "hg debuginstall" to list available '
	b'compression engines'			b'compression engines'
	),			),
	)			)

	# zlib is the historical default and doesn't need an explicit requirement.			# zlib is the historical default and doesn't need an explicit requirement.
	elif compengine == b'zstd':			if compengine == b'zstd':
	requirements.add(b'revlog-compression-zstd')			requirements.add(b'revlog-compression-zstd')
	elif compengine != b'zlib':			elif compengine != b'zlib':
	requirements.add(b'exp-compression-%s' % compengine)			requirements.add(b'exp-compression-%s' % compengine)

	if scmutil.gdinitconfig(ui):			if scmutil.gdinitconfig(ui):
	requirements.add(b'generaldelta')			requirements.add(b'generaldelta')
	if ui.configbool(b'format', b'sparse-revlog'):			if ui.configbool(b'format', b'sparse-revlog'):
	requirements.add(SPARSEREVLOG_REQUIREMENT)			requirements.add(SPARSEREVLOG_REQUIREMENT)

mercurial/upgrade.py

	for req in repo.requirements:			for req in repo.requirements:
	prefix = req.startswith			prefix = req.startswith
	if prefix(b'revlog-compression-') or prefix(b'exp-compression-'):			if prefix(b'revlog-compression-') or prefix(b'exp-compression-'):
	compression = req.split(b'-', 2)[2]			compression = req.split(b'-', 2)[2]
	return compression			return compression

	@classmethod			@classmethod
	def fromconfig(cls, repo):			def fromconfig(cls, repo):
	return repo.ui.config(b'format', b'revlog-compression')			compengines = repo.ui.configlist(b'format', b'revlog-compression')
				# return the first valid value as the selection code would do
				for comp in compengines:
				if comp in util.compengines:
				return comp

				# no valide compression found lets display it all for clarity
				return b','.join(compengines)


	@registerformatvariant			@registerformatvariant
	class compressionlevel(formatvariant):			class compressionlevel(formatvariant):
	name = b'compression-level'			name = b'compression-level'
	default = b'default'			default = b'default'

	description = _(b'compression level')			description = _(b'compression level')

tests/test-repo-compengines.t

	0x78 (x) : 1 (100.00%)			0x78 (x) : 1 (100.00%)
	0x78 (x) : 110 (100.00%)			0x78 (x) : 110 (100.00%)

	$ cd ..			$ cd ..

	Unknown compression engine to format.compression aborts			Unknown compression engine to format.compression aborts

	$ hg --config format.revlog-compression=unknown init unknown			$ hg --config format.revlog-compression=unknown init unknown
	abort: compression engine unknown defined by format.revlog-compression not available			abort: compression engines "unknown" defined by format.revlog-compression not available
				mharbison72Unsubmitted Done Nit: It might help readability (especially with a list) if the engines were delimited with quotes or parenthesis or something. It's mostly obvious here, but not if it's in a config file. (And actually, I missed that "unknown" was used above, and thought it was an awkward sentence at first.) mharbison72: Nit: It might help readability (especially with a list) if the engines were delimited with…
	(run "hg debuginstall" to list available compression engines)			(run "hg debuginstall" to list available compression engines)
	[255]			[255]

				unknown compression engine in a list with known one works fine

				$ hg --config format.revlog-compression=zlib,unknown init zlib-before-unknow
				$ hg --config format.revlog-compression=unknown,zlib init unknown-before-zlib

	A requirement specifying an unknown compression engine results in bail			A requirement specifying an unknown compression engine results in bail

	$ hg init unknownrequirement			$ hg init unknownrequirement
	$ cd unknownrequirement			$ cd unknownrequirement
	$ echo exp-compression-unknown >> .hg/requires			$ echo exp-compression-unknown >> .hg/requires
	$ hg log			$ hg log
	abort: repository requires features unknown to this Mercurial: exp-compression-unknown!			abort: repository requires features unknown to this Mercurial: exp-compression-unknown!
	(see https://mercurial-scm.org/wiki/MissingRequirement for more information)			(see https://mercurial-scm.org/wiki/MissingRequirement for more information)