This is an archive of the discontinued Mercurial Phabricator instance.

Differential D3393

bundle: introduce per-engine compression level
ClosedPublic

Authored by joerg.sonnenberger on Apr 16 2018, 9:08 PM.

Details

Reviewers
indygreg
Group Reviewers
hg-reviewers
Commits
rHG6a7ff5816c5f: bundle: introduce per-engine compression level
Summary

If experimental.bundlecomplevel.$engine is set, prefer it over the
generic experimental.bundlecomplevel. Given that compression levels have
widely different meanings across engines, this allows much saner
configuration.

Diff Detail

Repository
rHG Mercurial
Lint
Lint Skipped
Unit
Unit Tests Skipped

Event Timeline

indygreg added a subscriber: indygreg.Apr 17 2018, 1:40 AM

I'm -0 on this because it's yet more experimental code. Since this experimental option was added, the code for parsing bundle specs has been drastically improved. I would rather we declare a bundlespec parameter to define the compression level (e.g. `;complevel=4`) and have the bundler consume that. The code touched in commands.py already has access to the parsed bundlespec.

But if we don't want to do that so close to freeze, I'm fine adding yet more experimental config options.

joerg.sonnenberger added a comment.Apr 17 2018, 10:54 AM

I don't really disagree that the existing option naming is a bit stupid. There are useful points for including the compression level as part of the bundle spec, but I don't think that replaces the need for a config option for sane defaults.
As I said on IRC, a consistent approach for the three places where compression is used would be useful:

  • standalone "hg bundle" operations
  • wire compression
  • reflog compression

We currently don't allow tuning for the latter two, even though trade-offs between (write) performance and space make a lot of sense. As such, having a uniform mechanism for specifying the compression level sounds like a good idea.

indygreg accepted this revision.Apr 17 2018, 4:45 PM

I'm going to queue this because it is a useful feature. But I would strongly prefer we formalize things early in the 3.7 cycle. This functionality is too important to only live behind experimental flags.

This revision is now accepted and ready to land.Apr 17 2018, 4:45 PM
Closed by commit rHG6a7ff5816c5f: bundle: introduce per-engine compression level (authored by joerg.sonnenberger). · Explain WhyApr 17 2018, 4:45 PM
This revision was automatically updated to reflect the committed changes.

Revision Contents
Changeset List

PathPackages
Mmercurial/commands.py (5 lines)
Mmercurial/configitems.py (12 lines)
Mtests/test-bundle-type.t (5 lines)

Diff 8360

mercurial/commands.py

raise error.ProgrammingError( raise error.ProgrammingError(
'bundle: unexpected changegroup version %s' % cgversion) 'bundle: unexpected changegroup version %s' % cgversion)
# TODO compression options should be derived from bundlespec parsing. # TODO compression options should be derived from bundlespec parsing.
# This is a temporary hack to allow adjusting bundle compression # This is a temporary hack to allow adjusting bundle compression
# level without a) formalizing the bundlespec changes to declare it # level without a) formalizing the bundlespec changes to declare it
# b) introducing a command flag. # b) introducing a command flag.
compopts = {} compopts = {}
complevel = ui.configint('experimental',
'bundlecomplevel.' + bundlespec.compression)
if complevel is None:
complevel = ui.configint('experimental', 'bundlecomplevel') complevel = ui.configint('experimental', 'bundlecomplevel')
if complevel is not None: if complevel is not None:
compopts['level'] = complevel compopts['level'] = complevel
# Allow overriding the bundling of obsmarker in phases through # Allow overriding the bundling of obsmarker in phases through
# configuration while we don't have a bundle version that include them # configuration while we don't have a bundle version that include them
if repo.ui.configbool('experimental', 'evolution.bundle-obsmarker'): if repo.ui.configbool('experimental', 'evolution.bundle-obsmarker'):
bundlespec.contentopts['obsolescence'] = True bundlespec.contentopts['obsolescence'] = True
if repo.ui.configbool('experimental', 'bundle-phases'): if repo.ui.configbool('experimental', 'bundle-phases'):

mercurial/configitems.py

default=False, default=False,
) )
coreconfigitem('experimental', 'bundle2lazylocking', coreconfigitem('experimental', 'bundle2lazylocking',
default=False, default=False,
) )
coreconfigitem('experimental', 'bundlecomplevel', coreconfigitem('experimental', 'bundlecomplevel',
default=None, default=None,
) )
coreconfigitem('experimental', 'bundlecomplevel.bzip2',
default=None,
)
coreconfigitem('experimental', 'bundlecomplevel.gzip',
default=None,
)
coreconfigitem('experimental', 'bundlecomplevel.none',
default=None,
)
coreconfigitem('experimental', 'bundlecomplevel.zstd',
default=None,
)
coreconfigitem('experimental', 'changegroup3', coreconfigitem('experimental', 'changegroup3',
default=False, default=False,
) )
coreconfigitem('experimental', 'clientcompressionengines', coreconfigitem('experimental', 'clientcompressionengines',
default=list, default=list,
) )
coreconfigitem('experimental', 'copytrace', coreconfigitem('experimental', 'copytrace',
default='on', default='on',

tests/test-bundle-type.t

$ f --size gzip-v2.hg $ f --size gzip-v2.hg
gzip-v2.hg: size=468 gzip-v2.hg: size=468
$ hg --config experimental.bundlecomplevel=1 bundle -a -t gzip-v2 gzip-v2-level1.hg $ hg --config experimental.bundlecomplevel=1 bundle -a -t gzip-v2 gzip-v2-level1.hg
1 changesets found 1 changesets found
$ f --size gzip-v2-level1.hg $ f --size gzip-v2-level1.hg
gzip-v2-level1.hg: size=475 gzip-v2-level1.hg: size=475
$ hg --config experimental.bundlecomplevel.gzip=1 --config experimental.bundlelevel=9 bundle -a -t gzip-v2 gzip-v2-level1.hg
1 changesets found
$ f --size gzip-v2-level1.hg
gzip-v2-level1.hg: size=475
$ cd .. $ cd ..
#if zstd #if zstd
$ for t in "zstd" "zstd-v2"; do $ for t in "zstd" "zstd-v2"; do
> testbundle $t > testbundle $t
> done > done
% test bundle type zstd % test bundle type zstd