This is an archive of the discontinued Mercurial Phabricator instance.

Differential D9572

revlog: support none compression
ClosedPublic

Authored by joerg.sonnenberger on Dec 12 2020, 6:24 PM.

Details

Reviewers
None
Group Reviewers
hg-reviewers
Commits
rHG84130fd74a22: revlog: support none compression
Summary

revlog files had uncompressed revlog entries support since forever, but
it wasn't selectable or exposed explicitly. It is occassionally useful
for performance testing as it avoids the latency of zlib or zstd. It
also has the nice side effect of providing a non-default compression
engine out-of-the-box.

Diff Detail

Repository
rHG Mercurial
Lint
Automatic diff as part of commit; lint not applicable.
Unit
Automatic diff as part of commit; unit tests not applicable.

Event Timeline

joerg.sonnenberger added a commit: rHG84130fd74a22: revlog: support none compression.Dec 16 2020, 7:14 PM
This revision was not accepted when it landed; it landed in state Needs Review.
Closed by commit rHG84130fd74a22: revlog: support none compression (authored by joerg.sonnenberger). · Explain Why
This revision was automatically updated to reflect the committed changes.

Revision Contents
Changeset List

Diff 24312

mercurial/utils/compression.py

return b'none', b'UN' return b'none', b'UN'
# Clients always support uncompressed payloads. Servers don't because # Clients always support uncompressed payloads. Servers don't because
# unless you are on a fast network, uncompressed payloads can easily # unless you are on a fast network, uncompressed payloads can easily
# saturate your network pipe. # saturate your network pipe.
def wireprotosupport(self): def wireprotosupport(self):
return compewireprotosupport(b'none', 0, 10) return compewireprotosupport(b'none', 0, 10)
# We don't implement revlogheader because it is handled specially # revlog special cases the uncompressed case, but implementing
# in the revlog class. # revlogheader allows forcing uncompressed storage.
def revlogheader(self):
return b'\0'
def compressstream(self, it, opts=None): def compressstream(self, it, opts=None):
return it return it
def decompressorreader(self, fh): def decompressorreader(self, fh):
return fh return fh
class nooprevlogcompressor(object): class nooprevlogcompressor(object):

tests/test-repo-compengines.t

$ echo exp-compression-unknown >> .hg/requires $ echo exp-compression-unknown >> .hg/requires
$ hg log $ hg log
abort: repository requires features unknown to this Mercurial: exp-compression-unknown abort: repository requires features unknown to this Mercurial: exp-compression-unknown
(see https://mercurial-scm.org/wiki/MissingRequirement for more information) (see https://mercurial-scm.org/wiki/MissingRequirement for more information)
[255] [255]
$ cd .. $ cd ..
Specifying a new format.compression on an existing repo won't introduce data
with that engine or a requirement
$ cd default
$ touch bar
$ hg --config format.revlog-compression=none -q commit -A -m 'add bar with a lot of repeated repeated repeated text'
$ cat .hg/requires
dotencode
fncache
generaldelta
revlogv1
sparserevlog
store
testonly-simplestore (reposimplestore !)
$ hg debugrevlog -c | grep 0x78
0x78 (x) : 2 (100.00%)
0x78 (x) : 199 (100.00%)
$ cd ..
#if zstd #if zstd
$ hg --config format.revlog-compression=zstd init zstd $ hg --config format.revlog-compression=zstd init zstd
$ cd zstd $ cd zstd
$ cat .hg/requires $ cat .hg/requires
dotencode dotencode
fncache fncache
generaldelta generaldelta
revlog-compression-zstd revlog-compression-zstd
revlogv1 revlogv1
sparserevlog sparserevlog
store store
testonly-simplestore (reposimplestore !) testonly-simplestore (reposimplestore !)
$ touch foo $ touch foo
$ hg -q commit -A -m 'initial commit with a lot of repeated repeated repeated text' $ hg -q commit -A -m 'initial commit with a lot of repeated repeated repeated text'
$ hg debugrevlog -c | grep 0x28 $ hg debugrevlog -c | grep 0x28
0x28 : 1 (100.00%) 0x28 : 1 (100.00%)
0x28 : 98 (100.00%) 0x28 : 98 (100.00%)
$ cd .. $ cd ..
Specifying a new format.compression on an existing repo won't introduce data
with that engine or a requirement
$ cd default
$ touch bar
$ hg --config format.revlog-compression=zstd -q commit -A -m 'add bar with a lot of repeated repeated repeated text'
$ cat .hg/requires
dotencode
fncache
generaldelta
revlogv1
sparserevlog
store
testonly-simplestore (reposimplestore !)
$ hg debugrevlog -c | grep 0x78
0x78 (x) : 2 (100.00%)
0x78 (x) : 199 (100.00%)
#endif #endif
checking zlib options checking zlib options
===================== =====================
$ hg init zlib-level-default $ hg init zlib-level-default
$ hg init zlib-level-1 $ hg init zlib-level-1
> hg -R $repo commit -m some-commit > hg -R $repo commit -m some-commit
> } > }
$ for repo in zlib-level-default zlib-level-1 zlib-level-9; do $ for repo in zlib-level-default zlib-level-1 zlib-level-9; do
> commitone $repo > commitone $repo
> done > done
$ $RUNTESTDIR/f -s */.hg/store/data/* $ $RUNTESTDIR/f -s */.hg/store/data/*
default/.hg/store/data/foo.i: size=64 (pure !) default/.hg/store/data/bar.i: size=64
default/.hg/store/data/foo.i: size=64
zlib-level-1/.hg/store/data/a.i: size=4146 zlib-level-1/.hg/store/data/a.i: size=4146
zlib-level-9/.hg/store/data/a.i: size=4138 zlib-level-9/.hg/store/data/a.i: size=4138
zlib-level-default/.hg/store/data/a.i: size=4138 zlib-level-default/.hg/store/data/a.i: size=4138
zstd/.hg/store/data/foo.i: size=64 (zstd !)
Test error cases Test error cases
$ hg init zlib-level-invalid $ hg init zlib-level-invalid
$ cat << EOF >> zlib-level-invalid/.hg/hgrc $ cat << EOF >> zlib-level-invalid/.hg/hgrc
> [storage] > [storage]
> revlog.zlib.level=foobar > revlog.zlib.level=foobar
> EOF > EOF
$ commitone zlib-level-invalid $ commitone zlib-level-invalid
config error: storage.revlog.zlib.level is not a valid integer ('foobar') config error: storage.revlog.zlib.level is not a valid integer ('foobar')
config error: storage.revlog.zlib.level is not a valid integer ('foobar') config error: storage.revlog.zlib.level is not a valid integer ('foobar')
[30] [30]
$ hg init zlib-level-out-of-range $ hg init zlib-level-out-of-range
$ cat << EOF >> zlib-level-out-of-range/.hg/hgrc $ cat << EOF >> zlib-level-out-of-range/.hg/hgrc
> [storage] > [storage]
> revlog.zlib.level=42 > revlog.zlib.level=42
> EOF > EOF
$ commitone zlib-level-out-of-range $ commitone zlib-level-out-of-range
abort: invalid value for `storage.revlog.zlib.level` config: 42 abort: invalid value for `storage.revlog.zlib.level` config: 42
abort: invalid value for `storage.revlog.zlib.level` config: 42 abort: invalid value for `storage.revlog.zlib.level` config: 42
[255] [255]
checking details of none compression
====================================
$ hg init none-compression --config format.revlog-compression=none
$ commitone() {
> repo=$1
> cp $RUNTESTDIR/bundles/issue4438-r1.hg $repo/a
> hg -R $repo add $repo/a
> hg -R $repo commit -m some-commit
> }
$ commitone none-compression
$ hg log -R none-compression
changeset: 0:68b53da39cd8
tag: tip
user: test
date: Thu Jan 01 00:00:00 1970 +0000
summary: some-commit
$ cat none-compression/.hg/requires
dotencode
exp-compression-none
fncache
generaldelta
revlogv1
sparserevlog
store
testonly-simplestore (reposimplestore !)
$ $RUNTESTDIR/f -s none-compression/.hg/store/data/*
none-compression/.hg/store/data/a.i: size=4216
#if zstd #if zstd
checking zstd options checking zstd options
===================== =====================
$ hg init zstd-level-default --config format.revlog-compression=zstd $ hg init zstd-level-default --config format.revlog-compression=zstd
$ hg init zstd-level-1 --config format.revlog-compression=zstd $ hg init zstd-level-1 --config format.revlog-compression=zstd
$ cat << EOF >> zstd-level-1/.hg/hgrc $ cat << EOF >> zstd-level-1/.hg/hgrc