( )⚙ D10651 revlog: add a `_get_decompressor` method

This is an archive of the discontinued Mercurial Phabricator instance.

Differential D10651

revlog: add a `_get_decompressor` method
ClosedPublic

Authored by marmoute on May 4 2021, 10:14 AM.

Details

Reviewers
indygreg
Alphare
Group Reviewers
hg-reviewers
Commits
rHGeac3591abbf4: revlog: add a `_get_decompressor` method
Summary

This logic is non-trivial and we will need to reuse it.

Diff Detail

Repository
rHG Mercurial
Branch
default
Lint
No Linters Available
Unit
No Unit Test Coverage

Event Timeline

marmoute created this revision.May 4 2021, 10:14 AM
baymax updated this revision to Diff 27588.May 5 2021, 5:48 PM

✅ refresh by Heptapod after a successful CI run (🐙 💚)

baymax updated this revision to Diff 27890.May 11 2021, 1:46 PM

✅ refresh by Heptapod after a successful CI run (🐙 💚)

Alphare accepted this revision.May 17 2021, 5:49 PM
This revision is now accepted and ready to land.May 17 2021, 5:49 PM
marmoute added a commit: rHGeac3591abbf4: revlog: add a `_get_decompressor` method.May 17 2021, 5:55 PM
Closed by commit rHGeac3591abbf4: revlog: add a `_get_decompressor` method (authored by marmoute). · Explain Why
This revision was automatically updated to reflect the committed changes.

Revision Contents
Changeset List

PathPackages
Mmercurial/configitems.py (3 lines)
Mmercurial/revlog.py (26 lines)

Diff 27559

mercurial/configitems.py

# * proper streaming (race free) of the docker file # * proper streaming (race free) of the docker file
# * track garbage data to evemtually allow rewriting -existing- sidedata. # * track garbage data to evemtually allow rewriting -existing- sidedata.
# * Exchange-wise, we will also need to do something more efficient than # * Exchange-wise, we will also need to do something more efficient than
# keeping references to the affected revlogs, especially memory-wise when # keeping references to the affected revlogs, especially memory-wise when
# rewriting sidedata. # rewriting sidedata.
# * sidedata compression # * sidedata compression
# * introduce a proper solution to reduce the number of filelog related files. # * introduce a proper solution to reduce the number of filelog related files.
# * Improvement to consider # * Improvement to consider
# - track compression mode in the index entris instead of the chunks # - avoid compression header in chunk using the default compression?
# - forbid "inline" compression mode entirely?
# - split the data offset and flag field (the 2 bytes save are mostly trouble) # - split the data offset and flag field (the 2 bytes save are mostly trouble)
# - keep track of uncompressed -chunk- size (to preallocate memory better) # - keep track of uncompressed -chunk- size (to preallocate memory better)
# - keep track of chain base or size (probably not that useful anymore) # - keep track of chain base or size (probably not that useful anymore)
# - store data and sidedata in different files # - store data and sidedata in different files
coreconfigitem( coreconfigitem(
b'experimental', b'experimental',
b'revlogv2', b'revlogv2',
default=None, default=None,

mercurial/revlog.py

@util.propertycache @util.propertycache
def display_id(self): def display_id(self):
"""The public facing "ID" of the revlog that we use in message""" """The public facing "ID" of the revlog that we use in message"""
# Maybe we should build a user facing representation of # Maybe we should build a user facing representation of
# revlog.target instead of using `self.radix` # revlog.target instead of using `self.radix`
return self.radix return self.radix
def _get_decompressor(self, t):
try:
compressor = self._decompressors[t]
except KeyError:
try:
engine = util.compengines.forrevlogheader(t)
compressor = engine.revlogcompressor(self._compengineopts)
self._decompressors[t] = compressor
except KeyError:
raise error.RevlogError(
_(b'unknown compression type %s') % binascii.hexlify(t)
)
return compressor
@util.propertycache @util.propertycache
def _compressor(self): def _compressor(self):
engine = util.compengines[self._compengine] engine = util.compengines[self._compengine]
return engine.revlogcompressor(self._compengineopts) return engine.revlogcompressor(self._compengineopts)
def _indexfp(self): def _indexfp(self):
"""file object for the revlog's index file""" """file object for the revlog's index file"""
return self.opener(self._indexfile, mode=b"r") return self.opener(self._indexfile, mode=b"r")
% stringutil.forcebytestr(e) % stringutil.forcebytestr(e)
) )
# '\0' is more common than 'u' so it goes first. # '\0' is more common than 'u' so it goes first.
elif t == b'\0': elif t == b'\0':
return data return data
elif t == b'u': elif t == b'u':
return util.buffer(data, 1) return util.buffer(data, 1)
try: compressor = self._get_decompressor(t)
compressor = self._decompressors[t]
except KeyError:
try:
engine = util.compengines.forrevlogheader(t)
compressor = engine.revlogcompressor(self._compengineopts)
self._decompressors[t] = compressor
except KeyError:
raise error.RevlogError(
_(b'unknown compression type %s') % binascii.hexlify(t)
)
return compressor.decompress(data) return compressor.decompress(data)
def _addrevision( def _addrevision(
self, self,
node, node,
rawtext, rawtext,
transaction, transaction,