This is an archive of the discontinued Mercurial Phabricator instance.

Differential D9235

revlog: use LRU for the chain cache
ClosedPublic

Authored by joerg.sonnenberger on Oct 20 2020, 8:18 PM.

Details

Reviewers
indygreg
Group Reviewers
hg-reviewers
Commits
rHG8719a5b68419: revlog: use LRU for the chain cache
Summary

For a large repository, this reduces the number of filelog instances and
associated data a lot. For a 1% speed penalty, it reduces peak RSS by
20% for the full NetBSD test repository.

Diff Detail

Repository
rHG Mercurial
Lint
Automatic diff as part of commit; lint not applicable.
Unit
Automatic diff as part of commit; unit tests not applicable.

Event Timeline

joerg.sonnenberger added a commit: rHG8719a5b68419: revlog: use LRU for the chain cache.Oct 28 2020, 9:58 PM
This revision was not accepted when it landed; it landed in state Needs Review.
Closed by commit rHG8719a5b68419: revlog: use LRU for the chain cache (authored by joerg.sonnenberger). · Explain Why
This revision was automatically updated to reflect the committed changes.

Revision Contents
Changeset List

PathPackages
Mmercurial/revlog.py (4 lines)

Diff 23333

mercurial/revlog.py

except (ValueError, IndexError): except (ValueError, IndexError):
raise error.RevlogError( raise error.RevlogError(
_(b"index %s is corrupted") % self.indexfile _(b"index %s is corrupted") % self.indexfile
) )
self.index, self._chunkcache = d self.index, self._chunkcache = d
if not self._chunkcache: if not self._chunkcache:
self._chunkclear() self._chunkclear()
# revnum -> (chain-length, sum-delta-length) # revnum -> (chain-length, sum-delta-length)
self._chaininfocache = {} self._chaininfocache = util.lrucachedict(500)
# revlog header -> revlog compressor # revlog header -> revlog compressor
self._decompressors = {} self._decompressors = {}
@util.propertycache @util.propertycache
def _compressor(self): def _compressor(self):
engine = util.compengines[self._compengine] engine = util.compengines[self._compengine]
return engine.revlogcompressor(self._compengineopts) return engine.revlogcompressor(self._compengineopts)
end = rev * self._io.size end = rev * self._io.size
else: else:
end += rev * self._io.size end += rev * self._io.size
transaction.add(self.indexfile, end) transaction.add(self.indexfile, end)
# then reset internal state in memory to forget those revisions # then reset internal state in memory to forget those revisions
self._revisioncache = None self._revisioncache = None
self._chaininfocache = {} self._chaininfocache = util.lrucachedict(500)
self._chunkclear() self._chunkclear()
del self.index[rev:-1] del self.index[rev:-1]
def checksize(self): def checksize(self):
"""Check size of index and data files """Check size of index and data files
return a (dd, di) tuple. return a (dd, di) tuple.