This is an archive of the discontinued Mercurial Phabricator instance.

Differential D7710

revlog: split the content verification of a node into a separate method
ClosedPublic

Authored by mharbison72 on Dec 23 2019, 2:47 AM.

Details

Reviewers
indygreg
Group Reviewers
hg-reviewers
Commits
rHG7c3118b9a9db: revlog: split the content verification of a node into a separate method
Summary

This will be used by LFS to tune what is skipped.

In the future, this could also be used by LFS to indicate which nodes tagged
with skipread are simply in need of a blob fetch, so that they can be done in
a batch later. (Currently, skipread also indicates censored data and errors.)
Additionally, it could be used to cache the sha1 hash value for each blob so
that large blobs don't need to be re-read and re-hashed if they are used by
multiple nodes.

Diff Detail

Repository
rHG Mercurial
Lint
Automatic diff as part of commit; lint not applicable.
Unit
Automatic diff as part of commit; unit tests not applicable.

Event Timeline

mharbison72 created this revision.Dec 23 2019, 2:47 AM
indygreg accepted this revision.Dec 23 2019, 12:34 PM

The added function should arguably be a method. But it shouldn't really matter.

This revision is now accepted and ready to land.Dec 23 2019, 12:34 PM

Revision Contents
Changeset List

PathPackages
Mmercurial/revlog.py (16 lines)

Diff 18916

mercurial/revlog.py

def offset_type(offset, type): def offset_type(offset, type):
if (type & ~flagutil.REVIDX_KNOWN_FLAGS) != 0: if (type & ~flagutil.REVIDX_KNOWN_FLAGS) != 0:
raise ValueError(b'unknown revlog index flags') raise ValueError(b'unknown revlog index flags')
return int(int(offset) << 16 | type) return int(int(offset) << 16 | type)
def _verify_revision(rl, skipflags, state, node):
"""Verify the integrity of the given revlog ``node`` while providing a hook
point for extensions to influence the operation."""
if skipflags:
state[b'skipread'].add(node)
else:
# Side-effect: read content and verify hash.
rl.revision(node)
@attr.s(slots=True, frozen=True) @attr.s(slots=True, frozen=True)
class _revisioninfo(object): class _revisioninfo(object):
"""Information about a revision that allows building its fulltext """Information about a revision that allows building its fulltext
node: expected hash of the revision node: expected hash of the revision
p1, p2: parent revs of the revision p1, p2: parent revs of the revision
btext: built text cache consisting of a one-element list btext: built text cache consisting of a one-element list
cachedelta: (baserev, uncompressed_delta) or None cachedelta: (baserev, uncompressed_delta) or None
flags: flags associated to the revision storage flags: flags associated to the revision storage
# 2. hash check: depending on flag processor, we may need to # 2. hash check: depending on flag processor, we may need to
# use either "text" (external), or "rawtext" (in revlog). # use either "text" (external), or "rawtext" (in revlog).
try: try:
skipflags = state.get(b'skipflags', 0) skipflags = state.get(b'skipflags', 0)
if skipflags: if skipflags:
skipflags &= self.flags(rev) skipflags &= self.flags(rev)
if skipflags: _verify_revision(self, skipflags, state, node)
state[b'skipread'].add(node)
else:
# Side-effect: read content and verify hash.
self.revision(node)
l1 = self.rawsize(rev) l1 = self.rawsize(rev)
l2 = len(self.rawdata(node)) l2 = len(self.rawdata(node))
if l1 != l2: if l1 != l2:
yield revlogproblem( yield revlogproblem(
error=_(b'unpacked size is %d, %d expected') % (l2, l1), error=_(b'unpacked size is %d, %d expected') % (l2, l1),
node=node, node=node,