This is an archive of the discontinued Mercurial Phabricator instance.

Differential D4643

filelog: custom filelog to be used with narrow repos
ClosedPublic

Authored by indygreg on Sep 18 2018, 6:40 PM.

Details

Reviewers
None
Group Reviewers
hg-reviewers
Commits
rHG3e801ffd7269: filelog: custom filelog to be used with narrow repos
Summary

Narrow repos may have file revisions whose copy/rename metadata
references files not in the store. This can pose problems when
consumers attempt to access a missing referenced file revision.

The narrow extension hacks around this problem by implementing a
derived filelog type that provides custom implementations of
renamed(), size(), and cmp() which handle renames against files not
in the narrow spec by silently removing the rename metadata.

While silently dropping metadata isn't the most robust solution,
it is the easiest to implement.

This commit ports the custom narrow filelog class to core.

When a narrow repo is constructed, its ifilestorage creation
function will automatically use the new filelog type. This means
the extra logic is 0 cost for non-narrow repos and shouldn't
interfere with their operation.

Diff Detail

Repository
rHG Mercurial
Lint
Lint Skipped
Unit
Unit Tests Skipped

Event Timeline

indygreg created this revision.Sep 18 2018, 6:40 PM
Closed by commit rHG3e801ffd7269: filelog: custom filelog to be used with narrow repos (authored by indygreg). · Explain WhySep 23 2018, 5:38 AM
This revision was automatically updated to reflect the committed changes.

Revision Contents
Changeset List

PathPackages
Mmercurial/filelog.py (51 lines)
Mmercurial/localrepo.py (15 lines)

Diff 11177

mercurial/filelog.py

def length(self, rev): def length(self, rev):
return self._revlog.length(rev) return self._revlog.length(rev)
def compress(self, data): def compress(self, data):
return self._revlog.compress(data) return self._revlog.compress(data)
def _addrevision(self, *args, **kwargs): def _addrevision(self, *args, **kwargs):
return self._revlog._addrevision(*args, **kwargs) return self._revlog._addrevision(*args, **kwargs)
class narrowfilelog(filelog):
"""Filelog variation to be used with narrow stores."""
def __init__(self, opener, path, narrowmatch):
super(narrowfilelog, self).__init__(opener, path)
self._narrowmatch = narrowmatch
def renamed(self, node):
res = super(narrowfilelog, self).renamed(node)
# Renames that come from outside the narrowspec are problematic
# because we may lack the base text for the rename. This can result
# in code attempting to walk the ancestry or compute a diff
# encountering a missing revision. We address this by silently
# removing rename metadata if the source file is outside the
# narrow spec.
#
# A better solution would be to see if the base revision is available,
# rather than assuming it isn't.
#
# An even better solution would be to teach all consumers of rename
# metadata that the base revision may not be available.
#
# TODO consider better ways of doing this.
if res and not self._narrowmatch(res[0]):
return None
return res
def size(self, rev):
# Because we have a custom renamed() that may lie, we need to call
# the base renamed() to report accurate results.
node = self.node(rev)
if super(narrowfilelog, self).renamed(node):
return len(self.read(node))
else:
return super(narrowfilelog, self).size(rev)
def cmp(self, node, text):
different = super(narrowfilelog, self).cmp(node, text)
# Because renamed() may lie, we may get false positives for
# different content. Check for this by comparing against the original
# renamed() implementation.
if different:
if super(narrowfilelog, self).renamed(node):
t2 = self.read(node)
return t2 != text
return different

mercurial/localrepo.py

"""File storage when using revlogs.""" """File storage when using revlogs."""
def file(self, path): def file(self, path):
if path[0] == b'/': if path[0] == b'/':
path = path[1:] path = path[1:]
return filelog.filelog(self.svfs, path) return filelog.filelog(self.svfs, path)
@interfaceutil.implementer(repository.ilocalrepositoryfilestorage)
class revlognarrowfilestorage(object):
"""File storage when using revlogs and narrow files."""
def file(self, path):
if path[0] == b'/':
path = path[1:]
return filelog.narrowfilelog(self.svfs, path, self.narrowmatch())
def makefilestorage(requirements, **kwargs): def makefilestorage(requirements, **kwargs):
"""Produce a type conforming to ``ilocalrepositoryfilestorage``.""" """Produce a type conforming to ``ilocalrepositoryfilestorage``."""
if repository.NARROW_REQUIREMENT in requirements:
return revlognarrowfilestorage
else:
return revlogfilestorage return revlogfilestorage
# List of repository interfaces and factory functions for them. Each # List of repository interfaces and factory functions for them. Each
# will be called in order during ``makelocalrepository()`` to iteratively # will be called in order during ``makelocalrepository()`` to iteratively
# derive the final type for a local repository instance. # derive the final type for a local repository instance.
REPO_INTERFACES = [ REPO_INTERFACES = [
(repository.ilocalrepositorymain, makemain), (repository.ilocalrepositorymain, makemain),
(repository.ilocalrepositoryfilestorage, makefilestorage), (repository.ilocalrepositoryfilestorage, makefilestorage),
] ]