( )⚙ D10630 revlog: move the `trypending` logic from the `changelog` to the `revlog`

This is an archive of the discontinued Mercurial Phabricator instance.

Differential D10630

revlog: move the `trypending` logic from the `changelog` to the `revlog`
ClosedPublic

Authored by marmoute on May 3 2021, 7:55 AM.

Details

Reviewers
indygreg
Alphare
Group Reviewers
hg-reviewers
Commits
rHG4f38ada3fc26: revlog: move the `trypending` logic from the `changelog` to the `revlog`
Summary

We move the -reading- logic for the pending's '.a' suffixed index within the
revlog class. This is motivated by the fact the logic could be simpler and
cleaner if directly handled by the revlog docket. Before we can do so, we need
to teach the revlog code about reading "pending" changes. To be honest, we
already needed some special casing of the .a postfix, so this does not adds
much complexity.

The logic around -writing- the special '00changelog.i.a' remains in the
changelog class. Note that the revlog-v2 logic no longer use this logic.

The only remaining user of the postfix argument is the censor logic. We
could probably also make the revlog full aware of it (most of the code is
already implemented in revlog anyway) and get rid of the postfix argument and
logic. However this is an adventure for another time.

Since we have more information, we add more, paranoid, Programming error in case
we detect such "pending reader" trying to do a read (which does not happens
anyways).

Diff Detail

Repository
rHG Mercurial
Branch
default
Lint
No Linters Available
Unit
No Unit Test Coverage

Event Timeline

marmoute created this revision.May 3 2021, 7:55 AM
baymax updated this revision to Diff 27473.May 3 2021, 9:32 PM

✅ refresh by Heptapod after a successful CI run (🐙 💚)

baymax updated this revision to Diff 27876.May 11 2021, 1:44 PM

✅ refresh by Heptapod after a successful CI run (🐙 💚)

Alphare accepted this revision.May 17 2021, 5:48 PM
This revision is now accepted and ready to land.May 17 2021, 5:48 PM

Revision Contents
Changeset List

Diff 27876

mercurial/changelog.py

revision) data for a transaction that hasn't been finalized yet. revision) data for a transaction that hasn't been finalized yet.
It exists in a separate file to facilitate readers (such as It exists in a separate file to facilitate readers (such as
hooks processes) accessing data before a transaction is finalized. hooks processes) accessing data before a transaction is finalized.
``concurrencychecker`` will be passed to the revlog init function, see ``concurrencychecker`` will be passed to the revlog init function, see
the documentation there. the documentation there.
""" """
if trypending and opener.exists(b'00changelog.i.a'):
postfix = b'a'
else:
postfix = None
revlog.revlog.__init__( revlog.revlog.__init__(
self, self,
opener, opener,
target=(revlog_constants.KIND_CHANGELOG, None), target=(revlog_constants.KIND_CHANGELOG, None),
radix=b'00changelog', radix=b'00changelog',
postfix=postfix,
checkambig=True, checkambig=True,
mmaplargeindex=True, mmaplargeindex=True,
persistentnodemap=opener.options.get(b'persistent-nodemap', False), persistentnodemap=opener.options.get(b'persistent-nodemap', False),
concurrencychecker=concurrencychecker, concurrencychecker=concurrencychecker,
trypending=trypending,
) )
if self._initempty and (self._format_version == revlog.REVLOGV1): if self._initempty and (self._format_version == revlog.REVLOGV1):
# changelogs don't benefit from generaldelta. # changelogs don't benefit from generaldelta.
self._format_flags &= ~revlog.FLAG_GENERALDELTA self._format_flags &= ~revlog.FLAG_GENERALDELTA
self._generaldelta = False self._generaldelta = False

mercurial/revlog.py

_flagserrorclass = error.RevlogError _flagserrorclass = error.RevlogError
def __init__( def __init__(
self, self,
opener, opener,
target, target,
radix, radix,
postfix=None, postfix=None, # only exist for `tmpcensored` now
checkambig=False, checkambig=False,
mmaplargeindex=False, mmaplargeindex=False,
censorable=False, censorable=False,
upperboundcomp=None, upperboundcomp=None,
persistentnodemap=False, persistentnodemap=False,
concurrencychecker=None, concurrencychecker=None,
trypending=False,
): ):
""" """
create a revlog object create a revlog object
opener is a function that abstracts the file opening operation opener is a function that abstracts the file opening operation
and can be used to implement COW semantics or the like. and can be used to implement COW semantics or the like.
`target`: a (KIND, ID) tuple that identify the content stored in `target`: a (KIND, ID) tuple that identify the content stored in
this revlog. It help the rest of the code to understand what the revlog this revlog. It help the rest of the code to understand what the revlog
is about without having to resort to heuristic and index filename is about without having to resort to heuristic and index filename
analysis. Note: that this must be reliably be set by normal code, but analysis. Note: that this must be reliably be set by normal code, but
that test, debug, or performance measurement code might not set this to that test, debug, or performance measurement code might not set this to
accurate value. accurate value.
""" """
self.upperboundcomp = upperboundcomp self.upperboundcomp = upperboundcomp
self.radix = radix self.radix = radix
self._docket_file = None self._docket_file = None
self._indexfile = None self._indexfile = None
self._datafile = None self._datafile = None
self._nodemap_file = None self._nodemap_file = None
self.postfix = postfix self.postfix = postfix
self._trypending = trypending
self.opener = opener self.opener = opener
if persistentnodemap: if persistentnodemap:
self._nodemap_file = nodemaputil.get_nodemap_file(self) self._nodemap_file = nodemaputil.get_nodemap_file(self)
assert target[0] in ALL_KINDS assert target[0] in ALL_KINDS
assert len(target) == 2 assert len(target) == 2
self.target = target self.target = target
# When True, indexfile is opened with checkambig=True at writing, to # When True, indexfile is opened with checkambig=True at writing, to
if inst.errno != errno.ENOENT: if inst.errno != errno.ENOENT:
raise raise
return b'' return b''
def _loadindex(self): def _loadindex(self):
new_header, mmapindexthreshold, force_nodemap = self._init_opts() new_header, mmapindexthreshold, force_nodemap = self._init_opts()
if self.postfix is None: if self.postfix is not None:
entry_point = b'%s.i' % self.radix
else:
entry_point = b'%s.i.%s' % (self.radix, self.postfix) entry_point = b'%s.i.%s' % (self.radix, self.postfix)
elif self._trypending and self.opener.exists(b'%s.i.a' % self.radix):
entry_point = b'%s.i.a' % self.radix
else:
entry_point = b'%s.i' % self.radix
entry_data = b'' entry_data = b''
self._initempty = True self._initempty = True
entry_data = self._get_data(entry_point, mmapindexthreshold) entry_data = self._get_data(entry_point, mmapindexthreshold)
if len(entry_data) > 0: if len(entry_data) > 0:
header = INDEX_HEADER.unpack(entry_data[:4])[0] header = INDEX_HEADER.unpack(entry_data[:4])[0]
self._initempty = False self._initempty = False
else: else:
self._inline = False self._inline = False
# generaldelta implied by version 2 revlogs. # generaldelta implied by version 2 revlogs.
self._generaldelta = True self._generaldelta = True
# the logic for persistent nodemap will be dealt with within the # the logic for persistent nodemap will be dealt with within the
# main docket, so disable it for now. # main docket, so disable it for now.
self._nodemap_file = None self._nodemap_file = None
if self.postfix is None or self.postfix == b'a': if self.postfix is None:
self._datafile = b'%s.d' % self.radix self._datafile = b'%s.d' % self.radix
else: else:
self._datafile = b'%s.d.%s' % (self.radix, self.postfix) self._datafile = b'%s.d.%s' % (self.radix, self.postfix)
self.nodeconstants = sha1nodeconstants self.nodeconstants = sha1nodeconstants
self.nullid = self.nodeconstants.nullid self.nullid = self.nodeconstants.nullid
# sparse-revlog can't be on without general-delta (issue6056) # sparse-revlog can't be on without general-delta (issue6056)
if new_dfh is not None: if new_dfh is not None:
new_dfh.close() new_dfh.close()
def _nodeduplicatecallback(self, transaction, node): def _nodeduplicatecallback(self, transaction, node):
"""called when trying to add a node already stored.""" """called when trying to add a node already stored."""
@contextlib.contextmanager @contextlib.contextmanager
def _writing(self, transaction): def _writing(self, transaction):
if self._trypending:
msg = b'try to write in a `trypending` revlog: %s'
msg %= self.display_id
raise error.ProgrammingError(msg)
if self._writinghandles is not None: if self._writinghandles is not None:
yield yield
else: else:
r = len(self) r = len(self)
dsize = 0 dsize = 0
if r: if r:
dsize = self.end(r - 1) dsize = self.end(r - 1)
dfh = None dfh = None

mercurial/revlogutils/nodemap.py

"""find the revision associated with a given node""" """find the revision associated with a given node"""
entry = block.get(_to_int(node[0:1])) entry = block.get(_to_int(node[0:1]))
if isinstance(entry, dict): if isinstance(entry, dict):
return _find_node(entry, node[1:]) return _find_node(entry, node[1:])
return entry return entry
def get_nodemap_file(revlog): def get_nodemap_file(revlog):
if revlog.postfix == b'a': if revlog._trypending:
pending_path = revlog.radix + b".n.a" pending_path = revlog.radix + b".n.a"
if revlog.opener.exists(pending_path): if revlog.opener.exists(pending_path):
return pending_path return pending_path
return revlog.radix + b".n" return revlog.radix + b".n"