filelog: wrap revlog instead of inheriting it (API)
ClosedPublic

Authored by indygreg on Apr 5 2018, 8:51 PM.

Details

Summary

The revlog base class exposes a ton of methods. Inheriting the
revlog class for filelog will make it difficult to expose a
clean interface. There will be abstraction violations.

This commit breaks the inheritance of revlog by the filelog
class. Filelog instances now contain a reference to a revlog
instance. Various properties and methods are now proxied to
that instance.

There is precedence for doing this: manifestlog does something
similar. Although, manifestlog has a cleaner interface than
filelog. We'll get there with filelog...

The new filelog class exposes a handful of extra properties and
methods that aren't part of the declared filelog interface.
Every extra item was added in order to get a test to pass. The
set of tests that failed without these extra proxies has
significant overlap with the set of tests that don't work with
the simple store repo. There should be no surprise there.

Hopefully the hardest part about this commit to review are the
changes to bundlerepo and unionrepo. Both repository types
define a custom revlog or revlog-like class and then have a
custom filelog that inherits from both filelog and their custom
revlog. This code has been changed so the filelog types don't
inherit from revlog. Instead, they replace the revlog instance
on the created filelog. This is super hacky. I plan to fix this
in a future commit by parameterizing filelog.init.

Because Python function call overhead is a thing, this change
could impact performance by introducing a nearly empty proxy
function for various methods and properties. I would gladly
measure the performance impact of it, but I'm not sure what
operations have tight loops over filelog attribute lookups
or function calls. I know some of the DAG traversal code can
be sensitive about the performance of e.g. parentrevs(). However,
many of these functions are implemented on the revlog class and
therefore have direct access to self.parentrevs() and aren't
going through a proxy.

.. api::

filelog.filelog is now a standalone class and doesn't inherit
from revlog. Instead, it wraps a revlog instance at self._revlog.
This change was made in an attempt to formalize storage APIs and
prevent revlog implementation details leaking through to callers.

Diff Detail

Repository
rHG Mercurial
Lint
Automatic diff as part of commit; lint not applicable.
Unit
Automatic diff as part of commit; unit tests not applicable.
indygreg created this revision.Apr 5 2018, 8:51 PM

I want to draw attention to this patch for the potential performance concerns.

Like I said in the commit message, I'd love to know what the performance impact is. I'm just not sure which revlog/filelog attributes/methods are called in tight loops outside of revlog/filelog itself.

If the proxy methods are an issue, it /might/ be faster to use __new__ to hack up each instance so its self.__dict__[x] contained pointers to revlog functions bound as methods to the self._revlog instance. Super hacky. That's why I'd like evidence that performance is a real concern before implementing it.

durin42 accepted this revision.Apr 6 2018, 9:27 PM
This revision is now accepted and ready to land.Apr 6 2018, 9:27 PM
test-unionrepo.t:
+      t = self.revision(node)
+    File "/tmp/hgtests.wmoy8H/install/lib/python/mercurial/filelog.py", line 93, in revision
+      return self._revlog.revision(node, _df=_df, raw=raw)
+  TypeError: revision() got an unexpected keyword argument '_df'
+  [1]
durin42 requested changes to this revision.Apr 6 2018, 9:48 PM
This revision now requires changes to proceed.Apr 6 2018, 9:48 PM
test-unionrepo.t:
+      t = self.revision(node)
+    File "/tmp/hgtests.wmoy8H/install/lib/python/mercurial/filelog.py", line 93, in revision
+      return self._revlog.revision(node, _df=_df, raw=raw)
+  TypeError: revision() got an unexpected keyword argument '_df'
+  [1]

I remember this error and I thought I fixed it! But alas I can reproduce locally. I must have fixed it on a different class.

indygreg updated this revision to Diff 7860.Apr 7 2018, 1:40 AM
durin42 accepted this revision.Apr 9 2018, 6:30 PM
This revision is now accepted and ready to land.Apr 9 2018, 6:30 PM
This revision was automatically updated to reflect the committed changes.