This is an archive of the discontinued Mercurial Phabricator instance.

Differential D620

remotefilelog: using fastlog as a fast path to fix linkrev
ClosedPublic

Authored by shivramk on Sep 4 2017, 12:30 PM.
Tags
None
Subscribers
durham, pulkit

Details

Reviewers
ikostia
stash
ryanmce
durham
Group Reviewers
Restricted Project
Commits
rFBHGXbf48d31f3587: remotefilelog: using fastlog as a fast path to fix linkrev
Summary

The _adjustlinknode function in remotefilectx currently has a slow case where
it must prefetch history from the server to lookup the correct linkcode.
This change makes use of fastlog (if enabled for the repo) to speed up the
linknode lookup as fastlog will already have the correct linkrev. If fastlog
is not enabled or the fastlog call fails for some reason then it will fall
back to the old method.

Test Plan

Added new test cases in test-remotefilelog-linknodes.t

Diff Detail

Repository
rFBHGX Facebook Mercurial Extensions
Branch
arcpatch-D742 (bookmark) on default (branch)
Lint
Lint OK
Unit
Unit Tests OK

Event Timeline

shivramk created this revision.Sep 4 2017, 12:30 PM
Herald added a reviewer: Restricted Project. · View Herald TranscriptSep 4 2017, 12:30 PM
pulkit added a subscriber: pulkit.Sep 5 2017, 11:15 AM
pulkit added inline comments.
remotefilelog/remotefilectx.py
28

You should add a return after this otherwise, you may hit an AttributeError.

durham requested changes to this revision.Sep 5 2017, 11:58 AM
durham added a subscriber: durham.

How much faster is this than before?

remotefilelog/remotefilectx.py
55

If initialization fails, then conduit will still be None and this logic will get invoked every single time this code path his hit, and print a ton of warnings. Maybe we should do something like set conduit=False if initialization fails, then in this function we say if conduit is None: initialize, elif conduit is False: return None

347

I'd explicitly return None here. We generally shouldn't rely on pythons default return None behavior for functions with actual return values.

tests/test-remotefilelog-linknodes.t
231

Why set debug to true here if there is no output?

This revision now requires changes to proceed.Sep 5 2017, 11:58 AM
shivramk added a comment.Sep 5 2017, 12:27 PM

I haven't measured the actual performance improvement. Will try on one of our big repos.

remotefilelog/remotefilectx.py
55

Makes sense. Will set conduit to False if initialization fails

shivramk edited edge metadata.Sep 5 2017, 12:59 PM
shivramk updated this revision to Diff 1614.

addressed review comments

ikostia edited edge metadata.Sep 6 2017, 5:21 AM

This looks good to me, but I'd like @durham to take a final look.
One of the things that is missing in this diff however is the timeout. Were you going to publish another diff for that?

ryanmce added inline comments.Sep 6 2017, 8:44 AM
remotefilelog/remotefilectx.py
14

I'm not a fan of making conduit global when it could just be a property of the class. Why make it global?

ryanmce requested changes to this revision.Sep 6 2017, 8:44 AM

If there's a good reason, I'll reconsider.

This revision now requires changes to proceed.Sep 6 2017, 8:44 AM
shivramk added inline comments.Sep 6 2017, 8:51 AM
remotefilelog/remotefilectx.py
14

Well the idea was to have a single conduit shared by all remotefilelog instances and have it be initialized only once.

shivramk added a comment.Sep 6 2017, 8:53 AM

Having said it, if its not common to have multiple instances of remotefilectx in a single invocation of hg then it would indeed be better to have it as an instance variable.

ryanmce added a comment.Sep 6 2017, 9:00 AM

Ah good point. There will be plenty of remotefilectx, but probably there's only one remotefileserver. Perhaps the conduit can live there?

durham added a comment.Sep 6 2017, 1:10 PM

Since the goal of this is performance, I'd still like to know some perf numbers (keeping in mind this code review is open to the public so don't include any internal numbers like repo size). Preferably a comparison of before and after, but at least how fast it is when the code path is executed.

stash edited edge metadata.Sep 12 2017, 12:40 PM

Since the goal of this is performance, I'd still like to know some perf numbers (keeping in mind this code review is open to the public so don't include any internal numbers like repo size).

I can give it a go tomorrow. But I'm pretty sure it won't be worse than the remotefilelog prefetch heuristic we had before.

stash added a comment.Sep 12 2017, 12:43 PM

@shivramk, @ikostia just told me that you were planning to do the measurements yourself. Let me know if you have any problems with it

stash added a comment.Sep 13 2017, 1:09 PM

I've found a bug in implementation and I need some time to figure out how to fix it.

shivramk updated this revision to Diff 1877.Sep 18 2017, 6:22 PM

Moved the fastlog call out of the for loop

The previous version had a bug in that we were calling fastlog inside the for
loop which iterates over ancestor revisions. This would always return the latest
rev. This change moves it out of the for loop. We now call fastlog to retrieve
all revs where the file was modified, and then return the first rev which had
the same content.

I tested the changes with a file in one of our big repos and here are the
performance numbers.

Time taken for hg log with fastlog disabled:

time hg log strings.xml > /dev/null

hg log > /dev/null 10.08s user 0.64s system 99% cpu 10.728 total

With fastlog enabled:

time hg log strings.xml >/dev/null

hg log > /dev/null 1.14s user 0.47s system 59% cpu 2.710 total

stash requested changes to this revision.Sep 19 2017, 4:35 AM
stash added inline comments.
remotefilelog/remotefilectx.py
243–248

Sorry, I just realized I forgot to take draft phases into account :(.
In that case fastlog will be called even on draft revisions. Draft revisions are present only on your host, and that means that fastlog call will fail and it will slow things down.

To fix it we need to move this piece of code in the inner loop below, where we know that we are working with public revision.
See my comments below.

303

Fastlog call should be here, and it should be called only once, so I suggest to have

if seenpublic == False and repo.ui.configbool('fastlog', 'enabled'):
            lnode = self._linknodeviafastlog(repo, path, ancrev, fnode, cl, mfl)
            if lnode:
                return lnode

seenpublic = True

(note ancrev here self._linknodeviafastlog(repo, path, ancrev, fnode, cl, mfl))

One more important thing - in "_linknodeviafastlog" you are currently doing "for anc in results[1:]:". Since now we are checking only public revisions, we don't need to skip the first element, so it should be "for anc in results".

Also I think that we need to have a test that covers this behavior. Lmk if you need help with it!

P. S.
Adding a comment about what we are doing is also a good idea.

339

To route error messages correctly, you need to do:

repo.ui.log('fastloglinkrevfixup', _('fastlog returned 0 results\n'))

Can you please change all of the invocations of repo.ui.log?

341

See above: you don't need to skip first element if you query only public commits.

This revision now requires changes to proceed.Sep 19 2017, 4:35 AM
durham added a comment.Sep 19 2017, 12:25 PM

@shivramk should a test have caught the bug you found? Seems like a correctness issue, so I'm surprised nothing failed. Do we need to add a test?

shivramk updated this revision to Diff 1903.Sep 19 2017, 8:11 PM

Fixed handling of draft commits and logging calls

stash accepted this revision.Sep 20 2017, 5:58 AM

@shivramk It looks good to me, I'm going to accept the revision, but please address the comments, especially two most important ones about timeout and about calling fastlog only once.

Heads up @durham @ryanmce.

I just tested the change again. We'll see speed up on hg log FILENAME for certain files that are affected by linkrev issue. On my host I can reliably repro 7 secs speed up for one of such files - 11 secs down to 4 secs. It should also speed up other commands such as amend, maybe rebases etc.

What puzzled me for a long time is why we see this speed up in hg log command, while we already have fastlog.py extension. Turned out that fastlog.py extension works only for directories. From fastlog.py comment:

# bail on symlinks, and also bail on files for now
# with follow behavior, for files, we are supposed
# to track copies / renames, but it isn't convenient
# to do this through scmquery
remotefilelog/remotefilectx.py
297

Let's run fastlog only once - please run it only if seenpublic is False:

if not seenpublic and repo.ui.configbool('fastlog', 'enabled'):
298–299

Formatting is wrong:

lnode = self._linknodeviafastlog(repo, path, ancrev, fnode,
                                 cl, mfl)

I suggest to use Nuclide - it highlights such problems automatically

301

Can you also log successful fastlog calls:

if lnode:
  repo.ui.log('fastloglinkrevfixup', _('success\n'))
  return lnode
330–337

I completely forgot about timeout, but luckily the change is simple:

Please add these two lines -

diff --git a/remotefilelog/remotefilectx.py b/remotefilelog/remotefilectx.py
--- a/remotefilelog/remotefilectx.py
+++ b/remotefilelog/remotefilectx.py
@@ -325,8 +325,10 @@
             return None
         try:
             srchex = repo[srcrev].hex()
+            FASTLOG_TIMEOUT_IN_SECS = 0.5
             results = self._conduit.call_conduit(
                 'scmquery.log_v2',
+                timeout=FASTLOG_TIMEOUT_IN_SECS,
                 repo=reponame,
                 scm_type='hg',
                 rev=srchex,

Also I added depended revision in phabricator that adds timeout functionality - D742. So if you want to test locally, then first you need to do hgarc patch D742 to get my changes, then rebase your changes on top.

But don't worry, I tested them myself.

stash added a comment.EditedSep 21 2017, 10:01 AM

@shivramk I just pushed diff with timeout. Please address the comments and feel free to land it!

shivramk added inline comments.Sep 21 2017, 1:56 PM
remotefilelog/remotefilectx.py
297

The outer if condition is already checking for not seenpublic. Wouldn't this check be redundant?

shivramk updated this revision to Diff 1972.Sep 21 2017, 2:15 PM

Addressed review comments

stash added inline comments.Sep 22 2017, 5:15 AM
remotefilelog/remotefilectx.py
297

Right, makes sense

stash accepted this revision.Sep 22 2017, 5:16 AM
Closed by commit rFBHGXbf48d31f3587: remotefilelog: using fastlog as a fast path to fix linkrev (authored by shivramk). · Explain WhySep 22 2017, 1:33 PM
This revision was automatically updated to reflect the committed changes.

Revision Contents
Changeset List

Diff 1972

remotefilelog/remotefilectx.py

# remotefilectx.py - filectx/workingfilectx implementations for remotefilelog # remotefilectx.py - filectx/workingfilectx implementations for remotefilelog
# #
# Copyright 2013 Facebook, Inc. # Copyright 2013 Facebook, Inc.
# #
# This software may be used and distributed according to the terms of the # This software may be used and distributed according to the terms of the
# GNU General Public License version 2 or any later version. # GNU General Public License version 2 or any later version.
import collections import collections
from mercurial.i18n import _ from mercurial.i18n import _
from mercurial.node import bin, hex, nullid, nullrev from mercurial.node import bin, hex, nullid, nullrev
from mercurial import context, util, error, ancestor, phases from mercurial import context, util, error, ancestor, phases, extensions
propertycache = util.propertycache propertycache = util.propertycache
conduit = None
ryanmceUnsubmitted
Not Done

I'm not a fan of making conduit global when it could just be a property of the class. Why make it global?

ryanmce: I'm not a fan of making conduit global when it could just be a property of the class. Why make…
shivramkAuthorUnsubmitted
Not Done

Well the idea was to have a single conduit shared by all remotefilelog instances and have it be initialized only once.

shivramk: Well the idea was to have a single conduit shared by all remotefilelog instances and have it be…
FASTLOG_TIMEOUT_IN_SECS = 0.5
def createconduit(ui):
try:
conduit = extensions.find("fbconduit")
except KeyError:
try:
from hgext3rd import fbconduit as conduit
except ImportError:
ui.log('fastloglinkrevfixup',
_('unable to find fbconduit extension\n'))
return False
if not util.safehasattr(conduit, 'conduit_config'):
ui.log('fastloglinkrevfixup',
pulkitUnsubmitted
Not Done

You should add a return after this otherwise, you may hit an AttributeError.

pulkit: You should add a `return` after this otherwise, you may hit an `AttributeError`.
_('incompatible conduit module; disabling fastlog\n'))
return False
if not conduit.conduit_config(ui):
ui.log('fastloglinkrevfixup',
_('no conduit host specified in config; disabling fastlog\n'))
return False
return conduit
class remotefilectx(context.filectx): class remotefilectx(context.filectx):
def __init__(self, repo, path, changeid=None, fileid=None, def __init__(self, repo, path, changeid=None, fileid=None,
filelog=None, changectx=None, ancestormap=None): filelog=None, changectx=None, ancestormap=None):
if fileid == nullrev: if fileid == nullrev:
fileid = nullid fileid = nullid
if fileid and len(fileid) == 40: if fileid and len(fileid) == 40:
fileid = bin(fileid) fileid = bin(fileid)
super(remotefilectx, self).__init__(repo, path, changeid, super(remotefilectx, self).__init__(repo, path, changeid,
fileid, filelog, changectx) fileid, filelog, changectx)
self._ancestormap = ancestormap self._ancestormap = ancestormap
def size(self): def size(self):
return self._filelog.size(self._filenode) return self._filelog.size(self._filenode)
@propertycache @propertycache
def _conduit(self):
global conduit
if conduit is None:
conduit = createconduit(self._repo.ui)
durhamUnsubmitted
Not Done

If initialization fails, then conduit will still be None and this logic will get invoked every single time this code path his hit, and print a ton of warnings. Maybe we should do something like set conduit=False if initialization fails, then in this function we say if conduit is None: initialize, elif conduit is False: return None

durham: If initialization fails, then conduit will still be None and this logic will get invoked every…
shivramkAuthorUnsubmitted
Not Done

Makes sense. Will set conduit to False if initialization fails

shivramk: Makes sense. Will set conduit to False if initialization fails
# If createconduit fails, conduit will be set to False. We use this to
# avoid calling createconduit multiple times
if conduit is False:
return None
return conduit
@propertycache
def _changeid(self): def _changeid(self):
if '_changeid' in self.__dict__: if '_changeid' in self.__dict__:
return self._changeid return self._changeid
elif '_changectx' in self.__dict__: elif '_changectx' in self.__dict__:
return self._changectx.rev() return self._changectx.rev()
elif '_descendantrev' in self.__dict__: elif '_descendantrev' in self.__dict__:
# this file context was created from a revision with a known # this file context was created from a revision with a known
# descendant, we can (lazily) correct for linkrev aliases # descendant, we can (lazily) correct for linkrev aliases
flog = repo.file(path) flog = repo.file(path)
p2ctx = remotefilectx(repo, path, fileid=p2, filelog=flog, p2ctx = remotefilectx(repo, path, fileid=p2, filelog=flog,
ancestormap=ancestormap) ancestormap=ancestormap)
p2ctx._descendantrev = self.rev() p2ctx._descendantrev = self.rev()
results.append(p2ctx) results.append(p2ctx)
return results return results
def _nodefromancrev(self, ancrev, cl, mfl, path, fnode):
"""returns the node for <path> in <ancrev> if content matches <fnode>"""
ancctx = cl.read(ancrev) # This avoids object creation.
manifestnode, files = ancctx[0], ancctx[3]
# If the file was touched in this ancestor, and the content is similar
# to the one we are searching for.
if path in files and fnode == mfl[manifestnode].readfast().get(path):
return cl.node(ancrev)
return None
def _adjustlinknode(self, path, filelog, fnode, srcrev, inclusive=False): def _adjustlinknode(self, path, filelog, fnode, srcrev, inclusive=False):
"""return the first ancestor of <srcrev> introducing <fnode> """return the first ancestor of <srcrev> introducing <fnode>
If the linkrev of the file revision does not point to an ancestor of If the linkrev of the file revision does not point to an ancestor of
srcrev, we'll walk down the ancestors until we find one introducing srcrev, we'll walk down the ancestors until we find one introducing
this file revision. this file revision.
:repo: a localrepository object (used to access changelog and manifest) :repo: a localrepository object (used to access changelog and manifest)
revs = [p.rev() for p in self._repo[None].parents()] revs = [p.rev() for p in self._repo[None].parents()]
inclusive = True # we skipped the real (revless) source inclusive = True # we skipped the real (revless) source
else: else:
revs = [srcrev] revs = [srcrev]
if self._verifylinknode(revs, linknode): if self._verifylinknode(revs, linknode):
return linknode return linknode
pc = repo._phasecache pc = repo._phasecache
seenpublic = False seenpublic = False
iteranc = cl.ancestors(revs, inclusive=inclusive) iteranc = cl.ancestors(revs, inclusive=inclusive)
for ancrev in iteranc: for ancrev in iteranc:
# First, check locally-available history. # First, check locally-available history.
ancctx = cl.read(ancrev) # This avoids object creation. lnode = self._nodefromancrev(ancrev, cl, mfl, path, fnode)
stashUnsubmitted
Not Done

Sorry, I just realized I forgot to take draft phases into account :(.
In that case fastlog will be called even on draft revisions. Draft revisions are present only on your host, and that means that fastlog call will fail and it will slow things down.

To fix it we need to move this piece of code in the inner loop below, where we know that we are working with public revision.
See my comments below.

stash: Sorry, I just realized I forgot to take draft phases into account :(. In that case fastlog will…
manifestnode, files = ancctx[0], ancctx[3] if lnode is not None:
if path in files: return lnode
# The file was touched in this ancestor, so check if the
# content is similar to the one we are searching for.
if fnode == mfl[manifestnode].readfast().get(path):
return cl.node(ancrev)
# This next part is super non-obvious, so big comment block time! # This next part is super non-obvious, so big comment block time!
# #
# It is possible to get extremely bad performance here when a fairly # It is possible to get extremely bad performance here when a fairly
# common set of circumstances occur when this extension is combined # common set of circumstances occur when this extension is combined
# with a server-side commit rewriting extension like pushrebase. # with a server-side commit rewriting extension like pushrebase.
# #
# First, an engineer creates Commit A and pushes it to the server. # First, an engineer creates Commit A and pushes it to the server.
# #
# In rare circumstances (such as the server having a suboptimal # In rare circumstances (such as the server having a suboptimal
# linkrev for our use case), we will fall back to the old slow path. # linkrev for our use case), we will fall back to the old slow path.
# #
# We may want to add additional heuristics here in the future if # We may want to add additional heuristics here in the future if
# the slow path is used too much. One promising possibility is using # the slow path is used too much. One promising possibility is using
# obsolescence markers to find a more-likely-correct linkrev. # obsolescence markers to find a more-likely-correct linkrev.
if not seenpublic and pc.phase(repo, ancrev) == phases.public: if not seenpublic and pc.phase(repo, ancrev) == phases.public:
# If the commit is public and fastlog is enabled for this repo
# then we will can try to fetch the right linknode via fastlog
# since fastlog already has the right linkrev for all public
# commits
if repo.ui.configbool('fastlog', 'enabled'):
stashUnsubmitted
Not Done

Let's run fastlog only once - please run it only if seenpublic is False:

if not seenpublic and repo.ui.configbool('fastlog', 'enabled'):
stash: Let's run fastlog only once - please run it only if seenpublic is False: ``` if not seenpublic…
shivramkAuthorUnsubmitted
Not Done

The outer if condition is already checking for not seenpublic. Wouldn't this check be redundant?

shivramk: The outer if condition is already checking for not seenpublic. Wouldn't this check be redundant?
stashUnsubmitted
Not Done

Right, makes sense

stash: Right, makes sense
lnode = self._linknodeviafastlog(repo, path, ancrev, fnode,
cl, mfl)
stashUnsubmitted
Not Done

Formatting is wrong:

lnode = self._linknodeviafastlog(repo, path, ancrev, fnode,
                                 cl, mfl)

I suggest to use Nuclide - it highlights such problems automatically

stash: Formatting is wrong: ``` lnode = self._linknodeviafastlog(repo, path…
if lnode:
repo.ui.log('fastloglinkrevfixup', _('success\n'))
stashUnsubmitted
Not Done

Can you also log successful fastlog calls:

if lnode:
  repo.ui.log('fastloglinkrevfixup', _('success\n'))
  return lnode
stash: Can you also log successful fastlog calls: ``` if lnode: repo.ui.log('fastloglinkrevfixup'…
return lnode
seenpublic = True seenpublic = True
stashUnsubmitted
Not Done

Fastlog call should be here, and it should be called only once, so I suggest to have

if seenpublic == False and repo.ui.configbool('fastlog', 'enabled'):
            lnode = self._linknodeviafastlog(repo, path, ancrev, fnode, cl, mfl)
            if lnode:
                return lnode

seenpublic = True

(note ancrev here self._linknodeviafastlog(repo, path, ancrev, fnode, cl, mfl))

One more important thing - in "_linknodeviafastlog" you are currently doing "for anc in results[1:]:". Since now we are checking only public revisions, we don't need to skip the first element, so it should be "for anc in results".

Also I think that we need to have a test that covers this behavior. Lmk if you need help with it!

P. S.
Adding a comment about what we are doing is also a good idea.

stash: Fastlog call should be here, and it should be called only once, so I suggest to have ``` if…
try: try:
repo.fileservice.prefetch([(path, hex(fnode))], force=True) repo.fileservice.prefetch([(path, hex(fnode))], force=True)
# Now that we've downloaded a new blob from the server, # Now that we've downloaded a new blob from the server,
# we need to rebuild the ancestor map to recompute the # we need to rebuild the ancestor map to recompute the
# linknodes. # linknodes.
self._ancestormap = None self._ancestormap = None
linknode = self.ancestormap()[fnode][2] # 2 is linknode linknode = self.ancestormap()[fnode][2] # 2 is linknode
if self._verifylinknode(revs, linknode): if self._verifylinknode(revs, linknode):
return linknode return linknode
except Exception as e: except Exception as e:
errormsg = ('warning: failed to prefetch filepath %s ' + errormsg = ('warning: failed to prefetch filepath %s ' +
'while adjusting linknode %s (%s)\n(this is ' + 'while adjusting linknode %s (%s)\n(this is ' +
'generally benign but it may make ' + 'generally benign but it may make ' +
'this operation take longer to calculate ' + 'this operation take longer to calculate ' +
'things locally)') 'things locally)')
repo.ui.warn(_(errormsg) % (path, hex(linknode), e)) repo.ui.warn(_(errormsg) % (path, hex(linknode), e))
return linknode return linknode
def _linknodeviafastlog(self, repo, path, srcrev, fnode, cl, mfl):
reponame = repo.ui.config('fbconduit', 'reponame')
if self._conduit is None:
return None
try:
srchex = repo[srcrev].hex()
results = self._conduit.call_conduit(
'scmquery.log_v2',
timeout=FASTLOG_TIMEOUT_IN_SECS,
repo=reponame,
scm_type='hg',
rev=srchex,
file_paths=[path],
skip=0,
stashUnsubmitted
Not Done

I completely forgot about timeout, but luckily the change is simple:

Please add these two lines -

diff --git a/remotefilelog/remotefilectx.py b/remotefilelog/remotefilectx.py
--- a/remotefilelog/remotefilectx.py
+++ b/remotefilelog/remotefilectx.py
@@ -325,8 +325,10 @@
             return None
         try:
             srchex = repo[srcrev].hex()
+            FASTLOG_TIMEOUT_IN_SECS = 0.5
             results = self._conduit.call_conduit(
                 'scmquery.log_v2',
+                timeout=FASTLOG_TIMEOUT_IN_SECS,
                 repo=reponame,
                 scm_type='hg',
                 rev=srchex,

Also I added depended revision in phabricator that adds timeout functionality - D742. So if you want to test locally, then first you need to do hgarc patch D742 to get my changes, then rebase your changes on top.

But don't worry, I tested them myself.

stash: I completely forgot about timeout, but luckily the change is simple: Please add these two…
)
if results is None:
stashUnsubmitted
Not Done

To route error messages correctly, you need to do:

repo.ui.log('fastloglinkrevfixup', _('fastlog returned 0 results\n'))

Can you please change all of the invocations of repo.ui.log?

stash: To route error messages correctly, you need to do: ``` repo.ui.log('fastloglinkrevfixup', _…
repo.ui.log('fastloglinkrevfixup',
_('fastlog returned 0 results\n'))
stashUnsubmitted
Not Done

See above: you don't need to skip first element if you query only public commits.

stash: See above: you don't need to skip first element if you query only public commits.
return None
for anc in results:
ancrev = repo[str(anc['hash'])].rev()
lnode = self._nodefromancrev(ancrev, cl, mfl, path, fnode)
if lnode is not None:
return lnode
durhamUnsubmitted
Not Done

I'd explicitly return None here. We generally shouldn't rely on pythons default return None behavior for functions with actual return values.

durham: I'd explicitly return None here. We generally shouldn't rely on pythons default return None…
return None
except Exception as e:
repo.ui.log('fastloglinkrevfixup',
_('call to fastlog failed (%s)\n') % e)
return None
def _verifylinknode(self, revs, linknode): def _verifylinknode(self, revs, linknode):
""" """
Check if a linknode is correct one for the current history. Check if a linknode is correct one for the current history.
That is, return True if the linkrev is the ancestor of any of the That is, return True if the linkrev is the ancestor of any of the
passed in revs, otherwise return False. passed in revs, otherwise return False.
`revs` is a list that usually has one element -- usually the wdir parent `revs` is a list that usually has one element -- usually the wdir parent

tests/test-remotefilelog-linknodes.t

size: 6 bytes size: 6 bytes
path: $TESTTMP/hgcache/master/11/f6ad8ec52a2984abaafd7c3b516503785c2072/d4a3ed9310e5bd9887e3bf779da5077efab28216 path: $TESTTMP/hgcache/master/11/f6ad8ec52a2984abaafd7c3b516503785c2072/d4a3ed9310e5bd9887e3bf779da5077efab28216
key: d4a3ed9310e5 key: d4a3ed9310e5
node => p1 p2 linknode copyfrom node => p1 p2 linknode copyfrom
d4a3ed9310e5 => aee31534993a 000000000000 32e6611f6149 d4a3ed9310e5 => aee31534993a 000000000000 32e6611f6149
aee31534993a => 1406e7411862 000000000000 0632994590a8 aee31534993a => 1406e7411862 000000000000 0632994590a8
1406e7411862 => 000000000000 000000000000 b292c1e3311f 1406e7411862 => 000000000000 000000000000 b292c1e3311f
Test the same scenario as above but with fastlog enabled
$ cd ..
$ clearcache
$ rm -rf master
$ rm -rf shallow
$ hginit master
$ cd master
$ cat >> .hg/hgrc <<EOF
> [remotefilelog]
> server=True
> serverexpiration=-1
> EOF
$ echo x > x
$ hg commit -qAm x
$ echo x >> x
$ hg commit -Aqm xx
$ cd ..
$ hgcloneshallow ssh://user@dummy/master shallow -q
1 files fetched over 1 fetches - (1 misses, 0.00% hit ratio) over * (glob)
$ cd shallow
$ echo x >> x
$ hg commit -Aqm xx2
$ cd ../master
$ echo y >> y
$ hg commit -Aqm yy2
$ echo x >> x
$ hg commit -Aqm xx2-fake-rebased
$ echo y >> y
$ hg commit -Aqm yy3
$ cd ../shallow
$ hg pull -q
durhamUnsubmitted
Not Done

Why set debug to true here if there is no output?

durham: Why set debug to true here if there is no output?
$ hg update tip -q
1 files fetched over 1 fetches - (1 misses, 0.00% hit ratio) over *s (glob)
$ echo x > x
$ hg commit -qAm xx3
Verfiy correct linkrev despite fastlog failures
Case 1: fastlog service calls fails or times out
$ cat >> .hg/hgrc <<EOF
> [extensions]
> fbconduit=$TESTTMP/bad_conduit.py
> [fastlog]
> enabled=True
> EOF
$ cat > $TESTTMP/bad_conduit.py <<EOF
> def call_conduit(*args, **kwargs):
> raise Exception('error')
> def conduit_config(*args, **kwargs):
> return True
> EOF
$ hg log -f x -T '{node|short} {desc} {phase} {files}\n'
a5957b6bf0bd xx3 draft x
32e6611f6149 xx2-fake-rebased public x
0632994590a8 xx public x
b292c1e3311f x public x
1 files fetched over 1 fetches - (1 misses, 0.00% hit ratio) over *s (glob)
Case 2: fastlog returns empty results
$ cat > $TESTTMP/bad_conduit.py <<EOF
> def call_conduit(*args, **kwargs):
> return []
> def conduit_config(*args, **kwargs):
> return True
> EOF
$ rm $TESTTMP/bad_conduit.pyc
$ hg log -f x -T '{node|short} {desc} {phase} {files}\n'
a5957b6bf0bd xx3 draft x
32e6611f6149 xx2-fake-rebased public x
0632994590a8 xx public x
b292c1e3311f x public x
Case 3: fastlog returns a bad hash
$ cat > $TESTTMP/bad_conduit.py <<EOF
> def call_conduit(*args, **kwargs):
> return [{'hash': '123456'}]
> def conduit_config(*args, **kwargs):
> return True
> EOF
$ rm $TESTTMP/bad_conduit.pyc
$ hg log -f x -T '{node|short} {desc} {phase} {files}\n'
a5957b6bf0bd xx3 draft x
32e6611f6149 xx2-fake-rebased public x
0632994590a8 xx public x
b292c1e3311f x public x
Fastlog succeeds and returns the correct results
$ cat > $TESTTMP/bad_conduit.py <<EOF
> def call_conduit(*args, **kwargs):
> return [{'hash': '32e6611f6149e85f58def77ee0c22549bb6953a2'}]
> def conduit_config(*args, **kwargs):
> return True
> EOF
$ rm $TESTTMP/bad_conduit.pyc
$ hg log -f x -T '{node|short} {desc} {phase} {files}\n'
a5957b6bf0bd xx3 draft x
32e6611f6149 xx2-fake-rebased public x
0632994590a8 xx public x
b292c1e3311f x public x
Fastlog should never get called on draft commits
$ cat > $TESTTMP/bad_conduit.py <<EOF
> import sys
> def call_conduit(*args, **kwargs):
> if kwargs['rev'].startswith('a5957b6bf0bd'):
> sys.exit(80)
> return [{'hash': '32e6611f6149e85f58def77ee0c22549bb6953a2'}]
> def conduit_config(*args, **kwargs):
> return True
> EOF
$ rm $TESTTMP/bad_conduit.pyc
$ hg log -f x > /dev/null