This is an archive of the discontinued Mercurial Phabricator instance.

Differential D839

remotefilelog: time fastlog requests
ClosedPublic

Authored by stash on Sep 28 2017, 1:01 PM.
Tags
None
Subscribers
None

Details

Reviewers
quark
Group Reviewers
Restricted Project
Commits
rFBHGXf629119c6242: remotefilelog: time fastlog requests
Summary

The goal of the whole series is to have logging around linknode fixup. It's a slow operation, and we have two heuristics to make it faster. Unfortunately we have no idea about how well these heuristics actually help. This series of diffs aims to fix this problem. After this series is landed and logging is enabled, I hope that we can find out if we really these heuristics at all.

Test Plan

Run test-remotefilelog-linknodes.t

Diff Detail

Repository
rFBHGX Facebook Mercurial Extensions
Lint
Automatic diff as part of commit; lint not applicable.
Unit
Automatic diff as part of commit; unit tests not applicable.

Event Timeline

stash created this revision.Sep 28 2017, 1:01 PM
stash edited the summary of this revision. (Show Details)Sep 28 2017, 1:10 PM
quark accepted this revision.Sep 29 2017, 9:49 PM
This revision is now accepted and ready to land.Sep 29 2017, 9:49 PM
stash edited the summary of this revision. (Show Details)Sep 30 2017, 3:27 PM
stash updated this revision to Diff 2206.

rebase

Closed by commit rFBHGXf629119c6242: remotefilelog: time fastlog requests (authored by stash). · Explain WhyOct 1 2017, 9:00 AM
This revision was automatically updated to reflect the committed changes.

Revision Contents
Changeset List

Diff 2273

remotefilelog/remotefilectx.py

# remotefilectx.py - filectx/workingfilectx implementations for remotefilelog # remotefilectx.py - filectx/workingfilectx implementations for remotefilelog
# #
# Copyright 2013 Facebook, Inc. # Copyright 2013 Facebook, Inc.
# #
# This software may be used and distributed according to the terms of the # This software may be used and distributed according to the terms of the
# GNU General Public License version 2 or any later version. # GNU General Public License version 2 or any later version.
import collections import collections
import time
from mercurial.i18n import _ from mercurial.i18n import _
from mercurial.node import bin, hex, nullid, nullrev from mercurial.node import bin, hex, nullid, nullrev
from mercurial import context, util, error, ancestor, phases, extensions from mercurial import context, util, error, ancestor, phases, extensions
from . import shallowutil from . import shallowutil
propertycache = util.propertycache propertycache = util.propertycache
conduit = None conduit = None
FASTLOG_TIMEOUT_IN_SECS = 0.5 FASTLOG_TIMEOUT_IN_SECS = 0.5
'this operation take longer to calculate ' + 'this operation take longer to calculate ' +
'things locally)') 'things locally)')
repo.ui.warn(_(errormsg) % (path, hex(linknode), e)) repo.ui.warn(_(errormsg) % (path, hex(linknode), e))
return linknode return linknode
def _linknodeviafastlog(self, repo, path, srcrev, fnode, cl, mfl, def _linknodeviafastlog(self, repo, path, srcrev, fnode, cl, mfl,
commonlogkwargs): commonlogkwargs):
start = time.time()
reponame = repo.ui.config('fbconduit', 'reponame') reponame = repo.ui.config('fbconduit', 'reponame')
logmsg = '' logmsg = ''
if self._conduit is None: if self._conduit is None:
return None return None
try: try:
srchex = repo[srcrev].hex() srchex = repo[srcrev].hex()
results = self._conduit.call_conduit( results = self._conduit.call_conduit(
'scmquery.log_v2', 'scmquery.log_v2',
logmsg = 'fastlog succeded' logmsg = 'fastlog succeded'
return lnode return lnode
logmsg = 'fastlog succeded but linknode was not found' logmsg = 'fastlog succeded but linknode was not found'
return None return None
except Exception as e: except Exception as e:
logmsg = 'fastlog failed (%s)' % e logmsg = 'fastlog failed (%s)' % e
return None return None
finally: finally:
repo.ui.log('linkrevfixup', logmsg, **commonlogkwargs) elapsed = time.time() - start
repo.ui.log('linkrevfixup', logmsg, elapsed=elapsed * 1000,
**commonlogkwargs)
def _verifylinknode(self, revs, linknode): def _verifylinknode(self, revs, linknode):
""" """
Check if a linknode is correct one for the current history. Check if a linknode is correct one for the current history.
That is, return True if the linkrev is the ancestor of any of the That is, return True if the linkrev is the ancestor of any of the
passed in revs, otherwise return False. passed in revs, otherwise return False.

tests/test-remotefilelog-linknodes.t

$ cat >> $HGRCPATH <<EOF $ cat >> $HGRCPATH <<EOF
> [extensions] > [extensions]
> uilog=$TESTTMP/uilog.py > uilog=$TESTTMP/uilog.py
> EOF > EOF
Silencing stdout because we are interested only in ui.log output Silencing stdout because we are interested only in ui.log output
$ hg log -f x -T '{node|short} {desc} {phase} {files}\n' > /dev/null $ hg log -f x -T '{node|short} {desc} {phase} {files}\n' > /dev/null
linkrevfixup: adjusting linknode (filepath=x, fnode=d4a3ed9310e5bd9887e3bf779da5077efab28216, reponame=master, revs=a5957b6bf0bdeb9b96368bddd2838004ad966b7d, user=test) linkrevfixup: adjusting linknode (filepath=x, fnode=d4a3ed9310e5bd9887e3bf779da5077efab28216, reponame=master, revs=a5957b6bf0bdeb9b96368bddd2838004ad966b7d, user=test)
linkrevfixup: fastlog succeded (filepath=x, fnode=d4a3ed9310e5bd9887e3bf779da5077efab28216, reponame=master, revs=a5957b6bf0bdeb9b96368bddd2838004ad966b7d, user=test) linkrevfixup: fastlog succeded (elapsed=*, filepath=x, fnode=d4a3ed9310e5bd9887e3bf779da5077efab28216, reponame=master, revs=a5957b6bf0bdeb9b96368bddd2838004ad966b7d, user=test) (glob)
Fastlog fails Fastlog fails
$ cat > $TESTTMP/bad_conduit.py <<EOF $ cat > $TESTTMP/bad_conduit.py <<EOF
> def call_conduit(*args, **kwargs): > def call_conduit(*args, **kwargs):
> raise Exception('error') > raise Exception('error')
> def conduit_config(*args, **kwargs): > def conduit_config(*args, **kwargs):
> return True > return True
> EOF > EOF
$ rm $TESTTMP/bad_conduit.pyc $ rm $TESTTMP/bad_conduit.pyc
$ hg log -f x -T '{node|short} {desc} {phase} {files}\n' > /dev/null $ hg log -f x -T '{node|short} {desc} {phase} {files}\n' > /dev/null
linkrevfixup: adjusting linknode (filepath=x, fnode=d4a3ed9310e5bd9887e3bf779da5077efab28216, reponame=master, revs=a5957b6bf0bdeb9b96368bddd2838004ad966b7d, user=test) linkrevfixup: adjusting linknode (filepath=x, fnode=d4a3ed9310e5bd9887e3bf779da5077efab28216, reponame=master, revs=a5957b6bf0bdeb9b96368bddd2838004ad966b7d, user=test)
linkrevfixup: fastlog failed (error) (filepath=x, fnode=d4a3ed9310e5bd9887e3bf779da5077efab28216, reponame=master, revs=a5957b6bf0bdeb9b96368bddd2838004ad966b7d, user=test) linkrevfixup: fastlog failed (error) (elapsed=*, filepath=x, fnode=d4a3ed9310e5bd9887e3bf779da5077efab28216, reponame=master, revs=a5957b6bf0bdeb9b96368bddd2838004ad966b7d, user=test) (glob)