This is an archive of the discontinued Mercurial Phabricator instance.

Differential D1488

fastannotate: pass commitctx to filectx to prevent tree downloads
ClosedPublic

Authored by durham on Nov 21 2017, 5:32 PM.
Tags
None
Subscribers
quark

Details

Reviewers
quark
Group Reviewers
Restricted Project
Commits
rFBHGX41452c6d5805: fastannotate: pass commitctx to filectx to prevent tree downloads
Summary

Previously fastannotate was creating filectx's by doing commitctx[path]. This
invokes logic that resovles the filenode immediately, which require loading the
manifest for that commit. In a repo where manifests are downloaded lazily, this
can result in a lot of time spent downloading manifests.

Since commands like hg blame -u only need the filectx so they can resolve the
commitctx for commit information, let's just pass the commitctx straight to the
filectx. It can later derive the filenode if it needs to.

Diff Detail

Repository
rFBHGX Facebook Mercurial Extensions
Lint
Automatic diff as part of commit; lint not applicable.
Unit
Automatic diff as part of commit; unit tests not applicable.

Event Timeline

durham created this revision.Nov 21 2017, 5:32 PM
Herald added a reviewer: Restricted Project. · View Herald TranscriptNov 21 2017, 5:32 PM
quark accepted this revision.Nov 21 2017, 9:30 PM
quark added a subscriber: quark.

Thanks!

This revision is now accepted and ready to land.Nov 21 2017, 9:30 PM
Closed by commit rFBHGX41452c6d5805: fastannotate: pass commitctx to filectx to prevent tree downloads (authored by durham). · Explain WhyNov 28 2017, 5:59 PM
This revision was automatically updated to reflect the committed changes.

Revision Contents
Changeset List

Diff 3922

hgext3rd/fastannotate/context.py

that introduces the change (last modified the file). if adjustctx that introduces the change (last modified the file). if adjustctx
is 'linkrev', trust the linkrev and do not adjust it. this is noticeably is 'linkrev', trust the linkrev and do not adjust it. this is noticeably
faster for big repos but is incorrect for some cases. faster for big repos but is incorrect for some cases.
""" """
if resolverev and not isinstance(rev, int) and rev is not None: if resolverev and not isinstance(rev, int) and rev is not None:
ctx = _revsingle(repo, rev) ctx = _revsingle(repo, rev)
else: else:
ctx = repo[rev] ctx = repo[rev]
# If we don't need to adjust the linkrev, create the filectx using the
# changectx instead of using ctx[path]. This means it already has the
# changectx information, so blame -u will be able to look directly at the
# commitctx object instead of having to resolve it by going through the
# manifest. In a lazy-manifest world this can prevent us from downloading a
# lot of data.
if adjustctx is None:
# ctx.rev() is None means it's the working copy, which is a special
# case.
if ctx.rev() is None:
fctx = ctx[path]
else:
fctx = repo.filectx(path, changeid=ctx.rev())
else:
fctx = ctx[path] fctx = ctx[path]
if adjustctx is not None:
if adjustctx == 'linkrev': if adjustctx == 'linkrev':
introrev = fctx.linkrev() introrev = fctx.linkrev()
else: else:
introrev = fctx.introrev() introrev = fctx.introrev()
if introrev != ctx.rev(): if introrev != ctx.rev():
fctx._changeid = introrev fctx._changeid = introrev
fctx._changectx = repo[introrev] fctx._changectx = repo[introrev]
return fctx return fctx

tests/test-treemanifest-blame.t

  • This file was added.
$ . "$TESTDIR/library.sh"
$ PYTHONPATH=$TESTDIR/..:$PYTHONPATH
$ export PYTHONPATH
$ cat >> $HGRCPATH <<EOF
> [treemanifest]
> sendtrees=True
> [fastannotate]
> mainbranch=default
> modes=fctx
> EOF
Setup the server
$ hginit master
$ cd master
$ cat >> .hg/hgrc <<EOF
> [extensions]
> fastannotate=$TESTDIR/../hgext3rd/fastannotate
> treemanifest=$TESTDIR/../treemanifest
> [treemanifest]
> server=True
> [remotefilelog]
> server=True
> shallowtrees=True
> [fastannotate]
> server=True
> EOF
Make local commits on the server
$ mkdir subdir
$ echo x >> subdir/x
$ hg commit -qAm 'add subdir/x'
$ echo x >> subdir/x
$ hg commit -qAm 'modify subdir/x'
$ echo x >> subdir/x
$ hg commit -qAm 'modify subdir/x 2'
$ echo x >> subdir/x
$ hg commit -qAm 'modify subdir/x 3'
Run blame on client
$ cd ..
$ hgcloneshallow ssh://user@dummy/master client -q
1 files fetched over 1 fetches - (1 misses, 0.00% hit ratio) over * (glob)
$ cd client
$ cat >> .hg/hgrc <<EOF
> [extensions]
> fastannotate=$TESTDIR/../hgext3rd/fastannotate
> fastmanifest=$TESTDIR/../fastmanifest
> treemanifest=$TESTDIR/../treemanifest
> [fastmanifest]
> usecache=False
> usetree=True
> [treemanifest]
> demanddownload=True
> [fastannotate]
> clientfetchthreshold=2
> EOF
$ clearcache
$ hg prefetch -r 'tip^::tip'
4 trees fetched over * (glob)
2 files fetched over 1 fetches - (2 misses, 0.00% hit ratio) over * (glob)
- Verify no trees are downloaded
$ hg blame -r tip -u subdir/x --pager=off
test: x
test: x
test: x
test: x