This is an archive of the discontinued Mercurial Phabricator instance.

Differential D1456

treemanifest: add command for backfilling manifests revlog
ClosedPublic

Authored by durham on Nov 20 2017, 11:28 AM.
Tags
None
Subscribers
quark

Details

Reviewers
quark
Group Reviewers
Restricted Project
Commits
rFBHGX19d68f048608: treemanifest: add command for backfilling manifests revlog
Summary

Previously, once you transitioned to treeonly mode you had no way of switching
back to hybrid mode. This patch adds a hg backfillmanifestrevlog command to
refill a manifest revlog with any bits that are missing.

Diff Detail

Repository
rFBHGX Facebook Mercurial Extensions
Lint
Automatic diff as part of commit; lint not applicable.
Unit
Automatic diff as part of commit; unit tests not applicable.

Event Timeline

durham created this revision.Nov 20 2017, 11:28 AM
Herald added a reviewer: Restricted Project. · View Herald TranscriptNov 20 2017, 11:28 AM
durham updated this revision to Diff 3647.Nov 20 2017, 11:41 AM
quark accepted this revision.Nov 21 2017, 3:21 AM
quark added a subscriber: quark.
quark added inline comments.
treemanifest/__init__.py
682

This revset is expensive (O(len(changelog)) in Python code path). Maybe use

(heads(parents(roots(draft()))) + head()) & public()

instead.

683–684

More efficient if cl.node is inlined as a variable.

This revision is now accepted and ready to land.Nov 21 2017, 3:21 AM
durham added inline comments.Nov 21 2017, 9:59 AM
treemanifest/__init__.py
682

This won't get the same results though right? If some draft commits branch off some arbitrary spot in the history of master, that spot will be in the result, despite not being a head

Closed by commit rFBHGX19d68f048608: treemanifest: add command for backfilling manifests revlog (authored by durham). · Explain WhyNov 21 2017, 5:13 PM
This revision was automatically updated to reflect the committed changes.

Revision Contents
Changeset List

Diff 3753

tests/test-treemanifest-treeonly.t

tag: tip tag: tip
user: test user: test
date: Thu Jan 01 00:00:00 1970 +0000 date: Thu Jan 01 00:00:00 1970 +0000
summary: add y summary: add y
y | 1 + y | 1 +
1 files changed, 1 insertions(+), 0 deletions(-) 1 files changed, 1 insertions(+), 0 deletions(-)
Switch back to hybrid mode
TODO $ cd ../client
# log -T "{manifest}" #TODO: edit templatekw.showmanifest $ cat >> .hg/hgrc <<EOF
> [treemanifest]
> treeonly=False
> EOF
$ hg pull
pulling from ssh://user@dummy/master
searching for changes
adding changesets
adding manifests
transaction abort!
rollback completed
abort: 00manifest.i@d9920715ba88: unknown parent!
[255]
$ hg backfillmanifestrevlog
adding changesets
adding manifests
adding file changes
added 0 changesets with 0 changes to 0 files
$ hg pull
pulling from ssh://user@dummy/master
searching for changes
adding changesets
adding manifests
adding file changes
added 1 changesets with 0 changes to 0 files (+1 heads)
new changesets 4f84204095e0
(run 'hg heads .' to see heads, 'hg merge' to merge)

treemanifest/__init__.py

packpath = shallowutil.getcachepackpath(repo, PACK_CATEGORY) packpath = shallowutil.getcachepackpath(repo, PACK_CATEGORY)
if opts.get('skip_allowed_roots', False): if opts.get('skip_allowed_roots', False):
ui.setconfig('treemanifest', 'allowedtreeroots', None) ui.setconfig('treemanifest', 'allowedtreeroots', None)
with mutabledatapack(repo.ui, packpath) as dpack: with mutabledatapack(repo.ui, packpath) as dpack:
with mutablehistorypack(repo.ui, packpath) as hpack: with mutablehistorypack(repo.ui, packpath) as hpack:
recordmanifest(dpack, hpack, repo, mfrev1, mfrev2, recordmanifest(dpack, hpack, repo, mfrev1, mfrev2,
verify=opts.get('verify', False)) verify=opts.get('verify', False))
@command('backfillmanifestrevlog', [
], _('hg backfillmanifestrevlog'))
def backfillmanifestrevlog(ui, repo, *args, **opts):
"""Download any missing manifest revlog entries. This is useful when
transitioning back from a treeonly repo to a flat+tree hybrid repo."""
fallbackpath = getfallbackpath(repo)
with repo.connectionpool.get(fallbackpath) as conn:
remote = conn.peer
# _localrepo is needed for remotefilelog to work
if util.safehasattr(remote, '_callstream'):
remote._localrepo = repo
cl = repo.changelog
mfrevlog = repo.manifestlog._revlog
# We need to download any manifests the server has that we don't. We
# calculate that by saying we need all the public heads, and that we
# have some of them already. This might result in extra downloading but
# they become no-ops when attempting to be added to the revlog.
publicheads = repo.revs('heads(public())')
quarkUnsubmitted
Not Done

This revset is expensive (O(len(changelog)) in Python code path). Maybe use

(heads(parents(roots(draft()))) + head()) & public()

instead.

quark: This revset is expensive (O(len(changelog)) in Python code path). Maybe use (heads(parents…
durhamAuthorUnsubmitted
Not Done

This won't get the same results though right? If some draft commits branch off some arbitrary spot in the history of master, that spot will be in the result, despite not being a head

durham: This won't get the same results though right? If some draft commits branch off some arbitrary…
clnode = cl.node
heads = [clnode(r) for r in publicheads]
quarkUnsubmitted
Not Done

More efficient if cl.node is inlined as a variable.

quark: More efficient if `cl.node` is inlined as a variable.
common = [clnode(r) for r in publicheads if
cl.changelogrevision(r).manifest in mfrevlog.nodemap]
with repo.wlock(), repo.lock(), (
repo.transaction("backfillmanifest")) as tr:
cg = remote.getbundle('pull', common=common, heads=heads)
bundle2.applybundle(repo, cg, tr, 'pull', remote.url())
@command('backfilltree', [ @command('backfilltree', [
('l', 'limit', '10000000', _('')) ('l', 'limit', '10000000', _(''))
], _('hg backfilltree [OPTIONS]')) ], _('hg backfilltree [OPTIONS]'))
def backfilltree(ui, repo, *args, **opts): def backfilltree(ui, repo, *args, **opts):
with repo.wlock(), repo.lock(), repo.transaction('backfilltree') as tr: with repo.wlock(), repo.lock(), repo.transaction('backfilltree') as tr:
_backfill(tr, repo, int(opts.get('limit'))) _backfill(tr, repo, int(opts.get('limit')))
def _backfill(tr, repo, limit): def _backfill(tr, repo, limit):