( )⚙ D6532 export: don't prefetch *all* files in manifest

This is an archive of the discontinued Mercurial Phabricator instance.

Differential D6532

export: don't prefetch *all* files in manifest
ClosedPublic

Authored by martinvonz on Jun 14 2019, 5:27 PM.

Details

Reviewers
pulkit
Group Reviewers
hg-reviewers
Commits
rHG307f67d4aee3: export: don't prefetch *all* files in manifest
Summary

hg export only shows changed files, not all files, but we still
prefetched all files in cmdutil.export(). The same is true for the
other commands calling cmdutil.exportfile(). That meant that `hg
export` with remotefilelog (or lfs, I assume) could take much longer
than expected because it would download all the files in the repo.

Diff Detail

Repository
rHG Mercurial
Lint
Automatic diff as part of commit; lint not applicable.
Unit
Automatic diff as part of commit; unit tests not applicable.

Event Timeline

martinvonz created this revision.Jun 14 2019, 5:27 PM
pulkit accepted this revision.Jun 14 2019, 6:44 PM
This revision is now accepted and ready to land.Jun 14 2019, 6:44 PM
martinvonz added a commit: rHG307f67d4aee3: export: don't prefetch *all* files in manifest.Jun 14 2019, 7:15 PM
Closed by commit rHG307f67d4aee3: export: don't prefetch *all* files in manifest (authored by martinvonz). · Explain Why
This revision was automatically updated to reflect the committed changes.

Revision Contents
Changeset List

PathPackages
Mmercurial/cmdutil.py (12 lines)

Diff 15522

mercurial/cmdutil.py

with formatter.maybereopen(basefm, dest) as fm: with formatter.maybereopen(basefm, dest) as fm:
repo.ui.note("%s\n" % dest) repo.ui.note("%s\n" % dest)
for seqno, rev in filemap[dest]: for seqno, rev in filemap[dest]:
fm.startitem() fm.startitem()
ctx = repo[rev] ctx = repo[rev]
_exportsingle(repo, ctx, fm, match, switch_parent, seqno, _exportsingle(repo, ctx, fm, match, switch_parent, seqno,
diffopts) diffopts)
def _prefetchchangedfiles(repo, revs, match):
allfiles = set()
for rev in revs:
for file in repo[rev].files():
if not match or match(file):
allfiles.add(file)
scmutil.prefetchfiles(repo, revs, scmutil.matchfiles(repo, allfiles))
def export(repo, revs, basefm, fntemplate='hg-%h.patch', switch_parent=False, def export(repo, revs, basefm, fntemplate='hg-%h.patch', switch_parent=False,
opts=None, match=None): opts=None, match=None):
'''export changesets as hg patches '''export changesets as hg patches
Args: Args:
repo: The repository from which we're exporting revisions. repo: The repository from which we're exporting revisions.
revs: A list of revisions to export as revision numbers. revs: A list of revisions to export as revision numbers.
basefm: A formatter to which patches should be written. basefm: A formatter to which patches should be written.
fntemplate: An optional string to use for generating patch file names. fntemplate: An optional string to use for generating patch file names.
switch_parent: If True, show diffs against second parent when not nullid. switch_parent: If True, show diffs against second parent when not nullid.
Default is false, which always shows diff against p1. Default is false, which always shows diff against p1.
opts: diff options to use for generating the patch. opts: diff options to use for generating the patch.
match: If specified, only export changes to files matching this matcher. match: If specified, only export changes to files matching this matcher.
Returns: Returns:
Nothing. Nothing.
Side Effect: Side Effect:
"HG Changeset Patch" data is emitted to one of the following "HG Changeset Patch" data is emitted to one of the following
destinations: destinations:
fntemplate specified: Each rev is written to a unique file named using fntemplate specified: Each rev is written to a unique file named using
the given template. the given template.
Otherwise: All revs will be written to basefm. Otherwise: All revs will be written to basefm.
''' '''
scmutil.prefetchfiles(repo, revs, match) _prefetchchangedfiles(repo, revs, match)
if not fntemplate: if not fntemplate:
_exportfile(repo, revs, basefm, '<unnamed>', switch_parent, opts, match) _exportfile(repo, revs, basefm, '<unnamed>', switch_parent, opts, match)
else: else:
_exportfntemplate(repo, revs, basefm, fntemplate, switch_parent, opts, _exportfntemplate(repo, revs, basefm, fntemplate, switch_parent, opts,
match) match)
def exportfile(repo, revs, fp, switch_parent=False, opts=None, match=None): def exportfile(repo, revs, fp, switch_parent=False, opts=None, match=None):
"""Export changesets to the given file stream""" """Export changesets to the given file stream"""
scmutil.prefetchfiles(repo, revs, match) _prefetchchangedfiles(repo, revs, match)
dest = getattr(fp, 'name', '<unnamed>') dest = getattr(fp, 'name', '<unnamed>')
with formatter.formatter(repo.ui, fp, 'export', {}) as fm: with formatter.formatter(repo.ui, fp, 'export', {}) as fm:
_exportfile(repo, revs, fm, dest, switch_parent, opts, match) _exportfile(repo, revs, fm, dest, switch_parent, opts, match)
def showmarker(fm, marker, index=None): def showmarker(fm, marker, index=None):
"""utility function to display obsolescence marker in a readable way """utility function to display obsolescence marker in a readable way