This is an archive of the discontinued Mercurial Phabricator instance.

Differential D9951

narrow: add --no-backup option for narrowing
ClosedPublic

Authored by martinvonz on Feb 3 2021, 5:07 PM.

Details

Reviewers
durin42
pulkit
Group Reviewers
hg-reviewers
Commits
rHG3c360ab2688d: narrow: add --no-backup option for narrowing
Summary

Most of our users at Google use Mercurial on a file system that keeps
backups of previous versions of all files, including those in
.hg/. They therefore don't need a separate backup in the file system
when narrowing their repo (which they typically do by running `hg
tracked --auto-remove-includes`). Backups can be very slow. hg strip
already has a --no-backup option. This patch adds the same option to
hg tracked --removeinclude/--addexclude.

Diff Detail

Repository
rHG Mercurial
Lint
Automatic diff as part of commit; lint not applicable.
Unit
Automatic diff as part of commit; unit tests not applicable.

Event Timeline

martinvonz created this revision.Feb 3 2021, 5:07 PM
pulkit accepted this revision.Feb 10 2021, 7:08 AM
This revision is now accepted and ready to land.Feb 10 2021, 7:08 AM
pulkit added inline comments.Feb 10 2021, 7:09 AM
hgext/narrow/narrowcommands.py
276

Even when we backup, we can probably use soft-strip to make this faster.

martinvonz added a commit: rHG3c360ab2688d: narrow: add --no-backup option for narrowing.Feb 10 2021, 8:06 AM
Closed by commit rHG3c360ab2688d: narrow: add --no-backup option for narrowing (authored by martinvonz). · Explain Why
This revision was automatically updated to reflect the committed changes.
martinvonz added inline comments.Feb 10 2021, 12:22 PM
hgext/narrow/narrowcommands.py
276

Oh, good point! I initially thought that we couldn't, but I think you're right. The reason that narrowing strips commits is because local commits may have file revisions that we can't get back from the server one a later re-widening. That means that we need the backup. I didn't realize that soft-strip also supports creating a backup (and does so by default). I'll send a patch. Thanks!

martinvonz added inline comments.Feb 10 2021, 7:23 PM
hgext/narrow/narrowcommands.py
276

I finally remember the other reason for stripping commits (because tests reminded me). I was half right about later re-widening, but the backup is for users, and the real problem with re-widening is that don't apply a backup. So if your local commit touches some file and you then remove that path from your repo, we put the affected commit in a bundle, remove the commit, and then remove the revlog. When you later widen to include that path, we won't get the file revision from the server, which leaves that commit broken.

Revision Contents
Changeset List

Diff 25522

hgext/narrow/narrowcommands.py

repo, repo,
remote, remote,
commoninc, commoninc,
oldincludes, oldincludes,
oldexcludes, oldexcludes,
newincludes, newincludes,
newexcludes, newexcludes,
force, force,
backup,
): ):
oldmatch = narrowspec.match(repo.root, oldincludes, oldexcludes) oldmatch = narrowspec.match(repo.root, oldincludes, oldexcludes)
newmatch = narrowspec.match(repo.root, newincludes, newexcludes) newmatch = narrowspec.match(repo.root, newincludes, newexcludes)
# This is essentially doing "hg outgoing" to find all local-only # This is essentially doing "hg outgoing" to find all local-only
# commits. We will then check that the local-only commits don't # commits. We will then check that the local-only commits don't
# have any changes to files that will be untracked. # have any changes to files that will be untracked.
unfi = repo.unfiltered() unfi = repo.unfiltered()
b'(::%n) - %ln + null', b'(::%n) - %ln + null',
repo[b'.'].node(), repo[b'.'].node(),
visibletostrip, visibletostrip,
) )
) )
hg.clean(repo, urev) hg.clean(repo, urev)
overrides = {(b'devel', b'strip-obsmarkers'): False} overrides = {(b'devel', b'strip-obsmarkers'): False}
with ui.configoverride(overrides, b'narrow'): with ui.configoverride(overrides, b'narrow'):
repair.strip(ui, unfi, tostrip, topic=b'narrow') repair.strip(ui, unfi, tostrip, topic=b'narrow', backup=backup)
pulkitUnsubmitted
Not Done

Even when we backup, we can probably use soft-strip to make this faster.

pulkit: Even when we backup, we can probably use `soft-strip` to make this faster.
martinvonzAuthorUnsubmitted
Done

Oh, good point! I initially thought that we couldn't, but I think you're right. The reason that narrowing strips commits is because local commits may have file revisions that we can't get back from the server one a later re-widening. That means that we need the backup. I didn't realize that soft-strip also supports creating a backup (and does so by default). I'll send a patch. Thanks!

martinvonz: Oh, good point! I initially thought that we couldn't, but I think you're right. The reason that…
martinvonzAuthorUnsubmitted
Done

I finally remember the other reason for stripping commits (because tests reminded me). I was half right about later re-widening, but the backup is for users, and the real problem with re-widening is that don't apply a backup. So if your local commit touches some file and you then remove that path from your repo, we put the affected commit in a bundle, remove the commit, and then remove the revlog. When you later widen to include that path, we won't get the file revision from the server, which leaves that commit broken.

martinvonz: I finally remember the other reason for stripping commits (because tests reminded me). I was…
todelete = [] todelete = []
for f, f2, size in repo.store.datafiles(): for f, f2, size in repo.store.datafiles():
if f.startswith(b'data/'): if f.startswith(b'data/'):
file = f[5:-2] file = f[5:-2]
if not newmatch(file): if not newmatch(file):
todelete.append(f) todelete.append(f)
elif f.startswith(b'meta/'): elif f.startswith(b'meta/'):
( (
b'', b'',
b'force-delete-local-changes', b'force-delete-local-changes',
False, False,
_(b'forces deletion of local changes when narrowing'), _(b'forces deletion of local changes when narrowing'),
), ),
( (
b'', b'',
b'backup',
True,
_(b'back up local changes when narrowing'),
),
(
b'',
b'update-working-copy', b'update-working-copy',
False, False,
_(b'update working copy when the store has changed'), _(b'update working copy when the store has changed'),
), ),
] ]
+ commands.remoteopts, + commands.remoteopts,
_(b'[OPTIONS]... [REMOTE]'), _(b'[OPTIONS]... [REMOTE]'),
inferrepo=True, inferrepo=True,
repo, repo,
remote, remote,
commoninc, commoninc,
oldincludes, oldincludes,
oldexcludes, oldexcludes,
newincludes, newincludes,
newexcludes, newexcludes,
opts[b'force_delete_local_changes'], opts[b'force_delete_local_changes'],
opts[b'backup'],
) )
# _narrow() updated the narrowspec and _widen() below needs to # _narrow() updated the narrowspec and _widen() below needs to
# use the updated values as its base (otherwise removed includes # use the updated values as its base (otherwise removed includes
# and addedexcludes will be lost in the resulting narrowspec) # and addedexcludes will be lost in the resulting narrowspec)
oldincludes = newincludes oldincludes = newincludes
oldexcludes = newexcludes oldexcludes = newexcludes
if widening: if widening:

tests/test-narrow-trackedcmd.t

--auto-remove-includes automatically choose unused includes to --auto-remove-includes automatically choose unused includes to
remove remove
--addexclude VALUE [+] new paths to exclude --addexclude VALUE [+] new paths to exclude
--import-rules VALUE import narrowspecs from a file --import-rules VALUE import narrowspecs from a file
--removeexclude VALUE [+] old paths to no longer exclude --removeexclude VALUE [+] old paths to no longer exclude
--clear whether to replace the existing narrowspec --clear whether to replace the existing narrowspec
--force-delete-local-changes forces deletion of local changes when --force-delete-local-changes forces deletion of local changes when
narrowing narrowing
--[no-]backup back up local changes when narrowing
(default: on)
--update-working-copy update working copy when the store has --update-working-copy update working copy when the store has
changed changed
-e --ssh CMD specify ssh command to use -e --ssh CMD specify ssh command to use
--remotecmd CMD specify hg command to run on the remote side --remotecmd CMD specify hg command to run on the remote side
--insecure do not verify server certificate (ignoring --insecure do not verify server certificate (ignoring
web.cacerts config) web.cacerts config)
(use 'hg tracked -h' to show more help) (use 'hg tracked -h' to show more help)

tests/test-narrow.t

I path:d1 I path:d1
$ hg files $ hg files
d1/f d1/f
$ hg tracked --auto-remove-includes $ hg tracked --auto-remove-includes
comparing with ssh://user@dummy/master comparing with ssh://user@dummy/master
searching for changes searching for changes
looking for unused includes to remove looking for unused includes to remove
found no unused includes found no unused includes
Test --no-backup
$ hg tracked --addinclude d0 --addinclude d2 -q
$ hg unbundle .hg/strip-backup/*-narrow.hg -q
$ rm .hg/strip-backup/*
$ hg tracked --auto-remove-includes --no-backup
comparing with ssh://user@dummy/master
searching for changes
looking for unused includes to remove
path:d0
path:d2
remove these unused includes (yn)? y
looking for local changes to affected paths
deleting data/d0/f.i
deleting data/d2/f.i
deleting meta/d0/00manifest.i (tree !)
deleting meta/d2/00manifest.i (tree !)
$ ls .hg/strip-backup/