( )⚙ D12155 sparse: rework debugsparse's interface

This is an archive of the discontinued Mercurial Phabricator instance.

Differential D12155

sparse: rework debugsparse's interface
ClosedPublic

Authored by valentin.gatienbaron on Feb 8 2022, 11:32 PM.

Details

Reviewers
None
Group Reviewers
hg-reviewers
Commits
rHGa6efb9180764: sparse: rework debugsparse's interface
Summary

hg debugsparse supports arguments like --include, similar to `hg
tracked --addinclude` or hg log --include. But in hg debugsparse,
the pattern is not an argument of the flag, instead the patterns are
the anonymous command line arguments.

Not only is this surprising, it makes it impossible to use --include
and --exclude in the same invocation, or --reset --exclude.

So I propose making debugsparse making --include, --exclude take an
argument, and rejecting anonymous command line arguments, as well as
allowing mixing several of these flags in one invocations.

Diff Detail

Repository
rHG Mercurial
Lint
Automatic diff as part of commit; lint not applicable.
Unit
Automatic diff as part of commit; unit tests not applicable.

Event Timeline

Alphare added a subscriber: Alphare.Feb 9 2022, 11:18 AM

I don't know if we have dropped py2 by now or not.

We're dropping Python 2 right after the 6.1 release due for on or around March 1st. Provided this can wait about a month, we can just take it as is then.

As for the patch, I've not read it in detail yet, but the overall idea seems sane to me.

valentin.gatienbaron edited the summary of this revision. (Show Details)Feb 9 2022, 6:29 PM
valentin.gatienbaron updated this revision to Diff 32111.
valentin.gatienbaron added a comment.Feb 9 2022, 6:33 PM
In D12155#186341, @Alphare wrote:

I don't know if we have dropped py2 by now or not.

We're dropping Python 2 right after the 6.1 release due for on or around March 1st. Provided this can wait about a month, we can just take it as is then.

Ok. I just dropped the part of the diff that required py3 then. It's not essential at all.

valentin.gatienbaron added a commit: rHGa6efb9180764: sparse: rework debugsparse's interface.Feb 14 2022, 5:46 AM
This revision was not accepted when it landed; it landed in state Needs Review.
Closed by commit rHGa6efb9180764: sparse: rework debugsparse's interface (authored by valentin.gatienbaron). · Explain Why
This revision was automatically updated to reflect the committed changes.

Revision Contents
Changeset List

PathPackages
Mhgext/sparse.py (98 lines)
Mmercurial/sparse.py (47 lines)
Mtests/test-sparse-clear.t (2 lines)
Mtests/test-sparse.t (13 lines)

Diff 32163

hgext/sparse.py

tools/tests/** tools/tests/**
""" """
from __future__ import absolute_import from __future__ import absolute_import
from mercurial.i18n import _ from mercurial.i18n import _
from mercurial.pycompat import setattr from mercurial.pycompat import setattr
from mercurial import ( from mercurial import (
cmdutil,
commands, commands,
dirstate, dirstate,
error, error,
extensions, extensions,
logcmdutil, logcmdutil,
match as matchmod, match as matchmod,
merge as mergemod, merge as mergemod,
pycompat, pycompat,
revs = revs.filter(ctxmatch) revs = revs.filter(ctxmatch)
return revs return revs
extensions.wrapfunction(logcmdutil, b'_initialrevs', _initialrevs) extensions.wrapfunction(logcmdutil, b'_initialrevs', _initialrevs)
def _clonesparsecmd(orig, ui, repo, *args, **opts): def _clonesparsecmd(orig, ui, repo, *args, **opts):
include_pat = opts.get('include') include = opts.get('include')
exclude_pat = opts.get('exclude') exclude = opts.get('exclude')
enableprofile_pat = opts.get('enable_profile') enableprofile = opts.get('enable_profile')
narrow_pat = opts.get('narrow') narrow_pat = opts.get('narrow')
include = exclude = enableprofile = False
if include_pat:
pat = include_pat
include = True
if exclude_pat:
pat = exclude_pat
exclude = True
if enableprofile_pat:
pat = enableprofile_pat
enableprofile = True
if sum([include, exclude, enableprofile]) > 1:
raise error.Abort(_(b"too many flags specified."))
# if --narrow is passed, it means they are includes and excludes for narrow # if --narrow is passed, it means they are includes and excludes for narrow
# clone # clone
if not narrow_pat and (include or exclude or enableprofile): if not narrow_pat and (include or exclude or enableprofile):
def clonesparse(orig, ctx, *args, **kwargs): def clonesparse(orig, ctx, *args, **kwargs):
sparse.updateconfig( sparse.updateconfig(
ctx.repo().unfiltered(), ctx.repo().unfiltered(),
pat,
{}, {},
include=include, include=include,
exclude=exclude, exclude=exclude,
enableprofile=enableprofile, enableprofile=enableprofile,
usereporootpaths=True, usereporootpaths=True,
) )
return orig(ctx, *args, **kwargs) return orig(ctx, *args, **kwargs)
) )
def _add(orig, ui, repo, *pats, **opts): def _add(orig, ui, repo, *pats, **opts):
if opts.get('sparse'): if opts.get('sparse'):
dirs = set() dirs = set()
for pat in pats: for pat in pats:
dirname, basename = util.split(pat) dirname, basename = util.split(pat)
dirs.add(dirname) dirs.add(dirname)
sparse.updateconfig(repo, list(dirs), opts, include=True) sparse.updateconfig(repo, opts, include=list(dirs))
return orig(ui, repo, *pats, **opts) return orig(ui, repo, *pats, **opts)
extensions.wrapcommand(commands.table, b'add', _add) extensions.wrapcommand(commands.table, b'add', _add)
def _setupdirstate(ui): def _setupdirstate(ui):
"""Modify the dirstate to prevent stat'ing excluded files, """Modify the dirstate to prevent stat'ing excluded files,
and to prevent modifications to files outside the checkout. and to prevent modifications to files outside the checkout.
return orig(self, *args, **kwargs) return orig(self, *args, **kwargs)
extensions.wrapfunction(dirstate.dirstate, func, _wrapper) extensions.wrapfunction(dirstate.dirstate, func, _wrapper)
@command( @command(
b'debugsparse', b'debugsparse',
[ [
(b'I', b'include', False, _(b'include files in the sparse checkout')), (
(b'X', b'exclude', False, _(b'exclude files in the sparse checkout')), b'I',
(b'd', b'delete', False, _(b'delete an include/exclude rule')), b'include',
[],
_(b'include files in the sparse checkout'),
_(b'PATTERN'),
),
(
b'X',
b'exclude',
[],
_(b'exclude files in the sparse checkout'),
_(b'PATTERN'),
),
(
b'd',
b'delete',
[],
_(b'delete an include/exclude rule'),
_(b'PATTERN'),
),
( (
b'f', b'f',
b'force', b'force',
False, False,
_(b'allow changing rules even with pending changes'), _(b'allow changing rules even with pending changes'),
), ),
(b'', b'enable-profile', False, _(b'enables the specified profile')), (
(b'', b'disable-profile', False, _(b'disables the specified profile')), b'',
(b'', b'import-rules', False, _(b'imports rules from a file')), b'enable-profile',
[],
_(b'enables the specified profile'),
_(b'PATTERN'),
),
(
b'',
b'disable-profile',
[],
_(b'disables the specified profile'),
_(b'PATTERN'),
),
(
b'',
b'import-rules',
[],
_(b'imports rules from a file'),
_(b'PATTERN'),
),
(b'', b'clear-rules', False, _(b'clears local include/exclude rules')), (b'', b'clear-rules', False, _(b'clears local include/exclude rules')),
( (
b'', b'',
b'refresh', b'refresh',
False, False,
_(b'updates the working after sparseness changes'), _(b'updates the working after sparseness changes'),
), ),
(b'', b'reset', False, _(b'makes the repo full again')), (b'', b'reset', False, _(b'makes the repo full again')),
] ]
+ commands.templateopts, + commands.templateopts,
_(b'[--OPTION] PATTERN...'), _(b'[--OPTION]'),
helpbasic=True, helpbasic=True,
) )
def debugsparse(ui, repo, *pats, **opts): def debugsparse(ui, repo, **opts):
"""make the current checkout sparse, or edit the existing checkout """make the current checkout sparse, or edit the existing checkout
The sparse command is used to make the current checkout sparse. The sparse command is used to make the current checkout sparse.
This means files that don't meet the sparse condition will not be This means files that don't meet the sparse condition will not be
written to disk, or show up in any working copy operations. It does written to disk, or show up in any working copy operations. It does
not affect files in history in any way. not affect files in history in any way.
Passing no arguments prints the currently applied sparse rules. Passing no arguments prints the currently applied sparse rules.
force = opts.get(b'force') force = opts.get(b'force')
enableprofile = opts.get(b'enable_profile') enableprofile = opts.get(b'enable_profile')
disableprofile = opts.get(b'disable_profile') disableprofile = opts.get(b'disable_profile')
importrules = opts.get(b'import_rules') importrules = opts.get(b'import_rules')
clearrules = opts.get(b'clear_rules') clearrules = opts.get(b'clear_rules')
delete = opts.get(b'delete') delete = opts.get(b'delete')
refresh = opts.get(b'refresh') refresh = opts.get(b'refresh')
reset = opts.get(b'reset') reset = opts.get(b'reset')
count = sum( action = cmdutil.check_at_most_one_arg(
[ opts, b'import_rules', b'clear_rules', b'refresh'
include, )
exclude, updateconfig = bool(
enableprofile, include or exclude or delete or reset or enableprofile or disableprofile
disableprofile,
delete,
importrules,
refresh,
clearrules,
reset,
]
) )
count = sum([updateconfig, bool(action)])
if count > 1: if count > 1:
raise error.Abort(_(b"too many flags specified")) raise error.Abort(_(b"too many flags specified"))
if count == 0: if count == 0:
if repo.vfs.exists(b'sparse'): if repo.vfs.exists(b'sparse'):
ui.status(repo.vfs.read(b"sparse") + b"\n") ui.status(repo.vfs.read(b"sparse") + b"\n")
temporaryincludes = sparse.readtemporaryincludes(repo) temporaryincludes = sparse.readtemporaryincludes(repo)
if temporaryincludes: if temporaryincludes:
ui.status( ui.status(
_(b"Temporarily Included Files (for merge/rebase):\n") _(b"Temporarily Included Files (for merge/rebase):\n")
) )
ui.status((b"\n".join(temporaryincludes) + b"\n")) ui.status((b"\n".join(temporaryincludes) + b"\n"))
return return
else: else:
raise error.Abort( raise error.Abort(
_( _(
b'the debugsparse command is only supported on' b'the debugsparse command is only supported on'
b' sparse repositories' b' sparse repositories'
) )
) )
if include or exclude or delete or reset or enableprofile or disableprofile: if updateconfig:
sparse.updateconfig( sparse.updateconfig(
repo, repo,
pats,
opts, opts,
include=include, include=include,
exclude=exclude, exclude=exclude,
reset=reset, reset=reset,
delete=delete, delete=delete,
enableprofile=enableprofile, enableprofile=enableprofile,
disableprofile=disableprofile, disableprofile=disableprofile,
force=force, force=force,
) )
if importrules: if importrules:
sparse.importfromfiles(repo, opts, pats, force=force) sparse.importfromfiles(repo, opts, importrules, force=force)
if clearrules: if clearrules:
sparse.clearrules(repo, force=force) sparse.clearrules(repo, force=force)
if refresh: if refresh:
try: try:
wlock = repo.wlock() wlock = repo.wlock()
fcounts = map( fcounts = map(

mercurial/sparse.py

printchanges( printchanges(
repo.ui, opts, profilecount, includecount, excludecount, *fcounts repo.ui, opts, profilecount, includecount, excludecount, *fcounts
) )
def updateconfig( def updateconfig(
repo, repo,
pats,
opts, opts,
include=False, include=(),
exclude=False, exclude=(),
reset=False, reset=False,
delete=False, delete=(),
enableprofile=False, enableprofile=(),
disableprofile=False, disableprofile=(),
force=False, force=False,
usereporootpaths=False, usereporootpaths=False,
): ):
"""Perform a sparse config update. """Perform a sparse config update.
Only one of the actions may be performed.
The new config is written out and a working directory refresh is performed. The new config is written out and a working directory refresh is performed.
""" """
with repo.wlock(), repo.lock(), repo.dirstate.parentchange(): with repo.wlock(), repo.lock(), repo.dirstate.parentchange():
raw = repo.vfs.tryread(b'sparse') raw = repo.vfs.tryread(b'sparse')
oldinclude, oldexclude, oldprofiles = parseconfig( oldinclude, oldexclude, oldprofiles = parseconfig(
repo.ui, raw, b'sparse' repo.ui, raw, b'sparse'
) )
if reset: if reset:
newinclude = set() newinclude = set()
newexclude = set() newexclude = set()
newprofiles = set() newprofiles = set()
else: else:
newinclude = set(oldinclude) newinclude = set(oldinclude)
newexclude = set(oldexclude) newexclude = set(oldexclude)
newprofiles = set(oldprofiles) newprofiles = set(oldprofiles)
def normalize_pats(pats):
if any(os.path.isabs(pat) for pat in pats): if any(os.path.isabs(pat) for pat in pats):
raise error.Abort(_(b'paths cannot be absolute')) raise error.Abort(_(b'paths cannot be absolute'))
if not usereporootpaths: if usereporootpaths:
return pats
# let's treat paths as relative to cwd # let's treat paths as relative to cwd
root, cwd = repo.root, repo.getcwd() root, cwd = repo.root, repo.getcwd()
abspats = [] abspats = []
for kindpat in pats: for kindpat in pats:
kind, pat = matchmod._patsplit(kindpat, None) kind, pat = matchmod._patsplit(kindpat, None)
if kind in matchmod.cwdrelativepatternkinds or kind is None: if kind in matchmod.cwdrelativepatternkinds or kind is None:
ap = (kind + b':' if kind else b'') + pathutil.canonpath( ap = (kind + b':' if kind else b'') + pathutil.canonpath(
root, cwd, pat root, cwd, pat
) )
abspats.append(ap) abspats.append(ap)
else: else:
abspats.append(kindpat) abspats.append(kindpat)
pats = abspats return abspats
if include: include = normalize_pats(include)
newinclude.update(pats) exclude = normalize_pats(exclude)
elif exclude: delete = normalize_pats(delete)
newexclude.update(pats) disableprofile = normalize_pats(disableprofile)
elif enableprofile: enableprofile = normalize_pats(enableprofile)
newprofiles.update(pats)
elif disableprofile: newinclude.difference_update(delete)
newprofiles.difference_update(pats) newexclude.difference_update(delete)
elif delete: newprofiles.difference_update(disableprofile)
newinclude.difference_update(pats) newinclude.update(include)
newexclude.difference_update(pats) newprofiles.update(enableprofile)
newexclude.update(exclude)
profilecount = len(newprofiles - oldprofiles) - len( profilecount = len(newprofiles - oldprofiles) - len(
oldprofiles - newprofiles oldprofiles - newprofiles
) )
includecount = len(newinclude - oldinclude) - len( includecount = len(newinclude - oldinclude) - len(
oldinclude - newinclude oldinclude - newinclude
) )
excludecount = len(newexclude - oldexclude) - len( excludecount = len(newexclude - oldexclude) - len(

tests/test-sparse-clear.t

base.sparse base.sparse
data.py data.py
index.html index.html
readme.txt readme.txt
webpage.sparse webpage.sparse
Clear rules when there are excludes Clear rules when there are excludes
$ hg debugsparse --exclude *.sparse $ hg debugsparse -X base.sparse -X webpage.sparse
$ ls -A $ ls -A
.hg .hg
data.py data.py
index.html index.html
readme.txt readme.txt
$ hg debugsparse --clear-rules $ hg debugsparse --clear-rules
$ ls -A $ ls -A
.hg .hg

tests/test-sparse.t

$ hg debugsparse --delete 'show*' $ hg debugsparse --delete 'show*'
pending changes to 'hide' pending changes to 'hide'
abort: cannot change sparseness due to pending changes (delete the files or use --force to bring them back dirty) abort: cannot change sparseness due to pending changes (delete the files or use --force to bring them back dirty)
[255] [255]
Verify deleting sparseness with --force brings back files Verify deleting sparseness with --force brings back files
$ hg debugsparse --delete -f 'show*' $ hg debugsparse -f --delete 'show*'
pending changes to 'hide' pending changes to 'hide'
$ ls -A $ ls -A
.hg .hg
hide hide
hide2 hide2
hide3 hide3
show show
show2 show2
$ hg st $ hg st
M hide M hide
M show M show
? hide3 ? hide3
Verify editing sparseness fails if pending changes Verify editing sparseness fails if pending changes
$ hg debugsparse --include 'show*' $ hg debugsparse --include 'show*'
pending changes to 'hide' pending changes to 'hide'
abort: could not update sparseness due to pending changes abort: could not update sparseness due to pending changes
[255] [255]
Verify adding sparseness hides files Verify adding sparseness hides files
$ hg debugsparse --exclude -f 'hide*' $ hg debugsparse -f --exclude 'hide*'
pending changes to 'hide' pending changes to 'hide'
$ ls -A $ ls -A
.hg .hg
hide hide
hide3 hide3
show show
show2 show2
$ hg st $ hg st
$ hg up -C . $ hg up -C .
cleaned up 1 temporarily added file(s) from the sparse checkout cleaned up 1 temporarily added file(s) from the sparse checkout
1 files updated, 0 files merged, 0 files removed, 0 files unresolved 1 files updated, 0 files merged, 0 files removed, 0 files unresolved
$ hg debugsparse $ hg debugsparse
[exclude] [exclude]
hide* hide*
Multiple -I and -X can be passed at once
$ hg debugsparse --reset -I '*2' -X 'hide2'
$ ls -A
.hg
hide.orig
show2
$ hg debugsparse --reset -X 'hide*'
Verify strip -k resets dirstate correctly Verify strip -k resets dirstate correctly
$ hg status $ hg status
$ hg debugsparse $ hg debugsparse
[exclude] [exclude]
hide* hide*
$ hg log -r . -T '{rev}\n' --stat $ hg log -r . -T '{rev}\n' --stat