This is an archive of the discontinued Mercurial Phabricator instance.

narrow: widen when files are excluded by sparse and not included by narrow
ClosedPublic

Authored by charlesetc on May 18 2021, 4:15 PM.

Download Raw Diff

Details

Reviewers

durin42
martinvonz

Group Reviewers

hg-reviewers

Commits

rHGe4ccc341e65b: narrow: widen when files are excluded by sparse and not included by narrow
rHGf2e346c306e7: narrow: widen when files are excluded by sparse and not included by narrow

Summary

In a repo where some directories are included by narrow and the complement are
excluded by sparse, it was previously impossible to widen either because trying
to widen narrow would complain that the requested files are outside the sparse
checkout and trying to widen sparse would complain that the requested files are
outside the narrow checkout.

This changes the hg tracked --addinclude command to only actually update any
newly accessible files in the dirstate if they are also accessible via sparse.

Diff Detail

Repository

rHG Mercurial

Lint

Automatic diff as part of commit; lint not applicable.

Unit

Automatic diff as part of commit; unit tests not applicable.

Event Timeline

charlesetc created this revision.May 18 2021, 4:15 PM

Herald added a reviewer: durin42. · View Herald TranscriptMay 18 2021, 4:15 PM

Herald added a reviewer: hg-reviewers. · View Herald Transcript

Herald added a subscriber: mercurial-patches. · View Herald Transcript

charlesetc retitled this revision from Enable widen when files are excluded by sparse and not included by narrow to narrow: widen when files are excluded by sparse and not included by narrow.May 18 2021, 4:18 PM

Without the change in narrowspec.py, the new test case fails at the first hg tracked with:

   $ hg tracked -q --addinclude outside/f
+  abort: cannot add 'outside/f' - it is outside the sparse checkout
+  (include file with `hg debugsparse --include <pattern>` or use `hg add -s <file>` to include file directory while adding)
+  [255]

martinvonz added a subscriber: martinvonz.May 19 2021, 1:45 PM

martinvonz added inline comments.

tests/test-narrow-sparse.t
74–76	I was confused by this so I experimented a bit with it myself. It seems that it's `hg debugsparse` that fails. Could you update the text to clarify that? Alternatively, we could include coverage for the treemanifest case by manually updating `.hg/sparse`. I checked that that works, which was a bit of a relief (it seems less concerning if it's just the command for updating the sparse profile that fails).

charlesetc retitled this revision from narrow: widen when files are excluded by sparse and not included by narrow to Enable widen when files are excluded by sparse and not included by narrow.May 19 2021, 2:26 PM

charlesetc updated this revision to Diff 28129.

charlesetc added inline comments.May 19 2021, 2:30 PM

tests/test-narrow-sparse.t
74–76	I tried to word it more clearly, thanks! I'm not sure what you mean by 'manually updating .hg/sparse'.

martinvonz added inline comments.May 19 2021, 2:33 PM

tests/test-narrow-sparse.t
74–76	I mean this: $ cat >> .hg/sparse <<EOF > [exclude] > outside/f > widest/f > EOF

charlesetc added inline comments.May 19 2021, 4:11 PM

tests/test-narrow-sparse.t
74–76	Ah I see. We would also have to remove the test of `hg debugsparse -d outside/f` which isn't working with tree manifest but is necessary to fully show we can widen.

charlesetc marked an inline comment as not done.May 19 2021, 4:11 PM

martinvonz accepted this revision.May 20 2021, 1:25 PM

martinvonz added inline comments.

tests/test-narrow-sparse.t
74–76	I see. I've never used sparse so I didn't understand what the point of the command was (I thought it was just to update `.hg/sparse`). Sounds good then. I'll queue this patch.

This revision is now accepted and ready to land.May 20 2021, 1:25 PM

charlesetc added a commit: rHGf2e346c306e7: narrow: widen when files are excluded by sparse and not included by narrow.May 20 2021, 1:41 PM

Closed by commit rHGf2e346c306e7: narrow: widen when files are excluded by sparse and not included by narrow (authored by charlesetc). · Explain Why

This revision was automatically updated to reflect the committed changes.

charlesetc added inline comments.May 20 2021, 2:00 PM

tests/test-narrow-sparse.t
74–76	Ah yeah it also checks out the newly accessible files if any. Not sure if there's another way to do that. Thanks!

This change assumes the presence of the tree executable, which makes our CI fail. Please use find instead. I'll amend the changeset because it's a small amount of work.

charlesetc added a commit: rHGe4ccc341e65b: narrow: widen when files are excluded by sparse and not included by narrow.May 21 2021, 3:42 AM

Revision Contents
Changeset List

			Path	Packages
M			mercurial/narrowspec.py (3 lines)
M			tests/test-narrow-sparse.t (34 lines)

Diff	ID	Description	Created	Lint	Unit
Base		Base
Diff 1	28082		May 18 2021, 4:15 PM	★	★
Diff 2	28129		May 19 2021, 2:26 PM	★	★
Diff 3	28161	rHGf2e346c306e762d06dc7a89ea66fdb60ff4dc573	May 18 2021, 1:08 PM	★	★

Diff 28161

mercurial/narrowspec.py

	for f in sorted(status.unknown):			for f in sorted(status.unknown):
	repo.ui.status(_(b'not deleting unknown file %s\n') % uipathfn(f))			repo.ui.status(_(b'not deleting unknown file %s\n') % uipathfn(f))
	for f in sorted(status.ignored):			for f in sorted(status.ignored):
	repo.ui.status(_(b'not deleting ignored file %s\n') % uipathfn(f))			repo.ui.status(_(b'not deleting ignored file %s\n') % uipathfn(f))
	for f in clean + trackeddirty:			for f in clean + trackeddirty:
	ds.drop(f)			ds.drop(f)

	pctx = repo[b'.']			pctx = repo[b'.']

				# only update added files that are in the sparse checkout
				addedmatch = matchmod.intersectmatchers(addedmatch, sparse.matcher(repo))
	newfiles = [f for f in pctx.manifest().walk(addedmatch) if f not in ds]			newfiles = [f for f in pctx.manifest().walk(addedmatch) if f not in ds]
	for f in newfiles:			for f in newfiles:
	ds.normallookup(f)			ds.normallookup(f)
	_writeaddedfiles(repo, pctx, newfiles)			_writeaddedfiles(repo, pctx, newfiles)
	repo._updatingnarrowspec = False			repo._updatingnarrowspec = False

tests/test-narrow-sparse.t

	persistent-nodemap (rust !)			persistent-nodemap (rust !)
	revlog-compression-zstd (zstd !)			revlog-compression-zstd (zstd !)
	revlogv1			revlogv1
	sparserevlog			sparserevlog
	store			store
	treemanifest (tree !)			treemanifest (tree !)

	$ hg debugrebuilddirstate			$ hg debugrebuilddirstate

				We only make the following assertions for the flat test case since in the
				treemanifest test case debugsparse fails with "path ends in directory
				separator: outside/" which seems like a bug unrelated to the regression this is
				martinvonzUnsubmitted Not Done I was confused by this so I experimented a bit with it myself. It seems that it's `hg debugsparse` that fails. Could you update the text to clarify that? Alternatively, we could include coverage for the treemanifest case by manually updating `.hg/sparse`. I checked that that works, which was a bit of a relief (it seems less concerning if it's just the command for updating the sparse profile that fails). martinvonz: I was confused by this so I experimented a bit with it myself. It seems that it's `hg…
				charlesetcAuthorUnsubmitted Not Done I tried to word it more clearly, thanks! I'm not sure what you mean by 'manually updating .hg/sparse'. charlesetc: I tried to word it more clearly, thanks! I'm not sure what you mean by 'manually updating .
				martinvonzUnsubmitted Not Done I mean this: $ cat >> .hg/sparse <<EOF > [exclude] > outside/f > widest/f > EOF martinvonz: I mean this: ``` $ cat >> .hg/sparse <<EOF > [exclude] > outside/f > widest/f >…
				charlesetcAuthorUnsubmitted Not Done Ah I see. We would also have to remove the test of `hg debugsparse -d outside/f` which isn't working with tree manifest but is necessary to fully show we can widen. charlesetc: Ah I see. We would also have to remove the test of `hg debugsparse -d outside/f` which isn't…
				martinvonzUnsubmitted Done I see. I've never used sparse so I didn't understand what the point of the command was (I thought it was just to update `.hg/sparse`). Sounds good then. I'll queue this patch. martinvonz: I see. I've never used sparse so I didn't understand what the point of the command was (I…
				charlesetcAuthorUnsubmitted Done Ah yeah it also checks out the newly accessible files if any. Not sure if there's another way to do that. Thanks! charlesetc: Ah yeah it also checks out the newly accessible files if any. Not sure if there's another way…
				testing for.

				#if flat
				widening with both sparse and narrow is possible

				$ cat >> .hg/hgrc <<EOF
				> [extensions]
				> sparse =
				> narrow =
				> EOF

				$ hg debugsparse -X outside/f -X widest/f
				$ hg tracked -q --addinclude outside/f
				$ tree
				.
				`-- inside
				`-- f

				1 directory, 1 file

				$ hg debugsparse -d outside/f
				$ tree
				.
				\|-- inside
				\| `-- f
				`-- outside
				`-- f

				2 directories, 2 files
				#endif