This is an archive of the discontinued Mercurial Phabricator instance.

Differential D677

scmutil: handle conflicting files and dirs in origbackuppath
AbandonedPublic

Authored by mbthomas on Sep 11 2017, 11:55 AM.

Details

Reviewers
ryanmce
durham
Group Reviewers
hg-reviewers
Summary

When ui.origbackuppath is set, .orig files are stored outside of the working
copy. However conflicts can occur when files or directories end up having the
same name. These conflicts cause Mercurial to abort, even if they've been
created as a result of different sets of backup.

This is made worse by the .orig suffix that is appended to file names as they
are backed up. A file named x can conflict with a directory named x.orig from
a different backup, or perhaps even the same backup.

This change removes the .orig suffix from files stored in an out-of-tree
origbackuppath, and deals with the case where files and directories replace
each other in the backup directory.

Test Plan

Re-run unit tests. Add new unit test for conflicting paths.

Diff Detail

Repository
rHG Mercurial
Branch
dirfixes (bookmark) on default (branch)
Lint
No Linters Available
Unit
No Unit Test Coverage

Event Timeline

mbthomas created this revision.Sep 11 2017, 11:55 AM
durham accepted this revision.Sep 11 2017, 12:26 PM
ryanmce requested changes to this revision.Sep 11 2017, 12:35 PM

I think this will be a bit easier to ingest as two patches: one to deal with the conflicting directories and one to remove the .orig from files with a custom destination.

This is a relatively new feature so I don't think we need a config option to maintain BC about these file names, and I'd argue the current behavior is a bug so I think it's good to change. That being said, a origsuffix config might be a way to please everyone if there's any pushback on this front.

This revision now requires changes to proceed.Sep 11 2017, 12:35 PM
mbthomas abandoned this revision.Sep 11 2017, 2:04 PM

As suggested by @ryanmce I have split this into D679 and D680.

Revision Contents
Changeset List

Diff 1714

mercurial/help/config.txt

managed files. At template expansion, non-ASCII characters use the encoding managed files. At template expansion, non-ASCII characters use the encoding
specified by the ``--encoding`` global option, ``HGENCODING`` or other specified by the ``--encoding`` global option, ``HGENCODING`` or other
environment variables that govern your locale. If the encoding of the merge environment variables that govern your locale. If the encoding of the merge
markers is different from the encoding of the merged files, markers is different from the encoding of the merged files,
serious problems may occur. serious problems may occur.
``origbackuppath`` ``origbackuppath``
The path to a directory used to store generated .orig files. If the path is The path to a directory used to store generated .orig files. If the path is
not a directory, one will be created. not a directory, one will be created. If set, files stored in this
directory have the same name as the original file and do not have a .orig
suffix.
``paginate`` ``paginate``
Control the pagination of command output (default: True). See :hg:`help pager` Control the pagination of command output (default: True). See :hg:`help pager`
for details. for details.
``patch`` ``patch``
An optional external tool that ``hg import`` and some extensions An optional external tool that ``hg import`` and some extensions
will use for applying patches. By default Mercurial uses an will use for applying patches. By default Mercurial uses an

mercurial/scmutil.py

def matchfiles(repo, files, badfn=None): def matchfiles(repo, files, badfn=None):
'''Return a matcher that will efficiently match exactly these files.''' '''Return a matcher that will efficiently match exactly these files.'''
return matchmod.exact(repo.root, repo.getcwd(), files, badfn=badfn) return matchmod.exact(repo.root, repo.getcwd(), files, badfn=badfn)
def origpath(ui, repo, filepath): def origpath(ui, repo, filepath):
'''customize where .orig files are created '''customize where .orig files are created
Fetch user defined path from config file: [ui] origbackuppath = <path> Fetch user defined path from config file: [ui] origbackuppath = <path>
Fall back to default (filepath) if not specified Fall back to default (filepath with .orig suffix) if not specified
''' '''
origbackuppath = ui.config('ui', 'origbackuppath') origbackuppath = ui.config('ui', 'origbackuppath')
if origbackuppath is None: if origbackuppath is None:
return filepath + ".orig" return filepath + ".orig"
filepathfromroot = os.path.relpath(filepath, start=repo.root) filepathfromroot = os.path.relpath(filepath, start=repo.root)
fullorigpath = repo.wjoin(origbackuppath, filepathfromroot) fullorigpath = repo.wjoin(origbackuppath, filepathfromroot)
origbackupdir = repo.vfs.dirname(fullorigpath) origbackupdir = repo.vfs.dirname(fullorigpath)
if not repo.vfs.exists(origbackupdir): if not repo.vfs.isdir(origbackupdir):
ui.note(_('creating directory: %s\n') % origbackupdir) ui.note(_('creating directory: %s\n') % origbackupdir)
# Remove any files that conflict with the backup file's path
for f in util.finddirs(fullorigpath):
if repo.vfs.exists(f):
if repo.vfs.isfile(f):
ui.note(_('removing conflicting file: %s\n') % f)
repo.vfs.unlink(f)
break
util.makedirs(origbackupdir) util.makedirs(origbackupdir)
return fullorigpath + ".orig" if repo.vfs.isdir(fullorigpath):
ui.note(_('removing conflicting directory: %s\n') % fullorigpath)
repo.vfs.rmtree(fullorigpath, forcibly=True)
return fullorigpath
class _containsnode(object): class _containsnode(object):
"""proxy __contains__(node) to container.__contains__ which accepts revs""" """proxy __contains__(node) to container.__contains__ which accepts revs"""
def __init__(self, repo, revcontainer): def __init__(self, repo, revcontainer):
self._torev = repo.changelog.rev self._torev = repo.changelog.rev
self._revcontains = revcontainer.__contains__ self._revcontains = revcontainer.__contains__

tests/test-largefiles-misc.t

$ hg revert anotherlarge $ hg revert anotherlarge
$ hg st $ hg st
? sub/anotherlarge.orig ? sub/anotherlarge.orig
Test orig files go where we want them Test orig files go where we want them
$ echo moremore >> anotherlarge $ echo moremore >> anotherlarge
$ hg revert anotherlarge -v --config 'ui.origbackuppath=.hg/origbackups' $ hg revert anotherlarge -v --config 'ui.origbackuppath=.hg/origbackups'
creating directory: $TESTTMP/addrm2/.hg/origbackups/.hglf/sub (glob) creating directory: $TESTTMP/addrm2/.hg/origbackups/.hglf/sub (glob)
saving current version of ../.hglf/sub/anotherlarge as $TESTTMP/addrm2/.hg/origbackups/.hglf/sub/anotherlarge.orig (glob) saving current version of ../.hglf/sub/anotherlarge as $TESTTMP/addrm2/.hg/origbackups/.hglf/sub/anotherlarge (glob)
reverting ../.hglf/sub/anotherlarge (glob) reverting ../.hglf/sub/anotherlarge (glob)
creating directory: $TESTTMP/addrm2/.hg/origbackups/sub (glob) creating directory: $TESTTMP/addrm2/.hg/origbackups/sub (glob)
found 90c622cf65cebe75c5842f9136c459333faf392e in store found 90c622cf65cebe75c5842f9136c459333faf392e in store
found 90c622cf65cebe75c5842f9136c459333faf392e in store found 90c622cf65cebe75c5842f9136c459333faf392e in store
$ ls ../.hg/origbackups/sub $ ls ../.hg/origbackups/sub
anotherlarge.orig anotherlarge
$ cd .. $ cd ..
Test glob logging from the root dir Test glob logging from the root dir
$ hg log glob:**another* $ hg log glob:**another*
changeset: 1:9627a577c5e9 changeset: 1:9627a577c5e9
tag: tip tag: tip
user: test user: test
date: Thu Jan 01 00:00:00 1970 +0000 date: Thu Jan 01 00:00:00 1970 +0000

tests/test-merge-local.t

merging zzz2_merge_bad merging zzz2_merge_bad
warning: conflicts while merging zzz2_merge_bad! (edit, then use 'hg resolve --mark') warning: conflicts while merging zzz2_merge_bad! (edit, then use 'hg resolve --mark')
2 files updated, 1 files merged, 3 files removed, 1 files unresolved 2 files updated, 1 files merged, 3 files removed, 1 files unresolved
use 'hg resolve' to retry unresolved file merges use 'hg resolve' to retry unresolved file merges
[1] [1]
Are orig files from the last commit where we want them? Are orig files from the last commit where we want them?
$ ls .hg/origbackups $ ls .hg/origbackups
zzz2_merge_bad.orig zzz2_merge_bad
$ hg diff --nodates | grep "^[+-][^<>]" $ hg diff --nodates | grep "^[+-][^<>]"
--- a/zzz1_merge_ok --- a/zzz1_merge_ok
+++ b/zzz1_merge_ok +++ b/zzz1_merge_ok
+new last line +new last line
--- a/zzz2_merge_bad --- a/zzz2_merge_bad
+++ b/zzz2_merge_bad +++ b/zzz2_merge_bad
+another last line +another last line

tests/test-mq-qpush-fail.t

$ hg --config mq.keepchanges=1 qpush --force $ hg --config mq.keepchanges=1 qpush --force
applying p2 applying p2
now at: p2 now at: p2
$ hg st a $ hg st a
test previous qpop (with --force and --config) saved .orig files to where user test previous qpop (with --force and --config) saved .orig files to where user
wants them wants them
$ ls .hg/origbackups $ ls .hg/origbackups
b.orig b
$ rm -rf .hg/origbackups $ rm -rf .hg/origbackups
$ cd .. $ cd ..

tests/test-mq.t

[255] [255]
apply force, should discard changes in hello, but not bye apply force, should discard changes in hello, but not bye
$ hg qpush -f --verbose --config 'ui.origbackuppath=.hg/origbackups' $ hg qpush -f --verbose --config 'ui.origbackuppath=.hg/origbackups'
applying empty applying empty
creating directory: $TESTTMP/forcepush/.hg/origbackups (glob) creating directory: $TESTTMP/forcepush/.hg/origbackups (glob)
saving current version of hello.txt as $TESTTMP/forcepush/.hg/origbackups/hello.txt.orig (glob) saving current version of hello.txt as $TESTTMP/forcepush/.hg/origbackups/hello.txt (glob)
patching file hello.txt patching file hello.txt
committing files: committing files:
hello.txt hello.txt
committing manifest committing manifest
committing changelog committing changelog
now at: empty now at: empty
$ hg st $ hg st
M bye.txt M bye.txt
@@ -1,1 +1,3 @@ @@ -1,1 +1,3 @@
hello hello
+world +world
+universe +universe
test that the previous call to qpush with -f (--force) and --config actually put test that the previous call to qpush with -f (--force) and --config actually put
the orig files out of the working copy the orig files out of the working copy
$ ls .hg/origbackups $ ls .hg/origbackups
hello.txt.orig hello.txt
test popping revisions not in working dir ancestry test popping revisions not in working dir ancestry
$ hg qseries -v $ hg qseries -v
0 A empty 0 A empty
$ hg up qparent $ hg up qparent
1 files updated, 0 files merged, 0 files removed, 0 files unresolved 1 files updated, 0 files merged, 0 files removed, 0 files unresolved
$ hg qpop $ hg qpop

tests/test-origbackup-conflict.t

  • This file was added.
Set up repo
$ cat << EOF >> $HGRCPATH
> [ui]
> origbackuppath=.hg/origbackups
> [merge]
> checkunknown=warn
> EOF
$ hg init repo
$ cd repo
$ echo base > base
$ hg add base
$ hg commit -m "base"
Make a dir named b that contains a file
$ mkdir -p b
$ echo c1 > b/c
$ hg add b/c
$ hg commit -m "c1"
$ hg bookmark c1
Peform an update that causes b/c to be backed up
$ hg up 0
0 files updated, 0 files merged, 1 files removed, 0 files unresolved
(leaving bookmark c1)
$ mkdir -p b
$ echo c2 > b/c
$ hg up --verbose c1
resolving manifests
b/c: replacing untracked file
getting b/c
creating directory: $TESTTMP/repo/.hg/origbackups/b
1 files updated, 0 files merged, 0 files removed, 0 files unresolved
(activating bookmark c1)
$ test -f .hg/origbackups/b/c
Make a file named b
$ hg up 0
0 files updated, 0 files merged, 1 files removed, 0 files unresolved
(leaving bookmark c1)
$ echo b1 > b
$ hg add b
$ hg commit -m b1
created new head
$ hg bookmark b1
Perform an update that causes b to be backed up - it should replace the backup b dir
$ hg up 0
0 files updated, 0 files merged, 1 files removed, 0 files unresolved
(leaving bookmark b1)
$ echo b2 > b
$ hg up --verbose b1
resolving manifests
b: replacing untracked file
getting b
removing conflicting directory: $TESTTMP/repo/.hg/origbackups/b
1 files updated, 0 files merged, 0 files removed, 0 files unresolved
(activating bookmark b1)
$ test -f .hg/origbackups/b
Perform an update the causes b/c to be backed up again - it should replace the backup b file
$ hg up 0
0 files updated, 0 files merged, 1 files removed, 0 files unresolved
(leaving bookmark b1)
$ mkdir b
$ echo c3 > b/c
$ hg up --verbose c1
resolving manifests
b/c: replacing untracked file
getting b/c
creating directory: $TESTTMP/repo/.hg/origbackups/b
removing conflicting file: $TESTTMP/repo/.hg/origbackups/b
1 files updated, 0 files merged, 0 files removed, 0 files unresolved
(activating bookmark c1)
$ test -d .hg/origbackups/b

tests/test-resolve.t

$ hg resolve --all --verbose --config 'ui.origbackuppath=.hg/origbackups' $ hg resolve --all --verbose --config 'ui.origbackuppath=.hg/origbackups'
merging file1 merging file1
creating directory: $TESTTMP/repo/.hg/origbackups (glob) creating directory: $TESTTMP/repo/.hg/origbackups (glob)
merging file2 merging file2
warning: conflicts while merging file1! (edit, then use 'hg resolve --mark') warning: conflicts while merging file1! (edit, then use 'hg resolve --mark')
warning: conflicts while merging file2! (edit, then use 'hg resolve --mark') warning: conflicts while merging file2! (edit, then use 'hg resolve --mark')
[1] [1]
$ ls .hg/origbackups $ ls .hg/origbackups
file1.orig file1
file2.orig file2
$ grep '<<<' file1 > /dev/null $ grep '<<<' file1 > /dev/null
$ grep '<<<' file2 > /dev/null $ grep '<<<' file2 > /dev/null
resolve <file> should re-merge file resolve <file> should re-merge file
$ echo resolved > file1 $ echo resolved > file1
$ hg resolve -q file1 $ hg resolve -q file1
warning: conflicts while merging file1! (edit, then use 'hg resolve --mark') warning: conflicts while merging file1! (edit, then use 'hg resolve --mark')
[1] [1]

tests/test-revert.t

reverting e reverting e
Test creation of backup (.orig) file in configured file location Test creation of backup (.orig) file in configured file location
---------------------------------------------------------------- ----------------------------------------------------------------
$ echo z > e $ echo z > e
$ hg revert --all -v --config 'ui.origbackuppath=.hg/origbackups' $ hg revert --all -v --config 'ui.origbackuppath=.hg/origbackups'
creating directory: $TESTTMP/repo/.hg/origbackups (glob) creating directory: $TESTTMP/repo/.hg/origbackups (glob)
saving current version of e as $TESTTMP/repo/.hg/origbackups/e.orig (glob) saving current version of e as $TESTTMP/repo/.hg/origbackups/e
reverting e reverting e
$ rm -rf .hg/origbackups $ rm -rf .hg/origbackups
revert on clean file (no change) revert on clean file (no change)
-------------------------------- --------------------------------
$ hg revert a $ hg revert a
no changes needed to a no changes needed to a

tests/test-shelve.t

$ echo '' > root $ echo '' > root
$ hg shelve -q $ hg shelve -q
$ echo 'contADDent' > root $ echo 'contADDent' > root
$ hg unshelve -q --config 'ui.origbackuppath=.hg/origbackups' $ hg unshelve -q --config 'ui.origbackuppath=.hg/origbackups'
warning: conflicts while merging root! (edit, then use 'hg resolve --mark') warning: conflicts while merging root! (edit, then use 'hg resolve --mark')
unresolved conflicts (see 'hg resolve', then 'hg unshelve --continue') unresolved conflicts (see 'hg resolve', then 'hg unshelve --continue')
[1] [1]
$ ls .hg/origbackups $ ls .hg/origbackups
root.orig root
$ rm -rf .hg/origbackups $ rm -rf .hg/origbackups
test Abort unshelve always gets user out of the unshelved state test Abort unshelve always gets user out of the unshelved state
--------------------------------------------------------------- ---------------------------------------------------------------
Wreak havoc on the unshelve process Wreak havoc on the unshelve process
$ rm .hg/unshelverebasestate $ rm .hg/unshelverebasestate
$ hg unshelve --abort $ hg unshelve --abort
unshelve of 'default' aborted unshelve of 'default' aborted

tests/test-subrepo-git.t

$ hg status --subrepos $ hg status --subrepos
? s/barfoo ? s/barfoo
revert moves orig files to the right place revert moves orig files to the right place
$ echo 'bloop' > s/foobar $ echo 'bloop' > s/foobar
$ hg revert --all --verbose --config 'ui.origbackuppath=.hg/origbackups' $ hg revert --all --verbose --config 'ui.origbackuppath=.hg/origbackups'
reverting subrepo ../gitroot reverting subrepo ../gitroot
creating directory: $TESTTMP/tc/.hg/origbackups (glob) creating directory: $TESTTMP/tc/.hg/origbackups (glob)
saving current version of foobar as $TESTTMP/tc/.hg/origbackups/foobar.orig (glob) saving current version of foobar as $TESTTMP/tc/.hg/origbackups/foobar (glob)
$ ls .hg/origbackups $ ls .hg/origbackups
foobar.orig foobar
$ rm -rf .hg/origbackups $ rm -rf .hg/origbackups
show file at specific revision show file at specific revision
$ cat > s/foobar << EOF $ cat > s/foobar << EOF
> woop woop > woop woop
> fooo bar > fooo bar
> EOF > EOF
$ hg commit --subrepos -m "updated foobar" $ hg commit --subrepos -m "updated foobar"