This is an archive of the discontinued Mercurial Phabricator instance.

Differential D3117

tests: make test-convert-git.t more deterministic
AbandonedPublic

Authored by indygreg on Apr 5 2018, 12:38 AM.

Details

Reviewers
durin42
Group Reviewers
hg-reviewers
Summary

Previously, we had some unwrapped usage of git commit, which would
result in the current time being added to the commit object.

Our commit() shell function makes git commit deterministic. So
switch to that everywhere.

Because every call to commit() increments the counter for the current
time, inserting calls earlier in the test changes the hash of later
commits. So there was some test fallout associated with this. But no
meaningful behavior should have changed.

I'm not sure why the display order and the conversion order of some
things changed. If I had to venture a guess, it would be a) there
was a race condition in the previous code where commits with the same
second time were sorted differently from commits with different times
b) something somewhere was sorting by hash. I dunno.

Diff Detail

Repository
rHG Mercurial
Lint
Lint Skipped
Unit
Unit Tests Skipped

Event Timeline

indygreg created this revision.Apr 5 2018, 12:38 AM
durin42 accepted this revision.Apr 6 2018, 8:44 PM
This revision is now accepted and ready to land.Apr 6 2018, 8:44 PM
durin42 added a comment.Apr 6 2018, 9:24 PM

This seems to break things with git 1.8.3.1, which is what's on gcc112. Should we just ratchet our test requirements forward, or try and figure out what changed?

indygreg added a comment.Apr 6 2018, 9:42 PM
In D3117#50998, @durin42 wrote:

This seems to break things with git 1.8.3.1, which is what's on gcc112. Should we just ratchet our test requirements forward, or try and figure out what changed?

You can drop it for now. I don’t think it is important to the series. I was just poking around as part of investigating a test failure.

indygreg abandoned this revision.Apr 12 2018, 11:58 AM

This isn't worth my time to close the loop on.

Revision Contents
Changeset List

PathPackages
Mtests/test-convert-git.t (84 lines)

Diff 7699

tests/test-convert-git.t

[255] [255]
test converting certain branches test converting certain branches
$ mkdir git-testrevs $ mkdir git-testrevs
$ cd git-testrevs $ cd git-testrevs
$ git init $ git init
Initialized empty Git repository in $TESTTMP/git-testrevs/.git/ Initialized empty Git repository in $TESTTMP/git-testrevs/.git/
$ echo a >> a ; git add a > /dev/null; git commit -m 'first' > /dev/null $ echo a >> a ; git add a > /dev/null; commit -m 'first'
$ echo a >> a ; git add a > /dev/null; git commit -m 'master commit' > /dev/null $ echo a >> a ; git add a > /dev/null; commit -m 'master commit'
$ git checkout -b goodbranch 'HEAD^' $ git checkout -b goodbranch 'HEAD^'
Switched to a new branch 'goodbranch' Switched to a new branch 'goodbranch'
$ echo a >> b ; git add b > /dev/null; git commit -m 'good branch commit' > /dev/null $ echo a >> b ; git add b > /dev/null; commit -m 'good branch commit'
$ git checkout -b badbranch 'HEAD^' $ git checkout -b badbranch 'HEAD^'
Switched to a new branch 'badbranch' Switched to a new branch 'badbranch'
$ echo a >> c ; git add c > /dev/null; git commit -m 'bad branch commit' > /dev/null $ echo a >> c ; git add c > /dev/null; commit -m 'bad branch commit'
$ cd .. $ cd ..
$ hg convert git-testrevs hg-testrevs --rev master --rev goodbranch $ hg convert git-testrevs hg-testrevs --rev master --rev goodbranch
initializing destination hg-testrevs repository initializing destination hg-testrevs repository
scanning source... scanning source...
sorting... sorting...
converting... converting...
2 first 2 first
1 good branch commit 1 master commit
0 master commit 0 good branch commit
updating bookmarks updating bookmarks
$ cd hg-testrevs $ cd hg-testrevs
$ hg log -G -T '{rev} {bookmarks}' $ hg log -G -T '{rev} {bookmarks}'
o 2 master o 2 goodbranch
| |
| o 1 goodbranch | o 1 master
|/ |/
o 0 o 0
$ cd .. $ cd ..
test sub modules test sub modules
$ mkdir git-repo5 $ mkdir git-repo5
$ cd git-repo5 $ cd git-repo5
$ git init-db >/dev/null 2>/dev/null $ git init-db >/dev/null 2>/dev/null
$ echo 'sub' >> foo $ echo 'sub' >> foo
$ git add foo $ git add foo
$ commit -a -m 'addfoo' $ commit -a -m 'addfoo'
This is non-deterministic because `pwd` is likely inside a temp directory
and this value gets written into the .gitmodules file, thus changing hashes
of everything derived from it.
$ BASE=`pwd` $ BASE=`pwd`
$ cd .. $ cd ..
$ mkdir git-repo6 $ mkdir git-repo6
$ cd git-repo6 $ cd git-repo6
$ git init-db >/dev/null 2>/dev/null $ git init-db >/dev/null 2>/dev/null
$ git submodule add ${BASE} >/dev/null 2>/dev/null $ git submodule add ${BASE} >/dev/null 2>/dev/null
$ commit -a -m 'addsubmodule' >/dev/null 2>/dev/null $ commit -a -m 'addsubmodule' >/dev/null 2>/dev/null
test non-tab whitespace .gitmodules test non-tab whitespace .gitmodules
$ cat >> .gitmodules <<EOF $ cat >> .gitmodules <<EOF
> [submodule "git-repo5"] > [submodule "git-repo5"]
> path = git-repo5 > path = git-repo5
> url = git-repo5 > url = git-repo5
> EOF > EOF
$ git commit -q -a -m "weird white space submodule" $ commit -q -a -m "weird white space submodule"
$ cd .. $ cd ..
$ hg convert git-repo6 hg-repo6 $ hg convert git-repo6 hg-repo6
initializing destination hg-repo6 repository initializing destination hg-repo6 repository
scanning source... scanning source...
sorting... sorting...
converting... converting...
1 addsubmodule 1 addsubmodule
0 weird white space submodule 0 weird white space submodule
updating bookmarks updating bookmarks
$ rm -rf hg-repo6 $ rm -rf hg-repo6
$ cd git-repo6 $ cd git-repo6
$ git reset --hard 'HEAD^' > /dev/null $ git reset --hard 'HEAD^' > /dev/null
test missing .gitmodules test missing .gitmodules
$ git submodule add ../git-repo4 >/dev/null 2>/dev/null $ git submodule add ../git-repo4 >/dev/null 2>/dev/null
$ git checkout HEAD .gitmodules $ git checkout HEAD .gitmodules
$ git rm .gitmodules $ git rm .gitmodules
rm '.gitmodules' rm '.gitmodules'
$ git commit -q -m "remove .gitmodules" .gitmodules $ commit -q -m "remove .gitmodules" .gitmodules
$ git commit -q -m "missing .gitmodules" $ commit -q -m "missing .gitmodules"
$ cd .. $ cd ..
$ hg convert git-repo6 hg-repo6 --traceback 2>&1 | grep -v "fatal: Path '.gitmodules' does not exist" $ hg convert git-repo6 hg-repo6 --traceback 2>&1 | grep -v "fatal: Path '.gitmodules' does not exist"
initializing destination hg-repo6 repository initializing destination hg-repo6 repository
scanning source... scanning source...
sorting... sorting...
converting... converting...
2 addsubmodule 2 addsubmodule
1 remove .gitmodules 1 remove .gitmodules
converting... converting...
0 addsubmodule 0 addsubmodule
updating bookmarks updating bookmarks
$ hg -R git-repo6-hg log -v $ hg -R git-repo6-hg log -v
changeset: 0:* (glob) changeset: 0:* (glob)
bookmark: master bookmark: master
tag: tip tag: tip
user: nottest <test@example.org> user: nottest <test@example.org>
date: Mon Jan 01 00:00:23 2007 +0000 date: Mon Jan 01 00:00:27 2007 +0000
files: .hgsub .hgsubstate files: .hgsub .hgsubstate
description: description:
addsubmodule addsubmodule
committer: test <test@example.org> committer: test <test@example.org>
$ hg convert -q git-repo6 no-submodules --config convert.git.skipsubmodules=True $ hg convert -q git-repo6 no-submodules --config convert.git.skipsubmodules=True
$ hg -R no-submodules manifest --all $ hg -R no-submodules manifest --all
.gitmodules-renamed .gitmodules-renamed
convert using a different remote prefix convert using a different remote prefix
$ git init git-repo7 $ git init git-repo7
Initialized empty Git repository in $TESTTMP/git-repo7/.git/ Initialized empty Git repository in $TESTTMP/git-repo7/.git/
$ cd git-repo7 $ cd git-repo7
TODO: it'd be nice to use (?) lines instead of grep -v to handle the $ touch a && git add a && commit -am "commit a"
git output variance, but that doesn't currently work in the middle of $ git rev-parse HEAD
a block, so do this for now. b77d4d3dca7949aa425aba5d218042a385065923
$ touch a && git add a && git commit -am "commit a" | grep -v changed
[master (root-commit) 8ae5f69] commit a
Author: nottest <test@example.org>
create mode 100644 a
$ cd .. $ cd ..
$ git clone git-repo7 git-repo7-client $ git clone git-repo7 git-repo7-client
Cloning into 'git-repo7-client'... Cloning into 'git-repo7-client'...
done. done.
$ hg convert --config convert.git.remoteprefix=origin git-repo7-client hg-repo7 $ hg convert --config convert.git.remoteprefix=origin git-repo7-client hg-repo7
initializing destination hg-repo7 repository initializing destination hg-repo7 repository
scanning source... scanning source...
sorting... sorting...
converting... converting...
0 commit a 0 commit a
updating bookmarks updating bookmarks
$ hg -R hg-repo7 bookmarks $ hg -R hg-repo7 bookmarks
master 0:03bf38caa4c6 master 0:28a7d3f23427
origin/master 0:03bf38caa4c6 origin/master 0:28a7d3f23427
Run convert when the remote branches have changed Run convert when the remote branches have changed
(there was an old bug where the local convert read branches from the server) (there was an old bug where the local convert read branches from the server)
$ cd git-repo7 $ cd git-repo7
$ echo a >> a $ echo a >> a
$ git commit -q -am "move master forward" $ commit -q -am "move master forward"
$ cd .. $ cd ..
$ rm -rf hg-repo7 $ rm -rf hg-repo7
$ hg convert --config convert.git.remoteprefix=origin git-repo7-client hg-repo7 $ hg convert --config convert.git.remoteprefix=origin git-repo7-client hg-repo7
initializing destination hg-repo7 repository initializing destination hg-repo7 repository
scanning source... scanning source...
sorting... sorting...
converting... converting...
0 commit a 0 commit a
updating bookmarks updating bookmarks
$ hg -R hg-repo7 bookmarks $ hg -R hg-repo7 bookmarks
master 0:03bf38caa4c6 master 0:28a7d3f23427
origin/master 0:03bf38caa4c6 origin/master 0:28a7d3f23427
damaged git repository tests: damaged git repository tests:
In case the hard-coded hashes change, the following commands can be used to In case the hard-coded hashes change, the following commands can be used to
list the hashes and their corresponding types in the repository: list the hashes and their corresponding types in the repository:
cd git-repo4/.git/objects cd git-repo4/.git/objects
find . -type f | cut -c 3- | sed 's_/__' | xargs -n 1 -t git cat-file -t find . -type f | cut -c 3- | sed 's_/__' | xargs -n 1 -t git cat-file -t
cd ../../.. cd ../../..
#if no-windows git19 #if no-windows git19
test for escaping the repo name (CVE-2016-3069) test for escaping the repo name (CVE-2016-3069)
$ git init '`echo pwned >COMMAND-INJECTION`' $ git init '`echo pwned >COMMAND-INJECTION`'
Initialized empty Git repository in $TESTTMP/`echo pwned >COMMAND-INJECTION`/.git/ Initialized empty Git repository in $TESTTMP/`echo pwned >COMMAND-INJECTION`/.git/
$ cd '`echo pwned >COMMAND-INJECTION`' $ cd '`echo pwned >COMMAND-INJECTION`'
$ git commit -q --allow-empty -m 'empty' $ commit -q --allow-empty -m 'empty'
$ cd .. $ cd ..
$ hg convert '`echo pwned >COMMAND-INJECTION`' 'converted' $ hg convert '`echo pwned >COMMAND-INJECTION`' 'converted'
initializing destination converted repository initializing destination converted repository
scanning source... scanning source...
sorting... sorting...
converting... converting...
0 empty 0 empty
updating bookmarks updating bookmarks
$ test -f COMMAND-INJECTION $ test -f COMMAND-INJECTION
[1] [1]
test for safely passing paths to git (CVE-2016-3105) test for safely passing paths to git (CVE-2016-3105)
$ git init 'ext::sh -c echo% pwned% >GIT-EXT-COMMAND-INJECTION% #' $ git init 'ext::sh -c echo% pwned% >GIT-EXT-COMMAND-INJECTION% #'
Initialized empty Git repository in $TESTTMP/ext::sh -c echo% pwned% >GIT-EXT-COMMAND-INJECTION% #/.git/ Initialized empty Git repository in $TESTTMP/ext::sh -c echo% pwned% >GIT-EXT-COMMAND-INJECTION% #/.git/
$ cd 'ext::sh -c echo% pwned% >GIT-EXT-COMMAND-INJECTION% #' $ cd 'ext::sh -c echo% pwned% >GIT-EXT-COMMAND-INJECTION% #'
$ git commit -q --allow-empty -m 'empty' $ commit -q --allow-empty -m 'empty'
$ cd .. $ cd ..
$ hg convert 'ext::sh -c echo% pwned% >GIT-EXT-COMMAND-INJECTION% #' 'converted-git-ext' $ hg convert 'ext::sh -c echo% pwned% >GIT-EXT-COMMAND-INJECTION% #' 'converted-git-ext'
initializing destination converted-git-ext repository initializing destination converted-git-ext repository
scanning source... scanning source...
sorting... sorting...
converting... converting...
0 empty 0 empty
updating bookmarks updating bookmarks
$ test -f GIT-EXT-COMMAND-INJECTION $ test -f GIT-EXT-COMMAND-INJECTION
[1] [1]
#endif #endif
Conversion of extra commit metadata to extras works Conversion of extra commit metadata to extras works
$ git init gitextras >/dev/null 2>/dev/null $ git init gitextras >/dev/null 2>/dev/null
$ cd gitextras $ cd gitextras
$ touch foo $ touch foo
$ git add foo $ git add foo
$ commit -m initial $ commit -m initial
$ echo 1 > foo $ echo 1 > foo
$ tree=`git write-tree` $ tree=`git write-tree`
$ parent=`git rev-parse HEAD`
Git doesn't provider a user-facing API to write extra metadata into the Git doesn't provider a user-facing API to write extra metadata into the
commit, so create the commit object by hand commit, so create the commit object by hand
$ git hash-object -t commit -w --stdin << EOF $ git hash-object -t commit -w --stdin << EOF
> tree ${tree} > tree ${tree}
> parent ba6b1344e977ece9e00958dbbf17f1f09384b2c1 > parent ${parent}
> author test <test@example.com> 1000000000 +0000 > author test <test@example.com> 1000000000 +0000
> committer test <test@example.com> 1000000000 +0000 > committer test <test@example.com> 1000000000 +0000
> extra-1 extra-1 > extra-1 extra-1
> extra-2 extra-2 with space > extra-2 extra-2 with space
> convert_revision 0000aaaabbbbccccddddeeee > convert_revision 0000aaaabbbbccccddddeeee
> >
> message with extras > message with extras
> EOF > EOF
8123727c8361a4117d1a2d80e0c4e7d70c757f18 9a234dca4345fc0263417fd58f23d071233d4d45
$ git reset --hard 8123727c8361a4117d1a2d80e0c4e7d70c757f18 > /dev/null $ git reset --hard 9a234dca4345fc0263417fd58f23d071233d4d45 > /dev/null
$ cd .. $ cd ..
convert will not retain custom metadata keys by default convert will not retain custom metadata keys by default
$ hg convert gitextras hgextras1 $ hg convert gitextras hgextras1
initializing destination hgextras1 repository initializing destination hgextras1 repository
scanning source... scanning source...
sorting... sorting...
converting... converting...
1 initial 1 initial
0 message with extras 0 message with extras
updating bookmarks updating bookmarks
$ hg -R hgextras1 log --debug -r 1 $ hg -R hgextras1 log --debug -r 1
changeset: 1:e13a39880f68479127b2a80fa0b448cc8524aa09 changeset: 1:eb0a79a9d51fc8310cf9d569f1703d41a0acf823
bookmark: master bookmark: master
tag: tip tag: tip
phase: draft phase: draft
parent: 0:dcb68977c55cd02cbd13b901df65c4b6e7b9c4b9 parent: 0:7cc0c933b0b075a2a6c36cb9b1638f9bb54394b3
parent: -1:0000000000000000000000000000000000000000 parent: -1:0000000000000000000000000000000000000000
manifest: 0:6a3df4de388f3c4f8e28f4f9a814299a3cbb5f50 manifest: 0:6a3df4de388f3c4f8e28f4f9a814299a3cbb5f50
user: test <test@example.com> user: test <test@example.com>
date: Sun Sep 09 01:46:40 2001 +0000 date: Sun Sep 09 01:46:40 2001 +0000
extra: branch=default extra: branch=default
extra: convert_revision=8123727c8361a4117d1a2d80e0c4e7d70c757f18 extra: convert_revision=9a234dca4345fc0263417fd58f23d071233d4d45
description: description:
message with extras message with extras
Attempting to convert a banned extra is disallowed Attempting to convert a banned extra is disallowed
$ hg convert --config convert.git.extrakeys=tree,parent gitextras hgextras-banned $ hg convert --config convert.git.extrakeys=tree,parent gitextras hgextras-banned
initializing destination hgextras-banned repository initializing destination hgextras-banned repository
abort: copying of extra key is forbidden: parent, tree abort: copying of extra key is forbidden: parent, tree
[255] [255]
Converting a specific extra works Converting a specific extra works
$ hg convert --config convert.git.extrakeys=extra-1 gitextras hgextras2 $ hg convert --config convert.git.extrakeys=extra-1 gitextras hgextras2
initializing destination hgextras2 repository initializing destination hgextras2 repository
scanning source... scanning source...
sorting... sorting...
converting... converting...
1 initial 1 initial
0 message with extras 0 message with extras
updating bookmarks updating bookmarks
$ hg -R hgextras2 log --debug -r 1 $ hg -R hgextras2 log --debug -r 1
changeset: 1:d40fb205d58597e6ecfd55b16f198be5bf436391 changeset: 1:ee0741577061d2a30889ba3138797a59185010df
bookmark: master bookmark: master
tag: tip tag: tip
phase: draft phase: draft
parent: 0:dcb68977c55cd02cbd13b901df65c4b6e7b9c4b9 parent: 0:7cc0c933b0b075a2a6c36cb9b1638f9bb54394b3
parent: -1:0000000000000000000000000000000000000000 parent: -1:0000000000000000000000000000000000000000
manifest: 0:6a3df4de388f3c4f8e28f4f9a814299a3cbb5f50 manifest: 0:6a3df4de388f3c4f8e28f4f9a814299a3cbb5f50
user: test <test@example.com> user: test <test@example.com>
date: Sun Sep 09 01:46:40 2001 +0000 date: Sun Sep 09 01:46:40 2001 +0000
extra: branch=default extra: branch=default
extra: convert_revision=8123727c8361a4117d1a2d80e0c4e7d70c757f18 extra: convert_revision=9a234dca4345fc0263417fd58f23d071233d4d45
extra: extra-1=extra-1 extra: extra-1=extra-1
description: description:
message with extras message with extras
Converting multiple extras works Converting multiple extras works
$ hg convert --config convert.git.extrakeys=extra-1,extra-2 gitextras hgextras3 $ hg convert --config convert.git.extrakeys=extra-1,extra-2 gitextras hgextras3
initializing destination hgextras3 repository initializing destination hgextras3 repository
scanning source... scanning source...
sorting... sorting...
converting... converting...
1 initial 1 initial
0 message with extras 0 message with extras
updating bookmarks updating bookmarks
$ hg -R hgextras3 log --debug -r 1 $ hg -R hgextras3 log --debug -r 1
changeset: 1:0105af33379e7b6491501fd34141b7af700fe125 changeset: 1:e3ef3af18ebac06c213cd0194d306f6c591a0c9b
bookmark: master bookmark: master
tag: tip tag: tip
phase: draft phase: draft
parent: 0:dcb68977c55cd02cbd13b901df65c4b6e7b9c4b9 parent: 0:7cc0c933b0b075a2a6c36cb9b1638f9bb54394b3
parent: -1:0000000000000000000000000000000000000000 parent: -1:0000000000000000000000000000000000000000
manifest: 0:6a3df4de388f3c4f8e28f4f9a814299a3cbb5f50 manifest: 0:6a3df4de388f3c4f8e28f4f9a814299a3cbb5f50
user: test <test@example.com> user: test <test@example.com>
date: Sun Sep 09 01:46:40 2001 +0000 date: Sun Sep 09 01:46:40 2001 +0000
extra: branch=default extra: branch=default
extra: convert_revision=8123727c8361a4117d1a2d80e0c4e7d70c757f18 extra: convert_revision=9a234dca4345fc0263417fd58f23d071233d4d45
extra: extra-1=extra-1 extra: extra-1=extra-1
extra: extra-2=extra-2 with space extra: extra-2=extra-2 with space
description: description:
message with extras message with extras
convert.git.saverev can be disabled to prevent convert_revision from being written convert.git.saverev can be disabled to prevent convert_revision from being written
$ hg convert --config convert.git.saverev=false gitextras hgextras4 $ hg convert --config convert.git.saverev=false gitextras hgextras4
initializing destination hgextras4 repository initializing destination hgextras4 repository
scanning source... scanning source...
sorting... sorting...
converting... converting...
1 initial 1 initial
0 message with extras 0 message with extras
updating bookmarks updating bookmarks
$ hg -R hgextras4 log --debug -r 1 $ hg -R hgextras4 log --debug -r 1
changeset: 1:1dcaf4ffe5bee43fa86db2800821f6f0af212c5c changeset: 1:2f6315d25e96c4496516c4c93a439bc7f22c50b2
bookmark: master bookmark: master
tag: tip tag: tip
phase: draft phase: draft
parent: 0:a13935fec4daf06a5a87a7307ccb0fc94f98d06d parent: 0:ba282370acbe8adbb6c2b5b1036d4f7e8c10ff0c
parent: -1:0000000000000000000000000000000000000000 parent: -1:0000000000000000000000000000000000000000
manifest: 0:6a3df4de388f3c4f8e28f4f9a814299a3cbb5f50 manifest: 0:6a3df4de388f3c4f8e28f4f9a814299a3cbb5f50
user: test <test@example.com> user: test <test@example.com>
date: Sun Sep 09 01:46:40 2001 +0000 date: Sun Sep 09 01:46:40 2001 +0000
extra: branch=default extra: branch=default
description: description:
message with extras message with extras
convert.git.saverev and convert.git.extrakeys can be combined to preserve convert.git.saverev and convert.git.extrakeys can be combined to preserve
convert_revision from source convert_revision from source
$ hg convert --config convert.git.saverev=false --config convert.git.extrakeys=convert_revision gitextras hgextras5 $ hg convert --config convert.git.saverev=false --config convert.git.extrakeys=convert_revision gitextras hgextras5
initializing destination hgextras5 repository initializing destination hgextras5 repository
scanning source... scanning source...
sorting... sorting...
converting... converting...
1 initial 1 initial
0 message with extras 0 message with extras
updating bookmarks updating bookmarks
$ hg -R hgextras5 log --debug -r 1 $ hg -R hgextras5 log --debug -r 1
changeset: 1:574d85931544d4542007664fee3747360e85ee28 changeset: 1:730cc22ac35bf6010a5205dd37bd8e8bf679eb24
bookmark: master bookmark: master
tag: tip tag: tip
phase: draft phase: draft
parent: 0:a13935fec4daf06a5a87a7307ccb0fc94f98d06d parent: 0:ba282370acbe8adbb6c2b5b1036d4f7e8c10ff0c
parent: -1:0000000000000000000000000000000000000000 parent: -1:0000000000000000000000000000000000000000
manifest: 0:6a3df4de388f3c4f8e28f4f9a814299a3cbb5f50 manifest: 0:6a3df4de388f3c4f8e28f4f9a814299a3cbb5f50
user: test <test@example.com> user: test <test@example.com>
date: Sun Sep 09 01:46:40 2001 +0000 date: Sun Sep 09 01:46:40 2001 +0000
extra: branch=default extra: branch=default
extra: convert_revision=0000aaaabbbbccccddddeeee extra: convert_revision=0000aaaabbbbccccddddeeee
description: description:
message with extras message with extras