This is an archive of the discontinued Mercurial Phabricator instance.

Differential D2066

lfs: add a test showing bundle application could be broken
ClosedPublic

Authored by quark on Feb 6 2018, 8:14 PM.

Details

Reviewers
indygreg
Group Reviewers
hg-reviewers
Commits
rHG4e41b59633fa: lfs: add a test showing bundle application could be broken
Summary

When a bundle containing LFS delta uses non-LFS delta-base, or vice-versa,
the bundle will fail to apply.

Diff Detail

Repository
rHG Mercurial
Lint
Automatic diff as part of commit; lint not applicable.
Unit
Automatic diff as part of commit; unit tests not applicable.

Event Timeline

quark created this revision.Feb 6 2018, 8:14 PM
quark updated this revision to Diff 5268.Feb 6 2018, 8:52 PM
quark updated this revision to Diff 5272.Feb 6 2018, 10:17 PM
quark updated this revision to Diff 5275.Feb 6 2018, 11:43 PM
indygreg accepted this revision.Feb 7 2018, 5:27 PM
This revision is now accepted and ready to land.Feb 7 2018, 5:27 PM
quark updated this revision to Diff 5326.Feb 7 2018, 7:35 PM
mharbison72 added a subscriber: mharbison72.Feb 13 2018, 5:04 PM

This series worked for me in practice. I assume there's no chance that this lands on stable to help people that are pushing to a repo converted to LFS, but that didn't reclone after converting?

tests/test-lfs-bundle.t
23

This needs to be 'url=file:$TESTDIR/lfs-remote' to keep MSYS happy.

quark updated this revision to Diff 5730.Feb 14 2018, 7:42 PM
quark added a comment.Feb 14 2018, 7:59 PM
In D2066#36976, @mharbison72 wrote:

This series worked for me in practice. I assume there's no chance that this lands on stable to help people that are pushing to a repo converted to LFS, but that didn't reclone after converting?

That should be fine - there is no need to reclone. But upgrading is needed.

indygreg accepted this revision.Mar 6 2018, 12:59 PM
Closed by commit rHG4e41b59633fa: lfs: add a test showing bundle application could be broken (authored by quark). · Explain WhyMar 6 2018, 1:10 PM
This revision was automatically updated to reflect the committed changes.

Revision Contents
Changeset List

PathPackages
Mtests/drawdag.py (3 lines)
AMtests/test-lfs-bundle.t (100 lines)

Diff 6669

tests/drawdag.py

raise error.Abort(_('%s: too many parents: %s') raise error.Abort(_('%s: too many parents: %s')
% (k, b' '.join(v))) % (k, b' '.join(v)))
# parse comments to get extra file content instructions # parse comments to get extra file content instructions
files = collections.defaultdict(dict) # {(name, path): content} files = collections.defaultdict(dict) # {(name, path): content}
comments = list(_getcomments(text)) comments = list(_getcomments(text))
filere = re.compile(br'^(\w+)/([\w/]+)\s*=\s*(.*)$', re.M) filere = re.compile(br'^(\w+)/([\w/]+)\s*=\s*(.*)$', re.M)
for name, path, content in filere.findall(b'\n'.join(comments)): for name, path, content in filere.findall(b'\n'.join(comments)):
files[name][path] = content.replace(br'\n', b'\n') content = content.replace(br'\n', b'\n').replace(br'\1', b'\1')
files[name][path] = content
committed = {None: node.nullid} # {name: node} committed = {None: node.nullid} # {name: node}
# for leaf nodes, try to find existing nodes in repo # for leaf nodes, try to find existing nodes in repo
for name, parents in edges.items(): for name, parents in edges.items():
if len(parents) == 0: if len(parents) == 0:
try: try:
committed[name] = scmutil.revsingle(repo, name) committed[name] = scmutil.revsingle(repo, name)

tests/test-lfs-bundle.t

  • This file was added.
In this test, we want to test LFS bundle application on both LFS and non-LFS
repos.
To make it more interesting, the file revisions will contain hg filelog
metadata ('\1\n'). The bundle will have 1 file revision overlapping with the
destination repo.
# rev 1 2 3
# repo: yes yes no
# bundle: no (base) yes yes (deltabase: 2 if possible)
It is interesting because rev 2 could have been stored as LFS in the repo, and
non-LFS in the bundle; or vice-versa.
Init
$ cat >> $HGRCPATH << EOF
> [extensions]
> lfs=
> drawdag=$TESTDIR/drawdag.py
> [lfs]
> url=file:$TESTTMP/lfs-remote
> EOF
mharbison72Unsubmitted
Not Done

This needs to be 'url=file:$TESTDIR/lfs-remote' to keep MSYS happy.

mharbison72: This needs to be 'url=file:$TESTDIR/lfs-remote' to keep MSYS happy.
Helper functions
$ commitxy() {
> hg debugdrawdag "$@" <<'EOS'
> Y # Y/X=\1\nAAAA\nE\nF
> | # Y/Y=\1\nAAAA\nG\nH
> X # X/X=\1\nAAAA\nC\n
> # X/Y=\1\nAAAA\nD\n
> EOS
> }
$ commitz() {
> hg debugdrawdag "$@" <<'EOS'
> Z # Z/X=\1\nAAAA\nI\n
> | # Z/Y=\1\nAAAA\nJ\n
> | # Z/Z=\1\nZ
> Y
> EOS
> }
$ enablelfs() {
> cat >> .hg/hgrc <<EOF
> [lfs]
> track=all()
> EOF
> }
Generate bundles
$ for i in normal lfs; do
> NAME=src-$i
> hg init $TESTTMP/$NAME
> cd $TESTTMP/$NAME
> [ $i = lfs ] && enablelfs
> commitxy
> commitz
> hg bundle -q --base X -r Y+Z $TESTTMP/$NAME.bundle
> SRCNAMES="$SRCNAMES $NAME"
> done
Prepare destination repos
$ for i in normal lfs; do
> NAME=dst-$i
> hg init $TESTTMP/$NAME
> cd $TESTTMP/$NAME
> [ $i = lfs ] && enablelfs
> commitxy
> DSTNAMES="$DSTNAMES $NAME"
> done
Apply bundles
$ for i in $SRCNAMES; do
> for j in $DSTNAMES; do
> echo ---- Applying $i.bundle to $j ----
> cp -R $TESTTMP/$j $TESTTMP/tmp-$i-$j
> cd $TESTTMP/tmp-$i-$j
> if hg unbundle $TESTTMP/$i.bundle -q 2>/dev/null; then
> hg verify -q && echo OK
> else
> echo CRASHED
> fi
> done
> done
---- Applying src-normal.bundle to dst-normal ----
OK
---- Applying src-normal.bundle to dst-lfs ----
X@2: unpacking bcc7d23fa6b7: integrity check failed on data/X.i:2
Y@2: unpacking 46017a6640e7: integrity check failed on data/Y.i:2
2 integrity errors encountered!
(first damaged changeset appears to be 2)
---- Applying src-lfs.bundle to dst-normal ----
CRASHED
---- Applying src-lfs.bundle to dst-lfs ----
OK