This is an archive of the discontinued Mercurial Phabricator instance.

Differential D5960

test: stabilize test-remotefilelog-bgprefetch.t flaky output
ClosedPublic

Authored by lothiraldan on Feb 14 2019, 9:07 AM.

Details

Reviewers
None
Group Reviewers
hg-reviewers
Commits
rHG1c4d6ab2ecb8: test: stabilize test-remotefilelog-bgprefetch.t flaky output
rHG4cb2ac16d9af: test: stabilize test-remotefilelog-bgprefetch.t flaky output
rHG5b6ee1b16f9a: test: stabilize test-remotefilelog-bgprefetch.t flaky output
Summary

When running the test suite with multiple processes, we often get flaky
outputs, like here: https://ci.octobus.net/job/MercurialPy2/274/console

-  $TESTTMP/hgcache/master/packs/8f1443d44e57fec96f72fb2412e01d2818767ef2.histidx
-  $TESTTMP/hgcache/master/packs/8f1443d44e57fec96f72fb2412e01d2818767ef2.histpack
-  $TESTTMP/hgcache/master/packs/f4d50848e0b465e9bfd2875f213044c06cfd7407.dataidx
-  $TESTTMP/hgcache/master/packs/f4d50848e0b465e9bfd2875f213044c06cfd7407.datapack
+  $TESTTMP/hgcache/master/11/f6ad8ec52a2984abaafd7c3b516503785c2072/1406e74118627694268417491f018a4a883152f0
+  $TESTTMP/hgcache/master/11/f6ad8ec52a2984abaafd7c3b516503785c2072/ef95c5376f34698742fe34f315fd82136f8f68c0
+  $TESTTMP/hgcache/master/39/5df8f7c51f007019cb30201c49e884b46b92fa/69a1b67522704ec122181c0890bd16e9d3e7516a
+  $TESTTMP/hgcache/master/95/cb0bfd2977c761298d9624e4b4d4c72a39974a/076f5e2225b3ff0400b98c92aa6cdf403ee24cca
+  $TESTTMP/hgcache/master/af/f024fe4ab0fece4091de044c58c9ae4233383a/bb6ccd5dceaa5e9dc220e0dad65e051b94f69a2c

Add some sleeps after debugwaitonrepack calls as it seems it needs some
extra time to cleanup.

Diff Detail

Repository
rHG Mercurial
Lint
Automatic diff as part of commit; lint not applicable.
Unit
Automatic diff as part of commit; unit tests not applicable.

Event Timeline

lothiraldan created this revision.Feb 14 2019, 9:07 AM
durin42 added a subscriber: durin42.Feb 15 2019, 11:02 PM

Huh. That's strictly a bug, as debugwaitonrepack should actually be waiting.

lothiraldan added a comment.EditedFeb 18 2019, 5:03 AM
In D5960#87251, @durin42 wrote:

Huh. That's strictly a bug, as debugwaitonrepack should actually be waiting.

I've created a ticket to track it: https://bz.mercurial-scm.org/show_bug.cgi?id=6083

Do you have any idea what could be happening?

durin42 added a comment.Feb 19 2019, 10:09 AM
In D5960#87330, @lothiraldan wrote:
In D5960#87251, @durin42 wrote:

Huh. That's strictly a bug, as debugwaitonrepack should actually be waiting.

I've created a ticket to track it: https://phab.mercurial-scm.org/D5960
Do you have any idea what could be happening?

I've never seen this failure, so I'm suspicious of your test environment. Is there anything exotic, perhaps at the filesystem level?

lothiraldan added a comment.Feb 19 2019, 1:27 PM
In D5960#87357, @durin42 wrote:
In D5960#87330, @lothiraldan wrote:
In D5960#87251, @durin42 wrote:

Huh. That's strictly a bug, as debugwaitonrepack should actually be waiting.

I've created a ticket to track it: https://phab.mercurial-scm.org/D5960
Do you have any idea what could be happening?

I've never seen this failure, so I'm suspicious of your test environment. Is there anything exotic, perhaps at the filesystem level?

I'm using BTRFS localy, I reproduced inside a mounted Docker container locally. On the CI side, the slave seems to run with an OverlayFS disk.

Closed by commit rHG5b6ee1b16f9a: test: stabilize test-remotefilelog-bgprefetch.t flaky output (authored by lothiraldan). · Explain WhyFeb 20 2019, 3:16 PM
This revision was automatically updated to reflect the committed changes.

Revision Contents
Changeset List

Diff 14145

tests/test-remotefilelog-bgprefetch.t

updating bookmark foo updating bookmark foo
new changesets 6b4b6f66ef8c new changesets 6b4b6f66ef8c
(run 'hg update' to get a working copy) (run 'hg update' to get a working copy)
prefetching file contents prefetching file contents
$ sleep 0.5 $ sleep 0.5
$ hg debugwaitonprefetch >/dev/null 2>%1 $ hg debugwaitonprefetch >/dev/null 2>%1
$ sleep 0.5 $ sleep 0.5
$ hg debugwaitonrepack >/dev/null 2>%1 $ hg debugwaitonrepack >/dev/null 2>%1
$ sleep 0.5
$ find $CACHEDIR -type f | sort $ find $CACHEDIR -type f | sort
$TESTTMP/hgcache/master/packs/6e8633deba6e544e5f8edbd7b996d6e31a2c42ae.histidx $TESTTMP/hgcache/master/packs/6e8633deba6e544e5f8edbd7b996d6e31a2c42ae.histidx
$TESTTMP/hgcache/master/packs/6e8633deba6e544e5f8edbd7b996d6e31a2c42ae.histpack $TESTTMP/hgcache/master/packs/6e8633deba6e544e5f8edbd7b996d6e31a2c42ae.histpack
$TESTTMP/hgcache/master/packs/8ce5ab3745465ab83bba30a7b9c295e0c8404652.dataidx $TESTTMP/hgcache/master/packs/8ce5ab3745465ab83bba30a7b9c295e0c8404652.dataidx
$TESTTMP/hgcache/master/packs/8ce5ab3745465ab83bba30a7b9c295e0c8404652.datapack $TESTTMP/hgcache/master/packs/8ce5ab3745465ab83bba30a7b9c295e0c8404652.datapack
$TESTTMP/hgcache/master/packs/repacklock $TESTTMP/hgcache/master/packs/repacklock
$TESTTMP/hgcache/repos $TESTTMP/hgcache/repos
$ echo b > b $ echo b > b
$ hg commit -qAm b $ hg commit -qAm b
* files fetched over 1 fetches - (* misses, 0.00% hit ratio) over *s (glob) * files fetched over 1 fetches - (* misses, 0.00% hit ratio) over *s (glob)
$ hg bookmark temporary $ hg bookmark temporary
$ sleep 1 $ sleep 1
$ hg debugwaitonprefetch >/dev/null 2>%1 $ hg debugwaitonprefetch >/dev/null 2>%1
$ sleep 1 $ sleep 1
$ hg debugwaitonrepack >/dev/null 2>%1 $ hg debugwaitonrepack >/dev/null 2>%1
$ sleep 1
$ find $CACHEDIR -type f | sort $ find $CACHEDIR -type f | sort
$TESTTMP/hgcache/master/packs/8f1443d44e57fec96f72fb2412e01d2818767ef2.histidx $TESTTMP/hgcache/master/packs/8f1443d44e57fec96f72fb2412e01d2818767ef2.histidx
$TESTTMP/hgcache/master/packs/8f1443d44e57fec96f72fb2412e01d2818767ef2.histpack $TESTTMP/hgcache/master/packs/8f1443d44e57fec96f72fb2412e01d2818767ef2.histpack
$TESTTMP/hgcache/master/packs/f4d50848e0b465e9bfd2875f213044c06cfd7407.dataidx $TESTTMP/hgcache/master/packs/f4d50848e0b465e9bfd2875f213044c06cfd7407.dataidx
$TESTTMP/hgcache/master/packs/f4d50848e0b465e9bfd2875f213044c06cfd7407.datapack $TESTTMP/hgcache/master/packs/f4d50848e0b465e9bfd2875f213044c06cfd7407.datapack
$TESTTMP/hgcache/master/packs/repacklock $TESTTMP/hgcache/master/packs/repacklock
$TESTTMP/hgcache/repos $TESTTMP/hgcache/repos
$ hg rebase -s temporary -d foo $ hg rebase -s temporary -d foo
rebasing 3:58147a5b5242 "b" (temporary tip) rebasing 3:58147a5b5242 "b" (temporary tip)
saved backup bundle to $TESTTMP/shallow/.hg/strip-backup/58147a5b5242-c3678817-rebase.hg (glob) saved backup bundle to $TESTTMP/shallow/.hg/strip-backup/58147a5b5242-c3678817-rebase.hg (glob)
3 files fetched over 1 fetches - (3 misses, 0.00% hit ratio) over *s (glob) 3 files fetched over 1 fetches - (3 misses, 0.00% hit ratio) over *s (glob)
$ sleep 1 $ sleep 1
$ hg debugwaitonprefetch >/dev/null 2>%1 $ hg debugwaitonprefetch >/dev/null 2>%1
$ sleep 1 $ sleep 1
$ hg debugwaitonrepack >/dev/null 2>%1 $ hg debugwaitonrepack >/dev/null 2>%1
$ sleep 1
# Ensure that file 'y' was prefetched - it was not part of the rebase operation and therefore # Ensure that file 'y' was prefetched - it was not part of the rebase operation and therefore
# could only be downloaded by the background prefetch # could only be downloaded by the background prefetch
$ hg debugdatapack `ls -ct $TESTTMP/hgcache/master/packs/*.datapack | head -n 1` $ hg debugdatapack `ls -ct $TESTTMP/hgcache/master/packs/*.datapack | head -n 1`
$TESTTMP/hgcache/master/packs/f4d50848e0b465e9bfd2875f213044c06cfd7407: $TESTTMP/hgcache/master/packs/f4d50848e0b465e9bfd2875f213044c06cfd7407:
w: w:
Node Delta Base Delta Length Blob Size Node Delta Base Delta Length Blob Size