This is an archive of the discontinued Mercurial Phabricator instance.

tests: better determinism in test-chg.t
ClosedPublic

Authored by aalekseyev on Oct 21 2021, 7:19 AM.

Details

Summary

chg tests fail pretty often with "Sample count: *" line disappearing.
It disappears because the sample count is zero, in which case a custom message is printed.
This commit makes the test succeed in that case.

Diff Detail

Repository
rHG Mercurial
Lint
Automatic diff as part of commit; lint not applicable.
Unit
Automatic diff as part of commit; unit tests not applicable.

Event Timeline

aalekseyev created this revision.Oct 21 2021, 7:19 AM
Alphare accepted this revision.Oct 21 2021, 8:07 AM
Alphare added a subscriber: Alphare.

I'll queue this on stable to help benefit the 5.9.3 release as well, and try to merge into default.

This revision is now accepted and ready to land.Oct 21 2021, 8:07 AM
This revision was automatically updated to reflect the committed changes.

Are we sure these difference in message are valid and not some side effect from the use of sleep later in that tests?

The use of sleep is definitely an issue that should be fixed (should have been months ago but delays), but this stops it from being irritating for a bit while not actually really hiding a bug (IIUC)

I tried running some of these commands by hand and they reported very low sample counts (under 10 was common). I figured that the likelihood of getting 0 samples instead of 9 by pure chance is pretty high and did not look further.

I tried running some of these commands by hand and they reported very low sample counts (under 10 was common). I figured that the likelihood of getting 0 samples instead of 9 by pure chance is pretty high and did not look further.

I would appreciate if you could double check, that we are not disabling what the test is actually testing for here. (I did not check myself)

By looking at the code it seemed that the test is checking whether or not the profiling is enabled. The "No samples recorded" message is only printed when profiling is enabled (you can see that because some invocations print the sample count and some don't), so I claim that, to the first approximation, the test is still testing what it was supposed to test.
If there's a more subtle property being tested here, I'm afraid I won't be able to figure that out.

By looking at the code it seemed that the test is checking whether or not the profiling is enabled. The "No samples recorded" message is only printed when profiling is enabled (you can see that because some invocations print the sample count and some don't), so I claim that, to the first approximation, the test is still testing what it was supposed to test.
If there's a more subtle property being tested here, I'm afraid I won't be able to figure that out.

Thanks for looking into it. Seems like this changeset is as great as it initially looked :-)