( )⚙ D11263 debugcommands: add a `--paranoid` option to `debug-repair-issue-6528`

This is an archive of the discontinued Mercurial Phabricator instance.

Differential D11263

debugcommands: add a `--paranoid` option to `debug-repair-issue-6528`
ClosedPublic

Authored by Alphare on Aug 6 2021, 6:12 AM.

Details

Reviewers
marmoute
pulkit
Group Reviewers
hg-reviewers
Commits
rHG855463b5fe49: debugcommands: add a `--paranoid` option to `debug-repair-issue-6528`
Summary

See justification inline.

Diff Detail

Repository
rHG Mercurial
Branch
stable
Lint
No Linters Available
Unit
No Unit Test Coverage

Event Timeline

Alphare created this revision.Aug 6 2021, 6:12 AM
marmoute requested changes to this revision.Aug 6 2021, 6:53 AM
marmoute added a subscriber: marmoute.
marmoute added inline comments.
mercurial/revlogutils/rewrite.py
729–730

This should be an Abort since we are sending this to the User.

This revision now requires changes to proceed.Aug 6 2021, 6:53 AM
Alphare updated this revision to Diff 29837.Aug 6 2021, 1:50 PM
marmoute accepted this revision.Aug 6 2021, 5:36 PM
marmoute updated this revision to Diff 29842.Aug 7 2021, 6:13 AM
baymax updated this revision to Diff 29845.Aug 7 2021, 9:47 AM

✅ refresh by Heptapod after a successful CI run (🐙 💚)
⚠ This patch is intended for stable ⚠

baymax updated this revision to Diff 29857.Aug 7 2021, 6:09 PM

✅ refresh by Heptapod after a successful CI run (🐙 💚)
⚠ This patch is intended for stable ⚠

pulkit accepted this revision.Aug 10 2021, 8:17 AM
This revision is now accepted and ready to land.Aug 10 2021, 8:17 AM

Revision Contents
Changeset List

Diff 29837

mercurial/debugcommands.py

), ),
( (
b'', b'',
b'from-report', b'from-report',
b'', b'',
_(b'repair revisions listed in this report file'), _(b'repair revisions listed in this report file'),
_(b'FILE'), _(b'FILE'),
), ),
(
b'',
b'paranoid',
False,
_(b'check that both detection methods do the same thing'),
),
] ]
+ cmdutil.dryrunopts, + cmdutil.dryrunopts,
) )
def debug_repair_issue6528(ui, repo, **opts): def debug_repair_issue6528(ui, repo, **opts):
"""find affected revisions and repair them. See issue6528 for more details. """find affected revisions and repair them. See issue6528 for more details.
The `--to-report` and `--from-report` flags allow you to cache and reuse the The `--to-report` and `--from-report` flags allow you to cache and reuse the
computation of affected revisions for a given repository across clones. computation of affected revisions for a given repository across clones.
The report format is line-based (with empty lines ignored): The report format is line-based (with empty lines ignored):
``` ```
<ascii-hex of the affected revision>,... <unencoded filelog index filename> <ascii-hex of the affected revision>,... <unencoded filelog index filename>
``` ```
There can be multiple broken revisions per filelog, they are separated by There can be multiple broken revisions per filelog, they are separated by
a comma with no spaces. The only space is between the revision(s) and the a comma with no spaces. The only space is between the revision(s) and the
filename. filename.
Note that this does *not* mean that this repairs future affected revisions, Note that this does *not* mean that this repairs future affected revisions,
that needs a separate fix at the exchange level that hasn't been written yet that needs a separate fix at the exchange level that hasn't been written yet
(as of 5.9rc0). (as of 5.9rc0).
There is a `--paranoid` flag to test that the fast implementation is correct
by checking it against the slow implementation. Since this matter is quite
urgent and testing every edge-case is probably quite costly, we use this
method to test on large repositories as a fuzzing method of sorts.
""" """
cmdutil.check_incompatible_arguments( cmdutil.check_incompatible_arguments(
opts, 'to_report', ['from_report', 'dry_run'] opts, 'to_report', ['from_report', 'dry_run']
) )
dry_run = opts.get('dry_run') dry_run = opts.get('dry_run')
to_report = opts.get('to_report') to_report = opts.get('to_report')
from_report = opts.get('from_report') from_report = opts.get('from_report')
paranoid = opts.get('paranoid')
# TODO maybe add filelog pattern and revision pattern parameters to help # TODO maybe add filelog pattern and revision pattern parameters to help
# narrow down the search for users that know what they're looking for? # narrow down the search for users that know what they're looking for?
if requirements.REVLOGV1_REQUIREMENT not in repo.requirements: if requirements.REVLOGV1_REQUIREMENT not in repo.requirements:
msg = b"can only repair revlogv1 repositories, v2 is not affected" msg = b"can only repair revlogv1 repositories, v2 is not affected"
raise error.Abort(_(msg)) raise error.Abort(_(msg))
rewrite.repair_issue6528( rewrite.repair_issue6528(
ui, repo, dry_run=dry_run, to_report=to_report, from_report=from_report ui,
repo,
dry_run=dry_run,
to_report=to_report,
from_report=from_report,
paranoid=paranoid,
) )
@command(b'debugformat', [] + cmdutil.formatteropts) @command(b'debugformat', [] + cmdutil.formatteropts)
def debugformat(ui, repo, **opts): def debugformat(ui, repo, **opts):
"""display format information about the current repository """display format information about the current repository
Use --verbose to get extra information about current config value and Use --verbose to get extra information about current config value and

mercurial/revlogutils/rewrite.py

if not to_fix: if not to_fix:
msg = _(b"no affected revisions were found for '%s'\n") msg = _(b"no affected revisions were found for '%s'\n")
ui.write(msg % filename) ui.write(msg % filename)
continue continue
if not dry_run: if not dry_run:
_reorder_filelog_parents(repo, fl, sorted(to_fix)) _reorder_filelog_parents(repo, fl, sorted(to_fix))
def repair_issue6528(ui, repo, dry_run=False, to_report=None, from_report=None): def repair_issue6528(
ui, repo, dry_run=False, to_report=None, from_report=None, paranoid=False
):
from .. import store # avoid cycle from .. import store # avoid cycle
@contextlib.contextmanager @contextlib.contextmanager
def context(): def context():
if dry_run or to_report: # No need for locking if dry_run or to_report: # No need for locking
yield yield
else: else:
with repo.wlock(), repo.lock(): with repo.wlock(), repo.lock():
# Set of filerevs (or hex filenodes if `to_report`) that need fixing # Set of filerevs (or hex filenodes if `to_report`) that need fixing
to_fix = set() to_fix = set()
metadata_cache = {} metadata_cache = {}
for filerev in fl.revs(): for filerev in fl.revs():
affected = _is_revision_affected_fast( affected = _is_revision_affected_fast(
repo, fl, filerev, metadata_cache repo, fl, filerev, metadata_cache
) )
if paranoid:
slow = _is_revision_affected(fl, filerev)
if slow != affected:
msg = _(b"paranoid check failed for '%s' at node %s")
node = binascii.hexlify(fl.node(filerev))
raise error.Abort(msg % (filename, node))
marmouteUnsubmitted
Not Done

This should be an Abort since we are sending this to the User.

marmoute: This should be an Abort since we are sending this to the User.
if affected: if affected:
msg = b"found affected revision %d for filelog '%s'\n" msg = b"found affected revision %d for filelog '%s'\n"
ui.warn(msg % (filerev, path)) ui.warn(msg % (filerev, path))
found_nothing = False found_nothing = False
if not dry_run: if not dry_run:
if to_report: if to_report:
to_fix.add(binascii.hexlify(fl.node(filerev))) to_fix.add(binascii.hexlify(fl.node(filerev)))
else: else:

tests/test-completion.t

branches: active, closed, rev, template branches: active, closed, rev, template
bundle: force, rev, branch, base, all, type, ssh, remotecmd, insecure bundle: force, rev, branch, base, all, type, ssh, remotecmd, insecure
cat: output, rev, decode, include, exclude, template cat: output, rev, decode, include, exclude, template
clone: noupdate, updaterev, rev, branch, pull, uncompressed, stream, ssh, remotecmd, insecure clone: noupdate, updaterev, rev, branch, pull, uncompressed, stream, ssh, remotecmd, insecure
commit: addremove, close-branch, amend, secret, edit, force-close-branch, interactive, include, exclude, message, logfile, date, user, subrepos commit: addremove, close-branch, amend, secret, edit, force-close-branch, interactive, include, exclude, message, logfile, date, user, subrepos
config: untrusted, exp-all-known, edit, local, source, shared, non-shared, global, template config: untrusted, exp-all-known, edit, local, source, shared, non-shared, global, template
continue: dry-run continue: dry-run
copy: forget, after, at-rev, force, include, exclude, dry-run copy: forget, after, at-rev, force, include, exclude, dry-run
debug-repair-issue6528: to-report, from-report, dry-run debug-repair-issue6528: to-report, from-report, paranoid, dry-run
debugancestor: debugancestor:
debugantivirusrunning: debugantivirusrunning:
debugapplystreamclonebundle: debugapplystreamclonebundle:
debugbackupbundle: recover, patch, git, limit, no-merges, stat, graph, style, template debugbackupbundle: recover, patch, git, limit, no-merges, stat, graph, style, template
debugbuilddag: mergeable-file, overwritten-file, new-file debugbuilddag: mergeable-file, overwritten-file, new-file
debugbundle: all, part-type, spec debugbundle: all, part-type, spec
debugcapabilities: debugcapabilities:
debugchangedfiles: compute debugchangedfiles: compute

tests/test-issue6528.t

1 3 a58b36ad6b65 05b806ebe5ea 000000000000 1 3 a58b36ad6b65 05b806ebe5ea 000000000000
2 6 216a5fe8b8ed 000000000000 000000000000 2 6 216a5fe8b8ed 000000000000 000000000000
3 7 ea4f2f2463cc 216a5fe8b8ed 000000000000 3 7 ea4f2f2463cc 216a5fe8b8ed 000000000000
$ hg debugrevlogindex D.txt $ hg debugrevlogindex D.txt
rev linkrev nodeid p1 p2 rev linkrev nodeid p1 p2
0 6 2a8d3833f2fb 000000000000 000000000000 0 6 2a8d3833f2fb 000000000000 000000000000
1 7 2a80419dfc31 2a8d3833f2fb 000000000000 1 7 2a80419dfc31 2a8d3833f2fb 000000000000
Test the --paranoid option
$ hg debug-repair-issue6528 --dry-run --paranoid
found affected revision 1 for filelog 'data/D.txt.i'
found affected revision 1 for filelog 'data/b.txt.i'
found affected revision 3 for filelog 'data/b.txt.i'
$ hg st
M D.txt
M b.txt
$ hg debugrevlogindex b.txt
rev linkrev nodeid p1 p2
0 2 05b806ebe5ea 000000000000 000000000000
1 3 a58b36ad6b65 05b806ebe5ea 000000000000
2 6 216a5fe8b8ed 000000000000 000000000000
3 7 ea4f2f2463cc 216a5fe8b8ed 000000000000
$ hg debugrevlogindex D.txt
rev linkrev nodeid p1 p2
0 6 2a8d3833f2fb 000000000000 000000000000
1 7 2a80419dfc31 2a8d3833f2fb 000000000000
Run the fix Run the fix
$ hg debug-repair-issue6528 $ hg debug-repair-issue6528
found affected revision 1 for filelog 'data/D.txt.i' found affected revision 1 for filelog 'data/D.txt.i'
repaired revision 1 of 'filelog data/D.txt.i' repaired revision 1 of 'filelog data/D.txt.i'
found affected revision 1 for filelog 'data/b.txt.i' found affected revision 1 for filelog 'data/b.txt.i'
found affected revision 3 for filelog 'data/b.txt.i' found affected revision 3 for filelog 'data/b.txt.i'
repaired revision 1 of 'filelog data/b.txt.i' repaired revision 1 of 'filelog data/b.txt.i'
repaired revision 3 of 'filelog data/b.txt.i' repaired revision 3 of 'filelog data/b.txt.i'