This is an archive of the discontinued Mercurial Phabricator instance.

Differential D11271

issue6528: add a config option to control the fixing on the fly
ClosedPublic

Authored by marmoute on Aug 7 2021, 12:58 PM.

Details

Reviewers
pulkit
Group Reviewers
hg-reviewers
Commits
rHG2813d406b036: issue6528: add a config option to control the fixing on the fly
Summary

This will allow people who know to be safe to avoid any performance overhead
(and other potential issue).

Diff Detail

Repository
rHG Mercurial
Lint
Automatic diff as part of commit; lint not applicable.
Unit
Automatic diff as part of commit; unit tests not applicable.

Event Timeline

marmoute created this revision.Aug 7 2021, 12:58 PM
baymax updated this revision to Diff 29862.Aug 8 2021, 5:22 AM

✅ refresh by Heptapod after a successful CI run (🐙 💚)
⚠ This patch is intended for stable ⚠

Alphare added a subscriber: Alphare.Aug 8 2021, 10:07 AM

Note to other reviewers: I'm on vacation so I probably won't be queuing patches too much, I wanted to check up on the rest of my series.

tests/test-issue6528.t
595

Those comments say the opposite of what they're trying to

marmoute planned changes to this revision.Aug 8 2021, 4:08 PM
marmoute added inline comments.
tests/test-issue6528.t
595

The copy-paste devil strike again. Thanks for spotting this, update on the way.

baymax updated this revision to Diff 29864.Aug 8 2021, 6:59 PM

✅ refresh by Heptapod after a successful CI run (🐙 💚)
⚠ This patch is intended for stable ⚠

pulkit accepted this revision.Aug 10 2021, 8:19 AM
This revision is now accepted and ready to land.Aug 10 2021, 8:19 AM
pulkit added a comment.Aug 10 2021, 8:21 AM

Can we add details to releasenotes about debugcommand and config option introduced for the bug fix?

Revision Contents
Changeset List

Diff 29883

mercurial/configitems.py

experimental=True, experimental=True,
) )
coreconfigitem( coreconfigitem(
b'storage', b'storage',
b'revlog.optimize-delta-parent-choice', b'revlog.optimize-delta-parent-choice',
default=True, default=True,
alias=[(b'format', b'aggressivemergedeltas')], alias=[(b'format', b'aggressivemergedeltas')],
) )
coreconfigitem(
b'storage',
b'revlog.issue6528.fix-incoming',
default=True,
)
# experimental as long as rust is experimental (or a C version is implemented) # experimental as long as rust is experimental (or a C version is implemented)
coreconfigitem( coreconfigitem(
b'storage', b'storage',
b'revlog.persistent-nodemap.mmap', b'revlog.persistent-nodemap.mmap',
default=True, default=True,
) )
# experimental as long as format.use-persistent-nodemap is. # experimental as long as format.use-persistent-nodemap is.
coreconfigitem( coreconfigitem(

mercurial/filelog.py

target=(revlog_constants.KIND_FILELOG, path), target=(revlog_constants.KIND_FILELOG, path),
radix=b'/'.join((b'data', path)), radix=b'/'.join((b'data', path)),
censorable=True, censorable=True,
) )
# Full name of the user visible file, relative to the repository root. # Full name of the user visible file, relative to the repository root.
# Used by LFS. # Used by LFS.
self._revlog.filename = path self._revlog.filename = path
self.nullid = self._revlog.nullid self.nullid = self._revlog.nullid
opts = opener.options
self._fix_issue6528 = opts.get(b'issue6528.fix-incoming', True)
def __len__(self): def __len__(self):
return len(self._revlog) return len(self._revlog)
def __iter__(self): def __iter__(self):
return self._revlog.__iter__() return self._revlog.__iter__()
def hasnode(self, node): def hasnode(self, node):
_( _(
b'revlog storage does not support missing ' b'revlog storage does not support missing '
b'parents write mode' b'parents write mode'
) )
) )
with self._revlog._writing(transaction): with self._revlog._writing(transaction):
if self._fix_issue6528:
deltas = rewrite.filter_delta_issue6528(self._revlog, deltas) deltas = rewrite.filter_delta_issue6528(self._revlog, deltas)
return self._revlog.addgroup( return self._revlog.addgroup(
deltas, deltas,
linkmapper, linkmapper,
transaction, transaction,
addrevisioncb=addrevisioncb, addrevisioncb=addrevisioncb,
duplicaterevisioncb=duplicaterevisioncb, duplicaterevisioncb=duplicaterevisioncb,
) )

mercurial/helptext/config.txt

(default: True) (default: True)
``storage`` ``storage``
----------- -----------
Control the strategy Mercurial uses internally to store history. Options in this Control the strategy Mercurial uses internally to store history. Options in this
category impact performance and repository size. category impact performance and repository size.
``revlog.issue6528.fix-incoming``
Version 5.8 of Mercurial had a bug leading to altering the parent of file
revision with copy information (or any other metadata) on exchange. This
leads to the copy metadata to be overlooked by various internal logic. The
issue was fixed in Mercurial 5.8.1.
(See https://bz.mercurial-scm.org/show_bug.cgi?id=6528 for details)
As a result Mercurial is now checking and fixing incoming file revisions to
make sure there parents are in the right order. This behavior can be
disabled by setting this option to `no`. This apply to revisions added
through push, pull, clone and unbundle.
To fix affected revisions that already exist within the repository, one can
use :hg:`debug-repair-issue-6528`.
``revlog.optimize-delta-parent-choice`` ``revlog.optimize-delta-parent-choice``
When storing a merge revision, both parents will be equally considered as When storing a merge revision, both parents will be equally considered as
a possible delta base. This results in better delta selection and improved a possible delta base. This results in better delta selection and improved
revlog compression. This option is enabled by default. revlog compression. This option is enabled by default.
Turning this option off can result in large increase of repository size for Turning this option off can result in large increase of repository size for
repository with many merges. repository with many merges.

mercurial/localrepo.py

if chunkcachesize is not None: if chunkcachesize is not None:
options[b'chunkcachesize'] = chunkcachesize options[b'chunkcachesize'] = chunkcachesize
deltabothparents = ui.configbool( deltabothparents = ui.configbool(
b'storage', b'revlog.optimize-delta-parent-choice' b'storage', b'revlog.optimize-delta-parent-choice'
) )
options[b'deltabothparents'] = deltabothparents options[b'deltabothparents'] = deltabothparents
issue6528 = ui.configbool(b'storage', b'revlog.issue6528.fix-incoming')
options[b'issue6528.fix-incoming'] = issue6528
lazydelta = ui.configbool(b'storage', b'revlog.reuse-external-delta') lazydelta = ui.configbool(b'storage', b'revlog.reuse-external-delta')
lazydeltabase = False lazydeltabase = False
if lazydelta: if lazydelta:
lazydeltabase = ui.configbool( lazydeltabase = ui.configbool(
b'storage', b'revlog.reuse-external-delta-parent' b'storage', b'revlog.reuse-external-delta-parent'
) )
if lazydeltabase is None: if lazydeltabase is None:
lazydeltabase = not scmutil.gddeltaconfig(ui) lazydeltabase = not scmutil.gddeltaconfig(ui)

tests/test-issue6528.t

$ hg status $ hg status
And that the repair command does not find anything to fix And that the repair command does not find anything to fix
$ hg debug-repair-issue6528 $ hg debug-repair-issue6528
no affected revisions were found no affected revisions were found
$ cd .. $ cd ..
A config option can disable the fixing of the bad bundle on the fly
-------------------------------------------------------------------
from a v1 bundle
~~~~~~~~~~~~~~~~
$ hg debugbundle --spec "$TESTDIR"/bundles/issue6528.hg-v1
bzip2-v1
$ hg init unbundle-v1-no-fix
$ cd unbundle-v1-no-fix
$ hg unbundle "$TESTDIR"/bundles/issue6528.hg-v1 --config storage.revlog.issue6528.fix-incoming=no
adding changesets
adding manifests
adding file changes
added 8 changesets with 12 changes to 4 files
new changesets f5a5a568022f:3beabb508514 (8 drafts)
(run 'hg update' to get a working copy)
Check that revision were not fixed on the fly
$ hg debugrevlogindex b.txt
rev linkrev nodeid p1 p2
0 2 05b806ebe5ea 000000000000 000000000000
1 3 a58b36ad6b65 05b806ebe5ea 000000000000
2 6 216a5fe8b8ed 000000000000 000000000000
3 7 ea4f2f2463cc 216a5fe8b8ed 000000000000
$ hg debugrevlogindex D.txt
rev linkrev nodeid p1 p2
0 6 2a8d3833f2fb 000000000000 000000000000
1 7 2a80419dfc31 2a8d3833f2fb 000000000000
That we do see the symptoms of the bug
$ hg up -- -1
2 files updated, 0 files merged, 0 files removed, 0 files unresolved
$ hg status
M D.txt (?)
M b.txt (?)
And that the repair command find issue to fix.
$ hg debug-repair-issue6528 --dry-run
found affected revision 1 for filelog 'data/D.txt.i'
found affected revision 1 for filelog 'data/b.txt.i'
found affected revision 3 for filelog 'data/b.txt.i'
$ cd ..
from a v2 bundle
~~~~~~~~~~~~~~~~
$ hg debugbundle --spec "$TESTDIR"/bundles/issue6528.hg-v2
bzip2-v2
$ hg init unbundle-v2-no-fix
$ cd unbundle-v2-no-fix
$ hg unbundle "$TESTDIR"/bundles/issue6528.hg-v2 --config storage.revlog.issue6528.fix-incoming=no
adding changesets
adding manifests
adding file changes
added 8 changesets with 12 changes to 4 files
new changesets f5a5a568022f:3beabb508514 (8 drafts)
AlphareUnsubmitted
Not Done

Those comments say the opposite of what they're trying to

Alphare: Those comments say the opposite of what they're trying to
marmouteAuthorUnsubmitted
Done

The copy-paste devil strike again. Thanks for spotting this, update on the way.

marmoute: The copy-paste devil strike again. Thanks for spotting this, update on the way.
(run 'hg update' to get a working copy)
Check that revision were not fixed on the fly
$ hg debugrevlogindex b.txt
rev linkrev nodeid p1 p2
0 2 05b806ebe5ea 000000000000 000000000000
1 3 a58b36ad6b65 05b806ebe5ea 000000000000
2 6 216a5fe8b8ed 000000000000 000000000000
3 7 ea4f2f2463cc 216a5fe8b8ed 000000000000
$ hg debugrevlogindex D.txt
rev linkrev nodeid p1 p2
0 6 2a8d3833f2fb 000000000000 000000000000
1 7 2a80419dfc31 2a8d3833f2fb 000000000000
That we do see the symptoms of the bug
$ hg up -- -1
2 files updated, 0 files merged, 0 files removed, 0 files unresolved
$ hg status
M D.txt (?)
M b.txt (?)
And that the repair command find issue to fix.
$ hg debug-repair-issue6528 --dry-run
found affected revision 1 for filelog 'data/D.txt.i'
found affected revision 1 for filelog 'data/b.txt.i'
found affected revision 3 for filelog 'data/b.txt.i'
$ cd ..