This is an archive of the discontinued Mercurial Phabricator instance.

revlog: fix index computation during inline->non-inline transition
ClosedPublic

Authored by joerg.sonnenberger on May 17 2021, 8:41 PM.

Download Raw Diff

Details

Reviewers

indygreg
valentin.gatienbaron

Group Reviewers

hg-reviewers

Commits

rHG21ed126bab53: revlog: fix index computation during inline->non-inline transition
rHGf4117223239b: revlog: fix index computation during inline->non-inline transition
rHG762afde8c3a0: revlog: fix index computation during inline->non-inline transition

Summary

The computation in 63edc384d3b7 failed to factor in the index entries
themselve as revlog.start() doesn't count them. Found by Valtenin
Gatienbaron with a more precise test case from me.

Diff Detail

Repository

rHG Mercurial

Lint

Automatic diff as part of commit; lint not applicable.

Unit

Automatic diff as part of commit; unit tests not applicable.

Event Timeline

joerg.sonnenberger created this revision.May 17 2021, 8:41 PM

Herald added a reviewer: indygreg. · View Herald TranscriptMay 17 2021, 8:41 PM

Herald added a reviewer: hg-reviewers. · View Herald Transcript

Herald added a subscriber: mercurial-patches. · View Herald Transcript

joerg.sonnenberger updated this revision to Diff 28063.May 17 2021, 9:08 PM

joerg.sonnenberger edited the summary of this revision. (Show Details)May 18 2021, 2:48 PM

joerg.sonnenberger added a child revision: D10725: revlog: update data file record before index rename.

joerg.sonnenberger updated this revision to Diff 28079.

I knew zstd could be turned off, but I didn't knew there was an option to turn off even zlib. Cool.

joerg.sonnenberger added a commit: rHG762afde8c3a0: revlog: fix index computation during inline->non-inline transition.May 19 2021, 1:57 PM

This revision was not accepted when it landed; it landed in state Needs Review.

Closed by commit rHG762afde8c3a0: revlog: fix index computation during inline->non-inline transition (authored by joerg.sonnenberger). · Explain Why

This revision was automatically updated to reflect the committed changes.

FYI, this change breaks the CI since the error code is not correct. I'll amend it.

https://foss.heptapod.net/octobus/mercurial-devel/-/jobs/198659

joerg.sonnenberger added a commit: rHGf4117223239b: revlog: fix index computation during inline->non-inline transition.May 20 2021, 3:51 AM

joerg.sonnenberger added a commit: rHG21ed126bab53: revlog: fix index computation during inline->non-inline transition.May 20 2021, 4:36 AM

Revision Contents
Changeset List

		Path
M		mercurial/revlog.py (2 lines)
A	M	tests/helper-killhook.py (5 lines)
A	M	tests/test-transaction-rollback-on-revlog-split.t (29 lines)

Diff	ID	Description	Created	Lint	Unit
Base		Base
Diff 1	28062		May 17 2021, 8:41 PM	★	★
Diff 2	28063		May 17 2021, 9:08 PM	★	★
Diff 3	28079		May 18 2021, 2:48 PM	★	★
Diff 4	28124	rHG762afde8c3a051d80340da5c727818b1961738a6	May 17 2021, 8:35 PM	★	★

Status	Author	Revision
Closed	joerg.sonnenberger	D10726 recover: only apply last journal record per file (issue6423)
Closed	joerg.sonnenberger	D10725 revlog: update data file record before index rename
Closed	joerg.sonnenberger	D10724 revlog: fix index computation during inline->non-inline transition

Diff 28124

mercurial/revlog.py

	self._writinghandles = None			self._writinghandles = None

	new_dfh = self._datafp(b'w+')			new_dfh = self._datafp(b'w+')
	new_dfh.truncate(0) # drop any potentially existing data			new_dfh.truncate(0) # drop any potentially existing data
	try:			try:
	with self._indexfp() as read_ifh:			with self._indexfp() as read_ifh:
	for r in self:			for r in self:
	new_dfh.write(self._getsegmentforrevs(r, r, df=read_ifh)[1])			new_dfh.write(self._getsegmentforrevs(r, r, df=read_ifh)[1])
	if troffset <= self.start(r):			if troffset <= self.start(r) + r * self.index.entry_size:
	trindex = r			trindex = r
	new_dfh.flush()			new_dfh.flush()

	with self.__index_new_fp() as fp:			with self.__index_new_fp() as fp:
	self._format_flags &= ~FLAG_INLINE_DATA			self._format_flags &= ~FLAG_INLINE_DATA
	self._inline = False			self._inline = False
	for i in self:			for i in self:
	e = self.index.entry_binary(i)			e = self.index.entry_binary(i)

tests/helper-killhook.py

This file was added.

				import os


				def killme(ui, repo, hooktype, **wkargs):
				os._exit(80)

tests/test-transaction-rollback-on-revlog-split.t

This file was added.

				Test correctness of revlog inline -> non-inline transition
				----------------------------------------------------------

				Test offset computation to correctly factor in the index entries themselve.
				Test repo has one small, one moderate and one big change. The clone has
				the small and moderate change and will transition to non-inline storage when
				adding the big change.

				$ hg init troffset-computation --config format.revlog-compression=none
				$ cd troffset-computation
				$ printf '% 20d' '1' > file
				$ hg commit -Aqm_
				$ printf '% 1024d' '1' > file
				$ hg commit -Aqm_
				$ dd if=/dev/zero of=file bs=1k count=128 > /dev/null 2>&1
				$ hg commit -Aqm_
				$ cd ..

				$ hg clone -r 1 troffset-computation troffset-computation-copy --config format.revlog-compression=none -q
				$ cd troffset-computation-copy
				$ cat > .hg/hgrc <<EOF
				> [hooks]
				> pretxnchangegroup = python:$TESTDIR/helper-killhook.py:killme
				> EOF
				$ hg pull ../troffset-computation
				pulling from ../troffset-computation
				[80]
				$ cat .hg/store/journal \| tr -s '\000' ' ' \| grep data/file \| tail -1
				data/file.i 128