This is an archive of the discontinued Mercurial Phabricator instance.

Differential D7124

copies: move from a copy on branchpoint to a copy on write approach
ClosedPublic

Authored by marmoute on Oct 16 2019, 7:48 PM.

Download Raw Diff

Details

Reviewers

None

Group Reviewers

hg-reviewers

Commits

rHGffd04bc9f57d: copies: move from a copy on branchpoint to a copy on write approach

Summary

Before this changes, any branch points results in a copy of the dictionary containing the
copy information. This can be very costly for branchy history with few rename
information. Instead, we take a "copy on write" approach. Copying the input data
only when we are about to update them.

In practice we where already doing the copying in half of these case (because
_chain makes a copy), so we don't add a significant cost here even in the
linear case. However the speed up in branchy case is very significant. Here are
some timing on the pypy repository.

revision: large amount; added files: large amount; rename small amount; c3b14617fbd7 9ba6ab77fd29
before: ! wall 1.399863 comb 1.400000 user 1.370000 sys 0.030000 (median of 10)
after: ! wall 0.766453 comb 0.770000 user 0.750000 sys 0.020000 (median of 11)
revision: large amount; added files: small amount; rename small amount; c3b14617fbd7 f650a9b140d2
before: ! wall 1.876748 comb 1.890000 user 1.870000 sys 0.020000 (median of 10)
after: ! wall 1.167223 comb 1.170000 user 1.150000 sys 0.020000 (median of 10)
revision: large amount; added files: large amount; rename large amount; 08ea3258278e d9fa043f30c0
before: ! wall 0.242457 comb 0.240000 user 0.240000 sys 0.000000 (median of 39)
after: ! wall 0.211476 comb 0.210000 user 0.210000 sys 0.000000 (median of 45)
revision: small amount; added files: large amount; rename large amount; df6f7a526b60 a83dc6a2d56f
before: ! wall 0.013193 comb 0.020000 user 0.020000 sys 0.000000 (median of 224)
after: ! wall 0.013290 comb 0.010000 user 0.010000 sys 0.000000 (median of 222)
revision: small amount; added files: large amount; rename small amount; 4aa4e1f8e19a 169138063d63
before: ! wall 0.001673 comb 0.000000 user 0.000000 sys 0.000000 (median of 1000)
after: ! wall 0.001677 comb 0.000000 user 0.000000 sys 0.000000 (median of 1000)
revision: small amount; added files: small amount; rename small amount; 4bc173b045a6 964879152e2e
before: ! wall 0.000119 comb 0.000000 user 0.000000 sys 0.000000 (median of 8023)
after: ! wall 0.000119 comb 0.000000 user 0.000000 sys 0.000000 (median of 7997)
revision: medium amount; added files: large amount; rename medium amount; c95f1ced15f2 2c68e87c3efe
before: ! wall 0.201898 comb 0.210000 user 0.200000 sys 0.010000 (median of 48)
after: ! wall 0.167415 comb 0.170000 user 0.160000 sys 0.010000 (median of 58)
revision: medium amount; added files: medium amount; rename small amount; d343da0c55a8 d7746d32bf9d
before: ! wall 0.036820 comb 0.040000 user 0.040000 sys 0.000000 (median of 100)
after: ! wall 0.035797 comb 0.040000 user 0.040000 sys 0.000000 (median of 100)

The extra cost in the linear case can be reclaimed later with some extra logic.

Diff Detail

Repository

rHG Mercurial

Lint

Lint Skipped

Unit

Unit Tests Skipped

Event Timeline

marmoute created this revision.Oct 16 2019, 7:48 PM

Herald added a reviewer: hg-reviewers. · View Herald TranscriptOct 16 2019, 7:48 PM

Herald added a subscriber: mercurial-devel. · View Herald Transcript

marmoute added a child revision: D7127: sidedatacopies: only fetch information once for merge.Oct 17 2019, 7:45 AM

marmoute updated this revision to Diff 17331.Oct 17 2019, 7:46 PM

marmoute added a commit: rHGffd04bc9f57d: copies: move from a copy on branchpoint to a copy on write approach.Oct 19 2019, 1:38 PM

This revision was not accepted when it landed; it landed in state Needs Review.

Closed by commit rHGffd04bc9f57d: copies: move from a copy on branchpoint to a copy on write approach (authored by marmoute). · Explain Why

This revision was automatically updated to reflect the committed changes.

Revision Contents
Changeset List

			Path	Packages
M			mercurial/copies.py (14 lines)

Diff	ID	Description	Created	Lint	Unit
Base		Base
Diff 1	17280		Oct 16 2019, 7:48 PM	★	★
Diff 2	17331		Oct 17 2019, 7:46 PM	★	★
Diff 3	17361	rHGffd04bc9f57df1b686aeb99b826950901a002dbf	Oct 15 2019, 12:23 PM	★	★

Commit	Parents	Author	Summary	Date
082a7f34f527	cb57c62bace9	Pierre-Yves David		Oct 15 2019, 12:23 PM

Status	Author	Revision
Abandoned	marmoute	D7071 copies: use unchecked parentrevs function to access parent
Closed	marmoute	D7131 copies: simplify chain loop
Closed	marmoute	D7127 sidedatacopies: only fetch information once for merge
Closed	marmoute	D7124 copies: move from a copy on branchpoint to a copy on write approach
Closed	marmoute	D7076 copies: compute the exact set of revision to walk
Closed	marmoute	D7075 copies: add an explicit test using multiple roots
Closed	marmoute	D7074 sidedatacopies: fast path data fetching if revision has no sidedata
Abandoned	marmoute	D7073 revlog: introduce a `_uncheckedflags` shortcut
Closed	marmoute	D7072 sidedatacopies: directly fetch copies information from sidedata
Closed	marmoute	D7123 copies: use an unfiltered repository for the changeset centric algorithm
Closed	marmoute	D7070 copies: extract data extraction into a `revinfo` function
Closed	marmoute	D7122 copies: avoid instancing more changectx to access parent revisions
Closed	marmoute	D7121 copies: get copies information directly from _copies

Diff 17331

mercurial/copies.py

	else:			else:
	assert r == p2			assert r == p2
	parent = 2			parent = 2
	childcopies = p2copies			childcopies = p2copies
	if not alwaysmatch:			if not alwaysmatch:
	childcopies = {			childcopies = {
	dst: src for dst, src in childcopies.items() if match(dst)			dst: src for dst, src in childcopies.items() if match(dst)
	}			}
	# Copy the dict only if later iterations will also need it
	if i != len(children[r]) - 1:
	newcopies = copies.copy()
	else:
	newcopies = copies			newcopies = copies
	if childcopies:			if childcopies:
	newcopies = _chain(newcopies, childcopies)			newcopies = _chain(newcopies, childcopies)
				# _chain makes a copies, we can avoid doing so in some
				# simple/linear cases.
				assert newcopies is not copies
	for f in removed:			for f in removed:
	if f in newcopies:			if f in newcopies:
				if newcopies is copies:
				# copy on write to avoid affecting potential other
				# branches. when there are no other branches, this
				# could be avoided.
				newcopies = copies.copy()
	del newcopies[f]			del newcopies[f]
	othercopies = all_copies.get(c)			othercopies = all_copies.get(c)
	if othercopies is None:			if othercopies is None:
	all_copies[c] = newcopies			all_copies[c] = newcopies
	else:			else:
	# we are the second parent to work on c, we need to merge our			# we are the second parent to work on c, we need to merge our
	# work with the other.			# work with the other.
	#			#