( )⚙ D10084 copies: tests and fix parallel computation of changed file information

This is an archive of the discontinued Mercurial Phabricator instance.

Differential D10084

copies: tests and fix parallel computation of changed file information
ClosedPublic

Authored by marmoute on Mar 1 2021, 2:11 PM.

Details

Reviewers
SimonSapin
pulkit
Group Reviewers
hg-reviewers
Commits
rHGdde86beca388: copies: tests and fix parallel computation of changed file information
Summary

The code was mixing variable name and misbehaving in some case. This changeset
fix it and also add a tests to validate it does not regress. Without the fix,
the parallel-upgrade misbehavior in random ways.

Diff Detail

Repository
rHG Mercurial
Branch
default
Lint
No Linters Available
Unit
No Unit Test Coverage

Revision Contents
Changeset List

PathPackages
Mmercurial/metadata.py (6 lines)
Mtests/test-copies-chain-merge.t (59 lines)

Diff 26007

mercurial/metadata.py

# looking for. For example, if we need the sidedatamap for 42, and 43 is # looking for. For example, if we need the sidedatamap for 42, and 43 is
# received, when shelve 43 for later use. # received, when shelve 43 for later use.
staging = {} staging = {}
def sidedata_companion(revlog, rev): def sidedata_companion(revlog, rev):
data = {}, False data = {}, False
if util.safehasattr(revlog, b'filteredrevs'): # this is a changelog if util.safehasattr(revlog, b'filteredrevs'): # this is a changelog
# Is the data previously shelved ? # Is the data previously shelved ?
sidedata = staging.pop(rev, None) data = staging.pop(rev, None)
if sidedata is None: if data is None:
# look at the queued result until we find the one we are lookig # look at the queued result until we find the one we are lookig
# for (shelve the other ones) # for (shelve the other ones)
r, data = sidedataq.get() r, data = sidedataq.get()
while r != rev: while r != rev:
staging[r] = data staging[r] = data
r, sidedata = sidedataq.get() r, data = sidedataq.get()
tokens.release() tokens.release()
sidedata, has_copies_info = data sidedata, has_copies_info = data
new_flag = 0 new_flag = 0
if has_copies_info: if has_copies_info:
new_flag = sidedataflag.REVIDX_HASCOPIESINFO new_flag = sidedataflag.REVIDX_HASCOPIESINFO
return False, (), sidedata, new_flag, 0 return False, (), sidedata, new_flag, 0
return sidedata_companion return sidedata_companion

tests/test-copies-chain-merge.t

#testcases filelog compatibility changeset sidedata upgraded #testcases filelog compatibility changeset sidedata upgraded upgraded-parallel
===================================================== =====================================================
Test Copy tracing for chain of copies involving merge Test Copy tracing for chain of copies involving merge
===================================================== =====================================================
This test files covers copies/rename case for a chains of commit where merges This test files covers copies/rename case for a chains of commit where merges
are involved. It cheks we do not have unwanted update of behavior and that the are involved. It cheks we do not have unwanted update of behavior and that the
different options to retrieve copies behave correctly. different options to retrieve copies behave correctly.
processed revlogs: processed revlogs:
- all-filelogs - all-filelogs
- changelog - changelog
- manifest - manifest
#endif #endif
#if upgraded-parallel
$ cat >> $HGRCPATH << EOF
> [format]
> exp-use-side-data = yes
> exp-use-copies-side-data-changeset = yes
> [experimental]
> worker.repository-upgrade=yes
> [worker]
> enabled=yes
> numcpus=8
> EOF
$ hg debugformat -v
format-variant repo config default
fncache: yes yes yes
dotencode: yes yes yes
generaldelta: yes yes yes
share-safe: no no no
sparserevlog: yes yes yes
sidedata: no yes no
persistent-nodemap: no no no
copies-sdc: no yes no
plain-cl-delta: yes yes yes
compression: * (glob)
compression-level: default default default
$ hg debugupgraderepo --run --quiet
upgrade will perform the following actions:
requirements
preserved: * (glob)
added: exp-copies-sidedata-changeset, exp-sidedata-flag
processed revlogs:
- all-filelogs
- changelog
- manifest
#endif
#if no-compatibility no-filelog no-changeset #if no-compatibility no-filelog no-changeset
$ hg debugchangedfiles --compute 0 $ hg debugchangedfiles --compute 0
added : a, ; added : a, ;
added : b, ; added : b, ;
added : h, ; added : h, ;
added : p, ; added : p, ;
The result from mAEm is the same for the subsequent merge: The result from mAEm is the same for the subsequent merge:
$ hg status --copies --rev 'desc("i-0")' --rev 'desc("mAEm")' f $ hg status --copies --rev 'desc("i-0")' --rev 'desc("mAEm")' f
A f A f
a (filelog !) a (filelog !)
a (sidedata !) a (sidedata !)
a (upgraded !) a (upgraded !)
a (upgraded-parallel !)
$ hg status --copies --rev 'desc("i-0")' --rev 'desc("mAE,Km")' f $ hg status --copies --rev 'desc("i-0")' --rev 'desc("mAE,Km")' f
A f A f
a (filelog !) a (filelog !)
a (sidedata !) a (sidedata !)
a (upgraded !) a (upgraded !)
a (upgraded-parallel !)
$ hg status --copies --rev 'desc("i-0")' --rev 'desc("mK,AEm")' f $ hg status --copies --rev 'desc("i-0")' --rev 'desc("mK,AEm")' f
A f A f
a (filelog !) a (filelog !)
a (sidedata !) a (sidedata !)
a (upgraded !) a (upgraded !)
a (upgraded-parallel !)
The result from mEAm is the same for the subsequent merge: The result from mEAm is the same for the subsequent merge:
$ hg status --copies --rev 'desc("i-0")' --rev 'desc("mEAm")' f $ hg status --copies --rev 'desc("i-0")' --rev 'desc("mEAm")' f
A f A f
a (filelog !) a (filelog !)
b (sidedata !) b (sidedata !)
b (upgraded !) b (upgraded !)
b (upgraded-parallel !)
$ hg status --copies --rev 'desc("i-0")' --rev 'desc("mEA,Jm")' f $ hg status --copies --rev 'desc("i-0")' --rev 'desc("mEA,Jm")' f
A f A f
a (filelog !) a (filelog !)
b (sidedata !) b (sidedata !)
b (upgraded !) b (upgraded !)
b (upgraded-parallel !)
$ hg status --copies --rev 'desc("i-0")' --rev 'desc("mJ,EAm")' f $ hg status --copies --rev 'desc("i-0")' --rev 'desc("mJ,EAm")' f
A f A f
a (filelog !) a (filelog !)
b (sidedata !) b (sidedata !)
b (upgraded !) b (upgraded !)
b (upgraded-parallel !)
Subcase: chaining conflicting rename resolution Subcase: chaining conflicting rename resolution
``````````````````````````````````````````````` ```````````````````````````````````````````````
The "mPQm" and "mQPm" case create a rename tracking conflict on file 'v'. We The "mPQm" and "mQPm" case create a rename tracking conflict on file 'v'. We
add more change on the respective branch and merge again. These second merge add more change on the respective branch and merge again. These second merge
does not involve the file 'v' and the arbitration done within "mPQm" and "mQP" does not involve the file 'v' and the arbitration done within "mPQm" and "mQP"
about that file should stay unchanged. about that file should stay unchanged.
The result from mPQm is the same for the subsequent merge: The result from mPQm is the same for the subsequent merge:
$ hg status --copies --rev 'desc("i-0")' --rev 'desc("mPQm")' v $ hg status --copies --rev 'desc("i-0")' --rev 'desc("mPQm")' v
A v A v
r (filelog !) r (filelog !)
p (sidedata !) p (sidedata !)
p (upgraded !) p (upgraded !)
p (upgraded-parallel !)
$ hg status --copies --rev 'desc("i-0")' --rev 'desc("mPQ,Tm")' v $ hg status --copies --rev 'desc("i-0")' --rev 'desc("mPQ,Tm")' v
A v A v
r (filelog !) r (filelog !)
p (sidedata !) p (sidedata !)
p (upgraded !) p (upgraded !)
p (upgraded-parallel !)
$ hg status --copies --rev 'desc("i-0")' --rev 'desc("mT,PQm")' v $ hg status --copies --rev 'desc("i-0")' --rev 'desc("mT,PQm")' v
A v A v
r (filelog !) r (filelog !)
p (sidedata !) p (sidedata !)
p (upgraded !) p (upgraded !)
p (upgraded-parallel !)
The result from mQPm is the same for the subsequent merge: The result from mQPm is the same for the subsequent merge:
$ hg status --copies --rev 'desc("i-0")' --rev 'desc("mQPm")' v $ hg status --copies --rev 'desc("i-0")' --rev 'desc("mQPm")' v
A v A v
r (no-changeset no-compatibility !) r (no-changeset no-compatibility !)
$ hg status --copies --rev 'desc("i-0")' --rev 'desc("mQP,Sm")' v $ hg status --copies --rev 'desc("i-0")' --rev 'desc("mQP,Sm")' v
A v A v
r (no-changeset no-compatibility !) r (no-changeset no-compatibility !)
$ hg status --copies --rev 'desc("i-0")' --rev 'desc("mS,QPm")' v $ hg status --copies --rev 'desc("i-0")' --rev 'desc("mS,QPm")' v
A v A v
r (filelog !) r (filelog !)
r (sidedata !) r (sidedata !)
r (upgraded !) r (upgraded !)
r (upgraded-parallel !)
Subcase: chaining salvage information during a merge Subcase: chaining salvage information during a merge
```````````````````````````````````````````````````` ````````````````````````````````````````````````````
We add more change on the branch were the file was deleted. merging again We add more change on the branch were the file was deleted. merging again
should preserve the fact eh file was salvaged. should preserve the fact eh file was salvaged.
(for details about the filelog pick, check the mFGm/mGFm case) (for details about the filelog pick, check the mFGm/mGFm case)
$ hg status --copies --rev 'desc("i-0")' --rev 'desc("mFGm")' d $ hg status --copies --rev 'desc("i-0")' --rev 'desc("mFGm")' d
A d A d
a (filelog !) a (filelog !)
h (sidedata !) h (sidedata !)
h (upgraded !) h (upgraded !)
h (upgraded-parallel !)
$ hg status --copies --rev 'desc("i-0")' --rev 'desc("mGFm")' d $ hg status --copies --rev 'desc("i-0")' --rev 'desc("mGFm")' d
A d A d
a (filelog !) a (filelog !)
a (sidedata !) a (sidedata !)
a (upgraded !) a (upgraded !)
a (upgraded-parallel !)
Chained output Chained output
$ hg status --copies --rev 'desc("i-0")' --rev 'desc("mO,FGm")' d $ hg status --copies --rev 'desc("i-0")' --rev 'desc("mO,FGm")' d
A d A d
a (filelog !) a (filelog !)
h (sidedata !) h (sidedata !)
h (upgraded !) h (upgraded !)
h (upgraded-parallel !)
$ hg status --copies --rev 'desc("i-0")' --rev 'desc("mFG,Om")' d $ hg status --copies --rev 'desc("i-0")' --rev 'desc("mFG,Om")' d
A d A d
a (filelog !) a (filelog !)
h (sidedata !) h (sidedata !)
h (upgraded !) h (upgraded !)
h (upgraded-parallel !)
$ hg status --copies --rev 'desc("i-0")' --rev 'desc("mGF,Nm")' d $ hg status --copies --rev 'desc("i-0")' --rev 'desc("mGF,Nm")' d
A d A d
a (no-changeset no-compatibility !) a (no-changeset no-compatibility !)
$ hg status --copies --rev 'desc("i-0")' --rev 'desc("mN,GFm")' d $ hg status --copies --rev 'desc("i-0")' --rev 'desc("mN,GFm")' d
A d A d
a (no-changeset no-compatibility !) a (no-changeset no-compatibility !)
The result from mAEm is the same for the subsequent merge: The result from mAEm is the same for the subsequent merge:
$ hg status --copies --rev 'desc("i-0")' --rev 'desc("mAE-change-m")' f $ hg status --copies --rev 'desc("i-0")' --rev 'desc("mAE-change-m")' f
A f A f
a (filelog !) a (filelog !)
a (sidedata !) a (sidedata !)
a (upgraded !) a (upgraded !)
a (upgraded-parallel !)
$ hg status --copies --rev 'desc("i-0")' --rev 'desc("mAE-change,Km")' f $ hg status --copies --rev 'desc("i-0")' --rev 'desc("mAE-change,Km")' f
A f A f
a (filelog !) a (filelog !)
a (sidedata !) a (sidedata !)
a (upgraded !) a (upgraded !)
a (upgraded-parallel !)
$ hg status --copies --rev 'desc("i-0")' --rev 'desc("mK,AE-change-m")' f $ hg status --copies --rev 'desc("i-0")' --rev 'desc("mK,AE-change-m")' f
A f A f
a (no-changeset no-compatibility !) a (no-changeset no-compatibility !)
The result from mEAm is the same for the subsequent merge: The result from mEAm is the same for the subsequent merge:
$ hg status --copies --rev 'desc("i-0")' --rev 'desc("mEA-change-m")' f $ hg status --copies --rev 'desc("i-0")' --rev 'desc("mEA-change-m")' f
A f A f
a (filelog !) a (filelog !)
b (sidedata !) b (sidedata !)
b (upgraded !) b (upgraded !)
b (upgraded-parallel !)
$ hg status --copies --rev 'desc("i-0")' --rev 'desc("mEA-change,Jm")' f $ hg status --copies --rev 'desc("i-0")' --rev 'desc("mEA-change,Jm")' f
A f A f
a (filelog !) a (filelog !)
b (sidedata !) b (sidedata !)
b (upgraded !) b (upgraded !)
b (upgraded-parallel !)
$ hg status --copies --rev 'desc("i-0")' --rev 'desc("mJ,EA-change-m")' f $ hg status --copies --rev 'desc("i-0")' --rev 'desc("mJ,EA-change-m")' f
A f A f
a (filelog !) a (filelog !)
b (sidedata !) b (sidedata !)
b (upgraded !) b (upgraded !)
b (upgraded-parallel !)