This is an archive of the discontinued Mercurial Phabricator instance.

Differential D9487

upgrade: start moving the "to be happening" data in a dedicated object
ClosedPublic

Authored by marmoute on Dec 1 2020, 5:12 PM.

Details

Reviewers
pulkit
Group Reviewers
hg-reviewers
Commits
rHGc407513a44a3: upgrade: start moving the "to be happening" data in a dedicated object
rHG74923cb88537: upgrade: start moving the "to be happening" data in a dedicated object
Summary

The upgrade code has a lot of logic to determine which action needs to be
performed depending of various element (sometimes depending from each other). It
would be nice to have a consistent object representing this. That could be
cleanly passed and avoid some logic duplication.

So we create this object as a start.

Diff Detail

Repository
rHG Mercurial
Lint
Automatic diff as part of commit; lint not applicable.
Unit
Automatic diff as part of commit; unit tests not applicable.

Event Timeline

marmoute created this revision.Dec 1 2020, 5:12 PM
baymax updated this revision to Diff 24038.Dec 4 2020, 6:48 PM
pulkit requested changes to this revision.Dec 6 2020, 6:27 AM
pulkit added a subscriber: pulkit.
pulkit added inline comments.
mercurial/upgrade.py
184

Seems like incomplete statement.

This revision now requires changes to proceed.Dec 6 2020, 6:27 AM
baymax updated this revision to Diff 24071.Dec 6 2020, 10:09 AM

✅ refresh by Heptapod after a successful CI run (🐙 💚)

pulkit accepted this revision.Dec 7 2020, 3:48 AM
This revision is now accepted and ready to land.Dec 7 2020, 3:48 AM

Revision Contents
Changeset List

Diff 24074

mercurial/upgrade.py

if not revlogs: if not revlogs:
ui.write((b'no revlogs to process\n')) ui.write((b'no revlogs to process\n'))
else: else:
ui.write((b'processed revlogs:\n')) ui.write((b'processed revlogs:\n'))
for r in sorted(revlogs): for r in sorted(revlogs):
ui.write((b' - %s\n' % r)) ui.write((b' - %s\n' % r))
ui.write((b'\n')) ui.write((b'\n'))
upgrade_op = upgrade_actions.UpgradeOperation(
newreqs,
[a.name for a in actions],
revlogs,
)
pulkitUnsubmitted
Not Done

Seems like incomplete statement.

pulkit: Seems like incomplete statement.
if not run: if not run:
fromconfig = [] fromconfig = []
onlydefault = [] onlydefault = []
for d in deficiencies: for d in deficiencies:
if d.fromconfig(repo): if d.fromconfig(repo):
fromconfig.append(d) fromconfig.append(d)
elif d.default: elif d.default:
# Else we're in the run=true case. # Else we're in the run=true case.
ui.write(_(b'upgrade will perform the following actions:\n\n')) ui.write(_(b'upgrade will perform the following actions:\n\n'))
printrequirements() printrequirements()
printoptimisations() printoptimisations()
printupgradeactions() printupgradeactions()
print_affected_revlogs() print_affected_revlogs()
upgradeactions = [a.name for a in actions]
ui.status(_(b'beginning upgrade...\n')) ui.status(_(b'beginning upgrade...\n'))
with repo.wlock(), repo.lock(): with repo.wlock(), repo.lock():
ui.status(_(b'repository locked and read-only\n')) ui.status(_(b'repository locked and read-only\n'))
# Our strategy for upgrading the repository is to create a new, # Our strategy for upgrading the repository is to create a new,
# temporary repository, write data to it, then do a swap of the # temporary repository, write data to it, then do a swap of the
# data. There are less heavyweight ways to do this, but it is easier # data. There are less heavyweight ways to do this, but it is easier
# to create a new repo object than to instantiate all the components # to create a new repo object than to instantiate all the components
# (like the store) separately. # (like the store) separately.
) )
# clone ui without using ui.copy because repo.ui is protected # clone ui without using ui.copy because repo.ui is protected
repoui = repo.ui.__class__(repo.ui) repoui = repo.ui.__class__(repo.ui)
dstrepo = hg.repository(repoui, path=tmppath, create=True) dstrepo = hg.repository(repoui, path=tmppath, create=True)
with dstrepo.wlock(), dstrepo.lock(): with dstrepo.wlock(), dstrepo.lock():
backuppath = upgrade_engine.upgrade( backuppath = upgrade_engine.upgrade(
ui, repo, dstrepo, newreqs, upgradeactions, revlogs=revlogs ui, repo, dstrepo, upgrade_op
) )
if not (backup or backuppath is None): if not (backup or backuppath is None):
ui.status( ui.status(
_(b'removing old repository content%s\n') % backuppath _(b'removing old repository content%s\n') % backuppath
) )
repo.vfs.rmtree(backuppath, forcibly=True) repo.vfs.rmtree(backuppath, forcibly=True)
backuppath = None backuppath = None

mercurial/upgrade_utils/actions.py

newactions.append(d) newactions.append(d)
# FUTURE consider adding some optimizations here for certain transitions. # FUTURE consider adding some optimizations here for certain transitions.
# e.g. adding generaldelta could schedule parent redeltas. # e.g. adding generaldelta could schedule parent redeltas.
return newactions return newactions
class UpgradeOperation(object):
"""represent the work to be done during an upgrade"""
def __init__(self, requirements, actions, revlogs_to_process):
self.requirements = requirements
self.actions = actions
self.revlogs_to_process = revlogs_to_process
### Code checking if a repository can got through the upgrade process at all. # ### Code checking if a repository can got through the upgrade process at all. #
def requiredsourcerequirements(repo): def requiredsourcerequirements(repo):
"""Obtain requirements required to be present to upgrade a repo. """Obtain requirements required to be present to upgrade a repo.
An upgrade will not be allowed if the repository doesn't have the An upgrade will not be allowed if the repository doesn't have the
requirements returned by this function. requirements returned by this function.

mercurial/upgrade_utils/engine.py

def _finishdatamigration(ui, srcrepo, dstrepo, requirements): def _finishdatamigration(ui, srcrepo, dstrepo, requirements):
"""Hook point for extensions to perform additional actions during upgrade. """Hook point for extensions to perform additional actions during upgrade.
This function is called after revlogs and store files have been copied but This function is called after revlogs and store files have been copied but
before the new store is swapped into the original location. before the new store is swapped into the original location.
""" """
def upgrade( def upgrade(ui, srcrepo, dstrepo, upgrade_op):
ui, srcrepo, dstrepo, requirements, actions, revlogs=UPGRADE_ALL_REVLOGS
):
"""Do the low-level work of upgrading a repository. """Do the low-level work of upgrading a repository.
The upgrade is effectively performed as a copy between a source The upgrade is effectively performed as a copy between a source
repository and a temporary destination repository. repository and a temporary destination repository.
The source repository is unmodified for as long as possible so the The source repository is unmodified for as long as possible so the
upgrade can abort at any time without causing loss of service for upgrade can abort at any time without causing loss of service for
readers and without corrupting the source repository. readers and without corrupting the source repository.
""" """
assert srcrepo.currentwlock() assert srcrepo.currentwlock()
assert dstrepo.currentwlock() assert dstrepo.currentwlock()
ui.status( ui.status(
_( _(
b'(it is safe to interrupt this process any time before ' b'(it is safe to interrupt this process any time before '
b'data migration completes)\n' b'data migration completes)\n'
) )
) )
if b're-delta-all' in actions: if b're-delta-all' in upgrade_op.actions:
deltareuse = revlog.revlog.DELTAREUSENEVER deltareuse = revlog.revlog.DELTAREUSENEVER
elif b're-delta-parent' in actions: elif b're-delta-parent' in upgrade_op.actions:
deltareuse = revlog.revlog.DELTAREUSESAMEREVS deltareuse = revlog.revlog.DELTAREUSESAMEREVS
elif b're-delta-multibase' in actions: elif b're-delta-multibase' in upgrade_op.actions:
deltareuse = revlog.revlog.DELTAREUSESAMEREVS deltareuse = revlog.revlog.DELTAREUSESAMEREVS
elif b're-delta-fulladd' in actions: elif b're-delta-fulladd' in upgrade_op.actions:
deltareuse = revlog.revlog.DELTAREUSEFULLADD deltareuse = revlog.revlog.DELTAREUSEFULLADD
else: else:
deltareuse = revlog.revlog.DELTAREUSEALWAYS deltareuse = revlog.revlog.DELTAREUSEALWAYS
with dstrepo.transaction(b'upgrade') as tr: with dstrepo.transaction(b'upgrade') as tr:
_clonerevlogs( _clonerevlogs(
ui, ui,
srcrepo, srcrepo,
dstrepo, dstrepo,
tr, tr,
deltareuse, deltareuse,
b're-delta-multibase' in actions, b're-delta-multibase' in upgrade_op.actions,
revlogs=revlogs, revlogs=upgrade_op.revlogs_to_process,
) )
# Now copy other files in the store directory. # Now copy other files in the store directory.
# The sorted() makes execution deterministic. # The sorted() makes execution deterministic.
for p, kind, st in sorted(srcrepo.store.vfs.readdir(b'', stat=True)): for p, kind, st in sorted(srcrepo.store.vfs.readdir(b'', stat=True)):
if not _filterstorefile(srcrepo, dstrepo, requirements, p, kind, st): if not _filterstorefile(
srcrepo, dstrepo, upgrade_op.requirements, p, kind, st
):
continue continue
srcrepo.ui.status(_(b'copying %s\n') % p) srcrepo.ui.status(_(b'copying %s\n') % p)
src = srcrepo.store.rawvfs.join(p) src = srcrepo.store.rawvfs.join(p)
dst = dstrepo.store.rawvfs.join(p) dst = dstrepo.store.rawvfs.join(p)
util.copyfile(src, dst, copystat=True) util.copyfile(src, dst, copystat=True)
_finishdatamigration(ui, srcrepo, dstrepo, requirements) _finishdatamigration(ui, srcrepo, dstrepo, requirements)
# We first write the requirements file. Any new requirements will lock # We first write the requirements file. Any new requirements will lock
# out legacy clients. # out legacy clients.
ui.status( ui.status(
_( _(
b'finalizing requirements file and making repository readable ' b'finalizing requirements file and making repository readable '
b'again\n' b'again\n'
) )
) )
scmutil.writereporequirements(srcrepo, requirements) scmutil.writereporequirements(srcrepo, upgrade_op.requirements)
# The lock file from the old store won't be removed because nothing has a # The lock file from the old store won't be removed because nothing has a
# reference to its new location. So clean it up manually. Alternatively, we # reference to its new location. So clean it up manually. Alternatively, we
# could update srcrepo.svfs and other variables to point to the new # could update srcrepo.svfs and other variables to point to the new
# location. This is simpler. # location. This is simpler.
backupvfs.unlink(b'store/lock') backupvfs.unlink(b'store/lock')
return backuppath return backuppath