This is an archive of the discontinued Mercurial Phabricator instance.

Differential D7893

nodemap: update the index with the newly written data (when appropriate)
ClosedPublic

Authored by marmoute on Jan 15 2020, 9:56 AM.

Details

Reviewers
None
Group Reviewers
hg-reviewers
Commits
rHG6ecc34b31137: nodemap: update the index with the newly written data (when appropriate)
Summary

If we are to use mmap to read the nodemap data, and if the python code is
responsible for the IO, we need to refresh the mmap after each write and provide
it back to the index.

We start this dance without the mmap first.

Diff Detail

Repository
rHG Mercurial
Lint
Automatic diff as part of commit; lint not applicable.
Unit
Automatic diff as part of commit; unit tests not applicable.

Event Timeline

marmoute created this revision.Jan 15 2020, 9:56 AM
marmoute updated this revision to Diff 19774.Jan 31 2020, 10:30 AM
marmoute updated this revision to Diff 19801.Jan 31 2020, 6:01 PM
marmoute updated this revision to Diff 19845.Feb 2 2020, 3:20 AM

small doc update on .#s[1]

marmoute updated this revision to Diff 19905.Feb 4 2020, 7:26 PM

rebase to latest default

marmoute added a commit: rHG6ecc34b31137: nodemap: update the index with the newly written data (when appropriate).Feb 10 2020, 8:28 PM
This revision was not accepted when it landed; it landed in state Needs Review.
Closed by commit rHG6ecc34b31137: nodemap: update the index with the newly written data (when appropriate) (authored by marmoute). · Explain Why
This revision was automatically updated to reflect the committed changes.

Revision Contents
Changeset List

PathPackages
Mmercurial/revlogutils/nodemap.py (6 lines)

Diff 20126

mercurial/revlogutils/nodemap.py

data = None data = None
else: else:
datafile = _rawdata_filepath(revlog, target_docket) datafile = _rawdata_filepath(revlog, target_docket)
# EXP-TODO: if this is a cache, this should use a cache vfs, not a # EXP-TODO: if this is a cache, this should use a cache vfs, not a
# store vfs # store vfs
with revlog.opener(datafile, b'r+') as fd: with revlog.opener(datafile, b'r+') as fd:
fd.seek(target_docket.data_length) fd.seek(target_docket.data_length)
fd.write(data) fd.write(data)
fd.seek(0)
new_data = fd.read(target_docket.data_length + len(data))
target_docket.data_length += len(data) target_docket.data_length += len(data)
target_docket.data_unused += data_changed_count target_docket.data_unused += data_changed_count
if data is None: if data is None:
# otherwise fallback to a full new export # otherwise fallback to a full new export
target_docket = NodeMapDocket() target_docket = NodeMapDocket()
datafile = _rawdata_filepath(revlog, target_docket) datafile = _rawdata_filepath(revlog, target_docket)
if util.safehasattr(revlog.index, "nodemap_data_all"): if util.safehasattr(revlog.index, "nodemap_data_all"):
data = revlog.index.nodemap_data_all() data = revlog.index.nodemap_data_all()
else: else:
data = persistent_data(revlog.index) data = persistent_data(revlog.index)
# EXP-TODO: if this is a cache, this should use a cache vfs, not a # EXP-TODO: if this is a cache, this should use a cache vfs, not a
# store vfs # store vfs
new_data = data
with revlog.opener(datafile, b'w') as fd: with revlog.opener(datafile, b'w') as fd:
fd.write(data) fd.write(data)
target_docket.data_length = len(data) target_docket.data_length = len(data)
target_docket.tip_rev = revlog.tiprev() target_docket.tip_rev = revlog.tiprev()
# EXP-TODO: if this is a cache, this should use a cache vfs, not a # EXP-TODO: if this is a cache, this should use a cache vfs, not a
# store vfs # store vfs
with revlog.opener(revlog.nodemap_file, b'w', atomictemp=True) as fp: with revlog.opener(revlog.nodemap_file, b'w', atomictemp=True) as fp:
fp.write(target_docket.serialize()) fp.write(target_docket.serialize())
revlog._nodemap_docket = target_docket revlog._nodemap_docket = target_docket
if util.safehasattr(revlog.index, "update_nodemap_data"):
revlog.index.update_nodemap_data(target_docket, new_data)
# EXP-TODO: if the transaction abort, we should remove the new data and # EXP-TODO: if the transaction abort, we should remove the new data and
# reinstall the old one. # reinstall the old one.
# search for old index file in all cases, some older process might have # search for old index file in all cases, some older process might have
# left one behind. # left one behind.
olds = _other_rawdata_filepath(revlog, target_docket) olds = _other_rawdata_filepath(revlog, target_docket)
if olds: if olds:
realvfs = getattr(revlog, '_realopener', revlog.opener) realvfs = getattr(revlog, '_realopener', revlog.opener)