This is an archive of the discontinued Mercurial Phabricator instance.

Differential D1389

bundle2: only seek to beginning of part in bundlerepo
ClosedPublic

Authored by indygreg on Nov 13 2017, 11:59 PM.

Details

Reviewers
durin42
Group Reviewers
hg-reviewers
Commits
rHG2b72bc88043f: bundle2: only seek to beginning of part in bundlerepo
Summary

For reasons still not yet fully understood by me, bundlerepo
requires its changegroup bundle2 part to be seeked to beginning
after part iteration. As far as I can tell, it is the only
bundle2 part consumer that relies on this behavior.

This seeking was performed in the generic iterparts() API. Again,
I don't fully understand why it was here and not in bundlerepo.
Probably historical reasons.

What I do know is that all other bundle2 part consumers don't
need this special behavior (assuming the tests are comprehensive).
So, we move the code from bundle2's iterparts() to bundlerepo's
consumption of iterparts().

Diff Detail

Repository
rHG Mercurial
Lint
Lint Skipped
Unit
Unit Tests Skipped

Event Timeline

indygreg created this revision.Nov 13 2017, 11:59 PM
durin42 accepted this revision.Nov 20 2017, 6:40 PM
This revision is now accepted and ready to land.Nov 20 2017, 6:40 PM
Closed by commit rHG2b72bc88043f: bundle2: only seek to beginning of part in bundlerepo (authored by indygreg). · Explain WhyNov 20 2017, 6:49 PM
This revision was automatically updated to reflect the committed changes.

Revision Contents
Changeset List

PathPackages
Mmercurial/bundle2.py (4 lines)
Mmercurial/bundlerepo.py (16 lines)

Diff 3467

mercurial/bundle2.py

indebug(self.ui, 'start extraction of bundle2 parts') indebug(self.ui, 'start extraction of bundle2 parts')
headerblock = self._readpartheader() headerblock = self._readpartheader()
while headerblock is not None: while headerblock is not None:
part = seekableunbundlepart(self.ui, headerblock, self._fp) part = seekableunbundlepart(self.ui, headerblock, self._fp)
yield part yield part
# Ensure part is fully consumed so we can start reading the next # Ensure part is fully consumed so we can start reading the next
# part. # part.
part.consume() part.consume()
# But then seek back to the beginning so the code consuming this
# generator has a part that starts at 0.
part.seek(0, os.SEEK_SET)
headerblock = self._readpartheader() headerblock = self._readpartheader()
indebug(self.ui, 'end of bundle2 stream') indebug(self.ui, 'end of bundle2 stream')
def _readpartheader(self): def _readpartheader(self):
"""reads a part header size and return the bytes blob """reads a part header size and return the bytes blob
returns None if empty""" returns None if empty"""
headersize = self._unpack(_fpartheadersize)[0] headersize = self._unpack(_fpartheadersize)[0]

mercurial/bundlerepo.py

self.tempfile = None self.tempfile = None
f = util.posixfile(bundlepath, "rb") f = util.posixfile(bundlepath, "rb")
bundle = exchange.readbundle(ui, f, bundlepath) bundle = exchange.readbundle(ui, f, bundlepath)
if isinstance(bundle, bundle2.unbundle20): if isinstance(bundle, bundle2.unbundle20):
self._bundlefile = bundle self._bundlefile = bundle
self._cgunpacker = None self._cgunpacker = None
hadchangegroup = False cgpart = None
for part in bundle.iterparts(): for part in bundle.iterparts():
if part.type == 'changegroup': if part.type == 'changegroup':
if hadchangegroup: if cgpart:
raise NotImplementedError("can't process " raise NotImplementedError("can't process "
"multiple changegroups") "multiple changegroups")
hadchangegroup = True cgpart = part
self._handlebundle2part(bundle, part) self._handlebundle2part(bundle, part)
if not hadchangegroup: if not cgpart:
raise error.Abort(_("No changegroups found")) raise error.Abort(_("No changegroups found"))
# This is required to placate a later consumer, which expects
# the payload offset to be at the beginning of the changegroup.
# We need to do this after the iterparts() generator advances
# because iterparts() will seek to end of payload after the
# generator returns control to iterparts().
cgpart.seek(0, os.SEEK_SET)
elif isinstance(bundle, changegroup.cg1unpacker): elif isinstance(bundle, changegroup.cg1unpacker):
if bundle.compressed(): if bundle.compressed():
f = self._writetempbundle(bundle.read, '.hg10un', f = self._writetempbundle(bundle.read, '.hg10un',
header='HG10UN') header='HG10UN')
bundle = exchange.readbundle(ui, f, bundlepath, self.vfs) bundle = exchange.readbundle(ui, f, bundlepath, self.vfs)
self._bundlefile = bundle self._bundlefile = bundle
self._cgunpacker = bundle self._cgunpacker = bundle