This is an archive of the discontinued Mercurial Phabricator instance.

Differential D1649

infinitepush: add logic to support old clients on the basis of pushkey part
Needs RevisionPublic

Authored by pulkit on Dec 10 2017, 7:34 PM.
Tags
None
Subscribers
None

Details

Reviewers
durham
Group Reviewers
Restricted Project
Summary

This patch adds logic to check whether a push is an infinitepush or not from
an old client or client who don't have infinitepush client logic.

Diff Detail

Repository
rFBHGX Facebook Mercurial Extensions
Lint
Lint Skipped
Unit
Unit Tests Skipped

Event Timeline

pulkit created this revision.Dec 10 2017, 7:34 PM
Herald added a reviewer: Restricted Project. · View Herald TranscriptDec 10 2017, 7:34 PM
durham requested changes to this revision.Jan 9 2018, 11:08 AM

Could this have a test? Is there a config knob to disable bundle2 or force bundle1?

infinitepush/__init__.py
1055

I'd make it clear that this will have performance impact.

1080

Can we add a comment saying this could require a lot of memory, and that we might want to move this to being stored on disk in the future?

1108

I'd rename this supportoldclients, since it's a server configuration.

This revision now requires changes to proceed.Jan 9 2018, 11:08 AM
pulkit added a comment.EditedJan 11 2018, 4:47 AM

Status update: While adding tests, I found that I am reading the part twice, once for adding in bundlepart and once for the copied part. Since I can't read that twice, I need to do better here with the logic involved.

Revision Contents
Changeset List

PathPackages
Minfinitepush/__init__.py (60 lines)

Diff 4342

infinitepush/__init__.py

logger = partial(ui.log, 'infinitepush', user=username, logger = partial(ui.log, 'infinitepush', user=username,
requestid=requestid, hostname=hostname, requestid=requestid, hostname=hostname,
reponame=ui.config('infinitepush', 'reponame')) reponame=ui.config('infinitepush', 'reponame'))
op.records.add('infinitepushlogger', logger) op.records.add('infinitepushlogger', logger)
else: else:
logger = logger[0] logger = logger[0]
return logger return logger
def oldclientprocessor(orig, repo, op, unbundler):
""" Logic to check whether this part is a bundle2 part or not in case of old
client where we don't have capability to set things at client side.
Iterates over the bundle2 parts at time of unbundling, store a copy of each
part for later processing, grabs the bookmark from the pushkey part and
check if this is an infinitepush or not. If infinitepush, store the bundle
in the store otherwise, fallbacks to processing each part normally.
durhamUnsubmitted
Not Done

I'd make it clear that this will have performance impact.

durham: I'd make it clear that this will have performance impact.
This support of oldclient is turned of by default and can be used by setting
the config::
[infinitepush]
oldclient = True
"""
bundler = bundle2.bundle20(repo.ui)
isinfinitepush = False
# a list to store copy of parts to process them again
partslist = []
with bundle2.partiterator(repo, op, unbundler) as parts:
for part in parts:
if part.type == 'pushkey':
import pushkey
dec = pushkey.decode
namespace = dec(part.params['namespace'])
if namespace == 'bookmarks':
bname = dec(part.params['key'])
if _scratchbranchmatcher(bname):
isinfinitepush = True
bundler.addpart(part)
partslist.append(copiedpart(part))
durhamUnsubmitted
Not Done

Can we add a comment saying this could require a lot of memory, and that we might want to move this to being stored on disk in the future?

durham: Can we add a comment saying this could require a lot of memory, and that we might want to move…
if isinfinitepush:
buf = util.chunkbuffer(bundler.getchunks())
fd, bundlefile = tempfile.mkstemp()
try:
try:
fp = os.fdopen(fd, 'wb')
fp.write(buf.read())
finally:
fp.close()
# need some way to get cgparams too
storebundle(op, {}, bundlefile)
finally:
try:
os.unlink(bundlefile)
except Exception:
# we would rather see the original exception
pass
else:
for part in partslist:
bundle2._processpart(op, part)
def processparts(orig, repo, op, unbundler): def processparts(orig, repo, op, unbundler):
if unbundler.params.get('infinitepush') != 'True': if unbundler.params.get('infinitepush') != 'True':
# shall we support old clients
if repo.ui.configbool('infinitepush', 'oldclient'):
durhamUnsubmitted
Not Done

I'd rename this supportoldclients, since it's a server configuration.

durham: I'd rename this `supportoldclients`, since it's a server configuration.
return oldclientprocessor(orig, repo, op, unbundler)
return orig(repo, op, unbundler) return orig(repo, op, unbundler)
handleallparts = repo.ui.configbool('infinitepush', 'storeallparts') handleallparts = repo.ui.configbool('infinitepush', 'storeallparts')
partforwardingwhitelist = [] partforwardingwhitelist = []
try: try:
treemfmod = extensions.find('treemanifest') treemfmod = extensions.find('treemanifest')
partforwardingwhitelist.append(treemfmod.TREEGROUP_PARTTYPE2) partforwardingwhitelist.append(treemfmod.TREEGROUP_PARTTYPE2)