This is an archive of the discontinued Mercurial Phabricator instance.

Differential D9449

phabricator: introduce a `phabricator.retry` option
ClosedPublic

Authored by marmoute on Nov 28 2020, 3:04 PM.

Details

Reviewers
pulkit
Group Reviewers
hg-reviewers
Commits
rHG57183111a463: phabricator: introduce a `phabricator.retry` option
Summary

For the past 2 days, my connection to phab.mercurial-scm.org became extremely
poor. In practice this mean that any conduit call has a fairly high change to
hang and die. Giving the amount of call done by the phabricator extension, it
means the when I am lucky I can get 1 or 2 Diff to update after a few try, but
anything sizeable doesn't have any hope to get through.

This changesets introduce a new option for the fabricator extension to try retry
failed command itself. So that I can get Diff through.

As you can guess, this changeset managed to reach Phabricator thanks to itself.

Diff Detail

Repository
rHG Mercurial
Branch
default
Lint
No Linters Available
Unit
No Unit Test Coverage

Event Timeline

marmoute created this revision.Nov 28 2020, 3:04 PM
marmoute added a comment.Nov 28 2020, 3:14 PM

Do not take this yet, they are some compat thing to setup right first.

marmoute updated this revision to Diff 23790.Nov 28 2020, 3:18 PM
marmoute added a comment.Nov 28 2020, 3:19 PM

These should be fixed with the next update

marmoute updated this revision to Diff 23791.Nov 28 2020, 5:05 PM
marmoute updated this revision to Diff 23794.Nov 28 2020, 6:00 PM
marmoute updated this revision to Diff 23795.Nov 28 2020, 6:03 PM
marmoute added a comment.Nov 28 2020, 6:24 PM

I confirm that this is ready to review.

marmoute updated this revision to Diff 23800.Nov 28 2020, 6:26 PM
pulkit accepted this revision.Nov 29 2020, 1:55 PM
This revision is now accepted and ready to land.Nov 29 2020, 1:55 PM
marmoute added a commit: rHG57183111a463: phabricator: introduce a `phabricator.retry` option.Nov 29 2020, 2:55 PM
Closed by commit rHG57183111a463: phabricator: introduce a `phabricator.retry` option (authored by marmoute). · Explain Why
This revision was automatically updated to reflect the committed changes.

Revision Contents
Changeset List

PathPackages
Mhgext/phabricator.py (34 lines)

Diff 23786

hgext/phabricator.py

callsign = FOO callsign = FOO
# curl command to use. If not set (default), use builtin HTTP library to # curl command to use. If not set (default), use builtin HTTP library to
# communicate. If set, use the specified curl command. This could be useful # communicate. If set, use the specified curl command. This could be useful
# if you need to specify advanced options that is not easily supported by # if you need to specify advanced options that is not easily supported by
# the internal library. # the internal library.
curlcmd = curl --connect-timeout 2 --retry 3 --silent curlcmd = curl --connect-timeout 2 --retry 3 --silent
# retry failed command N time (default 0). Useful when using the extension
# over flakly connection.
#
# We wait `retry.interval` between each retry (default 1)
retry = 3
retry.interval = 10
[auth] [auth]
example.schemes = https example.schemes = https
example.prefix = phab.example.com example.prefix = phab.example.com
# API token. Get it from https://$HOST/conduit/login/ # API token. Get it from https://$HOST/conduit/login/
example.phabtoken = cli-xxxxxxxxxxxxxxxxxxxxxxxxxxxx example.phabtoken = cli-xxxxxxxxxxxxxxxxxxxxxxxxxxxx
""" """
from __future__ import absolute_import from __future__ import absolute_import
import base64 import base64
import contextlib import contextlib
import hashlib import hashlib
import itertools import itertools
import json import json
import mimetypes import mimetypes
import operator import operator
import re import re
import time
from mercurial.node import bin, nullid, short from mercurial.node import bin, nullid, short
from mercurial.i18n import _ from mercurial.i18n import _
from mercurial.pycompat import getattr from mercurial.pycompat import getattr
from mercurial.thirdparty import attr from mercurial.thirdparty import attr
from mercurial import ( from mercurial import (
cmdutil, cmdutil,
context, context,
# developer config: phabricator.repophid # developer config: phabricator.repophid
eh.configitem( eh.configitem(
b'phabricator', b'phabricator',
b'repophid', b'repophid',
default=None, default=None,
) )
eh.configitem( eh.configitem(
b'phabricator', b'phabricator',
b'retry',
default=0,
)
eh.configitem(
b'phabricator',
b'retry.interval',
default=1,
)
eh.configitem(
b'phabricator',
b'url', b'url',
default=None, default=None,
) )
eh.configitem( eh.configitem(
b'phabsend', b'phabsend',
b'confirm', b'confirm',
default=False, default=False,
) )
b'%s -d @- %s' % (curlcmd, procutil.shellquote(url)) b'%s -d @- %s' % (curlcmd, procutil.shellquote(url))
) )
sin.write(data) sin.write(data)
sin.close() sin.close()
body = sout.read() body = sout.read()
else: else:
urlopener = urlmod.opener(ui, authinfo) urlopener = urlmod.opener(ui, authinfo)
request = util.urlreq.request(pycompat.strurl(url), data=data) request = util.urlreq.request(pycompat.strurl(url), data=data)
max_try = ui.configint(b'phabricator', b'retry') + 1
for try_count in range(max_try):
try:
with contextlib.closing(urlopener.open(request)) as rsp: with contextlib.closing(urlopener.open(request)) as rsp:
body = rsp.read() body = rsp.read()
break
except util.urlerr.URLError as err:
ui.debug(
b'Conduit Request failed (try %d/%d: %r\n'
% (try_count + 1, max_try, err)
)
# failing request might come from overloaded server
retry_interval = ui.configint(b'phabricator', b'retry.interval')
time.sleep(retry_interval)
ui.debug(b'Conduit Response: %s\n' % body) ui.debug(b'Conduit Response: %s\n' % body)
parsed = pycompat.rapply( parsed = pycompat.rapply(
lambda x: encoding.unitolocal(x) lambda x: encoding.unitolocal(x)
if isinstance(x, pycompat.unicode) if isinstance(x, pycompat.unicode)
else x, else x,
# json.loads only accepts bytes from py3.6+ # json.loads only accepts bytes from py3.6+
pycompat.json_loads(encoding.unifromlocal(body)), pycompat.json_loads(encoding.unifromlocal(body)),
) )