This is an archive of the discontinued Mercurial Phabricator instance.

Differential D2408

debugcommands: support for sending "batch" requests
ClosedPublic

Authored by indygreg on Feb 23 2018, 3:51 PM.

Details

Reviewers
durin42
Group Reviewers
hg-reviewers
Commits
rHG097ad1079192: debugcommands: support for sending "batch" requests
rHG8c3e67747c4e: debugcommands: support for sending "batch" requests
Summary

Let's teach hg debugwireproto to send "batch" requests.

The easiest way to implement this was as a pair of instructions to
begin and end a batched operation. Otherwise, we would have to reinvent
the parsing wheel or factor out the parsing code.

To prove it works, we add a batched request to test-ssh-proto.t.

Diff Detail

Repository
rHG Mercurial
Lint
Lint Skipped
Unit
Unit Tests Skipped

Event Timeline

indygreg created this revision.Feb 23 2018, 3:51 PM
indygreg updated this revision to Diff 6253.Mar 1 2018, 11:30 AM
durin42 accepted this revision.Mar 1 2018, 4:38 PM
This revision is now accepted and ready to land.Mar 1 2018, 4:38 PM
Closed by commit rHG8c3e67747c4e: debugcommands: support for sending "batch" requests (authored by indygreg). · Explain WhyMar 1 2018, 4:55 PM
This revision was automatically updated to reflect the committed changes.

Revision Contents
Changeset List

PathPackages
Mmercurial/debugcommands.py (39 lines)
Mtests/test-ssh-proto.t (102 lines)

Diff 6253

mercurial/debugcommands.py

each line is ``<key> <value>``. e.g.:: each line is ``<key> <value>``. e.g.::
command listkeys command listkeys
namespace bookmarks namespace bookmarks
Values are interpreted as Python b'' literals. This allows encoding Values are interpreted as Python b'' literals. This allows encoding
special byte sequences via backslash escaping. special byte sequences via backslash escaping.
batchbegin
----------
Instruct the peer to begin a batched send.
All ``command`` blocks are queued for execution until the next
``batchsubmit`` block.
batchsubmit
-----------
Submit previously queued ``command`` blocks as a batch request.
This action MUST be paired with a ``batchbegin`` action.
close close
----- -----
Close the connection to the server. Close the connection to the server.
flush flush
----- -----
peer = None peer = None
else: else:
ui.write(_('creating ssh peer from handshake results\n')) ui.write(_('creating ssh peer from handshake results\n'))
peer = sshpeer.makepeer(ui, url, proc, stdin, stdout, stderr) peer = sshpeer.makepeer(ui, url, proc, stdin, stdout, stderr)
else: else:
raise error.Abort(_('only --localssh is currently supported')) raise error.Abort(_('only --localssh is currently supported'))
batchedcommands = None
# Now perform actions based on the parsed wire language instructions. # Now perform actions based on the parsed wire language instructions.
for action, lines in blocks: for action, lines in blocks:
if action in ('raw', 'raw+'): if action in ('raw', 'raw+'):
# Concatenate the data together. # Concatenate the data together.
data = ''.join(l.lstrip() for l in lines) data = ''.join(l.lstrip() for l in lines)
data = util.unescapestr(data) data = util.unescapestr(data)
stdin.write(data) stdin.write(data)
if len(fields) == 1: if len(fields) == 1:
key = fields[0] key = fields[0]
value = '' value = ''
else: else:
key, value = fields key, value = fields
args[key] = util.unescapestr(value) args[key] = util.unescapestr(value)
if batchedcommands is not None:
batchedcommands.append((command, args))
continue
ui.status(_('sending %s command\n') % command) ui.status(_('sending %s command\n') % command)
res = peer._call(command, **args) res = peer._call(command, **args)
ui.status(_('response: %s\n') % util.escapedata(res)) ui.status(_('response: %s\n') % util.escapedata(res))
elif action == 'batchbegin':
if batchedcommands is not None:
raise error.Abort(_('nested batchbegin not allowed'))
batchedcommands = []
elif action == 'batchsubmit':
# There is a batching API we could go through. But it would be
# difficult to normalize requests into function calls. It is easier
# to bypass this layer and normalize to commands + args.
ui.status(_('sending batch with %d sub-commands\n') %
len(batchedcommands))
for i, chunk in enumerate(peer._submitbatch(batchedcommands)):
ui.status(_('response #%d: %s\n') % (i, util.escapedata(chunk)))
batchedcommands = None
elif action == 'close': elif action == 'close':
peer.close() peer.close()
elif action == 'readavailable': elif action == 'readavailable':
fds = util.poll([stdout.fileno(), stderr.fileno()]) fds = util.poll([stdout.fileno(), stderr.fileno()])
if stdout.fileno() in fds: if stdout.fileno() in fds:
util.readpipe(stdout) util.readpipe(stdout)
if stderr.fileno() in fds: if stderr.fileno() in fds:
util.readpipe(stderr) util.readpipe(stderr)
elif action == 'readline': elif action == 'readline':
stdout.readline() stdout.readline()
else: else:
raise error.Abort(_('unknown action: %s') % action) raise error.Abort(_('unknown action: %s') % action)
if batchedcommands is not None:
raise error.Abort(_('unclosed "batchbegin" request'))
if peer: if peer:
peer.close() peer.close()
if proc: if proc:
proc.kill() proc.kill()

tests/test-ssh-proto.t

i> write(12) -> None: i> write(12) -> None:
i> namespace 6\n i> namespace 6\n
i> write(6) -> None: phases i> write(6) -> None: phases
i> flush() -> None i> flush() -> None
o> bufferedreadline() -> 3: o> bufferedreadline() -> 3:
o> 15\n o> 15\n
o> bufferedread(15) -> 15: publishing True o> bufferedread(15) -> 15: publishing True
response: publishing True response: publishing True
$ cd ..
Test batching of requests
$ hg init batching
$ cd batching
$ echo 0 > foo
$ hg add foo
$ hg -q commit -m initial
$ hg phase --public
$ echo 1 > foo
$ hg commit -m 'commit 1'
$ hg -q up 0
$ echo 2 > foo
$ hg commit -m 'commit 2'
created new head
$ hg book -r 1 bookA
$ hg book -r 2 bookB
$ debugwireproto << EOF
> batchbegin
> command heads
> command listkeys
> namespace bookmarks
> command listkeys
> namespace phases
> batchsubmit
> EOF
testing ssh1
creating ssh peer from handshake results
i> write(104) -> None:
i> hello\n
i> between\n
i> pairs 81\n
i> 0000000000000000000000000000000000000000-0000000000000000000000000000000000000000
i> flush() -> None
o> readline() -> 4:
o> 384\n
o> readline() -> 384:
o> capabilities: lookup changegroupsubset branchmap pushkey known getbundle unbundlehash batch streamreqs=generaldelta,revlogv1 $USUAL_BUNDLE2_CAPS_SERVER$ unbundle=HG10GZ,HG10BZ,HG10UN\n
o> readline() -> 2:
o> 1\n
o> readline() -> 1:
o> \n
sending batch with 3 sub-commands
i> write(6) -> None:
i> batch\n
i> write(4) -> None:
i> * 0\n
i> write(8) -> None:
i> cmds 61\n
i> write(61) -> None: heads ;listkeys namespace=bookmarks;listkeys namespace=phases
i> flush() -> None
o> bufferedreadline() -> 4:
o> 278\n
o> bufferedread(278) -> 278:
o> bfebe6bd38eebc6f8202e419c1171268987ea6a6 4ee3fcef1c800fa2bf23e20af7c83ff111d9c7ab\n
o> ;bookA 4ee3fcef1c800fa2bf23e20af7c83ff111d9c7ab\n
o> bookB bfebe6bd38eebc6f8202e419c1171268987ea6a6;4ee3fcef1c800fa2bf23e20af7c83ff111d9c7ab 1\n
o> bfebe6bd38eebc6f8202e419c1171268987ea6a6 1\n
o> publishing True
response #0: bfebe6bd38eebc6f8202e419c1171268987ea6a6 4ee3fcef1c800fa2bf23e20af7c83ff111d9c7ab\n
response #1: bookA 4ee3fcef1c800fa2bf23e20af7c83ff111d9c7ab\nbookB bfebe6bd38eebc6f8202e419c1171268987ea6a6
response #2: 4ee3fcef1c800fa2bf23e20af7c83ff111d9c7ab 1\nbfebe6bd38eebc6f8202e419c1171268987ea6a6 1\npublishing True
testing ssh2
creating ssh peer from handshake results
i> write(171) -> None:
i> upgrade * proto=exp-ssh-v2-0001\n (glob)
i> hello\n
i> between\n
i> pairs 81\n
i> 0000000000000000000000000000000000000000-0000000000000000000000000000000000000000
i> flush() -> None
o> readline() -> 62:
o> upgraded * exp-ssh-v2-0001\n (glob)
o> readline() -> 4:
o> 383\n
o> read(383) -> 383: capabilities: lookup changegroupsubset branchmap pushkey known getbundle unbundlehash batch streamreqs=generaldelta,revlogv1 $USUAL_BUNDLE2_CAPS_SERVER$ unbundle=HG10GZ,HG10BZ,HG10UN
o> read(1) -> 1:
o> \n
sending batch with 3 sub-commands
i> write(6) -> None:
i> batch\n
i> write(4) -> None:
i> * 0\n
i> write(8) -> None:
i> cmds 61\n
i> write(61) -> None: heads ;listkeys namespace=bookmarks;listkeys namespace=phases
i> flush() -> None
o> bufferedreadline() -> 4:
o> 278\n
o> bufferedread(278) -> 278:
o> bfebe6bd38eebc6f8202e419c1171268987ea6a6 4ee3fcef1c800fa2bf23e20af7c83ff111d9c7ab\n
o> ;bookA 4ee3fcef1c800fa2bf23e20af7c83ff111d9c7ab\n
o> bookB bfebe6bd38eebc6f8202e419c1171268987ea6a6;4ee3fcef1c800fa2bf23e20af7c83ff111d9c7ab 1\n
o> bfebe6bd38eebc6f8202e419c1171268987ea6a6 1\n
o> publishing True
response #0: bfebe6bd38eebc6f8202e419c1171268987ea6a6 4ee3fcef1c800fa2bf23e20af7c83ff111d9c7ab\n
response #1: bookA 4ee3fcef1c800fa2bf23e20af7c83ff111d9c7ab\nbookB bfebe6bd38eebc6f8202e419c1171268987ea6a6
response #2: 4ee3fcef1c800fa2bf23e20af7c83ff111d9c7ab 1\nbfebe6bd38eebc6f8202e419c1171268987ea6a6 1\npublishing True