This is an archive of the discontinued Mercurial Phabricator instance.

Differential D4485

wireprotov2: add bookmarks to "changesetdata" command
ClosedPublic

Authored by indygreg on Sep 5 2018, 12:20 PM.

Details

Reviewers
None
Group Reviewers
hg-reviewers
Commits
rHG9dffa99f9158: wireprotov2: add bookmarks to "changesetdata" command
Summary

Like we did for phases, we want to emit bookmarks data attached
to each changeset.

The approach here is very similar to phases: we emit bookmarks
data inline with requested revision data. But we emit
records for nodes that weren't requested as well so consumers have
access to the full set of defined bookmarks.

Diff Detail

Repository
rHG Mercurial
Lint
Automatic diff as part of commit; lint not applicable.
Unit
Automatic diff as part of commit; unit tests not applicable.

Event Timeline

indygreg created this revision.Sep 5 2018, 12:20 PM
indygreg updated this revision to Diff 10965.Sep 12 2018, 1:11 PM
Closed by commit rHG9dffa99f9158: wireprotov2: add bookmarks to "changesetdata" command (authored by indygreg). · Explain WhySep 14 2018, 11:16 PM
This revision was automatically updated to reflect the committed changes.

Revision Contents
Changeset List

Diff 11063

mercurial/help/internals/wireprotocolv2.txt

nodes nodes
(array of bytestrings) Changelog revisions to request explicitly. (array of bytestrings) Changelog revisions to request explicitly.
fields fields
(set of bytestring) Which data associated with changelog revisions to (set of bytestring) Which data associated with changelog revisions to
fetch. The following values are recognized: fetch. The following values are recognized:
bookmarks
Bookmarks associated with a revision.
parents parents
Parent revisions. Parent revisions.
phase phase
The phase state of a revision. The phase state of a revision.
revision revision
The raw, revision data for the changelog entry. The hash of this data The raw, revision data for the changelog entry. The hash of this data
map describing the next changeset revision will occur. map describing the next changeset revision will occur.
Each map has the following bytestring keys: Each map has the following bytestring keys:
node node
(bytestring) The node value for this revision. This is the SHA-1 hash of (bytestring) The node value for this revision. This is the SHA-1 hash of
the raw revision data. the raw revision data.
bookmarks (optional)
(array of bytestrings) Bookmarks attached to this revision. Only present
if ``bookmarks`` data is being requested and the revision has bookmarks
attached.
parents (optional) parents (optional)
(array of bytestrings) The nodes representing the parent revisions of this (array of bytestrings) The nodes representing the parent revisions of this
revision. Only present if ``parents`` data is being requested. revision. Only present if ``parents`` data is being requested.
phase (optional) phase (optional)
(bytestring) The phase that a revision is in. Recognized values are (bytestring) The phase that a revision is in. Recognized values are
``secret``, ``draft``, and ``public``. Only present if ``phase`` data ``secret``, ``draft``, and ``public``. Only present if ``phase`` data
is being requested. is being requested.
For example, if the request wants ``phase`` and ``revision`` data, For example, if the request wants ``phase`` and ``revision`` data,
the response may contain entries for each changeset in the common nodes the response may contain entries for each changeset in the common nodes
set with the ``phase`` key and without the ``revision`` key in order set with the ``phase`` key and without the ``revision`` key in order
to reflect a phase-only update. to reflect a phase-only update.
TODO support different revision selection mechanisms (e.g. non-public, specific TODO support different revision selection mechanisms (e.g. non-public, specific
revisions) revisions)
TODO support different hash "namespaces" for revisions (e.g. sha-1 versus other) TODO support different hash "namespaces" for revisions (e.g. sha-1 versus other)
TODO support emitting bookmarks data
TODO support emitting obsolescence data TODO support emitting obsolescence data
TODO support filtering based on relevant paths (narrow clone) TODO support filtering based on relevant paths (narrow clone)
TODO support depth limiting TODO support depth limiting
TODO support hgtagsfnodes cache / tags data TODO support hgtagsfnodes cache / tags data
TODO support branch heads cache TODO support branch heads cache
heads heads
----- -----

mercurial/wireprotov2server.py

# outgoing set. This is purely an optimization to reduce data # outgoing set. This is purely an optimization to reduce data
# size. # size.
for node in noderange[0]: for node in noderange[0]:
yield { yield {
b'node': node, b'node': node,
b'phase': b'public' if publishing else repo[node].phasestr() b'phase': b'public' if publishing else repo[node].phasestr()
} }
nodebookmarks = {}
for mark, node in repo._bookmarks.items():
nodebookmarks.setdefault(node, set()).add(mark)
# It is already topologically sorted by revision number. # It is already topologically sorted by revision number.
for node in outgoing: for node in outgoing:
d = { d = {
b'node': node, b'node': node,
} }
if b'parents' in fields: if b'parents' in fields:
d[b'parents'] = cl.parents(node) d[b'parents'] = cl.parents(node)
if b'phase' in fields: if b'phase' in fields:
if publishing: if publishing:
d[b'phase'] = b'public' d[b'phase'] = b'public'
else: else:
ctx = repo[node] ctx = repo[node]
d[b'phase'] = ctx.phasestr() d[b'phase'] = ctx.phasestr()
if b'bookmarks' in fields and node in nodebookmarks:
d[b'bookmarks'] = sorted(nodebookmarks[node])
del nodebookmarks[node]
revisiondata = None revisiondata = None
if b'revision' in fields: if b'revision' in fields:
revisiondata = cl.revision(node, raw=True) revisiondata = cl.revision(node, raw=True)
d[b'revisionsize'] = len(revisiondata) d[b'revisionsize'] = len(revisiondata)
yield d yield d
if revisiondata is not None: if revisiondata is not None:
yield revisiondata yield revisiondata
# If requested, send bookmarks from nodes that didn't have revision
# data sent so receiver is aware of any bookmark updates.
if b'bookmarks' in fields:
for node, marks in sorted(nodebookmarks.iteritems()):
yield {
b'node': node,
b'bookmarks': sorted(marks),
}
@wireprotocommand('heads', @wireprotocommand('heads',
args={ args={
'publiconly': False, 'publiconly': False,
}, },
permission='pull') permission='pull')
def headsv2(repo, proto, publiconly=False): def headsv2(repo, proto, publiconly=False):
if publiconly: if publiconly:
repo = repo.filtered('immutable') repo = repo.filtered('immutable')

tests/test-wireproto-command-changesetdata.t

}, },
{ {
b'node': b'\xea\xe5\xf8,.b#h\xd2}\xae\xcbv\xb7\xe3\x93\xd0\xf2B\x11', b'node': b'\xea\xe5\xf8,.b#h\xd2}\xae\xcbv\xb7\xe3\x93\xd0\xf2B\x11',
b'revisionsize': 61 b'revisionsize': 61
}, },
b'1b74476799ec8318045db759b1b4bcc9b839d0aa\ntest\n0 0\na\n\ncommit 3' b'1b74476799ec8318045db759b1b4bcc9b839d0aa\ntest\n0 0\na\n\ncommit 3'
] ]
Bookmarks key isn't present if no bookmarks data
$ sendhttpv2peer << EOF
> command changesetdata
> fields eval:[b'bookmarks']
> noderange eval:[[], [b'\x0b\xb8\xad\x89\x4a\x15\xb1\x53\x80\xb2\xa2\xa5\xb1\x83\xe2\x0f\x2a\x4b\x28\xdd', b'\xea\xe5\xf8\x2c\x2e\x62\x23\x68\xd2\x7d\xae\xcb\x76\xb7\xe3\x93\xd0\xf2\x42\x11']]
> EOF
creating http peer for wire protocol version 2
sending changesetdata command
s> POST /api/exp-http-v2-0001/ro/changesetdata HTTP/1.1\r\n
s> Accept-Encoding: identity\r\n
s> accept: application/mercurial-exp-framing-0005\r\n
s> content-type: application/mercurial-exp-framing-0005\r\n
s> content-length: 107\r\n
s> host: $LOCALIP:$HGPORT\r\n (glob)
s> user-agent: Mercurial debugwireproto\r\n
s> \r\n
s> c\x00\x00\x01\x00\x01\x01\x11\xa2Dargs\xa2Ffields\x81IbookmarksInoderange\x82\x80\x82T\x0b\xb8\xad\x89J\x15\xb1S\x80\xb2\xa2\xa5\xb1\x83\xe2\x0f*K(\xddT\xea\xe5\xf8,.b#h\xd2}\xae\xcbv\xb7\xe3\x93\xd0\xf2B\x11DnameMchangesetdata
s> makefile('rb', None)
s> HTTP/1.1 200 OK\r\n
s> Server: testing stub value\r\n
s> Date: $HTTP_DATE$\r\n
s> Content-Type: application/mercurial-exp-framing-0005\r\n
s> Transfer-Encoding: chunked\r\n
s> \r\n
s> 13\r\n
s> \x0b\x00\x00\x01\x00\x02\x011
s> \xa1FstatusBok
s> \r\n
received frame(size=11; request=1; stream=2; streamflags=stream-begin; type=command-response; flags=continuation)
s> 81\r\n
s> y\x00\x00\x01\x00\x02\x001
s> \xa1Jtotalitems\x04\xa1DnodeT3\x90\xef\x85\x00s\xfb\xc2\xf0\xdf\xff"D4,\x8e\x92)\x01:\xa1DnodeTu\x92\x91~\x1c>\x82g|\xb0\xa4\xbcq\\\xa2]\xd1-(\xc1\xa1DnodeT\x0b\xb8\xad\x89J\x15\xb1S\x80\xb2\xa2\xa5\xb1\x83\xe2\x0f*K(\xdd\xa1DnodeT\xea\xe5\xf8,.b#h\xd2}\xae\xcbv\xb7\xe3\x93\xd0\xf2B\x11
s> \r\n
received frame(size=121; request=1; stream=2; streamflags=; type=command-response; flags=continuation)
s> 8\r\n
s> \x00\x00\x00\x01\x00\x02\x002
s> \r\n
s> 0\r\n
s> \r\n
received frame(size=0; request=1; stream=2; streamflags=; type=command-response; flags=eos)
response: gen[
{
b'totalitems': 4
},
{
b'node': b'3\x90\xef\x85\x00s\xfb\xc2\xf0\xdf\xff"D4,\x8e\x92)\x01:'
},
{
b'node': b'u\x92\x91~\x1c>\x82g|\xb0\xa4\xbcq\\\xa2]\xd1-(\xc1'
},
{
b'node': b'\x0b\xb8\xad\x89J\x15\xb1S\x80\xb2\xa2\xa5\xb1\x83\xe2\x0f*K(\xdd'
},
{
b'node': b'\xea\xe5\xf8,.b#h\xd2}\xae\xcbv\xb7\xe3\x93\xd0\xf2B\x11'
}
]
Bookmarks are sent when requested
$ hg -R ../server bookmark -r 0bb8ad894a15b15380b2a2a5b183e20f2a4b28dd book-1
$ hg -R ../server bookmark -r eae5f82c2e622368d27daecb76b7e393d0f24211 book-2
$ hg -R ../server bookmark -r eae5f82c2e622368d27daecb76b7e393d0f24211 book-3
$ sendhttpv2peer << EOF
> command changesetdata
> fields eval:[b'bookmarks']
> noderange eval:[[], [b'\x0b\xb8\xad\x89\x4a\x15\xb1\x53\x80\xb2\xa2\xa5\xb1\x83\xe2\x0f\x2a\x4b\x28\xdd', b'\xea\xe5\xf8\x2c\x2e\x62\x23\x68\xd2\x7d\xae\xcb\x76\xb7\xe3\x93\xd0\xf2\x42\x11']]
> EOF
creating http peer for wire protocol version 2
sending changesetdata command
s> POST /api/exp-http-v2-0001/ro/changesetdata HTTP/1.1\r\n
s> Accept-Encoding: identity\r\n
s> accept: application/mercurial-exp-framing-0005\r\n
s> content-type: application/mercurial-exp-framing-0005\r\n
s> content-length: 107\r\n
s> host: $LOCALIP:$HGPORT\r\n (glob)
s> user-agent: Mercurial debugwireproto\r\n
s> \r\n
s> c\x00\x00\x01\x00\x01\x01\x11\xa2Dargs\xa2Ffields\x81IbookmarksInoderange\x82\x80\x82T\x0b\xb8\xad\x89J\x15\xb1S\x80\xb2\xa2\xa5\xb1\x83\xe2\x0f*K(\xddT\xea\xe5\xf8,.b#h\xd2}\xae\xcbv\xb7\xe3\x93\xd0\xf2B\x11DnameMchangesetdata
s> makefile('rb', None)
s> HTTP/1.1 200 OK\r\n
s> Server: testing stub value\r\n
s> Date: $HTTP_DATE$\r\n
s> Content-Type: application/mercurial-exp-framing-0005\r\n
s> Transfer-Encoding: chunked\r\n
s> \r\n
s> 13\r\n
s> \x0b\x00\x00\x01\x00\x02\x011
s> \xa1FstatusBok
s> \r\n
received frame(size=11; request=1; stream=2; streamflags=stream-begin; type=command-response; flags=continuation)
s> ac\r\n
s> \xa4\x00\x00\x01\x00\x02\x001
s> \xa1Jtotalitems\x04\xa1DnodeT3\x90\xef\x85\x00s\xfb\xc2\xf0\xdf\xff"D4,\x8e\x92)\x01:\xa1DnodeTu\x92\x91~\x1c>\x82g|\xb0\xa4\xbcq\\\xa2]\xd1-(\xc1\xa2Ibookmarks\x81Fbook-1DnodeT\x0b\xb8\xad\x89J\x15\xb1S\x80\xb2\xa2\xa5\xb1\x83\xe2\x0f*K(\xdd\xa2Ibookmarks\x82Fbook-2Fbook-3DnodeT\xea\xe5\xf8,.b#h\xd2}\xae\xcbv\xb7\xe3\x93\xd0\xf2B\x11
s> \r\n
received frame(size=164; request=1; stream=2; streamflags=; type=command-response; flags=continuation)
s> 8\r\n
s> \x00\x00\x00\x01\x00\x02\x002
s> \r\n
s> 0\r\n
s> \r\n
received frame(size=0; request=1; stream=2; streamflags=; type=command-response; flags=eos)
response: gen[
{
b'totalitems': 4
},
{
b'node': b'3\x90\xef\x85\x00s\xfb\xc2\xf0\xdf\xff"D4,\x8e\x92)\x01:'
},
{
b'node': b'u\x92\x91~\x1c>\x82g|\xb0\xa4\xbcq\\\xa2]\xd1-(\xc1'
},
{
b'bookmarks': [
b'book-1'
],
b'node': b'\x0b\xb8\xad\x89J\x15\xb1S\x80\xb2\xa2\xa5\xb1\x83\xe2\x0f*K(\xdd'
},
{
b'bookmarks': [
b'book-2',
b'book-3'
],
b'node': b'\xea\xe5\xf8,.b#h\xd2}\xae\xcbv\xb7\xe3\x93\xd0\xf2B\x11'
}
]
Bookmarks are sent when we make a no-new-revisions request
$ sendhttpv2peer << EOF
> command changesetdata
> fields eval:[b'bookmarks', b'revision']
> noderange eval:[[b'\xea\xe5\xf8\x2c\x2e\x62\x23\x68\xd2\x7d\xae\xcb\x76\xb7\xe3\x93\xd0\xf2\x42\x11'], [b'\x0b\xb8\xad\x89\x4a\x15\xb1\x53\x80\xb2\xa2\xa5\xb1\x83\xe2\x0f\x2a\x4b\x28\xdd', b'\xea\xe5\xf8\x2c\x2e\x62\x23\x68\xd2\x7d\xae\xcb\x76\xb7\xe3\x93\xd0\xf2\x42\x11']]
> EOF
creating http peer for wire protocol version 2
sending changesetdata command
s> POST /api/exp-http-v2-0001/ro/changesetdata HTTP/1.1\r\n
s> Accept-Encoding: identity\r\n
s> accept: application/mercurial-exp-framing-0005\r\n
s> content-type: application/mercurial-exp-framing-0005\r\n
s> content-length: 137\r\n
s> host: $LOCALIP:$HGPORT\r\n (glob)
s> user-agent: Mercurial debugwireproto\r\n
s> \r\n
s> \x81\x00\x00\x01\x00\x01\x01\x11\xa2Dargs\xa2Ffields\x82IbookmarksHrevisionInoderange\x82\x81T\xea\xe5\xf8,.b#h\xd2}\xae\xcbv\xb7\xe3\x93\xd0\xf2B\x11\x82T\x0b\xb8\xad\x89J\x15\xb1S\x80\xb2\xa2\xa5\xb1\x83\xe2\x0f*K(\xddT\xea\xe5\xf8,.b#h\xd2}\xae\xcbv\xb7\xe3\x93\xd0\xf2B\x11DnameMchangesetdata
s> makefile('rb', None)
s> HTTP/1.1 200 OK\r\n
s> Server: testing stub value\r\n
s> Date: $HTTP_DATE$\r\n
s> Content-Type: application/mercurial-exp-framing-0005\r\n
s> Transfer-Encoding: chunked\r\n
s> \r\n
s> 13\r\n
s> \x0b\x00\x00\x01\x00\x02\x011
s> \xa1FstatusBok
s> \r\n
received frame(size=11; request=1; stream=2; streamflags=stream-begin; type=command-response; flags=continuation)
s> 12f\r\n
s> \'\x01\x00\x01\x00\x02\x001
s> \xa1Jtotalitems\x02\xa2DnodeTu\x92\x91~\x1c>\x82g|\xb0\xa4\xbcq\\\xa2]\xd1-(\xc1Lrevisionsize\x18?X?7f144aea0ba742713887b564d57e9d12f12ff382\n
s> test\n
s> 0 0\n
s> a\n
s> b\n
s> \n
s> commit 1\xa3Ibookmarks\x81Fbook-1DnodeT\x0b\xb8\xad\x89J\x15\xb1S\x80\xb2\xa2\xa5\xb1\x83\xe2\x0f*K(\xddLrevisionsize\x18=X=37f0a2d1c28ffe4b879109a7d1bbf8f07b3c763b\n
s> test\n
s> 0 0\n
s> b\n
s> \n
s> commit 2\xa2Ibookmarks\x82Fbook-2Fbook-3DnodeT\xea\xe5\xf8,.b#h\xd2}\xae\xcbv\xb7\xe3\x93\xd0\xf2B\x11
s> \r\n
received frame(size=295; request=1; stream=2; streamflags=; type=command-response; flags=continuation)
s> 8\r\n
s> \x00\x00\x00\x01\x00\x02\x002
s> \r\n
s> 0\r\n
s> \r\n
received frame(size=0; request=1; stream=2; streamflags=; type=command-response; flags=eos)
response: gen[
{
b'totalitems': 2
},
{
b'node': b'u\x92\x91~\x1c>\x82g|\xb0\xa4\xbcq\\\xa2]\xd1-(\xc1',
b'revisionsize': 63
},
b'7f144aea0ba742713887b564d57e9d12f12ff382\ntest\n0 0\na\nb\n\ncommit 1',
{
b'bookmarks': [
b'book-1'
],
b'node': b'\x0b\xb8\xad\x89J\x15\xb1S\x80\xb2\xa2\xa5\xb1\x83\xe2\x0f*K(\xdd',
b'revisionsize': 61
},
b'37f0a2d1c28ffe4b879109a7d1bbf8f07b3c763b\ntest\n0 0\nb\n\ncommit 2',
{
b'bookmarks': [
b'book-2',
b'book-3'
],
b'node': b'\xea\xe5\xf8,.b#h\xd2}\xae\xcbv\xb7\xe3\x93\xd0\xf2B\x11'
}
]
Multiple fields can be transferred Multiple fields can be transferred
$ sendhttpv2peer << EOF $ sendhttpv2peer << EOF
> command changesetdata > command changesetdata
> fields eval:[b'parents', b'revision'] > fields eval:[b'parents', b'revision']
> nodes eval:[b'\xea\xe5\xf8\x2c\x2e\x62\x23\x68\xd2\x7d\xae\xcb\x76\xb7\xe3\x93\xd0\xf2\x42\x11'] > nodes eval:[b'\xea\xe5\xf8\x2c\x2e\x62\x23\x68\xd2\x7d\xae\xcb\x76\xb7\xe3\x93\xd0\xf2\x42\x11']
> EOF > EOF
creating http peer for wire protocol version 2 creating http peer for wire protocol version 2