narrow: send specs as bundle2 data instead of param (issue5952) (issue6019)
ClosedPublic

Authored by pulkit on Mon, Apr 8, 11:23 AM.

Details

Summary

Before this patch, when ACL is involved, narrowspecs are send as bundle2
parameter for narrow:spec bundle2 part. The limitation of bundle2 parts are they
cannot send data larger than 255 bytes. Includes and excludes in narrow are not
limited by size and they can grow over 255 bytes.

This patch introduces a new mandatory bundle2 part and send narrowspecs as data
of that. The new bundle2 part is introduced to keep things cleaner and easy to
distinguish related to backward compatibility.
The part is mandatory because without server's narrowspec, the local ACL narrow
repo won't work.

This patch makes clients compatible with servers which have older versions.
However I left a comment that we should drop the other bundle2 part soon as
that's broken and people should not rely on that.

I named the new bundle2 part 'Narrowspec' because:

  1. Capital 'N' to make it mandatory
  2. 'Narrow:spec' cannot be used because bundle2 enforces that there should not

be two different parts which resolve to same name when lowercased.

Diff Detail

Repository
rHG Mercurial
Lint
Lint Skipped
Unit
Unit Tests Skipped
pulkit created this revision.Mon, Apr 8, 11:23 AM
pulkit added inline comments.Mon, Apr 8, 11:25 AM
mercurial/exchange.py
2216

I don't feel good about the fact that we are not encoding data here. Is there exists some function which I can use to encode and decode this list of specs?

martinvonz added inline comments.Mon, Apr 8, 11:48 AM
mercurial/exchange.py
2216

You could probably reuse the function from D6184

pulkit updated this revision to Diff 14711.Fri, Apr 12, 10:24 AM
martinvonz added inline comments.Tue, Apr 16, 12:03 PM
hgext/narrow/narrowbundle2.py
156

I'd drop these checks

mercurial/exchange.py
2215–2216

And these

This revision was automatically updated to reflect the committed changes.
pulkit added inline comments.Tue, Apr 16, 1:11 PM
hgext/narrow/narrowbundle2.py
156

Sent D6241.

This patch is backwards incompatible over the wire protocol.

The problem is a new client will blindly send part data to an old server expecting part parameters. The old server won't read the part data and it would be as if the includes and excludes were not sent.

We need some kind of capability negotiation that allows the client to opt in to the newer behavior if the server advertises support for it.

Also, my personal preference is to create new bundle2 parts rather than change behavior of existing bundle2 parts. Doing things this way ensures that behavior for a named bundle2 part is constant over time. This keeps implementations simpler, as individual part handling can do one thing and one thing only.

Finally, the internals help docs should be updated to reflect changes to bundle2 part behavior.

This patch is backwards incompatible over the wire protocol.

The problem is a new client will blindly send part data to an old server expecting part parameters. The old server won't read the part data and it would be as if the includes and excludes were not sent.

It's an experimental feature and I suspect it's used only by Sandu (@idlsoft). Sandu, if we released this without the capability negotiation that Greg is talking about, you would need to make sure to upgrade the server before you upgrade your client(s). Are you okay with that? Is anyone aware of any other users of this feature? Greg, are you okay with making a breaking change (to an experimental feature) if the few existing users are okay with it?

We need some kind of capability negotiation that allows the client to opt in to the newer behavior if the server advertises support for it.

Also, my personal preference is to create new bundle2 parts rather than change behavior of existing bundle2 parts. Doing things this way ensures that behavior for a named bundle2 part is constant over time. This keeps implementations simpler, as individual part handling can do one thing and one thing only.

Finally, the internals help docs should be updated to reflect changes to bundle2 part behavior.

This patch is backwards incompatible over the wire protocol.

The problem is a new client will blindly send part data to an old server expecting part parameters. The old server won't read the part data and it would be as if the includes and excludes were not sent.

It's an experimental feature and I suspect it's used only by Sandu (@idlsoft). Sandu, if we released this without the capability negotiation that Greg is talking about, you would need to make sure to upgrade the server before you upgrade your client(s). Are you okay with that? Is anyone aware of any other users of this feature? Greg, are you okay with making a breaking change (to an experimental feature) if the few existing users are okay with it?

I agree with @martinvonz here. narrow extension is experimental right now, in 4.9 we had a lot of breaking changes. The narrowspecs are only send back in case when ACL is enabled. If there are users who rely on existing behavior, they must have hit the bug just like @idlsoft hit.

I am not sure how we can keep sending narrowspecs back using bundle2 param and fix the issues which this patch is trying to.

We need some kind of capability negotiation that allows the client to opt in to the newer behavior if the server advertises support for it.

Also, my personal preference is to create new bundle2 parts rather than change behavior of existing bundle2 parts. Doing things this way ensures that behavior for a named bundle2 part is constant over time. This keeps implementations simpler, as individual part handling can do one thing and one thing only.

Finally, the internals help docs should be updated to reflect changes to bundle2 part behavior.

Because the current client ignores the data completely, the only way to force it to fail I think is to change the name of the part.
This would make things cleaner probably, but I'll deal with whatever solution you guys settle on.

Because the current client ignores the data completely, the only way to force it to fail I think is to change the name of the part.
This would make things cleaner probably, but I'll deal with whatever solution you guys settle on.

I think bundle2 parts can be marked mandatory (by using uppercase in their name?). It seems to me like the ACL part should be mandatory. Is that correct, Sandu? So that's a good point and thanks for mentioning that. Pulkit, I think it's enough to change the name to be something like narrow:Spec or Narrow:spec (I'm thinking the former since some narrow parts are mandatory and some are not and then they all still start with narrow:).

If ACL is enabled, processing this part is mandatory, yes.
On clone, or pull the user doesn't specify includes, so reading this part is the only way the client can get them.

This is nitpicking, but there is a duplicate _NARROWACL_SECTION definition in narrowbundle2.py,
I think only the one in exchange.py should remain.
Btw it's still 'narrowhgacl' from the old days.

pulkit updated this revision to Diff 14870.Fri, Apr 19, 5:06 PM
pulkit edited the summary of this revision. (Show Details)