( )⚙ D4473 wireprotoframing: buffer emitted data to reduce frame count

This is an archive of the discontinued Mercurial Phabricator instance.

wireprotoframing: buffer emitted data to reduce frame count
ClosedPublic

Authored by indygreg on Sep 4 2018, 2:28 PM.

Details

Summary

An upcoming commit introduces a wire protocol command that can emit
hundreds of thousands of small objects. Without a buffering layer,
we would emit a single, small frame for every object. Performance
profiling revealed this to be a source of significant overhead for
both client and server.

This commit introduces a very crude buffering layer so that we emit
fewer, bigger frames in such a scenario. This code will likely get
rewritten in the future to be part of the streams API, as we'll
need a similar strategy for compressing data. I don't want to think
about it too much at the moment though.

server
before: user 32.500+0.000 sys 1.160+0.000
after: user 20.230+0.010 sys 0.180+0.000

client
before: user 133.400+0.000 sys 93.120+0.000
after: user 68.370+0.000 sys 32.950+0.000

This appears to indicate we have significant overhead in the frame
processing code on both client and server. It might be worth profiling
that at some point...

Diff Detail

Repository
rHG Mercurial
Lint
Lint Skipped
Unit
Unit Tests Skipped

Event Timeline

indygreg created this revision.Sep 4 2018, 2:28 PM
This revision was automatically updated to reflect the committed changes.