Page MenuHomePhabricator

remotefilelog: never deadlock on writing to a pipe
Needs RevisionPublic

Authored by ikostia on Nov 19 2017, 3:59 PM.

Details

Reviewers
durham
Group Reviewers
Restricted Project
Summary

Instead of guesstimating that 10000 (or any fixed number) steps
always fit in a buffer, we can just count the bytes as we write.

Depends on D1436

Test Plan
  • not tested yet

Diff Detail

Repository
rFBHGX Facebook Mercurial Extensions
Branch
default
Lint
Lint OK
Unit
Unit Tests OK

Event Timeline

ikostia created this revision.Nov 19 2017, 3:59 PM
wlis added a subscriber: wlis.Nov 20 2017, 12:16 PM
wlis added inline comments.
remotefilelog/fileserverclient.py
230

Probably remove the "True"

372

Is there a way to get this value from the OS, in case someone has some custom setting for the default pipe size?

ikostia updated this revision to Diff 3663.Nov 20 2017, 12:27 PM

remove unnecessary condition

durham requested changes to this revision.Nov 20 2017, 1:37 PM
durham added a subscriber: durham.

Have you tested the performance impact of this change? Previously we would issue 10k at a time, now if the buffer is 65k and if each entry is 100 bytes (40 byte hash + 60 byte path) then we're only doing 650 at a time. You may need to set remotefilelog.cacheprocess= to force it to fallback to normal ssh fetching to execute this code path in bulk.

remotefilelog/fileserverclient.py
238

i += 1

This revision now requires changes to proceed.Nov 20 2017, 1:37 PM