This is an archive of the discontinued Mercurial Phabricator instance.

Differential D170

remotefilelog: handle truncated server cache files
ClosedPublic

Authored by mbthomas on Jul 21 2017, 9:35 AM.
Tags
None
Subscribers
None

Details

Reviewers
simonfar
Group Reviewers
Restricted Project
Commits
rFBHGX82c678e60af8: remotefilelog: handle truncated server cache files
Summary

If remotefilelog server cache files are truncated, when the server
returns these files to the client, the client will crash trying to
decompress them. Detect truncated files and treat them as cache
misses.

Test Plan

Add unit test for truncated server cache files.

Diff Detail

Repository
rFBHGX Facebook Mercurial Extensions
Lint
Automatic diff as part of commit; lint not applicable.
Unit
Automatic diff as part of commit; unit tests not applicable.

Event Timeline

mbthomas created this revision.Jul 21 2017, 9:35 AM
simonfar accepted this revision.Jul 21 2017, 10:33 AM
This revision is now accepted and ready to land.Jul 21 2017, 10:33 AM
Closed by commit rFBHGX82c678e60af8: remotefilelog: handle truncated server cache files (authored by mbthomas). · Explain WhyJul 21 2017, 1:58 PM
This revision was automatically updated to reflect the committed changes.

Revision Contents
Changeset List

Diff 378

remotefilelog/remotefilelogserver.py

if cmd == 'getfiles': if cmd == 'getfiles':
return False return False
return orig(cmd) return orig(cmd)
wrapfunction(httpprotocol, 'iscmd', _iscmd) wrapfunction(httpprotocol, 'iscmd', _iscmd)
def _loadfileblob(repo, cachepath, path, node): def _loadfileblob(repo, cachepath, path, node):
filecachepath = os.path.join(cachepath, path, hex(node)) filecachepath = os.path.join(cachepath, path, hex(node))
if not os.path.exists(filecachepath): if not os.path.exists(filecachepath) or os.path.getsize(filecachepath) == 0:
filectx = repo.filectx(path, fileid=node) filectx = repo.filectx(path, fileid=node)
if filectx.node() == nullid: if filectx.node() == nullid:
repo.changelog = changelog.changelog(repo.svfs) repo.changelog = changelog.changelog(repo.svfs)
filectx = repo.filectx(path, fileid=node) filectx = repo.filectx(path, fileid=node)
text = createfileblob(filectx) text = createfileblob(filectx)
text = lz4wrapper.lzcompresshc(text) text = lz4wrapper.lzcompresshc(text)

tests/test-remotefilelog-corrupt-cache.t

> EOF > EOF
$ chmod u+w $CACHEDIR/master/11/f6ad8ec52a2984abaafd7c3b516503785c2072/1406e74118627694268417491f018a4a883152f0 $ chmod u+w $CACHEDIR/master/11/f6ad8ec52a2984abaafd7c3b516503785c2072/1406e74118627694268417491f018a4a883152f0
$ echo x > $CACHEDIR/master/11/f6ad8ec52a2984abaafd7c3b516503785c2072/1406e74118627694268417491f018a4a883152f0 $ echo x > $CACHEDIR/master/11/f6ad8ec52a2984abaafd7c3b516503785c2072/1406e74118627694268417491f018a4a883152f0
$ hg up tip $ hg up tip
3 files updated, 0 files merged, 0 files removed, 0 files unresolved 3 files updated, 0 files merged, 0 files removed, 0 files unresolved
1 files fetched over 1 fetches - (1 misses, 0.00% hit ratio) over *s (glob) 1 files fetched over 1 fetches - (1 misses, 0.00% hit ratio) over *s (glob)
$ cat .hg/remotefilelog_cache.log $ cat .hg/remotefilelog_cache.log
corrupt $TESTTMP/hgcache/master/11/f6ad8ec52a2984abaafd7c3b516503785c2072/1406e74118627694268417491f018a4a883152f0 during contains corrupt $TESTTMP/hgcache/master/11/f6ad8ec52a2984abaafd7c3b516503785c2072/1406e74118627694268417491f018a4a883152f0 during contains
Verify handling of corrupt server cache
$ rm -f ../master/.hg/remotefilelogcache/y/076f5e2225b3ff0400b98c92aa6cdf403ee24cca
$ touch ../master/.hg/remotefilelogcache/y/076f5e2225b3ff0400b98c92aa6cdf403ee24cca
$ clearcache
$ hg prefetch -r .
3 files fetched over 1 fetches - (3 misses, 0.00% hit ratio) over *s (glob)
$ test -s ../master/.hg/remotefilelogcache/y/076f5e2225b3ff0400b98c92aa6cdf403ee24cca
$ hg debugremotefilelog $CACHEDIR/master/95/cb0bfd2977c761298d9624e4b4d4c72a39974a/076f5e2225b3ff0400b98c92aa6cdf403ee24cca
size: 2 bytes
path: $TESTTMP/hgcache/master/95/cb0bfd2977c761298d9624e4b4d4c72a39974a/076f5e2225b3ff0400b98c92aa6cdf403ee24cca
key: 076f5e2225b3
node => p1 p2 linknode copyfrom
076f5e2225b3 => 000000000000 000000000000 f3d0bb0d1e48