This is an archive of the discontinued Mercurial Phabricator instance.

Differential D8850

dirstate: restore original estimation and update comment
ClosedPublic

Authored by joerg.sonnenberger on Jul 30 2020, 10:13 AM.

Details

Reviewers
Alphare
pulkit
Group Reviewers
hg-reviewers
Commits
rHGad7006830106: dirstate: restore original estimation and update comment
Summary

The former comment didn't reflect the content of the dirstate entries,
the two nodes are a fixed header in the file and not per-entry. Split
the documented entry into the path part and the fixed header. The
heuristic itself is still valid, e.g. for the NetBSD src tree a maximum
path size of 142 and an average of 49, resulting in 66 bytes per entry
on average.

Diff Detail

Repository
rHG Mercurial
Branch
default
Lint
No Linters Available
Unit
No Unit Test Coverage

Event Timeline

Alphare accepted this revision.Jul 30 2020, 10:57 AM
pulkit accepted this revision.Jul 31 2020, 7:40 AM
This revision is now accepted and ready to land.Jul 31 2020, 7:40 AM

Revision Contents
Changeset List

PathPackages
Mmercurial/dirstate.py (18 lines)

Diff 22154

mercurial/dirstate.py

if err.errno != errno.ENOENT: if err.errno != errno.ENOENT:
raise raise
return return
if not st: if not st:
return return
if util.safehasattr(parsers, b'dict_new_presized'): if util.safehasattr(parsers, b'dict_new_presized'):
# Make an estimate of the number of files in the dirstate based on # Make an estimate of the number of files in the dirstate based on
# its size. From a linear regression on a set of real-world repos, # its size. This trades wasting some memory for avoiding costly
# all over 10,000 files, the size of a dirstate entry is 2 nodes # resizes. Each entry have a prefix of 17 bytes followed by one or
# plus 45 bytes. The cost of resizing is significantly higher than the cost # two path names. Studies on various large-scale real-world repositories
# of filling in a larger presized dict, so subtract 20% from the # found 54 bytes a reasonable upper limit for the average path names.
# size. # Copy entries are ignored for the sake of this estimate.
# self._map = parsers.dict_new_presized(len(st) // 71)
# This heuristic is imperfect in many ways, so in a future dirstate
# format update it makes sense to just record the number of entries
# on write.
self._map = parsers.dict_new_presized(
len(st) // ((2 * self._nodelen + 45) * 4 // 5)
)
# Python's garbage collector triggers a GC each time a certain number # Python's garbage collector triggers a GC each time a certain number
# of container objects (the number being defined by # of container objects (the number being defined by
# gc.get_threshold()) are allocated. parse_dirstate creates a tuple # gc.get_threshold()) are allocated. parse_dirstate creates a tuple
# for each file in the dirstate. The C version then immediately marks # for each file in the dirstate. The C version then immediately marks
# them as not to be tracked by the collector. However, this has no # them as not to be tracked by the collector. However, this has no
# effect on when GCs are triggered, only on what objects the GC looks # effect on when GCs are triggered, only on what objects the GC looks
# into. This means that O(number of files) GCs are unavoidable. # into. This means that O(number of files) GCs are unavoidable.