This is an archive of the discontinued Mercurial Phabricator instance.

Differential D8850

dirstate: restore original estimation and update comment
ClosedPublic

Authored by joerg.sonnenberger on Jul 30 2020, 10:13 AM.

Details

Reviewers
Alphare
pulkit
Group Reviewers
hg-reviewers
Commits
rHGad7006830106: dirstate: restore original estimation and update comment
Summary

The former comment didn't reflect the content of the dirstate entries,
the two nodes are a fixed header in the file and not per-entry. Split
the documented entry into the path part and the fixed header. The
heuristic itself is still valid, e.g. for the NetBSD src tree a maximum
path size of 142 and an average of 49, resulting in 66 bytes per entry
on average.

Diff Detail

Repository
rHG Mercurial
Lint
Automatic diff as part of commit; lint not applicable.
Unit
Automatic diff as part of commit; unit tests not applicable.

Event Timeline

Alphare accepted this revision.Jul 30 2020, 10:57 AM
pulkit accepted this revision.Jul 31 2020, 7:40 AM
This revision is now accepted and ready to land.Jul 31 2020, 7:40 AM

Revision Contents
Changeset List

PathPackages
Mmercurial/dirstate.py (18 lines)

Diff 22158

mercurial/dirstate.py

if err.errno != errno.ENOENT: if err.errno != errno.ENOENT:
raise raise
return return
if not st: if not st:
return return
if util.safehasattr(parsers, b'dict_new_presized'): if util.safehasattr(parsers, b'dict_new_presized'):
# Make an estimate of the number of files in the dirstate based on # Make an estimate of the number of files in the dirstate based on
# its size. From a linear regression on a set of real-world repos, # its size. This trades wasting some memory for avoiding costly
# all over 10,000 files, the size of a dirstate entry is 2 nodes # resizes. Each entry have a prefix of 17 bytes followed by one or
# plus 45 bytes. The cost of resizing is significantly higher than the cost # two path names. Studies on various large-scale real-world repositories
# of filling in a larger presized dict, so subtract 20% from the # found 54 bytes a reasonable upper limit for the average path names.
# size. # Copy entries are ignored for the sake of this estimate.
# self._map = parsers.dict_new_presized(len(st) // 71)
# This heuristic is imperfect in many ways, so in a future dirstate
# format update it makes sense to just record the number of entries
# on write.
self._map = parsers.dict_new_presized(
len(st) // ((2 * self._nodelen + 45) * 4 // 5)
)
# Python's garbage collector triggers a GC each time a certain number # Python's garbage collector triggers a GC each time a certain number
# of container objects (the number being defined by # of container objects (the number being defined by
# gc.get_threshold()) are allocated. parse_dirstate creates a tuple # gc.get_threshold()) are allocated. parse_dirstate creates a tuple
# for each file in the dirstate. The C version then immediately marks # for each file in the dirstate. The C version then immediately marks
# them as not to be tracked by the collector. However, this has no # them as not to be tracked by the collector. However, this has no
# effect on when GCs are triggered, only on what objects the GC looks # effect on when GCs are triggered, only on what objects the GC looks
# into. This means that O(number of files) GCs are unavoidable. # into. This means that O(number of files) GCs are unavoidable.