This is an archive of the discontinued Mercurial Phabricator instance.

Differential D9236

utils: helper function to print top memory allocation site
ClosedPublic

Authored by joerg.sonnenberger on Oct 21 2020, 4:30 PM.

Details

Reviewers
None
Group Reviewers
hg-reviewers
Commits
rHG5b6c0af021da: utils: helper function to print top memory allocation site
rHG4f50d0303557: utils: helper function to print top memory allocation site
Summary

The memorytop function uses Python's tracemalloc module to show the
source lines / backtraces with the largest remaining allocations. This
allows identifying the origins of active memory by placing calls in
strategic locations. Allocations from C extensions will show up as long
as they are using the Python allocators.

Diff Detail

Repository
rHG Mercurial
Lint
Automatic diff as part of commit; lint not applicable.
Unit
Automatic diff as part of commit; unit tests not applicable.

Revision Contents
Changeset List

PathPackages
AMmercurial/utils/memorytop.py (44 lines)

Diff 23372

mercurial/utils/memorytop.py

  • This file was added.
# memorytop requires Python 3.4
#
# Usage: set PYTHONTRACEMALLOC=n in the environment of the hg invocation,
# where n>= is the number of frames to show in the backtrace. Put calls to
# memorytop in strategic places to show the current memory use by allocation
# site.
import tracemalloc
import gc
def memorytop(limit=10):
gc.collect()
snapshot = tracemalloc.take_snapshot()
snapshot = snapshot.filter_traces(
(
tracemalloc.Filter(False, "<frozen importlib._bootstrap>"),
tracemalloc.Filter(False, "<frozen importlib._bootstrap_external>"),
tracemalloc.Filter(False, "<unknown>"),
)
)
stats = snapshot.statistics('traceback')
total = sum(stat.size for stat in stats)
print("\nTotal allocated size: %.1f KiB\n" % (total / 1024))
print("Lines with the biggest net allocations")
for index, stat in enumerate(stats[:limit], 1):
print(
"#%d: %d objects using %.1f KiB"
% (index, stat.count, stat.size / 1024)
)
for line in stat.traceback.format(most_recent_first=True):
print(' ', line)
other = stats[limit:]
if other:
size = sum(stat.size for stat in other)
count = sum(stat.count for stat in other)
print(
"%s other: %d objects using %.1f KiB"
% (len(other), count, size / 1024)
)
print()