This is an archive of the discontinued Mercurial Phabricator instance.

Differential D8644

util: provide a helper function to estimate RAM size
ClosedPublic

Authored by joerg.sonnenberger on Jun 21 2020, 6:40 PM.

Details

Reviewers
pulkit
Group Reviewers
hg-reviewers
Commits
rHG02b17231f6c3: util: provide a helper function to estimate RAM size
rHGcfe0f4915793: util: provide a helper function to estimate RAM size
Summary

For POSIX systems, it uses sysconf. For Windows, it uses the win32 API
directly.

Diff Detail

Repository
rHG Mercurial
Lint
Automatic diff as part of commit; lint not applicable.
Unit
Automatic diff as part of commit; unit tests not applicable.

Event Timeline

joerg.sonnenberger added a comment.Jun 21 2020, 6:41 PM

This currently triggers a warning about using ui in util, so consider this a RFC for dealing with that.

pulkit added a subscriber: pulkit.Jul 7 2020, 2:08 PM
pulkit added inline comments.
mercurial/ui.py
2125

nit: can fail by returning None -> if we fails to calculate the memory, None is returned.

mercurial/util.py
3607

Can we move this try-expect in a if not windows: and also some comments on errors caught as why they are caught?

pulkit accepted this revision.Jul 10 2020, 6:02 AM
This revision is now accepted and ready to land.Jul 10 2020, 6:02 AM

Revision Contents
Changeset List

PathPackages
Mmercurial/configitems.py (4 lines)
Mmercurial/ui.py (16 lines)
Mmercurial/util.py (41 lines)

Diff 21824

mercurial/configitems.py

) )
coreconfigitem( coreconfigitem(
b'ui', b'archivemeta', default=True, b'ui', b'archivemeta', default=True,
) )
coreconfigitem( coreconfigitem(
b'ui', b'askusername', default=False, b'ui', b'askusername', default=False,
) )
coreconfigitem( coreconfigitem(
b'ui', b'available-memory', default=None,
)
coreconfigitem(
b'ui', b'clonebundlefallback', default=False, b'ui', b'clonebundlefallback', default=False,
) )
coreconfigitem( coreconfigitem(
b'ui', b'clonebundleprefers', default=list, b'ui', b'clonebundleprefers', default=list,
) )
coreconfigitem( coreconfigitem(
b'ui', b'clonebundles', default=True, b'ui', b'clonebundles', default=True,
) )

mercurial/ui.py

finally: finally:
for __, backup in backups.items(): for __, backup in backups.items():
self.restoreconfig(backup) self.restoreconfig(backup)
# just restoring ui.quiet config to the previous value is not enough # just restoring ui.quiet config to the previous value is not enough
# as it does not update ui.quiet class member # as it does not update ui.quiet class member
if (b'ui', b'quiet') in overrides: if (b'ui', b'quiet') in overrides:
self.fixconfig(section=b'ui') self.fixconfig(section=b'ui')
def estimatememory(self):
"""Provide an estimate for the available system memory in Bytes.
This can be overriden via ui.available-memory. It returns None, if
pulkitUnsubmitted
Not Done

nit: can fail by returning None -> if we fails to calculate the memory, None is returned.

pulkit: nit: `can fail by returning None` -> `if we fails to calculate the memory, None is returned.`
no estimate can be computed.
"""
value = self.config(b'ui', b'available-memory')
if value is not None:
try:
return util.sizetoint(value)
except error.ParseError:
raise error.ConfigError(
_(b"ui.available-memory value is invalid ('%s')") % value
)
return util._estimatememory()
class paths(dict): class paths(dict):
"""Represents a collection of paths and their configs. """Represents a collection of paths and their configs.
Data is initially derived from ui instances and the config files they have Data is initially derived from ui instances and the config files they have
loaded. loaded.
""" """

mercurial/util.py

shift += 7 shift += 7
# Passing the '' locale means that the locale should be set according to the # Passing the '' locale means that the locale should be set according to the
# user settings (environment variables). # user settings (environment variables).
# Python sometimes avoids setting the global locale settings. When interfacing # Python sometimes avoids setting the global locale settings. When interfacing
# with C code (e.g. the curses module or the Subversion bindings), the global # with C code (e.g. the curses module or the Subversion bindings), the global
# locale settings must be initialized correctly. Python 2 does not initialize # locale settings must be initialized correctly. Python 2 does not initialize
# the global locale settings on interpreter startup. Python 3 sometimes # the global locale settings on interpreter startup. Python 3 sometimes
pulkitUnsubmitted
Not Done

Can we move this try-expect in a if not windows: and also some comments on errors caught as why they are caught?

pulkit: Can we move this `try`-`expect` in a `if not windows:` and also some comments on errors caught…
# initializes LC_CTYPE, but not consistently at least on Windows. Therefore we # initializes LC_CTYPE, but not consistently at least on Windows. Therefore we
# explicitly initialize it to get consistent behavior if it's not already # explicitly initialize it to get consistent behavior if it's not already
# initialized. Since CPython commit 177d921c8c03d30daa32994362023f777624b10d, # initialized. Since CPython commit 177d921c8c03d30daa32994362023f777624b10d,
# LC_CTYPE is always initialized. If we require Python 3.8+, we should re-check # LC_CTYPE is always initialized. If we require Python 3.8+, we should re-check
# if we can remove this code. # if we can remove this code.
@contextlib.contextmanager @contextlib.contextmanager
def with_lc_ctype(): def with_lc_ctype():
oldloc = locale.setlocale(locale.LC_CTYPE, None) oldloc = locale.setlocale(locale.LC_CTYPE, None)
if oldloc == 'C': if oldloc == 'C':
try: try:
try: try:
locale.setlocale(locale.LC_CTYPE, '') locale.setlocale(locale.LC_CTYPE, '')
except locale.Error: except locale.Error:
# The likely case is that the locale from the environment # The likely case is that the locale from the environment
# variables is unknown. # variables is unknown.
pass pass
yield yield
finally: finally:
locale.setlocale(locale.LC_CTYPE, oldloc) locale.setlocale(locale.LC_CTYPE, oldloc)
else: else:
yield yield
def _estimatememory():
"""Provide an estimate for the available system memory in Bytes.
If no estimate can be provided on the platform, returns None.
"""
if pycompat.sysplatform.startswith(b'win'):
# On Windows, use the GlobalMemoryStatusEx kernel function directly.
from ctypes import c_long as DWORD, c_ulonglong as DWORDLONG
from ctypes.wintypes import Structure, byref, sizeof, windll
class MEMORYSTATUSEX(Structure):
_fields_ = [
('dwLength', DWORD),
('dwMemoryLoad', DWORD),
('ullTotalPhys', DWORDLONG),
('ullAvailPhys', DWORDLONG),
('ullTotalPageFile', DWORDLONG),
('ullAvailPageFile', DWORDLONG),
('ullTotalVirtual', DWORDLONG),
('ullAvailVirtual', DWORDLONG),
('ullExtendedVirtual', DWORDLONG),
]
x = MEMORYSTATUSEX()
x.dwLength = sizeof(x)
windll.kernel32.GlobalMemoryStatusEx(byref(x))
return x.ullAvailPhys
# On newer Unix-like systems and Mac OSX, the sysconf interface
# can be used. _SC_PAGE_SIZE is part of POSIX; _SC_PHYS_PAGES
# seems to be implemented on most systems.
try:
pagesize = os.sysconf(os.sysconf_names['SC_PAGE_SIZE'])
pages = os.sysconf(os.sysconf_names['SC_PHYS_PAGES'])
return pagesize * pages
except OSError: # sysconf can fail
pass
except KeyError: # unknown parameter
pass