Stowage/cpython
2
0
Fork 0
mirror of https://github.com/python/cpython.git synced 2026-02-07 02:10:37 +00:00
Code Issues Projects Releases Packages Wiki Activity

gh-144319: Fix huge page safety in pymalloc arenas (#144331)

Browse source 
The pymalloc huge page support had two problems. First, on
architectures where the default huge page size exceeds the arena
size (e.g. 32 MiB on PPC, 512 MiB on ARM64 with 64 KB base
pages), mmap with MAP_HUGETLB silently allocates a full huge page
even when the requested size is smaller. The subsequent munmap
with the original arena size then fails with EINVAL, permanently
leaking the entire huge page. Second, huge pages were always
attempted when compiled in, with no way to disable them at
runtime. On Linux, if the huge page pool is exhausted, page
faults including copy-on-write faults after fork deliver SIGBUS
and kill the process.

The arena allocator now queries the system huge page size from
/proc/meminfo and skips MAP_HUGETLB when the arena size is not a
multiple of it. Huge pages also now require explicit opt-in at
runtime via the PYTHON_PYMALLOC_HUGEPAGES environment variable,
which is read through PyConfig and respects -E and -I flags.
The config field pymalloc_hugepages is propagated to the runtime
allocators struct so the low-level arena allocator can check it
without calling getenv directly.
This commit is contained in:
Pablo Galindo Salgado 2026-01-30 18:18:56 +00:00 • committed by GitHub
parent a01694dacd
commit 96e4cd698a
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
10 changed files with 120 additions and 11 deletions
Download patch file Download diff file Expand all files Collapse all files

18
Python/initconfig.c
View file

@ -160,6 +160,7 @@ static const PyConfigSpec PYCONFIG_SPEC[] = {
SPEC(legacy_windows_stdio, BOOL, READ_ONLY, NO_SYS),
#endif
SPEC(malloc_stats, BOOL, READ_ONLY, NO_SYS),
SPEC(pymalloc_hugepages, BOOL, READ_ONLY, NO_SYS),
SPEC(orig_argv, WSTR_LIST, READ_ONLY, SYS_ATTR("orig_argv")),
SPEC(parse_argv, BOOL, READ_ONLY, NO_SYS),
SPEC(pathconfig_warnings, BOOL, READ_ONLY, NO_SYS),
@ -900,6 +901,7 @@ config_check_consistency(const PyConfig *config)
assert(config->show_ref_count >= 0);
assert(config->dump_refs >= 0);
assert(config->malloc_stats >= 0);
assert(config->pymalloc_hugepages >= 0);
assert(config->site_import >= 0);
assert(config->bytes_warning >= 0);
assert(config->warn_default_encoding >= 0);
@ -1879,6 +1881,18 @@ config_read_env_vars(PyConfig *config)
if (config_get_env(config, "PYTHONMALLOCSTATS")) {
config->malloc_stats = 1;
}
{
const char *env = _Py_GetEnv(use_env, "PYTHON_PYMALLOC_HUGEPAGES");
if (env) {
int value;
if (_Py_str_to_int(env, &value) < 0 || value < 0) {
/* PYTHON_PYMALLOC_HUGEPAGES=text or negative
behaves as PYTHON_PYMALLOC_HUGEPAGES=1 */
value = 1;
}
config->pymalloc_hugepages = (value > 0);
}
}
if (config->dump_refs_file == NULL) {
status = CONFIG_GET_ENV_DUP(config, &config->dump_refs_file,
@ -2812,6 +2826,10 @@ _PyConfig_Write(const PyConfig *config, _PyRuntimeState *runtime)
return _PyStatus_NO_MEMORY();
}
#ifdef PYMALLOC_USE_HUGEPAGES
runtime->allocators.use_hugepages = config->pymalloc_hugepages;
#endif
return _PyStatus_OK();
}