Implement more of PEP 810

This commit is contained in:
Pablo Galindo 2025-12-06 15:21:17 +00:00
parent b743eb03d8
commit e6cb131a53
16 changed files with 849 additions and 355 deletions

View file

@ -911,6 +911,43 @@ always available. Unless explicitly noted otherwise, all variables are read-only
.. versionadded:: 3.11 .. versionadded:: 3.11
.. function:: get_lazy_imports()
Returns the current lazy imports mode as a string.
* ``"normal"``: Only imports explicitly marked with the ``lazy`` keyword are lazy
* ``"all"``: All top-level imports are potentially lazy
* ``"none"``: All lazy imports are suppressed (even explicitly marked ones)
See also :func:`set_lazy_imports` and :pep:`810`.
.. versionadded:: 3.15
.. function:: get_lazy_imports_filter()
Returns the current lazy imports filter function, or ``None`` if no filter
is set.
The filter function is called for every potentially lazy import to determine
whether it should actually be lazy. See :func:`set_lazy_imports_filter` for
details on the filter function signature.
.. versionadded:: 3.15
.. function:: get_lazy_modules()
Returns a set of fully-qualified module names that have been lazily imported.
This is primarily useful for diagnostics and introspection.
Note that modules are removed from this set when they are reified (actually
loaded on first use).
.. versionadded:: 3.15
.. function:: getrefcount(object) .. function:: getrefcount(object)
Return the reference count of the *object*. The count returned is generally one Return the reference count of the *object*. The count returned is generally one
@ -1715,6 +1752,57 @@ always available. Unless explicitly noted otherwise, all variables are read-only
.. versionadded:: 3.11 .. versionadded:: 3.11
.. function:: set_lazy_imports(mode)
Sets the global lazy imports mode. The *mode* parameter must be one of the
following strings:
* ``"normal"``: Only imports explicitly marked with the ``lazy`` keyword are lazy
* ``"all"``: All top-level imports become potentially lazy
* ``"none"``: All lazy imports are suppressed (even explicitly marked ones)
This function is intended for advanced users who need to control lazy imports
across their entire application. Library developers should generally not use
this function as it affects the runtime execution of applications.
In addition to the mode, lazy imports can be controlled via the filter
provided by :func:`set_lazy_imports_filter`.
See also :func:`get_lazy_imports` and :pep:`810`.
.. versionadded:: 3.15
.. function:: set_lazy_imports_filter(filter)
Sets the lazy imports filter callback. The *filter* parameter must be a
callable or ``None`` to clear the filter.
The filter function is called for every potentially lazy import to determine
whether it should actually be lazy. It must have the following signature::
def filter(importing_module: str, imported_module: str,
fromlist: tuple[str, ...] | None) -> bool
Where:
* *importing_module* is the name of the module doing the import
* *imported_module* is the name of the module being imported
* *fromlist* is the tuple of names being imported (for ``from ... import``
statements), or ``None`` for regular imports
The filter should return ``True`` to allow the import to be lazy, or
``False`` to force an eager import.
This is an advanced feature intended for specialized users who need
fine-grained control over lazy import behavior.
See also :func:`get_lazy_imports_filter` and :pep:`810`.
.. versionadded:: 3.15
.. function:: setprofile(profilefunc) .. function:: setprofile(profilefunc)
.. index:: .. index::

View file

@ -65,6 +65,8 @@ Summary -- Release highlights
.. PEP-sized items next. .. PEP-sized items next.
* :pep:`810`: :ref:`Explicit lazy imports for faster startup times
<whatsnew315-pep810>`
* :pep:`799`: :ref:`A dedicated profiling package for organizing Python * :pep:`799`: :ref:`A dedicated profiling package for organizing Python
profiling tools <whatsnew315-sampling-profiler>` profiling tools <whatsnew315-sampling-profiler>`
* :pep:`686`: :ref:`Python now uses UTF-8 as the default encoding * :pep:`686`: :ref:`Python now uses UTF-8 as the default encoding
@ -77,6 +79,97 @@ Summary -- Release highlights
New features New features
============ ============
.. _whatsnew315-pep810:
:pep:`810`: Explicit lazy imports
---------------------------------
Large Python applications often suffer from slow startup times. A significant
contributor to this problem is the import system: when a module is imported,
Python must locate the file, read it from disk, compile it to bytecode, and
execute all top-level code. For applications with deep dependency trees, this
process can take seconds, even when most of the imported code is never actually
used during a particular run.
Developers have worked around this by moving imports inside functions, using
``importlib`` to load modules on demand, or restructuring code to avoid
unnecessary dependencies. These approaches work but make code harder to read
and maintain, scatter import statements throughout the codebase, and require
discipline to apply consistently.
Python now provides a cleaner solution through explicit lazy imports using the
new ``lazy`` soft keyword. When you mark an import as lazy, Python defers the
actual module loading until the imported name is first used. This gives you
the organizational benefits of declaring all imports at the top of the file
while only paying the loading cost for modules you actually use.
The ``lazy`` keyword works with both ``import`` and ``from ... import`` statements.
When you write ``lazy import heavy_module``, Python does not immediately load the
module. Instead, it creates a lightweight proxy object. The actual module loading
happens transparently when you first access the name:
.. code-block:: python
lazy import json
lazy from datetime import datetime
print("Starting up...") # json and datetime not loaded yet
data = json.loads('{"key": "value"}') # json loads here
now = datetime() # datetime loads here
This mechanism is particularly useful for applications that import many modules
at the top level but may only use a subset of them in any given run. The deferred
loading reduces startup latency without requiring code restructuring or conditional
imports scattered throughout the codebase.
When a lazy import eventually fails (for example, if the module does not exist),
Python raises the exception at the point of first use rather than at import time.
The traceback includes both the location where the name was accessed and the
original import statement, making it straightforward to diagnose the problem.
For cases where you want to enable lazy loading globally without modifying source
code, Python provides the :option:`-X lazy_imports <-X>` command-line option and
the :envvar:`PYTHON_LAZY_IMPORTS` environment variable. Both accept three values:
``all`` makes all imports lazy by default, ``none`` disables lazy imports entirely
(even explicit ``lazy`` statements become eager), and ``normal`` (the default)
respects the ``lazy`` keyword in source code. The :func:`sys.set_lazy_imports` and
:func:`sys.get_lazy_imports` functions allow changing and querying this mode at
runtime.
For more selective control, :func:`sys.set_lazy_imports_filter` accepts a callable
that determines whether a specific module should be loaded lazily. The filter
receives the fully-qualified module name and returns a boolean. This allows
patterns like making only your own application's modules lazy while keeping
third-party dependencies eager:
.. code-block:: python
import sys
sys.set_lazy_imports_filter(lambda name: name.startswith("myapp."))
sys.set_lazy_imports("all")
import myapp.slow_module # lazy (matches filter)
import json # eager (does not match filter)
For debugging and introspection, :func:`sys.get_lazy_modules` returns a set
containing the names of all modules that have been lazily imported but not yet
loaded. The proxy type itself is available as :data:`types.LazyImportType` for
code that needs to detect lazy imports programmatically.
There are some restrictions on where ``lazy`` can appear. Lazy imports are only
permitted at module scope; using ``lazy`` inside a function, class body, or
``try``/``except``/``finally`` block raises a :exc:`SyntaxError`. Star imports
cannot be lazy (``lazy from module import *`` is a syntax error), and future
imports cannot be lazy either (``lazy from __future__ import ...`` raises
:exc:`SyntaxError`).
.. seealso:: :pep:`810` for the full specification and rationale.
(Contributed by Pablo Galindo Salgado and Dino Viehland.)
.. _whatsnew315-sampling-profiler: .. _whatsnew315-sampling-profiler:
:pep:`799`: High frequency statistical sampling profiler :pep:`799`: High frequency statistical sampling profiler

View file

@ -318,6 +318,7 @@ struct _import_state {
PyObject *lazy_imports_filter; PyObject *lazy_imports_filter;
PyObject *lazy_importing_modules; PyObject *lazy_importing_modules;
PyObject *lazy_modules; PyObject *lazy_modules;
PyObject *lazy_modules_set; /* Set of fully-qualified module names lazily imported (PEP 810) */
/* The global import lock. */ /* The global import lock. */
_PyRecursiveMutex lock; _PyRecursiveMutex lock;
/* diagnostic info in PyImport_ImportModuleLevelObject() */ /* diagnostic info in PyImport_ImportModuleLevelObject() */

View file

@ -2571,302 +2571,6 @@ def test_disallowed_reimport(self):
self.assertIsNot(excsnap, None) self.assertIsNot(excsnap, None)
class LazyImportTests(unittest.TestCase):
def tearDown(self):
"""Make sure no modules pre-exist in sys.modules which are being used to
test."""
for key in list(sys.modules.keys()):
if key.startswith('test.test_import.data.lazy_imports'):
del sys.modules[key]
sys.set_lazy_imports_filter(None)
sys.set_lazy_imports(None)
def test_basic_unused(self):
try:
import test.test_import.data.lazy_imports.basic_unused
except ImportError as e:
self.fail('lazy import failed')
self.assertFalse("test.test_import.data.lazy_imports.basic2" in sys.modules)
def test_basic_unused_use_externally(self):
try:
from test.test_import.data.lazy_imports import basic_unused
except ImportError as e:
self.fail('lazy import failed')
self.assertFalse("test.test_import.data.lazy_imports.basic2" in sys.modules)
x = basic_unused.test.test_import.data.lazy_imports.basic2
self.assertTrue("test.test_import.data.lazy_imports.basic2" in sys.modules)
def test_basic_from_unused_use_externally(self):
try:
from test.test_import.data.lazy_imports import basic_from_unused
except ImportError as e:
self.fail('lazy import failed')
self.assertFalse("test.test_import.data.lazy_imports.basic2" in sys.modules)
x = basic_from_unused.basic2
self.assertTrue("test.test_import.data.lazy_imports.basic2" in sys.modules)
def test_basic_unused_dir(self):
try:
import test.test_import.data.lazy_imports.basic_unused
except ImportError as e:
self.fail('lazy import failed')
x = dir(test.test_import.data.lazy_imports.basic_unused)
self.assertIn("test", x)
self.assertFalse("test.test_import.data.lazy_imports.basic2" in sys.modules)
def test_basic_dir(self):
try:
from test.test_import.data.lazy_imports import basic_dir
except ImportError as e:
self.fail('lazy import failed')
self.assertIn("test", basic_dir.x)
self.assertFalse("test.test_import.data.lazy_imports.basic2" in sys.modules)
def test_basic_used(self):
try:
import test.test_import.data.lazy_imports.basic_used
except ImportError as e:
self.fail('lazy import failed')
self.assertTrue("test.test_import.data.lazy_imports.basic2" in sys.modules)
def test_global_off(self):
try:
import test.test_import.data.lazy_imports.global_off
except ImportError as e:
self.fail('lazy import failed')
self.assertTrue("test.test_import.data.lazy_imports.basic2" in sys.modules)
def test_global_on(self):
try:
import test.test_import.data.lazy_imports.global_on
except ImportError as e:
self.fail('lazy import failed')
self.assertFalse("test.test_import.data.lazy_imports.basic2" in sys.modules)
def test_global_filter(self):
try:
import test.test_import.data.lazy_imports.global_filter
except ImportError as e:
self.fail('lazy import failed')
self.assertTrue("test.test_import.data.lazy_imports.basic2" in sys.modules)
def test_global_filter_true(self):
try:
import test.test_import.data.lazy_imports.global_filter_true
except ImportError as e:
self.fail('lazy import failed')
self.assertFalse("test.test_import.data.lazy_imports.basic2" in sys.modules)
def test_global_filter_from(self):
try:
import test.test_import.data.lazy_imports.global_filter
except ImportError as e:
self.fail('lazy import failed')
self.assertTrue("test.test_import.data.lazy_imports.basic2" in sys.modules)
def test_global_filter_from_true(self):
try:
import test.test_import.data.lazy_imports.global_filter_true
except ImportError as e:
self.fail('lazy import failed')
self.assertFalse("test.test_import.data.lazy_imports.basic2" in sys.modules)
def test_compatibility_mode(self):
try:
import test.test_import.data.lazy_imports.basic_compatibility_mode
except ImportError as e:
self.fail('lazy import failed')
self.assertFalse("test.test_import.data.lazy_imports.basic2" in sys.modules)
def test_compatibility_mode_used(self):
try:
import test.test_import.data.lazy_imports.basic_compatibility_mode_used
except ImportError as e:
self.fail('lazy import failed')
self.assertTrue("test.test_import.data.lazy_imports.basic2" in sys.modules)
def test_compatibility_mode_func(self):
try:
import test.test_import.data.lazy_imports.compatibility_mode_func
except ImportError as e:
self.fail('lazy import failed')
self.assertTrue("test.test_import.data.lazy_imports.basic2" in sys.modules)
def test_compatibility_mode_try_except(self):
try:
import test.test_import.data.lazy_imports.compatibility_mode_try_except
except ImportError as e:
self.fail('lazy import failed')
self.assertTrue("test.test_import.data.lazy_imports.basic2" in sys.modules)
def test_compatibility_mode_relative(self):
try:
import test.test_import.data.lazy_imports.basic_compatibility_mode_relative
except ImportError as e:
self.fail('lazy import failed')
self.assertFalse("test.test_import.data.lazy_imports.basic2" in sys.modules)
def test_modules_dict(self):
try:
import test.test_import.data.lazy_imports.modules_dict
except ImportError as e:
self.fail('lazy import failed')
self.assertFalse("test.test_import.data.lazy_imports.basic2" in sys.modules)
def test_modules_geatattr(self):
try:
import test.test_import.data.lazy_imports.modules_getattr
except ImportError as e:
self.fail('lazy import failed')
self.assertTrue("test.test_import.data.lazy_imports.basic2" in sys.modules)
def test_modules_geatattr_other(self):
try:
import test.test_import.data.lazy_imports.modules_getattr_other
except ImportError as e:
self.fail('lazy import failed')
self.assertFalse("test.test_import.data.lazy_imports.basic2" in sys.modules)
def test_lazy_value_get(self):
try:
import test.test_import.data.lazy_imports.lazy_get_value
except ImportError as e:
self.fail('lazy import failed')
self.assertTrue("test.test_import.data.lazy_imports.basic2" in sys.modules)
def test_lazy_try_except(self):
with self.assertRaises(SyntaxError):
import test.test_import.data.lazy_imports.lazy_try_except
def test_lazy_try_except_from(self):
with self.assertRaises(SyntaxError):
import test.test_import.data.lazy_imports.lazy_try_except_from
def test_lazy_try_except_from_star(self):
with self.assertRaises(SyntaxError):
import test.test_import.data.lazy_imports.lazy_try_except_from_star
def test_try_except_eager(self):
sys.set_lazy_imports(True)
try:
import test.test_import.data.lazy_imports.try_except_eager
except ImportError as e:
self.fail('lazy import failed')
self.assertTrue("test.test_import.data.lazy_imports.basic2" in sys.modules)
def test_try_except_eager_from(self):
sys.set_lazy_imports(True)
try:
import test.test_import.data.lazy_imports.try_except_eager_from
except ImportError as e:
self.fail('lazy import failed')
self.assertTrue("test.test_import.data.lazy_imports.basic2" in sys.modules)
def test_lazy_with(self):
try:
import test.test_import.data.lazy_imports.lazy_with
except ImportError as e:
self.fail('lazy import failed')
self.assertFalse("test.test_import.data.lazy_imports.basic2" in sys.modules)
def test_lazy_with_from(self):
try:
import test.test_import.data.lazy_imports.lazy_with_from
except ImportError as e:
self.fail('lazy import failed')
self.assertFalse("test.test_import.data.lazy_imports.basic2" in sys.modules)
def test_lazy_import_func(self):
with self.assertRaises(SyntaxError):
import test.test_import.data.lazy_imports.lazy_import_func
def test_eager_import_func(self):
sys.set_lazy_imports(True)
try:
import test.test_import.data.lazy_imports.eager_import_func
except ImportError as e:
self.fail('lazy import failed')
f = test.test_import.data.lazy_imports.eager_import_func.f
self.assertEqual(type(f()), type(sys))
def test_lazy_import_pkg(self):
try:
import test.test_import.data.lazy_imports.lazy_import_pkg
except ImportError as e:
self.fail('lazy import failed')
self.assertTrue("test.test_import.data.lazy_imports.pkg" in sys.modules)
self.assertTrue("test.test_import.data.lazy_imports.pkg.bar" in sys.modules)
def test_lazy_import_pkg_cross_import(self):
try:
import test.test_import.data.lazy_imports.pkg.c
except ImportError as e:
self.fail('lazy import failed')
self.assertTrue("test.test_import.data.lazy_imports.pkg" in sys.modules)
self.assertTrue("test.test_import.data.lazy_imports.pkg.c" in sys.modules)
self.assertFalse("test.test_import.data.lazy_imports.pkg.b" in sys.modules)
g = test.test_import.data.lazy_imports.pkg.c.get_globals()
self.assertEqual(type(g["x"]), int)
self.assertEqual(type(g["b"]), types.LazyImportType)
def test_dunder_lazy_import(self):
try:
import test.test_import.data.lazy_imports.dunder_lazy_import
except ImportError as e:
self.fail('lazy import failed')
self.assertFalse("test.test_import.data.lazy_imports.basic2" in sys.modules)
def test_dunder_lazy_import_used(self):
try:
import test.test_import.data.lazy_imports.dunder_lazy_import_used
except ImportError as e:
self.fail('lazy import failed')
self.assertTrue("test.test_import.data.lazy_imports.basic2" in sys.modules)
def test_dunder_lazy_import_builtins(self):
"""__lazy_import__ uses modules __builtins__ to get __import__"""
try:
from test.test_import.data.lazy_imports import dunder_lazy_import_builtins
except ImportError as e:
self.fail('lazy import failed')
self.assertFalse("test.test_import.data.lazy_imports.basic2" in sys.modules)
self.assertEqual(dunder_lazy_import_builtins.basic, 42)
class TestSinglePhaseSnapshot(ModuleSnapshot): class TestSinglePhaseSnapshot(ModuleSnapshot):
"""A representation of a single-phase init module for testing. """A representation of a single-phase init module for testing.

View file

@ -5,7 +5,7 @@ def filter(module_name, imported_name, from_list):
assert imported_name == "test.test_import.data.lazy_imports.basic2" assert imported_name == "test.test_import.data.lazy_imports.basic2"
return True return True
sys.set_lazy_imports(None) sys.set_lazy_imports("normal")
sys.set_lazy_imports_filter(filter) sys.set_lazy_imports_filter(filter)
lazy import test.test_import.data.lazy_imports.basic2 as basic2 lazy import test.test_import.data.lazy_imports.basic2 as basic2

View file

@ -1,5 +1,5 @@
import sys import sys
sys.set_lazy_imports(False) sys.set_lazy_imports("none")
lazy import test.test_import.data.lazy_imports.basic2 as basic2 lazy import test.test_import.data.lazy_imports.basic2 as basic2

View file

@ -1,5 +1,5 @@
import sys import sys
sys.set_lazy_imports(True) sys.set_lazy_imports("all")
import test.test_import.data.lazy_imports.basic2 as basic2 import test.test_import.data.lazy_imports.basic2 as basic2

View file

@ -0,0 +1 @@
lazy from __future__ import annotations

View file

@ -2,6 +2,6 @@
def f(): def f():
x = globals() x = globals()
return x['basic2'].get() return x['basic2'].resolve()
f() f()

View file

@ -0,0 +1,524 @@
"""Tests for PEP 810 lazy imports."""
import sys
import types
import unittest
import threading
import textwrap
import subprocess
from test.support import import_helper
class LazyImportTests(unittest.TestCase):
"""Tests for basic lazy import functionality."""
def tearDown(self):
"""Clean up any test modules from sys.modules."""
for key in list(sys.modules.keys()):
if key.startswith('test.test_import.data.lazy_imports'):
del sys.modules[key]
sys.set_lazy_imports_filter(None)
sys.set_lazy_imports("normal")
def test_basic_unused(self):
"""Lazy imported module should not be loaded if never accessed."""
import test.test_import.data.lazy_imports.basic_unused
self.assertFalse("test.test_import.data.lazy_imports.basic2" in sys.modules)
def test_basic_unused_use_externally(self):
"""Lazy import should load module when accessed from outside."""
from test.test_import.data.lazy_imports import basic_unused
self.assertFalse("test.test_import.data.lazy_imports.basic2" in sys.modules)
x = basic_unused.test.test_import.data.lazy_imports.basic2
self.assertTrue("test.test_import.data.lazy_imports.basic2" in sys.modules)
def test_basic_from_unused_use_externally(self):
"""Lazy 'from' import should load when accessed from outside."""
from test.test_import.data.lazy_imports import basic_from_unused
self.assertFalse("test.test_import.data.lazy_imports.basic2" in sys.modules)
x = basic_from_unused.basic2
self.assertTrue("test.test_import.data.lazy_imports.basic2" in sys.modules)
def test_basic_unused_dir(self):
"""dir() on module should not trigger lazy import reification."""
import test.test_import.data.lazy_imports.basic_unused
x = dir(test.test_import.data.lazy_imports.basic_unused)
self.assertIn("test", x)
self.assertFalse("test.test_import.data.lazy_imports.basic2" in sys.modules)
def test_basic_dir(self):
"""dir() at module scope should not trigger lazy import reification."""
from test.test_import.data.lazy_imports import basic_dir
self.assertIn("test", basic_dir.x)
self.assertFalse("test.test_import.data.lazy_imports.basic2" in sys.modules)
def test_basic_used(self):
"""Lazy import should load when accessed within the module."""
import test.test_import.data.lazy_imports.basic_used
self.assertTrue("test.test_import.data.lazy_imports.basic2" in sys.modules)
class GlobalLazyImportModeTests(unittest.TestCase):
"""Tests for sys.set_lazy_imports() global mode control."""
def tearDown(self):
for key in list(sys.modules.keys()):
if key.startswith('test.test_import.data.lazy_imports'):
del sys.modules[key]
sys.set_lazy_imports_filter(None)
sys.set_lazy_imports("normal")
def test_global_off(self):
"""Mode 'none' should disable lazy imports entirely."""
import test.test_import.data.lazy_imports.global_off
self.assertTrue("test.test_import.data.lazy_imports.basic2" in sys.modules)
def test_global_on(self):
"""Mode 'all' should make regular imports lazy."""
import test.test_import.data.lazy_imports.global_on
self.assertFalse("test.test_import.data.lazy_imports.basic2" in sys.modules)
def test_global_filter(self):
"""Filter returning False should prevent lazy loading."""
import test.test_import.data.lazy_imports.global_filter
self.assertTrue("test.test_import.data.lazy_imports.basic2" in sys.modules)
def test_global_filter_true(self):
"""Filter returning True should allow lazy loading."""
import test.test_import.data.lazy_imports.global_filter_true
self.assertFalse("test.test_import.data.lazy_imports.basic2" in sys.modules)
def test_global_filter_from(self):
"""Filter should work with 'from' imports."""
import test.test_import.data.lazy_imports.global_filter
self.assertTrue("test.test_import.data.lazy_imports.basic2" in sys.modules)
def test_global_filter_from_true(self):
"""Filter returning True should allow lazy 'from' imports."""
import test.test_import.data.lazy_imports.global_filter_true
self.assertFalse("test.test_import.data.lazy_imports.basic2" in sys.modules)
class CompatibilityModeTests(unittest.TestCase):
"""Tests for __lazy_modules__ compatibility mode."""
def tearDown(self):
for key in list(sys.modules.keys()):
if key.startswith('test.test_import.data.lazy_imports'):
del sys.modules[key]
sys.set_lazy_imports_filter(None)
sys.set_lazy_imports("normal")
def test_compatibility_mode(self):
"""__lazy_modules__ should enable lazy imports for listed modules."""
import test.test_import.data.lazy_imports.basic_compatibility_mode
self.assertFalse("test.test_import.data.lazy_imports.basic2" in sys.modules)
def test_compatibility_mode_used(self):
"""Using a lazy import from __lazy_modules__ should load the module."""
import test.test_import.data.lazy_imports.basic_compatibility_mode_used
self.assertTrue("test.test_import.data.lazy_imports.basic2" in sys.modules)
def test_compatibility_mode_func(self):
"""Imports inside functions should be eager even in compatibility mode."""
import test.test_import.data.lazy_imports.compatibility_mode_func
self.assertTrue("test.test_import.data.lazy_imports.basic2" in sys.modules)
def test_compatibility_mode_try_except(self):
"""Imports in try/except should be eager even in compatibility mode."""
import test.test_import.data.lazy_imports.compatibility_mode_try_except
self.assertTrue("test.test_import.data.lazy_imports.basic2" in sys.modules)
def test_compatibility_mode_relative(self):
"""__lazy_modules__ should work with relative imports."""
import test.test_import.data.lazy_imports.basic_compatibility_mode_relative
self.assertFalse("test.test_import.data.lazy_imports.basic2" in sys.modules)
class ModuleIntrospectionTests(unittest.TestCase):
"""Tests for module dict and getattr behavior with lazy imports."""
def tearDown(self):
for key in list(sys.modules.keys()):
if key.startswith('test.test_import.data.lazy_imports'):
del sys.modules[key]
sys.set_lazy_imports_filter(None)
sys.set_lazy_imports("normal")
def test_modules_dict(self):
"""Accessing module.__dict__ should not trigger reification."""
import test.test_import.data.lazy_imports.modules_dict
self.assertFalse("test.test_import.data.lazy_imports.basic2" in sys.modules)
def test_modules_getattr(self):
"""Module __getattr__ for lazy import name should trigger reification."""
import test.test_import.data.lazy_imports.modules_getattr
self.assertTrue("test.test_import.data.lazy_imports.basic2" in sys.modules)
def test_modules_getattr_other(self):
"""Module __getattr__ for other names should not trigger reification."""
import test.test_import.data.lazy_imports.modules_getattr_other
self.assertFalse("test.test_import.data.lazy_imports.basic2" in sys.modules)
class LazyImportTypeTests(unittest.TestCase):
"""Tests for the LazyImportType and its resolve() method."""
def tearDown(self):
for key in list(sys.modules.keys()):
if key.startswith('test.test_import.data.lazy_imports'):
del sys.modules[key]
sys.set_lazy_imports_filter(None)
sys.set_lazy_imports("normal")
def test_lazy_value_resolve(self):
"""resolve() method should force the lazy import to load."""
import test.test_import.data.lazy_imports.lazy_get_value
self.assertTrue("test.test_import.data.lazy_imports.basic2" in sys.modules)
def test_lazy_import_type_exposed(self):
"""LazyImportType should be exposed in types module."""
self.assertTrue(hasattr(types, 'LazyImportType'))
self.assertEqual(types.LazyImportType.__name__, 'lazy_import')
class SyntaxRestrictionTests(unittest.TestCase):
"""Tests for syntax restrictions on lazy imports."""
def tearDown(self):
for key in list(sys.modules.keys()):
if key.startswith('test.test_import.data.lazy_imports'):
del sys.modules[key]
sys.set_lazy_imports_filter(None)
sys.set_lazy_imports("normal")
def test_lazy_try_except(self):
"""lazy import inside try/except should raise SyntaxError."""
with self.assertRaises(SyntaxError):
import test.test_import.data.lazy_imports.lazy_try_except
def test_lazy_try_except_from(self):
"""lazy from import inside try/except should raise SyntaxError."""
with self.assertRaises(SyntaxError):
import test.test_import.data.lazy_imports.lazy_try_except_from
def test_lazy_try_except_from_star(self):
"""lazy from import * should raise SyntaxError."""
with self.assertRaises(SyntaxError):
import test.test_import.data.lazy_imports.lazy_try_except_from_star
def test_lazy_future_import(self):
"""lazy from __future__ import should raise SyntaxError."""
with self.assertRaises(SyntaxError):
import test.test_import.data.lazy_imports.lazy_future_import
def test_lazy_import_func(self):
"""lazy import inside function should raise SyntaxError."""
with self.assertRaises(SyntaxError):
import test.test_import.data.lazy_imports.lazy_import_func
class EagerImportInLazyModeTests(unittest.TestCase):
"""Tests for imports that should remain eager even in lazy mode."""
def tearDown(self):
for key in list(sys.modules.keys()):
if key.startswith('test.test_import.data.lazy_imports'):
del sys.modules[key]
sys.set_lazy_imports_filter(None)
sys.set_lazy_imports("normal")
def test_try_except_eager(self):
"""Imports in try/except should be eager even with mode='all'."""
sys.set_lazy_imports("all")
import test.test_import.data.lazy_imports.try_except_eager
self.assertTrue("test.test_import.data.lazy_imports.basic2" in sys.modules)
def test_try_except_eager_from(self):
"""From imports in try/except should be eager even with mode='all'."""
sys.set_lazy_imports("all")
import test.test_import.data.lazy_imports.try_except_eager_from
self.assertTrue("test.test_import.data.lazy_imports.basic2" in sys.modules)
def test_eager_import_func(self):
"""Imports inside functions should return modules, not proxies."""
sys.set_lazy_imports("all")
import test.test_import.data.lazy_imports.eager_import_func
f = test.test_import.data.lazy_imports.eager_import_func.f
self.assertEqual(type(f()), type(sys))
class WithStatementTests(unittest.TestCase):
"""Tests for lazy imports in with statement context."""
def tearDown(self):
for key in list(sys.modules.keys()):
if key.startswith('test.test_import.data.lazy_imports'):
del sys.modules[key]
sys.set_lazy_imports_filter(None)
sys.set_lazy_imports("normal")
def test_lazy_with(self):
"""lazy import with 'with' statement should work."""
import test.test_import.data.lazy_imports.lazy_with
self.assertFalse("test.test_import.data.lazy_imports.basic2" in sys.modules)
def test_lazy_with_from(self):
"""lazy from import with 'with' statement should work."""
import test.test_import.data.lazy_imports.lazy_with_from
self.assertFalse("test.test_import.data.lazy_imports.basic2" in sys.modules)
class PackageTests(unittest.TestCase):
"""Tests for lazy imports with packages."""
def tearDown(self):
for key in list(sys.modules.keys()):
if key.startswith('test.test_import.data.lazy_imports'):
del sys.modules[key]
sys.set_lazy_imports_filter(None)
sys.set_lazy_imports("normal")
def test_lazy_import_pkg(self):
"""lazy import of package submodule should load the package."""
import test.test_import.data.lazy_imports.lazy_import_pkg
self.assertTrue("test.test_import.data.lazy_imports.pkg" in sys.modules)
self.assertTrue("test.test_import.data.lazy_imports.pkg.bar" in sys.modules)
def test_lazy_import_pkg_cross_import(self):
"""Cross-imports within package should preserve lazy imports."""
import test.test_import.data.lazy_imports.pkg.c
self.assertTrue("test.test_import.data.lazy_imports.pkg" in sys.modules)
self.assertTrue("test.test_import.data.lazy_imports.pkg.c" in sys.modules)
self.assertFalse("test.test_import.data.lazy_imports.pkg.b" in sys.modules)
g = test.test_import.data.lazy_imports.pkg.c.get_globals()
self.assertEqual(type(g["x"]), int)
self.assertEqual(type(g["b"]), types.LazyImportType)
class DunderLazyImportTests(unittest.TestCase):
"""Tests for __lazy_import__ builtin function."""
def tearDown(self):
for key in list(sys.modules.keys()):
if key.startswith('test.test_import.data.lazy_imports'):
del sys.modules[key]
sys.set_lazy_imports_filter(None)
sys.set_lazy_imports("normal")
def test_dunder_lazy_import(self):
"""__lazy_import__ should create lazy import proxy."""
import test.test_import.data.lazy_imports.dunder_lazy_import
self.assertFalse("test.test_import.data.lazy_imports.basic2" in sys.modules)
def test_dunder_lazy_import_used(self):
"""Using __lazy_import__ result should trigger module load."""
import test.test_import.data.lazy_imports.dunder_lazy_import_used
self.assertTrue("test.test_import.data.lazy_imports.basic2" in sys.modules)
def test_dunder_lazy_import_builtins(self):
"""__lazy_import__ should use module's __builtins__ for __import__."""
from test.test_import.data.lazy_imports import dunder_lazy_import_builtins
self.assertFalse("test.test_import.data.lazy_imports.basic2" in sys.modules)
self.assertEqual(dunder_lazy_import_builtins.basic, 42)
class SysLazyImportsAPITests(unittest.TestCase):
"""Tests for sys lazy imports API functions."""
def tearDown(self):
for key in list(sys.modules.keys()):
if key.startswith('test.test_import.data.lazy_imports'):
del sys.modules[key]
sys.set_lazy_imports_filter(None)
sys.set_lazy_imports("normal")
def test_set_lazy_imports_requires_string(self):
"""set_lazy_imports should reject non-string arguments."""
with self.assertRaises(TypeError):
sys.set_lazy_imports(True)
with self.assertRaises(TypeError):
sys.set_lazy_imports(None)
with self.assertRaises(TypeError):
sys.set_lazy_imports(1)
def test_set_lazy_imports_rejects_invalid_mode(self):
"""set_lazy_imports should reject invalid mode strings."""
with self.assertRaises(ValueError):
sys.set_lazy_imports("invalid")
with self.assertRaises(ValueError):
sys.set_lazy_imports("on")
with self.assertRaises(ValueError):
sys.set_lazy_imports("off")
def test_get_lazy_imports_returns_string(self):
"""get_lazy_imports should return string modes."""
sys.set_lazy_imports("normal")
self.assertEqual(sys.get_lazy_imports(), "normal")
sys.set_lazy_imports("all")
self.assertEqual(sys.get_lazy_imports(), "all")
sys.set_lazy_imports("none")
self.assertEqual(sys.get_lazy_imports(), "none")
def test_get_lazy_imports_filter_default(self):
"""get_lazy_imports_filter should return None by default."""
sys.set_lazy_imports_filter(None)
self.assertIsNone(sys.get_lazy_imports_filter())
def test_set_and_get_lazy_imports_filter(self):
"""set/get_lazy_imports_filter should round-trip filter function."""
def my_filter(name):
return name.startswith("test.")
sys.set_lazy_imports_filter(my_filter)
self.assertIs(sys.get_lazy_imports_filter(), my_filter)
def test_get_lazy_modules_returns_set(self):
"""get_lazy_modules should return a set per PEP 810."""
result = sys.get_lazy_modules()
self.assertIsInstance(result, set)
def test_lazy_modules_attribute_is_set(self):
"""sys.lazy_modules should be a set per PEP 810."""
self.assertIsInstance(sys.lazy_modules, set)
self.assertIs(sys.lazy_modules, sys.get_lazy_modules())
def test_lazy_modules_tracks_lazy_imports(self):
"""sys.lazy_modules should track lazily imported module names."""
code = textwrap.dedent("""
import sys
initial_count = len(sys.lazy_modules)
import test.test_import.data.lazy_imports.basic_unused
assert "test.test_import.data.lazy_imports.basic2" in sys.lazy_modules
assert len(sys.lazy_modules) > initial_count
print("OK")
""")
result = subprocess.run(
[sys.executable, "-c", code],
capture_output=True,
text=True
)
self.assertEqual(result.returncode, 0, f"stdout: {result.stdout}, stderr: {result.stderr}")
self.assertIn("OK", result.stdout)
class ThreadSafetyTests(unittest.TestCase):
"""Tests for thread-safety of lazy imports."""
def tearDown(self):
for key in list(sys.modules.keys()):
if key.startswith('test.test_import.data.lazy_imports'):
del sys.modules[key]
sys.set_lazy_imports_filter(None)
sys.set_lazy_imports("normal")
def test_concurrent_lazy_import_reification(self):
"""Multiple threads racing to reify the same lazy import should succeed."""
from test.test_import.data.lazy_imports import basic_unused
num_threads = 10
results = [None] * num_threads
errors = []
barrier = threading.Barrier(num_threads)
def access_lazy_import(idx):
try:
barrier.wait()
module = basic_unused.test.test_import.data.lazy_imports.basic2
results[idx] = module
except Exception as e:
errors.append((idx, e))
threads = [
threading.Thread(target=access_lazy_import, args=(i,))
for i in range(num_threads)
]
for t in threads:
t.start()
for t in threads:
t.join()
self.assertEqual(errors, [], f"Errors occurred: {errors}")
self.assertTrue(all(r is not None for r in results))
first_module = results[0]
for r in results[1:]:
self.assertIs(r, first_module)
def test_concurrent_reification_multiple_modules(self):
"""Multiple threads reifying different lazy imports concurrently."""
code = textwrap.dedent("""
import sys
import threading
sys.set_lazy_imports("all")
lazy import json
lazy import os
lazy import io
lazy import re
num_threads = 8
results = {}
errors = []
barrier = threading.Barrier(num_threads)
def access_modules(idx):
try:
barrier.wait()
mods = [json, os, io, re]
results[idx] = [type(m).__name__ for m in mods]
except Exception as e:
errors.append((idx, e))
threads = [
threading.Thread(target=access_modules, args=(i,))
for i in range(num_threads)
]
for t in threads:
t.start()
for t in threads:
t.join()
assert not errors, f"Errors: {errors}"
for idx, mods in results.items():
assert all(m == 'module' for m in mods), f"Thread {idx} got: {mods}"
print("OK")
""")
result = subprocess.run(
[sys.executable, "-c", code],
capture_output=True,
text=True
)
self.assertEqual(result.returncode, 0, f"stdout: {result.stdout}, stderr: {result.stderr}")
self.assertIn("OK", result.stdout)
if __name__ == '__main__':
unittest.main()

View file

@ -131,20 +131,20 @@ _PyLazyImport_GetName(PyObject *lazy_import)
} }
static PyObject * static PyObject *
lazy_get(PyObject *self, PyObject *args) lazy_resolve(PyObject *self, PyObject *args)
{ {
return _PyImport_LoadLazyImportTstate(PyThreadState_GET(), self); return _PyImport_LoadLazyImportTstate(PyThreadState_GET(), self);
} }
static PyMethodDef lazy_methods[] = { static PyMethodDef lazy_methods[] = {
{"get", lazy_get, METH_NOARGS, PyDoc_STR("gets the value that the lazy function references")}, {"resolve", lazy_resolve, METH_NOARGS, PyDoc_STR("resolves the lazy import and returns the actual object")},
{0} {0}
}; };
PyTypeObject PyLazyImport_Type = { PyTypeObject PyLazyImport_Type = {
PyVarObject_HEAD_INIT(&PyType_Type, 0) PyVarObject_HEAD_INIT(&PyType_Type, 0)
"LazyImport", /* tp_name */ "lazy_import", /* tp_name */
sizeof(PyLazyImportObject), /* tp_basicsize */ sizeof(PyLazyImportObject), /* tp_basicsize */
0, /* tp_itemsize */ 0, /* tp_itemsize */
(destructor)lazy_import_dealloc, /* tp_dealloc */ (destructor)lazy_import_dealloc, /* tp_dealloc */

View file

@ -1946,6 +1946,10 @@ _PyPegen_checked_future_import(Parser *p, identifier module, asdl_alias_seq * na
int is_lazy, int lineno, int col_offset, int end_lineno, int end_col_offset, int is_lazy, int lineno, int col_offset, int end_lineno, int end_col_offset,
PyArena *arena) { PyArena *arena) {
if (level == 0 && PyUnicode_CompareWithASCIIString(module, "__future__") == 0) { if (level == 0 && PyUnicode_CompareWithASCIIString(module, "__future__") == 0) {
if (is_lazy) {
RAISE_SYNTAX_ERROR("lazy from __future__ import is not allowed");
return NULL;
}
for (Py_ssize_t i = 0; i < asdl_seq_LEN(names); i++) { for (Py_ssize_t i = 0; i < asdl_seq_LEN(names); i++) {
alias_ty alias = asdl_seq_GET(names, i); alias_ty alias = asdl_seq_GET(names, i);
if (PyUnicode_CompareWithASCIIString(alias->name, "barry_as_FLUFL") == 0) { if (PyUnicode_CompareWithASCIIString(alias->name, "barry_as_FLUFL") == 0) {

View file

@ -1907,23 +1907,24 @@ sys_get_lazy_imports_filter(PyObject *module, PyObject *Py_UNUSED(ignored))
} }
PyDoc_STRVAR(sys_set_lazy_imports__doc__, PyDoc_STRVAR(sys_set_lazy_imports__doc__,
"set_lazy_imports($module, /, enabled)\n" "set_lazy_imports($module, /, mode)\n"
"--\n" "--\n"
"\n" "\n"
"Sets the global lazy imports flag.\n" "Sets the global lazy imports mode.\n"
"\n" "\n"
"True sets all imports at the top level as potentially lazy.\n" "The mode parameter must be one of the following strings:\n"
"False disables lazy imports for any explicitly marked imports.\n" "- \"all\": All top-level imports become potentially lazy\n"
"None causes only explicitly marked imports as lazy.\n" "- \"none\": All lazy imports are suppressed (even explicitly marked ones)\n"
"- \"normal\": Only explicitly marked imports (with \'lazy\' keyword) are lazy\n"
"\n" "\n"
"In addition to the mode lazy imports can be controlled via the filter\n" "In addition to the mode, lazy imports can be controlled via the filter\n"
"provided to sys.set_lazy_imports_filter"); "provided to sys.set_lazy_imports_filter");
#define SYS_SET_LAZY_IMPORTS_METHODDEF \ #define SYS_SET_LAZY_IMPORTS_METHODDEF \
{"set_lazy_imports", _PyCFunction_CAST(sys_set_lazy_imports), METH_FASTCALL|METH_KEYWORDS, sys_set_lazy_imports__doc__}, {"set_lazy_imports", _PyCFunction_CAST(sys_set_lazy_imports), METH_FASTCALL|METH_KEYWORDS, sys_set_lazy_imports__doc__},
static PyObject * static PyObject *
sys_set_lazy_imports_impl(PyObject *module, PyObject *enabled); sys_set_lazy_imports_impl(PyObject *module, PyObject *mode);
static PyObject * static PyObject *
sys_set_lazy_imports(PyObject *module, PyObject *const *args, Py_ssize_t nargs, PyObject *kwnames) sys_set_lazy_imports(PyObject *module, PyObject *const *args, Py_ssize_t nargs, PyObject *kwnames)
@ -1940,7 +1941,7 @@ sys_set_lazy_imports(PyObject *module, PyObject *const *args, Py_ssize_t nargs,
} _kwtuple = { } _kwtuple = {
.ob_base = PyVarObject_HEAD_INIT(&PyTuple_Type, NUM_KEYWORDS) .ob_base = PyVarObject_HEAD_INIT(&PyTuple_Type, NUM_KEYWORDS)
.ob_hash = -1, .ob_hash = -1,
.ob_item = { &_Py_ID(enabled), }, .ob_item = { &_Py_ID(mode), },
}; };
#undef NUM_KEYWORDS #undef NUM_KEYWORDS
#define KWTUPLE (&_kwtuple.ob_base.ob_base) #define KWTUPLE (&_kwtuple.ob_base.ob_base)
@ -1949,7 +1950,7 @@ sys_set_lazy_imports(PyObject *module, PyObject *const *args, Py_ssize_t nargs,
# define KWTUPLE NULL # define KWTUPLE NULL
#endif // !Py_BUILD_CORE #endif // !Py_BUILD_CORE
static const char * const _keywords[] = {"enabled", NULL}; static const char * const _keywords[] = {"mode", NULL};
static _PyArg_Parser _parser = { static _PyArg_Parser _parser = {
.keywords = _keywords, .keywords = _keywords,
.fname = "set_lazy_imports", .fname = "set_lazy_imports",
@ -1957,15 +1958,15 @@ sys_set_lazy_imports(PyObject *module, PyObject *const *args, Py_ssize_t nargs,
}; };
#undef KWTUPLE #undef KWTUPLE
PyObject *argsbuf[1]; PyObject *argsbuf[1];
PyObject *enabled; PyObject *mode;
args = _PyArg_UnpackKeywords(args, nargs, NULL, kwnames, &_parser, args = _PyArg_UnpackKeywords(args, nargs, NULL, kwnames, &_parser,
/*minpos*/ 1, /*maxpos*/ 1, /*minkw*/ 0, /*varpos*/ 0, argsbuf); /*minpos*/ 1, /*maxpos*/ 1, /*minkw*/ 0, /*varpos*/ 0, argsbuf);
if (!args) { if (!args) {
goto exit; goto exit;
} }
enabled = args[0]; mode = args[0];
return_value = sys_set_lazy_imports_impl(module, enabled); return_value = sys_set_lazy_imports_impl(module, mode);
exit: exit:
return return_value; return return_value;
@ -1975,11 +1976,11 @@ PyDoc_STRVAR(sys_get_lazy_imports__doc__,
"get_lazy_imports($module, /)\n" "get_lazy_imports($module, /)\n"
"--\n" "--\n"
"\n" "\n"
"Gets the global lazy imports flag.\n" "Gets the global lazy imports mode.\n"
"\n" "\n"
"Returns True if all top level imports are potentially lazy.\n" "Returns \"all\" if all top level imports are potentially lazy.\n"
"Returns False if all explicilty marked lazy imports are suppressed.\n" "Returns \"none\" if all explicitly marked lazy imports are suppressed.\n"
"Returns None if only explicitly marked imports are lazy."); "Returns \"normal\" if only explicitly marked imports are lazy.");
#define SYS_GET_LAZY_IMPORTS_METHODDEF \ #define SYS_GET_LAZY_IMPORTS_METHODDEF \
{"get_lazy_imports", (PyCFunction)sys_get_lazy_imports, METH_NOARGS, sys_get_lazy_imports__doc__}, {"get_lazy_imports", (PyCFunction)sys_get_lazy_imports, METH_NOARGS, sys_get_lazy_imports__doc__},
@ -1993,6 +1994,28 @@ sys_get_lazy_imports(PyObject *module, PyObject *Py_UNUSED(ignored))
return sys_get_lazy_imports_impl(module); return sys_get_lazy_imports_impl(module);
} }
PyDoc_STRVAR(sys_get_lazy_modules__doc__,
"get_lazy_modules($module, /)\n"
"--\n"
"\n"
"Gets the set of module names that have been lazily imported.\n"
"\n"
"Returns a set of fully-qualified module names that have been lazily imported\n"
"at some point (primarily for diagnostics and introspection). Note that modules\n"
"are removed from this set when they are reified (actually loaded).");
#define SYS_GET_LAZY_MODULES_METHODDEF \
{"get_lazy_modules", (PyCFunction)sys_get_lazy_modules, METH_NOARGS, sys_get_lazy_modules__doc__},
static PyObject *
sys_get_lazy_modules_impl(PyObject *module);
static PyObject *
sys_get_lazy_modules(PyObject *module, PyObject *Py_UNUSED(ignored))
{
return sys_get_lazy_modules_impl(module);
}
PyDoc_STRVAR(_jit_is_available__doc__, PyDoc_STRVAR(_jit_is_available__doc__,
"is_available($module, /)\n" "is_available($module, /)\n"
"--\n" "--\n"
@ -2120,4 +2143,4 @@ exit:
#ifndef SYS_GETANDROIDAPILEVEL_METHODDEF #ifndef SYS_GETANDROIDAPILEVEL_METHODDEF
#define SYS_GETANDROIDAPILEVEL_METHODDEF #define SYS_GETANDROIDAPILEVEL_METHODDEF
#endif /* !defined(SYS_GETANDROIDAPILEVEL_METHODDEF) */ #endif /* !defined(SYS_GETANDROIDAPILEVEL_METHODDEF) */
/*[clinic end generated code: output=dd304e713c0d089f input=a9049054013a1b77]*/ /*[clinic end generated code: output=cec3ca2ba0ad32cc input=a9049054013a1b77]*/

View file

@ -4267,10 +4267,26 @@ _PyImport_LazyImportModuleLevelObject(PyThreadState *tstate,
} }
PyObject *res = _PyLazyImport_New(builtins, abs_name, fromlist); PyObject *res = _PyLazyImport_New(builtins, abs_name, fromlist);
if (res == NULL) {
Py_DECREF(abs_name);
return NULL;
}
if (register_lazy_on_parent(tstate, abs_name, builtins) < 0) { if (register_lazy_on_parent(tstate, abs_name, builtins) < 0) {
Py_DECREF(res); Py_DECREF(res);
res = NULL; Py_DECREF(abs_name);
return NULL;
} }
// Add the module name to sys.lazy_modules set (PEP 810)
PyObject *lazy_modules_set = interp->imports.lazy_modules_set;
if (lazy_modules_set != NULL) {
if (PySet_Add(lazy_modules_set, abs_name) < 0) {
Py_DECREF(res);
Py_DECREF(abs_name);
return NULL;
}
}
Py_DECREF(abs_name); Py_DECREF(abs_name);
return res; return res;
} }
@ -4482,6 +4498,9 @@ _PyImport_ClearCore(PyInterpreterState *interp)
Py_CLEAR(IMPORTLIB(interp)); Py_CLEAR(IMPORTLIB(interp));
Py_CLEAR(IMPORT_FUNC(interp)); Py_CLEAR(IMPORT_FUNC(interp));
Py_CLEAR(LAZY_IMPORT_FUNC(interp)); Py_CLEAR(LAZY_IMPORT_FUNC(interp));
Py_CLEAR(interp->imports.lazy_modules);
Py_CLEAR(interp->imports.lazy_modules_set);
Py_CLEAR(interp->imports.lazy_importing_modules);
} }
void void

View file

@ -318,7 +318,7 @@ The following implementation-specific options are available:\n\
"\ "\
-X importtime[=2]: show how long each import takes; use -X importtime=2 to\n\ -X importtime[=2]: show how long each import takes; use -X importtime=2 to\n\
log imports of already-loaded modules; also PYTHONPROFILEIMPORTTIME\n\ log imports of already-loaded modules; also PYTHONPROFILEIMPORTTIME\n\
-X lazy_imports=[on|off|default]: control global lazy imports; default is auto;\n\ -X lazy_imports=[all|none|normal]: control global lazy imports; default is normal;\n\
also PYTHON_LAZY_IMPORTS\n\ also PYTHON_LAZY_IMPORTS\n\
-X int_max_str_digits=N: limit the size of int<->str conversions;\n\ -X int_max_str_digits=N: limit the size of int<->str conversions;\n\
0 disables the limit; also PYTHONINTMAXSTRDIGITS\n\ 0 disables the limit; also PYTHONINTMAXSTRDIGITS\n\
@ -2298,36 +2298,36 @@ config_init_lazy_imports(PyConfig *config)
const char *env = config_get_env(config, "PYTHON_LAZY_IMPORTS"); const char *env = config_get_env(config, "PYTHON_LAZY_IMPORTS");
if (env) { if (env) {
if (strcmp(env, "on") == 0) { if (strcmp(env, "all") == 0) {
lazy_imports = 1; lazy_imports = 1;
} }
else if (strcmp(env, "off") == 0) { else if (strcmp(env, "none") == 0) {
lazy_imports = 0; lazy_imports = 0;
} }
else if (strcmp(env, "default") == 0) { else if (strcmp(env, "normal") == 0) {
lazy_imports = -1; lazy_imports = -1;
} }
else { else {
return _PyStatus_ERR("PYTHON_LAZY_IMPORTS: invalid value; " return _PyStatus_ERR("PYTHON_LAZY_IMPORTS: invalid value; "
"expected 'on', 'off', or 'default'"); "expected 'all', 'none', or 'normal'");
} }
config->lazy_imports = lazy_imports; config->lazy_imports = lazy_imports;
} }
const wchar_t *x_value = config_get_xoption_value(config, L"lazy_imports"); const wchar_t *x_value = config_get_xoption_value(config, L"lazy_imports");
if (x_value) { if (x_value) {
if (wcscmp(x_value, L"on") == 0) { if (wcscmp(x_value, L"all") == 0) {
lazy_imports = 1; lazy_imports = 1;
} }
else if (wcscmp(x_value, L"off") == 0) { else if (wcscmp(x_value, L"none") == 0) {
lazy_imports = 0; lazy_imports = 0;
} }
else if (wcscmp(x_value, L"default") == 0) { else if (wcscmp(x_value, L"normal") == 0) {
lazy_imports = -1; lazy_imports = -1;
} }
else { else {
return _PyStatus_ERR("-X lazy_imports: invalid value; " return _PyStatus_ERR("-X lazy_imports: invalid value; "
"expected 'on', 'off', or 'default'"); "expected 'all', 'none', or 'normal'");
} }
config->lazy_imports = lazy_imports; config->lazy_imports = lazy_imports;
} }

View file

@ -2821,36 +2821,41 @@ sys_get_lazy_imports_filter_impl(PyObject *module)
/*[clinic input] /*[clinic input]
sys.set_lazy_imports sys.set_lazy_imports
enabled: object mode: object
Sets the global lazy imports flag. Sets the global lazy imports mode.
True sets all imports at the top level as potentially lazy. The mode parameter must be one of the following strings:
False disables lazy imports for any explicitly marked imports. - "all": All top-level imports become potentially lazy
None causes only explicitly marked imports as lazy. - "none": All lazy imports are suppressed (even explicitly marked ones)
- "normal": Only explicitly marked imports (with 'lazy' keyword) are lazy
In addition to the mode lazy imports can be controlled via the filter In addition to the mode, lazy imports can be controlled via the filter
provided to sys.set_lazy_imports_filter provided to sys.set_lazy_imports_filter
[clinic start generated code]*/ [clinic start generated code]*/
static PyObject * static PyObject *
sys_set_lazy_imports_impl(PyObject *module, PyObject *enabled) sys_set_lazy_imports_impl(PyObject *module, PyObject *mode)
/*[clinic end generated code: output=d601640d3e2d70fb input=d351054b5884eae5]*/ /*[clinic end generated code: output=1ff34ba6c4feaf73 input=f04e70d8bf9fe4f6]*/
{ {
PyImport_LazyImportsMode mode; PyImport_LazyImportsMode lazy_mode;
if (enabled == Py_None) { if (!PyUnicode_Check(mode)) {
mode = PyImport_LAZY_NORMAL; PyErr_SetString(PyExc_TypeError, "mode must be a string: 'normal', 'all', or 'none'");
} else if (enabled == Py_False) { return NULL;
mode = PyImport_LAZY_NONE; }
} else if (enabled == Py_True) { if (PyUnicode_CompareWithASCIIString(mode, "normal") == 0) {
mode = PyImport_LAZY_ALL; lazy_mode = PyImport_LAZY_NORMAL;
} else if (PyUnicode_CompareWithASCIIString(mode, "all") == 0) {
lazy_mode = PyImport_LAZY_ALL;
} else if (PyUnicode_CompareWithASCIIString(mode, "none") == 0) {
lazy_mode = PyImport_LAZY_NONE;
} else { } else {
PyErr_SetString(PyExc_ValueError, "expected None, True or False for enabled mode"); PyErr_SetString(PyExc_ValueError, "mode must be 'normal', 'all', or 'none'");
return NULL; return NULL;
} }
if (PyImport_SetLazyImportsMode(mode)) { if (PyImport_SetLazyImportsMode(lazy_mode)) {
return NULL; return NULL;
} }
Py_RETURN_NONE; Py_RETURN_NONE;
@ -2859,31 +2864,54 @@ sys_set_lazy_imports_impl(PyObject *module, PyObject *enabled)
/*[clinic input] /*[clinic input]
sys.get_lazy_imports sys.get_lazy_imports
Gets the global lazy imports flag. Gets the global lazy imports mode.
Returns True if all top level imports are potentially lazy. Returns "all" if all top level imports are potentially lazy.
Returns False if all explicilty marked lazy imports are suppressed. Returns "none" if all explicitly marked lazy imports are suppressed.
Returns None if only explicitly marked imports are lazy. Returns "normal" if only explicitly marked imports are lazy.
[clinic start generated code]*/ [clinic start generated code]*/
static PyObject * static PyObject *
sys_get_lazy_imports_impl(PyObject *module) sys_get_lazy_imports_impl(PyObject *module)
/*[clinic end generated code: output=4147dec48c51ae99 input=d7b25d814165c8ce]*/ /*[clinic end generated code: output=4147dec48c51ae99 input=8cb574f1e4e3003c]*/
{ {
switch (PyImport_GetLazyImportsMode()) { switch (PyImport_GetLazyImportsMode()) {
case PyImport_LAZY_NORMAL: case PyImport_LAZY_NORMAL:
Py_RETURN_NONE; return PyUnicode_FromString("normal");
case PyImport_LAZY_ALL: case PyImport_LAZY_ALL:
Py_RETURN_TRUE; return PyUnicode_FromString("all");
case PyImport_LAZY_NONE: case PyImport_LAZY_NONE:
Py_RETURN_FALSE; return PyUnicode_FromString("none");
default: default:
PyErr_SetString(PyExc_RuntimeError, "unknown lazy imports mode"); PyErr_SetString(PyExc_RuntimeError, "unknown lazy imports mode");
return NULL; return NULL;
} }
} }
/*[clinic input]
sys.get_lazy_modules
Gets the set of module names that have been lazily imported.
Returns a set of fully-qualified module names that have been lazily imported
at some point (primarily for diagnostics and introspection). Note that modules
are removed from this set when they are reified (actually loaded).
[clinic start generated code]*/
static PyObject *
sys_get_lazy_modules_impl(PyObject *module)
/*[clinic end generated code: output=4c641f8881ba87c0 input=06c7a1d05bcfa36a]*/
{
PyInterpreterState *interp = _PyInterpreterState_GET();
PyObject *lazy_modules_set = interp->imports.lazy_modules_set;
if (lazy_modules_set == NULL) {
return PySet_New(NULL);
}
return Py_NewRef(lazy_modules_set);
}
static PyMethodDef sys_methods[] = { static PyMethodDef sys_methods[] = {
/* Might as well keep this in alphabetic order */ /* Might as well keep this in alphabetic order */
SYS_ADDAUDITHOOK_METHODDEF SYS_ADDAUDITHOOK_METHODDEF
@ -2952,6 +2980,7 @@ static PyMethodDef sys_methods[] = {
SYS_SET_LAZY_IMPORTS_METHODDEF SYS_SET_LAZY_IMPORTS_METHODDEF
SYS_GET_LAZY_IMPORTS_FILTER_METHODDEF SYS_GET_LAZY_IMPORTS_FILTER_METHODDEF
SYS_SET_LAZY_IMPORTS_FILTER_METHODDEF SYS_SET_LAZY_IMPORTS_FILTER_METHODDEF
SYS_GET_LAZY_MODULES_METHODDEF
SYS__BASEREPL_METHODDEF SYS__BASEREPL_METHODDEF
#ifdef Py_STATS #ifdef Py_STATS
SYS__STATS_ON_METHODDEF SYS__STATS_ON_METHODDEF
@ -4065,6 +4094,14 @@ _PySys_InitCore(PyThreadState *tstate, PyObject *sysdict)
SET_SYS("path_importer_cache", PyDict_New()); SET_SYS("path_importer_cache", PyDict_New());
SET_SYS("path_hooks", PyList_New(0)); SET_SYS("path_hooks", PyList_New(0));
/* adding sys.lazy_modules set (PEP 810) */
PyObject *lazy_modules_set = PySet_New(NULL);
if (lazy_modules_set == NULL) {
goto err_occurred;
}
interp->imports.lazy_modules_set = lazy_modules_set;
SET_SYS("lazy_modules", Py_NewRef(lazy_modules_set));
if (_PyErr_Occurred(tstate)) { if (_PyErr_Occurred(tstate)) {
goto err_occurred; goto err_occurred;
} }