mirror of
https://github.com/python/cpython.git
synced 2025-10-31 13:41:24 +00:00
338 lines
14 KiB
ReStructuredText
338 lines
14 KiB
ReStructuredText
:mod:`compileall` --- Byte-compile Python libraries
|
|
===================================================
|
|
|
|
.. module:: compileall
|
|
:synopsis: Tools for byte-compiling all Python source files in a directory tree.
|
|
|
|
**Source code:** :source:`Lib/compileall.py`
|
|
|
|
--------------
|
|
|
|
This module provides some utility functions to support installing Python
|
|
libraries. These functions compile Python source files in a directory tree.
|
|
This module can be used to create the cached byte-code files at library
|
|
installation time, which makes them available for use even by users who don't
|
|
have write permission to the library directories.
|
|
|
|
|
|
Command-line use
|
|
----------------
|
|
|
|
This module can work as a script (using :program:`python -m compileall`) to
|
|
compile Python sources.
|
|
|
|
.. program:: compileall
|
|
|
|
.. cmdoption:: directory ...
|
|
file ...
|
|
|
|
Positional arguments are files to compile or directories that contain
|
|
source files, traversed recursively. If no argument is given, behave as if
|
|
the command line was ``-l <directories from sys.path>``.
|
|
|
|
.. cmdoption:: -l
|
|
|
|
Do not recurse into subdirectories, only compile source code files directly
|
|
contained in the named or implied directories.
|
|
|
|
.. cmdoption:: -f
|
|
|
|
Force rebuild even if timestamps are up-to-date.
|
|
|
|
.. cmdoption:: -q
|
|
|
|
Do not print the list of files compiled. If passed once, error messages will
|
|
still be printed. If passed twice (``-qq``), all output is suppressed.
|
|
|
|
.. cmdoption:: -d destdir
|
|
|
|
Directory prepended to the path to each file being compiled. This will
|
|
appear in compilation time tracebacks, and is also compiled in to the
|
|
byte-code file, where it will be used in tracebacks and other messages in
|
|
cases where the source file does not exist at the time the byte-code file is
|
|
executed.
|
|
|
|
.. cmdoption:: -s strip_prefix
|
|
.. cmdoption:: -p prepend_prefix
|
|
|
|
Remove (``-s``) or append (``-p``) the given prefix of paths
|
|
recorded in the ``.pyc`` files.
|
|
Cannot be combined with ``-d``.
|
|
|
|
.. cmdoption:: -x regex
|
|
|
|
regex is used to search the full path to each file considered for
|
|
compilation, and if the regex produces a match, the file is skipped.
|
|
|
|
.. cmdoption:: -i list
|
|
|
|
Read the file ``list`` and add each line that it contains to the list of
|
|
files and directories to compile. If ``list`` is ``-``, read lines from
|
|
``stdin``.
|
|
|
|
.. cmdoption:: -b
|
|
|
|
Write the byte-code files to their legacy locations and names, which may
|
|
overwrite byte-code files created by another version of Python. The default
|
|
is to write files to their :pep:`3147` locations and names, which allows
|
|
byte-code files from multiple versions of Python to coexist.
|
|
|
|
.. cmdoption:: -r
|
|
|
|
Control the maximum recursion level for subdirectories.
|
|
If this is given, then ``-l`` option will not be taken into account.
|
|
:program:`python -m compileall <directory> -r 0` is equivalent to
|
|
:program:`python -m compileall <directory> -l`.
|
|
|
|
.. cmdoption:: -j N
|
|
|
|
Use *N* workers to compile the files within the given directory.
|
|
If ``0`` is used, then the result of :func:`os.cpu_count()`
|
|
will be used.
|
|
|
|
.. cmdoption:: --invalidation-mode [timestamp|checked-hash|unchecked-hash]
|
|
|
|
Control how the generated byte-code files are invalidated at runtime.
|
|
The ``timestamp`` value, means that ``.pyc`` files with the source timestamp
|
|
and size embedded will be generated. The ``checked-hash`` and
|
|
``unchecked-hash`` values cause hash-based pycs to be generated. Hash-based
|
|
pycs embed a hash of the source file contents rather than a timestamp. See
|
|
:ref:`pyc-invalidation` for more information on how Python validates
|
|
bytecode cache files at runtime.
|
|
The default is ``timestamp`` if the :envvar:`SOURCE_DATE_EPOCH` environment
|
|
variable is not set, and ``checked-hash`` if the ``SOURCE_DATE_EPOCH``
|
|
environment variable is set.
|
|
|
|
.. cmdoption:: -o level
|
|
|
|
Compile with the given optimization level. May be used multiple times
|
|
to compile for multiple levels at a time (for example,
|
|
``compileall -o 1 -o 2``).
|
|
|
|
.. cmdoption:: -e dir
|
|
|
|
Ignore symlinks pointing outside the given directory.
|
|
|
|
.. cmdoption:: --hardlink-dupes
|
|
|
|
If two ``.pyc`` files with different optimization level have
|
|
the same content, use hard links to consolidate duplicate files.
|
|
|
|
.. versionchanged:: 3.2
|
|
Added the ``-i``, ``-b`` and ``-h`` options.
|
|
|
|
.. versionchanged:: 3.5
|
|
Added the ``-j``, ``-r``, and ``-qq`` options. ``-q`` option
|
|
was changed to a multilevel value. ``-b`` will always produce a
|
|
byte-code file ending in ``.pyc``, never ``.pyo``.
|
|
|
|
.. versionchanged:: 3.7
|
|
Added the ``--invalidation-mode`` option.
|
|
|
|
.. versionchanged:: 3.9
|
|
Added the ``-s``, ``-p``, ``-e`` and ``--hardlink-dupes`` options.
|
|
Raised the default recursion limit from 10 to
|
|
:py:func:`sys.getrecursionlimit()`.
|
|
Added the possibility to specify the ``-o`` option multiple times.
|
|
|
|
|
|
There is no command-line option to control the optimization level used by the
|
|
:func:`compile` function, because the Python interpreter itself already
|
|
provides the option: :program:`python -O -m compileall`.
|
|
|
|
Similarly, the :func:`compile` function respects the :attr:`sys.pycache_prefix`
|
|
setting. The generated bytecode cache will only be useful if :func:`compile` is
|
|
run with the same :attr:`sys.pycache_prefix` (if any) that will be used at
|
|
runtime.
|
|
|
|
Public functions
|
|
----------------
|
|
|
|
.. function:: compile_dir(dir, maxlevels=sys.getrecursionlimit(), ddir=None, force=False, rx=None, quiet=0, legacy=False, optimize=-1, workers=1, invalidation_mode=None, *, stripdir=None, prependdir=None, limit_sl_dest=None, hardlink_dupes=False)
|
|
|
|
Recursively descend the directory tree named by *dir*, compiling all :file:`.py`
|
|
files along the way. Return a true value if all the files compiled successfully,
|
|
and a false value otherwise.
|
|
|
|
The *maxlevels* parameter is used to limit the depth of the recursion; it
|
|
defaults to ``sys.getrecursionlimit()``.
|
|
|
|
If *ddir* is given, it is prepended to the path to each file being compiled
|
|
for use in compilation time tracebacks, and is also compiled in to the
|
|
byte-code file, where it will be used in tracebacks and other messages in
|
|
cases where the source file does not exist at the time the byte-code file is
|
|
executed.
|
|
|
|
If *force* is true, modules are re-compiled even if the timestamps are up to
|
|
date.
|
|
|
|
If *rx* is given, its ``search`` method is called on the complete path to each
|
|
file considered for compilation, and if it returns a true value, the file
|
|
is skipped. This can be used to exclude files matching a regular expression,
|
|
given as a :ref:`re.Pattern <re-objects>` object.
|
|
|
|
If *quiet* is ``False`` or ``0`` (the default), the filenames and other
|
|
information are printed to standard out. Set to ``1``, only errors are
|
|
printed. Set to ``2``, all output is suppressed.
|
|
|
|
If *legacy* is true, byte-code files are written to their legacy locations
|
|
and names, which may overwrite byte-code files created by another version of
|
|
Python. The default is to write files to their :pep:`3147` locations and
|
|
names, which allows byte-code files from multiple versions of Python to
|
|
coexist.
|
|
|
|
*optimize* specifies the optimization level for the compiler. It is passed to
|
|
the built-in :func:`compile` function. Accepts also a sequence of optimization
|
|
levels which lead to multiple compilations of one :file:`.py` file in one call.
|
|
|
|
The argument *workers* specifies how many workers are used to
|
|
compile files in parallel. The default is to not use multiple workers.
|
|
If the platform can't use multiple workers and *workers* argument is given,
|
|
then sequential compilation will be used as a fallback. If *workers*
|
|
is 0, the number of cores in the system is used. If *workers* is
|
|
lower than ``0``, a :exc:`ValueError` will be raised.
|
|
|
|
*invalidation_mode* should be a member of the
|
|
:class:`py_compile.PycInvalidationMode` enum and controls how the generated
|
|
pycs are invalidated at runtime.
|
|
|
|
The *stripdir*, *prependdir* and *limit_sl_dest* arguments correspond to
|
|
the ``-s``, ``-p`` and ``-e`` options described above.
|
|
They may be specified as ``str``, ``bytes`` or :py:class:`os.PathLike`.
|
|
|
|
If *hardlink_dupes* is true and two ``.pyc`` files with different optimization
|
|
level have the same content, use hard links to consolidate duplicate files.
|
|
|
|
.. versionchanged:: 3.2
|
|
Added the *legacy* and *optimize* parameter.
|
|
|
|
.. versionchanged:: 3.5
|
|
Added the *workers* parameter.
|
|
|
|
.. versionchanged:: 3.5
|
|
*quiet* parameter was changed to a multilevel value.
|
|
|
|
.. versionchanged:: 3.5
|
|
The *legacy* parameter only writes out ``.pyc`` files, not ``.pyo`` files
|
|
no matter what the value of *optimize* is.
|
|
|
|
.. versionchanged:: 3.6
|
|
Accepts a :term:`path-like object`.
|
|
|
|
.. versionchanged:: 3.7
|
|
The *invalidation_mode* parameter was added.
|
|
|
|
.. versionchanged:: 3.7.2
|
|
The *invalidation_mode* parameter's default value is updated to None.
|
|
|
|
.. versionchanged:: 3.8
|
|
Setting *workers* to 0 now chooses the optimal number of cores.
|
|
|
|
.. versionchanged:: 3.9
|
|
Added *stripdir*, *prependdir*, *limit_sl_dest* and *hardlink_dupes* arguments.
|
|
Default value of *maxlevels* was changed from ``10`` to ``sys.getrecursionlimit()``
|
|
|
|
.. function:: compile_file(fullname, ddir=None, force=False, rx=None, quiet=0, legacy=False, optimize=-1, invalidation_mode=None, *, stripdir=None, prependdir=None, limit_sl_dest=None, hardlink_dupes=False)
|
|
|
|
Compile the file with path *fullname*. Return a true value if the file
|
|
compiled successfully, and a false value otherwise.
|
|
|
|
If *ddir* is given, it is prepended to the path to the file being compiled
|
|
for use in compilation time tracebacks, and is also compiled in to the
|
|
byte-code file, where it will be used in tracebacks and other messages in
|
|
cases where the source file does not exist at the time the byte-code file is
|
|
executed.
|
|
|
|
If *rx* is given, its ``search`` method is passed the full path name to the
|
|
file being compiled, and if it returns a true value, the file is not
|
|
compiled and ``True`` is returned. This can be used to exclude files matching
|
|
a regular expression, given as a :ref:`re.Pattern <re-objects>` object.
|
|
|
|
If *quiet* is ``False`` or ``0`` (the default), the filenames and other
|
|
information are printed to standard out. Set to ``1``, only errors are
|
|
printed. Set to ``2``, all output is suppressed.
|
|
|
|
If *legacy* is true, byte-code files are written to their legacy locations
|
|
and names, which may overwrite byte-code files created by another version of
|
|
Python. The default is to write files to their :pep:`3147` locations and
|
|
names, which allows byte-code files from multiple versions of Python to
|
|
coexist.
|
|
|
|
*optimize* specifies the optimization level for the compiler. It is passed to
|
|
the built-in :func:`compile` function. Accepts also a sequence of optimization
|
|
levels which lead to multiple compilations of one :file:`.py` file in one call.
|
|
|
|
*invalidation_mode* should be a member of the
|
|
:class:`py_compile.PycInvalidationMode` enum and controls how the generated
|
|
pycs are invalidated at runtime.
|
|
|
|
The *stripdir*, *prependdir* and *limit_sl_dest* arguments correspond to
|
|
the ``-s``, ``-p`` and ``-e`` options described above.
|
|
They may be specified as ``str``, ``bytes`` or :py:class:`os.PathLike`.
|
|
|
|
If *hardlink_dupes* is true and two ``.pyc`` files with different optimization
|
|
level have the same content, use hard links to consolidate duplicate files.
|
|
|
|
.. versionadded:: 3.2
|
|
|
|
.. versionchanged:: 3.5
|
|
*quiet* parameter was changed to a multilevel value.
|
|
|
|
.. versionchanged:: 3.5
|
|
The *legacy* parameter only writes out ``.pyc`` files, not ``.pyo`` files
|
|
no matter what the value of *optimize* is.
|
|
|
|
.. versionchanged:: 3.7
|
|
The *invalidation_mode* parameter was added.
|
|
|
|
.. versionchanged:: 3.7.2
|
|
The *invalidation_mode* parameter's default value is updated to None.
|
|
|
|
.. versionchanged:: 3.9
|
|
Added *stripdir*, *prependdir*, *limit_sl_dest* and *hardlink_dupes* arguments.
|
|
|
|
.. function:: compile_path(skip_curdir=True, maxlevels=0, force=False, quiet=0, legacy=False, optimize=-1, invalidation_mode=None)
|
|
|
|
Byte-compile all the :file:`.py` files found along ``sys.path``. Return a
|
|
true value if all the files compiled successfully, and a false value otherwise.
|
|
|
|
If *skip_curdir* is true (the default), the current directory is not included
|
|
in the search. All other parameters are passed to the :func:`compile_dir`
|
|
function. Note that unlike the other compile functions, ``maxlevels``
|
|
defaults to ``0``.
|
|
|
|
.. versionchanged:: 3.2
|
|
Added the *legacy* and *optimize* parameter.
|
|
|
|
.. versionchanged:: 3.5
|
|
*quiet* parameter was changed to a multilevel value.
|
|
|
|
.. versionchanged:: 3.5
|
|
The *legacy* parameter only writes out ``.pyc`` files, not ``.pyo`` files
|
|
no matter what the value of *optimize* is.
|
|
|
|
.. versionchanged:: 3.7
|
|
The *invalidation_mode* parameter was added.
|
|
|
|
.. versionchanged:: 3.7.2
|
|
The *invalidation_mode* parameter's default value is updated to None.
|
|
|
|
To force a recompile of all the :file:`.py` files in the :file:`Lib/`
|
|
subdirectory and all its subdirectories::
|
|
|
|
import compileall
|
|
|
|
compileall.compile_dir('Lib/', force=True)
|
|
|
|
# Perform same compilation, excluding files in .svn directories.
|
|
import re
|
|
compileall.compile_dir('Lib/', rx=re.compile(r'[/\\][.]svn'), force=True)
|
|
|
|
# pathlib.Path objects can also be used.
|
|
import pathlib
|
|
compileall.compile_dir(pathlib.Path('Lib/'), force=True)
|
|
|
|
.. seealso::
|
|
|
|
Module :mod:`py_compile`
|
|
Byte-compile a single source file.
|