mirror of
https://github.com/python/cpython.git
synced 2026-01-07 16:02:55 +00:00
4279785b31
* Add profiling module documentation structure PEP 799 introduces a new `profiling` package that reorganizes Python's profiling tools under a unified namespace. This commit adds the documentation structure to match: a main entry point (profiling.rst) that helps users choose between profilers, detailed docs for the tracing profiler (profiling-tracing.rst), and separated pstats documentation. The tracing profiler docs note that cProfile remains as a backward-compatible alias, so existing code continues to work. The pstats module gets its own page since it's used by both profiler types and deserves focused documentation. * Add profiling.sampling documentation The sampling profiler is new in Python 3.15 and works fundamentally differently from the tracing profiler. It observes programs from outside by periodically capturing stack snapshots, which means zero overhead on the profiled code. This makes it practical for production use where you can attach to live servers. The docs explain the key concepts (statistical vs deterministic profiling), provide quick examples upfront, document all output formats (pstats, flamegraph, gecko, heatmap), and cover the live TUI mode. The defaults table helps users understand what happens without any flags. * Wire profiling docs into the documentation tree Add the new profiling module pages to the Debugging and Profiling toctree. The order places the main profiling.rst entry point first, followed by the two profiler implementations, then pstats, and finally the deprecated profile module last. * Convert profile.rst to deprecation stub The pure Python profile module is deprecated in 3.15 and scheduled for removal in 3.17. Users should migrate to profiling.tracing (or use the cProfile alias which continues to work). The page now focuses on helping existing users migrate: it shows the old vs new import style, keeps the shared API reference since both modules have the same interface, and preserves the calibration docs for anyone still using the pure Python implementation during the transition period. * Update CLI module references for profiling restructure Point cProfile to profiling.tracing docs and add profiling.sampling to the list of modules with CLI interfaces. The old profile-cli label no longer exists after the documentation restructure. * Update whatsnew to link to profiling module docs Enable cross-references to the new profiling module documentation and update the CLI examples to use the current syntax with the attach subcommand. Also reference profiling.tracing instead of cProfile since that's the new canonical name.
362 lines
12 KiB
ReStructuredText
362 lines
12 KiB
ReStructuredText
.. _pstats-module:
|
|
|
|
********************************************
|
|
:mod:`pstats` --- Statistics for profilers
|
|
********************************************
|
|
|
|
.. module:: pstats
|
|
:synopsis: Statistics object for analyzing profiler output.
|
|
|
|
**Source code:** :source:`Lib/pstats.py`
|
|
|
|
--------------
|
|
|
|
The :mod:`pstats` module provides tools for reading, manipulating, and
|
|
displaying profiling statistics generated by Python's profilers. It reads
|
|
output from both :mod:`profiling.tracing` (deterministic profiler) and
|
|
:mod:`profiling.sampling` (statistical profiler).
|
|
|
|
|
|
Reading and displaying profile data
|
|
===================================
|
|
|
|
The :class:`Stats` class is the primary interface for working with profile
|
|
data. It can read statistics from files or directly from a
|
|
:class:`~profiling.tracing.Profile` object.
|
|
|
|
Load statistics from a file and print a basic report::
|
|
|
|
import pstats
|
|
|
|
p = pstats.Stats('profile_output.prof')
|
|
p.print_stats()
|
|
|
|
The :class:`Stats` object provides methods for sorting and filtering the
|
|
data before printing. For example, to see the ten functions with the highest
|
|
cumulative time::
|
|
|
|
from pstats import SortKey
|
|
|
|
p = pstats.Stats('profile_output.prof')
|
|
p.sort_stats(SortKey.CUMULATIVE).print_stats(10)
|
|
|
|
|
|
Working with statistics
|
|
-----------------------
|
|
|
|
The :class:`Stats` class supports method chaining, making it convenient to
|
|
perform multiple operations::
|
|
|
|
p = pstats.Stats('restats')
|
|
p.strip_dirs().sort_stats(-1).print_stats()
|
|
|
|
The :meth:`~Stats.strip_dirs` method removes directory paths from filenames,
|
|
making the output more compact. The :meth:`~Stats.sort_stats` method accepts
|
|
various keys to control the sort order.
|
|
|
|
Different sort keys highlight different aspects of performance::
|
|
|
|
from pstats import SortKey
|
|
|
|
# Functions that consume the most cumulative time
|
|
p.sort_stats(SortKey.CUMULATIVE).print_stats(10)
|
|
|
|
# Functions that consume the most time in their own code
|
|
p.sort_stats(SortKey.TIME).print_stats(10)
|
|
|
|
# Functions sorted by name
|
|
p.sort_stats(SortKey.NAME).print_stats()
|
|
|
|
|
|
Filtering output
|
|
----------------
|
|
|
|
The :meth:`~Stats.print_stats` method accepts restrictions that filter
|
|
which functions are displayed. Restrictions can be integers (limiting the
|
|
count), floats between 0 and 1 (selecting a percentage), or strings (matching
|
|
function names via regular expression).
|
|
|
|
Print only the top 10%::
|
|
|
|
p.print_stats(.1)
|
|
|
|
Print only functions whose names contain "init"::
|
|
|
|
p.print_stats('init')
|
|
|
|
Combine restrictions (they apply sequentially)::
|
|
|
|
# Top 10%, then only those containing "init"
|
|
p.print_stats(.1, 'init')
|
|
|
|
# Functions in files matching "foo:", limited to top 50%
|
|
p.sort_stats(SortKey.FILENAME).print_stats('foo:', .5)
|
|
|
|
|
|
Analyzing call relationships
|
|
----------------------------
|
|
|
|
The :meth:`~Stats.print_callers` method shows which functions called each
|
|
displayed function::
|
|
|
|
p.print_callers()
|
|
|
|
The :meth:`~Stats.print_callees` method shows the opposite relationship,
|
|
listing which functions each displayed function called::
|
|
|
|
p.print_callees()
|
|
|
|
Both methods accept the same restriction arguments as :meth:`~Stats.print_stats`.
|
|
|
|
|
|
Combining multiple profiles
|
|
---------------------------
|
|
|
|
Statistics from multiple profiling runs can be combined into a single
|
|
:class:`Stats` object::
|
|
|
|
# Load multiple files at once
|
|
p = pstats.Stats('run1.prof', 'run2.prof', 'run3.prof')
|
|
|
|
# Or add files incrementally
|
|
p = pstats.Stats('run1.prof')
|
|
p.add('run2.prof')
|
|
p.add('run3.prof')
|
|
|
|
When files are combined, statistics for identical functions (same file, line,
|
|
and name) are accumulated, giving an aggregate view across all profiling runs.
|
|
|
|
|
|
The :class:`!Stats` class
|
|
=========================
|
|
|
|
.. class:: Stats(*filenames_or_profile, stream=sys.stdout)
|
|
|
|
Create a statistics object from profile data.
|
|
|
|
The arguments can be filenames (strings or path-like objects) or
|
|
:class:`~profiling.tracing.Profile` objects. If multiple sources are
|
|
provided, their statistics are combined.
|
|
|
|
The *stream* argument specifies where output from :meth:`print_stats` and
|
|
related methods is written. It defaults to :data:`sys.stdout`.
|
|
|
|
The profile data format is specific to the Python version that created it.
|
|
There is no compatibility guarantee between Python versions or between
|
|
different profilers.
|
|
|
|
.. method:: strip_dirs()
|
|
|
|
Remove leading path information from all filenames.
|
|
|
|
This method modifies the object in place and returns it for method
|
|
chaining. After stripping, the statistics are considered to be in
|
|
random order.
|
|
|
|
If stripping causes two functions to become indistinguishable (same
|
|
filename, line number, and function name), their statistics are
|
|
combined into a single entry.
|
|
|
|
.. method:: add(*filenames)
|
|
|
|
Add profiling data from additional files.
|
|
|
|
The files must have been created by the same profiler type. Statistics
|
|
for identical functions are accumulated.
|
|
|
|
.. method:: dump_stats(filename)
|
|
|
|
Save the current statistics to a file.
|
|
|
|
The file is created if it does not exist and overwritten if it does.
|
|
The saved data can be loaded by creating a new :class:`Stats` object.
|
|
|
|
.. method:: sort_stats(*keys)
|
|
|
|
Sort the statistics according to the specified criteria.
|
|
|
|
Each key can be a string or a :class:`SortKey` enum member. When
|
|
multiple keys are provided, later keys break ties in earlier keys.
|
|
|
|
Using :class:`SortKey` enum members is preferred over strings as it
|
|
provides better error checking::
|
|
|
|
from pstats import SortKey
|
|
p.sort_stats(SortKey.CUMULATIVE)
|
|
|
|
Valid sort keys:
|
|
|
|
+------------------+------------------------+----------------------+
|
|
| String | Enum | Meaning |
|
|
+==================+========================+======================+
|
|
| ``'calls'`` | ``SortKey.CALLS`` | call count |
|
|
+------------------+------------------------+----------------------+
|
|
| ``'cumulative'`` | ``SortKey.CUMULATIVE`` | cumulative time |
|
|
+------------------+------------------------+----------------------+
|
|
| ``'cumtime'`` | N/A | cumulative time |
|
|
+------------------+------------------------+----------------------+
|
|
| ``'file'`` | N/A | file name |
|
|
+------------------+------------------------+----------------------+
|
|
| ``'filename'`` | ``SortKey.FILENAME`` | file name |
|
|
+------------------+------------------------+----------------------+
|
|
| ``'module'`` | N/A | file name |
|
|
+------------------+------------------------+----------------------+
|
|
| ``'ncalls'`` | N/A | call count |
|
|
+------------------+------------------------+----------------------+
|
|
| ``'pcalls'`` | ``SortKey.PCALLS`` | primitive call count |
|
|
+------------------+------------------------+----------------------+
|
|
| ``'line'`` | ``SortKey.LINE`` | line number |
|
|
+------------------+------------------------+----------------------+
|
|
| ``'name'`` | ``SortKey.NAME`` | function name |
|
|
+------------------+------------------------+----------------------+
|
|
| ``'nfl'`` | ``SortKey.NFL`` | name/file/line |
|
|
+------------------+------------------------+----------------------+
|
|
| ``'stdname'`` | ``SortKey.STDNAME`` | standard name |
|
|
+------------------+------------------------+----------------------+
|
|
| ``'time'`` | ``SortKey.TIME`` | internal time |
|
|
+------------------+------------------------+----------------------+
|
|
| ``'tottime'`` | N/A | internal time |
|
|
+------------------+------------------------+----------------------+
|
|
|
|
All sorts on statistics are in descending order (most time consuming
|
|
first), while name, file, and line number sorts are ascending
|
|
(alphabetical).
|
|
|
|
The difference between ``SortKey.NFL`` and ``SortKey.STDNAME`` is that
|
|
NFL sorts line numbers numerically while STDNAME sorts them as strings.
|
|
``sort_stats(SortKey.NFL)`` is equivalent to
|
|
``sort_stats(SortKey.NAME, SortKey.FILENAME, SortKey.LINE)``.
|
|
|
|
For backward compatibility, the numeric arguments ``-1``, ``0``, ``1``,
|
|
and ``2`` are also accepted, meaning ``'stdname'``, ``'calls'``,
|
|
``'time'``, and ``'cumulative'`` respectively.
|
|
|
|
.. versionadded:: 3.7
|
|
The :class:`SortKey` enum.
|
|
|
|
.. method:: reverse_order()
|
|
|
|
Reverse the current sort order.
|
|
|
|
By default, the sort direction is chosen appropriately for the sort key
|
|
(descending for time-based keys, ascending for name-based keys). This
|
|
method inverts that choice.
|
|
|
|
.. method:: print_stats(*restrictions)
|
|
|
|
Print a report of the profiling statistics.
|
|
|
|
The output includes a header line summarizing the data, followed by a
|
|
table of function statistics sorted according to the last
|
|
:meth:`sort_stats` call.
|
|
|
|
Restrictions filter the output. Each restriction is either:
|
|
|
|
- An integer: limits output to that many entries
|
|
- A float between 0.0 and 1.0: selects that fraction of entries
|
|
- A string: matches function names via regular expression
|
|
|
|
Restrictions are applied sequentially. For example::
|
|
|
|
print_stats(.1, 'foo:')
|
|
|
|
First limits to the top 10%, then filters to functions matching 'foo:'.
|
|
|
|
.. method:: print_callers(*restrictions)
|
|
|
|
Print the callers of each function in the statistics.
|
|
|
|
For each function in the filtered results, shows which functions called
|
|
it and how often.
|
|
|
|
With :mod:`profiling.tracing` (or ``cProfile``), each caller line
|
|
shows three numbers: the number of calls from that caller, and the
|
|
total and cumulative times for those specific calls.
|
|
|
|
Accepts the same restriction arguments as :meth:`print_stats`.
|
|
|
|
.. method:: print_callees(*restrictions)
|
|
|
|
Print the functions called by each function in the statistics.
|
|
|
|
This is the inverse of :meth:`print_callers`, showing which functions
|
|
each listed function called.
|
|
|
|
Accepts the same restriction arguments as :meth:`print_stats`.
|
|
|
|
.. method:: get_stats_profile()
|
|
|
|
Return a ``StatsProfile`` object containing the statistics.
|
|
|
|
The returned object provides programmatic access to the profile data,
|
|
with function names mapped to ``FunctionProfile`` objects
|
|
containing timing and call count information.
|
|
|
|
.. versionadded:: 3.9
|
|
|
|
|
|
.. class:: SortKey
|
|
|
|
An enumeration of valid sort keys for :meth:`Stats.sort_stats`.
|
|
|
|
.. attribute:: CALLS
|
|
|
|
Sort by call count.
|
|
|
|
.. attribute:: CUMULATIVE
|
|
|
|
Sort by cumulative time.
|
|
|
|
.. attribute:: FILENAME
|
|
|
|
Sort by file name.
|
|
|
|
.. attribute:: LINE
|
|
|
|
Sort by line number.
|
|
|
|
.. attribute:: NAME
|
|
|
|
Sort by function name.
|
|
|
|
.. attribute:: NFL
|
|
|
|
Sort by name, then file, then line number (numeric line sort).
|
|
|
|
.. attribute:: PCALLS
|
|
|
|
Sort by primitive (non-recursive) call count.
|
|
|
|
.. attribute:: STDNAME
|
|
|
|
Sort by standard name (string-based line sort).
|
|
|
|
.. attribute:: TIME
|
|
|
|
Sort by internal time (time in function excluding subcalls).
|
|
|
|
|
|
.. _pstats-cli:
|
|
|
|
Command-line interface
|
|
======================
|
|
|
|
The :mod:`pstats` module can be invoked as a script to interactively browse
|
|
profile data::
|
|
|
|
python -m pstats profile_output.prof
|
|
|
|
This opens a line-oriented interface (built on :mod:`cmd`) for examining the
|
|
statistics. Type ``help`` at the prompt for available commands.
|
|
|
|
|
|
.. seealso::
|
|
|
|
:mod:`profiling`
|
|
Overview of Python profiling tools.
|
|
|
|
:mod:`profiling.tracing`
|
|
Deterministic tracing profiler.
|
|
|
|
:mod:`profiling.sampling`
|
|
Statistical sampling profiler.
|