mirror of
https://github.com/python/cpython.git
synced 2026-02-22 07:00:51 +00:00
01602ae403
pstats is really useful or profiling and printing the output of the execution of some block of code, but I've found on multiple occasions when I'd like to access this output directly in an easily usable dictionary on which I can further analyze or manipulate.
The proposal is to add a function called get_profile_dict inside of pstats that'll automatically return this data the data in an easily accessible dict.
The output of the following script:
```
import cProfile, pstats
import pprint
from pstats import func_std_string, f8
def fib(n):
if n == 0:
return 0
if n == 1:
return 1
return fib(n-1) + fib(n-2)
pr = cProfile.Profile()
pr.enable()
fib(5)
pr.create_stats()
ps = pstats.Stats(pr).sort_stats('tottime', 'cumtime')
def get_profile_dict(self, keys_filter=None):
"""
Returns a dict where the key is a function name and the value is a dict
with the following keys:
- ncalls
- tottime
- percall_tottime
- cumtime
- percall_cumtime
- file_name
- line_number
keys_filter can be optionally set to limit the key-value pairs in the
retrieved dict.
"""
pstats_dict = {}
func_list = self.fcn_list[:] if self.fcn_list else list(self.stats.keys())
if not func_list:
return pstats_dict
pstats_dict["total_tt"] = float(f8(self.total_tt))
for func in func_list:
cc, nc, tt, ct, callers = self.stats[func]
file, line, func_name = func
ncalls = str(nc) if nc == cc else (str(nc) + '/' + str(cc))
tottime = float(f8(tt))
percall_tottime = -1 if nc == 0 else float(f8(tt/nc))
cumtime = float(f8(ct))
percall_cumtime = -1 if cc == 0 else float(f8(ct/cc))
func_dict = {
"ncalls": ncalls,
"tottime": tottime, # time spent in this function alone
"percall_tottime": percall_tottime,
"cumtime": cumtime, # time spent in the function plus all functions that this function called,
"percall_cumtime": percall_cumtime,
"file_name": file,
"line_number": line
}
func_dict_filtered = func_dict if not keys_filter else { key: func_dict[key] for key in keys_filter }
pstats_dict[func_name] = func_dict_filtered
return pstats_dict
pp = pprint.PrettyPrinter(depth=6)
pp.pprint(get_profile_dict(ps))
```
will produce:
```
{"<method 'disable' of '_lsprof.Profiler' objects>": {'cumtime': 0.0,
'file_name': '~',
'line_number': 0,
'ncalls': '1',
'percall_cumtime': 0.0,
'percall_tottime': 0.0,
'tottime': 0.0},
'create_stats': {'cumtime': 0.0,
'file_name': '/usr/local/Cellar/python/3.7.4/Frameworks/Python.framework/Versions/3.7/lib/python3.7/cProfile.py',
'line_number': 50,
'ncalls': '1',
'percall_cumtime': 0.0,
'percall_tottime': 0.0,
'tottime': 0.0},
'fib': {'cumtime': 0.0,
'file_name': 'get_profile_dict.py',
'line_number': 5,
'ncalls': '15/1',
'percall_cumtime': 0.0,
'percall_tottime': 0.0,
'tottime': 0.0},
'total_tt': 0.0}
```
As an example, this can be used to generate a stacked column chart using various visualization tools which will assist in easily identifying program bottlenecks.
https://bugs.python.org/issue37958
Automerge-Triggered-By: @gpshead
|
||
|---|---|---|
| .. | ||
| c-api | ||
| data | ||
| distributing | ||
| distutils | ||
| extending | ||
| faq | ||
| howto | ||
| includes | ||
| install | ||
| installing | ||
| library | ||
| reference | ||
| tools | ||
| tutorial | ||
| using | ||
| whatsnew | ||
| about.rst | ||
| bugs.rst | ||
| conf.py | ||
| contents.rst | ||
| copyright.rst | ||
| glossary.rst | ||
| license.rst | ||
| make.bat | ||
| Makefile | ||
| README.rst | ||
| requirements.txt | ||
| runtime.txt | ||
Python Documentation README ~~~~~~~~~~~~~~~~~~~~~~~~~~~ This directory contains the reStructuredText (reST) sources to the Python documentation. You don't need to build them yourself, `prebuilt versions are available <https://docs.python.org/dev/download.html>`_. Documentation on authoring Python documentation, including information about both style and markup, is available in the "`Documenting Python <https://devguide.python.org/documenting/>`_" chapter of the developers guide. Building the docs ================= The documentation is built with several tools which are not included in this tree but are maintained separately and are available from `PyPI <https://pypi.org/>`_. * `Sphinx <https://pypi.org/project/Sphinx/>`_ * `blurb <https://pypi.org/project/blurb/>`_ * `python-docs-theme <https://pypi.org/project/python-docs-theme/>`_ The easiest way to install these tools is to create a virtual environment and install the tools into there. Using make ---------- To get started on UNIX, you can create a virtual environment with the command :: make venv That will install all the tools necessary to build the documentation. Assuming the virtual environment was created in the ``venv`` directory (the default; configurable with the VENVDIR variable), you can run the following command to build the HTML output files:: make html By default, if the virtual environment is not created, the Makefile will look for instances of sphinxbuild and blurb installed on your process PATH (configurable with the SPHINXBUILD and BLURB variables). On Windows, we try to emulate the Makefile as closely as possible with a ``make.bat`` file. If you need to specify the Python interpreter to use, set the PYTHON environment variable instead. Available make targets are: * "clean", which removes all build files. * "venv", which creates a virtual environment with all necessary tools installed. * "html", which builds standalone HTML files for offline viewing. * "htmlview", which re-uses the "html" builder, but then opens the main page in your default web browser. * "htmlhelp", which builds HTML files and a HTML Help project file usable to convert them into a single Compiled HTML (.chm) file -- these are popular under Microsoft Windows, but very handy on every platform. To create the CHM file, you need to run the Microsoft HTML Help Workshop over the generated project (.hhp) file. The make.bat script does this for you on Windows. * "latex", which builds LaTeX source files as input to "pdflatex" to produce PDF documents. * "text", which builds a plain text file for each source file. * "epub", which builds an EPUB document, suitable to be viewed on e-book readers. * "linkcheck", which checks all external references to see whether they are broken, redirected or malformed, and outputs this information to stdout as well as a plain-text (.txt) file. * "changes", which builds an overview over all versionadded/versionchanged/ deprecated items in the current version. This is meant as a help for the writer of the "What's New" document. * "coverage", which builds a coverage overview for standard library modules and C API. * "pydoc-topics", which builds a Python module containing a dictionary with plain text documentation for the labels defined in `tools/pyspecific.py` -- pydoc needs these to show topic and keyword help. * "suspicious", which checks the parsed markup for text that looks like malformed and thus unconverted reST. * "check", which checks for frequent markup errors. * "serve", which serves the build/html directory on port 8000. * "dist", (Unix only) which creates distributable archives of HTML, text, PDF, and EPUB builds. Without make ------------ First, install the tool dependencies from PyPI. Then, from the ``Doc`` directory, run :: sphinx-build -b<builder> . build/<builder> where ``<builder>`` is one of html, text, latex, or htmlhelp (for explanations see the make targets above). Deprecation header ================== You can define the ``outdated`` variable in ``html_context`` to show a red banner on each page redirecting to the "latest" version. The link points to the same page on ``/3/``, sadly for the moment the language is lost during the process. Contributing ============ Bugs in the content should be reported to the `Python bug tracker <https://bugs.python.org>`_. Bugs in the toolset should be reported to the tools themselves. You can also send a mail to the Python Documentation Team at docs@python.org, and we will process your request as soon as possible. If you want to help the Documentation Team, you are always welcome. Just send a mail to docs@python.org.