cpython/Doc
Kevin Modzelewski 214eb2cce5
gh-90536: Add support for the BOLT post-link binary optimizer (gh-95908)
* Add support for the BOLT post-link binary optimizer

Using [bolt](https://github.com/llvm/llvm-project/tree/main/bolt)
provides a fairly large speedup without any code or functionality
changes. It provides roughly a 1% speedup on pyperformance, and a
4% improvement on the Pyston web macrobenchmarks.

It is gated behind an `--enable-bolt` configure arg because not all
toolchains and environments are supported. It has been tested on a
Linux x86_64 toolchain, using llvm-bolt built from the LLVM 14.0.6
sources (their binary distribution of this version did not include bolt).

Compared to [a previous attempt](https://github.com/faster-cpython/ideas/issues/224),
this commit uses bolt's preferred "instrumentation" approach, as well as adds some non-PIE
flags which enable much better optimizations from bolt.

The effects of this change are a bit more dependent on CPU microarchitecture
than other changes, since it optimizes i-cache behavior which seems
to be a bit more variable between architectures. The 1%/4% numbers
were collected on an Intel Skylake CPU, and on an AMD Zen 3 CPU I
got a slightly larger speedup (2%/4%), and on a c6i.xlarge EC2 instance
I got a slightly lower speedup (1%/3%).

The low speedup on pyperformance is not entirely unexpected, because
BOLT improves i-cache behavior, and the benchmarks in the pyperformance
suite are small and tend to fit in i-cache.

This change uses the existing pgo profiling task (`python -m test --pgo`),
though I was able to measure about a 1% macrobenchmark improvement by
using the macrobenchmarks as the training task. I personally think that
both the PGO and BOLT tasks should be updated to use macrobenchmarks,
but for the sake of splitting up the work this PR uses the existing pgo task.

* Simplify the build flags

* Add a NEWS entry

* Update Makefile.pre.in

Co-authored-by: Dong-hee Na <donghee.na92@gmail.com>

* Update configure.ac

Co-authored-by: Dong-hee Na <donghee.na92@gmail.com>

* Add myself to ACKS

* Add docs

* Other review comments

* fix tab/space issue

* Make it more clear that --enable-bolt is experimental

* Add link to bolt's github page

Co-authored-by: Dong-hee Na <donghee.na92@gmail.com>
2022-08-19 06:33:54 +09:00
..
c-api gh-93103: Doc uses PyConfig rather than deprecated vars (#96070) 2022-08-18 16:58:38 +02:00
data gh-93274: Expose receiving vectorcall in the Limited API (GH-95717) 2022-08-08 14:12:05 +02:00
distributing gh-85454: Remove links from historical mentions of distutils (GH-95192) 2022-07-25 12:20:09 +02:00
extending gh-91838: Use HTTPS links in docs for resources which redirect to HTTPS (GH-95527) 2022-08-04 10:13:49 +03:00
faq gh-91838: Resolve HTTP links which redirect to HTTPS (GH-95642) 2022-08-04 13:30:05 +03:00
howto Docs: replace 'currying' by 'partial function'. (#91814) 2022-08-11 01:23:40 -05:00
includes gh-95273: Improve sqlite3.complete_statement docs (#95840) 2022-08-12 01:05:12 +02:00
install gh-91838: Resolve HTTP links which redirect to HTTPS (GH-95642) 2022-08-04 13:30:05 +03:00
installing gh-93851: Fix all broken links in Doc/ (GH-93853) 2022-06-21 20:55:18 +02:00
library GH-95861: Add support for Spearman's rank correlation coefficient (GH-95863) 2022-08-18 13:48:27 -05:00
reference gh-94619: Remove long deprecated methods module_repr() and load_module() (#94624) 2022-08-04 17:24:26 -07:00
tools gh-95451: Update docs for wasm32-emscripten and -wasi platforms (GH-95452) 2022-08-02 21:00:41 +02:00
tutorial Doc: Add omitted secondary prompt in inputoutput tutorial (GH-30317) 2022-07-23 17:25:42 +08:00
using gh-90536: Add support for the BOLT post-link binary optimizer (gh-95908) 2022-08-19 06:33:54 +09:00
whatsnew gh-90536: Add support for the BOLT post-link binary optimizer (gh-95908) 2022-08-19 06:33:54 +09:00
about.rst gh-91838: Resolve HTTP links which redirect to HTTPS (GH-95642) 2022-08-04 13:30:05 +03:00
bugs.rst Add link to documentation translation list (#91560) 2022-04-16 03:18:10 +02:00
conf.py gh-91207: Fix CSS bug in Windows CHM help file and add deprecation message (GH-95607) 2022-08-03 20:23:20 +01:00
contents.rst gh-85454: Remove distutils documentation (#95239) 2022-07-25 15:50:46 +02:00
copyright.rst Update copyright year to 2022. (GH-30335) 2022-01-02 12:08:48 -08:00
glossary.rst gh-84910: Tweak IDLE Glossary entry (#95866) 2022-08-10 22:04:36 -04:00
license.rst gh-91838: Use HTTPS links in docs for resources which redirect to HTTPS (GH-95527) 2022-08-04 10:13:49 +03:00
make.bat Restore default role check in make check. (#92290) 2022-05-15 17:34:52 +02:00
Makefile Speedup: build docs in parallel (GH-92733) 2022-05-16 13:52:00 +02:00
README.rst Link to GitHub instead of BPO for CPython bug tracker (GH-92221) 2022-05-03 22:40:36 +08:00
requirements.txt bpo-47126: Update to canonical PEP URLs specified by PEP 676 (GH-32124) 2022-03-30 12:00:27 +01:00

Python Documentation README
~~~~~~~~~~~~~~~~~~~~~~~~~~~

This directory contains the reStructuredText (reST) sources to the Python
documentation.  You don't need to build them yourself, `prebuilt versions are
available <https://docs.python.org/dev/download.html>`_.

Documentation on authoring Python documentation, including information about
both style and markup, is available in the "`Documenting Python
<https://devguide.python.org/documenting/>`_" chapter of the
developers guide.


Building the docs
=================

The documentation is built with several tools which are not included in this
tree but are maintained separately and are available from
`PyPI <https://pypi.org/>`_.

* `Sphinx <https://pypi.org/project/Sphinx/>`_
* `blurb <https://pypi.org/project/blurb/>`_
* `python-docs-theme <https://pypi.org/project/python-docs-theme/>`_

The easiest way to install these tools is to create a virtual environment and
install the tools into there.

Using make
----------

To get started on UNIX, you can create a virtual environment and build
documentation with the commands::

  make venv
  make html

The virtual environment in the ``venv`` directory will contain all the tools
necessary to build the documentation downloaded and installed from PyPI.
If you'd like to create the virtual environment in a different location,
you can specify it using the ``VENVDIR`` variable.

You can also skip creating the virtual environment altogether, in which case
the Makefile will look for instances of ``sphinxbuild`` and ``blurb``
installed on your process ``PATH`` (configurable with the ``SPHINXBUILD`` and
``BLURB`` variables).

On Windows, we try to emulate the Makefile as closely as possible with a
``make.bat`` file. If you need to specify the Python interpreter to use,
set the PYTHON environment variable.

Available make targets are:

* "clean", which removes all build files and the virtual environment.

* "clean-venv", which removes the virtual environment directory.

* "venv", which creates a virtual environment with all necessary tools
  installed.

* "html", which builds standalone HTML files for offline viewing.

* "htmlview", which re-uses the "html" builder, but then opens the main page
  in your default web browser.

* "htmlhelp", which builds HTML files and a HTML Help project file usable to
  convert them into a single Compiled HTML (.chm) file -- these are popular
  under Microsoft Windows, but very handy on every platform.

  To create the CHM file, you need to run the Microsoft HTML Help Workshop
  over the generated project (.hhp) file.  The make.bat script does this for
  you on Windows.

* "latex", which builds LaTeX source files as input to "pdflatex" to produce
  PDF documents.

* "text", which builds a plain text file for each source file.

* "epub", which builds an EPUB document, suitable to be viewed on e-book
  readers.

* "linkcheck", which checks all external references to see whether they are
  broken, redirected or malformed, and outputs this information to stdout as
  well as a plain-text (.txt) file.

* "changes", which builds an overview over all versionadded/versionchanged/
  deprecated items in the current version. This is meant as a help for the
  writer of the "What's New" document.

* "coverage", which builds a coverage overview for standard library modules and
  C API.

* "pydoc-topics", which builds a Python module containing a dictionary with
  plain text documentation for the labels defined in
  ``tools/pyspecific.py`` -- pydoc needs these to show topic and keyword help.

* "suspicious", which checks the parsed markup for text that looks like
  malformed and thus unconverted reST.

* "check", which checks for frequent markup errors.

* "serve", which serves the build/html directory on port 8000.

* "dist", (Unix only) which creates distributable archives of HTML, text,
  PDF, and EPUB builds.


Without make
------------

First, install the tool dependencies from PyPI.

Then, from the ``Doc`` directory, run ::

   sphinx-build -b<builder> . build/<builder>

where ``<builder>`` is one of html, text, latex, or htmlhelp (for explanations
see the make targets above).

Deprecation header
==================

You can define the ``outdated`` variable in ``html_context`` to show a
red banner on each page redirecting to the "latest" version.

The link points to the same page on ``/3/``, sadly for the moment the
language is lost during the process.


Contributing
============

Bugs in the content should be reported to the
`Python bug tracker <https://github.com/python/cpython/issues>`_.

Bugs in the toolset should be reported to the tools themselves.

You can also send a mail to the Python Documentation Team at docs@python.org,
and we will process your request as soon as possible.

If you want to help the Documentation Team, you are always welcome.  Just send
a mail to docs@python.org.