cpython/Misc
Kevin Modzelewski 214eb2cce5
gh-90536: Add support for the BOLT post-link binary optimizer (gh-95908)
* Add support for the BOLT post-link binary optimizer

Using [bolt](https://github.com/llvm/llvm-project/tree/main/bolt)
provides a fairly large speedup without any code or functionality
changes. It provides roughly a 1% speedup on pyperformance, and a
4% improvement on the Pyston web macrobenchmarks.

It is gated behind an `--enable-bolt` configure arg because not all
toolchains and environments are supported. It has been tested on a
Linux x86_64 toolchain, using llvm-bolt built from the LLVM 14.0.6
sources (their binary distribution of this version did not include bolt).

Compared to [a previous attempt](https://github.com/faster-cpython/ideas/issues/224),
this commit uses bolt's preferred "instrumentation" approach, as well as adds some non-PIE
flags which enable much better optimizations from bolt.

The effects of this change are a bit more dependent on CPU microarchitecture
than other changes, since it optimizes i-cache behavior which seems
to be a bit more variable between architectures. The 1%/4% numbers
were collected on an Intel Skylake CPU, and on an AMD Zen 3 CPU I
got a slightly larger speedup (2%/4%), and on a c6i.xlarge EC2 instance
I got a slightly lower speedup (1%/3%).

The low speedup on pyperformance is not entirely unexpected, because
BOLT improves i-cache behavior, and the benchmarks in the pyperformance
suite are small and tend to fit in i-cache.

This change uses the existing pgo profiling task (`python -m test --pgo`),
though I was able to measure about a 1% macrobenchmark improvement by
using the macrobenchmarks as the training task. I personally think that
both the PGO and BOLT tasks should be updated to use macrobenchmarks,
but for the sake of splitting up the work this PR uses the existing pgo task.

* Simplify the build flags

* Add a NEWS entry

* Update Makefile.pre.in

Co-authored-by: Dong-hee Na <donghee.na92@gmail.com>

* Update configure.ac

Co-authored-by: Dong-hee Na <donghee.na92@gmail.com>

* Add myself to ACKS

* Add docs

* Other review comments

* fix tab/space issue

* Make it more clear that --enable-bolt is experimental

* Add link to bolt's github page

Co-authored-by: Dong-hee Na <donghee.na92@gmail.com>
2022-08-19 06:33:54 +09:00
..
NEWS.d gh-90536: Add support for the BOLT post-link binary optimizer (gh-95908) 2022-08-19 06:33:54 +09:00
rhel7 gh-95957: Add instructions for Tcl/Tk and OpenSSL on RHEL/CentOS 7 (#95964) 2022-08-16 13:44:02 +02:00
ACKS gh-90536: Add support for the BOLT post-link binary optimizer (gh-95908) 2022-08-19 06:33:54 +09:00
coverity_model.c bpo-35808: Retire pgen and use pgen2 to generate the parser (GH-11814) 2019-03-01 15:34:44 -08:00
gdbinit gdbinit: Use proper define syntax (GH-19557) 2020-04-23 05:49:26 -07:00
HISTORY Fix links to old SF bugs (#95648) 2022-08-04 18:12:35 +02:00
indent.pro
Porting bpo-30737: Update DevGuide links to new URL (GH-3228) 2017-08-30 09:37:43 -07:00
python-config.in bpo-38468 : Refactor python-config (#16749) 2019-10-15 11:18:47 -03:00
python-config.sh.in bpo-37925: Mention --embed in python-config usage (GH-15458) 2019-08-26 23:45:36 +02:00
python-embed.pc.in bpo-36721: Add --embed option to python-config (GH-13500) 2019-05-23 03:30:23 +02:00
python.man gh-90300: split --help output into separate options (#30331) 2022-06-01 05:50:01 -04:00
python.pc.in bpo-36721: Add --embed option to python-config (GH-13500) 2019-05-23 03:30:23 +02:00
README bpo-23469: Delete Wing IDE configuration files (GH-30067) 2021-12-14 15:01:13 +02:00
README.AIX bpo-42087: Remove support for AIX 5.3 and below (GH-22830) 2020-11-16 16:16:10 +01:00
README.coverity
README.valgrind closes docs: remove references to Py_USING_MEMORY_DEBUGGER (GH-30284) 2021-12-29 16:20:42 -06:00
requirements-test.txt Update tzdata to 2020.3 (GH-22856) 2020-10-21 06:40:43 -07:00
SpecialBuilds.txt gh-78607: Replace __ltrace__ with __lltrace__ (GH-91619) 2022-04-16 18:57:00 -04:00
stable_abi.toml gh-93274: Expose receiving vectorcall in the Limited API (GH-95717) 2022-08-08 14:12:05 +02:00
svnmap.txt bpo-32159: Revert Misc/svnmap.txt (#4639) 2017-11-29 18:58:33 +01:00
valgrind-python.supp bpo-35561: Supress valgrind false alarm on epoll_ctl(event) (GH-18060) 2020-01-19 23:38:37 +01:00
vgrindefs

Python Misc subdirectory
========================

This directory contains files that wouldn't fit in elsewhere.  Some
documents are only of historic importance.

Files found here
----------------

ACKS                    Acknowledgements
gdbinit                 Handy stuff to put in your .gdbinit file, if you use gdb
HISTORY                 News from previous releases -- oldest last
indent.pro              GNU indent profile approximating my C style
NEWS                    News for this release (for some meaning of "this")
Porting                 Mini-FAQ on porting to new platforms
python-config.in        Python script template for python-config
python.man              UNIX man page for the python interpreter
python.pc.in            Package configuration info template for pkg-config
README                  The file you're reading now
README.AIX              Information about using Python on AIX
README.coverity         Information about running Coverity's Prevent on Python
README.valgrind         Information for Valgrind users, see valgrind-python.supp
SpecialBuilds.txt       Describes extra symbols you can set for debug builds
svnmap.txt              Map of old SVN revs and branches to hg changeset ids,
                        help history-digging
valgrind-python.supp    Valgrind suppression file, see README.valgrind
vgrindefs               Python configuration for vgrind (a generic pretty printer)