cpython/Misc
Petr Viktorin 49f6beb56a
[3.12] gh-113993: Make interned strings mortal (GH-120520, GH-121364, GH-121903, GH-122303) (#123065)
This backports several PRs for gh-113993, making interned strings mortal so they can be garbage-collected when no longer needed.

* Allow interned strings to be mortal, and fix related issues (GH-120520)

  * Add an InternalDocs file describing how interning should work and how to use it.

  * Add internal functions to *explicitly* request what kind of interning is done:
    - `_PyUnicode_InternMortal`
    - `_PyUnicode_InternImmortal`
    - `_PyUnicode_InternStatic`

  * Switch uses of `PyUnicode_InternInPlace` to those.

  * Disallow using `_Py_SetImmortal` on strings directly.
    You should use `_PyUnicode_InternImmortal` instead:
    - Strings should be interned before immortalization, otherwise you're possibly
      interning a immortalizing copy.
    - `_Py_SetImmortal` doesn't handle the `SSTATE_INTERNED_MORTAL` to
      `SSTATE_INTERNED_IMMORTAL` update, and those flags can't be changed in
      backports, as they are now part of public API and version-specific ABI.

  * Add private `_only_immortal` argument for `sys.getunicodeinternedsize`, used in refleak test machinery.

   Make sure the statically allocated string singletons are unique. This means these sets are now disjoint:
    - `_Py_ID`
    - `_Py_STR` (including the empty string)
    - one-character latin-1 singletons

    Now, when you intern a singleton, that exact singleton will be interned.

  * Add a `_Py_LATIN1_CHR` macro, use it instead of `_Py_ID`/`_Py_STR` for one-character latin-1 singletons everywhere (including Clinic).

  * Intern `_Py_STR` singletons at startup.

  * Beef up the tests. Cover internal details (marked with `@cpython_only`).

  * Add lots of assertions

* Don't immortalize in PyUnicode_InternInPlace; keep immortalizing in other API (GH-121364)

  * Switch PyUnicode_InternInPlace to _PyUnicode_InternMortal, clarify docs

  * Document immortality in some functions that take `const char *`

  This is PyUnicode_InternFromString;
  PyDict_SetItemString, PyObject_SetAttrString;
  PyObject_DelAttrString; PyUnicode_InternFromString;
  and the PyModule_Add convenience functions.

  Always point out a non-immortalizing alternative.

  * Don't immortalize user-provided attr names in _ctypes

* Immortalize names in code objects to avoid crash (GH-121903)

* Intern latin-1 one-byte strings at startup (GH-122303)

There are some 3.12-specific changes, mainly to allow statically allocated strings in deepfreeze. (In 3.13, deepfreeze switched to the general `_Py_ID`/`_Py_STR`.)

Co-authored-by: Eric Snow <ericsnowcurrently@gmail.com>
2024-09-27 13:28:48 -07:00
..
NEWS.d [3.12] gh-113993: Make interned strings mortal (GH-120520, GH-121364, GH-121903, GH-122303) (#123065) 2024-09-27 13:28:48 -07:00
rhel7 gh-95957: Add instructions for Tcl/Tk and OpenSSL on RHEL/CentOS 7 (#95964) 2022-08-16 13:44:02 +02:00
ACKS [3.12] gh-124248: Fix crash in struct when processing 0p fields (GH-124251) (#124278) 2024-09-20 10:29:43 +00:00
coverity_model.c [3.12] Fix typos (#123775) (#123867) 2024-09-09 13:22:13 +00:00
externals.spdx.json [3.12] gh-123418: Update OpenSSL to 3.0.15 on Windows (GH-123691) 2024-09-04 16:21:37 -05:00
HISTORY [3.12] gh-123492: Remove unnecessary :func: parentheses (gh-123493) (#123512) 2024-08-30 20:09:55 +03:00
indent.pro
Porting
python-config.in
python-config.sh.in
python-embed.pc.in
python.man [3.12] gh-121101: Document -Wall option (an alias for -Walways) (GH-121102) (#121147) 2024-06-29 13:57:22 +05:30
python.pc.in gh-99942: python.pc on android/cygwin should link to libpython per configure.ac (GH-100356) 2023-02-21 17:21:24 -08:00
README gh-102809: Remove gdbinit mention in Misc/README (#103269) 2023-04-08 00:04:23 -07:00
README.AIX bpo-42087: Remove support for AIX 5.3 and below (GH-22830) 2020-11-16 16:16:10 +01:00
README.coverity
README.valgrind closes docs: remove references to Py_USING_MEMORY_DEBUGGER (GH-30284) 2021-12-29 16:20:42 -06:00
sbom.spdx.json [3.12] gh-123678: Upgrade libexpat 2.6.3 (GH-123689) (GH-123708) 2024-09-04 20:19:33 +00:00
SpecialBuilds.txt gh-78607: Replace __ltrace__ with __lltrace__ (GH-91619) 2022-04-16 18:57:00 -04:00
stable_abi.toml [3.12] gh-102304: Fix Py_INCREF() stable ABI in debug mode (#104763) (#105352) 2023-06-06 16:22:04 +02:00
svnmap.txt
valgrind-python.supp [3.12] gh-121096: Ignore dlopen() leaks in Valgrind suppression file (GH-121097) (#121123) 2024-06-28 11:17:30 +00:00
vgrindefs

Python Misc subdirectory
========================

This directory contains files that wouldn't fit in elsewhere.  Some
documents are only of historic importance.

Files found here
----------------

ACKS                    Acknowledgements
HISTORY                 News from previous releases -- oldest last
indent.pro              GNU indent profile approximating my C style
NEWS                    News for this release (for some meaning of "this")
Porting                 Mini-FAQ on porting to new platforms
python-config.in        Python script template for python-config
python.man              UNIX man page for the python interpreter
python.pc.in            Package configuration info template for pkg-config
README                  The file you're reading now
README.AIX              Information about using Python on AIX
README.coverity         Information about running Coverity's Prevent on Python
README.valgrind         Information for Valgrind users, see valgrind-python.supp
SpecialBuilds.txt       Describes extra symbols you can set for debug builds
svnmap.txt              Map of old SVN revs and branches to hg changeset ids,
                        help history-digging
valgrind-python.supp    Valgrind suppression file, see README.valgrind
vgrindefs               Python configuration for vgrind (a generic pretty printer)