gh-103142: Upgrade binary builds and CI to OpenSSL 1.1.1u (GH-105174)
Upgrade builds to OpenSSL 1.1.1u.
This OpenSSL version addresses a pile if less-urgent CVEs since 1.1.1t.
The Mac/BuildScript/build-installer.py was already updated.
Also updates _ssl_data_111.h from OpenSSL 1.1.1u, _ssl_data_300.h from 3.0.9, and adds a new _ssl_data_31.h file from 3.1.1 along with the ssl.c code to use it.
Manual edits to the _ssl_data_300.h file prevent it from removing any existing definitions in case those exist in some peoples builds and were important (avoiding regressions during backporting).
backports of this prior to 3.12 will not include the openssl 3.1 header.
(cherry picked from commit ede89af605)
Co-authored-by: Gregory P. Smith [Google] <greg@krypto.org>
We now compile a `re.Pattern` object for the entire pattern. This is made
more difficult by `fnmatch` not treating directory separators as special
when evaluating wildcards (`*`, `?`, etc), and so we arrange the path parts
onto separate *lines* in a string, and ensure we don't set `re.DOTALL`.
Co-authored-by: Hugo van Kemenade <hugovk@users.noreply.github.com>
Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
gh-105017: Include CRLF lines in strings and column numbers (GH-105030)
(cherry picked from commit 96fff35325)
Co-authored-by: Marta Gómez Macías <mgmacias@google.com>
Co-authored-by: Pablo Galindo <pablogsal@gmail.com>
For backwards compatibility, accept backslashes as path separators in
`PurePosixPath` if an instance of `PureWindowsPath` is supplied.
This restores behaviour from Python 3.11.
(cherry picked from commit 328422ce61)
Co-authored-by: Barney Gale <barney.gale@gmail.com>
Co-authored-by: Gregory P. Smith <greg@krypto.org>
Use `str.lower()` rather than `ntpath.normcase()` to normalize case of
Windows paths. This restores behaviour from Python 3.11.
(cherry picked from commit ad0be361c9)
Co-authored-by: Barney Gale <barney.gale@gmail.com>
Co-authored-by: Gregory P. Smith <greg@krypto.org>
gh-104972: Ensure that line attributes in tokens in the tokenize module are correct (GH-104975)
(cherry picked from commit 3fdb55c482)
Co-authored-by: Pablo Galindo Salgado <Pablogsal@gmail.com>
gh-102024: Reduced _idle_semaphore.release calls (GH-102025)
Reduced _idle_semaphore.release calls in concurrent.futures.thread._worker
_idle_semaphore.release() is now only called if only work_queue is empty.
---------
(cherry picked from commit 0242e9a57a)
Co-authored-by: Andrii Kuzmin <jack.cvr@gmail.com>
Co-authored-by: blurb-it[bot] <43283697+blurb-it[bot]@users.noreply.github.com>
Co-authored-by: Shantanu <12621235+hauntsaninja@users.noreply.github.com>
gh-104372: Drop the GIL around the vfork() call. (GH-104782)
On Linux where the `subprocess` module can use the `vfork` syscall for
faster spawning, prevent the parent process from blocking other threads
by dropping the GIL while it waits for the vfork'ed child process `exec`
outcome. This prevents spawning a binary from a slow filesystem from
blocking the rest of the application.
Fixes GH-104372.
(cherry picked from commit d08679212d)
Co-authored-by: Gregory P. Smith <gps@python.org>
gh-99108: Refresh HACL* (GH-104808)
Refresh HACL* from upstream to improve SHA2 performance and fix a 32-bit issue in SHA3.
(cherry picked from commit 160321e530)
Co-authored-by: Jonathan Protzenko <protz@microsoft.com>
gh-104719: IDLE - test existence of all tokenize references. (GH-104767)
Class editor.IndentSearcher contains all editor references to tokenize module.
Module io tokenize reference cover those other modules.
(cherry picked from commit e561c09975)
Co-authored-by: Terry Jan Reedy <tjreedy@udel.edu>
Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com>
gh-99108: Release the GIL around hashlib built-in computation (GH-104675)
This matches the GIL releasing behavior of our existing `_hashopenssl`
module, extending it to the HACL* built-ins.
Includes adding comments to better describe the ENTER/LEAVE macros
purpose and explain the lock strategy in both existing and new code.
(cherry picked from commit 2e5d8a90aa)
Co-authored-by: Gregory P. Smith [Google] <greg@krypto.org>
Apply BOLT optimizations to libpython for shared builds. Most of the C
code is in libpython so it is critical to apply BOLT there fully realize
BOLT benefits.
This change also reworks how BOLT instrumentation is applied. It
effectively removes the readelf based logic added in gh-101525 and
replaces it with a mechanism that saves a copy of the pre-bolt binary
and restores that copy when necessary. This allows us to perform BOLT
optimizations without having to manually delete the output binary to
force a new bolt run.
Also:
- add a clean-bolt target for purging BOLT files and hook that up to the
clean target
- .gitignore BOLT related files
Before and after this refactor, `make` will no-op after a previous run.
Both versions should also share common make DAG deficiencies where
targets fail to trigger as often as they need to or can trigger
prematurely in certain scenarios. e.g. after this change you may need to
`rm profile-bolt-stamp` to force a BOLT run because there aren't
appropriate non-phony targets for BOLT's make target to depend on.
To make it easier to iterate on custom BOLT settings, the flags to pass
to instrumentation and application are now defined in configure and can
be overridden by passing BOLT_INSTRUMENT_FLAGS and BOLT_APPLY_FLAGS.
Fix a race condition in the internal `multiprocessing.process` cleanup
logic that could manifest as an unintended `AttributeError` when calling
`BaseProcess.close()`.
---------
Co-authored-by: Oleg Iarygin <oleg@arhadthedev.net>
Co-authored-by: Gregory P. Smith <greg@krypto.org>
* Support for conversion specifiers o (octal) and X (uppercase hexadecimal).
* Support for length modifiers j (intmax_t) and t (ptrdiff_t).
* Length modifiers are now applied to all integer conversions.
* Support for wchar_t C strings (%ls and %lV).
* Support for variable width and precision (*).
* Support for flag - (left alignment).
This commit replaces the Python implementation of the tokenize module with an implementation
that reuses the real C tokenizer via a private extension module. The tokenize module now implements
a compatibility layer that transforms tokens from the C tokenizer into Python tokenize tokens for backward
compatibility.
As the C tokenizer does not emit some tokens that the Python tokenizer provides (such as comments and non-semantic newlines), a new special mode has been added to the C tokenizer mode that currently is only used via
the extension module that exposes it to the Python layer. This new mode forces the C tokenizer to emit these new extra tokens and add the appropriate metadata that is needed to match the old Python implementation.
Co-authored-by: Pablo Galindo <pablogsal@gmail.com>
bpo-17258: `multiprocessing` now supports stronger HMAC algorithms for inter-process connection authentication rather than only HMAC-MD5.
Signed-off-by: Christian Heimes <christian@python.org>
gpshead: I Reworked to be more robust while keeping the idea.
The protocol modification idea remains, but we now take advantage of the
message length as an indicator of legacy vs modern protocol version. No
more regular expression usage. We now default to HMAC-SHA256, but do so
in a way that will be compatible when communicating with older clients
or older servers. No protocol transition period is needed.
More integration tests to verify these claims remain true are required. I'm
unaware of anyone depending on multiprocessing connections between
different Python versions.
---------
Signed-off-by: Christian Heimes <christian@python.org>
Co-authored-by: Gregory P. Smith [Google] <greg@krypto.org>
This PR updates `math.nextafter` to add a new `steps` argument. The behaviour is as though `math.nextafter` had been called `steps` times in succession.
---------
Co-authored-by: Mark Dickinson <mdickinson@enthought.com>
