Commit graph

30258 commits

Author SHA1 Message Date
Guido van Rossum
da86db56cb
gh-106529: Implement JUMP_FORWARD in uops (with test) (#106651)
Note that this may generate two SAVE_IP uops in a row.
Removing unneeded SAVE_IP uops is the optimizer's job.
2023-07-11 15:13:57 -07:00
Serhiy Storchaka
4bf43710d1
gh-106307: C API: Add PyMapping_GetOptionalItem() function (GH-106308)
Also add PyMapping_GetOptionalItemString() function.
2023-07-11 23:04:12 +03:00
Serhiy Storchaka
579aa89e68
gh-106521: Add PyObject_GetOptionalAttr() function (GH-106522)
It is a new name of former _PyObject_LookupAttr().

Add also PyObject_GetOptionalAttrString().
2023-07-11 22:13:27 +03:00
Guido van Rossum
cabd6e8a10
gh-106529: Support JUMP_BACKWARD in Tier 2 (uops) (#106543)
During superblock generation, a JUMP_BACKWARD instruction is translated to either a JUMP_TO_TOP micro-op (when the target of the jump is exactly the beginning of the superblock, closing the loop), or a SAVE_IP + EXIT_TRACE pair, when the jump goes elsewhere.

The new JUMP_TO_TOP instruction includes a CHECK_EVAL_BREAKER() call, so a closed loop can still be interrupted.
2023-07-11 18:08:10 +00:00
Kirill Podoprigora
292ac4bfe9
gh-102541: Add test case for help() for non_existent_module (#106340)
Test fix for when one enters, for instance, 'abd' at the 'help>' prompt.
---------

Co-authored-by: Terry Jan Reedy <tjreedy@udel.edu>
2023-07-11 14:04:09 -04:00
Jelle Zijlstra
945d3cbf2e
gh-106403: Restore weakref support for TypeVar and friends (#106418) 2023-07-11 08:43:24 -07:00
Terry Jan Reedy
a2d54d4e8a
gh-106498: Revert incorrect colorsys.rgb_to_hls change (#106627)
gh-86618 assumed a-b-c = a-(b+c) = a-d where d = b+d.
For floats 2.0, 1.0, and 0.9999999999999999, this assumption
is false.  The net change of 1.1102230246251565e-16 to 0.0
results in division by 0.  Revert the replacement.  Add test.
2023-07-11 11:07:20 -04:00
Ethan Furman
95b7426f45
gh-105497: [Enum] Fix flag mask inversion when unnamed flags exist (#106468)
For example:

    class Flag(enum.Flag):
        A = 0x01
        B = 0x02
        MASK = 0xff

    ~Flag.MASK is Flag(0)
2023-07-11 13:35:54 +02:00
Victor Stinner
1f2921b72c
gh-106572: Convert PyObject_DelAttr() to a function (#106611)
* Convert PyObject_DelAttr() and PyObject_DelAttrString() macros to
  functions.
* Add PyObject_DelAttr() and PyObject_DelAttrString() functions to
  the stable ABI.
* Replace PyObject_SetAttr(obj, name, NULL) with
  PyObject_DelAttr(obj, name).
2023-07-11 11:38:22 +02:00
Chris Withers
e6379f72cb
Remove unused branches from mock module (#106617)
* lambda has a name of __none__, but no async lambda so this branch is not needed

* _get_signature_object only returns None for bound builtins. There are no async builtins so this branch isn't needed

* Exclude  a couple of methods from coverage checking in the downstream rolling backport of mock
2023-07-11 08:52:12 +00:00
Dong-hee Na
115df8491a
gh-104635: Add a test case for variables that have a dependency. (gh-106583) 2023-07-11 10:14:53 +09:00
Guido van Rossum
22988c323a
gh-106529: Implement POP_JUMP_IF_XXX uops (#106551)
- Hand-written uops JUMP_IF_{TRUE,FALSE}.
  These peek at the top of the stack.
  The jump target (in superblock space) is absolute.

- Hand-written translation for POP_JUMP_IF_{TRUE,FALSE},
  assuming the jump is unlikely.
  Once we implement jump-likelihood profiling,
  we can implement the jump-unlikely case (in another PR).

- Tests (including some test cleanup).

- Improvements to len(ex) and ex[i] to expose the whole trace.
2023-07-10 16:04:26 -07:00
Thomas Dwyer
18dfbd0357
gh-102988: Detect email address parsing errors and return empty tuple to indicate the parsing error (old API) (#105127)
Detect email address parsing errors and return empty tuple to indicate the parsing error (old API). This fixes or at least ameliorates CVE-2023-27043.

---------

Co-authored-by: Gregory P. Smith <greg@krypto.org>
2023-07-10 23:00:55 +00:00
Louis Paulot
6782fc0502
gh-94777: Fix deadlock in ProcessPoolExecutor (#94784)
Fixes a hang in multiprocessing process pool executor when a child process crashes and code could otherwise block on writing to the pipe.  See GH-94777 for more details.
2023-07-10 21:45:27 +00:00
Ijtaba Hussain
9d582250d8
gh-103186: Fix or catch 'extra' stderr output from unittests (#103196)
Reduce test noise by fixing or catching and testing stderr messages from individual tests.

test_cmd_line_script.test_script_as_dev_fd calls spawn_python and hence subprocess.Popen with incompatible arguments. On POSIX, pass_fds forces close_fds to be True (subprocess.py line 848). Correct the call.

test_uuid.test_cli_namespace_required_for_uuid3: when the namespace is omitted, uuid.main calls argparse.Argument_Parser.error, which prints to stderr before calling sys.exit, which raises SystemExit. Unittest assertRaises catches the exception but not the previous output. Catch the output and test it.

test_warnings.test_catchwarnings_with_simplefilter_error similarly prints before raising. Catch the output and test it.
---------

Co-authored-by: Oleg Iarygin <oleg@arhadthedev.net>
2023-07-10 16:29:03 -04:00
Serhiy Storchaka
51ea664d18
gh-99593: Add tests for Unicode C API (part 3) (GH-104728)
Add tests for codecs.
2023-07-10 14:04:34 +03:00
Hugo van Kemenade
34c14147a2
gh-106487: Allow the 'count' argument of str.replace to be a keyword (#106488) 2023-07-10 12:52:36 +03:00
Serhiy Storchaka
8cb6f9761e
Move implementation specific RE tests to separate class (GH-106563) 2023-07-09 12:48:36 +03:00
Serhiy Storchaka
74ec02e949
gh-106510: Fix DEBUG output for atomic group (GH-106511) 2023-07-08 14:31:25 +03:00
Radislav Chugunov
69a39bd9ad
gh-105873: Make _xxsubinterpreters use exception type name in shared exception (#105874) 2023-07-08 08:44:50 +00:00
Radislav Chugunov
2ef1dc37f0
gh-106524: Fix a crash in _sre.template() (GH-106525)
Some items remained uninitialized if _sre.template() was called with invalid
indices. Then attempt to clear them in the destructor led to dereferencing
of uninitialized pointer.
2023-07-08 10:47:01 +03:00
Desmond Cheong
1fb9bd222b
gh-103200: Fix performance issues with zipimport.invalidate_caches() (GH-103208)
Co-authored-by: blurb-it[bot] <43283697+blurb-it[bot]@users.noreply.github.com>
Co-authored-by: Brett Cannon <brett@python.org>
2023-07-07 22:02:13 +00:00
Nikita Sobolev
6e6a4cd523
gh-106300: Improve assertRaises(Exception) usages in tests (GH-106302) 2023-07-07 13:42:40 -07:00
Guido van Rossum
b3648f036e
gh-104584: Allow unspecialized instructions in superblocks (#106497)
This adds several of unspecialized opcodes to superblocks:

TO_BOOL, BINARY_SUBSCR, STORE_SUBSCR,
UNPACK_SEQUENCE, LOAD_GLOBAL, LOAD_ATTR,
COMPARE_OP, BINARY_OP.

While we may not want that eventually, for now this helps finding bugs.

There is a rudimentary test checking for UNPACK_SEQUENCE.

Once we're ready to undo this, that would be simple:
just replace the call to variable_used_unspecialized
with a call to variable_used (as shown in a comment).
Or add individual opcdes to FORBIDDEN_NAMES_IN_UOPS.
2023-07-07 18:03:27 +00:00
Mark Shannon
24fb627ea7
GH-106057: Handle recursion errors in inline class calls properly. (GH-106108) 2023-07-07 11:09:26 +01:00
Guido van Rossum
e1d45b8ed4
gh-104584: Handle EXTENDED_ARG in superblock creation (#106489)
With test.
2023-07-06 16:46:06 -07:00
Gregory P. Smith
c60df361ce
gh-90876: Restore the ability to import multiprocessing when sys.executable is None (#106464)
Prevent `multiprocessing.spawn` from failing to *import* in environments
where `sys.executable` is `None`.  This regressed in 3.11 with the addition
of support for path-like objects in multiprocessing.

Adds a test decorator to have tests only run when part of test_multiprocessing_spawn to `_test_multiprocessing.py` so we can start to avoid re-running the same not-global-state specific test in all 3 modes when there is no need.
2023-07-06 22:46:50 +00:00
Guido van Rossum
76fac7bce5
gh-104584: Clean up and fix uops tests and fix crash (#106492)
The uops test wasn't testing anything by default,
and was failing when run with -Xuops.

Made the two executor-related context managers global,
so TestUops can use them (notably `with temporary_optimizer(opt)`).

Made clear_executor() a little more thorough.

Fixed a crash upon finalizing a uop optimizer,
by adding a `tp_dealloc` handler.
2023-07-06 15:45:56 -07:00
Dustin Rodrigues
67a798888d
closes gh-106479: fix typo in __cplusplus macro (gh-106480) 2023-07-06 17:34:00 -05:00
Mario Corchero
56353b1002
gh-106458: Mark testthreadingmock.py with @requires_working_threading (GH-106366)
Mark `testthreadingmock.py` with `threading_helper.requires_working_threading`.

Also add longer delays to reduce the change of a race conditions on the tests that validate short timeouts.
2023-07-06 10:54:45 -07:00
Carl Meyer
104d7b760f
gh-105340: include hidden fast-locals in locals() (#105715)
* gh-105340: include hidden fast-locals in locals()
2023-07-05 17:05:02 -06:00
Carl Meyer
838406b4fc
gh-106292: restore checking __dict__ in cached_property.__get__ (#106380)
* gh-106292: restore checking __dict__ in cached_property.__get__

Co-authored-by: Dong-hee Na <donghee.na92@gmail.com>
2023-07-05 17:01:35 -06:00
Dong-hee Na
217f47d6e5
gh-96844: Improve error message of list.remove (gh-106455) 2023-07-06 07:19:49 +09:00
Erlend E. Aasland
9d1d4f9c73
gh-64595: Fix regression in file write logic in Argument Clinic (#106449)
Revert the two commits that introduced the regressions:

 - gh-104152
 - gh-104507
2023-07-05 13:23:22 +02:00
Erlend E. Aasland
a941bd6c53
gh-104683: Rename Lib/test/clinic.test as Lib/test/clinic.test.c (#106443) 2023-07-05 08:33:05 +00:00
Erlend E. Aasland
2fb9480c83
gh-106368: Add tests for formatting helpers in Argument Clinic (#106415)
Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
2023-07-04 22:35:57 +00:00
Mario Corchero
2dfc7fae78
gh-61215: Rename wait_until_any_call to wait_until_any_call_with (#106414)
mock: Rename `wait_until_any_call` to `wait_until_any_call_with`

Rename the method to be more explicit that it expects the args and
kwargs to wait for.
2023-07-04 18:34:43 +01:00
Inada Naoki
8a4bba8b97
gh-106162: array: suppress warning in test_array (#106404)
array: suppress warning in test_array
2023-07-05 02:02:49 +09:00
Mark Shannon
318ea2c72e
GH-106360: Support very basic superblock introspection (#106422)
* Add len() and indexing support to uop superblocks.
2023-07-04 17:23:00 +01:00
Lysandros Nikolaou
dfe4de2038
gh-106396: Special-case empty format spec to gen empty JoinedStr node (#106401) 2023-07-04 14:19:08 +02:00
Erlend E. Aasland
8f6df5e9cb
gh-106368: Add tests for permutation helpers in Argument Clinic (#106407)
Added new test class PermutationTests()
2023-07-04 11:36:40 +00:00
Victor Stinner
8a73b57b9b
gh-106320: Remove _PyUnicode_TransformDecimalAndSpaceToASCII() (#106398)
Remove private _PyUnicode_TransformDecimalAndSpaceToASCII() and other
private _PyUnicode C API functions: move them to the internal C API
(pycore_unicodeobject.h). No longer most of these functions.

Replace _testcapi.unicode_transformdecimalandspacetoascii() with
_testinternal._PyUnicode_TransformDecimalAndSpaceToASCII().
2023-07-04 08:59:09 +00:00
Erlend E. Aasland
3406f8cce5
gh-106368: Increase Argument Clinic test coverage (#106389)
Add:

- test_disallowed_gropuing__no_matching_bracket
- test_double_slash
2023-07-03 23:58:27 +00:00
Erlend E. Aasland
648688c137
gh-106368: Harden Argument Clinic parser tests (#106384) 2023-07-03 23:10:47 +00:00
Erlend E. Aasland
3ee8dac7a1
gh-106368: Clean up Argument Clinic tests (#106373) 2023-07-03 21:57:20 +00:00
Barney Gale
b4efdf8cda
GH-106330: Fix matching of empty path in pathlib.PurePath.match() (GH-106331)
We match paths using the `_lines` attribute, which is derived from the
path's string representation. The bug arises because an empty path's string
representation is `'.'` (not `''`), which is matched by the `'*'` wildcard.
2023-07-03 21:29:44 +01:00
Erlend E. Aasland
7f4c8121db
gh-106368: Increase Argument Clinic test coverage (#106369)
Add tests for 'self' and 'defining_class' converter requirements.
2023-07-03 20:16:50 +00:00
Erlend E. Aasland
0da4c883cf
gh-106359: Fix corner case bugs in Argument Clinic converter parser (#106361)
DSLParser.parse_converter() could return unusable kwdicts in some rare cases

Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
2023-07-03 13:14:59 +00:00
Mario Corchero
d65b783b69
gh-61215: New mock to wait for multi-threaded events to happen (#16094)
mock: Add `ThreadingMock` class

Add a new class that allows to wait for a call to happen by using
`Event` objects. This mock class can be used to test and validate
expectations of multithreading code.

It uses two attributes for events to distinguish calls with any argument
and calls with specific arguments.

The calls with specific arguments need a lock to prevent two calls in
parallel from creating the same event twice.

The timeout is configured at class and constructor level to allow users
to set a timeout, we considered passing it as an argument to the
function but it could collide with a function parameter. Alternatively
we also considered passing it as positional only but from an API
caller perspective it was unclear what the first number meant on the
function call, think `mock.wait_until_called(1, "arg1", "arg2")`, where
1 is the timeout.

Lastly we also considered adding the new attributes to magic mock
directly rather than having a custom mock class for multi threading
scenarios, but we preferred to have specialised class that can be
composed if necessary. Additionally, having added it to `MagicMock`
directly would have resulted in `AsyncMock` having this logic, which
would not work as expected, since when if user "waits" on a
coroutine does not have the same meaning as waiting on a standard
call.

Co-authored-by: Karthikeyan Singaravelan <tir.karthi@gmail.com>
2023-07-03 07:56:54 +01:00
Victor Stinner
feb51f3a64
gh-106320: Remove private _PyTraceMalloc C API functions (#106324)
* Remove private _PyTraceMalloc C API functions: move them to the
  internal C API.
* Don't export most of these functions anymore, but still export
  _PyTraceMalloc_GetTraceback() used by tests.
* Rename Include/tracemalloc.h to Include/cpython/tracemalloc.h
2023-07-02 00:49:18 +00:00