cpython/Lib/test/support
Victor Stinner 31c2945f14
gh-108834: regrtest reruns failed tests in subprocesses (#108839)
When using --rerun option, regrtest now re-runs failed tests
in verbose mode in fresh worker processes to have more
deterministic behavior. So it can write its final report even
if a test killed a worker progress.

Add --fail-rerun option to regrtest: exit with non-zero exit code
if a test failed pass passed when re-run in verbose mode (in a
fresh process). That's now more useful since tests can pass
when re-run in a fresh worker progress, whereas they failed
when run after other tests when tests are run sequentially.

Rename --verbose2 option (-w) to --rerun. Keep --verbose2 as a
deprecated alias.

Changes:

* Fix and enhance statistics in regrtest summary. Add "(filtered)"
  when --match and/or --ignore options are used.
* Add RunTests class.
* Add TestResult.get_rerun_match_tests() method
* Rewrite code to serialize/deserialize worker arguments as JSON
  using a new WorkerJob class.
* Fix stats when a test is run with --forever --rerun.
* If failed test names cannot be parsed, log a warning and don't
  filter tests.
* test_regrtest.test_rerun_success() now uses a marker file, since
  the test is re-run in a separated process.
* Add tests on normalize_test_name() function.
* Add test_success() and test_skip() tests to test_regrtest.
2023-09-03 21:37:15 +00:00
..
_hypothesis_stubs GH-86275: Implementation of hypothesis stubs for property-based tests, with zoneinfo tests (#22863) 2023-05-12 08:35:53 -04:00
__init__.py gh-108834: regrtest reruns failed tests in subprocesses (#108839) 2023-09-03 21:37:15 +00:00
ast_helper.py gh-100518: Add tests for ast.NodeTransformer (#100521) 2023-01-21 21:44:41 +00:00
asynchat.py gh-72719: Remove asyncore and asynchat modules (#96580) 2022-11-08 16:48:58 +01:00
asyncore.py gh-102799: use sys.exception() instead of sys.exc_info() in tests (#103293) 2023-04-06 11:08:25 +01:00
bytecode_helper.py gh-104615: don't make unsafe swaps in apply_static_swaps (#104620) 2023-05-18 21:22:03 +00:00
hashlib_helper.py bpo-45042: Now test classes decorated with requires_hashdigest are not skipped (GH-28060) 2021-09-04 23:42:36 +03:00
hypothesis_helper.py gh-86275: improve Hypothesis configuration for CI and local runs (#104468) 2023-05-21 05:52:29 -06:00
import_helper.py gh-104310: Add importlib.util.allowing_all_extensions() (gh-104311) 2023-05-08 16:56:01 -06:00
interpreters.py Update test.support.interpreters to include missing RunFailedError import (#103841) 2023-06-26 17:30:20 +02:00
logging_helper.py bpo-40275: Avoid importing logging in test.support (GH-19601) 2020-04-25 11:35:18 +03:00
os_helper.py gh-100809: Fix handling of drive-relative paths in pathlib.Path.absolute() (GH-100812) 2023-02-17 14:08:14 +00:00
script_helper.py bpo-40280: Skip subprocess-based tests on wasm32-emscripten (GH-30615) 2022-01-25 08:09:06 +01:00
socket_helper.py gh-104773: PEP 594: Remove the nntplib module (#104894) 2023-05-24 21:15:43 +00:00
testcase.py gh-103791: Make contextlib.suppress also act on exceptions within an ExceptionGroup (#103792) 2023-04-24 22:17:02 +00:00
testresult.py gh-108834: regrtest reruns failed tests in subprocesses (#108839) 2023-09-03 21:37:15 +00:00
threading_helper.py gh-104812: Run Pending Calls in any Thread (gh-104813) 2023-06-13 15:02:19 -06:00
warnings_helper.py gh-103109: Document ignore_warnings() test support helper (#103110) 2023-04-03 00:18:25 +02:00