gh-109566, regrtest: Add --fast-ci and --slow-ci options (#109570)

* Add --fast-ci and --slow-ci options to libregrtest:

  * --fast-ci uses a default timeout of 10 minutes and "-u all,-cpu"
    (skip slowest tests).
  * --slow-ci uses a default timeout of 20 minues and "-u all" (run
    all tests).

* regrtest header now lists test resources.
* Makefile changes:

  * "make test", "make hostrunnertest" and "make coverage-report" now
    use --fast-ci option and TESTTIMEOUT variable.
  * "make buildbottest" now uses "--slow-ci". Remove options which
    became redundant with "--slow-ci".
  * "make testall" and "make testuniversal" now use --slow-ci option
    and TESTTIMEOUT variable.
  * "make testall" now uses "find -exec rm ..." instead of
    "find ... -print|xargs rm ...", same as "make clean".

* GitHub Actions workflow:

  * Ubuntu and Address Sanitizer jobs now use "make test". Remove
    options which became redundant with "--fast-ci".
  * Windows jobs now use --fast-ci option.
  * Use -j0 to detect the number of CPUs.

* Set Makefile TESTTIMEOUT default to an empty string, since
  --slow-ci and --fast-ci use different default timeout. It's now
  accepted to pass "--timeout=" to regrtest: treated as not timeout.
* Tools/scripts/run_tests.py now uses --fast-ci option.
* Tools/buildbot/test.bat now uses --slow-ci option. Remove
  --timeout=1200 option, redundant with --slow-ci.
This commit is contained in:
Victor Stinner 2023-09-26 17:22:50 +02:00 committed by GitHub
parent 19bf398695
commit 859618c8cd
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23
11 changed files with 161 additions and 51 deletions

View file

@ -182,7 +182,7 @@ jobs:
- name: Display build info
run: .\python.bat -m test.pythoninfo
- name: Tests
run: .\PCbuild\rt.bat -p Win32 -d -q -uall -u-cpu -rwW --slowest --timeout=1200 -j0
run: .\PCbuild\rt.bat -p Win32 -d -q --fast-ci
build_win_amd64:
name: 'Windows (x64)'
@ -201,7 +201,7 @@ jobs:
- name: Display build info
run: .\python.bat -m test.pythoninfo
- name: Tests
run: .\PCbuild\rt.bat -p x64 -d -q -uall -u-cpu -rwW --slowest --timeout=1200 -j0
run: .\PCbuild\rt.bat -p x64 -d -q --fast-ci
build_win_arm64:
name: 'Windows (arm64)'
@ -252,7 +252,7 @@ jobs:
- name: Display build info
run: make pythoninfo
- name: Tests
run: make buildbottest TESTOPTS="-j4 -uall,-cpu"
run: make test
build_ubuntu:
name: 'Ubuntu'
@ -319,7 +319,7 @@ jobs:
run: sudo mount $CPYTHON_RO_SRCDIR -oremount,rw
- name: Tests
working-directory: ${{ env.CPYTHON_BUILDDIR }}
run: xvfb-run make buildbottest TESTOPTS="-j4 -uall,-cpu"
run: xvfb-run make test
build_ubuntu_ssltests:
name: 'Ubuntu SSL tests with OpenSSL'
@ -535,7 +535,7 @@ jobs:
- name: Display build info
run: make pythoninfo
- name: Tests
run: xvfb-run make buildbottest TESTOPTS="-j4 -uall,-cpu"
run: xvfb-run make test
all-required-green: # This job does nothing and is only used for the branch protection
name: All required checks pass