Webkit and Mobile safari are comically unreliable, will fail for unexplainable reasons and are very hard to run locally in comparison with the other supported platforms. I do not remember the last time where these two platforms were able to catch a regression where the other platforms did not.
I would like to stress, for the historical record, that many hours has been devoted into adjusting the tests and following best practices to make these two platforms more stable but despite those, IMO wasted, efforts these two platforms are causing many hours of wasted CPU time simply because they are flaky and make (new) contributors nervous if their change contains a regression or not.
To my knowledge, the tests are not broken for these two platforms. If you go to the issue tracker you will not find issues by users that use these two platforms and report that Forgejo is broken. It does not reflect reality.
This is the sunk cost fallacy, bite the bullet and agree that these platforms will not contribute positively to Forgejo's excellent test suite.
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/10103
Reviewed-by: Michael Kriese <michael.kriese@gmx.de>
Reviewed-by: Mathieu Fenniak <mfenniak@noreply.codeberg.org>
Reviewed-by: 0ko <0ko@noreply.codeberg.org>
Co-authored-by: Gusted <postmaster@gusted.xyz>
Co-committed-by: Gusted <postmaster@gusted.xyz>
Goals:
- speedup
- less flakiness
- best practices and more use
- documentation
config:
- sync ports in Makefile and playwright config
(otherwise, some tests fail locally because they assert the full URL including the (wrong) port)
- even more generous timeouts
- limit workers to one again (because I finally understand how
Playwright works)
- allow nested functions to group them together with the related test
all:
- deprecate waitForLoadState('networkidle')
- it is discouraged as per https://playwright.dev/docs/api/class-page#page-wait-for-load-state
- I could not find a usage that seems to require it actually (see
added documentation in README)
- adding an exception should be made explicitly
- it does not do what you might expect anyway in most cases
- only log in when necessary
webauthn:
- verify that login is possible after disabling key
- otherwise, the cleanup was not necessary after the previous refactor to create a fresh user each
issue-sidebar / WIP toggle:
- split into smaller chunks
- restore original state first
- add missed assertion to fix race condition (not waiting
before state was reached)
- explicitly toggle the state to detect mismatch earlier
issue-sidebar / labels:
- restore original state first
- better waiting for background request
