The output format for js-benchmarks is going to change, and while the
webhook parsing the results has not yet been updated pin the
js-benchmark checkout to a specific commit.
This will build both aarch64 and x86_64 flatpaks nightly. The test jobs
I did with the default GitHub runners took ~1h30m for aarch64 and
~2h30m for x86_64, but this will hopefully improve over time as the
cache is built up. And be better time-wise on the blacksmith runners.
Disk space is a large concern with these flatpak builds. A naive script
on my local machine showed that we needed a max of 10.3 GiB of free
space for the x86_64 build, with clang as the compiler. For some reason,
building with default gcc 14 uses more disk space than that, bumping
up against the default 14 GiB free space limit guaranteed by GitHub for
their default runners.
This was a cache race condition between the fuzzers and sanitizer
builds, where the vcpkg binary cache could have been updated before the
sanitizer builds started doing their vcpkg install, causing the source
assets to never be updated at all.
These existed because of their `vcpkg_ci` base preset, whose goal was to
enable vcpkg-supported caching on GitHub. We now handle vcpkg caching in
the CI steps, removing the necessity for all these *_CI preset variants.
This allows us to reuse `inputs.build_preset` in the test step, and we
can do away with the Windows-specific test step since it is now almost
identical to the Linux/macOS test step.
I've left in the Windows_CI presets, because that one actually results
in a different set of compiled files compared to the
Windows_Experimental presets.
I've set up a shared vcpkg source assets cache using Azurite. By
default, it runs in read-only mode allowing anyone who builds Ladybird
to use the cache to speed up downloads of source tarballs. This should
alleviate some of the more slower vcpkgs downloads I've been seeing
lately.
CI runs on master put vcpkg in readwrite mode, updating the cache for
everyone.
For macOS, we now default to ENABLE_QT=OFF and use the same build step
as for the other platforms. We keep a single separate macOS + Qt build
step to prevent bitrotting that part of the code.
The Windows step ran in a different directory and skipped the install
step; we can just use the Linux behavior here.
The CI runs are supposed to upload test dumps for failed LibWeb tests as
artifacts. However, the path for the artifact upload was wrong, it
likely regressed in 9a5b31ccd1.
Having the pull_request 'labeled' event trigger the main CI is much too
impactful, since it cancels running jobs and does not start anything if
the label is anything else than 'windows'. Let's go with a different
approach and put the Windows CI job in a separate workflow.
How this works:
* Jobs are only run if the 'windows' label is present on the PR.
* If a PR is opened or updated, existing jobs are canceled.
* If a PR is (un)labeled, existing jobs are only canceled if the label
was 'windows'. Other labels cause the job to be rescheduled.
As far as I can see, there's no way to prevent the job from being
rescheduled when labels other than 'windows' are being added or removed.
However, by not canceling the existing Windows job, we can try to create
a cache so the next run will be much quicker.
Each matrix item now defines whether it is enabled, and if not, the job
conditionally disables itself. Apparantly we cannot use expressions in
`matrix.exclude`, but it should work in the items' definitions.
This has the added benefit that this makes the job show up as "skipped",
which could serve as a hint that it could be run on a PR.
Our CI matrix now includes Windows if the 'windows' label is added to
the PR. The job still takes too long to include it for every PR, but it
is certainly useful to have it run on Windows-specific PRs and PRs that
might break Windows builds.
Github Actions just updated windows-2025 to LLVM 20, which is the
minimum version required for us to build and run tests with sanitizers
Now that we've added support, enable the Sanitizer build in CI.
As we now are only officially supporting the Ninja generator for
Windows, the old preset structure can now be simplified. Also, we
now follow the naming conventions of the non-hidden presets.
We could use flake8 for linting, but ruff is compatible with black
formatting out-of-the-box. It also seems to catch more than flake8,
such as unnecessary f-strings.
By default, matrix jobs generate a name for themselves by concatenating
their job name and all matrix variables into a big string. Changing the
runner labels causes the job name to change, which means we need to go
into GitHub and change the required checks since those are name-based.
Give all matrix workflows an explicit, more stable name.
The GitHub-provided runner frequently times out and is plain slow most
of the time. And since Blacksmith does not yet offer macOS runners,
let's use our self-hosted runner that is currently idle most of the time
(when it's not running JS benchmarks).
This is now always a JSON array of runner labels, inside a string. We
need it to be a string because `workflow_call` works with inputs that
do not natively support an array type.
No functional changes.
We haven't built a universal binary in over 11 months. The name is
confusing, and actually breaks esvu on macOS. The fact that nobody
has complained suggests that this is not a common use case..
Their cache action only works on their runners. For jobs that run on
other runners, we have use the default cache action. At least until they
update their cache product to work or fallback on other runners.
Everything that's not self-hosted or macOS is now pointing to
Blacksmith.sh. Nightly jobs and JS artifact builds use 8VCPU machines,
while regular integration builds & tests use 16VCPU machines.
The people over at Blacksmith.sh have generously offered usage of their
runners for our organization, so let's try to switch over some simple
workflows. The runners should be drop-in replacements.
When we try to retrieve benchmark results in the webhook call, we cannot
use the `head_sha` parameter since the workflow run might have a
different `head_sha` associated with it than the upstream workflow run.
This can happen when the JS repl binary workflow runs, a new commit is
pushed to master, followed by a JS benchmarks workflow run causing this
latter run to be associated with a different commit ID.
This extends the webhook payload to include the current run ID, which
can eventually be used by the webhook script to specifically download
the benchmark results associated with the current run.
Additionally, this changes the JS artifact download to use the upstream
run ID which seems nicer to do anyway.
This job uses the windows_ci_ninja preset to build just the
components and unit tests that are known to work with ClangCL on the
amd64-pc-windows-msvc target triple.
As a nightly job, its failures are non-blocking for any PRs, though
they should be fixed eventually or the job will get turned off by
email-annoyed maintainers.
For our js-benchmarks and libjs-test262 workflow runs, we already know
that they're provisioned with these repositories and can skip adding the
key and repo altogether.
We were using both wget and curl arbitrarily; use curl exclusively since
that is installed by default on our machines and containers. Fixes the
js-benchmarks workflow.
