This is the only job that actually needs to use the environment secrets,
so the environment must be declared. We were using secrets in the
prepare job to do a permissions check, but this is unnecessary, because
that job does not do anything that is security sensitive.
Only the upload job needs to have these permission checks and these are
already included in the upload-release-artifact composite action.
We haven't really being tagged in new PRs anymore, so it looks like this
is not working.
Revert for now until we can fix this, so we don't miss PRs.
This reverts commit 04f124f615.
This should improve timezone coverage for keeping the bazel build
working due to libc breakages, e.g. #209433 broke things and the bazel
fixer bot sent out #209689. But since it wasn't landed until just
recently, manual fixes were needed for other changes like #209449. This
wouldn't be as bad if the bazel fixer bot could handle layered
breakages.
The macOS premerge job built inline instead of via a `monolithic-*.sh`
script, so it never emitted a GitHub Job Summary or JUnit XML like
Linux/Windows do on failure
This resolves a TODO to allow selecting which machine we're running on.
As I am working on automation to dispatch these benchmarking jobs, it
turns out that we always want to specify the machine, and we always want
to specify a single machine to run per workflow, since this is the only
way we can track which commits have been attempted (or are still in
flight) on a given machine.
Therefore, the ability to run benchmarks on all machines at once is
removed from this workflow (but not from the related PR A/B benchmarking
workflow).
This patch also includes the machine name and the commit being
benchmarked into the runs's name, which is necessary for automation to
know what the workflow was benchmarking.
The LNT instance on http://lnt.llvm.org was wiped since I created the
original schema, and we can now start over clean with a new test suite
named `libcxx`.
The upload-release-artifact action uses the require-team-membership
action so we need to make sure that latter is checkout out when calling
upload-release-artifact.
Since we're expanding libc++'s pre-commit CI to add other types of
tests, (e.g. tools tests, performance tests, etc), it makes sense to be
a bit more precise about what each workflow does, and to use a
consistent pattern across workflow names.
Also, update stale references to old names in the documentation and in
some infrastructure scripts.
We have a few scripts under libcxx/utils, and they wouldn't be tested on
a regular basis. As a result, changes to e.g. the build system could
break these scripts, which in turn could break some Github workflows
like the libc++ PR benchmarking workflow.
This patch adds a new `test-tools` configuration to `run-buildbot` which
allows testing these tools locally, and a new dedicated Github workflow
to run it.
On the macOS self-hosted runners, we need to install dependencies via
Homebrew and pinning the Xcode version is good for reproducibility. This
applies the guidelines documented in #211622 to libc++'s CI jobs.
We have various scripts that build and test the library at pinpointed
commits: benchmark-historical, run-benchmarks, build-at-commit and
test-at-commit. They were handling the compiler in different ways: some
scripts would just run the libc++ build (or test suite configuration)
without specifying the compiler, which means the $CXX environment
variable was used if present. Other scripts (e.g. run-benchmarks) would
accept the compiler as an argument, but would fail to pass it down when
configuring the test suite, which led to issues.
This patch passes the compiler explicitly in all scripts: this removes
any potential confusion around how the compiler should be specified (env
var or argument). The only exception is build-at-commit, where the
compiler is still specified by passing the appropriate CMake arguments.
The reason for this exception is that passing arguments to CMake is
actually the way we want to configure aspects of the build (and the test
suite) in the long term, it's just that the test suite doesn't support
this cleanly due to the CMake/Lit split at the moment.
In the longer term, `test-at-commit` should also lose its `--compiler`
argument in favour of being able to pass CMake parameters to the test
suite configuration, but we are not there yet.
Add Pre-commit CI to test compiler-rt builtins with libc to check it
works without issues
Split from #197950
Part of #197824
---------
Co-authored-by: Michael Jones <michaelrj@google.com>
f0ae26c9 allowed specifying multiple commits to /cherry-pick, but that
only works for plain commit hashes; it doesn't work for the URL form.
Adjust the regex to allow multiple URLs.
Switch GitHub API interactions to use actions/github-script instead of
pygithub, as is done in other Github workflows in the repository. This
is simpler and it also resolves the issue that if setting up the Python
virtual environment fails, there's no way to communicate the issue back
since doing so would require pygithub.
As a drive-by, also report the failure when a step in the libc++ PR
benchmark job fails, instead of leaving a "Running benchmarks in ..."
comment indefinitely.
Assisted by Claude
Fixes#210985
This should increase our capacity and allow better turn around times,
and potentially unblock additional test coverage.
Note that this change implies that the self-hosted runners provide a
version of Xcode that is supported by libc++ (the latest released one),
which is currently the case.
Missed thing in #210414. We need to symlink the LLVM tools that are
under `/usr/lib/llvm-23/bin/` to avoid errors like this because the LLVM
tools have a `-23` suffix when they're installed inside the container
```
# RUN: at line 2
/usr/bin/clang-23 -gline-tables-only -Wthread-safety -Wthread-safety-reference -Wthread-safety-beta -DCOMPILER_RT_HAS_FLOAT16 -nobuiltininc -I/__w/llvm-project/llvm-project/compiler-rt/include -idirafter /usr/lib/llvm-23/lib/clang/23/include -resource-dir=/__w/llvm-project/llvm-project/build/compiler-rt -Wl,-rpath,/__w/llvm-project/llvm-project/build/compiler-rt/lib/linux -fno-builtin -I /__w/llvm-project/llvm-project/compiler-rt/lib/builtins -nodefaultlibs /__w/llvm-project/llvm-project/compiler-rt/test/builtins/Unit/aarch64/emupac.c -lm -Wl,--start-group /__w/llvm-project/llvm-project/build/compiler-rt/lib/linux/libclang_rt.builtins-aarch64.a -lc -Wl,--end-group -o /__w/llvm-project/llvm-project/build/compiler-rt/test/builtins/Unit/AARCH64LinuxConfig/aarch64/Output/emupac.c.tmp
# executed command: /usr/bin/clang-23 -gline-tables-only -Wthread-safety -Wthread-safety-reference -Wthread-safety-beta -DCOMPILER_RT_HAS_FLOAT16 -nobuiltininc -I/__w/llvm-project/llvm-project/compiler-rt/include -idirafter /usr/lib/llvm-23/lib/clang/23/include -resource-dir=/__w/llvm-project/llvm-project/build/compiler-rt -Wl,-rpath,/__w/llvm-project/llvm-project/build/compiler-rt/lib/linux -fno-builtin -I /__w/llvm-project/llvm-project/compiler-rt/lib/builtins -nodefaultlibs /__w/llvm-project/llvm-project/compiler-rt/test/builtins/Unit/aarch64/emupac.c -lm -Wl,--start-group /__w/llvm-project/llvm-project/build/compiler-rt/lib/linux/libclang_rt.builtins-aarch64.a -lc -Wl,--end-group -o /__w/llvm-project/llvm-project/build/compiler-rt/test/builtins/Unit/AARCH64LinuxConfig/aarch64/Output/emupac.c.tmp
# RUN: at line 3
/__w/llvm-project/llvm-project/build/compiler-rt/test/builtins/Unit/AARCH64LinuxConfig/aarch64/Output/emupac.c.tmp 1
# executed command: /__w/llvm-project/llvm-project/build/compiler-rt/test/builtins/Unit/AARCH64LinuxConfig/aarch64/Output/emupac.c.tmp 1
# RUN: at line 4
/__w/llvm-project/llvm-project/build/compiler-rt/test/builtins/Unit/AARCH64LinuxConfig/aarch64/Output/emupac.c.tmp 2
# executed command: /__w/llvm-project/llvm-project/build/compiler-rt/test/builtins/Unit/AARCH64LinuxConfig/aarch64/Output/emupac.c.tmp 2
# RUN: at line 5
not --crash /__w/llvm-project/llvm-project/build/compiler-rt/test/builtins/Unit/AARCH64LinuxConfig/aarch64/Output/emupac.c.tmp 3
# executed command: not --crash /__w/llvm-project/llvm-project/build/compiler-rt/test/builtins/Unit/AARCH64LinuxConfig/aarch64/Output/emupac.c.tmp 3
# .---command stderr------------
# | 'not': command not found
# `-----------------------------
```
Currently, the libc container doesn't contain `llvm-tools`. we need to
use them in #200196 because we are not able to use `llvm-lit` without
building clang from source
Since https://github.com/llvm/llvm-project/pull/209258, lldb uses
`lldb-server.exe` by default if libxml2 is available at build time. This
means that lldb-server requires installing libxml2 in the CI
environments to test it.
This patch adds a step to install libxml2 in the docker container used
for lldb pre-merge testing on Windows.
In #188049 we missed some test failures because GitHub CI does not have
libxml2 installed. I checked the logs and found a couple of other things
missing.
In this change I'm adding to the Linux container:
* libedit
* libxml2
* liblzma
I am not adding ncurses because this would enable text user interface
tests that are often unstable. They do run post-commit, and don't change
that often anyway.
I am not adding Lua or TreeSitter because they are more obscure and are
tested somewhere post-commit (in Green Dragon I think).
This allows triggering a workflow that benchmarks a given commit on both
macOS and Linux, and optionally submits the results to a LNT instance
(by default lnt.llvm.org).
Triggering this workflow requires write access to the LLVM monorepo, so
the inputs can be trusted to a large extent.
The intent is for this workflow to be triggered semi-automatically by a
combination of scripts and manual intervention. For example, we can
reuse a script like libcxx/utils/ci/lnt/commit-watch to automatically
trigger the workflow on new commits, while also requesting new commits
when bisecting regressions, etc. Since workflows can be dispatched via
the Github API, this should easily be scriptable.
A few important improvements are currently missing from this workflow
and left for future patches:
- The ability to select only a subset of configurations to run
- Additional configurations, such as unstable ABI, hardening, etc.
