Manual-sync mirror of chromium.googlesource.com/external/github.com/llvm/llvm-project (bare monorepo, no subtree suffix)
Find a file
Robert Imschweiler e9acb01904
[OpenMP][offload] Cross-team reductions with variable number of teams (#195102)
This is a part of a series of patches that rework OpenMP cross-team
reductions.

This patch changes the cross-team reduction runtime to no longer work
through larger number of teams in chunks. Instead, we allocate a
suitable-sized global buffer for the team values and let all teams run
at once. The last team that finishes uses a strided loop to reduce the
team values from the global buffer.

We also use `mapping::getNumberOfThreadsInBlock()` instead of
`omp_get_num_threads()` because the reduction of the team values runs
outside of the parallel region device code, which would make
`omp_get_num_threads()` always return 1. For Generic-SPMD mode, we also
want to use all available threads, which means that we need to copy the
reduction data from LDS (where it lives in that mode by default) to
scratch in codegen before calling the cross-team reduction.

Performance:
All performance tests can be reproduced with
https://github.com/ro-i/xteam-test @ commit
2f09ae1f566f13189d20b414d8b83bf2986564d2. To reproduce, simply create a
`local.mk` file in the cloned directory with a suitable `OFFLOAD_ARCH`
for your machine and `CXX_trunk` + `CXX_trunk_dev` set to the paths of
the clang++ binaries for llvm/main and this patch. (llvm/main should
best be at the commit that is currently the base for this PR. At the
moment, this is fe87c971bf). Then, run
`make trunk trunk_dev` to build the benchmark binaries for 208 and 10400
teams. Run them with `./run_bench.sh -rq -n10 red_trunk_208
red_trunk_dev_208 red_trunk_10400 red_trunk_dev_10400` to get the avg
performance numbers over 10 rounds.
This tests multiple reduction workloads, including reductions that run
in the Generic-SPMD mode, with 208 teams and with 10400 teams, both à
512 threads, and with a reduction array size of 177,777,777. I tested on
a gfx942 and didn't find any regressions - here are some numbers showing
the performance of this patch relative to the baseline:
```
red_comb_sep_arr_32    double   dev_208:   +84.37%   dev_10400:   +54.06%
red_sum_arr_32         double   dev_208:    +9.72%   dev_10400:   +51.29%
red_comb               double   dev_208:    +2.73%   dev_10400:    +8.73%
red_comb_sep           double   dev_208:    +1.89%   dev_10400:   +87.13%
red_dot                double   dev_208:    +1.46%   dev_10400:   +25.66%
red_indirect           double   dev_208:    +1.51%   dev_10400:   +23.54%
red_kernel_part        double   dev_208:    +1.27%   dev_10400:   +88.28%
red_max                double   dev_208:    +1.29%   dev_10400:   +22.69%
red_mult               double   dev_208:    +1.96%   dev_10400:   +22.02%
red_sum                double   dev_208:    +2.03%   dev_10400:   +22.00%
red_pi                 double   dev_208:    +1.08%   dev_10400:    +1.22%
red_comb_sep_arr_32    uint     dev_208:  +139.10%   dev_10400:   +88.02%
red_sum_arr_32         uint     dev_208:   +49.70%   dev_10400:  +143.24%
red_dot                uint     dev_208:    +0.77%   dev_10400:   +24.91%
red_max                uint     dev_208:    +3.43%   dev_10400:   +36.61%
red_sum                uint     dev_208:    +4.56%   dev_10400:   +36.95%
red_comb_sep_arr_32    ulong    dev_208:   +85.22%   dev_10400:   +58.17%
red_sum_arr_32         ulong    dev_208:    +9.01%   dev_10400:   +55.14%
red_dot                ulong    dev_208:    +2.94%   dev_10400:   +26.89%
red_max                ulong    dev_208:    +2.83%   dev_10400:   +23.10%
red_sum                ulong    dev_208:    +1.59%   dev_10400:   +22.82%
red_comb_sep_arr_32    Value    dev_208:   +29.88%   dev_10400:   +24.17%
red_sum_arr_32         Value    dev_208:    +7.26%   dev_10400:   +38.14%
red_dot                Value    dev_208:    +2.65%   dev_10400:    +8.60%
red_max                Value    dev_208:    +2.86%   dev_10400:    +4.77%
red_sum                Value    dev_208:    +0.47%   dev_10400:   +10.22%
```

This patch uses ideas from Johannes Doerfert, ideas from the AOMP
cross-team reduction implementation, and was assisted by Claude.
2026-06-20 07:12:30 +02:00
.ci [CI][Offload] Fix offload depends on openmp (#202541) 2026-06-09 09:45:56 +00:00
.github workflows/subscriber: Update to latest github automation container (#204692) 2026-06-19 14:28:44 -07:00
bolt [BOLT][AArch64] Align tentative layout bases using per-section alignment (#204262) 2026-06-19 19:35:43 -07:00
clang [OpenMP][offload] Cross-team reductions with variable number of teams (#195102) 2026-06-20 07:12:30 +02:00
clang-tools-extra cp -r replaces symlinks, update to cp -R (#201463) 2026-06-18 16:09:10 -07:00
cmake [docs] try again to handle doxygen everywhere (#203081) 2026-06-11 08:58:36 -04:00
compiler-rt Revert "Revert "[Compiler-rt][test] Fix circular link dependency between builtins and libc"" (#204728) 2026-06-19 15:50:21 +05:30
cross-project-tests [RISCV][P-ext] Support Packed Absolute Value and Absolute Difference (#203840) 2026-06-18 22:32:00 +08:00
flang [Flang] Standardize coarray TODO() diagnostic messages (#204708) 2026-06-19 16:27:10 -07:00
flang-rt [Flang] [Runtime ]Fix write endfile abort (#191633) 2026-06-15 10:27:29 -05:00
libc Revert "[libc] Implement basename and dirname in libgen.h (#204554)" (#204856) 2026-06-19 17:39:37 +01:00
libclc [libclc] Use unchecked division for f64 AMDGPU reciprocal (#203809) 2026-06-15 07:26:32 -05:00
libcxx [libc++][byte] Apply [[nodiscard]] to std::byte (#204674) 2026-06-19 16:04:21 -04:00
libcxxabi [Demangle] Guard DEMANGLE_ABI and add missing annotation (#202920) 2026-06-11 02:48:04 -07:00
libsycl [libsycl] Fix _LIBSYCL_EXPORT placement (#201364) 2026-06-09 16:12:58 +02:00
libunwind [libunwind] Add SME detection for Apple platforms (#193630) 2026-05-05 02:25:33 +00:00
lld [ELF][AArch64] Relax zero TLSLE add to nop (#204286) 2026-06-19 09:10:34 +01:00
lldb [lldb] Survive ptrace(PT_DENY_ATTACH) when attaching (#204688) 2026-06-19 10:51:02 -07:00
llvm [OpenMP][offload] Cross-team reductions with variable number of teams (#195102) 2026-06-20 07:12:30 +02:00
llvm-libgcc [Runtimes] Introduce variables containing resource dir paths (#177953) 2026-04-02 10:32:14 +00:00
mlir [OpenMP][offload] Cross-team reductions with variable number of teams (#195102) 2026-06-20 07:12:30 +02:00
offload [OpenMP][offload] Cross-team reductions with variable number of teams (#195102) 2026-06-20 07:12:30 +02:00
openmp [OpenMP][offload] Cross-team reductions with variable number of teams (#195102) 2026-06-20 07:12:30 +02:00
orc-rt [orc-rt] Fix assertion, add fail-to-connect unit test. (#204720) 2026-06-19 13:11:59 +10:00
polly Revert "[SimpleLoopUnswitch] Generalize the notion of trivial unswitching" (#204538) 2026-06-18 13:23:48 +03:00
runtimes [Flang] Add mock flang driver (#203481) 2026-06-17 16:13:40 +02:00
third-party Fix a warning and an uninitialized variable in benchmark (#200927) 2026-06-02 13:49:06 -07:00
utils/bazel [Bazel] Fixes 95e3219 (#204873) 2026-06-19 15:56:54 -07:00
.clang-format Enforce Unix line endings for Clang/LLVM/MLIR projects (#161460) 2025-10-02 06:09:12 -07:00
.clang-format-ignore [Docs] Update coding standard for TD files (#200848) 2026-06-01 21:35:58 +01:00
.clang-tidy Add default empty header filter regex to root .clang-tidy (#167386) 2025-11-11 01:36:25 +03:00
.git-blame-ignore-revs Add "Split clang/lib/CodeGen/CGBuiltin.cpp" to .git-blame-ignore-revs (#203419) 2026-06-12 12:03:28 +01:00
.gitattributes [libclang/python] Enable packaging clang python bindings (#125806) 2025-11-24 17:21:24 -08:00
.gitignore Revert "[LifetimeSafety] Support C Language in LifetimeSafety" (#204481) 2026-06-18 00:31:31 +02:00
.mailmap [mailmap] Add mailmap entry for myself (#189447) 2026-03-30 18:35:34 +00:00
CODE_OF_CONDUCT.md
CONTRIBUTING.md
LICENSE.TXT
pyproject.toml List psutil as a dependency in pyproject.toml 2026-05-04 13:10:21 -07:00
README.md
SECURITY.md [SECURITY.md] add pointers to "what is a security issue" (#177372) 2026-01-22 17:19:09 -07:00

The LLVM Compiler Infrastructure

OpenSSF Scorecard OpenSSF Best Practices libc++

Welcome to the LLVM project!

This repository contains the source code for LLVM, a toolkit for the construction of highly optimized compilers, optimizers, and run-time environments.

The LLVM project has multiple components. The core of the project is itself called "LLVM". This contains all of the tools, libraries, and header files needed to process intermediate representations and convert them into object files. Tools include an assembler, disassembler, bitcode analyzer, and bitcode optimizer.

C-like languages use the Clang frontend. This component compiles C, C++, Objective-C, and Objective-C++ code into LLVM bitcode -- and from there into object files, using LLVM.

Other components include: the libc++ C++ standard library, the LLD linker, and more.

Getting the Source Code and Building LLVM

Consult the Getting Started with LLVM page for information on building and running LLVM.

For information on how to contribute to the LLVM project, please take a look at the Contributing to LLVM guide.

Getting in touch

Join the LLVM Discourse forums, Discord chat, LLVM Office Hours or Regular sync-ups.

The LLVM project has adopted a code of conduct for participants to all modes of communication within the project.