This path transforms `vector.contract(A, B, Acc)` into
`vector.contract(A, B, 0) + Acc` to decouple the contraction computation
from the accumulator update for contractions over the loop.
Add a lane-level XeGPU operation that re-distributes a subgroup's
fragments across its lanes without changing the element type.
The op takes a 1D vector — the fragment held by one lane — and returns a
fragment of the same type. A pack/unpack mode selects the direction:
viewing the subgroup as an S x N element grid (S = subgroup size, N =
elements per lane), pack moves element j of lane i from logical position
j*S + i to i*N + j, so a lane's elements end up at consecutive
positions; unpack is the reverse.
This implements xegpu.convert_layout semantics at the lane level when
lane_layout is unchanged but lane_data differs.
assisted-by-claude
---------
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Share LLVM dialect metadata materialization through `ModuleTranslation`
so named metadata and metadata-as-value lowering use one conversion
path. Resolve metadata symbol references to functions, globals, aliases,
and ifuncs, and diagnose malformed required metadata before lowering.
Rename `MDFuncAttr` to `MDGlobalValueAttr` to reflect its support for
symbol-backed global values.
The im2col patterns for conv_2d_nhwc_hwcf, conv_2d_nchw_fchw and
conv_2d_nhwc_fhwc avoided non-unit dilations, but the restriction
doesn't seem to be fundamental: dilation only affects the gather step,
which can fold it into its indexing map. Generalize the convolved index
expression from `oh * stride + fh` to `oh * stride + fh * dilation` and
drop the match failures.
Verified by comparing the results of a dilated convolution lowered
directly to loops against the im2col decomposition with mlir-runner.
Assisted-By: Claude Code (for generating tests).
Example:
```fortran
!$acc parallel loop gang reduction(+:a)
do i = 1, N
!$acc loop worker reduction(+:a)
do j = 1, M
a(i) = a(i) + b(j,i)
end do
end do
```
In this code, the worker accumulator is block-shared, so every worker
updates
the same element with a plain read-modify-write and all but one partial
is lost.
Fix: make in-place updates of a block-shared array accumulator atomic,
unless
the element index varies per thread, in which case each thread owns its
element
and the plain update is already race-free.
---------
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Currently, only gpu.launch_func supports stream-based asynchronicity. This
patch adds the same mode to gpu.launch, and makes kernel outlining preserve
that when converting to gpu.launch_func.
This patch also documents the stream-based model in the operations description,
which was previously missing.
This patch changes the builders for the two launch operations:
To retain the current behavior for gpu.launch one needs to add a `nullptr` for
the new `Value asyncObject` argument after the `ValueRange asyncDependencies`,
and make sure to pass a valid type for the `Type asyncTokenType` parameter if
dependency-based async behavior is intended.
For gpu.launch_func, both stream-based and dependency-based async models now go
through the same builders, which take all three `Type asyncTokenType,
ValueRange asyncDependencies, Value asyncObject` arguments. To retain
stream-based behavior, pass `nullptr, {}` to the first two parameters, while to
retain dependency-based behavior, pass `nullptr` to the third parameter.
Assisted-by: Claude Code
`arith-expand` expanded `arith.maximumf`/`minimumf`/`maxnumf`/`minnumf`
and the signed/unsigned integer max/min ops into `cmpf`/`cmpi` +
`select` sequences. These ops also have a direct arith-to-llvm lowering
to the `llvm.intr.maximum`/`minimum`/... intrinsics, which are a single
hardware instruction on many targets. Pipelines that run arith-to-llvm
after arith-expand (e.g. the GPU-to-XeVM pipeline) therefore paid a
large, avoidable overhead.
Add `include-min-max-f` (the float ops
maximumf/minimumf/maxnumf/minnumf) and `include-min-max-i` (the
signed/unsigned integer ops maxsi/maxui/minsi/minui) option (default `false`, changing the existing
behavior) that controls whether these min/max ops are expanded. The
min/max converters are factored into new
`populateExpandMinMaxFPatterns` and
`populateExpandMinMaxIPatterns`; the ceil/floor-div and scaling ext/trunc
expansions (which have no LLVM lowering) always run.
---------
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
This PR fixes an issue in getDpasSubgroupLayouts(): It failed to find
valid subgroup layouts for otherwise-legal DPAS ops. It treated the K
(contraction) dimension as distributed across subgroups — requiring
wgShape % sgLayout == 0 on every dim and gating on a
checkAlignedSgDataAB() equality — when K is actually broadcast: its full
extent stays in every subgroup. This rejected the only valid candidate,
causing layout propagation to bail out.
Update the two affected tests, which now propagate successfully instead
of failing.
assisted-by-claude
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
This commit adds a version of the existing s_buffer_load intrinsic that
more accurately models the memory semantics of the s_buffer_load
instruction, namely that it is, in fact, a memory load.
To preserve the existing behavior that the "nomem" s.buffer.load
intrinsic was using, Clang and MLIR add !invariant.load metadata when
constructing the intrinsic (matching documented requirements on
scalarazable buffer loads) and a late codegen pass adds the metadata
just to be safe.
Tests that were "about" s.buffer.load have been copied to create
versions that use the new intrinsic, as was done for the other
*.ptr.buffer.* operations.
Other tests have been upgraded to use the new intrinsic. This has mainly
resulted in minor instruction ordering changes in prologues, if any
change at all. However, CodeGen/AMDGPU/dagcombine-fma-fmad.ll has seen a
v_fma => v_mad pattern fail to match in one case, I don't know if this
is a real regression.
While I was here, the s_buffer_load => buffer_load fallback has been
updated to preserve cachepolity.
AI disclosure: Primarily AI-written code, but I have at least looked at
and tried to find the worst of the silliness.
---------
Co-authored-by: Codex <codex@openai.com>
This PR enhance `setupInsertStridedSliceResultLayout` to treat a size-1
source dim as a broadcast dim: keep its lane_data at 1 instead of
requiring divisibility.
Add regression tests for both the layout propagation setup and the
subgroup-to-lane distribution of such an op.
Assisted-by-claude
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
This commit removes the "QueryProfileInterface" and
"QueryExtensionInterface" as they are superfluous in most cases, with
TosaComplianceData.h.inc already providing much of the data we need.
Removing these also helps prevent mistakes as observed in:
https://github.com/llvm/llvm-project/pull/206687
This commit removes these interfaces and updates the profile conformance
to extract supported profile/extension information from
TosaComplianceData.h.inc instead.
One factor that isn't duplicated is operations that rely on extensions,
but do not declare any data type support. This is a small set of
operations, such as tosa.custom, tosa.cond_if, tosa.while_loop and
tosa.const_shape, and we can add them to the TosaComplianceData.h.inc
file as needed.
The CancellingBlockScaledCastsOptimization optimises away sequences
where a block-scaled tensor is cast to fp32 and then cast back to its
original block-scaled type.
This patch extends it to also cover cases where the intermediate type is
bf16, since the 7 bits of mantissa and 8 bits in the exponent are
sufficient to encode any `element * scale` product with valid
block-scaled type combinations.
---------
Signed-off-by: Ian Tayler Lessa <ian.taylerlessa@arm.com>
This PR integrates constant bound inference into the existing
`AffineLoopNormalize` pass under a new `useExpensiveMath` option. When
`useExpensiveMath` is enabled (disabled by default due to potential
compilation time overhead), the pass leverages
`ValueBoundsConstraintSet` (uss presburger) analysis to refine dynamic
`affine.for` loop bounds into compile-time constant bounds. RFC:
https://discourse.llvm.org/t/rfc-mlir-enable-dynamic-and-tighter-affine-unrolling-via-valueboundsconstraintset/91055/2
---------
Co-authored-by: Oleksandr "Alex" Zinenko <azinenko@amd.com>
# Recategorize QuantileType as Storage Format, Not Quantization Scheme
## Summary
Reclassify `QuantileType` from inheriting `QuantizedType` to functioning
as a pure storage type.
## Motivation
### QuantileType is a Storage Format, Not a Quantization Scheme
After careful consideration, there was a conclusion reached that
**QuantileType should not be registered as a subtype of QuantizedType**.
The architectural distinction is critical:
- **QuantizedType** represents different quantization schemes (uniform,
per-axis, etc.), semantic operations that map values into a
lower-precision domain
- **QuantileType** is purely a **storage format** that specifies how to
interpret indices within a lookup table (LUT)
QuantileType was never meant to function as a quantization scheme on its
own. Instead, its purpose is to indicate **how to interpret indices
based on the given LUT**. It is an orthogonal concern to quantization
semantics; it describes data organization, not mathematical
transformation.
### Relaxed Type Conditions
The previous implementation restricted the LUT values to float types
only. However, float types are not the only types that can exist inside
a LUT, integer types are valid as well.
This PR removes the type restriction, allowing **both floating-point and
integer-based LUT interpretations** to coexist, better matching actual
use cases.
## Changes
- **Removed**: Incorrect inheritance of `QuantizedType` by
`QuantileType`
- **Relaxed**: Type constraints to accept both float and integer LUT
value types
This pass emits MLIR remarks for all OpenACC data mapping actions.
Intended to be used when all the data actions are committed (typically
right before codegen to call runtime).
The design of the ACC routine lowering supports multiple device side
specializations of an ACC routine function. Earlier in the pipeline, all
calls to the function are expected to refer the original host function
and not their device side specialized versions. Once there is enough
parallelism context, the calls can be changed to point to the
appropriate specialized routine.
The nohost handling in `ACCRoutineLowering` goes against this design.
This change removes the special handling of nohost routines and this
clean up should be done in later passes.
Assisted-by: Codex
- Added FP8 and all six MXFP extension/type combinations
- Updated RESIZE availability and enforced NEAREST_NEIGHBOR for MXFP
- Added positive, negative, version, extension, mixed-type, and verifier
tests.
Signed-off-by: Peng Sun <peng.sun@arm.com>
Example:
```fortran
!$acc parallel loop vector reduction(+:b)
do i = 1, n
b(i) = i
end do
```
In this code the array reduction is thread(vector)-only with no
gang/block launch dim, so a within-block `all_reduce` is a complete
reduction. ACCCGToGPU still rejected it as NYI because the accumulate
had no block context.
Fix: keep the NYI only when the region actually launches block
dimensions; allow the existing per-element `all_reduce` path for
single-block thread-only array accumulates.
per
[#210738](https://github.com/llvm/llvm-project/pull/210738#discussion_r3622744153)
Currently, the following crashes on verification with a segfault:
```
func.func @broadcast_rank0_tensor(%arg0: tensor<i32>, %arg1: tensor<32x2xi32>) -> tensor<32x2xi32> {
%0 = linalg.broadcast ins(%arg0 : tensor<i32>) outs(%arg1 : tensor<32x2xi32>) dimensions = [0, 0]
return %0 : tensor<32x2xi32>
}
```
The verifier should reject cases like this where we have a duplicate
dimension. I've also added a couple of tests covering `linalg.broadcast`
on various input scenarios to confirm they are being handled properly.
Add value-preserving folds mirroring LLVM's InstructionSimplify for the
integer shift ops:
```
shli/shrui/shrsi(0, x) -> 0
shrui/shrsi(x, x) -> 0
shrsi(-1, x) -> -1
```
A shift amount greater than or equal to the bit width yields poison, so
refining these cases to a constant is valid. For `shr(x, x)` any
in-range amount `v` satisfies `v >> v == 0 (v < 2^v)`.
Folding an out-of-range shift amount (`c >= bit width`) to poison is
left as a TODO: it would make the arith dialect depend on the ub dialect
to materialize `ub.poison`.
LLVM performs the same simplifications: https://godbolt.org/z/7hsMz9E7Y
Verified with Alive2:
| Fold | shl | lshr | ashr |
| :--- | :---: | :---: | :---: |
| `0 << x`, `0 >> x -> 0` | [✓](https://alive2.llvm.org/ce/z/Sz2fT1) |
[✓](https://alive2.llvm.org/ce/z/mlq7k6) |
[✓](https://alive2.llvm.org/ce/z/jbeNH9) |
| `x >> x -> 0` | — | [✓](https://alive2.llvm.org/ce/z/JCPBCZ) |
[✓](https://alive2.llvm.org/ce/z/YLjo3M) |
| `-1 >> x -> -1` | — | — | [✓](https://alive2.llvm.org/ce/z/sT6pNC) |
---
Code partially generated by Claude Code.
Signed-off-by: Víctor Pérez Carrasco <victor.pc.upm@gmail.com>
`Builder::getZeroAttr` asserts when asked to build a constant of a
ranked tensor with a dynamic shape (`DenseElementsAttr` requires a
static shape). Three self-identity folds/patterns reached it without a
guard and aborted on dynamic-shaped operands:
```mlir
arith.xori %x, %x : tensor<?xi32>
arith.subui_extended %x, %x : tensor<?xi32>, tensor<?xi1>
arith.subi (arith.subi %a, %b), %a : tensor<?xi32> (SubISubILHSRHSLHS)
```
Guard each with a static-shape check (as `arith.subi(x,x)` already does)
so they leave the op unfolded instead of crashing. The TableGen pattern
gets a new `IsScalarOrHasStaticShape` constraint.
Signed-off-by: Víctor Pérez Carrasco <victor.pc.upm@gmail.com>
Add value-preserving folds mirroring LLVM's InstructionSimplify for the
integer division and remainder ops:
```
divui/divsi/ceildivui/ceildivsi/floordivsi(0, x) -> 0
divui/divsi/ceildivui/ceildivsi/floordivsi(x, x) -> 1
remui/remsi(0, x) -> 0
remui/remsi(x, x) -> 0
```
The self and zero-dividend cases are valid refinements because division
or remainder by zero is undefined behaviour; no overflow flags are
required. The folds return a scalar or splat constant and bail out on
shaped types with a dynamic shape.
Folding `x / 0` and `x % 0` to poison is left as a TODO: it would make
the arith dialect depend on the ub dialect to materialize `ub.poison`.
LLVM performs the same simplifications: https://godbolt.org/z/dzTjYjs4e
Verified with Alive2:
| Fold | udiv | sdiv | urem | srem |
| :--- | :---: | :---: | :---: | :---: |
| `x / x -> 1`, `x % x -> 0` | [✓](https://alive2.llvm.org/ce/z/YjZCEA)
| [✓](https://alive2.llvm.org/ce/z/BSWQrm) |
[✓](https://alive2.llvm.org/ce/z/gx5sRH) |
[✓](https://alive2.llvm.org/ce/z/VUYkr4) |
| `0 / x -> 0`, `0 % x -> 0` | [✓](https://alive2.llvm.org/ce/z/keqeCZ)
| [✓](https://alive2.llvm.org/ce/z/jyf5wm) |
[✓](https://alive2.llvm.org/ce/z/fZa6ND) |
[✓](https://alive2.llvm.org/ce/z/fo6ss8) |
----
Code generated by Claude Code.
Signed-off-by: Víctor Pérez Carrasco <victor.pc.upm@gmail.com>
Adds two missing ops to the WasmSSA dialect.
- `wasmssa.global_set`
- `wasmssa.select`
---------
Co-authored-by: Ferdinand Lemaire <flscminecraft@gmail.com>
`tosa.mul` and `tosa.intdiv` fold constant splat operands by building a
`DenseElementsAttr` of the result type, which asserts when that type has
a dynamic shape. When the operands are constants but the result type is
dynamic, folding aborted. Guard `mulBinaryFolder` and the `IntDivOp`
splat path on a static result shape, matching the other fold paths in
these ops.
Signed-off-by: Víctor Pérez Carrasco <victor.pc.upm@gmail.com>
Make the `input_layout` attribute of `xegpu.convert_layout` optional.
When it is not set, the effective input layout defaults to
`target_layout`, i.e. the source is assumed to already be in the target
distribution.
assisted-by-Claude
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
The diagnostic says the transform is not performed as the source may be
a scalar. But we can make it if we can check the source is not a scalar.
AI assisted.
Relax the create_mem_desc source operand constraint to accept a
statically shaped shared-memory memref of any rank, replacing the
1D/2D-only StaticShared{1,2}DMemRefOf classes with a rank-agnostic
StaticSharedMemRefOf.
Add a verifier requiring the source memref to be contiguous row-major,
update the op documentation, and add valid/invalid lit tests.
assisted-by-claude
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
## [flang][OpenMP] Implement the `error` directive
### Summary
Implements the OpenMP `error` directive (OpenMP 5.1, 2.5.4) in Flang.
The directive prints a message and, with `severity(fatal)`, stops.
Handling depends on the `at` clause:
- **`at(compilation)`** (the default): handled in semantics. Emits a
compile-time warning or error and produces no IR.
- **`at(execution)`**: lowered to a new `omp.error` op, which becomes a
call to `__kmpc_error`.
Before this patch the directive was unimplemented and hit a `not yet
implemented: OmpErrorDirective` error during lowering.
Fixes#204240.
### Details
Changes go through the compiler pipeline.
**Flang semantics**
- `Enter(OmpErrorDirective)` emits the compile-time diagnostic for the
`at(compilation)` form, using the directive's severity and the `MESSAGE`
text when present.
- `Enter(OmpClause::Message)` checks that the `MESSAGE` expression is a
scalar, default-kind `CHARACTER`, which is required to pass it to the
runtime as a `const char*`.
**Flang lowering**
- `genErrorDirective` lowers the `at(execution)` form only.
- A constant message is stored in the `message` attribute. A
non-constant message is copied to a null-terminated buffer
(`genNullTerminatedString`) and passed as the `message_expr` operand,
because `__kmpc_error` expects a NUL-terminated string.
**New `omp.error` op (MLIR OpenMP dialect)**
- Adds a `ClauseSeverity` enum (`warning = 1`, `fatal = 2`, matching the
`__kmpc_error` argument) and `ClauseSeverityAttr`.
- `omp.error` takes a required `severity` attribute and an optional
message, given either as:
- `message` — a constant string attribute, or
- `message_expr` — a pointer to a null-terminated string.
- A verifier rejects providing both `message` and `message_expr`.
**MLIR → LLVM IR translation**
- Translates `omp.error` to `__kmpc_error(ident_t *, i32 severity, ptr
message)`.
**OMPIRBuilder**
- Adds `OpenMPIRBuilder::createError(...)`, which builds the call, maps
`fatal`→`2` and `warning`→`1`, and passes a null pointer when there is
no message.
### Tests
- **Parser** (`error-unparse.f90`): unparse and parse-tree output.
`at(compilation) severity(fatal)` cases that would stop compilation were
moved to `at(execution)`.
- **Semantics** (`Semantics/OpenMP/error.f90`): compilation
warning/error/default-severity cases, plus the new `MESSAGE` type, rank,
and kind errors.
- **Lowering** (`Lower/OpenMP/error.f90`): constant message, no message,
empty message, runtime `message_expr`, and `at(compilation)` cases that
must not produce `omp.error`. The old `Lower/OpenMP/Todo/error.f90` test
is removed.
- **MLIR** (`ops.mlir`, `invalid.mlir`): round-trip of each `omp.error`
form and the both-messages verifier error.
- **LLVM IR** (`openmp-llvm.mlir`): `__kmpc_error` calls for warning and
fatal, constant and runtime messages, and the null-message cases.
Assisted-by: Github Copilot
Fixes#206628 crash by adding an earlier body argument verifier for
`affine.for`. This crash is caused because `LoopLikeOpInterface`
verifier would call `getRegionIterArgs()` and assumed induction var of
`affine.for` exists.
Added the `class-name-format` option that takes a format string used to
generate the wrapper class name.
This provides greater control over the generated `ClassOp` name. When
multiple identically structured IR files need to be programmatically
lowered with this pass and linked, there will be naming conflicts which
this patch provides a method of resolution for.
During compute lowering, also set the collapse count attribute when
creating scf.parallel ops from acc.loop ops. While this collapse count
is implied by the number of IVs, later canonicalizations may use this
information to decide whether or not to collapse the loop nest further
Example:
```fortran
!$acc parallel loop collapse(2) num_gangs(10) reduction(+:a)
do i = 1, n
do j = 1, n
do k = 1, m
a(k) = a(k) + x(k,i) * y(k,j)
end do
end do
end do
```
In this code, ACCCG pads ThreadX to the subgroup size because
`gpu.all_reduce` uses subgroup shuffles. This unnecessarily increases
private reduction storage when only one Y/Z row is active.
Fix: preserve partial ThreadX launches for a single row, while retaining
alignment when Y/Z rows could share a subgroup.
`ACCLoopTiling` used to lower an N-dimensional `tile()` clause by
*uncollapsing* the fused loop into a deep nest of single-IV `acc.loop`s
(`tile_1 → … → tile_N → elem_1 → … → elem_N`). This discards the natural
grouping of the tile clause: the 2N single-IV loops no longer express
"these are the tile iterations" and "these are the in-tile iterations"
as collapsible units, and gang/vector end up spread across a deep nest
where only the outermost loop of each group carries the attribute. This
could potentially lead to poor parallelism assignment.
With this MR:
`tileACCLoops` now rewrites the single fused `acc.loop` in-place into
exactly two multi-IV loops, each carrying all the tiled induction
variables:
- a **tile-group** loop whose steps are the original steps scaled by the
tile sizes, and
- a nested **element-group** loop that walks the iterations within one
tile, with upper bounds clamped to `min(origUB, tileStart +
tileExtent)`.
Keeping each group as a single multi-IV `acc.loop` preserves the
tile/element structure as two collapsible units instead of a 2N-deep
single-IV nest, which is a simpler canonical form. The
gang/worker/vector distribution across the two groups is unchanged from
before.
This change also diagnoses `tile` + `collapse` on the same loop because
these clauses give conflicting loop-association counts. The pass now
emits a NYI diagnostic instead of silently dropping the `collapse`
clause. Added a new invalid-input test to verify this.
With the pass no longer uncollapsing, `uncollapseLoops` and the
vector-of-loops `tileACCLoops(SmallVector<LoopOp>&, …)` overload (and
the now-orphaned `createInnerLoop` helper) are deleted, along with their
unit tests. New unit tests are added to cover the surviving fused-loop
overload.
Sequential `gpu_block_redundant` ancestors need selective launch-dim
handling in `getAncestorParDims`: including them always widens
gang-private to per-thread; omitting them always can predicate
block-redundant bodies on `blockIdx` after partition.
Include launch dims when the block-redundant loop is worksharing or is
the innermost parallel parent; skip them for an outer sequential
block-redundant wrapper around nested worksharing.
Extends `vector.compressstore` + `vector.expandload` to support scalable
vectors and updates relevant tests.
An e2e test for `vector.compressstore` is added. For
`vector.expandload`, we need to wait for QEMU support:
https://github.com/llvm/llvm-project/issues/210942.
`SparseVectorization` assumes direct loop accesses (`a[lo:hi]`) are
contiguous and vectorizes them with `vector.maskedload/maskedstore`.
This is false for `sparse_tensor.coordinates` of a level inside a
trailing AoS COO region, whose buffer is interleaved with other levels:
a silent miscompile.
The true stride is already known from the tensor's encoding, even though
the memref type is still dynamic at this point. Use it to fall back to a
scalar loop when the stride is provably non-unit.
---------
Signed-off-by: Federico Bruzzone <federico.bruzzone.i@gmail.com>
Update both:
* CastAwayExtractStridedSliceLeadingOneDim
* CastAwayInsertStridedSliceLeadingOneDim
to use vector.shape_cast, rather than vector.extract and
vector.broadcast, as the canonical form for stripping unit dimensions.
This change was originally implemented by @krzysz00 in #196206, but was
subsequently reverted in #199546. This PR intentionally restores only a
subset of #196206, making it easier to identify and triage any potential
regressions.
Co-authored-by: Krzysztof Drewniak <Krzysztof.Drewniak@amd.com>
This commit refactors the defined TOSA types to combine unranked and
ranked tensor types into a single type `TosaTensorOf`. This helps
simplify the type definitions and allows all tensor types to support
both unranked and ranked tensors.
Example:
```fortran
integer :: r(:,:)
!$acc parallel loop reduction(+:r)
```
In this code, the reduction accumulator has dynamic dimensions and rank
greater than one.
Fix: use runtime dimension sizes to delinearize flattened
array-reduction indices.
Example:
```fortran
!$acc parallel loop gang worker reduction(+:sum)
do j = 1, n
!$acc loop vector reduction(+:sum)
do i = 1, n
sum = sum + a(i, j)
end do
end do
```
In this code, each worker has a private partial result, but combine
predication previously allowed only ThreadY row zero to contribute.
Fix: keep ThreadY active only for proven atomic worker combines, while
preserving legacy predication or reporting NYI for unsafe combinations.
Fold math.fpowi using the base's own floating-point semantics via
iterative
square-and-multiply, matching the multiply sequence ExpandPowI builds in
SelectionDAGBuilder. The prior pow/powf-based fold computed in
float/double
and rounded differently from the runtime expansion, disagreeing with
x**n by ~1 ULP.
Example:
```fortran
subroutine reduce(a, x, n)
integer :: n, i
real :: a(:), x(:)
!$acc parallel loop reduction(+:a)
do i = 1, n
a(:) = a(:) + x(i)
end do
end subroutine
```
`ACCCGToGPU` represents each gang/thread-private array as a dynamically
offset `memref.subview`. Converting this subview directly to a FIR
pointer-like type loses the memref offset, causing different private
copies to alias the same storage.
Fix: extract the selected subview’s element offset, convert it to bytes,
and create a zero-offset view at that address before converting to the
FIR pointer-like type. This preserves the selected gang/thread-private
slice.
Before:
```mlir
%subview = memref.subview %view[%block_id, 0] [1, %extent] [1, 1]
: memref<?x?xf64> to memref<?xf64, strided<[1], offset: ?>>
%result = fir.convert %subview
: (memref<?xf64, strided<[1], offset: ?>>) ->
!fir.heap<!fir.array<?xf64>>
```
After:
```mlir
%subview = memref.subview %view[%block_id, 0] [1, %extent] [1, 1]
: memref<?x?xf64> to memref<?xf64, strided<[1], offset: ?>>
%base, %offset, %sizes, %strides =
memref.extract_strided_metadata %subview
: memref<?xf64, strided<[1], offset: ?>> ->
memref<f64>, index, index, index
%c8 = arith.constant 8 : index
%byte_offset = arith.muli %offset, %c8 : index
%private_view = memref.view %buffer[%byte_offset][%extent]
: memref<?xi8> to memref<?xf64>
%result = fir.convert %private_view
: (memref<?xf64>) -> !fir.heap<!fir.array<?xf64>>
```
