[flang][OpenMP] Fix alias analysis for omp.private copy region block arguments to help inline hlfir.assign to improve time taken in LTO. (#209539)

## Summary

This PR fixes https://github.com/llvm/llvm-project/issues/200922 - a
compile-time regression affecting firstprivate clauses on simple arrays
in OpenMP target regions.

When `InlineHLFIRAssign` checks whether to inline `hlfir.assign`
operations in `omp.private` copy regions, the alias analysis
conservatively returns `MayAlias` for the copy region block arguments
(`%arg0` mold vs `%arg1` private). This prevents inlining and forces a
fallback to `fir.call @_FortranAAssign`, which:

- Pulls in 89 runtime functions via LTO
- Creates 550K+ abstract attributes in OpenMPOpt
- Causes severe compile-time degradation (60%+ increase in some cases)

**Example:**
```fortran
!$omp target firstprivate(array)
  ! ... use array ...
!$omp end target
```

The firstprivate copy generates an `omp.private` copy region where the
assignment from the original array to the private copy cannot be inlined
due to conservative aliasing assumptions.

## Root Cause and Solution

The FIR alias analysis did not recognize that `omp.private` copy region
block arguments are guaranteed by the OpenMP specification to reference
different memory locations:
- `%arg0` (mold) - the original variable
- `%arg1` (private) - freshly allocated private storage created in the
init region

Since the private copy is allocated fresh, these arguments **cannot
alias**.

So, this PR adds special-case handling in
`flang/lib/Optimizer/Analysis/AliasAnalysis.cpp` to recognize
`omp.private` copy region block arguments and correctly return
`NoAlias`. This allows `InlineHLFIRAssign` to inline the assignment into
an element-wise loop, avoiding the expensive runtime call entirely.

Assisted by: Claude Sonnet

---------

Co-authored-by: Claude Sonnet 4 <noreply@anthropic.com>
This commit is contained in:
Pranav Bhandarkar 2026-07-24 13:56:30 -05:00 committed by GitHub
parent ba61615b65
commit e949b65442
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
4 changed files with 199 additions and 50 deletions

View file

@ -662,6 +662,71 @@ AliasResult AliasAnalysis::alias(Source lhsSrc, Source rhsSrc, mlir::Value lhs,
if (lhs == rhs || getZeroOffsetViewRoot(lhs) == getZeroOffsetViewRoot(rhs))
return AliasResult::MustAlias;
// OpenMP private clause copy region arguments are guaranteed to be disjoint.
// The copy region has two arguments: %arg0 (original/mold) and %arg1
// (private). By omp.private semantics, %arg1 is initialized with freshly
// allocated memory (from the init region), while %arg0 references the
// original variable. These cannot alias. This check handles both direct block
// argument comparisons and cases where one value is loaded from a block
// argument (common pattern: hlfir.assign %loaded to %arg1 where %loaded =
// fir.load %arg0).
auto getBlockArgOrLoadSource = [](mlir::Value v) -> mlir::BlockArgument {
if (auto arg = mlir::dyn_cast<BlockArgument>(v))
return arg;
// Check if this is a load from a block argument
if (auto defOp = v.getDefiningOp()) {
if (auto loadOp = mlir::dyn_cast<fir::LoadOp>(defOp)) {
return mlir::dyn_cast<BlockArgument>(loadOp.getMemref());
}
}
return {};
};
auto lhsArg = getBlockArgOrLoadSource(lhs);
auto rhsArg = getBlockArgOrLoadSource(rhs);
if (lhsArg && rhsArg &&
lhsArg.getParentRegion() == rhsArg.getParentRegion()) {
if (auto *parentOp = lhsArg.getParentRegion()->getParentOp()) {
if (auto privateOp = mlir::dyn_cast<omp::PrivateClauseOp>(parentOp)) {
auto &copyRegion = privateOp.getCopyRegion();
if (lhsArg.getParentRegion() == &copyRegion) {
// Both trace to block args from the copy region - check if they're
// the defined pair
auto moldArg = privateOp.getCopyMoldArg();
auto privArg = privateOp.getCopyPrivateArg();
if ((lhsArg == moldArg && rhsArg == privArg) ||
(lhsArg == privArg && rhsArg == moldArg)) {
// Check if this is a POINTER type.
// For POINTER + FIRSTPRIVATE: the descriptor is copied (private)
// but pointer association is preserved, so the DATA they point to
// is the same → ALIASING.
// For ALLOCATABLE: fresh memory is allocated → NO ALIASING.
auto isPointerType = [](mlir::Type ty) -> bool {
ty = fir::unwrapRefType(ty);
if (auto boxTy = mlir::dyn_cast<fir::BaseBoxType>(ty))
return boxTy.isPointer();
return false;
};
if (isPointerType(lhsArg.getType()) ||
isPointerType(rhsArg.getType())) {
LLVM_DEBUG(llvm::dbgs()
<< " may alias: omp.private POINTER preserves "
<< "association (data aliases)\n");
return AliasResult::MayAlias;
}
LLVM_DEBUG(llvm::dbgs()
<< " no alias: omp.private copy region arguments "
<< "(mold vs private)\n");
return AliasResult::NoAlias;
}
}
}
}
}
bool approximateSource = lhsSrc.approximateSource || rhsSrc.approximateSource;
LLVM_DEBUG(llvm::dbgs() << "\nAliasAnalysis::alias\n";
llvm::dbgs() << " lhs: " << lhs << "\n";

View file

@ -0,0 +1,50 @@
// Test that InlineHLFIRAssign successfully inlines hlfir.assign operations
// in omp.private copy regions when the fix for omp.private block argument
// aliasing is present. Without the fix, alias analysis returns MayAlias and
// InlineHLFIRAssign fails.
//
// RUN: fir-opt %s --inline-hlfir-assign --split-input-file | FileCheck %s
// Fortran source (simplified):
// program minimal
// integer :: arr(8)
// arr(1) = 1
// !$omp target firstprivate(arr)
// arr(1) = 42
// !$omp end target
// end program
omp.private {type = firstprivate} @arr_privatizer : !fir.box<!fir.array<8xi32>> init {
^bb0(%arg0: !fir.ref<!fir.box<!fir.array<8xi32>>>, %arg1: !fir.ref<!fir.box<!fir.array<8xi32>>>):
%c8 = arith.constant 8 : index
%c1 = arith.constant 1 : index
%0 = fir.shape %c8 : (index) -> !fir.shape<1>
%1 = fir.allocmem !fir.array<8xi32> {bindc_name = ".tmp", uniq_name = ""}
%2:2 = hlfir.declare %1(%0) {uniq_name = ".tmp"} : (!fir.heap<!fir.array<8xi32>>, !fir.shape<1>) -> (!fir.heap<!fir.array<8xi32>>, !fir.heap<!fir.array<8xi32>>)
%3 = fir.shape_shift %c1, %c8 : (index, index) -> !fir.shapeshift<1>
%4 = fir.embox %2#0(%3) : (!fir.heap<!fir.array<8xi32>>, !fir.shapeshift<1>) -> !fir.box<!fir.array<8xi32>>
fir.store %4 to %arg1 : !fir.ref<!fir.box<!fir.array<8xi32>>>
omp.yield(%arg1 : !fir.ref<!fir.box<!fir.array<8xi32>>>)
} copy {
^bb0(%arg0: !fir.ref<!fir.box<!fir.array<8xi32>>>, %arg1: !fir.ref<!fir.box<!fir.array<8xi32>>>):
// This hlfir.assign should be inlined into a loop by InlineHLFIRAssign
// because alias analysis should return NoAlias for %0 (loaded from %arg0 mold)
// vs %arg1 (private copy).
%0 = fir.load %arg0 : !fir.ref<!fir.box<!fir.array<8xi32>>>
hlfir.assign %0 to %arg1 : !fir.box<!fir.array<8xi32>>, !fir.ref<!fir.box<!fir.array<8xi32>>>
omp.yield(%arg1 : !fir.ref<!fir.box<!fir.array<8xi32>>>)
}
// CHECK-LABEL: omp.private {type = firstprivate} @arr_privatizer
// CHECK: } copy {
// CHECK: ^bb0(%[[MOLD:.*]]: !fir.ref<!fir.box<!fir.array<8xi32>>>, %[[PRIV:.*]]: !fir.ref<!fir.box<!fir.array<8xi32>>>):
// CHECK-NOT: hlfir.assign {{.*}} to {{.*}} : !fir.box<!fir.array<8xi32>>
// CHECK: fir.do_loop
// CHECK: hlfir.designate
// CHECK: hlfir.designate
// CHECK: hlfir.assign {{.*}} to {{.*}} : i32
// CHECK: omp.yield
func.func @test_omp_private_copy_inlined() {
return
}

View file

@ -0,0 +1,34 @@
// Test alias analysis for POINTER + FIRSTPRIVATE in omp.private copy region.
// For POINTER variables, the descriptor is copied but pointer association
// is preserved. Both the mold descriptor and private descriptor point to
// the SAME target data, therefore they ALIAS.
//
// RUN: fir-opt %s -pass-pipeline='builtin.module(test-fir-alias-analysis)' -split-input-file --mlir-disable-threading 2>&1 | FileCheck %s
// CHECK-LABEL: Testing : <<NULL ATTRIBUTE>>
// CHECK: mold_data#0 <-> priv_data#0: MayAlias
// This test verifies that alias analysis correctly returns MayAlias for POINTER
// variables in omp.private copy regions. For POINTER, the descriptor is copied
// but pointer association is preserved, so both descriptors point to the SAME
// target data.
omp.private {type = firstprivate} @ptr_privatizer : !fir.box<!fir.ptr<!fir.array<8xi32>>> init {
^bb0(%arg0: !fir.ref<!fir.box<!fir.ptr<!fir.array<8xi32>>>>, %arg1: !fir.ref<!fir.box<!fir.ptr<!fir.array<8xi32>>>>):
%c8 = arith.constant 8 : index
%c1 = arith.constant 1 : index
%0 = fir.zero_bits !fir.ptr<!fir.array<8xi32>>
%1 = fir.shape %c8 : (index) -> !fir.shape<1>
%2 = fir.embox %0(%1) : (!fir.ptr<!fir.array<8xi32>>, !fir.shape<1>) -> !fir.box<!fir.ptr<!fir.array<8xi32>>>
fir.store %2 to %arg1 : !fir.ref<!fir.box<!fir.ptr<!fir.array<8xi32>>>>
omp.yield(%arg1 : !fir.ref<!fir.box<!fir.ptr<!fir.array<8xi32>>>>)
} copy {
^bb0(%arg0: !fir.ref<!fir.box<!fir.ptr<!fir.array<8xi32>>>>, %arg1: !fir.ref<!fir.box<!fir.ptr<!fir.array<8xi32>>>>):
// For POINTER: the descriptor is copied, but pointer association is preserved.
// Load both the mold (%arg0) and private (%arg1) to test alias analysis.
// Both descriptors point to THE SAME target data => should ALIAS.
%mold_desc = fir.load %arg0 {test.ptr = "mold_data"} : !fir.ref<!fir.box<!fir.ptr<!fir.array<8xi32>>>>
fir.store %mold_desc to %arg1 : !fir.ref<!fir.box<!fir.ptr<!fir.array<8xi32>>>>
%priv_desc = fir.load %arg1 {test.ptr = "priv_data"} : !fir.ref<!fir.box<!fir.ptr<!fir.array<8xi32>>>>
omp.yield(%arg1 : !fir.ref<!fir.box<!fir.ptr<!fir.array<8xi32>>>>)
}

View file

@ -83,72 +83,72 @@ end subroutine
! CHECK: omp.private.copy12: ; preds = %omp.private.copy
! [begin firstprivate copy for first var]
! [read the length, is it non-zero?]
! CHECK: br i1 %{{.*}}, label %omp.private.copy13, label %omp.private.copy14
! CHECK: br i1 %{{.*}}, label %omp.private.copy13, label %omp.private.copy22
! CHECK: omp.private.copy14: ; preds = %omp.private.copy13, %omp.private.copy12
! CHECK: omp.private.copy22: ; preds = %omp.private.copy21, %omp.private.copy12
! CHECK-NEXT: br label %omp.region.cont11
! CHECK: omp.region.cont11: ; preds = %omp.private.copy14
! CHECK: omp.region.cont11: ; preds = %omp.private.copy22
! CHECK-NEXT: %{{.*}} = phi ptr
! CHECK-NEXT: br label %omp.private.copy16
! CHECK-NEXT: br label %omp.private.copy24
! CHECK: omp.private.copy16: ; preds = %omp.region.cont11
! CHECK: omp.private.copy24: ; preds = %omp.region.cont11
! [begin firstprivate copy for second var]
! [read the length, is it non-zero?]
! CHECK: br i1 %{{.*}}, label %omp.private.copy17, label %omp.private.copy18
! CHECK: br i1 %{{.*}}, label %omp.private.copy25, label %omp.private.copy34
! CHECK: omp.private.copy18: ; preds = %omp.private.copy17, %omp.private.copy16
! CHECK-NEXT: br label %omp.region.cont15
! CHECK: omp.private.copy34: ; preds = %omp.private.copy33, %omp.private.copy24
! CHECK-NEXT: br label %omp.region.cont23
! CHECK: omp.region.cont15: ; preds = %omp.private.copy18
! CHECK: omp.region.cont23: ; preds = %omp.private.copy34
! CHECK-NEXT: %{{.*}} = phi ptr
! CHECK-NEXT: br label %omp.reduction.init
! CHECK: omp.reduction.init: ; preds = %omp.region.cont15
! CHECK: omp.reduction.init: ; preds = %omp.region.cont23
! [deferred stores for results of reduction alloc regions]
! CHECK: br label %[[VAL_96:.*]]
! CHECK: omp.reduction.neutral: ; preds = %omp.reduction.init
! [start of reduction initialization region]
! [null check:]
! CHECK: br i1 %{{.*}}, label %omp.reduction.neutral20, label %omp.reduction.neutral21
! CHECK: br i1 %{{.*}}, label %omp.reduction.neutral36, label %omp.reduction.neutral37
! CHECK: omp.reduction.neutral21: ; preds = %omp.reduction.neutral
! CHECK: omp.reduction.neutral37: ; preds = %omp.reduction.neutral
! [malloc and assign the default value to the reduction variable]
! CHECK: br label %omp.reduction.neutral22
! CHECK: br label %omp.reduction.neutral38
! CHECK: omp.reduction.neutral22: ; preds = %omp.reduction.neutral20, %omp.reduction.neutral21
! CHECK-NEXT: br label %omp.region.cont19
! CHECK: omp.reduction.neutral38: ; preds = %omp.reduction.neutral36, %omp.reduction.neutral37
! CHECK-NEXT: br label %omp.region.cont35
! CHECK: omp.region.cont19: ; preds = %omp.reduction.neutral22
! CHECK: omp.region.cont35: ; preds = %omp.reduction.neutral38
! CHECK-NEXT: %{{.*}} = phi ptr
! CHECK-NEXT: br label %omp.reduction.neutral24
! CHECK-NEXT: br label %omp.reduction.neutral40
! CHECK: omp.reduction.neutral24: ; preds = %omp.region.cont19
! CHECK: omp.reduction.neutral40: ; preds = %omp.region.cont35
! [start of reduction initialization region]
! [null check:]
! CHECK: br i1 %{{.*}}, label %omp.reduction.neutral25, label %omp.reduction.neutral26
! CHECK: br i1 %{{.*}}, label %omp.reduction.neutral41, label %omp.reduction.neutral42
! CHECK: omp.reduction.neutral26: ; preds = %omp.reduction.neutral24
! CHECK: omp.reduction.neutral42: ; preds = %omp.reduction.neutral40
! [malloc and assign the default value to the reduction variable]
! CHECK: br label %omp.reduction.neutral27
! CHECK: br label %omp.reduction.neutral43
! CHECK: omp.reduction.neutral27: ; preds = %omp.reduction.neutral25, %omp.reduction.neutral26
! CHECK-NEXT: br label %omp.region.cont23
! CHECK: omp.reduction.neutral43: ; preds = %omp.reduction.neutral41, %omp.reduction.neutral42
! CHECK-NEXT: br label %omp.region.cont39
! CHECK: omp.region.cont23: ; preds = %omp.reduction.neutral27
! CHECK: omp.region.cont39: ; preds = %omp.reduction.neutral43
! CHECK-NEXT: %{{.*}} = phi ptr
! CHECK-NEXT: br label %omp.par.region29
! CHECK-NEXT: br label %omp.par.region45
! CHECK: omp.par.region29: ; preds = %omp.region.cont23
! CHECK: omp.par.region45: ; preds = %omp.region.cont39
! [call SUM runtime function]
! [if (sum(a) == 1)]
! CHECK: br i1 %{{.*}}, label %omp.par.region30, label %omp.par.region31
! CHECK: br i1 %{{.*}}, label %omp.par.region46, label %omp.par.region47
! CHECK: omp.par.region31: ; preds = %omp.par.region29
! CHECK-NEXT: br label %omp.region.cont28
! CHECK: omp.par.region47: ; preds = %omp.par.region45
! CHECK-NEXT: br label %omp.region.cont44
! CHECK: omp.region.cont28: ; preds = %omp.par.region31
! CHECK: omp.region.cont44: ; preds = %omp.par.region47
! [omp parallel region done, call into the runtime to complete reduction]
! CHECK: %[[VAL_233:.*]] = call i32 @__kmpc_reduce(
! CHECK: switch i32 %[[VAL_233]], label %reduce.finalize [
@ -156,16 +156,16 @@ end subroutine
! CHECK-NEXT: i32 2, label %reduce.switch.atomic
! CHECK-NEXT: ]
! CHECK: reduce.switch.atomic: ; preds = %omp.region.cont28
! CHECK: reduce.switch.atomic: ; preds = %omp.region.cont44
! CHECK-NEXT: unreachable
! CHECK: reduce.switch.nonatomic: ; preds = %omp.region.cont28
! CHECK: reduce.switch.nonatomic: ; preds = %omp.region.cont44
! CHECK-NEXT: %[[red_private_value_0:.*]] = load ptr, ptr %{{.*}}, align 8
! CHECK-NEXT: br label %omp.reduction.nonatomic.body
! [various blocks implementing the reduction]
! CHECK: omp.region.cont36: ; preds =
! CHECK: omp.region.cont52: ; preds =
! CHECK-NEXT: %{{.*}} = phi ptr
! CHECK-NEXT: call void @__kmpc_end_reduce(
! CHECK-NEXT: br label %reduce.finalize
@ -182,38 +182,38 @@ end subroutine
! CHECK: omp.reduction.cleanup: ; preds = %.fini
! [null check]
! CHECK: br i1 %{{.*}}, label %omp.reduction.cleanup42, label %omp.reduction.cleanup43
! CHECK: br i1 %{{.*}}, label %omp.reduction.cleanup58, label %omp.reduction.cleanup59
! CHECK: omp.reduction.cleanup43: ; preds = %omp.reduction.cleanup42, %omp.reduction.cleanup
! CHECK-NEXT: br label %omp.region.cont41
! CHECK: omp.reduction.cleanup59: ; preds = %omp.reduction.cleanup58, %omp.reduction.cleanup
! CHECK-NEXT: br label %omp.region.cont57
! CHECK: omp.region.cont41: ; preds = %omp.reduction.cleanup43
! CHECK: omp.region.cont57: ; preds = %omp.reduction.cleanup59
! CHECK-NEXT: %{{.*}} = load ptr, ptr
! CHECK-NEXT: br label %omp.reduction.cleanup45
! CHECK-NEXT: br label %omp.reduction.cleanup61
! CHECK: omp.reduction.cleanup45: ; preds = %omp.region.cont41
! CHECK: omp.reduction.cleanup61: ; preds = %omp.region.cont57
! [null check]
! CHECK: br i1 %{{.*}}, label %omp.reduction.cleanup46, label %omp.reduction.cleanup47
! CHECK: br i1 %{{.*}}, label %omp.reduction.cleanup62, label %omp.reduction.cleanup63
! CHECK: omp.par.region30: ; preds = %omp.par.region29
! CHECK: omp.par.region46: ; preds = %omp.par.region45
! CHECK-NEXT: call void @_FortranAStopStatement
! CHECK-NEXT: unreachable
! CHECK: omp.reduction.neutral25: ; preds = %omp.reduction.neutral24
! CHECK: omp.reduction.neutral41: ; preds = %omp.reduction.neutral40
! [source length was zero: finish initializing array]
! CHECK: br label %omp.reduction.neutral27
! CHECK: br label %omp.reduction.neutral43
! CHECK: omp.reduction.neutral20: ; preds = %omp.reduction.neutral
! CHECK: omp.reduction.neutral36: ; preds = %omp.reduction.neutral
! [source length was zero: finish initializing array]
! CHECK: br label %omp.reduction.neutral22
! CHECK: br label %omp.reduction.neutral38
! CHECK: omp.private.copy17: ; preds = %omp.private.copy16
! CHECK: omp.private.copy33: ; preds = %omp.private.copy32, %omp.private.copy29
! [source length was non-zero: call assign runtime]
! CHECK: br label %omp.private.copy18
! CHECK: br label %omp.private.copy34
! CHECK: omp.private.copy13: ; preds = %omp.private.copy12
! CHECK: omp.private.copy21: ; preds = %omp.private.copy20, %omp.private.copy17
! [source length was non-zero: call assign runtime]
! CHECK: br label %omp.private.copy14
! CHECK: br label %omp.private.copy22
! CHECK: omp.private.init8: ; preds = %omp.private.init7
! [var extent was non-zero: malloc a private array]
@ -223,5 +223,5 @@ end subroutine
! [var extent was non-zero: malloc a private array]
! CHECK: br label %omp.private.init5
! CHECK: omp.par.exit.exitStub: ; preds = %omp.region.cont51
! CHECK: omp.par.exit.exitStub: ; preds = %omp.region.cont67
! CHECK-NEXT: ret void