Cold-Air-Networks/mirror-llvm-project
2
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions 7

[IR] Enable dereferenceable-at-point semantics (#204795)

Browse source 
The precise semantics of `dereferenceable` are currently undocumented,
but de facto they are:

* Argument position `dereferenceable` implies that the memory remains
dereferenceable while the function executes.
* Return position `dereferenceable` implies that the memory remains
dereferenceable forever.

This is not coherent with how these attributes are used, in at least two
ways:

* C++ reference arguments are annotated as `dereferenceable` by Clang.
However, C++ does not actually make any guarantees that the underlying
memory does not get freed during the execution of the function. (Unlike
Rust, which does guarantee this via protectors in SB/TB.)
* We infer `dereferenceable` on allocator return values (from
`allocsize`). Allocator return values quite obviously do not stay
dereferenceable forever, as they can be freed.

This PR changes the semantics of `dereferenceable` to apply at-point,
i.e. only directly on function entry (for arguments), or at the position
of the call (for return values). The fact that the memory cannot be
freed between the point where the `dereferenceable` fact has been
established and the point where we're trying to speculate needs to be
explicitly proven now.

The main ways this happens is:

* Via `nofree` function attributes (inferred).
* Via `noalias nofree` argument attributes. The latter is inferred, but
the former generally requires frontend annotation. In Rust the
combination of `noalias nofree dereferenceable` is used, so it should
not be materially affected by these changes.
* Context-sensitive analysis to check for frees (i.e. non-nofree calls
or synchronizing atomics) between the point of definition and the point
of speculation.

It's expected that this will regress optimization of C++ code in some
cases, because we can no longer use dereferenceability facts that are
not, in fact, guaranteed to hold.

This finishes implementing an old RFC:
https://discourse.llvm.org/t/rfc-decomposing-deref-n-into-deref-n-nofree/57873
This commit is contained in:
Nikita Popov 2026-07-03 08:08:47 +02:00 committed by GitHub
parent aa7773bef3
commit 5969d640a3
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
20 changed files with 87 additions and 69 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

16
llvm/docs/LangRef.rst
View file

@ -1620,6 +1620,12 @@ Currently, only the following parameter attributes are defined:
otherwise it is undefined behavior. This means ``dereferenceable(<n>)``
implies ``noundef``.
The ``dereferenceable`` attribute only implies dereferenceability at the
point of the attribute (i.e. on function entry for arguments or at the
point of the call for return values). The underlying object may still get
freed after that point. Other attributes such as ``nofree`` can be used
to exclude frees.
.. _attr_dereferenceable_or_null:
``dereferenceable_or_null(<n>)``
@ -7838,12 +7844,15 @@ it is attached to is completely unpredictable.
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
The existence of the ``!dereferenceable`` metadata on the instruction
tells the optimizer that the value loaded is known to be dereferenceable,
otherwise the behavior is undefined.
tells the optimizer that the value loaded is known to be dereferenceable
at the current program point, otherwise the behavior is undefined.
The number of bytes known to be dereferenceable is specified by the integer
value in the metadata node. This is analogous to the ''dereferenceable''
attribute on parameters and return values.
The ``!deferenceable`` metadata can be combined with the ``!nofree`` metadata
to indicate that the pointer will stay dereferenceable forever.
.. _md_dereferenceable_or_null:
'``dereferenceable_or_null``' Metadata
@ -7851,7 +7860,8 @@ attribute on parameters and return values.
The existence of the ``!dereferenceable_or_null`` metadata on the
instruction tells the optimizer that the value loaded is known to be either
dereferenceable or null, otherwise the behavior is undefined.
dereferenceable at the current program point or null, otherwise the behavior is
undefined.
The number of bytes known to be dereferenceable is specified by the integer
value in the metadata node. This is analogous to the ''dereferenceable_or_null''
attribute on parameters and return values.

4
llvm/docs/ReleaseNotes.md
View file

@ -107,6 +107,10 @@ Makes programs 10x faster by doing Special New Thing.
graph section generation. See the [callgraph Metadata](https://llvm.org/docs/LangRef.html#callgraph-metadata)
section of the LangRef for details.
* The `dereferenceable` and `dereferenceable_or_null` attributes (and
corresponding metadata) now apply only at the point of definition, instead of
for the execution of the function (for arguments) or forever (for returns).
### Changes to LLVM infrastructure
* Removed ``Constant::isZeroValue``. It was functionally identical to

2
llvm/lib/IR/Value.cpp
View file

@ -38,7 +38,7 @@
using namespace llvm;
static cl::opt<bool> UseDerefAtPointSemantics(
"use-dereferenceable-at-point-semantics", cl::Hidden, cl::init(false),
"use-dereferenceable-at-point-semantics", cl::Hidden, cl::init(true),
cl::desc("Deref attributes and metadata infer facts at definition only"));
//===----------------------------------------------------------------------===//

2
llvm/test/CodeGen/AMDGPU/amdhsa-trap-num-sgprs.ll
View file

@ -2,7 +2,7 @@
; RUN: llc -mtriple=amdgcn-amd-amdhsa -mcpu=gfx803 -mattr=-trap-handler < %s | FileCheck %s --check-prefixes=GCN,TRAP-HANDLER-DISABLE
; GCN-LABEL: {{^}}amdhsa_trap_num_sgprs
; TRAP-HANDLER-ENABLE: NumSgprs: 61
; TRAP-HANDLER-ENABLE: NumSgprs: 63
; TRAP-HANDLER-DISABLE: NumSgprs: 77
define amdgpu_kernel void @amdhsa_trap_num_sgprs(
ptr addrspace(1) %out0, i32 %in0,

15
llvm/test/CodeGen/AMDGPU/asyncmark-gfx12plus.ll
View file

@ -172,12 +172,13 @@ define amdgpu_kernel void @test_pipelined_loop(ptr addrspace(1) %foo, ptr addrsp
; SDAG-NEXT: s_cbranch_scc1 .LBB1_1
; SDAG-NEXT: ; %bb.2: ; %epilog
; SDAG-NEXT: s_lshl2_add_u32 s0, s3, s2
; SDAG-NEXT: ; wait_asyncmark(1)
; SDAG-NEXT: s_wait_asynccnt 0x1
; SDAG-NEXT: v_mov_b32_e32 v2, 0
; SDAG-NEXT: s_add_co_i32 s0, s0, -8
; SDAG-NEXT: s_delay_alu instid0(SALU_CYCLE_1)
; SDAG-NEXT: v_dual_mov_b32 v2, 0 :: v_dual_mov_b32 v0, s0
; SDAG-NEXT: v_mov_b32_e32 v0, s0
; SDAG-NEXT: s_load_b64 s[0:1], s[4:5], 0x34 nv
; SDAG-NEXT: ; wait_asyncmark(1)
; SDAG-NEXT: s_wait_asynccnt 0x1
; SDAG-NEXT: ds_load_b32 v0, v0
; SDAG-NEXT: ; wait_asyncmark(0)
; SDAG-NEXT: s_wait_dscnt 0x0
@ -228,14 +229,14 @@ define amdgpu_kernel void @test_pipelined_loop(ptr addrspace(1) %foo, ptr addrsp
; GISEL-NEXT: s_cbranch_scc1 .LBB1_1
; GISEL-NEXT: ; %bb.2: ; %epilog
; GISEL-NEXT: s_lshl_b32 s0, s3, 2
; GISEL-NEXT: ; wait_asyncmark(1)
; GISEL-NEXT: s_wait_asynccnt 0x1
; GISEL-NEXT: s_add_co_u32 s0, s2, s0
; GISEL-NEXT: v_mov_b32_e32 v1, 0
; GISEL-NEXT: s_add_co_u32 s0, s2, s0
; GISEL-NEXT: s_delay_alu instid0(SALU_CYCLE_1) | instskip(NEXT) | instid1(SALU_CYCLE_1)
; GISEL-NEXT: s_add_co_u32 s0, s0, -8
; GISEL-NEXT: s_delay_alu instid0(SALU_CYCLE_1)
; GISEL-NEXT: v_mov_b32_e32 v0, s0
; GISEL-NEXT: s_load_b64 s[0:1], s[4:5], 0x34 nv
; GISEL-NEXT: ; wait_asyncmark(1)
; GISEL-NEXT: s_wait_asynccnt 0x1
; GISEL-NEXT: ds_load_b32 v0, v0
; GISEL-NEXT: ; wait_asyncmark(0)
; GISEL-NEXT: s_wait_dscnt 0x0

14
llvm/test/CodeGen/AMDGPU/insert-delay-alu-bug.ll
View file

@ -117,18 +117,20 @@ define amdgpu_kernel void @f2(i32 %arg, i32 %arg1, i32 %arg2, i1 %arg3, i32 %arg
; GFX11-NEXT: s_cmp_eq_u32 s1, 0
; GFX11-NEXT: s_cbranch_scc0 .LBB2_9
; GFX11-NEXT: ; %bb.6: ; %bb18.preheader
; GFX11-NEXT: s_load_b128 s[28:31], s[18:19], 0x44
; GFX11-NEXT: s_clause 0x1
; GFX11-NEXT: s_load_b64 s[0:1], s[18:19], 0x44
; GFX11-NEXT: s_load_b32 s9, s[18:19], 0x4c
; GFX11-NEXT: v_mov_b32_e32 v0, 0
; GFX11-NEXT: s_mov_b32 vcc_lo, 0
; GFX11-NEXT: s_waitcnt lgkmcnt(0)
; GFX11-NEXT: s_mul_hi_u32 s1, s29, s28
; GFX11-NEXT: s_mul_i32 s0, s29, s28
; GFX11-NEXT: s_delay_alu instid0(SALU_CYCLE_1) | instskip(SKIP_4) | instid1(SALU_CYCLE_1)
; GFX11-NEXT: s_lshr_b64 s[0:1], s[0:1], 1
; GFX11-NEXT: s_mul_hi_u32 s29, s1, s0
; GFX11-NEXT: s_mul_i32 s28, s1, s0
; GFX11-NEXT: s_delay_alu instid0(SALU_CYCLE_1) | instskip(SKIP_2) | instid1(SALU_CYCLE_1)
; GFX11-NEXT: s_lshr_b64 s[0:1], s[28:29], 1
; GFX11-NEXT: s_mov_b32 s1, 0
; GFX11-NEXT: s_or_b32 s0, s0, 1
; GFX11-NEXT: s_lshr_b32 s0, s0, s9
; GFX11-NEXT: s_mov_b32 s9, s1
; GFX11-NEXT: s_lshr_b32 s0, s0, s30
; GFX11-NEXT: s_mul_i32 s0, s0, s22
; GFX11-NEXT: s_delay_alu instid0(SALU_CYCLE_1) | instskip(NEXT) | instid1(SALU_CYCLE_1)
; GFX11-NEXT: s_mul_i32 s0, s0, s20

24
llvm/test/CodeGen/AMDGPU/madak.ll
View file

@ -1323,14 +1323,13 @@ define amdgpu_kernel void @madak_constant_bus_violation(i32 %arg1, [8 x i32], fl
; GFX6-NEXT: buffer_store_dword v0, off, s[0:3], 0
; GFX6-NEXT: s_waitcnt vmcnt(0)
; GFX6-NEXT: .LBB9_2: ; %bb4
; GFX6-NEXT: s_load_dword s0, s[4:5], 0x12
; GFX6-NEXT: s_mov_b32 s3, 0xf000
; GFX6-NEXT: s_mov_b32 s2, -1
; GFX6-NEXT: s_waitcnt expcnt(0)
; GFX6-NEXT: v_mov_b32_e32 v1, 0x42280000
; GFX6-NEXT: s_waitcnt expcnt(0) lgkmcnt(0)
; GFX6-NEXT: buffer_load_dword v0, off, s[0:3], 0 glc
; GFX6-NEXT: s_waitcnt vmcnt(0)
; GFX6-NEXT: s_load_dword s0, s[4:5], 0x12
; GFX6-NEXT: v_mov_b32_e32 v1, 0x42280000
; GFX6-NEXT: s_waitcnt lgkmcnt(0)
; GFX6-NEXT: v_mac_f32_e64 v1, s0, 0.5
; GFX6-NEXT: v_mul_f32_e32 v0, v1, v0
; GFX6-NEXT: buffer_store_dword v0, off, s[0:3], 0
@ -1348,9 +1347,9 @@ define amdgpu_kernel void @madak_constant_bus_violation(i32 %arg1, [8 x i32], fl
; GFX8-NEXT: flat_store_dword v[0:1], v0
; GFX8-NEXT: s_waitcnt vmcnt(0)
; GFX8-NEXT: .LBB9_2: ; %bb4
; GFX8-NEXT: s_load_dword s0, s[4:5], 0x48
; GFX8-NEXT: flat_load_dword v0, v[0:1] glc
; GFX8-NEXT: s_waitcnt vmcnt(0)
; GFX8-NEXT: s_load_dword s0, s[4:5], 0x48
; GFX8-NEXT: v_mov_b32_e32 v1, 0x42280000
; GFX8-NEXT: s_waitcnt lgkmcnt(0)
; GFX8-NEXT: v_mac_f32_e64 v1, s0, 0.5
@ -1370,9 +1369,9 @@ define amdgpu_kernel void @madak_constant_bus_violation(i32 %arg1, [8 x i32], fl
; GFX9-NEXT: global_store_dword v[0:1], v0, off
; GFX9-NEXT: s_waitcnt vmcnt(0)
; GFX9-NEXT: .LBB9_2: ; %bb4
; GFX9-NEXT: s_load_dword s0, s[4:5], 0x48
; GFX9-NEXT: global_load_dword v0, v[0:1], off glc
; GFX9-NEXT: s_waitcnt vmcnt(0)
; GFX9-NEXT: s_load_dword s0, s[4:5], 0x48
; GFX9-NEXT: v_mov_b32_e32 v1, 0x42280000
; GFX9-NEXT: s_waitcnt lgkmcnt(0)
; GFX9-NEXT: v_mac_f32_e64 v1, s0, 0.5
@ -1392,9 +1391,9 @@ define amdgpu_kernel void @madak_constant_bus_violation(i32 %arg1, [8 x i32], fl
; GFX10-MAD-NEXT: global_store_dword v[0:1], v0, off
; GFX10-MAD-NEXT: s_waitcnt_vscnt null, 0x0
; GFX10-MAD-NEXT: .LBB9_2: ; %bb4
; GFX10-MAD-NEXT: s_load_dword s0, s[4:5], 0x48
; GFX10-MAD-NEXT: global_load_dword v0, v[0:1], off glc dlc
; GFX10-MAD-NEXT: s_waitcnt vmcnt(0)
; GFX10-MAD-NEXT: s_load_dword s0, s[4:5], 0x48
; GFX10-MAD-NEXT: v_mov_b32_e32 v1, 0.5
; GFX10-MAD-NEXT: s_waitcnt lgkmcnt(0)
; GFX10-MAD-NEXT: v_madak_f32 v1, s0, v1, 0x42280000
@ -1414,10 +1413,9 @@ define amdgpu_kernel void @madak_constant_bus_violation(i32 %arg1, [8 x i32], fl
; GFX11-MAD-NEXT: global_store_b32 v[0:1], v0, off dlc
; GFX11-MAD-NEXT: s_waitcnt_vscnt null, 0x0
; GFX11-MAD-NEXT: .LBB9_2: ; %bb4
; GFX11-MAD-NEXT: global_load_b32 v0, v[0:1], off glc dlc
; GFX11-MAD-NEXT: s_waitcnt vmcnt(0)
; GFX11-MAD-NEXT: s_load_b32 s0, s[4:5], 0x48
; GFX11-MAD-NEXT: s_waitcnt lgkmcnt(0)
; GFX11-MAD-NEXT: global_load_b32 v0, v[0:1], off glc dlc
; GFX11-MAD-NEXT: s_waitcnt vmcnt(0) lgkmcnt(0)
; GFX11-MAD-NEXT: v_mul_f32_e64 v1, s0, 0.5
; GFX11-MAD-NEXT: s_delay_alu instid0(VALU_DEP_1) | instskip(NEXT) | instid1(VALU_DEP_1)
; GFX11-MAD-NEXT: v_add_f32_e32 v1, 0x42280000, v1
@ -1437,9 +1435,9 @@ define amdgpu_kernel void @madak_constant_bus_violation(i32 %arg1, [8 x i32], fl
; GFX942-FMA-NEXT: global_store_dword v[0:1], v0, off sc0 sc1
; GFX942-FMA-NEXT: s_waitcnt vmcnt(0)
; GFX942-FMA-NEXT: .LBB9_2: ; %bb4
; GFX942-FMA-NEXT: s_load_dword s0, s[4:5], 0x48
; GFX942-FMA-NEXT: global_load_dword v0, v[0:1], off sc0 sc1
; GFX942-FMA-NEXT: s_waitcnt vmcnt(0)
; GFX942-FMA-NEXT: s_load_dword s0, s[4:5], 0x48
; GFX942-FMA-NEXT: v_mov_b32_e32 v1, 0x42280000
; GFX942-FMA-NEXT: s_waitcnt lgkmcnt(0)
; GFX942-FMA-NEXT: v_fmac_f32_e64 v1, s0, 0.5
@ -1459,9 +1457,9 @@ define amdgpu_kernel void @madak_constant_bus_violation(i32 %arg1, [8 x i32], fl
; GFX10-FMA-NEXT: global_store_dword v[0:1], v0, off
; GFX10-FMA-NEXT: s_waitcnt_vscnt null, 0x0
; GFX10-FMA-NEXT: .LBB9_2: ; %bb4
; GFX10-FMA-NEXT: s_load_dword s0, s[4:5], 0x48
; GFX10-FMA-NEXT: global_load_dword v0, v[0:1], off glc dlc
; GFX10-FMA-NEXT: s_waitcnt vmcnt(0)
; GFX10-FMA-NEXT: s_load_dword s0, s[4:5], 0x48
; GFX10-FMA-NEXT: v_mov_b32_e32 v1, 0.5
; GFX10-FMA-NEXT: s_waitcnt lgkmcnt(0)
; GFX10-FMA-NEXT: v_fmaak_f32 v1, s0, v1, 0x42280000
@ -1481,9 +1479,9 @@ define amdgpu_kernel void @madak_constant_bus_violation(i32 %arg1, [8 x i32], fl
; GFX11-FMA-NEXT: global_store_b32 v[0:1], v0, off dlc
; GFX11-FMA-NEXT: s_waitcnt_vscnt null, 0x0
; GFX11-FMA-NEXT: .LBB9_2: ; %bb4
; GFX11-FMA-NEXT: s_load_b32 s0, s[4:5], 0x48
; GFX11-FMA-NEXT: global_load_b32 v0, v[0:1], off glc dlc
; GFX11-FMA-NEXT: s_waitcnt vmcnt(0)
; GFX11-FMA-NEXT: s_load_b32 s0, s[4:5], 0x48
; GFX11-FMA-NEXT: v_mov_b32_e32 v1, 0.5
; GFX11-FMA-NEXT: s_waitcnt lgkmcnt(0)
; GFX11-FMA-NEXT: s_delay_alu instid0(VALU_DEP_1) | instskip(NEXT) | instid1(VALU_DEP_1)

2
llvm/test/CodeGen/AMDGPU/rewrite-mfma-form-spill-cost-reset.ll
View file

@ -533,7 +533,7 @@ define amdgpu_kernel void @test_spill_cost_reset(
; CHECK-NEXT: S_BRANCH %bb.2
; CHECK-NEXT: {{ $}}
; CHECK-NEXT: bb.2.epilogue:
; CHECK-NEXT: [[S_LOAD_DWORDX2_IMM:%[0-9]+]]:sreg_64_xexec_xnull = S_LOAD_DWORDX2_IMM [[COPY]](p4), 0, 0 :: (dereferenceable invariant load (s64) from %ir.out.kernarg.offset550, align 16, addrspace 4)
; CHECK-NEXT: [[S_LOAD_DWORDX2_IMM:%[0-9]+]]:sreg_64_xexec_xnull = S_LOAD_DWORDX2_IMM [[COPY]](p4), 0, 0 :: (invariant load (s64) from %ir.out.kernarg.offset550, align 16, addrspace 4)
; CHECK-NEXT: [[V_MOV_B32_e32_13:%[0-9]+]]:vgpr_32 = V_MOV_B32_e32 0, implicit $exec
; CHECK-NEXT: GLOBAL_STORE_DWORD_SADDR [[V_MOV_B32_e32_13]], [[V_MFMA_F32_16X16X32_F16_vgprcd_e64_2]].sub0, [[S_LOAD_DWORDX2_IMM]], 0, 0, implicit $exec :: (store (s32) into %ir.out.load, addrspace 1)
; CHECK-NEXT: S_ENDPGM 0

2
llvm/test/CodeGen/X86/load-partial.ll
View file

@ -150,7 +150,7 @@ define <4 x float> @load_float4_float3_as_float2_float_0122(ptr nocapture readon
ret <4 x float> %10
}
define <4 x float> @load_float4_float3_trunc(ptr nocapture readonly dereferenceable(16)) {
define <4 x float> @load_float4_float3_trunc(ptr nocapture readonly dereferenceable(16)) nofree {
; SSE-LABEL: load_float4_float3_trunc:
; SSE: # %bb.0:
; SSE-NEXT: movaps (%rdi), %xmm0

10
llvm/test/CodeGen/X86/oddsubvector.ll
View file

@ -91,11 +91,11 @@ define void @PR40815(ptr nocapture readonly dereferenceable(64), ptr nocapture d
;
; AVX512-LABEL: PR40815:
; AVX512: # %bb.0:
; AVX512-NEXT: vmovaps 48(%rdi), %xmm0
; AVX512-NEXT: vmovups 16(%rdi), %ymm1
; AVX512-NEXT: vinsertf128 $1, (%rdi), %ymm1, %ymm1
; AVX512-NEXT: vinsertf128 $1, 32(%rdi), %ymm0, %ymm0
; AVX512-NEXT: vinsertf64x4 $1, %ymm1, %zmm0, %zmm0
; AVX512-NEXT: vmovaps 16(%rdi), %xmm0
; AVX512-NEXT: vmovaps 48(%rdi), %xmm1
; AVX512-NEXT: vinsertf128 $1, (%rdi), %ymm0, %ymm0
; AVX512-NEXT: vinsertf128 $1, 32(%rdi), %ymm1, %ymm1
; AVX512-NEXT: vinsertf64x4 $1, %ymm0, %zmm1, %zmm0
; AVX512-NEXT: vmovups %zmm0, (%rsi)
; AVX512-NEXT: vzeroupper
; AVX512-NEXT: retq

2
llvm/test/Transforms/GVN/PRE/volatile.ll
View file

@ -153,7 +153,7 @@ exit:
; Another volatile PRE case - two paths through a loop
; load in preheader, one path read only, one not
define i32 @test8(i1 %b, i1 %c, ptr noalias align 4 dereferenceable(4) %p, ptr noalias %q) {
define i32 @test8(i1 %b, i1 %c, ptr noalias align 4 dereferenceable(4) %p, ptr noalias %q) nofree {
; CHECK-LABEL: @test8(
; CHECK-NEXT: entry:
; CHECK-NEXT: [[Y1:%.*]] = load i32, ptr [[P:%.*]], align 4

2
llvm/test/Transforms/LoopVectorize/AArch64/simple_early_exit.ll
View file

@ -3,7 +3,7 @@
target triple = "aarch64-unknown-linux-gnu"
declare void @init_mem(ptr, i64);
declare void @init_mem(ptr, i64) nofree
define i64 @same_exit_block_pre_inc_use1() #1 {
; CHECK-LABEL: define i64 @same_exit_block_pre_inc_use1(

2
llvm/test/Transforms/LoopVectorize/early_exit_legality.ll
View file

@ -2,7 +2,7 @@
; REQUIRES: asserts
; RUN: opt -S < %s -p loop-vectorize -debug-only=loop-vectorize,vplan -force-vector-width=4 -disable-output 2>&1 | FileCheck %s
declare void @init_mem(ptr, i64);
declare void @init_mem(ptr, i64) nofree
; == SOME LEGAL EXAMPLES ==

18
llvm/test/Transforms/LoopVectorize/early_exit_with_outer_loop.ll
View file

@ -122,7 +122,7 @@ exit:
; Tests when an early-exit inner loop has multiple exits and all exits
; leave the outer loop.
define i32 @multi_early_exit_all_leave_outer_loop(i1 %c, ptr dereferenceable(1024) %src) {
define i32 @multi_early_exit_all_leave_outer_loop(i1 %c, ptr dereferenceable(1024) %src) nofree {
; CHECK-LABEL: Loop info for function 'multi_early_exit_all_leave_outer_loop':
; CHECK-NEXT: Loop at depth 1 containing: %outer.header<header>,%outer.latch<latch><exiting>,%vector.ph,%vector.body<exiting>,%vector.body.interim,%middle.block
; CHECK-NEXT: Loop at depth 2 containing: %vector.body<header><exiting>,%vector.body.interim<latch><exiting>
@ -159,7 +159,7 @@ early.exit.2:
; Tests when an inner loop has two early exits at different loop levels
; (one staying in the outer loop, one leaving all loops).
define i32 @multi_early_exit_different_loop_levels(i1 %c, ptr dereferenceable(1024) %src) {
define i32 @multi_early_exit_different_loop_levels(i1 %c, ptr dereferenceable(1024) %src) nofree {
; CHECK-LABEL: Loop info for function 'multi_early_exit_different_loop_levels':
; CHECK-NEXT: Loop at depth 1 containing: %outer.header<header>,%early.exit.outer,%outer.latch<latch>,%outer.latch.loopexit,%vector.ph,%vector.body,%vector.body.interim,%middle.block,%vector.early.exit.check<exiting>,%vector.early.exit.0
; CHECK-NEXT: Loop at depth 2 containing: %vector.body<header><exiting>,%vector.body.interim<latch><exiting>
@ -196,7 +196,7 @@ early.exit.leave:
; Same as above, but the early exit order is reversed: the first early exit
; leaves all loops and the second stays in the outer loop.
define i32 @multi_early_exit_different_loop_levels_reversed(i1 %c, ptr dereferenceable(1024) %src) {
define i32 @multi_early_exit_different_loop_levels_reversed(i1 %c, ptr dereferenceable(1024) %src) nofree {
; CHECK-LABEL: Loop info for function 'multi_early_exit_different_loop_levels_reversed':
; CHECK-NEXT: Loop at depth 1 containing: %outer.header<header>,%early.exit.outer,%outer.latch<latch>,%outer.latch.loopexit,%vector.ph,%vector.body,%vector.body.interim,%middle.block,%vector.early.exit.check<exiting>,%vector.early.exit.1
; CHECK-NEXT: Loop at depth 2 containing: %vector.body<header><exiting>,%vector.body.interim<latch><exiting>
@ -233,7 +233,7 @@ early.exit.leave:
; Tests when an inner loop has two early exits at different loop levels
; (one going to a middle loop, one going to the outer loop).
define i32 @multi_early_exit_two_different_loops(i1 %c, ptr dereferenceable(1024) %src) {
define i32 @multi_early_exit_two_different_loops(i1 %c, ptr dereferenceable(1024) %src) nofree {
; CHECK-LABEL: Loop info for function 'multi_early_exit_two_different_loops':
; CHECK-NEXT: Loop at depth 1 containing: %outer.header<header>,%middle.header,%early.exit.outer,%early.exit.middle,%middle.latch,%outer.latch<latch>,%middle.latch.loopexit,%outer.latch.loopexit,%vector.ph,%vector.body,%vector.body.interim,%middle.block,%vector.early.exit.check,%vector.early.exit.1,%vector.early.exit.0
; CHECK-NEXT: Loop at depth 2 containing: %middle.header<header>,%early.exit.middle,%middle.latch<latch><exiting>,%middle.latch.loopexit,%vector.ph,%vector.body,%vector.body.interim,%middle.block,%vector.early.exit.check<exiting>,%vector.early.exit.0
@ -277,7 +277,7 @@ early.exit.outer:
; Same as above, but the early exit order is reversed: the first early exit
; goes to the outer loop and the second goes to the middle loop.
define i32 @multi_early_exit_two_different_loops_reversed(i1 %c, ptr dereferenceable(1024) %src) {
define i32 @multi_early_exit_two_different_loops_reversed(i1 %c, ptr dereferenceable(1024) %src) nofree {
; CHECK-LABEL: Loop info for function 'multi_early_exit_two_different_loops_reversed':
; CHECK-NEXT: Loop at depth 1 containing: %outer.header<header>,%middle.header,%early.exit.middle,%middle.latch,%early.exit.outer,%outer.latch<latch>,%middle.latch.loopexit,%outer.latch.loopexit,%vector.ph,%vector.body,%vector.body.interim,%middle.block,%vector.early.exit.check,%vector.early.exit.1,%vector.early.exit.0
; CHECK-NEXT: Loop at depth 2 containing: %middle.header<header>,%early.exit.middle,%middle.latch<latch><exiting>,%middle.latch.loopexit,%vector.ph,%vector.body,%vector.body.interim,%middle.block,%vector.early.exit.check<exiting>,%vector.early.exit.1
@ -323,7 +323,7 @@ early.exit.outer:
; intermediate exit-routing blocks have multiple successors (i.e., the
; vector.early.exit.check block itself has >1 successor), and all exits leave
; all loops.
define i32 @multi_early_exit_all_leave_with_multi_succ_exit_check(i1 %c.1, i1 %c.2, ptr dereferenceable(1024) %src) {
define i32 @multi_early_exit_all_leave_with_multi_succ_exit_check(i1 %c.1, i1 %c.2, ptr dereferenceable(1024) %src) nofree {
; CHECK-LABEL: Loop info for function 'multi_early_exit_all_leave_with_multi_succ_exit_check':
; CHECK-NEXT: Loop at depth 1 containing: %outer.header<header>,%outer.latch<latch>,%vector.ph,%vector.body<exiting>,%vector.body.interim,%middle.block
; CHECK-NEXT: Loop at depth 2 containing: %vector.body<header><exiting>,%vector.body.interim<latch><exiting>
@ -368,7 +368,7 @@ early.exit.3:
; has 3 early exits that ALL leave ALL loops. The exit routing blocks have
; multiple successors so connectToPredecessors places them in the vectorized
; loop. The fixup pass must move them all the way out (not just one level up).
define i32 @multi_early_exit_all_leave_three_nested_loops(i1 %c.1, i1 %c.2, ptr dereferenceable(1024) %src) {
define i32 @multi_early_exit_all_leave_three_nested_loops(i1 %c.1, i1 %c.2, ptr dereferenceable(1024) %src) nofree {
; CHECK-LABEL: Loop info for function 'multi_early_exit_all_leave_three_nested_loops':
; CHECK-NEXT: Loop at depth 1 containing: %outer.header<header>,%middle.header,%middle.latch,%outer.latch<latch>,%vector.ph,%vector.body<exiting>,%vector.body.interim,%middle.block
; CHECK-NEXT: Loop at depth 2 containing: %middle.header<header>,%middle.latch<latch><exiting>,%vector.ph,%vector.body<exiting>,%vector.body.interim,%middle.block
@ -419,7 +419,7 @@ early.exit.3:
; Tests that dispatch blocks are correctly placed in the outermost loop when
; the innermost loop has two uncountable early exits at different loop depths:
; one to the outer loop and one outside all loops.
define i64 @multi_early_exit_to_outer_and_outside(ptr dereferenceable(1024) %p1, ptr dereferenceable(1024) %p2, i1 %c) {
define i64 @multi_early_exit_to_outer_and_outside(ptr dereferenceable(1024) %p1, ptr dereferenceable(1024) %p2, i1 %c) nofree {
; CHECK-LABEL: Loop info for function 'multi_early_exit_to_outer_and_outside':
; CHECK-NEXT: Loop at depth 1 containing: %loop.outer<header>,%loop.middle,%loop.inner.end,%loop.middle.end,%loop.inner.found,%loop.outer.latch<latch><exiting>,%vector.ph,%vector.body,%vector.body.interim,%middle.block,%vector.early.exit.0,%vector.early.exit.check<exiting>
; CHECK-NEXT: Loop at depth 2 containing: %loop.middle<header>,%loop.inner.end<latch><exiting>,%vector.ph,%vector.body<exiting>,%vector.body.interim,%middle.block
@ -476,7 +476,7 @@ exit:
; Same as above but with early exits reversed: the first early exit goes outside
; all loops and the second goes to the outer loop.
define i64 @multi_early_exit_to_outside_and_outer(ptr dereferenceable(1024) %p1, ptr dereferenceable(1024) %p2, i1 %c) {
define i64 @multi_early_exit_to_outside_and_outer(ptr dereferenceable(1024) %p1, ptr dereferenceable(1024) %p2, i1 %c) nofree {
; CHECK-LABEL: Loop info for function 'multi_early_exit_to_outside_and_outer':
; CHECK-NEXT: Loop at depth 1 containing: %loop.outer<header>,%loop.middle,%loop.inner.end,%loop.middle.end,%loop.inner.found,%loop.outer.latch<latch><exiting>,%vector.ph,%vector.body,%vector.body.interim,%middle.block,%vector.early.exit.1,%vector.early.exit.check<exiting>
; CHECK-NEXT: Loop at depth 2 containing: %loop.middle<header>,%loop.inner.end<latch><exiting>,%vector.ph,%vector.body<exiting>,%vector.body.interim,%middle.block

4
llvm/test/Transforms/LoopVectorize/single_early_exit.ll
View file

@ -289,9 +289,9 @@ for.end:
ret i32 0
}
define void @inner_loop_trip_count_depends_on_outer_iv(ptr align 8 dereferenceable(1792) %this, ptr %dst) {
define void @inner_loop_trip_count_depends_on_outer_iv(ptr align 8 dereferenceable(1792) %this, ptr %dst) nofree {
; CHECK-LABEL: define void @inner_loop_trip_count_depends_on_outer_iv(
; CHECK-SAME: ptr align 8 dereferenceable(1792) [[THIS:%.*]], ptr [[DST:%.*]]) {
; CHECK-SAME: ptr align 8 dereferenceable(1792) [[THIS:%.*]], ptr [[DST:%.*]]) #[[ATTR0:[0-9]+]] {
; CHECK-NEXT: entry:
; CHECK-NEXT: [[GEP_SRC:%.*]] = getelementptr i8, ptr [[THIS]], i64 1000
; CHECK-NEXT: br label [[OUTER_HEADER:%.*]]

2
llvm/test/Transforms/LoopVectorize/single_early_exit_live_outs.ll
View file

@ -1,7 +1,7 @@
; NOTE: Assertions have been autogenerated by utils/update_test_checks.py UTC_ARGS: --version 4
; RUN: opt -S < %s -p loop-vectorize -force-vector-width=4 | FileCheck %s
declare void @init_mem(ptr, i64);
declare void @init_mem(ptr, i64) nofree
define i64 @same_exit_block_pre_inc_use1() {
; CHECK-LABEL: define i64 @same_exit_block_pre_inc_use1() {

2
llvm/test/Transforms/MergeICmps/X86/no-gep-other-work.ll
View file

@ -1,5 +1,5 @@
; NOTE: Assertions have been autogenerated by utils/update_test_checks.py
; RUN: opt -S -passes=mergeicmps < %s | FileCheck %s --check-prefixes=CHECK,GLOBAL
; RUN: opt -S -passes=mergeicmps -use-dereferenceable-at-point-semantics=0 < %s | FileCheck %s --check-prefixes=CHECK,GLOBAL
; RUN: opt -S -passes=mergeicmps -use-dereferenceable-at-point-semantics < %s | FileCheck %s --check-prefixes=CHECK,AT-POINT
; This does not use a GEP for the zero-offset comparison and requires a

17
llvm/test/Transforms/PhaseOrdering/AArch64/constraint-elimination-placement.ll
View file

@ -97,17 +97,18 @@ define void @test2(ptr %this) #0 {
; CHECK-SAME: ptr nofree writeonly captures(none) [[THIS:%.*]]) local_unnamed_addr #[[ATTR2:[0-9]+]] {
; CHECK-NEXT: entry:
; CHECK-NEXT: [[CALL1_I_I:%.*]] = tail call i1 @test2_fn4(i8 undef)
; CHECK-NEXT: br i1 [[CALL1_I_I]], label [[COND_TRUE_I_I:%.*]], label [[COMMON_RET:%.*]]
; CHECK: cond.true.i.i:
; CHECK-NEXT: [[CALL2_I_I:%.*]] = load i64, ptr inttoptr (i64 8 to ptr), align 8
; CHECK-NEXT: [[COND38:%.*]] = icmp eq i64 [[CALL2_I_I]], 13
; CHECK-NEXT: [[COND:%.*]] = select i1 [[CALL1_I_I]], i1 [[COND38]], i1 false
; CHECK-NEXT: br i1 [[COND]], label [[TEST2_FN2_EXIT12:%.*]], label [[COMMON_RET:%.*]]
; CHECK: test2_fn2.exit12:
; CHECK-NEXT: br i1 [[COND38]], label [[TEST2_FN2_EXIT14:%.*]], label [[COMMON_RET]]
; CHECK: test2_fn2.exit14:
; CHECK-NEXT: [[CALL8_I_I8:%.*]] = tail call fastcc noundef i32 @test2_fn6()
; CHECK-NEXT: [[CMP4_I11:%.*]] = icmp eq i32 [[CALL8_I_I8]], 0
; CHECK-NEXT: br i1 [[CMP4_I11]], label [[TEST2_FN2_EXIT24:%.*]], label [[COMMON_RET]]
; CHECK: common.ret:
; CHECK-NEXT: ret void
; CHECK: test2_fn2.exit24:
; CHECK: test2_fn2.exit28:
; CHECK-NEXT: store i8 0, ptr [[THIS]], align 4
; CHECK-NEXT: br label [[COMMON_RET]]
;
@ -136,8 +137,12 @@ define i1 @test2_fn2(ptr %__rhs) #0 {
; CHECK-NEXT: entry:
; CHECK-NEXT: [[CALL:%.*]] = tail call i64 @strlen(ptr noundef nonnull dereferenceable(1) [[__RHS]])
; CHECK-NEXT: [[CALL1_I:%.*]] = tail call i1 @test2_fn4(i8 undef)
; CHECK-NEXT: br i1 [[CALL1_I]], label [[COND_TRUE_I:%.*]], label [[ENTRY:%.*]]
; CHECK: cond.true.i:
; CHECK-NEXT: [[CALL2_I:%.*]] = load i64, ptr inttoptr (i64 8 to ptr), align 8
; CHECK-NEXT: [[COND_I:%.*]] = select i1 [[CALL1_I]], i64 [[CALL2_I]], i64 0
; CHECK-NEXT: br label [[ENTRY]]
; CHECK: test2_fn3.exit:
; CHECK-NEXT: [[COND_I:%.*]] = phi i64 [ [[CALL2_I]], [[COND_TRUE_I]] ], [ 0, [[ENTRY1:%.*]] ]
; CHECK-NEXT: [[CMP2_NOT:%.*]] = icmp eq i64 [[CALL]], [[COND_I]]
; CHECK-NEXT: br i1 [[CMP2_NOT]], label [[IF_END:%.*]], label [[CLEANUP:%.*]]
; CHECK: if.end:
@ -145,7 +150,7 @@ define i1 @test2_fn2(ptr %__rhs) #0 {
; CHECK-NEXT: [[CMP4:%.*]] = icmp eq i32 [[CALL8_I]], 0
; CHECK-NEXT: br label [[CLEANUP]]
; CHECK: cleanup:
; CHECK-NEXT: [[RETVAL_0:%.*]] = phi i1 [ [[CMP4]], [[IF_END]] ], [ false, [[ENTRY:%.*]] ]
; CHECK-NEXT: [[RETVAL_0:%.*]] = phi i1 [ [[CMP4]], [[IF_END]] ], [ false, [[ENTRY]] ]
; CHECK-NEXT: ret i1 [[RETVAL_0]]
;
entry:

14
llvm/test/Transforms/SimplifyCFG/fold-branch-to-common-dest.ll
View file

@ -1295,17 +1295,15 @@ define void @load_derefable_two_preds(i8 %v0, i8 %v1, i8 %v2, ptr dereferenceabl
; CHECK-NEXT: [[C0:%.*]] = icmp eq i8 [[V0:%.*]], 0
; CHECK-NEXT: br i1 [[C0]], label [[PRED0:%.*]], label [[PRED1:%.*]]
; CHECK: pred0:
; CHECK-NEXT: [[C1:%.*]] = icmp eq i8 [[V1:%.*]], 0
; CHECK-NEXT: [[V3_OLD:%.*]] = load i8, ptr [[P:%.*]], align 1
; CHECK-NEXT: [[C3_OLD:%.*]] = icmp eq i8 [[V3_OLD]], 0
; CHECK-NEXT: [[OR_COND1:%.*]] = select i1 [[C1]], i1 true, i1 [[C3_OLD]]
; CHECK-NEXT: br i1 [[OR_COND1]], label [[FINAL_LEFT:%.*]], label [[FINAL_RIGHT:%.*]]
; CHECK-NEXT: [[C3_OLD:%.*]] = icmp eq i8 [[V3_OLD:%.*]], 0
; CHECK-NEXT: br i1 [[C3_OLD]], label [[FINAL_LEFT:%.*]], label [[DISPATCH:%.*]]
; CHECK: pred1:
; CHECK-NEXT: [[C2:%.*]] = icmp eq i8 [[V2:%.*]], 0
; CHECK-NEXT: [[V3:%.*]] = load i8, ptr [[P]], align 1
; CHECK-NEXT: br i1 [[C2]], label [[DISPATCH]], label [[FINAL_RIGHT:%.*]]
; CHECK: dispatch:
; CHECK-NEXT: [[V3:%.*]] = load i8, ptr [[P:%.*]], align 1
; CHECK-NEXT: [[C3:%.*]] = icmp eq i8 [[V3]], 0
; CHECK-NEXT: [[OR_COND:%.*]] = select i1 [[C2]], i1 [[C3]], i1 false
; CHECK-NEXT: br i1 [[OR_COND]], label [[FINAL_LEFT]], label [[FINAL_RIGHT]]
; CHECK-NEXT: br i1 [[C3]], label [[FINAL_LEFT]], label [[FINAL_RIGHT]]
; CHECK: common.ret:
; CHECK-NEXT: ret void
; CHECK: final_left:

2
polly/test/ScopInfo/invariant_load_dereferenceable.ll
View file

@ -16,7 +16,7 @@
; CHECK-NOT: Function: foo_undereferanceable
define void @foo_dereferanceable(ptr %A, ptr %B, ptr dereferenceable(8) align 8 %sizeA_ptr,
i32 %lb.i, i32 %lb.j, i32 %ub.i, i32 %ub.j) {
i32 %lb.i, i32 %lb.j, i32 %ub.i, i32 %ub.j) nofree {
entry:
br label %for.i