Commit graph

385 commits

Author SHA1 Message Date
Nikolas Klauser
090b6569b6
[libc++] Make __has_array_cookie a variable template (#212767)
Using variable templates is slightly faster to compile and more
readable, so we might as well use them.
2026-07-31 13:24:04 +02:00
Louis Dionne
8ea9597ccb
[libc++] Defer removal of two escape hatches to LLVM 25 (#211553)
Rather than removing the escape hatches for trivially default
constructible allocator and bitset's const reference type in LLVM 24,
keep them around for another release since they have not been widely
deployed yet.
2026-07-28 12:47:43 -04:00
Avi Patel
1b3d208c77
[libc++] Implement P1901R2: owner_hash and owner_equal for shared_ptr/weak_ptr (#210551)
Fixes #105372.

## Acknowledgements

Claude assisted with test development.
2026-07-27 18:00:45 +08:00
Emmett
eb5bc2694a
[libc++][unique_ptr] Implement LWG 4144: Disallow unique_ptr<T&, D> (#209018)
Implement [LWG 4144](https://cplusplus.github.io/LWG/issue4144) by
explicitly rejecting `unique_ptr<T, D>` when `T*` is not a valid type.

This adds a pointability check and tests reference and abominable
function types.

Fixes #118359
2026-07-13 21:39:06 +03:00
Nikolas Klauser
62bf9d924c
[libc++] Fix shared_ptr not accepting allocators with explicit conversions (#208439)
Allocators are only required to be explicitly convertible between
different types. `shared_ptr` currently requires implicit conversions,
however.

This is a regression introduced by #200401.
2026-07-10 15:31:02 +02:00
inquisitivecrystal
b5fa9eee67
[libc++] Implement LWG3918: copy elision in std::uninitialized_move/_n (#207692)
This implements [LWG3918](https://wg21.link/LWG3918), which guarantees
copy elision for rvalues in `std::uninitialized_move/_n`. It also
implements [LWG4452](https://wg21.link/LWG4452), a minor correction that
makes the helper added by LWG3918 constexpr.

This additionally fixes a bug in `std::uninitialized_move/_n` where they
could create and then access dangling references. The previous
implementation used the following lambda as an implementation detail:
```c++
[](auto&& __iter) -> decltype(auto) { return std::move(*__iter); }
```
When `__iter` is a prvalue, this creates a temporary object within the
lambda's body and `std::move` then returns a reference to that object.
The reference dangles as soon as control leaves the lambda. This
behavior was not permitted by the standard even before LWG3918.

The new implementation fixes the bug by using a helper,
`std::__deref_move(__iter)`, which returns `*__iter` directly when
`*__iter` is already an rvalue.

Resolves #118339.
Resolves #171416.
2026-07-08 20:06:57 +08:00
Nikolas Klauser
1b2efd2e07
[libc++] Split the multidimensional algorithms out of uninitialized_algorithms.h (#207447)
While these algorithms are conceptually related to other
`uninitialized_*` algorithms, they're likely only ever going to be used
by `shared_ptr`. `uninitialized_algorithms.h` itself is also getting
rather large, so it makes sense to split the multidimensional versions
into their own header.

Fixes #207417
2026-07-06 09:28:56 +02:00
A. Jiang
bb6868abb8
[libc++] Fix behavior for get_temporary_buffer with non-positive size (#206871)
Per C++17 [depr.temporary.buffer]/4, `get_temporary_buffer` is required
to return `{nullptr, 0}` when the size argument is zero or negative.
libc++ used to correctly handle this, but the refactoring in
94e7c0b051 got this wrong.

GCC generally warns on negative size due to `-Walloc-size-larger-than=`,
which is false positive due to incomplete control flow analysis. The
warning is coupled with optimizations, and this patch make tests
suppress it instead.
2026-07-01 22:29:19 +08:00
Nikolas Klauser
ad2fbea797
[libc++] Consistently use version guards outside namespace macros (#181136)
This has the benefit of consistent placement for these macros (outside
the namespace, not inside it). This, in turn, makes it more obvious that
the condition applies to the entire namespace scope. Compile times are
also reduced a bit, since the compiler doesn't have to open a namespace
just to close it again.

This is enforced with a clang-tidy check.
As a drive-by this also fixes a few macro comments.
2026-06-29 14:44:44 +02:00
Nikolas Klauser
aab67db385
[libc++] Fix shared_ptr rebinding allocators to incomplete types (#206145)
https://github.com/llvm/llvm-project/pull/200401 caused various ways to
create a `shared_ptr` control block to reject any allocators which
required a complete type. This patch allows allocators to again require
a complete type.
2026-06-27 08:12:19 +02:00
Nikolas Klauser
23e960431d
[libc++] Fix move-only types in std::make_shared (#205819)
https://github.com/llvm/llvm-project/pull/200401 broke `make_shared` and
friends support for move-only types.
2026-06-25 21:39:13 +02:00
A. Jiang
755bc5c9a5
[libc++] Add missing attribute usages to <__memory/shared_ptr.h> (#205776)
Since 44546e0e32, lack of
`_LIBCPP_NODEBUG` and `_LIBCPP_HIDE_FROM_ABI` are caught by clang-tidy.
This patch adds them wherever expected.
2026-06-25 16:13:28 +02:00
Nikolas Klauser
7083c06372
[libc++] Remove _LIBCPP_COMPRESSED_PAIR/TRIPLE from shared_ptr (#200401)
The `shared_ptr` control blocks don't need to be ABI stable as long as
we rename them. This can be thought of as adding a new control block,
which we've done before, and simply never generating the old one again.
This is done via an ABI tag on the control blocks, allowing us to change
their contents in whichever way we want without needing to consider
their ABI. In this case specifically, `_LIBCPP_COMPRESSED_{PAIR,TRIPLE}`
is removed, simplifying the code.
2026-06-25 11:45:55 +02:00
Nikolas Klauser
44546e0e32
[libc++] Enable clang-tidy in the CI again (#195047)
clang-tidy currently doesn't run in the CI because CMake doesn't find
the appropriate `ClangConfig.cmake` anymore. This adds the path so
clang-tidy is built again and fixes most issues. Some things are
disabled for now to get most checks back. They will be re-enabled later.
2026-06-25 09:11:33 +02:00
A. Jiang
44cc7971ff
[libc++] P3369R0: constexpr for uninitialized_default_construct (#200163)
Remarks:
- Tests also verify that `uninitialized_default_construct(_n)`
algorithms do not initialize trivially default-constructible elements
(`int` in these tests) to determined values during constant evaluation.
2026-06-13 09:56:56 +08:00
Nikolas Klauser
3e2ccf5a4d
[libc++] Assume that <atomic> is available (#199674)
We always define either `_LIBCPP_HAS_C_ATOMIC_IMP` or
`_LIBCPP_HAS_GCC_ATOMIC_IMP`, so we can remove any special handling of
not having an `<atomic>` header.
2026-06-06 04:34:31 +02:00
Nikolas Klauser
c88cefb9f1
[libc++] Simplify unique_ptr constructor SFINAE (#201305)
This patch does a couple of things:
- inline aliases to `__enable_if_t`s, making it easier to understand
what's actually going on
- make the `enable_if`s dependent via a `class _Deleter = deleter_type`
instead of a `bool` and `__dependent_type`, reducing the number of
instantiated classes
- remove `__unique_ptr_deleter_sfinae`
2026-06-04 10:41:46 +02:00
Nikolas Klauser
f9207d7c4b
[libc++][NFC] Move functions in shared_ptr.h into the class body (#200343)
This removes some boilerplate code and makes it more consistent with
most of the rest of the code base.
2026-05-29 15:43:39 +02:00
Nikolas Klauser
9d47197593
[libc++] Use variable templates instead of class templates in shared_ptr.h (#199481)
This improves readability and compile times a bit.
2026-05-29 09:52:29 +02:00
William Tran-Viet
4306b4ab9c
[libc++] Implement P3508R0: Wording for "constexpr for specialized memory algorithms" (#197313)
Resolves #118379

- Add `constexpr` to affected areas
- Add `constexpr` tests
- Update documentation

Tests were created with AI assistance.
2026-05-28 16:02:21 +08:00
Nikolas Klauser
a917c00ab1
[libc++] Use __in_out_result in the remaining appropriate algorithms (#198156)
We've already switched some algorithms to use `__in_out_result` instead
of `pair` as the return type. This updates the remaining appropriate
algorithms.
2026-05-26 19:06:01 +02:00
Nikolas Klauser
d06febc3a5
[libc++] Fix shared_ptr(Y*) constraint check on GCC (#199429)
GCC has fixed SFINAEing on `delete` since at least GCC 11, so we can
properly constrain the constructor.
2026-05-25 07:10:15 +02:00
Yuxuan Chen
859c550e95
[libc++] Require the exact assignment expression to be trivial in __uninitialized_allocator_copy_impl (#196648)
Fixes https://github.com/llvm/llvm-project/issues/196645

`__uninitialized_allocator_copy_impl` has an optimization that replaces
`allocator_traits::construct` with `std::copy` for raw pointer ranges
when the element type is trivially copy constructible and trivially copy
assignable.

The copy-assignment trait only checks whether assignment from `const T&`
is trivial. That is weaker than the expression used by `std::copy`,
which evaluates `*out = *in`. If overload resolution selects a different
non-trivial assignment operator for that expression, `std::copy` can
call that operator on uninitialized storage.

Check `is_trivially_assignable<_Out&, _In&>` instead. This matches the
assignment expression used by `std::copy`, preserves the optimized path
when that assignment is actually trivial, and avoids making non-const
raw pointer callers select the generic `allocator_traits::construct`
overload due to a qualification conversion.

Add a vector copy-constructor regression test with a type whose
defaulted copy assignment is trivial but whose templated assignment
operator is selected for non-const lvalue sources.
2026-05-20 21:37:35 +02:00
Nikolas Klauser
013b87ee4a
[libc++] Simplify the implementation of to_address a bit (#198256) 2026-05-20 17:23:18 +02:00
A. Jiang
2caea408ab
[libc++] LWG4324: unique_ptr<void>::operator* is not SFINAE-friendly (#190919)
---------

Co-authored-by: Hristo Hristov <zingam@outlook.com>
2026-05-09 23:53:26 +08:00
Nikolas Klauser
2e2d90b986
[libc++] Introduce implicit and explicit ABI annotations (#193045)
This patch introduces `_LIBCPP_{BEGIN/END}_EXPLICIT_ABI_ANNOTATIONS` and
marks everything within an
`_LIBCPP_{BEGIN,END}_UNVERSIONED_NAMESPACE_STD` (and any derivatives
like `_LIBCPP_{BEGIN,END}_NAMESPACE_STD`) implicitly by default. This
allows us to drop `_LIBCPP_HIDE_FROM_ABI` in most of the code base,
except for functions which shouldn't be `_LIBCPP_HIDE_FROM_ABI`.

This patch doesn't remove any `_LIBCPP_HIDE_FROM_ABI`s, since we have
over 13k of them in the code base. Actually dropping them will happen
over some time to avoid too many merge conflicts.
2026-05-08 12:21:31 +02:00
A. Jiang
470ba560b7
[libc++] Properly calculate rounded-up size for {allocate,make}_shared (#190315)
Previously, the allocated size might be a bit too small for certain
cases. This patch makes the size calculated with potentially larger
alignment of the whole control block.
2026-04-08 17:49:04 +08:00
Michael Levine
46c79a0c1f
[libc++] Use internal std::__is_sufficiently_aligned to check requirement in atomic ref constructors (#188530)
This PR introduces a `std::__is_sufficiently_aligned` helper function that
can be used for the `atomic_ref` C++20 constructors while the public
C++26 `std::is_sufficiently_aligned` delegates to the new helper
function.

Fixes #179803

Assisted-by: Claude Code

Signed-off-by: mlevine55 <mlevine55@bloomberg.net>
2026-03-26 14:38:24 -04:00
Nikolas Klauser
c9863ceacc
[libc++] Specialize allocator_traits for std::allocator (#160278)
This reduces the time to instantiate `allocator_traits<allocator<T>>`.
2026-02-05 11:43:47 +01:00
Nikolas Klauser
dd7348420d
[libc++][NFC] Use forward declarations in {in,}out_ptr.h (#176341)
This doesn't improve compile times currently, but it does make it clear
that we don't need the definition of `shared_ptr` or `unique_ptr` for
`{in,}out_ptr`. It also reduces compile times quite a bit if we ever use
these utilities in another header.
2026-01-19 15:13:47 -05:00
Nikolas Klauser
4304106dc2
Reapply "[libc++] Optimize std::find_if" (#175903) (#175921)
#175913 removed that `__builtin_assume_dereferenceable(ptr, 0)` implies
`ptr != nullptr`, which should allow us to use the builtin with LLVM 23.

This reverts commit 776c09c212.
2026-01-15 10:16:45 +01:00
Nikolas Klauser
776c09c212
Revert "[libc++] Optimize std::find_if" (#175903)
`__builtin_assume_dereferenceable` currently implies that the pointer
given to it is non-null, even if the size is zero. This causes
miscompilations, since we consider [nullptr, nullptr) to be a valid
range.

Reverts llvm/llvm-project#167697
2026-01-14 11:15:21 +01:00
Nikolas Klauser
6189512f73
[libc++] Optimize std::find_if (#167697)
```
Benchmark                       4ecfaa602f    80d5ac247d34    Difference    % Difference
----------------------------  --------------  --------------  ------------  --------------
bm_find_if_autovectorization         1901.51          306.12      -1595.39         -83.90%
```
2026-01-13 11:13:54 +01:00
Hristo Hristov
e593dbc184
[libc++][memory] Applied [[nodiscard]] to more functions (#172131)
`[[nodiscard]]` should be applied to functions where discarding the
return value is most likely a correctness issue.

- https://libcxx.llvm.org/CodingGuidelines.html
- https://wg21.link/allocator.traits
- https://wg21.link/specialized.addressof
- https://wg21.link/ptr.align
- https://timsong-cpp.github.io/cppwp/n4659/depr.default.allocator
- https://timsong-cpp.github.io/cppwp/n4659/depr.storage.iterator

Towards #172124
2026-01-08 12:05:43 +02:00
saipubw
1c06165c9b
[libc++] Make std::align an inline function (#167472)
`std::align` is heavily used in memory allocators. When we attempted to
switch from libstdc++ to libc++, we observed a **50%** performance
regression in a database query bench: the issue is that `std::align` in
libc++ is not an inline function, which prevents the compiler from
performing inlining optimizations.

make `std::align` an inline function will run about 2x faster. See
[benchmark
result](https://quick-bench.com/q/wPTnt9JCGn2S-3bu5gY9YrEf6KU).

---------

Co-authored-by: Louis Dionne <ldionne.2@gmail.com>
2025-12-18 17:55:28 +08:00
Nikolas Klauser
df6c27e752
[libc++] Make std::allocator always trivially default constructible (#169914)
This is technically ABI breaking, since `is_trivial` and
`is_trivially_default_constructible` now return different results.
However, I don't think that's a significant issue, since `allocator` is
almost always used in classes which own memory, making them non-trivial
anyways.
2025-12-11 10:27:43 +01:00
Hristo Hristov
4125e73cdc
[libc++] Applied [[nodiscard]] to hash<shared_ptr>, hash<unique_ptr>, etc. (#170674)
`[[nodiscard]]` should be applied to functions where discarding the
return value is most likely a correctness issue.

- https://libcxx.llvm.org/CodingGuidelines.html


1. `hash<shared_ptr>`, `hash<unique_ptr>`, `std::integer_sequence<>`
etc.
2. Also implements fixes to
https://github.com/llvm/llvm-project/issues/169634 on the go (issues
discovered during current implementation)

---------

Co-authored-by: A. Jiang <de34@live.cn>
Co-authored-by: Hristo Hristov <zingam@outlook.com>
2025-12-08 06:56:33 +02:00
Nikolas Klauser
3dcdb4c765
[libc++][NFC] Move __memory/aligned_alloc.h into src/ (#166172)
This header is only ever used inside `src/`, so we might as well move it
there. As a drive-by this also removes some dead code.
2025-11-24 22:36:13 +01:00
Hristo Hristov
3f151a3fa6
[libc++][memory] Applied [[nodiscard]] to smart pointers (#168483)
Applied `[[nodiscard]]` where relevant to smart pointers and related
functions.

- [x] - `std::unique_ptr`
- [x] - `std::shared_ptr`
- [x] - `std::weak_ptr`

See guidelines:
-
https://libcxx.llvm.org/CodingGuidelines.html#apply-nodiscard-where-relevant
- `[[nodiscard]]` should be applied to functions where discarding the
return value is most likely a correctness issue. For example a locking
constructor in unique_lock.

---------

Co-authored-by: Hristo Hristov <zingam@outlook.com>
2025-11-20 04:19:15 +02:00
Hristo Hristov
9036e2359f
[libc++] Apply [[nodiscard]] to in/out_ptr (#167097)
...according to Coding Guidelines: `[[nodiscard]]` should be applied to
functions where discarding the return value is most likely a correctness
issue.

Changes to:
  - [x] `inout_ptr()` 
  - [x] `out_ptr()` 


At the time of impelentation the `[[nodiscard]]` policy has not been
established yet.

---------

Co-authored-by: Hristo Hristov <zingam@outlook.com>
2025-11-15 03:00:04 +02:00
Nikolas Klauser
f038dfd22d
[libc++] Merge is_{,un}bounded_array.h into is_array.h (#167479)
These headers are incredibly simple and closely related, so this merges
them into a single one.
2025-11-13 09:29:28 +01:00
Nikolas Klauser
2ac9e59d97
[libc++] Simplify the implementation of the unique_ptr -> shared_ptr converting constructor (#165619)
This also backports LWG2415 as a drive-by.
2025-11-13 09:09:44 +01:00
Louis Dionne
17ce48f2b6
[libc++] Remove __is_replaceable emulation (#167355)
The Trivial Relocation feature has been removed from the C++26 working
draft. Based on discussions in Kona, it is unlikely that the
"replaceable" type concept will come back in the C++29 time frame.

Since we don't have a use for the type trait in the library at the
moment, remove the code associated to it. If we end up needing something
like it in the future, we can always add it back.
2025-11-11 07:53:56 -05:00
Nikolas Klauser
c0bca9cb84
[libc++] Remove some of the uses of aligned_storage inside the library (#161635)
`aligned_storage` has been deprecated and will most likely be removed in
a future version of C++. This patch removes some of its uses to avoid
having to work around its removal in the future.
2025-11-11 11:03:02 +01:00
Nikolas Klauser
471dbb90cb
[libc++] Replace __libcpp_is_final with a variable template (#167137) 2025-11-10 11:00:05 +01:00
Nikolas Klauser
9f9b8d56f4
[libc++] Simplify the implementation of destroy_at a bit (#165392) 2025-10-31 12:56:37 +01:00
Louis Dionne
b3a199469c
[libc++] Properly implement array cookies in the ARM ABI (#160182)
When we implemented array cookie support for hardening std::unique_ptr,
the implementation was only done for the Itanium ABI. I did not
initially realize that ARM was using a different ABI for array cookies,
so unique_ptr should not have been hardened on ARM.

However, we were also incorrectly setting the ABI-detection macro: we
were pretending to be using a vanilla Itanium ABI when in reality the
(similar but different) ARM ABI was in use. As a result, unique_ptr was
using the wrong representation for array cookies on ARM, which
fortunately only mattered in the case of overaligned types.

This patch fixes that.

rdar://160852193
2025-10-16 23:40:58 -04:00
Nikolas Klauser
446d950524
[libc++] Use std::__{scope,exception}_guard throughout the code base (#161322)
This simplifies the code quite a bit and seems to improve code size
slightly in some cases.
2025-10-09 09:32:40 +02:00
Nikolas Klauser
cc95c5680e
[libc++] Remove alignment attributes from _LIBCPP_COMPRESSED_PAIR (#158595)
These attributes aren't required anymore, since the potential
overalignemnt of the objects is handled by the surrounding anonymous
struct.
2025-09-25 19:59:08 +02:00
Nikolas Klauser
b5bb50d09a
[libc++] Simplify __memory/shared_count.h a bit (#160048)
This removes a few checks that aren't required anymore and moves some
code around to the places where it's actually used.
2025-09-24 10:40:36 +02:00