This PR fixes a bug in `std::mersenne_twister_engine::seed(_Sseq&)`
where the initialization twist is bypassed when the vectorized engine
optimization (`_LIBCPP_ABI_VECTORIZED_MERSENNE_TWISTER_ENGINE`) is
enabled.
In the vectorized implementation introduced in #206423, the engine state
must be pre-twisted during initialization by calling
`__update_all_states()` at the end of the `seed` functions.
However, in `seed(_Sseq& __q)`, the all-zero state check contains an
early `return;` when a non-zero state is verified. This early return
bypasses the trailing `__update_all_states()` call. Consequently, the
generator is initialized with raw, untwisted seed values (tempered)
rather than the standard twisted values.
This only manifests when compiled with unstable ABI settings
(`_LIBCPP_ABI_VERSION >= 2`) which enables the vectorized
implementation.
I replaced the early `return;` inside the all-zero state verification
block with a loop break, and conditionalized the zero-state fix using
the loop index `__i`, ensuring `__update_all_states()` is always
executed at the end of the function. Added regression tests to
`seed_sseq.pass.cpp` asserting the exact deterministic outputs for both
32-bit and 64-bit engines.
Instead of updating the status array every time `operator()` is called,
we can instead update it once every time the entire array has been read.
That allows the compiler to vectorize the updating code, improving
performance.
Apple M4:
```
Benchmark old new Difference % Difference
---------------------------------------- ------- ------- ------------ --------------
std::mt19937::operator() 1.39 0.65 -0.74 -53.26%
std::mt19937::operator()_(into_array) 172.92 80.69 -92.23 -53.34%
std::mt19937_64::operator() 1.46 0.99 -0.47 -31.99%
std::mt19937_64::operator()_(into_array) 187.31 108.10 -79.21 -42.29%
```
Fixes#197221
This has the benefit of consistent placement for these macros (outside
the namespace, not inside it). This, in turn, makes it more obvious that
the condition applies to the entire namespace scope. Compile times are
also reduced a bit, since the compiler doesn't have to open a namespace
just to close it again.
This is enforced with a clang-tidy check.
As a drive-by this also fixes a few macro comments.
Instead of pushing and popping warnings we can just mark the offending
member as `[[maybe_unused]]`, improving compile times a bit and
simplifying the code.
This patch introduces `_LIBCPP_{BEGIN/END}_EXPLICIT_ABI_ANNOTATIONS` and
marks everything within an
`_LIBCPP_{BEGIN,END}_UNVERSIONED_NAMESPACE_STD` (and any derivatives
like `_LIBCPP_{BEGIN,END}_NAMESPACE_STD`) implicitly by default. This
allows us to drop `_LIBCPP_HIDE_FROM_ABI` in most of the code base,
except for functions which shouldn't be `_LIBCPP_HIDE_FROM_ABI`.
This patch doesn't remove any `_LIBCPP_HIDE_FROM_ABI`s, since we have
over 13k of them in the code base. Actually dropping them will happen
over some time to avoid too many merge conflicts.
Libc++ currently redeclares ::lgamma_r on platforms that provide it.
This causes issues when building with modules, and redeclaring functions
provided by another library (here the C library) is bad hygiene.
Instead, use an asm declaration to call the right function without
having to redeclare it.
Several components in libc++ aren't defending against overloaded
`operator,(T, Iter)` currently. Existing deleted overloads in
`test_iterators.h` are insufficient for such cases.
This PR adds corresponding deleted overloads with reversed order and
fixes these libc++ components.
- `piecewise_linear_distribution`'s iterator pair constructor,
- `piecewise_linear_distribution::param_type`'s iterator pair
constructor,
- `piecewise_constant_distribution`'s iterator pair constructor,
- `piecewise_constant_distribution::param_type`'s iterator pair
constructor,
- `money_get::do_get`,
- `money_put::do_put`, and
- `num_put::do_put`.
An older PR https://github.com/llvm/llvm-project/pull/102036 suggested
that LLVM libc declares `lgamma_r` as noexcept and is incompatible with
this redeclaration. However, I recently discovered that glibc also
declares the math functions to be noexcept under C++ mode.
This line usually don't cause issues because both the glibc and this
file are included as "system headers". According to [this
godbolt](https://godbolt.org/z/o7Wd9PP58), both GCC and clang ignore the
different exception specification between multiple declarations if they
are in system headers.
However, this seems not the case for NVCC/EDG, so a fix for this
redeclaration is still desirable. This patch proposes that we should
declare the function as noexcept under known libc integrations to keep
the declared function consistent.
This reverts commit c861fe8a71.
Unfortunately, this use of hidden visibility attributes causes
user-defined specializations of standard-library types to also be marked
hidden by default, which is incorrect. See discussion thread on #131156.
...and also reverts the follow-up commits:
Revert "[libc++] Add explicit ABI annotations to functions from the block runtime declared in <__functional/function.h> (#140592)"
This reverts commit 3e4c9dc299.
Revert "[libc++] Make ABI annotations explicit for windows-specific code (#140507)"
This reverts commit f73287e623.
Revert "[libc++][NFC] Replace a few "namespace std" with the correct macro (#140510)"
This reverts commit 1d411f27c7.
This patch introduces `_LIBCPP_{BEGIN,END}_EXPLICIT_ABI_ANNOTATIONS`,
which allow us to have implicit annotations for most functions, and just
where it's not "hide_from_abi everything" we add explicit annotations.
This allows us to drop the `_LIBCPP_HIDE_FROM_ABI` macro from most
functions in libc++.
This is technically not necessary in most cases to prevent issues with ADL,
but let's be consistent. This allows us to remove the libcpp-qualify-declval
clang-tidy check, which is now enforced by the robust-against-adl clang-tidy check.
The variables are all `constexpr`, which implies `inline`. Since they
aren't `constexpr` in C++03 they're also not `inline` there. Because of
that we define them out-of-line currently. Instead we can use the C++17
extension of `inline` variables, which results in the same weak
definitions of the variables but without having all the boilerplate.
Currently, the library-internal feature test macros are only defined if
the feature is not available, and always have the prefix
`_LIBCPP_HAS_NO_`. This patch changes that, so that they are always
defined and have the prefix `_LIBCPP_HAS_` instead. This changes the
canonical use of these macros to `#if _LIBCPP_HAS_FEATURE`, which means
that using an undefined macro (e.g. due to a missing include) is
diagnosed now. While this is rather unlikely currently, a similar change
in `<__configuration/availability.h>` caught a few bugs. This also
improves readability, since it removes the double-negation of `#ifndef
_LIBCPP_HAS_NO_FEATURE`.
The current patch only touches the macros defined in `<__config>`. If
people are happy with this approach, I'll make a follow-up PR to also
change the macros defined in `<__config_site>`.
This patch adds a large number of missing includes in the libc++ headers
and the test suite. Those were found as part of the effort to move
towards a mostly monolithic top-level std module.
Many headers include `<cstddef>` just for size_t, and pulling in
additional content (e.g. the traits used for std::byte) is unnecessary.
To solve this problem, this patch splits up `<cstddef>` into
subcomponents so that headers can include only the parts that they
actually require.
This has the added benefit of making the modules build a lot stricter
with respect to IWYU, and also providing a canonical location where we
define `std::size_t` and friends (which were previously defined in
multiple headers like `<cstddef>` and `<ctime>`).
After this patch, there's still many places in the codebase where we
include `<cstddef>` when `<__cstddef/size_t.h>` would be sufficient.
This patch focuses on removing `<cstddef>` includes from __type_traits
to make these headers non-circular with `<cstddef>`. Additional
refactorings can be tackled separately.
We use lgamma_r for the random normal distribution support. In this
code we redeclare it, which causes issues with the LLVM C library as
this function is marked noexcept in LLVM libc. This patch ensures that
we don't redeclare that function when targeting LLVM libc.
This PR is a followup to #81080.
This PR makes two major changes to how the LCG operation is computed:
The first is that I added an additional case where `ax + c` might
overflow the intermediate variable, but `ax` by itself won't. In this
case, it's much better to use `(ax mod m) + c mod m` than the previous
behavior of falling back to Schrage's algorithm. The addition modulo is
done in the same way as when using Schrage's algorithm (i.e. `x += c -
(x >= m - c)*m`), but the multiplication modulo is calculated directly,
which is faster.
The second is that I added handling for the case where the `ax`
intermediate might overflow, but Schrage's algorithm doesn't apply (i.e.
r > q). In this case, the only real option is to increase the precision
of the intermediate values. The good news is that - for `x`, `a`, and
`c` being n-bit values - `ax + c` will never overflow a 2n-bit
intermediary, meaning this promotion can only happen once, and will
always be able to use the simplest implementation. This is already the
case for 16-bit LCGs, as libcxx chooses to compute them with 32-bit
intermediate values. For 32-bit LCGs, I simply added code similar to the
16-bit case to use the existing 64-bit implementations. Lastly, for
64-bit LCGs, I wrote a case that calculates it using `unsigned __int128`
if it is available to use.
While this implementation covers a *lot* of the missing cases from
#81080, this still won't compile **every** possible
`linear_congruential_engine`. Specifically, if `a`, `c`, and `m` are
chosen such that it needs 128-bit integers, but the platform doesn't
support `__int128` (eg. 32-bit x86), then it will fail to compile.
However, this is a fairly rare case to see actually used, and libcxx
would be in good company with this, as [libstdc++ also fails to compile
under these
circumstances](https://gcc.gnu.org/bugzilla/show_bug.cgi?id=87744).
Fixing **this** gap would require even **more** work of further
complexity, so that would probably be best handled by a different PR
(I'll put more details on what that PR would entail in a comment).
Originally, we used __libcpp_verbose_abort to handle assertion failures.
That function was declared from all public headers. Since we don't use
that mechanism anymore, we don't need to declare __libcpp_verbose_abort
from all public headers, and we can clean up a lot of unnecessary
includes.
This patch also moves the definition of the various assertion categories
to the <__assert> header, since we now rely on regular IWYU for these
assertion macros.
rdar://105510916
This fixes two major mistakes in the implementation of
`linear_congruential_engine` that allowed it to produce incorrect
output. Specifically, these mistakes are in `__lce_alg_picker`, which is
used to determine whether Schrage's algorithm is valid and needed.
The first mistake is in the definition of `_OverflowOK`. The code
comment and the description of [D65041](https://reviews.llvm.org/D65041)
both indicate that it's supposed to be true iff `m` is a power of two.
However, the definition used does not work out to that, and instead is
true whenever `m` is even. This could result in
`linear_congruential_engine` using an invalid implementation, as it
would incorrectly assume that any integer overflow can't change the
result. I changed the implementation to one that accurately checks if
`m` is a power of two. Technically, this implementation has an edge case
where it considers `0` to be a power of two, but in this case this is
actually accurate behavior, as `m = 0` indicates a modulus of 2^w where
w is the size of `result_type` in bits, which *is* a power of two.
The second mistake is in the static assert. The original static assert
erroneously included an unnecessary `a != 0 || m != 0`. Combined with
the `|| !_MightOverflow`, this actually resulted in the static assert
being impossible to fail. Applying De Morgan's law and expanding
`_MightOverflow` gives that the only way this static assert can be
triggered is if `a == 0 && m == 0 && a != 0 && m != 0 && ...`, which
clearly cannot be true. I simply removed the explicit checks against `a`
and `m`, as the intended checks are already included in `_MightOverflow`
and `_SchrageOK`, and their inclusion doesn't provide any obvious
semantic benefit.
This should fix all the current instances where
`linear_congruential_engine` uses an invalid implementation. This
technically isn't a complete implementation, though, since the static
assert will cause some instantiations of `linear_congruential_engine`
not disallowed by the standard from compiling. However, this should
still be an improvement, as all compiling instantiations of
`linear_congruential_engine` should use a valid implementation. Fixing
the cases where the static assert triggers will require adding
additional implementations, some of which will be fairly non-trivial, so
I'd rather leave those for another PR so they don't hold up these more
important fixes.
Fixes#33554
Also introduce `_LIBCPP_ASSERT_PEDANTIC` for assertions violating which
results in a no-op or other benign behavior, but which may nevertheless
indicate a bug in the invoking code.
This patch runs clang-format on all of libcxx/include and libcxx/src, in
accordance with the RFC discussed at [1]. Follow-up patches will format
the benchmarks, the test suite and remaining parts of the code. I'm
splitting this one into its own patch so the diff is a bit easier to
review.
This patch was generated with:
find libcxx/include libcxx/src -type f \
| grep -v 'module.modulemap.in' \
| grep -v 'CMakeLists.txt' \
| grep -v 'README.txt' \
| grep -v 'libcxx.imp' \
| grep -v '__config_site.in' \
| xargs clang-format -i
A Git merge driver is available in libcxx/utils/clang-format-merge-driver.sh
to help resolve merge and rebase issues across these formatting changes.
[1]: https://discourse.llvm.org/t/rfc-clang-formatting-all-of-libc-once-and-for-all
In preparation for running clang-format on the whole code base, we are
also removing mentions of the legacy _LIBCPP_INLINE_VISIBILITY macro in
favor of the newer _LIBCPP_HIDE_FROM_ABI.
We're still leaving the definition of _LIBCPP_INLINE_VISIBILITY to avoid
creating needless breakage in case some older patches are checked-in
with mentions of the old macro. After we branch for LLVM 18, we can do
another pass to clean up remaining uses of the macro that might have
gotten introduced by mistake (if any) and remove the macro itself at the
same time. This is just a minor convenience to smooth out the transition
as much as possible.
See
https://discourse.llvm.org/t/rfc-clang-formatting-all-of-libc-once-and-for-all
for the clang-format proposal.
This brings most of the enable_ifs in libc++ to the same style. It also has the nice side-effect of reducing the size of names of these symbols, since the arguments don't get mangled anymore.
Reviewed By: #libc, Mordante
Spies: Mordante, libcxx-commits
Differential Revision: https://reviews.llvm.org/D157748
This brings most of the enable_ifs in libc++ to the same style. It also has the nice side-effect of reducing the size of names of these symbols, since the depedent return type is shorter.
Reviewed By: #libc, ldionne
Spies: ldionne, libcxx-commits
Differential Revision: https://reviews.llvm.org/D157736
Replace most uses of `_LIBCPP_ASSERT` with
`_LIBCPP_ASSERT_UNCATEGORIZED`.
This is done as a prerequisite to introducing hardened mode to libc++.
The idea is to make enabling assertions an opt-in with (somewhat)
fine-grained controls over which categories of assertions are enabled.
The vast majority of assertions are currently uncategorized; the new
macro will allow turning on `_LIBCPP_ASSERT` (the underlying mechanism
for all kinds of assertions) without enabling all the uncategorized
assertions (in the future; this patch preserves the current behavior).
Differential Revision: https://reviews.llvm.org/D153816
These macros are always defined identically, so we can simplify the code a bit by merging them.
Reviewed By: ldionne, #libc
Spies: libcxx-commits, krytarowski, smeenai
Differential Revision: https://reviews.llvm.org/D152652
This patch makes are code less dependant on transitive includes.
This was part of D145800. This patch will be abandoned, but these
changes are still useful. I manually verified declarations of the new
includes are used in these files.
Reviewed By: #libc, philnik
Differential Revision: https://reviews.llvm.org/D148645
We already have a clang-tidy check for making sure that `_LIBCPP_HIDE_FROM_ABI` is on free functions. This patch extends this to class members. The places where we don't check for `_LIBCPP_HIDE_FROM_ABI` are classes for which we have an instantiation in the library.
Reviewed By: ldionne, Mordante, #libc
Spies: jplehr, mikhail.ramalho, sstefan1, libcxx-commits, krytarowski, miyuki, smeenai
Differential Revision: https://reviews.llvm.org/D142332
