The fact that `hdr/elf_proxy.h` was generated with headergen was causing
dependency issues. This shouldn't be necessary, and this PR moves it out
into `include` to avoid needing headergen for an overlay build.
Assisted-by: Automated tooling, human reviewed.
Summary:
Fixes some failing tests. Fmod's builtin has improper handling in the
NVPTX instruction so we should just use the generic version for now. The
Float128 implementation seems to have had some undefined behavior for
out of range behavior that only showed up now. Hopefully someone can
tell me if this is fixed correctly.
Implement `statfs` and `fstatfs` functions on Linux.
We already have `struct statfs` and syscall wrappers defined (to
implement POSIX `<sys/statvfs.h>`). Use them to provide Linux-specific
functions as well.
This is a Linux extension that allows to obtain credentials of a peer
process connected to a socket, e.g. via passing `SO_PEERCRED` to
`getsockopt` function.
Use the provided alignment for the pointer to the returned
memory chunk instead of the size.
This fixes occasional segfaults in `hsearch` tests after
ce62aab97e, since
hash-table bitmask implementation uses SSE2 intrinsics with 16-byte
alignment.
* Extract statfs/fstatfs Linux syscall wrappers to
`linux/syscall_wrappers` directory;
* Provide our own definition of `struct statfs` type and a corresponding
proxy header (to use system type in overlay mode);
* Migrate callers (e.g. statfs->statvfs translation) to use the new
syscall wrapper.
This would allow us to add Linux-specific (non-POSIX) `<sys/statfs.h>`
header as a next step.
Assisted by: Gemini, human-reviewed
Added struct passwd type definition, pwd.yaml header specification, and
parse_passwd_line utility returning ErrorOr<struct passwd> for
colon-separated
password file parsing with hermetic unit tests.
Note: Additional POSIX pwd.h functions (getpwuid, getpwnam, getpwuid_r,
getpwnam_r, fgetpwent) are omitted from this initial change and will be
added in follow-up PRs.
* Added libc/include/llvm-libc-types/struct_passwd.h and proxy header
* Added libc/include/pwd.yaml header specification
* Added pwd_utils.h and pwd_utils.cpp implementing ErrorOr<struct
passwd> parse_passwd_line
* Added pwd.h to target public headers for linux architectures
* Added unit tests in libc/test/src/pwd/pwd_utils_test.cpp
Assisted-by: Automated tooling, human reviewed.
This is useful for several reasons:
- it makes it possible to use the macros outside of the test methods,
aligning with googletest. Although this isn't recommended, the
alternatives are often not worth it.
- it makes it possible (with additional changes) to expose this
infrastructure to C_TESTs (which currently use the assert() macro, which
is a no-op in release mode)
- it makes it possible to add (and have assertions in) googletest style
SetUp/TearDownTestSuite functions.
I've converted the main assertion macros for now, to test the waters.
I'm planning to handle EXPECT/ASSERT_THAT and EXPECT/ASSERT_EXIT/DEATH
in a follow-up.
Add EXPECT_STREQ and ASSERT_STREQ macros to IntegrationTest/test.h using
cpp::string_view. Note that, unlike the unit test macro (but like
googletest), this version treats NULL as distinct from "". The
inconsistency is unfortunate, but I think it's important as the code is
used for testing functions like getenv(), where "" and NULL have very
different meanings.
We should probably follow this up with a change to make the unit test
macro behave the same way.
Update integration tests to use the new macros instead of handrolled
string equality functions (my_streq), strcmp, inline_strcmp, or
string_view wrappers.
Currently the tests skip themselves because this gets treated as a
(missing) dependency.
I also enable tests that were explicitly skipped with UNIT_TEST_ONLY.
Some of the FP tests are claiming to be failing on GPUs (in 2023). If
that is still true, I'll skip them with if(TARGET_IS_GPU).
Extend the `<dirent.h>` header with macro and types specified in recent
POSIX.1-2024:
* Add `posix_dent` structure, which has more fields than `dirent`, that
are actually used in practice. This struct would be identical to
`dirent` that we have on Linux
* Add `reclen_t` type for `d_reclen` field.
* Add macro `DT_BLK` and friends
Also, extend the tests to verify the values of `d_type` field, now that
we have the proper macro defined.
Assisted by: Gemini, human-verified
This commit updates realpath to call `readlinkat` on symlinks during
path resolution.
Some notes on implementation:
- `PendingPath` uses a `cpp::string` to store unprocessed path
components. This could be more efficient by using a `PATH_MAX` buffer
and storing the path at the end of the buffer, which would allow for
easy prepending. However, using a `cpp::string` avoids the pointer
arithmetic and memory management that would involve.
- This PR implements `cpp::string::replace` to avoid pointer arithmetic
inside `realpath`. This should hopefully not be too inefficient, since
`prepend` reuses space from processed path components there's enough
room, there certainly exist some bad inputs that would cause many
`memmove` operations.
Add missing type dependencies to the sys_types header target in
libc/include/CMakeLists.txt for types defined in sys/types.yaml,
including __off_t, __off64_t, __uint64_t, off64_t, and BSD/System V
types added in commit 39df67d0ed.
Without these DEPENDS entries, CMake and Ninja do not copy the
corresponding type headers into the build and install directories,
causing missing header includes (e.g. __off64_t.h) when consuming
generated <sys/types.h>.
Assisted-by: Automated tooling, human reviewed.
Bypass custom operator delete redirection in new.h and new.cpp when
building with AddressSanitizer (LIBC_HAS_ADDRESS_SANITIZER).
cpp::unique_ptr is the first heap-owning utility in src/__support/CPP/
that executes delete/delete[] expressions. Previously, new.h redirected
operator delete via asm-renaming to __llvm_libc_delete (calling free()),
which caused ASan to report an alloc-dealloc-mismatch (operator new vs
free) when objects allocated via host operator new were deleted by
cpp::unique_ptr.
Guarding the custom operator delete implementations under ASan enables
ASan deallocation tracking while preserving hermetic libc behavior for
non-ASan builds.
Assisted-by: Automated tooling, human reviewed.
[libc] Implement cpp::unique_ptr utility
Implemented cpp::unique_ptr and cpp::default_delete in
libc/src/__support/CPP/unique_ptr.h for internal use within LLVM-libc.
* Added cpp::default_delete and its array specialization.
* Implemented cpp::unique_ptr with support for custom deleters and
array specialization.
* Applied LIBC_TRIVIAL_ABI to unique_ptr and LIBC_NO_UNIQUE_ADDRESS to
the
deleter to enable empty member optimization and register-passing ABI.
* Defined portable LIBC_NO_UNIQUE_ADDRESS and LIBC_TRIVIAL_ABI macros in
attributes.h.
* Included is_assignable.h in type_traits.h to support conversion
assignment constraints.
* Modernized template constraints using enable_if_t and
is_convertible_v.
* Added comprehensive unit tests in
libc/test/src/__support/CPP/unique_ptr_test.cpp, including coverage for
reset sequencing, custom deleters, and conversion constructors.
This is a simplified implementation with the following limitations:
* No support for custom pointer types (hardcoded to T*).
* No support for reference deleter types or array conversion.
Adding this to LLVM libc because some things (like regex) require
dynamic
memory allocation, and this will help with correct memory management.
Assisted-by: Automated tooling, human reviewed.
`fscanf` family of functions should return EOF if:
- end of input is reached before either the first successful conversion
or a matching failure occurs;
- a read error occurs.
This wasn't handled correctly before - in "system FILE" mode (in overlay
build) `EOF` return value from `getc` was passed through to parser,
violating `Reader` interface, which triggered infinite loops on
"while-not-EOF" `fscanf` loops. In llvm-libc-FILE mode `fscanf` simply
returned zero instead of EOF, because the code only checked error
indicator on a stream.
This PR removes _any_ lookups of eof/error indicators on an input stream
- instead we can simply rely on the `getc` (system or internal) function
behavior, which returns `EOF` on unsuccessful reads.
Extend the test cases for `fscanf` to cover various return values in
cases when `EOF` is reached (before or after conversions / matching
errors).
Add header YAML specifications for memory.h, sys/poll.h, sys/unistd.h,
sys/fcntl.h, and sys/signal.h using public_includes.
POSIX.1-2017 and historical X/Open System Interfaces (XSI) standards
define these headers as alternate or legacy header locations that
forward to string.h, poll.h, unistd.h, fcntl.h, and signal.h
respectively. The YAML header specs allow hdrgen to generate the public
headers automatically with proper license headers and guards.
Updated Linux target headers.txt configuration files to register the new
public header targets for installation.
Assisted-by: Automated tooling, human reviewed.
Added the POSIX putenv() function and its internal support.
Implemented EnvironmentManager::put() to insert caller-provided
"name=value" strings directly into the environment array, managing
ownership correctly (caller retains ownership).
Registered for x86_64, aarch64, and riscv. Integration tests cover basic
operations, ownership semantics, validation, and edge cases.
Assisted-by: Automated tooling, human reviewed.
Aligned allocation is now supported, but the test burns through our 1MB
never-freeing bump pointer allocation buffer. I tweak the test to reuse
the buffer between loop iterations, reducing its memory usage by about
300x :P.
The only issue was in pkey_test, which uses function-local static to do
lazy initialization. This requires the c++ runtime
(__cxa_guard_acquire/release).
There are no threads in this test, so we can just use the
(non-thread-safe) hand-rolled version of that. This is currently our
only test with such a pattern.
This would be something that in gtest would go into a SetUpTestSuite
method, but our test framework currently doesn't have an equivalent.
We never unlocked the ThreadAtExit mutex, which meant the second call to
exit deadlocked. This was detected in the unit tests as they were
calling __cxa_thread_atexit from the system C library.
This PR adds a stub for `strptime` under
`LLVM_LIBC_ENABLE_EXPERIMENTAL_ENTRYPOINTS` for x86_64 and aarch64 as a
first step in implementing the function.
`aarch64/inline_memset.h` defines type aliases `uint128_t`, `uint256_t`
and `uint512_t` at file scope. `aarch64/inline_memmove.h`, alongside it,
defines the same aliases at function scope. If both headers are included
by the same source file, this can lead to a warning about the
function-scope aliases shadowing the file-scope ones, which turns into a
compile error if you build with `clang -Werror -Wshadow`.
Commit a81db64570 (PR #210895) exposed this latent compile failure,
by making `CPP/string.h` include both files. But there's nothing wrong
with that, so the right fix is to make it _safe_ to include both files,
not to stop doing it.
This commit moves the function-scope aliases in memmove up to file
scope. That makes them duplicate definitions in the _same_ scope, which
doesn't cause the same error.
Other options would be to move the memset aliases down into function
scope (duplicating them in two functions), or to move them into a tiny
AArch64-specific header file with an include guard. Both of those are
more intrusive than this fix.
Resolved compiler warnings in several libc unit and integration tests:
- Fixed -Wmissing-designated-field-initializers in localtime_r_test.cpp
by using zero-initialisation instead of partial designated initialisers.
- Fixed -Wimplicit-int-conversion in sigaltstack_test.cpp by explicitly
casting the loop index to uint8_t.
- Fixed -Wunused-parameter in pthread_barrier_test.cpp by omitting
unused parameter names in function definitions.
Assisted-by: Automated tooling, human reviewed.
This PR generally updates `cpp::string` to avoid incidental allocations.
Specifically, it:
- Updates `opreator=(string_view)` to avoid allocating a temporary
string:
67ebc4b221/libc/src/__support/CPP/string.h (L106-L108)
- Changes `operator+=(const string&)` to accept a `string_view` so that
strings may be appended without allocation.
- Makes the `string(string_view)` constructor explicit. Before, there
were non-obvious allocations because of the implicit conversion.
As a side effect, this PR has to more carefully handle self-assignment
and self-append. This PR updates append and assignment to avoid calling
`realloc` during append / assignment, as this may invalidate input
pointers held by `string_view` if they point to data held by the string.
This also fixes self-assignment, which previously didn't work, eg
`cpp::string s = "abc"; s = s;` would zero out `s`.
Synchronised child thread execution in pthread_setschedparam_test using
a mutex lock. This prevents race conditions where the child thread
exited before scheduling parameter assertions ran on multi-core
platforms such as aarch64.
Added missing pthread target dependencies to CMakeLists.txt.
Assisted-by: Automated tooling, human reviewed.
Extend SIGCHLD-specific `si_code` values, already available in
`signal-macros.h` with Linux values for the rest of the signal-specific
codes specified in the POSIX.1-2024.
Currently, `capacity()` includes the null terminator of the backing
buffer, which does not align with C++20's `std::string`. That is,
`s.reserve(s.capacity())` should be a no-op. While there's no
requirement these be in-sync, it is less surprising to align its
behavior.
Also, this fixes the case where `reserve()` is called on an empty
string, where previously the null terminator was not set.
Add missing header macro constants:
* endian-macros.h: __BYTE_ORDER, __LITTLE_ENDIAN, and __BIG_ENDIAN
* sys-stat-macros.h: S_IREAD, S_IWRITE, S_IEXEC, and ACCESSPERMS
* limits-macros.h: _POSIX_HOST_NAME_MAX and HOST_NAME_MAX
* unistd-macros.h: TEMP_FAILURE_RETRY
POSIX.1-2017 specifies _POSIX_HOST_NAME_MAX as 255. The Linux
gethostname(2) man page specifies HOST_NAME_MAX as 64. The
double-underscore byte order macros and TEMP_FAILURE_RETRY macro follow
the Linux and GNU C library conventions.
Assisted-by: Automated tooling, human reviewed.
Add missing type definitions to llvm-libc-types and sys/types.yaml:
* POSIX standard types: id_t, useconds_t
* BSD and System V compatibility typedefs: caddr_t, u_int, u_int8_t,
u_int16_t, u_long, u_short, ulong, ushort
* Large File Support (LFS) and Linux kernel internal types: ino64_t,
off64_t, __off_t, __off64_t, __uint64_t
POSIX.1-2017 defines id_t and useconds_t in sys/types.h. The BSD and
System V standards specify the legacy unsigned integer shorthand types,
and Linux kernel/glibc interfaces use the double-underscore prefix
offset and integer types.
Assisted-by: Automated tooling, human reviewed.
When compiling `inline_memmove` with MSVC for non-x86 architectures, `op_generic.h` picks
a non-builtin type for generic_v128 and friends:
cad3c95a89/libc/src/string/memory_utils/op_generic.h (L54-L57)
This causes calls to `load` and `store` to trigger ADL, which is
ambiguous with `load` definition in `memory_utils/utils.h`.
```
C:\Users\jtstogel\Github\llvm-project\libc\src/string/memory_utils/op_generic.h(348): error C2668: '__llvm_libc_24_0_0_git::generic::store': ambiguous call to overloaded function
C:\Users\jtstogel\Github\llvm-project\libc\src/string/memory_utils/op_generic.h(138): note: could be 'void __llvm_libc_24_0_0_git::generic::store<T>(__llvm_libc_24_0_0_git::Ptr,T)'
with
[
T=__llvm_libc_24_0_0_git::inline_memmove_aarch64::uint512_t
]
C:\Users\jtstogel\Github\llvm-project\libc\src/string/memory_utils/utils.h(221): note: or 'void __llvm_libc_24_0_0_git::store<T>(__llvm_libc_24_0_0_git::Ptr,T)' [found using argument-dependent lookup]
with
[
T=__llvm_libc_24_0_0_git::inline_memmove_aarch64::uint512_t
]
C:\Users\jtstogel\Github\llvm-project\libc\src/string/memory_utils/op_generic.h(348): note: while trying to match the argument list '(__llvm_libc_24_0_0_git::Ptr, const T)'
with
[
T=__llvm_libc_24_0_0_git::inline_memmove_aarch64::uint512_t
]
```
Using a pipe avoids races between "hermetic" and "unit" versions of the
same test and avoids leaving filesystem artifacts around if the test
fails. If I'm successful, we won't have the worry about the first
problem anymore, but I think it's still a nice cleanup.
Assisted by Gemini.
To make this work, I needed to remove the getauxval definition from
HermeticTestUtils.cpp. Even though it was weak, it still prevented the
real one from being used because the linker will not extract an archive
member if the dependency has already been satisfied by a weak
definition.
I *think* I've removed the need for it by changing how the getauxval
dependency is declared in cmake (it fixes the errors on the presubmit
aarch64 bot), but I don't know if that was the only issue.
This reverts https://github.com/llvm/llvm-project/pull/210889
(b7f642fdb2,
"Revert
recent changes to the hermetic tests"), re-applying
https://github.com/llvm/llvm-project/pull/209999
(9e2e9b33e1,
"[libc] Port process
utilities to hermetic mode and enable some tests") and
https://github.com/llvm/llvm-project/pull/210715
(4bd1a447e1,
"[libc] Make hermetic test
syscall deps linux-only"), which was reverted due to failures on the GPU
bots.
There are a couple of small changes w.r.t the original patches:
- This version disables the exit tests on builds that do not
support subprocess tests (via the newly introduced
LIBC_TEST_SUBPROCESS_TESTS). This is necessary because the GPU builds
contain the relevant entry points, which means their tests are not
skipped automatically.
- fix a typo in a variable name
- make the dependencies conditional on LIBC_TEST_SUBPROCESS_TESTS
instead of listing the target OSs directly. This is a no-op, as that's
exactly how the variable is defined, but it makes the code cleaner.
Use `LIBC_NO_SANITIZE_OOB_ACCESS` for small SIMD helper functions
(load/store/gather/scatter/expand/compress). Even though the actual
functions which have logical OOB reads (such as
`clang_vector::string_length`) already have
`LIBC_NO_SANITIZE_OOB_ACCESS` attribute to ignore OOB reads, it's not
enough - as we see downstream reports from ASan builds of llvm-libc
(using tip-of-trunk Clang), both with `-O1` and `-O2`. We simply can't
rely on the SIMD helpers being inlined into the caller function, with
their memory reads/writes ignored.
Thus, apply the `no_sanitize` attribute to the helpers themselves. This
is clearly suboptimal, as we're effectively disabling sanitizer checks
for *all* the code using SIMD to read/write from memory, but it seems to
be the easiest reasonable fix. After all, code using SIMD (like code
using explicit intrinsics or inline assembly) should be written only in
special cases, with author knowing what they're doing.
Implemented the pthread_setschedparam and pthread_getschedparam
functions.
Added Linux syscall wrappers:
* sched_setscheduler
* sched_getscheduler
* sched_getparam
Updated the Thread class to support getting and setting scheduling
parameters.
Added integration tests to verify the implementation.
Assisted-by: Automated tooling, human reviewed.
