Implement `statfs` and `fstatfs` functions on Linux.
We already have `struct statfs` and syscall wrappers defined (to
implement POSIX `<sys/statvfs.h>`). Use them to provide Linux-specific
functions as well.
Use the provided alignment for the pointer to the returned
memory chunk instead of the size.
This fixes occasional segfaults in `hsearch` tests after
ce62aab97e, since
hash-table bitmask implementation uses SSE2 intrinsics with 16-byte
alignment.
Added struct passwd type definition, pwd.yaml header specification, and
parse_passwd_line utility returning ErrorOr<struct passwd> for
colon-separated
password file parsing with hermetic unit tests.
Note: Additional POSIX pwd.h functions (getpwuid, getpwnam, getpwuid_r,
getpwnam_r, fgetpwent) are omitted from this initial change and will be
added in follow-up PRs.
* Added libc/include/llvm-libc-types/struct_passwd.h and proxy header
* Added libc/include/pwd.yaml header specification
* Added pwd_utils.h and pwd_utils.cpp implementing ErrorOr<struct
passwd> parse_passwd_line
* Added pwd.h to target public headers for linux architectures
* Added unit tests in libc/test/src/pwd/pwd_utils_test.cpp
Assisted-by: Automated tooling, human reviewed.
This is useful for several reasons:
- it makes it possible to use the macros outside of the test methods,
aligning with googletest. Although this isn't recommended, the
alternatives are often not worth it.
- it makes it possible (with additional changes) to expose this
infrastructure to C_TESTs (which currently use the assert() macro, which
is a no-op in release mode)
- it makes it possible to add (and have assertions in) googletest style
SetUp/TearDownTestSuite functions.
I've converted the main assertion macros for now, to test the waters.
I'm planning to handle EXPECT/ASSERT_THAT and EXPECT/ASSERT_EXIT/DEATH
in a follow-up.
Add EXPECT_STREQ and ASSERT_STREQ macros to IntegrationTest/test.h using
cpp::string_view. Note that, unlike the unit test macro (but like
googletest), this version treats NULL as distinct from "". The
inconsistency is unfortunate, but I think it's important as the code is
used for testing functions like getenv(), where "" and NULL have very
different meanings.
We should probably follow this up with a change to make the unit test
macro behave the same way.
Update integration tests to use the new macros instead of handrolled
string equality functions (my_streq), strcmp, inline_strcmp, or
string_view wrappers.
Currently the tests skip themselves because this gets treated as a
(missing) dependency.
I also enable tests that were explicitly skipped with UNIT_TEST_ONLY.
Some of the FP tests are claiming to be failing on GPUs (in 2023). If
that is still true, I'll skip them with if(TARGET_IS_GPU).
Extend the `<dirent.h>` header with macro and types specified in recent
POSIX.1-2024:
* Add `posix_dent` structure, which has more fields than `dirent`, that
are actually used in practice. This struct would be identical to
`dirent` that we have on Linux
* Add `reclen_t` type for `d_reclen` field.
* Add macro `DT_BLK` and friends
Also, extend the tests to verify the values of `d_type` field, now that
we have the proper macro defined.
Assisted by: Gemini, human-verified
This commit updates realpath to call `readlinkat` on symlinks during
path resolution.
Some notes on implementation:
- `PendingPath` uses a `cpp::string` to store unprocessed path
components. This could be more efficient by using a `PATH_MAX` buffer
and storing the path at the end of the buffer, which would allow for
easy prepending. However, using a `cpp::string` avoids the pointer
arithmetic and memory management that would involve.
- This PR implements `cpp::string::replace` to avoid pointer arithmetic
inside `realpath`. This should hopefully not be too inefficient, since
`prepend` reuses space from processed path components there's enough
room, there certainly exist some bad inputs that would cause many
`memmove` operations.
[libc] Implement cpp::unique_ptr utility
Implemented cpp::unique_ptr and cpp::default_delete in
libc/src/__support/CPP/unique_ptr.h for internal use within LLVM-libc.
* Added cpp::default_delete and its array specialization.
* Implemented cpp::unique_ptr with support for custom deleters and
array specialization.
* Applied LIBC_TRIVIAL_ABI to unique_ptr and LIBC_NO_UNIQUE_ADDRESS to
the
deleter to enable empty member optimization and register-passing ABI.
* Defined portable LIBC_NO_UNIQUE_ADDRESS and LIBC_TRIVIAL_ABI macros in
attributes.h.
* Included is_assignable.h in type_traits.h to support conversion
assignment constraints.
* Modernized template constraints using enable_if_t and
is_convertible_v.
* Added comprehensive unit tests in
libc/test/src/__support/CPP/unique_ptr_test.cpp, including coverage for
reset sequencing, custom deleters, and conversion constructors.
This is a simplified implementation with the following limitations:
* No support for custom pointer types (hardcoded to T*).
* No support for reference deleter types or array conversion.
Adding this to LLVM libc because some things (like regex) require
dynamic
memory allocation, and this will help with correct memory management.
Assisted-by: Automated tooling, human reviewed.
`fscanf` family of functions should return EOF if:
- end of input is reached before either the first successful conversion
or a matching failure occurs;
- a read error occurs.
This wasn't handled correctly before - in "system FILE" mode (in overlay
build) `EOF` return value from `getc` was passed through to parser,
violating `Reader` interface, which triggered infinite loops on
"while-not-EOF" `fscanf` loops. In llvm-libc-FILE mode `fscanf` simply
returned zero instead of EOF, because the code only checked error
indicator on a stream.
This PR removes _any_ lookups of eof/error indicators on an input stream
- instead we can simply rely on the `getc` (system or internal) function
behavior, which returns `EOF` on unsuccessful reads.
Extend the test cases for `fscanf` to cover various return values in
cases when `EOF` is reached (before or after conversions / matching
errors).
Added the POSIX putenv() function and its internal support.
Implemented EnvironmentManager::put() to insert caller-provided
"name=value" strings directly into the environment array, managing
ownership correctly (caller retains ownership).
Registered for x86_64, aarch64, and riscv. Integration tests cover basic
operations, ownership semantics, validation, and edge cases.
Assisted-by: Automated tooling, human reviewed.
Aligned allocation is now supported, but the test burns through our 1MB
never-freeing bump pointer allocation buffer. I tweak the test to reuse
the buffer between loop iterations, reducing its memory usage by about
300x :P.
The only issue was in pkey_test, which uses function-local static to do
lazy initialization. This requires the c++ runtime
(__cxa_guard_acquire/release).
There are no threads in this test, so we can just use the
(non-thread-safe) hand-rolled version of that. This is currently our
only test with such a pattern.
This would be something that in gtest would go into a SetUpTestSuite
method, but our test framework currently doesn't have an equivalent.
We never unlocked the ThreadAtExit mutex, which meant the second call to
exit deadlocked. This was detected in the unit tests as they were
calling __cxa_thread_atexit from the system C library.
This PR adds a stub for `strptime` under
`LLVM_LIBC_ENABLE_EXPERIMENTAL_ENTRYPOINTS` for x86_64 and aarch64 as a
first step in implementing the function.
Resolved compiler warnings in several libc unit and integration tests:
- Fixed -Wmissing-designated-field-initializers in localtime_r_test.cpp
by using zero-initialisation instead of partial designated initialisers.
- Fixed -Wimplicit-int-conversion in sigaltstack_test.cpp by explicitly
casting the loop index to uint8_t.
- Fixed -Wunused-parameter in pthread_barrier_test.cpp by omitting
unused parameter names in function definitions.
Assisted-by: Automated tooling, human reviewed.
This PR generally updates `cpp::string` to avoid incidental allocations.
Specifically, it:
- Updates `opreator=(string_view)` to avoid allocating a temporary
string:
67ebc4b221/libc/src/__support/CPP/string.h (L106-L108)
- Changes `operator+=(const string&)` to accept a `string_view` so that
strings may be appended without allocation.
- Makes the `string(string_view)` constructor explicit. Before, there
were non-obvious allocations because of the implicit conversion.
As a side effect, this PR has to more carefully handle self-assignment
and self-append. This PR updates append and assignment to avoid calling
`realloc` during append / assignment, as this may invalidate input
pointers held by `string_view` if they point to data held by the string.
This also fixes self-assignment, which previously didn't work, eg
`cpp::string s = "abc"; s = s;` would zero out `s`.
Synchronised child thread execution in pthread_setschedparam_test using
a mutex lock. This prevents race conditions where the child thread
exited before scheduling parameter assertions ran on multi-core
platforms such as aarch64.
Added missing pthread target dependencies to CMakeLists.txt.
Assisted-by: Automated tooling, human reviewed.
Currently, `capacity()` includes the null terminator of the backing
buffer, which does not align with C++20's `std::string`. That is,
`s.reserve(s.capacity())` should be a no-op. While there's no
requirement these be in-sync, it is less surprising to align its
behavior.
Also, this fixes the case where `reserve()` is called on an empty
string, where previously the null terminator was not set.
Using a pipe avoids races between "hermetic" and "unit" versions of the
same test and avoids leaving filesystem artifacts around if the test
fails. If I'm successful, we won't have the worry about the first
problem anymore, but I think it's still a nice cleanup.
Assisted by Gemini.
To make this work, I needed to remove the getauxval definition from
HermeticTestUtils.cpp. Even though it was weak, it still prevented the
real one from being used because the linker will not extract an archive
member if the dependency has already been satisfied by a weak
definition.
I *think* I've removed the need for it by changing how the getauxval
dependency is declared in cmake (it fixes the errors on the presubmit
aarch64 bot), but I don't know if that was the only issue.
This reverts https://github.com/llvm/llvm-project/pull/210889
(b7f642fdb2,
"Revert
recent changes to the hermetic tests"), re-applying
https://github.com/llvm/llvm-project/pull/209999
(9e2e9b33e1,
"[libc] Port process
utilities to hermetic mode and enable some tests") and
https://github.com/llvm/llvm-project/pull/210715
(4bd1a447e1,
"[libc] Make hermetic test
syscall deps linux-only"), which was reverted due to failures on the GPU
bots.
There are a couple of small changes w.r.t the original patches:
- This version disables the exit tests on builds that do not
support subprocess tests (via the newly introduced
LIBC_TEST_SUBPROCESS_TESTS). This is necessary because the GPU builds
contain the relevant entry points, which means their tests are not
skipped automatically.
- fix a typo in a variable name
- make the dependencies conditional on LIBC_TEST_SUBPROCESS_TESTS
instead of listing the target OSs directly. This is a no-op, as that's
exactly how the variable is defined, but it makes the code cleaner.
Implemented the pthread_setschedparam and pthread_getschedparam
functions.
Added Linux syscall wrappers:
* sched_setscheduler
* sched_getscheduler
* sched_getparam
Updated the Thread class to support getting and setting scheduling
parameters.
Added integration tests to verify the implementation.
Assisted-by: Automated tooling, human reviewed.
This replaces the ENABLE_SUBPROCESS_TESTS preprocessor flag with a cmake
variable. By lifting the logic into cmake, we select which test
framework files to build, and also exclude whole tests, if they do not
make sense without subprocess support.
This enables us to fix failures from (now reverted) #209999 and
#210889, as it will let us skip the *exit family of tests. The problem
there was that GPU targets support the *exit entry points (so the
auto-skipping logic does not kick in), but they do not have (and maybe
cannot have?) the functionality necessary to test them.
I convert two libgen tests (the only users of ENABLE_SUBPROCESS_TESTS)
to the new framework.
The code will be updated and used for wide character printf functions
implemented in wchar/, so __support/ is a more consistent shared
location.
Assisted-by: Automated tooling, human reviewed.
PR #209999 (9e2e9b33e1) "broke" the GPU
build by causing many/most tests to not run.
The follow-up in #210715 (4bd1a447e1)
broke it for real
(https://lab.llvm.org/buildbot/#/builders/10/builds/32254) by causing
too many tests to run.
I think we just need to disable the extra tests, but I want to start out
by reverting both patches so that it's easier to compare the
before/after state in the next attempt.
Add definitions for struct group_req and struct group_source_req in
llvm-libc-types and wire them into <netinet/in.h>.
Also define the multicast socket options using these structs:
- MCAST_JOIN_GROUP
- MCAST_LEAVE_GROUP
- MCAST_BLOCK_SOURCE
- MCAST_UNBLOCK_SOURCE
- MCAST_JOIN_SOURCE_GROUP
- MCAST_LEAVE_SOURCE_GROUP
Add layout assertions for struct group_req and struct group_source_req
to the netinet/in.h unit test.
Assisted by Gemini.
When writing the test, I assumed that (linux) domain sockets do not
support OOB data, and that sockatmark returns 0, because it can never
read it.
In fact, as of 2021, linux does support OOB on domain sockets, but this
feature can be turned off at build time (CONFIG_AF_UNIX_OOB). In this
case (or in the case of older kernels), the kernel returns an error (and
the test fails).
Armed with this knowledge, I modify the test to test both "1" (OOB
present) and "0" (no OOB data) cases by actually sending OOB data into
the socket. I use the send call to determine the presence of OOB
support, and have the test skip itself if it is absent. OOB data only
makes sense on stream sockets, so change the socketpair type to that.
---------
Co-authored-by: Jeff Bailey <jbailey@raspberryginger.com>
This is achieved by calling internal libc functions in hermetic mode. In
the overlay mode I keep calling the system functions so that the tests
work even on systems which don't have these implemented.
I also needed to implement the delete operators as both the libc proper
and the test framework uses them nowadays.
This is sufficient to enable all stdlib tests in hermetic mode, except
for one, which actually exposes a bug in the implementation. I'll deal
with that in a separate patch.
Follow-up PRs will add path validation etc to `realpath`, which will
require paths to exist in the filesystem.
This PR bumps the memory limit in `HermeticTestUtils.cpp`. The realpath
unit tests allocate quite a few strings, and since memory in hermetic
tests is never freed, the unit test quickly reaches the limit.
My cleverness with the union backfired as it introduced an
(architecture-specific) padding between before it. Drop the union to
avoid that.
This probably wouldn't break much as the structure exists just to
allocate space (and this made it larger), but we should fix it
nonetheless.
This patch adds the following sys/socket functions and syscall wrapper
libraries to BUILD.bazel:
- accept
- accept4
- bind
- connect
- getpeername
- getsockname
- listen
- sockatmark
I'm also enabling the unit tests for these functions. Running AF_UNIX
socket tests under Bazel ran into a problem where the absolute path to
the test sandbox directory exceeds the 108-byte limit of
sockaddr_un::sun_path. To work around that, I added a TestDirectoryScope
helper that temporarily chdirs into the test directory so the tests can
use short relative filenames, and restores the original working
directory on exit.
Assisted by Gemini.
The issue was reported by Paul Zimmermann.
Solution: for denormal inputs, we use 1.0 + x trick after proper scaling
to mimick rounding at denormal scale.
