Commit graph

580 commits

Author SHA1 Message Date
Yorhel
beac59fb12 Use std.c.fstatat() instead of @cImport()ed version
Because translate-c can't handle struct stat as defined by musl.
(Should have done this in the first place, but wasn't aware fstatat()
had been properly wrapped in std.c)
2025-04-28 13:22:03 +02:00
Yorhel
d97a7f73dd Fix integer overflow in binary export
And allocate an appropriately-sized initial block when
--export-block-size is set.

Fixes #257.
2025-04-28 12:43:04 +02:00
Yorhel
35a9faadb2 Work around panic in Zig fstatat wrapper
https://github.com/ziglang/zig/issues/23463
2025-04-06 10:36:36 +02:00
Eric Joldasov
e43d22ba3f
build.zig: change -Dpie option default to be target-dependant
`null` here indicates to Zig that it should decide for itself whether
to enable PIE or not. On some targets (like macOS or OpenBSD), it is
required and so current default will cause unneccessary build error.

Behavior for `-Dpie=false` and `-Dpie=true` is not changed.

Signed-off-by: Eric Joldasov <bratishkaerik@landless-city.net>
2025-03-06 01:23:28 +05:00
Eric Joldasov
f4e4694612
build.zig: link zstd instead of libzstd
Seems like it was used here because `pkg-config` does not work with
`zstd` passed but works with `libzstd`.

In 0.14.0 there was a change with allows pkg-config to be used here:
it now tries to use `lib+libname.pc` if `libname.pc` was not found:
https://github.com/ziglang/zig/pull/22069

This change allows Zig to use plain file search if pkg-config was not
found (it would now search `libzstd.so` instead of `liblibzstd.so`).

Signed-off-by: Eric Joldasov <bratishkaerik@landless-city.net>
2025-03-06 01:14:14 +05:00
Eric Joldasov
c9f3d39d3e
build.zig: update to new API for 0.14.0
Consolidates some options in one single module now, which is helpful
when you want to edit linked libraries etc.

Useful for making future changes here, or patching by distros (1 place
to change instead of 2).

Signed-off-by: Eric Joldasov <bratishkaerik@landless-city.net>
2025-03-06 01:10:55 +05:00
Yorhel
2b4c1ca03e Version 2.8 2025-03-05 11:10:31 +01:00
Yorhel
af7163acf6 Revise @branchHint probabilities
Arguably, my previous use of @setCold() was incorrect in some cases, but
it happened to work out better than not providing any hints.  Let's use
the more granular hints now that Zig 0.14 supports them.
2025-03-05 10:56:46 +01:00
Yorhel
5438312440 Fix tests
Broken in 5d5182ede3
2025-03-05 10:44:14 +01:00
Eric Joldasov
0918096301
fix std.Target.isDarwin function replaced with direct version
See https://github.com/ziglang/zig/pull/22589 .

Signed-off-by: Eric Joldasov <bratishkaerik@landless-city.net>
2025-03-03 22:46:51 +05:00
Eric Joldasov
ee1d80da6a
std.fmt.digits now accept u8 instead of usize
See https://github.com/ziglang/zig/pull/22864 .

Signed-off-by: Eric Joldasov <bratishkaerik@landless-city.net>
2025-03-03 22:46:51 +05:00
Eric Joldasov
93a81a3898
ArrayList.pop now returns optional like removed popOrNull
See https://github.com/ziglang/zig/pull/22720 .

Signed-off-by: Eric Joldasov <bratishkaerik@landless-city.net>
2025-03-03 22:46:51 +05:00
Eric Joldasov
cf3a8f3043
update to new allocator interface
See https://github.com/ziglang/zig/pull/20511 .

Signed-off-by: Eric Joldasov <bratishkaerik@landless-city.net>
2025-03-03 22:46:51 +05:00
Eric Joldasov
f7fe61194b
update to new panic interface
See https://github.com/ziglang/zig/pull/22594 .

Signed-off-by: Eric Joldasov <bratishkaerik@landless-city.net>
2025-03-03 22:46:50 +05:00
Eric Joldasov
456cde16df
remove anonymous struct types
See https://github.com/ziglang/zig/pull/21817 .

Signed-off-by: Eric Joldasov <bratishkaerik@landless-city.net>
2025-03-03 22:46:50 +05:00
Eric Joldasov
3c77dc458a
replace deprecated std.Atomic(T).fence with load
See https://github.com/ziglang/zig/pull/21585 .
I went with second option listed in `Conditional Barriers` section.

Signed-off-by: Eric Joldasov <bratishkaerik@landless-city.net>
2025-03-03 22:46:50 +05:00
Eric Joldasov
ce9921846c
update to new names in std.builtin.Type
See https://github.com/ziglang/zig/pull/21225 .

Signed-off-by: Eric Joldasov <bratishkaerik@landless-city.net>
2025-03-03 22:46:49 +05:00
Eric Joldasov
e0ab5d40c7
change deprecated @setCold(true) to @branchHint(.cold)
New `@branchHint` builtin is more expressive than `@setCold`, therefore
latter was removed.
See https://github.com/ziglang/zig/pull/21214 .

Signed-off-by: Eric Joldasov <bratishkaerik@landless-city.net>
2025-03-03 22:46:49 +05:00
Eric Joldasov
607b07a30e
change deprecated timespec.tv_sec to timespec.sec
Part of the reorganization of `std.c` namespace.
See https://github.com/ziglang/zig/pull/20679 .

Signed-off-by: Eric Joldasov <bratishkaerik@landless-city.net>
2025-03-03 22:46:49 +05:00
Eric Joldasov
b4dc9f1d4d
change deprecated std.fs.MAX_PATH_BYTES to max_path_bytes
It was deprecated before, and became hard compile error.
See https://github.com/ziglang/zig/pull/19847 .

Signed-off-by: Eric Joldasov <bratishkaerik@landless-city.net>
2025-03-03 22:46:45 +05:00
Yorhel
2e5c767d4c List all options in --help
Fixes #251. Not as bloated as I had expected, this seems workable.
2025-03-03 13:31:51 +01:00
Yorhel
5d5182ede3 Add support for @-prefix to ignore errors in config file
Forward-ported from the C version: ff830ac2bf
2025-03-01 13:33:29 +01:00
Henrik Bengtsson
db96bc698c SI units: The unit is kB, not KB 2024-12-22 11:21:36 -08:00
Yorhel
4873a7c765 Version 2.7 2024-11-19 14:41:50 +01:00
Yorhel
49d43f89a1 Fix build on 32bit systems + static build adjustments
I was trying to remove the need for that strip command with some
-fno-exceptions -fno-unwind-tables -fno-asynchronous-unwind-tables,
but passing even one of those to the ncurses build would result in an
ncdu binary that's twice as large and fails to run. I don't get it.
2024-11-17 11:51:29 +01:00
Yorhel
e5a6a1c5ea build: Expose -Dstrip flag
Compiling with -fstrip generates *much* smaller binaries, even compared
to running 'strip' after a regular build.
2024-11-17 09:30:42 +01:00
Yorhel
5593fa2233 Expand ~ and ~user in config file
Fixes #243
2024-11-16 11:38:12 +01:00
Yorhel
9d51df02c1 Add --export-block-size option + minor man page adjustments 2024-11-15 11:08:26 +01:00
Yorhel
7ed209a8e5 Properly fix zstd streaming decompression
I had a feeling my last workaround wasn't correct, turns out my basic
assumption about ZSTD_decompressStream() was wrong: rather than
guaranteeing some output when there's enough input, it always guarantees
to consume input when there's space in the output.

Fixed the code and adjusted the buffers again.
2024-11-14 10:46:24 +01:00
Yorhel
4bd6e3daba Fix handling short reads when uncompressed json
Turns out that zstd can consume compressed data without returning any
decompressed data when the input buffer isn't full enough. I just
increased the input buffer as a workaround.
2024-11-13 15:28:10 +01:00
Yorhel
2fcd7f370c Drop ncdubinexp.pl
Was a useful testing tool during development, but now been replaced with
a more robust 'ncdutils validate' in https://code.blicky.net/yorhel/ncdutils
2024-11-03 13:13:55 +01:00
Yorhel
232a4f8741 JSON import: support reading escaped UTF-16 surrogate pairs
Fixes #245

json/scanner.zig in std notes inconsistencies in the standard as to
whether unpaired surrogate halves are allowed. That implementation
disallows them and so does this commit.
2024-11-03 10:40:57 +01:00
Yorhel
bdc730f1e5 Bin export: fix incorrectly setting prev=0 on the root node 2024-10-29 14:45:10 +01:00
Yorhel
df5845baad Support writing zstd-compressed json, add --compress option 2024-10-26 19:30:16 +02:00
Yorhel
0e6967498f Support reading zstd-compressed json
Oddly enough this approach is slightly slower than just doing
`zstdcat x | ncdu -f-`, need to investigate.
2024-10-26 15:49:42 +02:00
Yorhel
bd442673d2 Consolidate @cImports into a single c.zig
Which is, AFAIK, a recommended practice. Reduces the number of times
translate-c is being run and (most likely) simplifies a possible future
transition if/when @cImport is thrown out of the language.

Also uses zstd.h instead of my own definitions, mainly because I plan to
use the streaming API as well and those need more definitions.
2024-10-26 14:35:05 +02:00
Yorhel
28d9eaecab Version 2.6 2024-09-27 10:49:22 +02:00
Yorhel
61d7fc8473 man: Mention new flags in the synopsis 2024-09-27 10:40:13 +02:00
Yorhel
e142d012f0 Man page updates
Haven't mentioned the new -O flag in the examples section yet. Let's
first keep it as a slightly lower-profile feature while the format gains
wider testing and adoption.
2024-09-26 15:07:09 +02:00
Yorhel
39517c01a8 Remove kernfs dev id cache
Kernfs checking was previously done for every directory scanned, but the
new parallel scanning code only performs the check when the dev id is
different from parent, which isn't nearly as common.
(In fact, in typical scenarios this only ever happens once per dev id,
rendering the cache completely useless. But even people will 10k bind
mounts are unlikely to notice a performance impact)
2024-08-25 09:29:41 +02:00
Yorhel
cc26ead5f8 Fix integer overflow and off-by-one in binfmt itemref parsing 2024-08-23 09:53:33 +02:00
Yorhel
ca46c7241f Fix off-by-one in binfmt reader 2024-08-18 08:38:57 +02:00
Yorhel
e324804cdd Strip stack unwinding info from static binaries
Saves another 70k on the x86_64 binary, more on x86.

None of the included C or Zig code will unwind the stack at any point,
so these sections seem pretty useless.
2024-08-11 16:26:40 +02:00
Yorhel
26229d7a63 binfmt: Remove "rawlen" field, require use of ZSTD_getFrameContentSize()
The zstd frame format already supports this functionality and I don't
really see a benefit in not making use of that.
2024-08-11 15:56:14 +02:00
Yorhel
4ef9c3e817 bin_export: Adaptively adjust block size 2024-08-11 10:56:43 +02:00
Yorhel
c30699f93b Track which extended mode fields we have + bugfixes
This prevents displaying invalid zero values or writing such values out
in JSON/bin exports. Very old issue, actually, but with the new binfmt
experiments it's finally started annoying me.
2024-08-09 18:32:47 +02:00
Yorhel
6b7983b2f5 binfmt: Support larger (non-data) block sizes
I realized that the 16 MiB limitation implied that the index block could
only hold ((2^24)-16)/8 =~ 2 mil data block pointers. At the default
64k data block size that means an export can only reference up to
~128 GiB of uncompressed data. That's pretty limiting.

This change increases the maximum size of the index block to 256 MiB,
supporting ~33 mil data block pointers and ~2 TiB of uncompressed data
with the default data block size.
2024-08-09 09:40:29 +02:00
Yorhel
9418079da3 binfmt: Remove CBOR-null-based padding hack
Seems like unnecessary complexity.
2024-08-09 09:19:27 +02:00
Yorhel
18f322c532 Throw an error when running out of DevId numbers
I find it hard to imagine that this will happen on a real filesystem,
but it can be triggered by a malicious export file. Better protect
against that than invoke undefined behavior.
2024-08-08 15:38:56 +02:00
Yorhel
252f7fc253 Use u64 for item counts in binary export
They're still clamped to a u32 upon reading, but at least the file will
have correct counts and can be read even when it exceeds 4.2 billion
items.
2024-08-08 11:37:55 +02:00