Managed ArrayLists are deprecated in 0.15. "ArrayList" in 0.15 is the
same as "ArrayListUnmanaged" in 0.14. The latter alias is still
available in 0.15, so let's stick with that for now. When dropping
support for 0.14, we can do s/ArrayListUnmanaged/ArrayList/.
Arguably, my previous use of @setCold() was incorrect in some cases, but
it happened to work out better than not providing any hints. Let's use
the more granular hints now that Zig 0.14 supports them.
New `@branchHint` builtin is more expressive than `@setCold`, therefore
latter was removed.
See https://github.com/ziglang/zig/pull/21214 .
Signed-off-by: Eric Joldasov <bratishkaerik@landless-city.net>
I was trying to remove the need for that strip command with some
-fno-exceptions -fno-unwind-tables -fno-asynchronous-unwind-tables,
but passing even one of those to the ncurses build would result in an
ncdu binary that's twice as large and fails to run. I don't get it.
Which is, AFAIK, a recommended practice. Reduces the number of times
translate-c is being run and (most likely) simplifies a possible future
transition if/when @cImport is thrown out of the language.
Also uses zstd.h instead of my own definitions, mainly because I plan to
use the streaming API as well and those need more definitions.
This prevents displaying invalid zero values or writing such values out
in JSON/bin exports. Very old issue, actually, but with the new binfmt
experiments it's finally started annoying me.
I realized that the 16 MiB limitation implied that the index block could
only hold ((2^24)-16)/8 =~ 2 mil data block pointers. At the default
64k data block size that means an export can only reference up to
~128 GiB of uncompressed data. That's pretty limiting.
This change increases the maximum size of the index block to 256 MiB,
supporting ~33 mil data block pointers and ~2 TiB of uncompressed data
with the default data block size.
And do dynamic buffer allocation for bin_export, removing 128k of
.rodata that I accidentally introduced earlier and reducing memory use
for parallel scans.
Static binaries now also include the minimal version of zstd, current
sizes for x86_64 are:
582k ncdu-2.5
601k ncdu-new-nocompress
765k ncdu-new-zstd
That's not great, but also not awful. Even zlib or LZ4 would've resulted
in a 700k binary.
This isn't the low-memory browsing experience I was hoping to implement,
yet, but it serves as a good way to test the new format and such a
sink-based import is useful to have anyway.
Performance is much better than I had expected, and I haven't even
profiled anything yet.
This ended up a little different than I had originally planned.
The bad part is that my idea for the 'prevlnk' references wasn't going
to work out. For one because the reader has no efficient way to
determine the head reference of this list and implementing a lookup
table would be pretty costly and complex, and second because even with
those references working, they'd be pretty useless because there's no
way to go from an itemref to a full path. I don't see an easy way to
solve these problems, so I'm afraid the efficient hardlink list feature
will have to be disabled when reading from this new format. :(
The good news is that removing these references simplifies the hardlink
counting implementation and removes the requirement for a global inode
map and associated mutex. \o/
Performance is looking really good so far, too.
The goals of this format being:
- Streaming parallel export with minimal mandatory buffering.
- Exported data includes cumulative directory stats, so reader doesn't
have to go through the entire tree to calculate these.
- Fast-ish directory listings without reading the entire file.
- Built-in compression.
Current implementation is missing compression, hardlink counting and
actually reading the file. Also need to tune and measure stuff.
