This should really have been accepted from the start. Adding extra
elements to the top-level array might be useful for aux. data related to
the scanned files, e.g. map of seen uid/gids to names, information about
the scanned dev ids, etc. But since ncdu would throw an error on further
array elements, adding such information can't be done without breaking
compat with older versions. :(
Bumping the minor version of the file format to '2'.
The "ino" field is only interesting for hardlinks, so we can save space
by not exporting it for other entries.
The hlink count will be interesting later on when I implement tracking
of shared data between directories. It's currently ignored on import.
The "nlink" field makes the "hlnkc" field redundant, but let's keep
including that field anyway for backwards compatibility.
This is a best-effort approach to save ncdu state when memory is low.
There's likely allocation in libraries that isn't being checked
(ncurses, printf).
Fixes#132 (it actually doesn't, that needs a 64bit static binary too,
but I'll get to that)
Unfortunately, there wasn't a single bit free in struct dir.flags, so I
had to increase its size to 16 bit. This commit is just the initial
preparation, there's still a few things to do:
- Add "extended information" cli flag to enable/disable this
functionality.
- Export and import extended information when requested
- Do something with the data.
I also did a few memory measurements on a file list with 12769842 items:
before this commit: 1.239 GiB
without extended info: 1.318 GiB
with extended info: 1.698 GiB
It's surprising what adding a single byte to a struct can do to the
memory usage. :(
Fixes https://dev.yorhel.nl/ncdu/bug/103
I don't think a stack overflow as a result of recursion is exploitable
on a modern system. It should just result in an unfortunate write to a
page that is not writable, followed by a crash.
This should fix https://dev.yorhel.nl/ncdu/bug/99 - with the downside
that this requires a C99 compiler.
I also replaced all occurrences of static allocation of struct dir with
use dynamic allocation, because I wasn't really sure if static
allocation of flexible structs is allowed. In the case of dirlist.c the
dynamic allocation is likely required anyway, because it does store a
few bytes in the name field.
Turns out that being able to open an empty directory actually has its
uses:
- If you delete the last file in a directory, you now won't be directed
to the parent directory anymore. This allows keeping 'd' pressed
without worrying that you'll delete stuff outside of the current dir.
(This is the primary motivation for doing this)
- You can now scan and later refresh an empty directory, as suggested by
#2 in http://dev.yorhel.nl/ncdu/bug/15
Some measurements importing a gzip-compressed file (zcat .. | ncdu -f -)
containing a bit under 6 million items and a few choices of how often to
call input_handle():
Called on every item:
real 0m13.745s
user 0m12.576s
sys 0m4.566s
Called on every 8 items:
real 0m7.932s
user 0m9.636s
sys 0m1.623s
Called on every 16 items:
real 0m7.559s
user 0m9.553s
sys 0m1.323s
Called on every 32 items:
real 0m7.279s
user 0m9.353s
sys 0m1.277s
Called on every 64 items:
real 0m7.166s
user 0m9.389s
sys 0m1.117s
Called on every 256 items:
real 0m7.073s
user 0m9.439s
sys 0m1.027s
32 seemed like a good compromise.
- errors in item() didn't propagate properly
- empty [] and {} values weren't allowed
- fractional numbers weren't allowed
- parsing of escaped characters didn't ensure that enough data was in
the buffer
- E() didn't propagate errors properly in all cases
I'll do some more testing later on, but the current code seems to be
working quite well already.
