Unfortunately, there wasn't a single bit free in struct dir.flags, so I
had to increase its size to 16 bit. This commit is just the initial
preparation, there's still a few things to do:
- Add "extended information" cli flag to enable/disable this
functionality.
- Export and import extended information when requested
- Do something with the data.
I also did a few memory measurements on a file list with 12769842 items:
before this commit: 1.239 GiB
without extended info: 1.318 GiB
with extended info: 1.698 GiB
It's surprising what adding a single byte to a struct can do to the
memory usage. :(
Fixes https://dev.yorhel.nl/ncdu/bug/103
I don't think a stack overflow as a result of recursion is exploitable
on a modern system. It should just result in an unfortunate write to a
page that is not writable, followed by a crash.
This should fix https://dev.yorhel.nl/ncdu/bug/99 - with the downside
that this requires a C99 compiler.
I also replaced all occurrences of static allocation of struct dir with
use dynamic allocation, because I wasn't really sure if static
allocation of flexible structs is allowed. In the case of dirlist.c the
dynamic allocation is likely required anyway, because it does store a
few bytes in the name field.
Turns out that being able to open an empty directory actually has its
uses:
- If you delete the last file in a directory, you now won't be directed
to the parent directory anymore. This allows keeping 'd' pressed
without worrying that you'll delete stuff outside of the current dir.
(This is the primary motivation for doing this)
- You can now scan and later refresh an empty directory, as suggested by
#2 in http://dev.yorhel.nl/ncdu/bug/15
Some measurements importing a gzip-compressed file (zcat .. | ncdu -f -)
containing a bit under 6 million items and a few choices of how often to
call input_handle():
Called on every item:
real 0m13.745s
user 0m12.576s
sys 0m4.566s
Called on every 8 items:
real 0m7.932s
user 0m9.636s
sys 0m1.623s
Called on every 16 items:
real 0m7.559s
user 0m9.553s
sys 0m1.323s
Called on every 32 items:
real 0m7.279s
user 0m9.353s
sys 0m1.277s
Called on every 64 items:
real 0m7.166s
user 0m9.389s
sys 0m1.117s
Called on every 256 items:
real 0m7.073s
user 0m9.439s
sys 0m1.027s
32 seemed like a good compromise.
- errors in item() didn't propagate properly
- empty [] and {} values weren't allowed
- fractional numbers weren't allowed
- parsing of escaped characters didn't ensure that enough data was in
the buffer
- E() didn't propagate errors properly in all cases
I'll do some more testing later on, but the current code seems to be
working quite well already.
