mirror of
https://github.com/coreos/chunkah.git
synced 2026-11-03 21:49:44 -09:00
ba99f6a80b
The old algorithm used a greedy merge approach (BinaryHeap-based,
minimizing total expected value loss per merge) which was fundamentally
unstable: small changes in input caused completely different merge
decisions, resulting in poor layer reuse across updates.
Additionally, because of its greediness, it easily fell into a local
optimum with a giant catch-all bucket with low stability and safer more
stable items in the other layers.
Replace it with a two-phase statistical partitioning approach inspired
by rpm-ostree's chunking algorithm:
- Phase 1 classifies components by size, giving large components
(linux-firmware, kernel, firefox) their own singleton layers.
- Phase 2 assigns all remaining components to bins using stability tiers
(high/mid/low) and deterministic name-based hashing, which ensures
stable bin membership across builds without needing to track prior
build state.
Also remove the stability fallback that assigned min(known)/2 to
components without stability data (xattr, bigfiles, unclaimed). These
now stay at 0.0 and are naturally handled by the stability tiers.
Benchmarked against rpm-ostree's native chunking on FCOS F43 (11
biweekly stable releases) and Silverblue F43 (3 weekly builds):
FCOS: 46.4% reuse, 916 MiB avg download uncompressed (rpm-ostree: 34.0%, 1.2 GiB)
Silverblue: 47.9% reuse, 2.8 GiB avg download uncompressed (rpm-ostree: 38.5%, 3.5 GiB)
Assisted-by: OpenCode (Claude Opus 4.6)
|
||
|---|---|---|
| .. | ||
| alpm.rs | ||
| bigfiles.rs | ||
| mod.rs | ||
| rpm.rs | ||
| xattr.rs | ||