Commit graph

233 commits

Author SHA1 Message Date
Jonathan Lebon
b23541a811 tools: add release.py for cutting releases
Add a new `tools/release.py` script to help cut releases and hook it up
to `just release`.

Use `--no-push` to prepare the release locally without pushing, allowing
inspection of artifacts before committing to the release.

Release notes are saved to `.release-notes-{version}.md` after editing
so they can be recovered if a later step fails.

Assisted-by: Claude Opus 4.5
2026-01-30 13:10:54 -05:00
Jonathan Lebon
50b6a2bf76 Justfile: convert buildimg's nochunk to proper flag
TIL that `just` supports recipe flags, which seems cleaner for this than
a stringy positional arg.

Assisted-by: Claude Opus 4.5
2026-01-30 13:10:54 -05:00
Jonathan Lebon
edbac85e06 Containerfile: disable weak deps in dnf installs
Let's default to a minimal footprint.

Assisted-by: Claude Opus 4.5
2026-01-30 13:10:54 -05:00
Jonathan Lebon
d7992dc269 Cargo.toml: switch rpm-qa dependency to crates.io
Use the published crates.io version instead of the git repository now
that it's published!

Assisted-by: Claude Opus 4.5
2026-01-30 13:10:54 -05:00
Jonathan Lebon
9c3a20b957 .gitignore: ignore .opencode/plans 2026-01-30 13:10:54 -05:00
Jonathan Lebon
7aefd6f138 .claude: move skill .opencode
Migrate the extract-oci-archive skill to the native OpenCode skills
directory structure. Symlink .claude so it still works with Claude Code.

Assisted-by: Claude Opus 4.5
2026-01-30 13:10:54 -05:00
Jonathan Lebon
d004999c74 components: add bigfiles repo to claim large unclaimed files
Add a new component repo that claims unclaimed files >= 1 MB into
separate standalone components. This solves a conceptual issue in the
unclaimed files logic: by grouping all unclaimed files together, they
can't ever be broken back out into separate layers since the packer
considers each component as one monolithic unit. By breaking out large
files, the packer can choose to merge them back in (or leave them
separate) as it sees fit.

Conceptually, every unclaimed file should be considered separately, but
it's overkill to be this granular so we just filter for >= 1 MB. Files
like initramfs.img and rpmdb.sqlite are good examples of large volatile
files that benefit from being in their own layers.

Hardlinked files (same inode) are grouped into the same component so
that tar can emit them properly as hardlinks.

On FCOS, this drops the unclaimed component from ~165 MB to ~9 MB by
extracting just 5 files totaling 156 MB.

Assisted-by: Claude Opus 4.5
2026-01-28 23:17:16 -05:00
Jonathan Lebon
307a1bba04 components: set unknown stabilities to half the minimum
Instead of using 0.0 for components with unknown stability, use half
the minimum stability of all claimed components with known values. This
ensures unclaimed and xattr content are always considered less stable
than any known component while still having a non-zero value so the
packing algorithm can make meaningful TEV loss calculations when them.

This is also prep for breaking down the unclaimed component into
multiple ones.

Assisted-by: Claude Opus 4.5
2026-01-28 23:17:16 -05:00
Jonathan Lebon
3e9b12a7e4 AGENTS.md: add explicit example for running just one test
This was hard somehow for Opus 4.5 to intuit.
2026-01-28 23:17:16 -05:00
Jonathan Lebon
835efb0263 tools: add profiling and benchmarking justfile
Add `tools/perf/` with Justfile and Containerfile to make it easier
to do basic performance analysis in a realistic setup. Includes two
targets:

- profile: generates `flamegraph.svg` using cargo-flamegraph
- benchmark: runs hyperfine for timing measurements

Both targets build a container with the necessary tools, pull the target
image if missing, and mount it for chunkah to process. I kept the bulk
of the logic in a separate Justfile to keep the main one clean, but did
add forwarding targets for convenience:

    just profile [image]
    just benchmark [image]

Assisted-by: Claude Opus 4.5
2026-01-28 23:17:16 -05:00
Jonathan Lebon
0bf9fb6411 tools: add inspect-layers.sh for analyzing split images
Add a script to inspect chunkah-split images, displaying layer size,
stability, and component information.

Assisted-by: Claude Opus 4.5
2026-01-28 23:17:16 -05:00
Jonathan Lebon
2dde0663bf tests/e2e: add FCOS image splitting test
Having coverage for a bootable container is overdue at this point. Now
that we've fixed RPM path canonicalization, let's do that.

This test still shows there's work to be done in handling the unclaimed
bucket. More to come in that area.

Assisted-by: Claude Opus 4.5
2026-01-28 13:22:38 -05:00
Jonathan Lebon
e6a11933fd rpm: canonicalize RPM paths to match filesystem scan
RPM database stores paths as they were at package build time, which may
use compat symlinks (e.g., /lib/modules/... instead of
/usr/lib/modules/...). This caused ~316 MB of kernel content to go to
"unclaimed" instead of being claimed by rpm/kernel-* packages.

Add path canonicalization to RpmRepo::load() that resolves symlinks in
directory components only before building the path-to-component mapping.
The final component is intentionally not resolved, so that if a file
that is supposed to be a file/directory according to the rpmdb turns out
to be a symlink, the RPM does not claim it.

Assisted-by: Claude Opus 4.5
2026-01-28 13:22:38 -05:00
Jonathan Lebon
8a86cba790 AGENTS.md: remove obsolete information
Missed this when I ripped out the `prescan` command.
2026-01-27 15:17:35 -05:00
Jonathan Lebon
c14dc672bb deps: drop cap-std dep; use cap_std_ext re-export
Remove the direct cap-std v3 dependency and instead use the cap_std
re-export from cap-std-ext v4. That way we guarantee it's always deduped
and also avoid having to maintain that dep separately.

Assisted-by: Claude Opus 4.5
2026-01-27 15:14:31 -05:00
dependabot[bot]
0f2276610d deps: bump the rust-deps group with 6 updates
Bumps the rust-deps group with 6 updates:

| Package | From | To |
| --- | --- | --- |
| [chrono](https://github.com/chronotope/chrono) | `0.4.42` | `0.4.43` |
| [clap](https://github.com/clap-rs/clap) | `4.5.53` | `4.5.54` |
| [flate2](https://github.com/rust-lang/flate2-rs) | `1.1.5` | `1.1.8` |
| [libc](https://github.com/rust-lang/libc) | `0.2.178` | `0.2.180` |
| [serde_json](https://github.com/serde-rs/json) | `1.0.145` | `1.0.149` |
| [tempfile](https://github.com/Stebalien/tempfile) | `3.23.0` | `3.24.0` |


Updates `chrono` from 0.4.42 to 0.4.43
- [Release notes](https://github.com/chronotope/chrono/releases)
- [Changelog](https://github.com/chronotope/chrono/blob/main/CHANGELOG.md)
- [Commits](https://github.com/chronotope/chrono/compare/v0.4.42...v0.4.43)

Updates `clap` from 4.5.53 to 4.5.54
- [Release notes](https://github.com/clap-rs/clap/releases)
- [Changelog](https://github.com/clap-rs/clap/blob/master/CHANGELOG.md)
- [Commits](https://github.com/clap-rs/clap/compare/clap_complete-v4.5.53...clap_complete-v4.5.54)

Updates `flate2` from 1.1.5 to 1.1.8
- [Release notes](https://github.com/rust-lang/flate2-rs/releases)
- [Commits](https://github.com/rust-lang/flate2-rs/compare/1.1.5...1.1.8)

Updates `libc` from 0.2.178 to 0.2.180
- [Release notes](https://github.com/rust-lang/libc/releases)
- [Changelog](https://github.com/rust-lang/libc/blob/0.2.180/CHANGELOG.md)
- [Commits](https://github.com/rust-lang/libc/compare/0.2.178...0.2.180)

Updates `serde_json` from 1.0.145 to 1.0.149
- [Release notes](https://github.com/serde-rs/json/releases)
- [Commits](https://github.com/serde-rs/json/compare/v1.0.145...v1.0.149)

Updates `tempfile` from 3.23.0 to 3.24.0
- [Changelog](https://github.com/Stebalien/tempfile/blob/master/CHANGELOG.md)
- [Commits](https://github.com/Stebalien/tempfile/compare/v3.23.0...v3.24.0)

---
updated-dependencies:
- dependency-name: chrono
  dependency-version: 0.4.43
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: rust-deps
- dependency-name: clap
  dependency-version: 4.5.54
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: rust-deps
- dependency-name: flate2
  dependency-version: 1.1.8
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: rust-deps
- dependency-name: libc
  dependency-version: 0.2.180
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: rust-deps
- dependency-name: serde_json
  dependency-version: 1.0.149
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: rust-deps
- dependency-name: tempfile
  dependency-version: 3.24.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: rust-deps
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-01-27 19:51:34 +00:00
dependabot[bot]
02cfb4ddbb ci: bump actions/checkout from 5 to 6 in the actions group
Bumps the actions group with 1 update: [actions/checkout](https://github.com/actions/checkout).


Updates `actions/checkout` from 5 to 6
- [Release notes](https://github.com/actions/checkout/releases)
- [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md)
- [Commits](https://github.com/actions/checkout/compare/v5...v6)

---
updated-dependencies:
- dependency-name: actions/checkout
  dependency-version: '6'
  dependency-type: direct:production
  update-type: version-update:semver-major
  dependency-group: actions
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-01-27 14:50:27 -05:00
Jonathan Lebon
e33176ef48 ci: add Dependabot with auto-merge for minor/patch updates
Add dependabot.yml to enable weekly dependency updates for Cargo and
GitHub Actions. Minor and patch updates are grouped together to reduce
PR noise, while major updates remain separate for manual review.

Add dependabot-automerge.yml workflow that enables auto-merge (rebase)
for non-major updates after CI passes.

Assisted-by: Claude Opus 4.5
2026-01-27 14:45:30 -05:00
Jonathan Lebon
92f81d21bd build: add --prune flag to exclude paths
Add a --prune flag that allows excluding paths from the rootfs scan (and
thus, the resulting tar layers).

If a directory ends with /*, its contents are excluded but not the
directory itself. No other glob patterns are supported.

My main motivation for this right now is to prune `/sysroot/ostree` in
OSTree-based bootable container images, but it's obviously generally
useful.

Assisted-by: Claude Opus 4.5
2026-01-27 00:18:31 -05:00
Jonathan Lebon
17de0d7506 scan: refactor to use Scanner builder pattern
Introduce a Scanner struct with builder pattern similar to
ocibuilder::Builder. The Scanner owns a cloned Dir and provides
builder methods for configuration before scanning.

This improves ergonomics as more scanning options are added.

Assisted-by: Claude Opus 4.5
2026-01-27 00:18:31 -05:00
Jonathan Lebon
1376e12029 scan: inline scan_for_components into cmd_build
Move the scan_for_components logic directly into cmd_build::run() as
a preparatory step for refactoring the scanner code to use a builder
pattern. That abstraction didn't seem worth it anyway; it wasn't used
in unit tests, and it feels cleaner to not have the scanner module know
about components.

Assisted-by: Claude Opus 4.5
2026-01-27 00:18:31 -05:00
Jonathan Lebon
3496a73166 tools/split-and-diff: flatten multi-layer images before splitting
When the source image has multiple layers, squash it to a single layer
before passing to chunkah. This makes it easier to compare against
chunkah's `--max-layers 1` output.

While we're here, use the `localhost/` prefix for image names we create
to avoid reference resolution issues between skopeo, buildah, and
podman.

Assisted-by: Claude Opus 4.5
2026-01-27 00:18:31 -05:00
Jonathan Lebon
b3f62f929a tests/e2e: move cap handling test to test-buildtime.sh
The cap handling test requires building a derived image, which we're
already doing in `test-buildtime.sh`. So let's move it there.

Assisted-by: Claude Opus 4.5
2026-01-22 00:55:26 -05:00
Jonathan Lebon
396922ec8d tests/e2e: add build-time splitting tests
We need to cover the build-time path, which is going to be very common.
And it allows us to verify support for the new --skip-special-files.

While we're here, rename existing tests to better reflect that they test
splitting existing images.

Assisted-by: Claude Opus 4.5
2026-01-22 00:55:26 -05:00
Jonathan Lebon
0bdf546ef3 tests/e2e: extract shared helpers to lib.sh
Add lib.sh with common helper functions to reduce duplication
and provide consistent validation logic.

Assisted-by: Claude Opus 4.5
2026-01-22 00:55:26 -05:00
Jonathan Lebon
43bcf620bd tests/e2e: add docstrings to tests missing them 2026-01-22 00:55:26 -05:00
Jonathan Lebon
a681bf4e3e build: add --skip-special-files flag
Currently, we error out when special files are encountered. This is by
design; chunkah aims to stay true to the input rootfs it's being fed.
Ideally unsupported content should be deleted upfront.

That said, it definitely makes sense to provide an easy escape hatch.
This patch adds --skip-special-files which silently skips them instead
of erroring.

Closes: https://github.com/jlebon/chunkah/issues/2

Assisted-by: Claude Opus 4.5
2026-01-22 00:55:26 -05:00
tangowithfoxtrot
6e506b5f4a fix(docs): unquoted docker inspect strings
if left unquoted, the `CHUNKAH_CONFIG_STR` will just be the last line of the JSON output (`]`), breaking the usage in the examples
2026-01-20 12:39:09 -05:00
Jonathan Lebon
f75b1d5d2b
ci: don't try to support pushing to the dev registry from PRs
It doesn't work because forks don't have write access to the package
registry, causing the push step to fail. Oh well.
2026-01-20 12:31:43 -05:00
Jonathan Lebon
3499e471cb
tar: preserve hardlinks for symlinks within the same layer
Symlinks can also be hardlinked, but the hardlink detection logic was
only applied to regular files. This extends the hardlink tracking to
handle symlinks as well.

Assisted-by: Claude Opus 4.5
2026-01-20 11:23:23 -05:00
Jonathan Lebon
0b5a4b9f3a
README.md: update xattr repo section to reflect reality
We switched to always reading xattrs upfront because it's metadata we
want to persist anyway.
2026-01-20 10:33:27 -05:00
Jonathan Lebon
704f7ad94c
tools/split-and-diff: add --no-cleanup option
This option skips removing the original and split images from
containers-storage on exit, which is useful for debugging or inspecting
the images after the script runs.

Assisted-by: Claude Opus 4.5
2026-01-19 20:37:19 -05:00
Jonathan Lebon
11ca5eb66a
tree: initial commit 2026-01-19 15:59:23 -05:00