This makes way more sense, nobody who builds a local image wants to use
a potentially older fedora release, especially on the same branch, this
is also a mismatch between what is built in CI as that builds from a
stable branch instead of main.
This is faster and less resource intensive. It usually takes a couple
seconds on my machine.
We only care about the label on that one image, and not the 3000 other
images in the ublue-os/akmods namespace.
* feat: bump testing to Fedora 45
We are using docker from their repo because it usually ships newer
versions faster than Fedora. In the meantime we can just ship Fedora's
docker version which is good enough.
xref: https://github.com/ublue-os/aurora/issues/2758
* renovate
* fix: cleanup hidden files in /tmp
Something specific to the DX build creates a bunch of socket files
(/tmp/.{ICE,X11,XIM,font}-unix) now. I have no idea what it is, it is
not the new docker package.
* feat: make compatible with unsealed composefs
The bootc package ships a dracut configuration that adds the bootc
module (which is needed for the installation of the composefs backend)
to the initramfs which we currently don't have and ostree is also set
there.
I want this because bootc install is significantly faster when used with
the composefs backend, because it does some magic zeor-copy reflink
things. It is wired up with `just disk-image --backend composefs`.
For a long time systemd-boot was the only supported bootloader on
composefs backend, but grub works now. So we don't have to unnecessarily
add systemd-boot to the image.
* use grub for composefs
This format is a little bit more verbose, this will immediately make it
obvious in CI from which "branch" from common we are getting the image.
Same goes for chunkah if we ever use the dev tag for some nice upcoming
features.
Should help with reproducibility and might make updates smaller for
unpackaged contents like wallpapers, homebrew.
Only downside so far that I see is that `podman images` and the like
will show up at the bottom with 56 years ago and so on. Labels like
`org.opencontainers.image.created` are of course not affected by this.
xref: https://github.com/ublue-os/aurora/issues/2350 and
https://github.com/ublue-os/aurora/issues/2579
When you make a PR to aurora on a Sunday before the daily renovate bump
kicked off (POINT=1) (before 4AM UTC) then this run would not have any
cache.
PRs always should have cache.
* feat(ci): setup runner with containers policy
Currently for ublue-os/akmods and kinoite (not pinned by digest), we
verify them and pull them right after. We can't trust the authenticity
of those images, as the tag in the meantime could point to a different,
potentially compromised digest.
To avoid this, we set up our runner so that it will refuse to pull
anything from Universal Blue or quay.io/fedora-ostree-desktops without
first getting verified with the corresponding public key in our repo.
With this, we can completely get rid of our verify-container recipe and
benefit from the retry logic that is implemented in the podman pulls we
already do beforehand.
This would mean that local builds would no longer verify the pulled
images, as the policy is very likely not configured, but I don't think
that is super important.
* fix: install cosign in non-production builds
* feat(ci): package cache with OCI artifacts
We are hitting a couple limitations in regards to actions/cache [1] with
our new testing branch workflow, as we previously relied on scheduled
builds from the main branch, which we no longer do with this model.
Now we make new cache on merge_group on the first build on Sundays, I
chose Sundays just because. Which means we run only a single build with
no cache per week and use that same build to generate a new fresh cache.
Else we would end up with 3 versions of vscode in our cache.
There is no point uploading cache on *every* PR.
For simplicity sake this has no handling/differentiation for our stable
and main branch. When we bump the fedora version of our main/testing
images then the cache for stable will just get stale.
We are mixing root/rootless usage of oras/cosign, I made it use root or
else we will get premission issues.
[1]: https://github.com/actions/cache/issues/1537
fixes: https://github.com/ublue-os/aurora/issues/2351
* define blessed day
* chore: remove command -v prefixes
not needed, running everything rootless now
* feat(ci): rootless CI
26.04 runners use sudo-rs, which does not support this flag. It doesn't
seem to have broken too much, Only thing I could see is the github token
not being passed to the build.
```
sudo: preserving the entire environment is not supported, '-E' is ignored
```
So this is a good time to move over to a rootless CI I guess.
* chore: unset target-dir for container-storage-action
This defaults to the user container storage
If you specified --kernel-pin from cli it will now correctly verify and
donwload the right version (instead of the newest one), if it's
specified in the Justfile and CLI then CLI wins.
The pin that is always set to an empty string would override it.
* refactor(ci): initial work for using flag/arg-based workflow
The goal of this is to make the buildsystem function like any other CLI
program. The only recipes I haven't touched in this PR are the very
simple recipes like verify-container, retag-nvidia-ghcr and
login-registry. In the future I want to split up this Justfile in
multiple smaller ones where it feels right to do that.
resolves: https://github.com/ublue-os/aurora/issues/2436
* document minimum just version
* build recipe
* feat: redo the way we do kernel pins
This allows an independent pinning of kernels for future ARM builds
regardless of x86. Probably the way we should have done it from the
beginning.
* get rid of build-ghcr and build-pipeline related things
We are not using the build-pipeline recipe anywhere right now. I think a
build-rechunk recipe is good enough for now.
I think build-ghcr doesn't make any sense anymore as it's now just a
single additional flag. The rechunk recipe with chunkah currently
doesn't alter any behavior when run in ghcr so we can delete this. But
this will likely change when chunkah implements some form of passing a
previous manifest.
I want to do a recipe at some point which replaces the general idea of
the pipeline recipe and is as close to github as possible. Build ->
Rechunk -> Secureboot-> tag + push an image to a local registry bootc
install.
* do deprecated build-chunked-oci recipe
* remove build-pipeline parts of ostree-rechunk
* export oci
* load rootful recipe
currently not used in CI as we are running everything as root.
* secureboot
* generate-point
* gen-sbom
* package cache
* bootc install
* some remaining shebangs
best practice to use env bash here instead
* push recipe
* redo generate-build-tags and tag-images
I removed the version input here. This has been broken for probably a
very long time, this didn't do anything for months at least, oldest log
I could find here:
```
Error: no such object: "aurora:latest"
Tags for this Action...
latest latest-43.20260424 latest-20260424 43 43-43.20260424 43-20260424
```
Kernel input is now done through just, so no reason to pass that, no
idea why we had that here in the first place.
* fix kernel pin for build-rechunk
This convenience recipe needs to take all the options from the build one
This is probably the way we should be doing it instead of having a
separate pull, also works for the other images, mainly useful for akmods
and kinoite as we do not pin those by a digest. Which we should be doing
at some point.
In the future we probably want to have some kind of mechanism that
ensures that we are only pulling the blessed verified image (configuring
the runner environment with a policy).
Currently we are verifying the tag and then pulling the image a short
while later. This is not very likely to happen but leaves a couple
seconds of room where an untrusted image could be published that we
did not verify the authenticity of.
This ties a little bit into [1].
[1]: https://github.com/ublue-os/aurora/issues/2381
* chore: Revert "chore: delete bootc install config (#1568)"
This is one of the things we need for bcvk to work with our images.
This reverts commit 5f76360321.
* chore: drop filesystem option for bootc install
This should minimize the network flakes we run into when we have to pull
images as part of the many `FROM`s, the default is 3 tries with 2s
delay, which is really low.
I'm not 100% sure it will help us, let's just try it!
See: https://github.com/ublue-os/aurora/issues/2337
We are not using docker in CI and I don't think it's worth the
complexity to keep (rather poorly) supporting builds with docker. Bootc
and Fedora only test/use the podman/buildah ecosystem so we should
probably do the same.
Docker prefixes images with docker.io instead of localhost like podman
does if you don't specific a tag prefix, so we can drop those prefixes
everywhere.
I'm sure that some things are already broken to some degree with docker
as I don't use it at all to test the various recipes.
I'm keeping the PODMAN variable to make it easy to at least replace the
path of the podman binary, compiling podman from source, homebrew...
* chore: simplify container pinning for common brew
This is just less complex and fully uses image-versions.yml
* make akmods_flavor a separate recipe
Helps in further splitting up the big monolith that is the build recipe.
* set right just group
* set full name for alpine image
This recipe doesn't actually work yet without any changes to the image
because we don't add the bootc module to the initramfs and don't install
systemd-boot.
This makes it easier to test the composefs native backend.
Chunkah is actively maintained and the algorithm seems to be more stable
for the layer plan which results in smaller sizes for weekly updates
compared to rpm-ostree. Although it looks like daily updates are
slightly bigger for the time being.
In the future there may be a way to consume previous build manifests as
well to further shrink down image sizes.
I am not deleting the old build-chunked-oci code yet, to make this PR
easier to revert in case we have to.
Rpm-ostree is basically the only blocker for rootless CI, which is
generally what we should be doing, PolP and all that. Now we only need
root privileges for bootc install. Althought to keep this PR more simple
I will implement this later.
It seems like bootc install needs more space now, not sure exactly why
that happens but seems to be related due to pruned /sysroot.
xref https://github.com/ublue-os/aurora/issues/2324
Co-authored-by: inffy <10782843+inffy@users.noreply.github.com>
This label was previously created by the hhd/legacy-rechunker instead of
on the image build process. We are now using the build-chunked-oci and
in the near future chunkah which do not mess with the labels.
Our changelogs script expects this label to be present in order for the
commits since last release to show up.
fixes: https://github.com/ublue-os/aurora/issues/2338
Currently we are tagging these:
```
latest
latest-44.20260531
latest-20260531
latest-44
44-44.20260531
44-20260531
```
and this PR will add:
```
latest-44.20260531.2
latest-20260531.2
44-44.20260531.2
44-20260531.2
```
and for stable:
```
stable
stable-44.20260531
stable-20260531
stable-daily
44.20260531
stable-daily-44.20260531
stable-daily-20260531
```
```
stable-44.20260531.1
stable-20260531.1
stable-daily-44.20260531.1
stable-daily-20260531.1
```
Today there has been no stable image released so it starts out with .1
but there has been a single latest image, so it bumps it to .2
Very much inspired by ublue-os/main's solution [1], but this
implementation has a bug where where it will find the first "missing"
POINT (not the highest!) in the tags.
This would falsely choose the wrong POINT in the case we delete a bad
image and there is a "gap".
For example, we have the images .1-3 already on the registry, then
delete .2 and this implementation would correctly choose the next image
as .4 (instead of .2) as it's not just a dumb while loop.
This is very much an edge case and makes the jq command quite cursed but
I think this should always be reliable.
This feature has been broken previously because we didn't tag the image
with an index point number so no matter how many images we have built
that day, it was always .1 in the various VESRION fields in os-release.
fixes: https://github.com/ublue-os/aurora/issues/1833
[1]: ef33e7ca96/Justfile (L239-L251)
The ubuntu 26.04 provided version is still too old, recently they
stabilized the --fmt flag [1]. This is consistent with the other
invokations and also allows us to have some whitespace between variable
definitions for example as they seemed to have changed that.
[1]: 21b3833bf0
* fix(ci): push production tags after signing
If the signing step fails, causes for this can be ghcr jank or a
sigstore outage and not testing our changes properly beforehand. Then we
have already pushed images to the registry with production tags that
users can update to.
The clients won't actually update to these images as we enforce
verification on the client side to pull images from ghcr.io/ublue-os/
only with a valid signature attached. Otherwise this leads to an "update
failed" on the users POV, which is very confusing. This has been a
longstanding issue across the org [1] [2] [3].
To workaround this, we push a tag to the registry first that we don't
care about and clients don't check against for updates. I chose the job
id because it always changes.
This also happens to get rid of the action we use for retries to
workaround ghcr jank, we were forced to do this in the past because the
old podman version we used at the time didn't have those features.
Using podman to push signatures and images at the same time is not
something that is viable at the moment unfortunately but is definitely
the approach we should be using once it works well.
This effort can be seen at [4]. Although we are likely hitting [5] here
where all layers get invalidated because of this bug. We are effectively
pushing full images/no shared layers every time we do podman push. Which
causes the overall push step in CI to take upwards of 20 minutes.
fixes: https://github.com/ublue-os/aurora/issues/2135
[1]: https://github.com/ublue-os/bazzite/pull/4902
[2]: https://github.com/ublue-os/main/issues/643
[3]: https://github.com/ublue-os/bazzite/issues/3044
[4]: https://github.com/containers/podman/issues/27796
[5]: https://github.com/ublue-os/aurora/pull/2199
Signed-off-by: renner <renner0@posteo.de>
* fix: cosign verify
It seems like cosign tries to verify all .sig files on the registry for
that digest, one of which is generated by cosign sign with a static key
and one is generated by actions/attest using an OIDC.
This exactly broke, which is what we have been using for the longest
time:
```
cosign verify --key cosign.pub ghcr.io/ublue-os/aurora:latest
Error: no matching attestations: expected key signature, not certificate
error during command execution: no matching attestations: expected key
signature, not certificate
```
And this currently works to verify the image, although it is of course
not the same thing as singing with the ublue cosign key.
This will not work going forward after this commit:
```
cosign verify \
--certificate-oidc-issuer https://token.actions.githubusercontent.com \
--certificate-identity-regexp="github.com/ublue-os/aurora/.github/workflows/*" \
ghcr.io/ublue-os/aurora:latest
```
Signed-off-by: renner <renner0@posteo.de>
* feat: use staging for the temporary tag
This is just a minor cosmetic thing and recudes the amount of tags we
have because it's static and not changing unlike the run id.
Signed-off-by: renner <renner0@posteo.de>
* fix: use empty string by default for temporary tag
Signed-off-by: renner <renner0@posteo.de>
* fix: don't use signing config
Or else we are getting this error with new cosign 3.X.X
```
Error: must provide --new-bundle-format or --bundle where applicable
with --signing-config or --use-signing-config error during command
```
Signed-off-by: renner <renner0@posteo.de>
* chore: bump cosign-installer to v4.1.2
* chore: move oras install to beginning as well
if this fails at the beginning because of jank then it's not so bad
* fix(ci): make sbom signing work
```
Error: error getting signing config from TUF: error setting TUF options:
error setting TUF mirror: error reading remote.json: open
/home/runner/.sigstore/root/remote.json: permission denied
```
---------
Signed-off-by: renner <renner0@posteo.de>
* feat(ci): stop building stable-daily images
These do not make super much sense, this is in preparation for the
testing branches so we don't build too many images.
We will still build stable images in PRs to find out if things are
broken on those images only.
To not break existing users, point this to `:stable` instead,
effectively lying about the cadence now. We have to do it this way
because stable-daily has ZFS and `:latest` doesn't to avoid breaking
that use case. In the future when we have removed ZFS we can point this
to `:latest`.
* fix: only publish :stable on schedule
We do not want to publish stable on every PR merge
---------
Co-authored-by: inffy <10782843+inffy@users.noreply.github.com>
