mirror-ublue-os-akmods/README.md
Benjamin Sherman 95d16290f1
feat(ci): skip rebuilding unchanged akmods images (#620)
Every nightly rebuilds all 33 targets even when nothing changed. Each
rebuild is another chance to hit a mirror flake, and published images
record nothing about what they were built from.

Label each image with a fingerprint of its inputs: the kernel release,
the git objects that feed the build, the target config, and overrides.
The build job skips test, build, and push when the published image
matches, is signed, and is under AKMODS_MAX_AGE_DAYS old (default 7).
The manifest job likewise skips regenerating indexes that are already
current and signed, but attestation still runs every time. Any doubt
means build. PR runs and the new force input bypass the guard.

NVIDIA builds pin akmod-nvidia to the fingerprinted version, and the
userspace packages to the same driver version, so the label describes
what was actually built.
2026-10-25 05:21:52 +00:00

291 lines
22 KiB
Markdown

# ublue-os akmods
[![OpenSSF Scorecard](https://api.scorecard.dev/projects/github.com/ublue-os/akmods/badge)](https://scorecard.dev/viewer/?uri=github.com/ublue-os/akmods)[![Build COREOS-STABLE akmods](https://github.com/ublue-os/akmods/actions/workflows/build-akmods-coreos-stable.yml/badge.svg)](https://github.com/ublue-os/akmods/actions/workflows/build-akmods-coreos-stable.yml)[![Build COREOS-TESTING akmods](https://github.com/ublue-os/akmods/actions/workflows/build-akmods-coreos-testing.yml/badge.svg)](https://github.com/ublue-os/akmods/actions/workflows/build-akmods-coreos-testing.yml)[![Build LONGTERM-6.18 akmods](https://github.com/ublue-os/akmods/actions/workflows/build-akmods-longterm-6.18.yml/badge.svg)](https://github.com/ublue-os/akmods/actions/workflows/build-akmods-longterm-6.18.yml)[![Build OGC akmods](https://github.com/ublue-os/akmods/actions/workflows/build-akmods-ogc.yml/badge.svg)](https://github.com/ublue-os/akmods/actions/workflows/build-akmods-ogc.yml)[![Build OGC-LTS akmods](https://github.com/ublue-os/akmods/actions/workflows/build-akmods-ogc-lts.yml/badge.svg)](https://github.com/ublue-os/akmods/actions/workflows/build-akmods-ogc-lts.yml)[![Build MAIN akmods](https://github.com/ublue-os/akmods/actions/workflows/build-akmods-main.yml/badge.svg)](https://github.com/ublue-os/akmods/actions/workflows/build-akmods-main.yml)
OCI images providing a set of cached kernel RPMs and extra kernel modules to Universal Blue images. Used for better hardware support and consistent build process.
## How it's organized
The [`akmods` images](https://github.com/orgs/ublue-os/packages?repo_name=akmods) are built and published daily. However, there's not a single image but several, given various kernels we now support.
The akmods packages are divided up for building in a few different "groups":
- `common` - any kmod installed by default in Aurora/Bazzite/Bluefin (or were originally in main images pre-Fedora 39)
- `extra` - less common kmods used by only one Universal Blue project.
- `nvidia` - only the nvidia proprietary kmod and addons
- `nvidia-open` - only the nvidia-open kmod and addons
- `zfs` - only the zfs kmod and utilities built for select kernels
Each of these images contains a cached copy of the respective kernel RPMs compatible with the respective kmods for the image.
Builds also run for different kernels:
- `main` - Mainline Fedora Kernel
- `coreos-stable` - Current Fedora CoreOS stable kernel version
- `coreos-testing` - Current Fedora CoreOS testing kernel version
- `Longterm-6.18` - Fedora Kernel on Kernel 6.18 LTS
- `ogc` - [Open Gaming Collective](https://github.com/OpenGamingCollective/kernel-packages-fedora) kernel
- `ogc-lts` - Open Gaming Collective kernel, LTS branch (`:lts` tag)
See `images.yaml` for which akmods packages are built for each Kernel
## Features
### Overview
The `common` images contain related kmod packages, plus:
- `ublue-os-akmods-addons` - installs extra repos and our kmods signing key; install and import to allow SecureBoot systems to use these kmods
The `nvidia` and `nvidia-open` images contains
- `ublue-os-nvidia-addons` - installs extra repos enabling our nvidia support
- [nvidia container selinux policy](https://github.com/NVIDIA/dgx-selinux/tree/master/src/nvidia-container-selinux) - uses RHEL9 policy as the closest match
- [nvidia-container-toolkit repo](https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/latest/install-guide.html#installing-with-yum-or-dnf) - version 1.14 (and newer) provide CDI for podman use of nvidia gpus
- `ublue-os-ucore-nvidia` - a slightly lighter `ublue-os-nvidia-addons` for CoreOS/uCore systems
### Kmod Packages
| Group | Package | Description | Source |
|-------|---------|-------------|--------|
| common | [openrazer](https://openrazer.github.io/) | kernel module adding additional features to Razer hardware | [Terra](https://github.com/terrapkg/packages) |
| common | [v4l2loopback](https://github.com/umlaeute/v4l2loopback) | allows creating "virtual video devices" | [RPMFusion - free](https://rpmfusion.org/) |
| common | [wl](https://github.com/rpmfusion/broadcom-wl/) | support for some legacy broadcom wifi devices | [RPMFusion - nonfree](https://rpmfusion.org/) |
| common | [xone](https://github.com/dlundqvist/xone) | xbox one controller USB wired/RF driver modified to work along-side xpad (built from [xonedo](https://github.com/OpenGamingCollective/xonedo) fork) | [![badge](https://copr.fedorainfracloud.org/coprs/ublue-os/akmods/package/xone-kmod/status_image/last_build.png)](https://copr.fedorainfracloud.org/coprs/ublue-os/akmods/package/xone-kmod) |
| common | [xpadneo](https://github.com/atar-axis/xpadneo) | xbox one controller bluetooth driver | [negativo17 - fedora-multimedia](https://negativo17.org/) |
| extra | [ayaneo-leds](https://github.com/TiPSilva/ayaneo-leds) | AYANEO joystick ring RGB LEDs (legacy EC models) | [Terra](https://github.com/terrapkg/packages) |
| extra | [ayaneo-platform](https://github.com/ShadowBlip/ayaneo-platform) | AYANEO x86 handheld RGB LED control and bypass charging | [Terra](https://github.com/terrapkg/packages) |
| extra | [evdi](https://github.com/DisplayLink/evdi) | DisplayLink USB graphics adapter support | [negativo17 - fedora-multimedia](https://negativo17.org/) |
| extra | [gcadapter_oc](https://github.com/hannesmann/gcadapter-oc-kmod) | Gamecube controller adapter overclocking | [![badge](https://copr.fedorainfracloud.org/coprs/ublue-os/akmods/package/gcadapter_oc-kmod/status_image/last_build.png)](https://copr.fedorainfracloud.org/coprs/ublue-os/akmods/package/gcadapter_oc-kmod) |
| extra | [hid-fanatecff](https://github.com/gotzl/hid-fanatecff) | Fanatec wheel base force feedback | [Terra](https://github.com/terrapkg/packages) |
| extra | [hid-tmff2](https://github.com/Kimplul/hid-tmff2) | Thrustmaster T300RS, T248, TX, T128, T598, T-GT II, TS-XW force feedback | [Terra](https://github.com/terrapkg/packages) |
| extra | [kvmfr](https://github.com/gnif/looking-glass) | KVM framebuffer relay kernel module for use with Looking Glass | [![badge](https://copr.fedorainfracloud.org/coprs/hikariknight/looking-glass-kvmfr/package/kvmfr-kmod/status_image/last_build.png)](https://copr.fedorainfracloud.org/coprs/hikariknight/looking-glass-kvmfr/package/kvmfr-kmod) |
| extra | [logitech-trueforce](https://github.com/mescon/logitech-trueforce-linux-driver) | Logitech RS50, G PRO, and G923 force feedback and TrueForce support | [Terra](https://github.com/terrapkg/packages) |
| extra | [nct6687d](https://github.com/Fred78290/nct6687d) | Linux kernel module for Nuvoton NCT6687D/NCT6687-R motherboard fan control | [Terra](https://github.com/terrapkg/packages) |
| extra | [new-lg4ff](https://github.com/berarma/new-lg4ff) | Logitech force feedback wheels | [Terra](https://github.com/terrapkg/packages) |
| extra | [ryzen-smu](https://github.com/leogx9r/ryzen_smu) | AMD Ryzen SMU (System Management Unit) driver | [![badge](https://copr.fedorainfracloud.org/coprs/ublue-os/akmods/package/ryzen-smu-kmod/status_image/last_build.png)](https://copr.fedorainfracloud.org/coprs/ublue-os/akmods/package/ryzen-smu-kmod) |
| extra | [sc0710](https://github.com/Nakildias/sc0710) | Elgato 4K60 Pro MK.2 / 4K Pro capture card driver | [Terra](https://github.com/terrapkg/packages) |
| extra | [system76](https://github.com/pop-os/system76-driver) | System76 laptop hardware support | [![badge](https://copr.fedorainfracloud.org/coprs/ssweeny/system76-hwe/package/system76-driver-kmod/status_image/last_build.png)](https://copr.fedorainfracloud.org/coprs/ssweeny/system76-hwe/package/system76-driver-kmod) |
| extra | [system76-io](https://github.com/pop-os/system76-io-dkms) | System76 Thelio I/O board driver | [![badge](https://copr.fedorainfracloud.org/coprs/ssweeny/system76-io/package/system76-io-kmod/status_image/last_build.png)](https://copr.fedorainfracloud.org/coprs/ssweeny/system76-io/package/system76-io-kmod) |
| extra | [t150-driver](https://github.com/scarburato/t150_driver) | Thrustmaster T150 steering wheel | [Terra](https://github.com/terrapkg/packages) |
| extra | [zenergy](https://github.com/BoukeHaarsma23/zenern) | AMD Zen energy monitoring driver | [Terra](https://github.com/terrapkg/packages) |
| nvidia-open | [nvidia](https://negativo17.org/nvidia-driver/) | nvidia-open GPU drivers | [negativo17 - fedora-nvidia](https://negativo17.org/) |
| zfs | [zfs](https://github.com/openzfs/zfs) | OpenZFS advanced file system and volume manager | [zfs](https://github.com/openzfs/zfs) |
## Notes
### NVIDIA Hardware Support
We build both the open and closed drivers from NVIDIA. The open driver is the only option for supporting the latest hardware. The closed driver is required for older hardware, but even it doesn't support "legacy" hardware.
`nvidia-open` - newest current and most open driver supports the following hardware:
- GeForce RTX: 50 Series, 40 Series, 30 Series, 20 Series
- GeForce: 16 Series
- and more: [NVIDIA Compatible GPUs list](https://github.com/NVIDIA/open-gpu-kernel-modules#compatible-gpus)
`nvidia` - closed proprietary driver supports the following hardware:
- GeForce RTX: 40 Series, 30 Series, 20 Series
- GeForce: 16 Series, 10 Series, 900 Series, 700 Series
- NVIDIA Turing: T4, T4G
- NVIDIA Volta: V100
- NVIDIA Pascal: Quadro: P2000, P4000, P5000, P6000, GP100; Tesla: P100, P40, P4
- NVIDIA Maxwell: Quadro: K2200, M2000, M4000, M5000, M6000, M6000 24GB; - - Tesla: M60, M40, M6, M4
### Pinning the coreos-stable kernel
`coreos_stable_kernel_pin` in the `Justfile` is a temporary ceiling on the kernel used for **every** `coreos-stable` target (`common`, `nvidia-lts`, `nvidia-open`, `zfs`). It exists so publication can continue when Fedora CoreOS stable ships a kernel that OpenZFS cannot yet build.
Empty `''` means follow upstream `fedora-coreos:stable`. A value like `'6.17.12'` is a `major.minor.patch` cap applied in `get-kernel-version` for `coreos-stable` only. Newer detected kernels are rewritten to that pin plus the detected release suffix, then verified in Koji. Kernels at or below the pin pass through.
Use the pin when **all** of these are true:
1. `coreos-stable` ZFS builds are **reliably** failing, not flaking.
2. The failure is kernel-version incompatibility: OpenZFS `META` `Linux-Maximum` is below the current CoreOS stable kernel, or `configure` rejects the kernel as unsupported.
3. Waiting for an OpenZFS release would keep the entire `coreos-stable` check job red, blocking publication of the other targets (and kernel CVE / nvidia updates) to downstream stable images such as `ucore:stable`.
Do **not** use the pin for transient CI or network flakes, failures that are not “kernel newer than OpenZFS supports,” `coreos-testing` (set `zfs.linux_experimental` in `images.yaml` if you intentionally want to track newer kernels), or as a long-term fork of the CoreOS stable kernel. Prefer the pin over inventing local OpenZFS kernel-compat backports when the only goal is to restore publication. Carry an already-upstreamed ZFS fix only when the team has explicitly decided to do that.
To apply the pin:
1. Read the last successful `major.minor.patch` from a published image (`ostree.linux` on `ghcr.io/ublue-os/akmods-zfs:coreos-stable-<ver>`) or from a prior `build/*/KCWD/rpms/cache.json`.
2. Set `coreos_stable_kernel_pin := 'X.Y.Z'` in the `Justfile`, then commit and merge.
3. Clear it back to `''` as soon as ZFS builds on the current CoreOS stable kernel.
Downstream stable images keep the older kernel until the pin is cleared. That is intentional: operations continue, but new kernel CVEs in the unpinned CoreOS kernel wait.
A `coreos-stable` ZFS job that fails for this unsupported-kernel reason prints a pointer to this section.
## Usage
To install one of these kmods, you'll need to install any of their specific dependencies (checkout the `build-prep.sh` and the specific `build-FOO.sh` script for details), and ensure you are on a compatible kernel.
Using common images as an example, add something like this to your Containerfile, replacing `TAG` with the appropriate tag for the image:
COPY --from=ghcr.io/ublue-os/akmods:TAG / /tmp/akmods-common
RUN find /tmp/akmods-common
## optionally install remove old and install new kernel
# dnf -y remove --no-autoremove kernel kernel-core kernel-modules kernel-modules-core kernel-modules-extra
## install ublue support package and desired kmod(s)
RUN dnf install /tmp/rpms/ublue-os/ublue-os-akmods*.rpm
RUN dnf install /tmp/rpms/kmods/kmod-v4l2loopback*.rpm
For NVIDIA images, add something like this to your Containerfile, replacing `TAG` with the appropriate tag for the image:
COPY --from=ghcr.io/ublue-os/akmods-nvidia:TAG / /tmp/akmods-nvidia
RUN find /tmp/akmods-nvidia
## optionally install remove old and install new kernel
# dnf -y remove --no-autoremove kernel kernel-core kernel-modules kernel-modules-core kernel-modules-extra
## install ublue support package and desired kmod(s)
RUN dnf install /tmp/rpms/ublue-os/ublue-os-nvidia*.rpm
RUN dnf install /tmp/rpms/kmods/kmod-nvidia*.rpm
*Note* There is an nvidia-install script that most universal-blue images use located with the *main* repo that works for desktop Fedora installs.
## Verification
### Image Signatures
All architecture images and manifest indexes are signed by Podman with the shared Universal Blue static key. Prefer verifying immutable, digest-pinned subjects with Cosign v3.1.2:
```bash
cosign verify \
--key https://raw.githubusercontent.com/ublue-os/main/main/cosign.pub \
ghcr.io/ublue-os/akmods@sha256:DIGEST
```
For historical manifest digests with registry-attached GitHub provenance, use legacy bundle discovery:
```bash
cosign verify \
--new-bundle-format=false \
--key https://raw.githubusercontent.com/ublue-os/main/main/cosign.pub \
ghcr.io/ublue-os/akmods@sha256:DIGEST
```
The `ublue-os/main/cosign.pub` key is the Universal Blue image trust root. Mutable tags such as `main-44` or `coreos-stable-44` are convenient for discovery, but a digest is the preferred verification target.
### Build Provenance
GitHub build provenance is a separate trust decision from image signatures: it uses a short-lived GitHub OIDC certificate rather than the Universal Blue static key. Provenance currently covers the release and kernel-version manifest indexes, not architecture-only image digests.
```bash
gh attestation verify \
oci://ghcr.io/ublue-os/akmods@sha256:DIGEST \
--repo ublue-os/akmods
```
Provenance is stored in GitHub's attestations API. It is no longer published as a registry referrer, so registry provenance and GitHub linked-artifact storage records are not available for newly published digests.
## Local Building/Testing
You can build these akmods locally with our test keys using the included `Justfile`. We strongly recommend using the provided devcontainer which contains all dependencies for building this project.
### Using the Devcontainer
The repo's `.devcontainer.json` points at `ghcr.io/ublue-os/devcontainer:latest`, a public prebuilt image with `just`, `podman`, `yq`, `jq`, and everything else the `Justfile` needs, so you don't have to install build dependencies on your host. Pulling it requires no authentication.
**With VS Code:** install the [Dev Containers extension](https://marketplace.visualstudio.com/items?itemName=ms-vscode-remote.remote-containers), open this repo, then run "Dev Containers: Reopen in Container" from the Command Palette. Once it's open, use the integrated terminal exactly as you would locally (e.g. `just build`).
**From the command line (no VS Code):** install the [devcontainer CLI](https://github.com/devcontainers/cli) (`npm install -g @devcontainers/cli`), then start/exec into the container with Podman as the backend. A few aliases make this painless:
```bash
alias dcup='devcontainer up --docker-path=podman --workspace-folder .'
alias dcexec='devcontainer exec --docker-path=podman --workspace-folder .'
alias dcjust='devcontainer exec --docker-path=podman --workspace-folder . just'
alias dcpodman='devcontainer exec --docker-path=podman --workspace-folder . podman'
```
```bash
dcup # build and start the devcontainer (run once, or after changing .devcontainer.json)
dcjust build # run `just build` inside the devcontainer
dcpodman images # run an arbitrary podman command inside the devcontainer
```
### How to Use the Justfile
To build an akmods package, run the following:
```bash
just build
```
Since nothing additional was set. The following will occur. The build scripts will determine the current fedora kernel version, download the RPMs, and sign the kernel with the test key. It will then build the common set of akmods. To modify what gets built, modify the following environment variables:
- AKMODS_KERNEL - The kernel flavor you are building
- AKMODS_VERSION - The release version
- AKMODS_TARGET - The akmods package to build
- ZFS_VERSION - Overrides `zfs.*.version` from `images.yaml` (a full `X.Y.Z`). `ZFS_MINOR_VERSION` is no longer read. Renovate manages the pin.
- RPMFUSION_MIRROR - Optional base URL for an RPM Fusion mirror (for example `https://mirrors.ocf.berkeley.edu/rpmfusion`). When set, the build installs the rpmfusion release RPMs from it and points the rpmfusion repos at it instead of the metalink. When unset, the build uses `https://download1.rpmfusion.org` and the normal metalink. It can go in `.env`.
```bash
AKMODS_KERNEL=longterm-6.18 AKMODS_VERSION=44 AKMODS_TARGET=zfs just build
```
Will determine the current longterm kernel version, download the rpms and sign them, and will then build the zfs package.
You can also populate a `.env` file to store your current settings.
You can see your current settings with `just --evaluate`
Additionally you can pass values as key/value pairs.
```bash
just kernel_flavor=main version=42 akmods_target=extra build
```
Which will build the extra package for main.
Note, the `Justfile` will compare your inputs to the `images.yaml` file to ensure you have a valid combination.
### How to Use images.yaml
All build targets are defined in the `images.yaml` file. This is where the top level targets are defined. You can view the targets using:
```bash
yq 'explode(.).images' images.yaml
```
### Regenerating GitHub Workflows
The `.github/workflows/build-akmods-*.yml` files are generated from `images.yaml` and the templates in `workflow-templates/` (`workflow.yaml.in`, `cache_kernel.yaml.in`, `job.yaml.in`, `check-status.yaml.in`) by running:
```bash
just generate-workflows
```
Run this and commit the resulting diff whenever you edit `images.yaml` (adding/removing a kernel, flavor, or target) or anything in `workflow-templates/`. Nothing in CI does this for you.
Never hand-edit files under `.github/workflows/build-akmods-*.yml` directly — they're regenerated from the templates above, and manual edits will be silently discarded the next time someone runs `just generate-workflows`.
Each flavor's daily build is scheduled at a staggered UTC minute assigned once and preserved across regenerations, so an unrelated `images.yaml` edit shouldn't produce cron diffs unless that flavor's generated file was missing beforehand.
### Skipping Unchanged Builds
Each image has an `io.github.ublue-os.akmods.fingerprint` label with a sha256 of the build inputs and an `io.github.ublue-os.akmods.fingerprint-inputs` label with the inputs themselves as JSON. An image labeled `fingerprint=unavailable` was built without a fingerprint and never matches.
The inputs are the target, version, architecture, and kernel release. They also include the git hashes of `build_files/`, `files/`, `Containerfile.in`, `Justfile`, `fetch-kernel.sh`, the committed public and test keys, `reusable-build.yml`, and `reusable-cache-kernel.yml`. The target's `images.yaml` entry, the ZFS version, any `RPMFUSION_MIRROR` override, and for NVIDIA targets the `akmod-nvidia` EVR are in there too. Local uncommitted edits add a random nonce, so a dirty build never matches. The fingerprint logic lives in the `Justfile`, which is itself an input, so changing how the fingerprint is computed changes every hash. Keep it in a file that's an input.
Some inputs are left out on purpose: the builder image digest, akmod versions from rpmfusion, copr, and terra, NVIDIA userspace package releases, and signing key rotation. A rebuild after 7 days picks those up. To pick them up sooner, run a forced build.
Before testing, CI runs `just fingerprint` once per target and arch. That writes `build/<flavor>-<version>/fingerprint-<target>-<arch>.json`, and both `just test` and `just build` read that same file. For NVIDIA targets, the file pins the `akmod-nvidia` EVR, and the build installs exactly that version and the driver packages for that version. Without a valid file, the build runs unpinned and the image is labeled `unavailable`. For a labeled local build, run:
```bash
just fingerprint && just test && just build
```
`just build-needed` compares the local fingerprint with the published per-arch image. It skips test, build, and push only when the hashes match, the published image's signature verifies, and the image is no older than `AKMODS_MAX_AGE_DAYS` (default 7, settable as a repository variable). Any error or missing value means build. `just manifest-needed` skips regenerating the manifests when both indexes already point at the current per-arch digests and are signed. Attestations are created on every run either way.
Pull requests always build. Scheduled, merge-queue, and manual runs are guarded. In the merge queue, a change to the reusable build workflows still rebuilds. Changes that only touch other workflows, docs, or cron minutes don't rebuild there, since their pull request run already built. To force every target in a flavor, run the workflow manually with `force` checked.
### Adding Kernels and KMODs
Generally speaking, Kernels are only added if they will be used internally to Universal Blue.
KMODs as well will likely only be included if there is a need/desire to include them within the Universal Blue Project. Generally, KMODs for hardware enablement will be considered for inclusion or ones that fix/resolve a known feature gap.
Don't hesitate to file an issue asking about inclusion.
## Metrics
![Alt](https://repobeats.axiom.co/api/embed/a7ddeb1a3d2e0ce534ccf7cfa75c33b35183b106.svg "Repobeats analytics image")