mirror-cockpit-project-cockpit/HACKING.md

# Hacking on Cockpit

Start by getting the code:

    git clone https://github.com/cockpit-project/cockpit
    cd cockpit/

The remainder of the commands assume you're in the top level of the
Cockpit git repository checkout.

**Do not clone a fork!** This will not work for various reasons (missing tags,
determining version, integration with bots commands). Please keep `origin` as
the read-only actual upstream project, and add your fork as a separate writable
remote, for example with

    git remote add my git@github.com:yourgithubid/cockpit.git

## Setting up development container

The cockpit team maintains a [cockpit/tasks container](https://ghcr.io/cockpit-project/tasks)
for both local development and CI. If you can install [toolbx](https://containertoolbx.org/) or
[distrobox](https://distrobox.privatedns.org/) on your system, it is highly
recommended to do that:

 - It is *the* official environment for CI, known to work, and gives you reproducible results.
 - It avoids having to install development packages on your main machine.
 - It avoids having to map the build and test dependencies to package names of various distributions.

1. Install `toolbox`

   - Fedora/CentOS/RHEL based distributions:

         sudo dnf install toolbox

   - Debian/Ubuntu based distributions:

         sudo apt install podman-toolbox

2. Create a development toolbox for Cockpit

       toolbox create --image ghcr.io/cockpit-project/tasks -c cockpit

3. Enter the toolbox:

       toolbox enter cockpit

Your home directory, user D-Bus, etc. are shared with the host, so you can
edit files as you normally would. Building and running tests happens inside the
toolbox container. If desired, you can install additional packages with
`sudo dnf install`.

The Cockpit team occasionally refreshes the `tasks` container image.
To re-create your development container from the latest image, run:

    podman pull ghcr.io/cockpit-project/tasks
    toolbox rm cockpit

...and then repeat steps 2 and 3 from above.

## Working on Cockpit's session pages

Most contributors want to work on the web (HTML, JavaScript, CSS) parts of Cockpit.

### Install Cockpit

First, install Cockpit on your local machine as described in:

<https://cockpit-project.org/running.html>

### Build session pages

Next, run this command from your top level Cockpit checkout directory, and make
sure to run it as the same user that you'll use to log into Cockpit below.

    mkdir -p ~/.local/share/
    ln -s $(pwd)/dist ~/.local/share/cockpit

This will cause Cockpit to read JavaScript, HTML, and CSS files directly from the
locally built package output directory instead of using the system-installed Cockpit
files.

The recommended way to build bundles is to use the "watch" mode
(`-w` or`--watch`) on the page you are working on. For example, if you want to
work on anything in [pkg/systemd](./pkg/systemd/), run:

    ./build.js -w systemd

See [pkg/](./pkg/) for a list of all pages.

If you work on a change that affects multiple pages (such as a file in
pkg/lib/), you can also build all pages:

    ./build.js -w

Now you can log into Cockpit on your local Linux machine at the following
address, using the same username and password as your desktop login:

<http://localhost:9090>

Watch mode automatically rebuilds when source files are modified. Once it
finishes building, refresh your browser to see the changes in Cockpit.
Press `Ctrl-C` to stop watch mode when you are done changing the code.

### Testing

You often need to test code changes in a VM. You can set the `$RSYNC` env
variable to copy the built page into the given SSH target's
/usr/local/share/cockpit/ directory. If you use Cockpit's own test VMs and set up the
SSH `c` alias as described in [test/README.md](./test/README.md), you can use
one of these commands:

    RSYNC=c ./build.js -w kdump
    RSYNC=c ./build.js -w

### Returning to system packages

To make Cockpit use system packages again, instead of your checkout directory,
remove the symlink with the following command and log back into Cockpit:

    rm ~/.local/share/cockpit

## Building and unit tests

Cockpit uses autotools, so there are familiar `./configure` script and
Makefile targets.

After a fresh clone of the Cockpit sources, you need to prepare them by running
`autogen.sh` like this:

    ./autogen.sh --prefix=/usr --enable-debug

As shown, `autogen.sh` runs 'configure' with the given options, and it also
prepares the build tree by downloading various nodejs dependencies.

When working with a Git clone, it is best to always run `./autogen.sh`
instead of `./configure`.

Then run

    make

to build everything. Cockpit has a single non-recursive Makefile. You can only
run `make` from the top-level and it will always rebuild the whole project.

You can run unit tests of the current checkout:

    make check

These should finish very quickly. It is a good practice to do this often.

For debugging individual tests, there are compiled binaries in the build
directory. For QUnit tests (JavaScript), you can run

    ./test-server

which will output a URL to connect to with a browser, such as
<http://localhost:8765/qunit/base1/test-dbus.html>. Adjust the path for different
tests and inspect the results there.

QUnit tests are run as part of a pytest test called `test_browser`.  You can
run individual tests via `pytest -k`, like so:

    pytest -k test-fsinfo.html

You can see JavaScript code coverage information for QUnit tests.  For a
summary table:

    pytest -k test_browser --js-cov

And for detailed output on uncovered sections in a specific file, something
like:

    pytest -k test-fsinfo.html --js-cov-files='*/fsinfo.ts'

Coverage information is gathered into the pytest tmpdir, regardless of which
coverage-related commandline flags are given, so it's also possible to drill
down after the fact — without re-running tests — using something like:

    test/common/js_coverage.py -m '*/fsinfo.ts' /tmp/pytest-of-*/pytest-current/js-coverage/*

There are also static code and syntax checks which you should run often:

    test/common/static-code

It is highly recommended to set up a git pre-push hook, to avoid pushing PRs
that will fail on trivial errors:

    ln -s ../../tools/git-hook-pre-push .git/hooks/pre-push

This calls `test/common/static-code` for each commit you're trying to push.

You can also set up a post-commit hook to do the same, after each commit:

    ln -s ../../tools/git-hook-post-commit .git/hooks/post-commit

We also have a hook to ameliorate one of the more annoying drawbacks of using
git submodules:

    ln -s ../../tools/git-hook-pre-rebase .git/hooks/pre-rebase

## Running the integration test suite

Refer to the [testing README](test/README.md) for details on running the Cockpit
integration tests locally.

## Bridge

The Cockpit bridge is the initial program launched in a Cockpit Linux session:
Its stdin/out is connected to the web socket (and thus to JavaScript in the
[pages](pkg/)), where it speaks a [JSON protocol](doc/protocol.md) that
multiplexes "channels" -- abstractions of operating system APIs that the pages
use to implement their functionality. This protocol is translated into
operating system calls such as opening or writing files, D-Bus calls, or HTTP
queries. Think of the bridge as the moral equivalent of "bash" in a human SSH
session.

The bridge resides in `src/cockpit` with most of its rules in `src/Makefile.am`.
This directory was chosen because it matches the standard so-called "src
layout" convention for Python packages, where each package (`cockpit`) is a
subdirectory of the `src` directory.

### Running the bridge

The bridge can be used interactively on a local machine out of the source tree:

    PYTHONPATH=src python3 -m cockpit.bridge

To make it easy to test out channels without having to write out messages
manually, `cockpit.misc.print` can be used:

    PYTHONPATH=src python3 -m cockpit.misc.print open fsinfo path=/etc 'attrs=["type", "entries"]' | PYTHONPATH=src python3 -m cockpit.bridge

These shell aliases might be useful when experimenting with the protocol:

    alias cpy='PYTHONPATH=src python3 -m cockpit.bridge'
    alias cpf='PYTHONPATH=src python3 -m cockpit.misc.print'

You can then for example run a metrics channel:

    cpf open metrics1 source=internal interval=1000 metrics='[{"name": "cpu.basic.user", "derive": "rate"}]' : wait | cpy

To enable debug logging in journal on a test image, you can pass `--debug` to
`image-prepare`. This will set `COCKPIT_DEBUG=all` to `/etc/environment`, if
you are only interested channel debug messages change `all` to
`cockpit.channel`.

### Iterating on the bridge with the ws container

Our fedora-coreos and -bootc images use the [cockpit/ws container](https://quay.io/repository/cockpit/ws)
instead of `cockpit-bridge.rpm`. You can do fast "edit bridge" → "run
integration test" iterations by modifying `MachineCase.login_and_go()` to
upload the local dev tree bridge code to the VM, then onwards into the running
container, and bind-mount it into the correct path:

```diff
--- test/common/testlib.py
+++ test/common/testlib.py
@@ -1933,6 +1933,14 @@ class MachineCase(unittest.TestCase):
         if enable_root_login:
             self.enable_root_login()
         self.machine.start_cockpit(tls=tls)
+
+        m = self.machine
+        m.execute("umount /usr/lib/python3.14/site-packages/cockpit || true")
+        m.upload(["../src/cockpit"], "/tmp/")
+        m.execute("podman cp /tmp/cockpit ws:/tmp/")
+        m.execute("podman exec ws mount -o bind /tmp/cockpit /usr/lib/python3.14/site-packages/cockpit")
+
         # first load after starting cockpit tends to take longer, due to on-demand service start
         with self.browser.wait_timeout(30):
             self.browser.login_and_go(path, user=user, password=password, host=host, superuser=superuser,
```

Replace `3.14` with the Python version du jour.

### Testing the bridge

There are a growing number of [pytest](https://docs.pytest.org) tests being written to test
the bridge code.  You can run these with `make pytest` or
`make pytest-cov`.  Those are both just rules to make sure that the
`systemd_ctypes` submodule is checked out before running `pytest` from the
source directory.

The tests require at least `pytest` 7.0.0 or higher to run.

## Running eslint

Cockpit uses [ESLint](https://eslint.org/) to automatically check JavaScript
code style in `.js` and `.jsx` files.

The linter is executed as part of `test/common/static-code`.

For developer convenience, the ESLint can be started explicitly by:

    npm run eslint

Most rule violations can be automatically fixed by running:

    npm run eslint:fix

Rules configuration can be found in the `.eslintrc.json` file.

During fast iterative development, you can also choose to not run eslint, by
running `./build.js` with the `-e`/`--no-eslint` option. This
speeds up the build and avoid build failures due to ill-formatted comments,
unused identifiers, and other JavaScript-related issues.

## Running stylelint

Cockpit uses [Stylelint](https://stylelint.io/) to automatically check CSS code
style in `.css` and `.scss` files.

The linter is executed as part of `test/common/static-code`.

For developer convenience, the Stylelint can be started explicitly by:

    npm run stylelint

But note that this only covers files in `pkg/`. `test/common/static-code` covers
*all* (S)CSS files tracked in git.

Some rule violations can be automatically fixed by running:

    npm run stylelint:fix

Rules configuration can be found in the `.stylelintrc.json` file.

During fast iterative development, you can also choose to not run stylelint, by
running `./build.js` with the `-s`/`--no-stylelint` option. This speeds up the
build and avoids build failures due to ill-formatted CSS or other issues.

## Working on your local machine: systemd-sysext

If you want to safely test your local changes directly on it, Cockpit supports
installation as a [systemd-sysext](https://www.freedesktop.org/software/systemd/man/latest/systemd-sysext.html).
This covers all parts of Cockpit (ws, tls, session, bridge, login page, systemd
units, PAM configuration, and the session pages) except for the SELinux policy.
It gets installed into `/run/extensions/`, so nothing ever hits the disk and
this also works on read-only installations (CoreOS, OSTree, bootc).

Just run:

    tools/make-sysext

This runs `./autogen.sh` if necessary, and then just re-`make`s your tree,
re-installs it into `/run/extensions`, and reloads the sysext in systemd.
Afterwards you can connect to http://localhost:9090 as usual.

To remove this, reboot or run

    tools/make-sysext stop

**Attention**: This is not currently compatible with SELinux in enforcing mode.
If you have that, you need to disable it:

    sudo setenforce 0

## Working on your local machine: Web server

If the above systemd-sysext approach does not work for you, you can also test
changes with some bind mounts.

To test changes to the login page, you can bind-mount the build tree's
`dist/` directory over the system one:

    sudo mount -o bind dist /usr/share/cockpit

Likewise, to test changes to the branding, use:

    sudo mount -o bind src/branding/ /usr/share/cockpit/branding/

Do `mkdir dist/branding` if you run both of these commands at the same time.

After that, run `systemctl stop cockpit.service` to ensure that the web server
restarts on the next browser request.

To make Cockpit use system-installed code again, umount the paths:

    sudo umount /usr/share/cockpit /usr/share/cockpit/branding/
    systemctl stop cockpit.service

Similarly, if you change `cockpit-ws` itself, you can make the system (systemd
units, cockpit-tls, etc.) use this:

    sudo mount -o bind cockpit-ws /usr/libexec/cockpit-ws

On Debian-based OSes (including Ubuntu), the path will be
`/usr/lib/cockpit/cockpit-ws` instead.

On Fedora, CentOS, Red Hat Enterprise Linux, and related distributions, you also
need to disable SELinux with:

    sudo setenforce 0

for this to work, as your local build tree does not otherwise have the expected
SELinux type.

Some cockpit binaries rely on specific paths in /usr/share or libexecdir to be
set correctly. By default they are set to `/usr/local`.

On RPM based systems, this can be set using an autogen.sh argument;
afterwards you need to rebuild:

    ./autogen.sh rpm

## Installation from upstream sources

    make
    sudo make install

This will install Cockpit and all support files. If you have a
Fedora/RHEL/CentOS based distribution, install a PAM configuration with

    sudo cp tools/cockpit.pam /etc/pam.d/cockpit

If you have a Debian/Ubuntu based distribution, install this PAM config instead:

    sudo cp tools/cockpit.debian.pam /etc/pam.d/cockpit

For other distributions you need to create a PAM config yourself.

If you prefer to install to a different `--prefix` and would prefer
that `make install` not write outside that prefix, then specify the
`--enable-prefix-only` option to `autogen.sh`. This will result in an
installation of Cockpit that does not work without further tweaking.
For advanced users only.

## Build distribution packages

Instead of a direct `make install` as above, you can also build distribution
packages and install them. This is generally more robust, as they upgrade and
remove cleanly, and don't interfere with distribution packages in `/usr`.

In a Fedora/RHEL build environment you can build binary RPMs with

    tools/make-rpms --quick

In a Debian/Ubuntu build environment you can build debs with

    tools/make-debs --quick

## Updating `node_modules`

During a normal build from a git checkout, the `node_modules` will be
automatically unpacked from a cache kept in a separate git repository.  You can
force the unpack to occur using the `tools/node-modules checkout` command, but
this shouldn't be necessary.  In the event that you need to modify
`package.json` (to install a new module, for example) then you'll need to run
`tools/node-modules install` to create a new cache from the result of running
`npm install` on your new `package.json`.

Your locally rebuilt changes to `node_modules` won't be used by others.  A new
version will be created by a GitHub workflow when you open your pull request.

The `tools/node-modules` script inspects the `GITHUB_BASE` environment variable
to determine the correct repository to use when fetching and pushing.  It will
strip the repository name (leaving the project- or username) and use the
`node-cache.git` repository in that namespace.  If `GITHUB_BASE` is unset, it
will default to `cockpit-project/node-cache.git`.

A local cache is maintained in `~/.cache/cockpit-dev`.

## Contributing a change

Make a pull request on github.com with your change. All changes get reviewed,
tested, and iterated on before getting into Cockpit. The general workflow is
described in the [wiki](https://github.com/cockpit-project/cockpit/wiki/Workflow).

You need to be familiar with git to contribute a change. Do your changes
on a branch. Your change should be one or more git commits that each contain one
single logical simple reviewable change, without modifications that are
unrelated to the commit message.

Don't feel bad if there's multiple steps back and forth asking for changes or
tweaks before your change gets in. If you fix your commits after getting a
review, just force-push to your branch -- this will update the pull request
automatically. Do *not* close it and open a new one; that would destroy the
conversation and reviews.

Cockpit is a designed project. Anything that the user will see should have
design done first. This is done on the wiki and mailing list.

Bigger changes need to be discussed on the
[#cockpit:fedoraproject.org](https://matrix.to/#/#cockpit:fedoraproject.org)
Matrix channel or our mailing list
[cockpit-devel@lists.fedoraproject.org](https://lists.fedorahosted.org/admin/lists/cockpit-devel.lists.fedorahosted.org/)
before you invest too much time and energy.

Feature changes should have a video and/or screenshots that show the change.
This video should be uploaded directly to GitHub on the pull request or issue
or uploaded to YouTube or another service that allows video embedding.

Use a command like this to record a video including the browser
frame:

```
recordmydesktop -x 1 -y 200 --width 1024 --height 576 \
   --fps 24 --freq 44100 --v_bitrate 2000000
```

(This command only works on X11 and requires the `recordmydesktop` program to
be installed.)

You can also resize your browser window and move it to the right location with
a script. In Firefox you can open the Scratchpad (`Shift+F4`) and enter the
following commands:

    window.resizeTo(1024, 576);
    window.moveTo(1, 200);

Then run it with `Ctrl+R` when the browser is showing an empty tab, e.g.
`about:newtab`. You may need to adjust the positions for your environment.

## Adding a release note in PR

Optional but highly quality of life feature we have relates to automatic
release blog post generation.

There is an automated release note process for PRs where any PRs that include
a label `release-note` and contains a release note segment in the description
will be used when generating the release blog post for https://cockpit-project.org.

To create a release note you need a h2-header at the end of your PR description
where everything underneath will be taken by the automation for the blog
generation. It will also take any images and videos uploaded and add them to
the blog post PR as well. For example:

```md
[...truncated PR description...]

## Release note title

Release note text that will be picked up by automation

<img for release note>
```

You can see this in action in [this cockpit-machines
PR](https://github.com/cockpit-project/cockpit-machines/pull/2268) where we
have an h2-header together with text which in turn was taken by the
automation and shown inside [the blog
post](https://cockpit-project.org/blog/cockpit-347.html) for the release
that included it.

## Debug logging

All messages from the various cockpit processes go to the journal and can be
seen with commands like:

    sudo journalctl -f

Cockpit's web server has verbose internal debug logging that can be enabled when
trying to track down a problem. To turn it on add a file to your system like
this:

    sudo mkdir -p /etc/systemd/system/cockpit-wsinstance-http.service.d
    sudo sh -c 'printf "[Service]\nEnvironment=G_MESSAGES_DEBUG=all\n" > /etc/systemd/system/cockpit-wsinstance-http.service.d/debug.conf'
    sudo systemctl daemon-reload
    sudo systemctl stop cockpit

Instead of `all`, you can also specify a log domain:

 * cockpit-protocol: Very verbose low level traffic logging
 * cockpit-ws: Cockpit Web Service detailed debug messages
 * WebSocket: Verbose low level WebSocket logging

To revert the above logging changes:

    sudo rm /etc/systemd/system/cockpit-wsinstance-http.service.d/debug.conf
    sudo systemctl daemon-reload
    sudo systemctl stop cockpit

To debug HTTPS connections, replace `http` with `https@` in the above commands.

The bridge runs in the user session, not in a systemd service. See "Running the
bridge" section for how to enable debug logging.

## Debug logging in Javascript console

Various javascript methods in Cockpit can show debug messages. You can turn them
on by setting a `window.debugging` global, or setting up a `debugging` property
in the browser storage. To do this run the following in your javascript console:

    >> sessionStorage.debugging = "all"

You'll notice that there's a ton of messages that get shown. If you
want to be more specific, instead of "all" use one or more of the following
specific types:

    "all"      // All available debug messages
    "channel"  // All channel messages sent to server
    "dbus"     // DBus related debug messages
    "http"     // HTTP (via the server) related debug messages
    "spawn"    // Debug messages related to executing processes

There are other strings related to the code you may be working on. For example,
the metrics page shows debug information with the value `metrics`. Do
`git grep window.debugging pkg` to find out all available ones.

In addition, if you want your debug setting to survive a browser refresh or
Cockpit log out, use something like:

    >> localStorage.debugging = "spawn"

## Using React Developer Tools

Cockpit uses React for the JavaScript frontend, [React Developer
Tools](https://react.dev/learn/react-developer-tools) is a browser extension to
inspect React components, edit props and state. Out of the box the developer
tools do not work with Cockpit due to the pages being loaded in a separate
iframe. A workaround is to load the page directly by embedding, for example for
the system overview page:

<http://localhost:9090/cockpit/@localhost/system/index.html>

This loads the system overview as a standalone page allowing React Developer
tools to inspect its state.

## Running Cockpit processes under a debugger

You may want to run cockpit-ws under a debugger such as valgrind or gdb. You can
run these processes as your own user, although you won't be able to debug all
the authentication logic in those cases.

First of all, make sure Cockpit is correctly installed. Even though we will be
running `cockpit-ws` from the built sources, this still relies on some of the right
software being installed in order for Cockpit to work. (Such as: PAM stack,
UI files, cockpit-bridge, etc.)

This is how you would run cockpit-ws under gdb:

    export G_DEBUG=fatal-criticals
    export G_MESSAGES_DEBUG=cockpit-ws,cockpit-wrapper,cockpit-bridge
    gdb --args ./cockpit-ws --port 10000 --no-tls

And you can run cockpit-ws and cockpit-bridge under valgrind like this:

    export G_DEBUG=fatal-criticals
    export G_MESSAGES_DEBUG=cockpit-ws,cockpit-wrapper,cockpit-bridge
    valgrind --trace-children=yes --trace-children-skip='*unix_chkpwd*' \
          ./cockpit-ws --port 10000 --no-tls

Note that cockpit-session and cockpit-bridge will run from the installed
prefix, rather than your build tree.

## Debug hard-to-grab elements

In most cases, you can use the browser debugger to directly attach breakpoints
to interesting elements. But that's not directly possible with e.g. popup menus
or other elements which react to `mouseenter`/`mouseleave`. For these, run this
in the developer console:

```js
setTimeout(() => { debugger }, 5000)
```

then do the mouse action (like hovering over an element) and wait until the
timeout.

## Start login session and web socket with curl

For iterating on the login / web socket ←→ bridge integration, this command
logs into our standard test VM (https://127.0.0.2:9091) and establishes the web
socket connection and user session:

```sh
curl -ksS -D- -u admin:foobar --cookie-jar /tmp/cookie https://127.0.0.2:9091/cockpit/login --no-buffer -H"Connection: Upgrade" -H"Upgrade: websocket" -H"Host: 127.0.0.2:9091" -H"Origin: https://127.0.0.2:9091" -H"Sec-Websocket-Key: 3sc2c9IzwRUc3BlSIYwtSA==" -H"Sec-WebSocket-Version: 13" https://127.0.0.2:9091/cockpit/socket
```

It replies with a message like
```
{"csrf-token":"e3074fc5e06cb3804ad8c3463fc6727c67ac245dd3c39590e486644759a42818"}HTTP/1.1 101 Switching Protocols
```

which contains the session token. You can use the `--cookie-jar` to issue
further authenticated HTTP requests. Note that as there is no browser attached,
the session times out after some 10 seconds of inactivity.

## Manually installing the development dependencies

_If at all possible, use the cockpit/tasks container with toolbox/distrobox as
documented above. Installing all necessary development packages manually on
your machine is intrusive, error prone, difficult, and hard to debug._

You will need at least node.js and NPM.

On Fedora or CentOS (>= 9):

    sudo dnf install npm

On Debian/Ubuntu:

    sudo apt install npm

For running tests, the following dependencies are required:

    sudo dnf install curl expect xz rpm-build chromium-headless dbus-daemon \
        libvirt-daemon-driver-storage-core libvirt-daemon-driver-qemu libvirt-client python3-libvirt \
        python3-pyyaml

For compiling the C parts, you will need the package build dependencies:

    sudo dnf install dnf-utils python-srpm-macros
    sudo dnf builddep --spec tools/cockpit.spec