As part of running `./autogen.sh` I get a `configure~` generated
alongside `configure`. This seems to be an intermittent step and should
be ignored.
For reference, this is ignored by automake itself
080b6dbea2/.gitignore (L1)
Signed-off-by: Freya Gustavsson <freya@venefilyn.se>
There's an awful lot of code in `src/common` and `src/websocket` which
is used only by `cockpit-ws`. Move all of those files (and the tests
for them) into `src/ws`.
Assisted-by: Claude <noreply@anthropic.com>
Add a `make ws-container-build-tree` target which copies the bundles and
binaries from the build tree into the container image. This is much
faster than a full image-prepare and building rpms in a VM.
This could be optimized further to not always reinstall the rpms during
each container build, but reuse these layers. However, the Containerfile
currently does the `ADD .` before the dnf install, and all the install
steps happen in an order which breaks efficient caching. This is
annoyingly hard to fix (cue: `COPY` and `ADD` barf with nonexisting
files/directories, even with globbing).
Only use when you know what you are doing -- if it breaks, you get to
keep both halves.
With the created AsciiDoc files for documentation we can now update our
build structure to generate the HTML and Manpage files both during build
and package distribution.
Changes
- Uses AsciiDoctor or asciidoc-py and a2x to generate HTML and Manpages.
- Generated HTML and Manpage files moved to `doc/output/`.
- Support packaging docs on all distributions Cockpit deploys to.
- Add package dependencies for AsciiDoctor and asciidoc-py.
- Safety fallback in automake file to search for either asciidoctor or
asciidoc-py in terms of priority.
Known issues
- Links in guide pages goes to new files for the pages themselves, which
meant we now generate HTML files for all `.adoc` files and not just
`index.html`.
With the removal of fonts we also managed to break the build pipeline
that automake generated for us. As the doc guide fonts make rule was
removed, which relied on `$(DIST_STAMP)` rule, automake optimized the
build process and in turn broke the workflow. As a workaround the
`$(DIST_STAMP)` rule was added to the index rule as a dep to help
automake generate it the way we want it.
As the workaround is a hack and not a real solution, this needs to be
fixed in another PR to make sure these dependency issues do not occur if
we were to remove `$(DIST_STAMP)` from a rule.
We also need the `$(DIST_STAMP)` to appear last in the doc generation or
something, likely race condition, may cause the `index.html` to only
contain `{}`.
Related-to: https://github.com/cockpit-project/cockpit/issues/22648
Related-to: https://github.com/cockpit-project/cockpit/issues/22647
Co-authored-by: Aleksandra Fedorova <afedorova@redhat.com>
Signed-off-by: Freya Gustavsson <freya@venefilyn.se>
Our package.json's `dependencies` has transitive/optional dependencies
which don't actually end up in the bundle, e.g. `argparse`,
`attr-accept`, or `js-tokens`; or `react-dropzone` which is a
`@patternfly/react-table` dependency, but not actually used in
cockpit.
The next commit will aggravate this further.
So make the computation of the `Provides: bundled(npm(..))` lines more
precise: Ask esbuild about which modules it actually included, which it
tells us via the "metafile". Change build.js to always output that (drop
the `--metafile` option), compute a ./runtime-npm-modules.txt list, and
use that to build the spec file.
Ported from https://github.com/cockpit-project/cockpit-files/commit/e15a518fddad1
This isn't really a systemd file, but it fits in well with the rest of
the systems integration stuff in that directory. Besides: this isn't
tied to the C bridge anymore: we use it from Python as well, and will be
getting rid of the C bridge soon.
There's a theoretical functional change here which makes no difference
in practice: we now install the policy file unconditionally. We'll soon
remove the `--disable-polkit` option anyway, and the only two things
that call `--disable-polkit` don't call `make install`
(`tools/make-dist` and the flatpak build).
Nothing uses this any more, superseded by cockpit-beiboot. This gets rid
of the libssh build dependency.
Drop the `Provides: cockpit-ssh` from Debian. No package ever related to
that virtual package name, and it's meaningless these days.
We hardcode the failure page into cockpit-ws because we want to be able
to serve it even in situations where the install is totally broken. We
do that by use of a very trivial script which escapes the html file as a
string in a temporary .c file which gets included as part of cockpit-ws.
Unfortunately, the name of that file — fail.html.c — was poorly chosen.
make has a built-in implicit rule for turning %.c into % by running the
C compiler. When it sees that fail.html is older than fail.html.c it
concludes that the correct thing to do is to bring it up to date by
compiling it, replacing the original HTML page with a binary. That's
wrong of course, and doesn't work anyway, since there's no main().
The only reason that this hasn't been a problem up to now is that there
seems to be some complicated side-effects caused by having .html
included in our TEST_EXTENSIONS list, but the next commit wants to
remove that.
Let's just avoid the problem up by calling the file by a different name
that won't match that implicit rule.
While we're at it, remove a single stray tab character that somehow made
its way into the fail page.
Having `checkLib: false` in `tsconfig.conf` is a bad idea: it means that
if we want to check `.d.ts` files, we need a separate invocation of the
typescript compiler to do that. It also means that `.d.ts` files won't
be checked in your editor by default. Our workaround for this in
`test/static-code` only checks `cockpit.d.ts`, and we'll soon have other
declaration files.
We've been using `checkLib: false` as a crutch to avoid reporting errors
against `node_modules/`. The relevant TypeScript issue is still
unaddressed: https://github.com/microsoft/TypeScript/issues/30511
Modify `test/static-code` to only report errors if errors were found on
something outside of node_modules, but in that case, report all errors.
We do this with two invocations of `tsc`, to keep things simple, but
we'll only do the second invocation if we're reporting errors. To keep
things speedy, we turn on "incremental" mode (which writes a cache file
to speed things up). `.gitignore` that. It now takes only ~2s to do an
incremental check in the "no errors" case.
By default, `tsc` outputs a very nice format that shows the exact
content of errors, but this doesn't work by default in
`test/static-code` because we capture the output before displaying it.
Detect if we're running on a TTY (or if `FORCE_COLOR=1` as from our
workflows) and enable the "pretty" output in that case.
This separates the bits which we want to install from the generated
QUnit tests, which should stay in the build tree. This will become
relevant once we drop Makefile.deps and its WEBPACK_INSTALL variable.
These are the only files which get put into pkg/ during build. This is
conceptually awkward, as we want to treat pkg/ as pure source for
webpack related input, and not write make-built files into $(srcdir).
It's also technically awkward, as we soon want to drop the Makefile.deps
mechanism, and instead have a global dynamic pkg/ → dist/ dependency.
`make` changing files in pkg/ would unnecessarily build the webpack at
each invocation.
So move them to src/appstream/ instead. They fit better there, as they
are not part of webpacks (in dist/), and are better handled by make than
by webpack.
Since we dropped the Makefile rule for creating cockpit-bridge.pyz, the
pybridge scenario has been broken.
Add some minimal pyproject.toml metadata required to build a wheel of
the Python package. Build this wheel on the outside, and inject it into
the VM via image-customize.
Note: we no longer bother with building the C parts of the package
anymore — we rely on the distro packages already present in the image.
Doing it this way instead of ad-hoc allows us to improve the way we run
the unit-tests container: we can now check out the submodule as part of
the preparation step, eliminating the need for network access inside the
container.
Set up pytest and coverage via pyproject.toml. Add some hints for
static linters, as well.
Add some tests for the Transport classes. These already found several
bugs (fixed in previous commits).
Add support for creating cockpit-bridge.pyz as a zipapp, containing the
Python bridge, systemd_ctypes, plus a full set of packages from dist/.
Add a target `make pycheck` to run the QUnit tests against this new
bridge package.
The logic is broken for builddir != srcdir builds, since it doesn't
include $(srcdir). It's also a bit silly to explicitly list all of the
files when we can easily do it with `$(wildcard)`.
Switch to `$(wildcard)`, include `$(srcdir)` and inline the whole thing.
Drop the weird thing where we source the 'prepare' script and do some
stuff before running the 'prepare' function defined inside of it. This
is now a simple script, with two arguments: the source location and the
checksum. If the source is unspecified, it will be created. If the
checksum is omitted, and the source is local, it will be calculated.
The prepare script writes the manifest to the working directory where it
was invoked and prints the name of the file to stdout.
Adjust containers/flatpak/install for these changes, and take advantage
of the fact that the output is now placed in the working directory:
change to tmp/ first, and do the build there as well. That lets us
clean up some .gitignore noise as well.
We drop the update-flathub machinery for now. It'll be brought back
very soon as part of a revamped release script.
Create a tool to turn the point form notes in our tags into a proper set
of <release/> tags for metainfo and fill this information in when
building a tarball.
Install the metainfo if building with --enable-client. Validate it
at install time as part of building the flatpak — since we no longer
handle it separately, it's more difficult to do that from the outside
now.
Strip the old metainfo handling out of the containers/flatpak scripts.
Add a nice automake-style output for generating the manifest .yml.
This will simplify the process of releasing the flatpak.
The version of Python we have on the GitHub `ubuntu-latest` runner is
lacking the `indent` function in ElementTree, so add a hack to disable
it. We do it this way to make it easier to back this out in the future.
Extract *.metainfo.xml strings into our PO template, and use po files to
generate the installed metainfo.xml files.
Amend the distcheck unit test to ensure that translatable strings are
extracted from our cockpit webapp AppStream data (which was already the
case) as well as for our extensions (which is fixed with this commit).
Adjust package build dependencies to ensure gettext support. Current
gettext version 0.21 ships `metainfo.its`, for older releases (RHEL
8 and Ubuntu 20.04) this needs the additional appstream package.
Fixes https://bugzilla.redhat.com/show_bug.cgi?id=2018389
Add a new script, tools/make-compile-commands, which can generate the
compile_commands.json script used by many static analysis tools.
Run the script at the end of ./configure.
Now that we don't have to deal with on-the-fly compression of .gz files,
we can use a much easier approach to installing our webpack output.
automake's nobase_ prefix is pretty much exactly designed for this sort
of "preserve tree structure" installation. There's only one catch:
since we have no way to drop the `dist/` prefix, we do a workaround:
create a symlink from `cockpit/` to `dist/` and install the files, via
that symlink, directly into `$(datadir)`.
That gets us names like `/usr/share/cockpit/base1/cockpit.js.gz`, as
before.
It also means that it's now possible to point XDG_DATA_DIRS directly at
the build directory, for testing purposes.
One issue this introduces is how to deal with the manifests for pcp and
ssh: the tar rule managed to strip the pkg/ prefix from those, but now
we have to install them a bit more explicitly.
We use a dist-hook combined with tar for some fairly complicated
reasons, constituting one of the most non-obvious corners of the build
system. The main reasons for that are:
1) directly adding a list of files in git to EXTRA_DIST is not
something that can be done reasonably in make syntax, and
particularly when we also need it to work when building from tar
2) (and this is complicated) we support typing `make dist` directly
after configuring the package in order to get a .tar release,
containing the files built by webpack (which will be implicitly
built as a part of `make dist`). We know what files to ship because
`webpack-make` writes a set of `Makefile.deps` files, setting the
`WEBPACK_INPUTS` and `WEBPACK_OUTPUTS` variables. At the time of
the first `make dist` invocation on the freshly-configured project,
we immediately call `make distdir-am` which is what invokes
webpack-make. However: this invocation of make doesn't get a chance
to include the Makefiles that weren't present at the time it was
invoked, and as a result, doesn't have access to the final values of
`WEBPACK_INPUTS` and `WEBPACK_OUTPUTS`. We've solved this by using
`dist-hook` (which gets a fresh recursive make invocation, and a new
chance to read the Makefiles.deps).
3) we want to make some minor adjustments to the distdir (adding a
.tarball file, adjusting timestamps, etc).
Our use of tar is pretty evil here, though, and it breaks badly when
combined with out-of-tree builds. It turns out that automake has quite
some logic for dealing with VPATH demangling, if only we can get the
files that we want to ship into `EXTRA_DIST` at the time of the `make
distdir-am` invocation.
We achieve this by using our (already existing) `distdir:` target
override to gain more control over the `make distdir-am` invocation.
The three points above are addressed, as follows:
1) If we're in a git tree, we create (or update) a .committed_files
file on `make dist` and pass its contents to be added to
`EXTRA_DIST` in the `make distdir-am` invocation. We add this file
itself to the dist, avoiding the need to use `find` in the
"non-git" case.
2) By adding `$(DISTFILES)` as a dependency of the `distdir:` target,
we can be sure our webpack (and other things in parallel) will be
built by the time we do the recursive `make distdir-am` invocation.
This specifically means that the Makfefile.deps will be available
to be included, allowing `WEBPACK_INPUTS` and `WEBPACK_OUTPUTS` to
be added to `EXTRA_DIST` in the usual way.
3) We can make our minor adjustments to the distdir directly from the
`distdir:` target, after the `make distdir-am` invocation.
In addition to working with out-of-tree builds, the new approach is, to
my eye, much easier to understand in depth.
Drop all .gitignore files in subdirectories, leaving only the toplevel.
The rules contained in those files were mostly outdated, which shows the
usefulness of having one file in a central location.
Add a couple of more general-purpose rules for things like coverage data
(--enable-coverage) and make some of the wildcard rules more specific
(and self-documenting).
Take the SELinux policy building out of the spec file and add a proper
set of Makefile rules for it. Stop blanket-disting the selinux/
subdirectory, and treat it more like a normal source directory.
The new rules are only enabled if --enable-selinux-policy=<type> is
explicitly given.
From the parts remaining in the spec file, clean them up a bit: remove
the with_selinux conditional and use the pre- and post- rules
unconditionally: we're only building the cockpit-ws package, anyway, in
exactly the case that we'd want SELinux policy enabled.
We can also make use of the macro provided by the selinux-policy package
to avoid having to guess the version number for ourselves. We need to
keep our own dependencies (instead of using %selinux_requires), though
for two reasons:
- chicken and egg problem: without a static BuildRequires on
selinux-policy, the macros themselves wouldn't be available during
the SRPM build, which means that we don't know that we need
selinux-policy.
- the post dependencies defined by the macro are a bit too aggressive
and aren't conditional on selinux being available at all. This
would break container builds, for example.
Our %build stage now consists of a single %configure invocation and a
single %make_build invocation.
When building a release, let's include a complete tools/cockpit.spec.
First, the existing tools/cockpit.spec isn't a valid spec file at all,
because it requires pre-processing before it can be used. Rename it to
cockpit.spec.in to reflect this reality.
Replace the %{}-style variable substitutions with @@-style ones to make
it clear that these substitutions are not RPM variables, and that
they're handled on a different level.
Drop the existing tools/gen-spec-dependencies (with replaces its output
in-place) and tools/min-base-version (called from gen-spec-dependencies)
with a single script which operates more like a standard shell utility,
with a separate output file. This tool expands all of the @@ variables.
Drop the extra hacks from tools/make-srpm, as they are no longer needed.
We can also reduce the CentOS 8 hacks in our .packit.yml file, but
unfortunately we still need to produce a valid spec file during the
`post-upstream-clone` phase, because packit cannot continue without it.
There's a --build-all flag to create-spec (and some small changes to the
spec file) to reproduce the old "one srpm please" behaviour.
Our increasingly large web of systemd integration mostly lives in
src/ws/, for mostly historical reasons. These days, cockpit-tls has
at least as much integration with systemd as does cockpit-ws, and
cockpit-session is about to gain some.
Move everything to a new src/systemd/ directory.
We use glib-compile-resources to get a single html file bundled. That
seems like overkill.
Ideally, we'd use #embed for this, but that seems like it'll take a
while longer, so use a small Python script to convert the .html into C,
and compile the resulting file in the usual way.
Formerly, cockpitwebresponse would default to serving a minimalist
internally-hardcoded document for errors, unless the resource was
provided. Since this resource was provided from both -bridge and -ws,
the internal document was only used for testing. That was for good
reason: the full document would be a pain to test on, and we'd have to
update the tests any time someone modified the html.
Let's turn the logic around there and default to the real document and
give the tests a mechanism to inject their minimal one. We use a
pointer instead of a constant, allowing it to be directly overridden
from the tests, and simplifying the code in cockpitwebresponse a bit.
We tried another approach with the usual `ld -b binary` tricks to
convert the file directly to a linkable symbol, but this was causing
strange crashes on rhel and fedora. Going via C is a sure thing.
Add some scripts and metadata to support packaging cockpit-client as a
flatpak.
Add a GitHub workflow to do some initial smoke testing of the flatpak
(thanks Martin!).
Having a / at the end of the ignore pattern causes a symlink not to be
ignored. For some time now, tools/make-bots can put a symlink there, so
ignore that too.
