Commit graph

24 commits

Author SHA1 Message Date
Allison Karlitskaya
0c46dd5729 urls-check: avoid using 'task' from bots
We're trying to get rid of this module and this is one of the last
users.  It isn't really using anything from the module either, other
than the `api` GitHub instance (which it could easily create for itself
directly).

Let's use the GitHub methods directly for everything.  One minor fix:
you close GitHub issues with PATCH, not POST.
2026-06-02 15:46:31 +02:00
Martin Pitt
a7ae147258 REUSE: Convert license headers to SPDX format
Replace full LGPL/GPL/MIT/BSD license header with
SPDX-License-Identifier in all source files. This is more concise and
makes the license information machine readable.

Keep the full text of ISC for src/common/cockpitbase64.[hc], it's small
enough and only used in this pair of files. Also keep the CC-BY-SA
paragraph in pkg/storaged/icons/gnome-icons.jsx as explained in the
previous commit.

See https://reuse.software/spec-2.0/#2-include-a-copyright-notice-and-license-in-each-file

https://issues.redhat.com/browse/COCKPIT-1742
2026-02-04 10:24:11 +01:00
Allison Karlitskaya
848e36dd0b test: adjust bots imports
We moved some files from task/ to lib/, so update the imports so we can
drop the shims.
2026-01-26 13:32:52 +01:00
Jelle van der Waa
d304d86176 treewide: remove unused Python imports 2025-10-10 06:29:44 +02:00
Martin Pitt
52d1c415c4 tools: Ignore gnome.org in urls-check
https://gnome.org keeps failing with "403 Forbidden" when running in a
GitHub action, presumably due to some IP range (or User-Agent) ban? It
works fine locally.

We know that https://www.gnome.org is fine, so generalize our existing
ignore mechanism for example.com and add it to that list.

Fixes #21340
2024-11-28 11:07:28 +01:00
Martin Pitt
aaa4902aff tools: Ignore SVG/XML namespace URLs in urls-check
Some of our code generates SVGs or processes XML. These are namespace
identifiers and should not be changed to https.
2024-08-19 09:23:02 +02:00
Martin Pitt
08cfc7a878 tools: Fix handling of redirects in urls-check
Show redirections with `urls-check -n`, and also show them if there are
failed URLs.
2024-08-19 09:23:02 +02:00
Martin Pitt
c016cf302e all: Move gnu.org/license URLs to https://
To skip the automatic redirect.

This was done mechanically with:
```sh
git grep -l http://www.gnu.org/licenses/ | xargs sed  -i 's@http://www.gnu.org@https://www.gnu.org@g'
```
2024-08-19 09:23:02 +02:00
Jelle van der Waa
69041116a8 tools: URLError has no attribute code 2024-06-19 14:49:30 +02:00
Martin Pitt
0c69347fb1 tools: Print failure details
This might tell us what's wrong about that dmtf.org URL when running in
GitHub (issue #20167).
2024-03-13 16:30:00 +01:00
Jelle van der Waa
076f732ea8 tools: add dry run option to urls-check 2024-02-14 12:16:36 +01:00
Jelle van der Waa
1907213c2f ruff: Enable UP032 f-string rule
Convert .format() to f-strings with ruff's auto-fixer.
2023-09-27 17:44:24 +02:00
Allison Karlitskaya
4a9e30bc86 all: replace import parent with an interpreter line hack
We'd like to run our executable python scripts using our new `pywrap`
feature as a better way to get their PYTHONPATH setup properly.  This
script lives in our source tree, at a fixed relative location to each
script that we're interested in, but has no absolute system path.

The #!interpreter syntax does supports relative paths, but they are
resolved relative to the current working directory, not relative to the
file in which they appear, which makes this feature pretty useless.

There are some tricks we can do to get our designed 'relative'
behaviour, though.

Something like this works:

    #!/usr/bin/env -S sh -c '"${0%/*}/pywrap" "$0"'

but it causes various file auto-detection schemes (including our own
test/static-code) to improperly conclude that this is a shell script,
rather than a Python script.  (n)vim also gets it wrong for highlighting
and linting purposes.

Fortunately there's a somewhat longer, but still acceptable version of
this we can do in Python:

    #!/usr/bin/python3 -cimport os, sys; os.execv(os.path.dirname(sys.argv[1]) + "/pywrap", sys.argv)

This fits well under our 118 column limit.

Note that unlike `sh`, the `-c` option in Python doesn't require the
script to be passed as a separate argument, which means we don't need to
use the non-POSIX splitting tricks of `/usr/bin/env -S`.

Note also: it appears as though our hardcoding of /usr/bin/python3 here
might be venv-unfriendly, but it's not: /usr/bin/python3 (which surely
exists) is used only for running this one tiny fragment of code.  Our
pywrap script itself finds python3 in the path.

Let's move all of our previous users of `import parent` over to the new
approach.

We can remove a couple of `parent.py` symlinks we had lying
around, as well.

We also remove `parent.py` itself, since otherwise vulture complains
about it being unused.  Subprojects will be forced to adjust on upgrade.

Remove a CodeQL exemption that was necessitated by our `import parent`
shenanigans.
2023-05-26 16:17:29 +02:00
Allison Karlitskaya
e4654eba6e various: fix up issues identified by ruff
These are all rather trivial changes made in response to suggestions
from running ruff.
2023-05-26 09:08:06 +02:00
Martin Pitt
1d08051f72 tools: More careful hostname check
Parse the URL and only compare the hostname against "example.com". Not
very relevant, but calms down CodeQL "Incomplete URL substring
sanitization" check.
2022-11-17 13:57:40 +01:00
Allison Karlitskaya
1a5a6e2ad1 tools/urls-check: some cleanups
Move user agent to the top as a constant.

Move the PatternFly exception also to the top by converting the
exceptions list into fnmatch patterns.

Get rid of the non-idiomatic use of /usr/bin/env to find python3.

Clean up some string formatting to use f-strings and be more generally
readable.
2022-04-20 19:18:18 +02:00
Matej Marusak
9c16c4b6e8 urls-check: Ignore patternfly redirects
PF maintainers often transfer issues between their repositories. This is
nothing we need to worry about.
2022-01-03 15:23:50 +01:00
Matej Marusak
94601ee5c3 tools: Update user-agent in urls-check script
It seems that just `Mozilla/5.0` was not enough for https://bugzilla.mozilla.org
as it was returning `400 - Bad request`. Putting there something more
real makes it work again.

Fixes #16221
2021-08-11 09:16:26 +02:00
Allison Karlitskaya
2a036ed2f2 various python: clean up sys.path hacks
The reason that the files in test/verify/ don't need to do sys.path
hacks is because parent.py has been symlinked into that directory.  Do
the same with the two other directories which contain scripts which need
such hacks, allowing a substantial cleanup.

Make some minor code adjustments to use the variables defined in
parent.py instead of the home-brew versions.

Add a somewhat dubious cwd= parameter to the `git grep` subprocess
invocation in tools/url-check.  This is strictly correct (since we want
to check the base directory, regardless of the current directory when
the tool was invoked) but we're really doing as a convenient way to
avoid a message about an unused import.
2021-04-07 10:14:05 +02:00
Allison Karlitskaya
2d0ec23cee urls-check: drop superfluous dont_write_bytecode = True
This has no impact because it's done after all the imports.  It's also
unprecedented: nothing else in the project does this, outside of the
run-tests scripts (which are in a bit of a different category, because
they import the testcases themselves).
2021-04-07 10:14:05 +02:00
Allison Karlitskaya
4077dee3d2 test-static-code: more pycodestyle adjustments
Our recent changes to test-static-code were causing node_modules/ and
bots/ to be scanned.  Steal some code from bots/ (thanks Matej) to make
sure we only scan files that are in git.  That should also speed things
up a little bit.

This causes us to find a couple of new files, so make the usual tweaks
to fix those ones up.

If we're not in a git checkout, #skip the test.
2021-04-06 10:14:28 +02:00
Matej Marusak
ccc8f64e9c tools: Link checker warns about redirections
Currently it would succeed, but would print this output:
```
Checked 133 URLs and all are valid.
Following URLs are redirected:
    http://cloud.redhat.com/insights
    http://cloud.redhat.com/insights/inventory/
    https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/8/html/managing_systems_using_the_rhel_8_web_console/managing-networking-in-the-web-console_system-management-using-the-rhel-8-web-console
    https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/8/html/managing_systems_using_the_rhel_8_web_console/managing-networking-in-the-web-console_system-management-using-the-rhel-8-web-console#configuring-network-bonds-using-the-web-console_managing-networking-in-the-web-console
    https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/8/html/managing_systems_using_the_rhel_8_web_console/managing-storage-devices-in-the-web-console_system-management-using-the-rhel-8-web-console
    https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/8/html/managing_systems_using_the_rhel_8_web_console/performing-basic-system-administration-tasks-in-the-web-console_system-management-using-the-rhel-8-web-console
    https://access.redhat.com/security/updates/classification/#
    https://apple.com/safari/
    https://developer.mozilla.org/en-US/docs/Web/API/WindowBase64/Base64_encoding_and_decoding
    https://developer.mozilla.org/en-US/docs/Web/API/WindowTimers/setTimeout
    https://developer.mozilla.org/en-US/docs/Web/Events/beforeunload
    https://firefox.com/
    https://github.com/patternfly/patternfly/issues/3053
    https://github.com/patternfly/patternfly-react/issues/4777
    https://github.com/storaged-project/storaged/issues/26
    https://github.com/storaged-project/storaged/pull/68
    https://github.com/systemd/systemd/blob/master/src/systemctl/systemctl.c
    https://google.com/chrome
    https://microsoftedge.com/
    https://patternfly-next-pr-2757.surge.sh/documentation/core/components/tabs
    https://udisks.freedesktop.org/docs/latest/gdbus-org.freedesktop.UDisks2.Block.html#gdbus-method-org-freedesktop-UDisks2-Block.GetSecretConfiguration
    https://www.patternfly.org/v4/components/table
    https://www.redhat.com/archives/libvir-list/2019-January/msg01104.html
```
2021-02-16 13:46:36 +01:00
Matej Marusak
caf26394d6 tools: Do not scan .svg files fro urls
Fixes #13632
Closes #13748
2020-03-18 17:39:16 +01:00
Matej Marusak
350273fa3a tools: Introduce script for checking of URLs
Closes #13461
2020-01-30 15:28:59 +01:00