Commit Graph

141 Commits

Author SHA1 Message Date
Adam Williamson
5da2faac67 openqa/server: allow OAuth2 authentication, enable on lab
OpenID support in FAS is going away. openQA has OAuth2 support.
I've tested this config to work with manual edits on lab, now
ansiblizing it (for lab only to start with).

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2025-03-28 13:40:57 -07:00
Michal Konecny
2ec055db6f Use first uppercase letter for all handlers
This will unify all the handlers to use first uppercase letter for
ansible-lint to stop complaining.

I went through all `notify:` occurrences and fixed them by running
```
set TEXT "text_to_replace"; set REPLACEMENT "replacement_text"; git grep
-rlz "$TEXT" . | xargs -0 sed -i "s/$TEXT/$REPLACEMENT/g"
```

Then I went through all the changes and removed the ones that wasn't
expected to be changed.

Fixes https://pagure.io/fedora-infrastructure/issue/12391

Signed-off-by: Michal Konecny <mkonecny@redhat.com>
2025-02-10 20:31:49 +00:00
Kevin Fenzi
6c38d7b61a various: fix some more shell variables that were accidentally converted to builtin.shell
Signed-off-by: Kevin Fenzi <kevin@scrye.com>
2025-01-15 17:26:50 -08:00
Ryan Lerch
47c68f478d ansiblelint fixes - fqcn[action-core] - template to ansible.builtin.template
Replaces references to template: with ansible.builtin.template

Signed-off-by: Ryan Lerch <rlerch@redhat.com>
2025-01-15 11:30:29 +10:00
Ryan Lerch
3c41882bb0 ansiblelint fixes - fqcn[action-core] - shell to ansible.builtin.shell
Replaces references to shell: with ansible.builtin.shell

Signed-off-by: Ryan Lerch <rlerch@redhat.com>
2025-01-15 11:29:10 +10:00
Ryan Lerch
25391e95b7 ansiblelint fixes - fqcn[action-core] - package to ansible.builtin.package
Replaces many references to  package: with ansible.builtin.package

Signed-off-by: Ryan Lerch <rlerch@redhat.com>
2025-01-15 11:28:00 +10:00
Ryan Lerch
462176464b ansiblelint fixes-- fqcn[action-core] - command to ansible.builtin.command
Replaces many references to  command: with ansible.builtin.command

Signed-off-by: Ryan Lerch <rlerch@redhat.com>
2025-01-15 11:26:47 +10:00
Ryan Lerch
6a3816dfdc ansiblelint fixes-- fqcn[action-core] - copy to ansible.builtin.copy
Replaces many references to 'copy' with ansible.builtin.copy

Signed-off-by: Ryan Lerch <rlerch@redhat.com>
2025-01-15 10:43:31 +10:00
Ryan Lerch
62952df107 ansiblelint fixes-- fqcn[action-core] - file to ansible.builtin.file
Replaces many references to  file: with ansible.builtin.file

Signed-off-by: Ryan Lerch <rlerch@redhat.com>
2025-01-15 10:41:52 +10:00
Ryan Lerch
691adee6ee Fix name[casing] ansible-lint issues
fix 1900 failures of the following case issue:

`name[casing]: All names should start with an uppercase letter.`

Signed-off-by: Ryan Lerch <rlerch@redhat.com>
2025-01-14 20:20:07 +10:00
Ryan Lerch
89f6f1fc32 Fix majority of remaining yamllint warnings and errors
Signed-off-by: Ryan Lerch <rlerch@redhat.com>
2024-11-28 17:31:45 +10:00
Adam Williamson
4d801444a9 openqa: set up a side repo for prod as well as lab
Sometimes we want to deploy something to prod before it goes
stable (or even to u-t).

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2024-11-25 17:06:34 -08:00
Adam Williamson
1a537f38ce openqa/server: correct scratchrepo removal
d'oh. this has been broken for some time...

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2024-11-20 17:27:57 -08:00
Adam Williamson
530f69d967 openqa: use an external side repo for test builds
It's overall simpler and more idempotent to just use a side repo
maintained outside of ansible than re-create one on each system
on each run of the plays.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2023-10-27 11:20:58 -07:00
Adam Williamson
de979123fa openQA: don't install the fedoraupdaterestart plugin any more
We don't need it, we use upstream RETRY now.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-12-19 16:16:11 -08:00
Adam Williamson
8e891fe4d5 openqa/server: update for git default branch rename
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-07-14 11:58:09 -07:00
Adam Williamson
7ba67fdc12 openQA: don't enable FedoraUpdateRestart plugin
Upstream implemented a feature that we can use to do the same
thing using just a test variable, so we're switching to that.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-07-06 10:42:26 -07:00
Adam Williamson
a91dfc29e9 openqa: twiddle with the delegation stuff again
Ugh, we delegate for the assetsize stuff too and there's tons of
that, splitting it would be awful. Let's try a different approach
with a new optional variable for the delegate target.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-06-07 16:32:04 -07:00
Adam Williamson
42e930e97f openqa-onebox: tweak db host stuff
Using the machine's own hostname works for the ansible delegate
stuff but doesn't work for openQA itself (if you try and access
the DB by hostname like this, postgres denies access; you have
to use 'localhost' for postgres to allow it). Using 'localhost'
works for postgres but doesn't do the right thing for delegation.
Let's use 'localhost' and split the two play steps into
delegated and non-delegated versions.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-06-07 16:17:29 -07:00
Adam Williamson
6c2991306c openqa/server: only install nfs-utils when needed
If there are no NFS workers, we don't need the NFS server.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-06-07 15:21:15 -07:00
Adam Williamson
0cf8a59fd5 openqa: fix openqa_nfs_{worker,client}s confusion again
Missed from previous commit.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-06-07 13:26:22 -07:00
Adam Williamson
58dd80c799 openqa/server: reduce PPC update group asset size
We need to treat it and the x86_64 update group separately to
do this, but it really doesn't need 200G. We have images from
three weeks ago, and we don't need that kind of buffer, and space
is a bit tight.

Note: there is no aarch64 updates group as we do not currently
run updates tests on aarch64.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2022-03-22 16:17:17 -07:00
Adam Williamson
3dec01a15a openqa/server: set httpd_can_network_connect boolean again :(
Seems there's one more port that needs to be tagged before we
can finally unset this:
https://bugzilla.redhat.com/show_bug.cgi?id=1277312#c9

Keep the custom policy as well, though, so we just need to
update it when that port gets done.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2021-12-14 16:33:19 -08:00
Adam Williamson
2320eef5ee openqa/worker: create custom SELinux module directory first
Whoops. Also order these things a bit better.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2021-12-14 15:54:38 -08:00
Adam Williamson
edc4caa833 openqa/server: use custom SELinux policy instead of boolean
We've been using the httpd_can_network_connect boolean for years
to allow httpd to connect to the openQA server processes. This
is an unnecessarily large hammer when we only need it to be
able to connect to exactly the two openQA ports. This uses a
custom SELinux policy to allow connecting to those ports only,
and ensures the boolean is set back to off.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2021-12-14 15:48:34 -08:00
Adam Williamson
67eb9bb288 openqa/server: clean up and trim package requirements
Several of these requirements are old ones that were only needed
for createhdds, when we ran createhdds on the servers. All of
those can go. Also make the list line-by-line for easier git
blame tracking in future (and add comments for the remaining
entries so we know why they're there).

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2021-12-14 14:43:29 -08:00
Adam Williamson
813bbc4d2a openqa/server: allow group to write to factory dirs
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-11-05 17:16:28 -08:00
Adam Williamson
95f062c07a openQA: allow all workers NFS write access, other tweaks
The main goal of these changes is to allow all workers in each
deployment NFS write access to the factory share. This is because
I want to try using os-autoinst's at-job-run-time decompression
of disk images instead of openQA's at-asset-download-time
decompression; it avoids some awkwardness with the asset file
name, and should also actually allow us to drop the decompression
code from openQA I think.

I also rejigged various other things at the same time as they
kinda logically go together. It's mostly cleanups and tweaks to
group variables. I tried to handle more things explicitly with
variables, as it's better for use of these plays outside of
Fedora infra.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-11-05 16:10:32 -08:00
Adam Williamson
c2023d5560 openQA: try to make NFS mount changes more robust
On client end, restart mount unit (with daemon-reload) if mount
file changes. On server end, run exportfs -r if export config
file changes.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-10-30 14:06:07 -07:00
Adam Williamson
c3b87d88d1 openqa/server: allow template dump to fail
It will on first deployment. That's fine.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-06-05 16:05:18 -07:00
Adam Williamson
0dc0dd6659 openqa: slightly broaden "(N|n)ot a git repository" check
I wrote it as "Not" before, so I presumably saw the message that
way, but today it seems to be "not". Let's just skip the letter.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-06-01 17:09:38 -07:00
Adam Williamson
32f9933aad openqa/server: drop createhdds stuff
This was disabled due to a bug for some time now. Originally I
meant to turn it back on, but now I don't think I do: it makes
more sense to just keep letting the worker hosts handle disk
image building, it doesn't make any sense to have the server do
it for x86_64 but worker hosts do it for other arches. If the
server can't do it *all*, we may as well be consistent across
arches and always have the worker hosts do it.

This does mean that on initial deployment using these plays there
is a time where the server is up and running but any jobs run
that need the base disk images will fail because the worker play
won't have built them yet. But I think that's not a big problem,
and it was already the case for non-x86_64 arches anyhow.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-05-06 14:27:37 -07:00
Adam Williamson
26005bf805 openqa: correct scratch repo config filename
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-04-30 12:38:48 -07:00
Adam Williamson
ba8c7b49ff openqa: create repodata for scratchrepo
Whoops.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-04-30 12:35:00 -07:00
Adam Williamson
255ce6ebad openqa/server: use jq for JSON comparison as json_diff died
Swiped from https://stackoverflow.com/questions/31930041/

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-04-30 12:33:50 -07:00
Adam Williamson
bb1525bdef openqa/{server,worker}: enhance package handling
This provides a mechanism for deploying scratch builds, and also
for controlling whether or not to install openQA and os-autoinst
from updates-testing.

I have been doing the scratch build thing for years already, just
manually by ssh'ing into the boxes. This is getting tiring now
we have like 15 worker hosts.

The scratch build mechanism isn't properly idempotent, but fixing
that would be hard and I really only intend to use it transiently
when I'm updating the packages, so I don't think it's worth the
effort.

This also adds a notification for restarting openQA worker
services when the packages or config are updated, and fixes the
worker playbook to enable the last worker service.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-04-30 12:23:57 -07:00
Adam Williamson
d86a76b4d1 openqa/server: fix issues reported by ansible-lint
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-04-29 17:32:16 -07:00
Adam Williamson
52d7450a9c openqa, check-compose, relvalconsumer: drop remaining fedmsg bits
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-04-24 21:34:26 +02:00
Adam Williamson
235e5f9e65 openqa/server: fix FIF template loading
Need to run from correct directory...

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-04-24 21:34:23 +02:00
Adam Williamson
c95f8c78ff openqa/server: update fifloader executable name
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-04-24 21:34:22 +02:00
Adam Williamson
bc681d703d openqa/server: handle FIF format template loading
We just invented a new format for openQA templates. This makes
the openqa/server role handle loading templates in either format.
I'll remove old-format loading when we're done tweaking the new
setup and it's deployed to prod.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-04-24 21:34:21 +02:00
Adam Williamson
29a666e2a3 openqa/server: create a cloud-init ISO
We're going to test Cloud images in openQA now autocloud is
retiring. We need a cloud-init ISO to be able to boot them.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-04-24 21:34:16 +02:00
Adam Williamson
f29f9a38e5 openqa: update libsemanage python package names too
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-04-24 21:34:14 +02:00
Adam Williamson
b916a7cba6 openqa: update python selinux package name for Python 3
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-04-24 21:34:14 +02:00
Adam Williamson
45b6adb9e1 openqa/server: fixes for the AMQP publishing config
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-04-24 21:34:07 +02:00
Adam Williamson
ab4a46bf9e openQA: try and set up fedora-messaging publishing on staging
Also tweak how we do the plugin config a bit, I don't like the
whole 'do special stuff if deployment_mode is set' thing any
more.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-04-24 21:34:07 +02:00
Paul W. Frields
7ef3be1d94 openqa: all roles change dnf -> package 2019-06-11 14:39:52 +00:00
Adam Williamson
b30732e722 openqa etc.: drop Python 2 -> 3 migration bits
Both stg and prod are migrated now, so let's clean up the plays
to be pure Python 3.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2018-11-30 10:13:56 -08:00
Adam Williamson
31115c12ab openqa: Enable openqa-livehandler.service for prod too
...since I'm about to upgrade it. Whee

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2018-11-27 14:42:00 -08:00
Adam Williamson
6c390c669b First cut at migrating openqa-stg to fedmsg-3
This includes some tweaks to the core fedmsg roles to allow a
'generic' way of indicating that a box should use fedmsg-hub-3
not fedmsg-hub, and make the restart notification work for that.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2018-11-26 10:53:16 -08:00