OpenID support in FAS is going away. openQA has OAuth2 support.
I've tested this config to work with manual edits on lab, now
ansiblizing it (for lab only to start with).
Signed-off-by: Adam Williamson <awilliam@redhat.com>
forgot to do this before, we don't want it now as I'm about to
push an untested build there for stg to try out.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
so, this was working before somehow, but it was pretty clearly
wrong. We were using queues owned by openqa.stg on the prod
rabbitmq instance for the cases where the openQA "stg" consumers
need to listen to prod queues. This can only have been working
with an openqa.stg user on prod, which seems wrong. Instead,
these three consumers should do it the way the relval and
relvalami consumers do - use a queue owned by the "openqa" user,
but with a suffix so they have a different queue from the actual
prod queue. The upshot of this is that in the configs, we should
go from:
amqp_url = "amqps://openqa:@rabbitmq.fedoraproject.org/%2Fpubsub"
...
queue = "openqa.stg_scheduler"
- which is weird and I have no idea how it ever worked - to:
amqp_url = "amqps://openqa:@rabbitmq.fedoraproject.org/%2Fpubsub"
...
queue = "openqa_scheduler_stg"
- which seems much more sensible.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
This is triggered by
https://pagure.io/fedora-infrastructure/issue/11375 , but the
changes are rather extensive. Unfortunately, some of the
relevant files got messed up by the alphabetical sort thing that
got run on several group variable files a while ago, so that
confuses the diff a bit - I had to unwind those changes to make
the files readable again in order to make these changes.
Ultimately the goal here is to make the config more consistent
and more functional - the variables used and their names should
be more consistently related to what they're actually *for*,
which I didn't entirely understand when setting this up. So
we have variables for the username being used in each case and
we use that variable where we're referring to the username, for
instance. This should also make the whole thing about the cases
where listeners on the openQA stg/lab instance need to listen
to prod messages clearer, too. It also makes the user creation
clearer by doing it explicitly, just once per user, instead of
haphazardly doing it implicitly through the queue definitions.
And finally it should also actually fix 11375, by giving the
appropriate write permissions to each user.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
The 2023-05 git builds look good on stg, and are now in u-t, so
drop the scratch repos on stg and enable u-t on prod so we get
the builds from u-t everywhere.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
The update isn't in u-t yet (waiting for push) so I'm using the
builds, but this has been in testing on stg for a week and it
seems fine.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
I think the recent git builds are good and I want them everywhere
so I can merge a bunch of things and clean up before the weekend.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
Upstream implemented a feature that we can use to do the same
thing using just a test variable, so we're switching to that.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
The build prod is on has hit stable, I want to send a new build
to testing for lab to test out, don't want it on prod yet.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
This reverts commit edb9704acc.
The new prod deployment seems to have some issues (message
publishing is timing out) so let's go back to using the old
deployment for now.
We don't need the scratch builds on stg any more now we've sent
the new builds to updates-testing. We *do* want to use the builds
from u-t on prod now (I don't want to wait a week for the stable
push).
Signed-off-by: Adam Williamson <awilliam@redhat.com>
We don't need scratch builds on lab or updates-testing on prod
any more, latest builds are stable.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
This unifies prod and stg onto the ways of doing things for the
latest packages, and rejigs the swtpm stuff a bit to tear down
more (we shouldn't need the custom SELinux policy any more).
Signed-off-by: Adam Williamson <awilliam@redhat.com>
This was done using yq (
https://mikefarah.gitbook.io/yq/operators/sort-keys )
Doing things this way makes it much easier to see if a variable is set
in a file or if two hosts differ in what variables they set. Hopefully
we can keep things sorted moving forward.
Basically this means just sort a-z anything you add to any host or group
vaiable and it will be in the right place.
Additionally, this enforces 'normal' intent rules for all the variable
files which we should also try and obey. 2 spaces for first level, 3 for
next, etc. When in doubt you can run yq on it.
This should cause NO actual vairable changes, it's all just readability
fixing for humans, ansible parses it exactly the same.
Signed-off-by: Kevin Fenzi <kevin@scrye.com>
We don't need the scratch builds any more, they're in u-t. We
also don't need to use a side branch on lab createhdds any more,
or deploy from u-t on prod.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
The latest versions that were previously deployed as scratch
builds on lab are now in u-t, and ran fine on lab for a week, so
let's just deploy them from u-t to both instances now.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
Drop updates-testing repo for prod (though might have to put it
back next week). Use a scratch os-autoinst current git build on
lab to see if it works.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
Latest builds in u-t are quite well tested and I want them on
prod, don't want to wait a week for the stable push.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
The main goal of these changes is to allow all workers in each
deployment NFS write access to the factory share. This is because
I want to try using os-autoinst's at-job-run-time decompression
of disk images instead of openQA's at-asset-download-time
decompression; it avoids some awkwardness with the asset file
name, and should also actually allow us to drop the decompression
code from openQA I think.
I also rejigged various other things at the same time as they
kinda logically go together. It's mostly cleanups and tweaks to
group variables. I tried to handle more things explicitly with
variables, as it's better for use of these plays outside of
Fedora infra.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
I just enhanced the scheduler code so we can share the stock
WANTED definition (which now includes all arches) between prod
and lab, but filter the arches with a config file setting. This
means we don't have to carry and install a whole custom WANTED
file with the extra arches for lab any more, we just set the
appropriate value in the config file. Also drop some stuff from
the config file that's been useless since we switched to
fedora-messaging.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
This pretty much nukes all inventory entries and variables
relating to the old phx2 openQA deployments, as I don't think
they're ever coming up again. I left a few group entries around
for now just in case they're necessary for decommissioning.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
stg's disk is groaning at the seams, let's drop this to 400GB to
try and give it a bit of room to breathe.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
I enhanced the fedora_openqa job scheduling consumer to have a
setting for which arches update tests should be run on. This
change adds that new setting to the config we write, and enables
ppc64le update tests in staging. If this goes well we will also
enable them in production, and later enable aarch64 as well when
we have more workers.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
Also tweak how we do the plugin config a bit, I don't like the
whole 'do special stuff if deployment_mode is set' thing any
more.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
Now I've built the whole thing and it works, I feel like this
is a cleaner design: we use a lot more role variables in the
messaging config files, set defaults for them that are
appropriate for non-infra use, and set the correct values for
the infra servers in the group vars. Let's just hope I got it
all right...
Signed-off-by: Adam Williamson <awilliam@redhat.com>
There are several variables we set to the same value for openQA
prod and staging servers. Instead of duplicating these between
the openqa and openqa_stg groups, let's invent an openqa_common
and put them in there, and add both servers to it.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
env_suffix isn't actually set to .stg on openQA staging because
it's not a normal staging host. Let's not set it in case it
breaks other plays, but use a custom var instead.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
This is a big change to support fedora-messaging based consumers
instead of fedmsg ones in the openqa/dispatcher role. This needs
the fedora-messaging branch of fedora_openqa, until that is
merged to stable. For now this is all conditionalized so prod
will continue to use fedmsg. This also improves some variable
usage and documentation in the fedmsg path too.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
This is a slightly inexact science, but I think we have solid
free space on both servers and it does seem like assets are
getting thrown away pretty aggressively, so let's see how this
does.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
