Commit Graph

128 Commits

Author SHA1 Message Date
Adam Williamson
aa2a002a96 Change how we get the HTML file accessible in fedora_nightlies
Just can't get Apache config Alias to work for some reason, so
let's go with the flow and stick the file in openQA's public
directory. This works!

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-11-21 18:37:03 -08:00
Adam Williamson
efb353bc02 Let's make that IncludeOptional so lab doesn't die
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-11-21 17:47:23 -08:00
Adam Williamson
4851dc8d65 Try and do fedora_nightlies Apache config without breaking openQA
Er, oops. This involves a hack, but at least it doesn't take the
openQA web UI offline.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-11-21 17:43:55 -08:00
Adam Williamson
813bbc4d2a openqa/server: allow group to write to factory dirs
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-11-05 17:16:28 -08:00
Adam Williamson
61251d0b11 More syntax...sigh
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-11-05 16:24:27 -08:00
Adam Williamson
95f062c07a openQA: allow all workers NFS write access, other tweaks
The main goal of these changes is to allow all workers in each
deployment NFS write access to the factory share. This is because
I want to try using os-autoinst's at-job-run-time decompression
of disk images instead of openQA's at-asset-download-time
decompression; it avoids some awkwardness with the asset file
name, and should also actually allow us to drop the decompression
code from openQA I think.

I also rejigged various other things at the same time as they
kinda logically go together. It's mostly cleanups and tweaks to
group variables. I tried to handle more things explicitly with
variables, as it's better for use of these plays outside of
Fedora infra.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-11-05 16:10:32 -08:00
Adam Williamson
c2023d5560 openQA: try to make NFS mount changes more robust
On client end, restart mount unit (with daemon-reload) if mount
file changes. On server end, run exportfs -r if export config
file changes.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-10-30 14:06:07 -07:00
Adam Williamson
c3b87d88d1 openqa/server: allow template dump to fail
It will on first deployment. That's fine.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-06-05 16:05:18 -07:00
Adam Williamson
0dc0dd6659 openqa: slightly broaden "(N|n)ot a git repository" check
I wrote it as "Not" before, so I presumably saw the message that
way, but today it seems to be "not". Let's just skip the letter.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-06-01 17:09:38 -07:00
Adam Williamson
32f9933aad openqa/server: drop createhdds stuff
This was disabled due to a bug for some time now. Originally I
meant to turn it back on, but now I don't think I do: it makes
more sense to just keep letting the worker hosts handle disk
image building, it doesn't make any sense to have the server do
it for x86_64 but worker hosts do it for other arches. If the
server can't do it *all*, we may as well be consistent across
arches and always have the worker hosts do it.

This does mean that on initial deployment using these plays there
is a time where the server is up and running but any jobs run
that need the base disk images will fail because the worker play
won't have built them yet. But I think that's not a big problem,
and it was already the case for non-x86_64 arches anyhow.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-05-06 14:27:37 -07:00
Adam Williamson
26005bf805 openqa: correct scratch repo config filename
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-04-30 12:38:48 -07:00
Adam Williamson
ba8c7b49ff openqa: create repodata for scratchrepo
Whoops.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-04-30 12:35:00 -07:00
Adam Williamson
255ce6ebad openqa/server: use jq for JSON comparison as json_diff died
Swiped from https://stackoverflow.com/questions/31930041/

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-04-30 12:33:50 -07:00
Adam Williamson
bb1525bdef openqa/{server,worker}: enhance package handling
This provides a mechanism for deploying scratch builds, and also
for controlling whether or not to install openQA and os-autoinst
from updates-testing.

I have been doing the scratch build thing for years already, just
manually by ssh'ing into the boxes. This is getting tiring now
we have like 15 worker hosts.

The scratch build mechanism isn't properly idempotent, but fixing
that would be hard and I really only intend to use it transiently
when I'm updating the packages, so I don't think it's worth the
effort.

This also adds a notification for restarting openQA worker
services when the packages or config are updated, and fixes the
worker playbook to enable the last worker service.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-04-30 12:23:57 -07:00
Adam Williamson
d86a76b4d1 openqa/server: fix issues reported by ansible-lint
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-04-29 17:32:16 -07:00
Adam Williamson
52d7450a9c openqa, check-compose, relvalconsumer: drop remaining fedmsg bits
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-04-24 21:34:26 +02:00
Adam Williamson
235e5f9e65 openqa/server: fix FIF template loading
Need to run from correct directory...

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-04-24 21:34:23 +02:00
Adam Williamson
c95f8c78ff openqa/server: update fifloader executable name
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-04-24 21:34:22 +02:00
Adam Williamson
bc681d703d openqa/server: handle FIF format template loading
We just invented a new format for openQA templates. This makes
the openqa/server role handle loading templates in either format.
I'll remove old-format loading when we're done tweaking the new
setup and it's deployed to prod.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-04-24 21:34:21 +02:00
Adam Williamson
08c8bd379a openqa/server: tweak usernames/passwords for cloud-init config
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-04-24 21:34:16 +02:00
Adam Williamson
29a666e2a3 openqa/server: create a cloud-init ISO
We're going to test Cloud images in openQA now autocloud is
retiring. We need a cloud-init ISO to be able to boot them.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-04-24 21:34:16 +02:00
Adam Williamson
f29f9a38e5 openqa: update libsemanage python package names too
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-04-24 21:34:14 +02:00
Adam Williamson
b916a7cba6 openqa: update python selinux package name for Python 3
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-04-24 21:34:14 +02:00
Adam Williamson
45b6adb9e1 openqa/server: fixes for the AMQP publishing config
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-04-24 21:34:07 +02:00
Adam Williamson
ab4a46bf9e openQA: try and set up fedora-messaging publishing on staging
Also tweak how we do the plugin config a bit, I don't like the
whole 'do special stuff if deployment_mode is set' thing any
more.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-04-24 21:34:07 +02:00
Paul W. Frields
7ef3be1d94 openqa: all roles change dnf -> package 2019-06-11 14:39:52 +00:00
Kevin Fenzi
4b31ac5152 ansible: Change all our group names from foo-bar to foo_bar or foo-bar-baz to foo_bar_baz
In ansible 2.8 the - character isn't supposed to be valid in group names.
While we could override this, might has well just bite the bullet and change it.
So, just switch all group names to use _ instead of -

Signed-off-by: Kevin Fenzi <kevin@scrye.com>
2019-05-20 17:38:09 +00:00
Adam Williamson
ed4f45b314 openqa/server: set config option not to cancel whole job cluster
This is the config option added in the most recent openqa build
which prevents it cancelling an entire cluster of parallel jobs
if just one of the children fails. See:

https://progress.opensuse.org/issues/46295

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2019-03-05 10:04:48 -08:00
Adam Williamson
b30732e722 openqa etc.: drop Python 2 -> 3 migration bits
Both stg and prod are migrated now, so let's clean up the plays
to be pure Python 3.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2018-11-30 10:13:56 -08:00
Adam Williamson
31115c12ab openqa: Enable openqa-livehandler.service for prod too
...since I'm about to upgrade it. Whee

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2018-11-27 14:42:00 -08:00
Adam Williamson
6c390c669b First cut at migrating openqa-stg to fedmsg-3
This includes some tweaks to the core fedmsg roles to allow a
'generic' way of indicating that a box should use fedmsg-hub-3
not fedmsg-hub, and make the restart notification work for that.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2018-11-26 10:53:16 -08:00
Adam Williamson
4f684bf9e1 openqa/server: enable additional service for new openqa
Only on stg for now (as we're only deploying new openqa there
for now).

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2018-11-19 15:30:26 -08:00
Adam Williamson
c69c2fd485 openqa/relvalconsumer: fix all other dnf loop cases to use lists
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2018-10-09 13:50:39 -07:00
Kevin Fenzi
e231493b1e Fix typo. I have no idea how this worked or ran since march. 2018-07-11 22:10:01 +00:00
Kevin Fenzi
a8714caab3 first cut at changing all the old |changed to is changed per ansible deprecations 2018-05-07 23:51:48 +00:00
Adam Williamson
a8ee935cbe Fix 'when' condition for previous commits
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2018-03-22 16:14:10 -07:00
Adam Williamson
e6fbc29d42 Try and fix previous commit (I hate sql)
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2018-03-22 16:11:12 -07:00
Adam Williamson
e069489ecc Tweak openQA asset size settings again
The UI makes this easier to investigate now, and it looks like
we could give x86_64 a bit more space, but other arches less.
So let's tweak things to do that. This should also reduce
overall usage on staging a bit, as it's up against its limits.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2018-03-22 16:06:42 -07:00
Adam Williamson
672bf06b04 openqa: do createhdds on x86_64 workers for now
1539330 is turning out to be pretty intransigent and the images
are getting very old, plus we need f28 branched base images. So
let's do image builds on the x86_64 workers for now. Also re-
enable image builds on workers - I never should've turned that
off in the first place, as they shouldn't be susceptible to the
bug.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2018-02-23 17:14:28 -08:00
Adam Williamson
69ed1bb554 openqa: disable createhdds cron jobs (#1539330)
Yeah, daily crashes are bad.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2018-01-28 01:08:44 +01:00
Adam Williamson
25d1c79a4b Tweak openQA asset size settings again
Staging is running out of space...let's kick it back down to
300, and also create a separate setting for update group asset
size. We test lots of updates, and for each update we only need
to upload one disk image, so we really don't need 300GB of
asset space for update job groups, that just means we'll keep
like 300 update disk images lying around. If PPC starts getting
incompletions again I'll have to, uh, do something? Yeahhh.
Something.
2017-11-30 11:48:22 -08:00
Adam Williamson
4a6d33d863 openqa, relvalconsumer: need fedfind/wikitcms from u-t for now 2017-11-10 18:34:10 -08:00
Adam Williamson
8969b508a8 Whoops, == not =
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2017-09-07 10:39:57 -07:00
Adam Williamson
ba788afe08 openqa/dispatcher: don't check out tests when locally modified
Also don't check out the tests if we're on master branch, but
have local modifications.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2017-09-07 09:56:23 -07:00
Adam Williamson
e6ee55eb78 openqa/server: no root squash for rw access to factory share
Might be better to do this with some other account, but uh, for
now...
2017-08-17 14:22:42 -07:00
Adam Williamson
b5f825741a openqa/server: allow rw access to nfs share for ppc workers
We need to create the ppc hard disk images from the ppc workers,
so they need rw access to the share.
2017-08-17 14:17:18 -07:00
Adam Williamson
e9ba26289a openqa/server: *REALLY* fix the Apache status thing
OK, this version actually seems to work.
2017-03-14 13:10:27 -07:00
Adam Williamson
db1150c824 Revert "openqa/server: try fixing the Apache status thing again"
This reverts commit d6e32c9901.
It didn't work.
2017-03-14 12:55:51 -07:00
Adam Williamson
d6e32c9901 openqa/server: try fixing the Apache status thing again
Seems like the config needs to be in the VirtualHost definition.
2017-03-14 12:53:38 -07:00
Adam Williamson
acf096e829 openqa/server: Mostly revert previous commit
Actually openQA playbook already includes the apache role, which
sets up the config file. The 404s are for some other reason. But
keep the comment change.
2017-03-14 12:16:12 -07:00