Commit Graph

66 Commits

Author SHA1 Message Date
Adam Williamson
f4133798f6 openqa/server: make geekotest static UID/GID optional
I'm still kinda trying to make the openqa roles usable outside
of infra, so now I have a minute, let's do this: it makes the
static UID/GID for geekotest optional and configurable, instead
of hard coding it. For infra we set the value to 601, as we
are already using, in the openqa and openqa-stg group config.
2016-03-19 16:28:09 -07:00
Adam Williamson
461aac031c openqa_server: tweak the asset size changed_when a bit
just in case we wind up with multiple job groups, this should
be more correct.
2016-03-19 16:14:03 -07:00
Adam Williamson
2bbac61464 whoops, add a missing quote 2016-03-19 16:10:14 -07:00
Adam Williamson
cb7bfcb614 openqa: allow specifying asset size limit
this is a database value and there's no openQA API function to
set it, so we have to do it directly in the database...this
*should* work. I think. I should add equivalent functionality
for sqlite use as well, really...
2016-03-19 16:09:20 -07:00
Adam Williamson
eb44a82979 openqa/server: run createhdds with python3
there seems to be a bug in python2-guestfs which causes the
disk image with an updates.img file to be broken (the updates
image is only 4096 bytes long and incomplete). When built with
Python 3 the image seems to be fine. So, run the script with
python3 (as its hashbang says) and ensure necessary packages
are installed.
2016-03-18 15:37:21 -07:00
Adam Williamson
c588b55419 openqa/server: whoops, gotta clean this too 2016-03-10 20:20:31 -08:00
Adam Williamson
39c72cc3e4 clean up openqa/server a bit more
I'd like to revise this a bit to be usable outside infra again,
but tomorrow...or later...
2016-03-10 20:19:39 -08:00
Adam Williamson
8d4a492b02 drop the openqa/fixes role, threebean says it's unneeded
apparently the policy.py file shouldn't cause any problems.
2016-03-10 20:06:02 -08:00
Adam Williamson
2d5b6329ef openqa: drop fedmsg/relay role (not needed), drop relay 'fixes'
let's start over and see where we're at.
2016-03-10 18:54:08 -08:00
Adam Williamson
d7c79c5cb0 openqa/fixes: try a custom relay.py for prod...
seems like we need the internal inbound relay but the public
outbound relay? I don't even know...but we definitely can't
connect to busgateway01.phx2.fedoraproject.org:3999
2016-03-10 17:30:09 -08:00
Adam Williamson
52c2bb1cc0 openqa/fixes: don't do the relay fix on prod
i think the relay 'fix' is only needed for stg, because there
was a firewall rule added for prod but not stg. It's not really
a 'fix' either (it'll stop messages getting out) but it at
least prevents fedmsg-relay failing, so keep it for now.
2016-03-10 17:18:51 -08:00
Adam Williamson
1a33048416 openqa: install openqa-plugin-fedmsg 2016-03-10 17:07:13 -08:00
Adam Williamson
5de040a201 openqa/server: enable fedmsg plugin 2016-03-10 16:56:47 -08:00
Adam Williamson
aa8cf397ee openqa: add a dumb 'fixes' role to tweak fedmsg config
Ralph *mostly* fixed it, but the config we get from fedmsg/base
still doesn't quite work, so this just hacks it up after that
role's done. This will go away with a couple more fixes to the
fedmsg/base role.
2016-03-10 16:55:31 -08:00
Adam Williamson
2feaf0e2ef openqa/server: allow apache to read from NFS 2016-03-08 14:26:46 -08:00
Adam Williamson
a8eba01a56 openqa/check-compose: tweak package installs and sources
we need some from updates-testing for now. also, we need
openqa-httpd for the server.
2016-03-08 14:09:28 -08:00
Adam Williamson
fc588796e5 openqa/server: actually set geekotest user group to geekotest 2016-03-08 13:59:03 -08:00
Adam Williamson
59cbb73597 actually python2-fedfind as we explicitly run it under python2 2016-03-08 12:47:43 -08:00
Adam Williamson
a167d37411 openqa/server: createhdds.py needs python3-fedfind 2016-03-08 12:45:14 -08:00
Adam Williamson
b718b8fbd4 openqa: test sourcing server data from NFS on stg
Well hey, let's hope this works!
2016-03-08 12:16:24 -08:00
Adam Williamson
61355acddf openqa/dispatcher: drop the old service cleanup stuff
we only needed it once.
2016-03-07 16:51:34 -08:00
Adam Williamson
cb951141c5 openqa_dispatcher: correct a paste-o in fedmsg consumer copy 2016-03-07 12:13:15 -08:00
Adam Williamson
9d4634759b openqa_dispatcher: update a comment 2016-03-07 12:09:00 -08:00
Adam Williamson
651061ea10 openqa_dispatcher: revise for fedmsg hub consumer
ditch the old systemd consumer service, install the fedmsg
consumer config file, add some fedmsg-hub reload notifications.
2016-03-07 12:06:41 -08:00
Adam Williamson
73da02a655 openqa: make openQA and FAS credentials root.fedmsg 0640
with openQA jobs being scheduled and wiki results reported
(well, when we get that working again) by fedmsg-hub consumers,
we have to let the fedmsg user read these files.
2016-03-07 11:26:16 -08:00
Adam Williamson
f7ea6b51c0 also trim openqa-dispatcher fedmsg reqs 2016-03-04 14:39:43 -08:00
Adam Williamson
7c30dbe0ff drop some old cleanup bits now they've run 2016-03-03 18:09:00 -08:00
Adam Williamson
95a286e23b openqa_dispatcher and check-compose: package dep fixes 2016-03-03 15:59:05 -08:00
Adam Williamson
35824c1ad4 openqa-consumer: restart service on git changes
...otherwise you end up scheduling jobs with stale code and
they fail, NOT THAT THAT'S EVER HAPPENED TO ME
2016-03-02 08:07:57 -08:00
Adam Williamson
389adc835a openqa-dispatcher: damnit, package name 2016-02-24 11:06:51 -08:00
Adam Williamson
0b4e759d01 openqa-dispatcher: requires python-fedmsg now 2016-02-24 11:02:57 -08:00
Adam Williamson
514f9ecba6 openqa: d'oh, type (service not server) 2016-02-24 10:51:55 -08:00
Adam Williamson
24435918ea openqa: adjust for changes to scheduler design
mainly systemd unit changes, clean up the old ones, enable the
new.
2016-02-24 10:49:05 -08:00
Adam Williamson
016f04c30d openqa: enable asset downloading from fedoraproject.org 2016-02-18 12:06:46 -08:00
Adam Williamson
7eacf03ebc drop use of COPR for openQA
all required packages except openqa itself are now in main repo,
openqa is in infra repo.
2016-02-15 16:16:19 -08:00
Kevin Fenzi
9006a36832 Update all calls to sudo to use 'become' and sudo_user to 'become_user' 2016-02-02 21:24:45 +00:00
Adam Williamson
330d58375d openqa_dispatcher: adjust for changed path in tools
we moved the openqa scheduler within the git checkout.
2016-01-22 13:43:39 -08:00
Adam Williamson
78755e7a36 openqa: enable openQA-stg COPR on staging hosts 2016-01-15 17:33:30 -08:00
Adam Williamson
b3851786fa openqa-server: never register 'changed' for this (check) step 2016-01-12 17:21:36 -08:00
Kevin Fenzi
f0c80375b5 Remove action: in all roles. 2016-01-06 21:58:31 +00:00
Adam Williamson
cc8d3896a7 openqa: use 'shell' not 'command' for a step that needs it
this step uses && and >, needs a shell.
2016-01-04 17:10:19 -08:00
Adam Williamson
c1f4d29dd8 openqa: drop some more always_runs
we really shouldn't always_run these, as they actually *do*
stuff. That does mean that in check mode we won't check if
database init and admin user creation are necessary, but these
are only ever necessary exactly one time at initial deployment
which is likely to be kicked off manually anyway, so it should
not be a problem. We adjust the 'when' condition for the 'wait
for services to start up' action, so it will work in check mode
with the 'services' variable undefined (because the step that
defines it now won't be run).
2016-01-04 17:01:12 -08:00
Adam Williamson
87ba685b13 openqa: drop always_run from git actions
it causes problems (prevents updated code being installed) and
is not necessary (git module supports check).
2016-01-04 16:51:22 -08:00
Adam Williamson
3a114ab7ca openqa: no, seriously, fuck dnf 2015-12-18 19:52:34 -08:00
Adam Williamson
227cdf23b9 openqa: fucking dnf 2015-12-18 19:51:17 -08:00
Adam Williamson
3bd08a441f openqa: run createhdds.py with py2 for now
and install the additional deps.
2015-12-18 19:49:39 -08:00
Adam Williamson
ee8ab4e662 enable openqa disk image generation
the servers have nested virt enabled now, so this should be OK
2015-12-18 19:43:47 -08:00
Adam Williamson
3373f9938d openqa_server: switch to createhdds.py, disable creation for now
This puts in place the bits needed to use the new createhdds.py
for image generation, which works much nicer with ansible than
the old createhdds.sh. For now, the disk creation step is
disabled, because creating some images (including the ones that
get refreshed regularly) requires running VMs, and the openqa
server hosts are VMs themselves and do not currently have nested
virt enabled, so the creation would run extremely slow. Once we
get nested virt enabled, I'll change the condition for the image
creation step to what it should be. Until then I'll carry on
handling image creation manually (both hosts have all the images
at present anyhow).
2015-12-17 13:29:16 -08:00
Adam Williamson
d8ef7c0ad6 openqa_server: try a new approach for checking test changes
basically, dump the config from the server before loading the
templates file, dump it after, and use json_diff to compare.

I wanted to make the actual template loading script from openQA
do some kind of check in '--clean' mode, but given how it's
designed that's not really terribly practical without doing
some major re-architecting, and this seems like a relatively
simple alternative. With this change, the actual "Load tests"
step will never show 'changed', but the subsequent 'Check if
the tests changed in previous step' step will show as changed
when appropriate.
2015-12-15 14:18:14 -08:00
Adam Williamson
402bfe6215 openqa_server: use guestfs direct backend for creating disks
this avoids the need to start and stop libvirtd.
2015-12-15 11:30:13 -08:00