The capability to handle a variance between prod and staging
here is just temporary while I'm testing the new fixed asset
handling stuff by deploying it on staging. Once it's tested
and merged we'll just have prod and staging do the same thing.
But for now we need to cleanly handle them having the static
disk images in different places.
I've enhanced `createhdds check` to exit 1 if all images are
present but some are old, and 2 if any images are missing. We
use this to only create images if any are missing here in the
play; we rely on the daily cron job to rebuild old images.
This is kind of a band-aid for a weird issue on openqa01 where
virt-install runs just don't seem to work properly after the
box has been running for a while, so createhdds doesn't actually
work and any playbook run gets hung up on it for a long time.
This doesn't fix that, but does at least mean we can run the
playbook without being bothered by it. To get createhdds to run
properly and actually regenerate the outdated images, we have
to reboot the system and run it right away, it seems to work
fine right after the system boots up.
we need to install some additional packages for the revised
createhdds (but we no longer need pexpect), and ensure libvirtd
is running before running createhdds.
it's not really fatal when it fails (except on first deployment)
and nothing else later depends on it, so we can go ahead and
continue the run even if it fails
instead of just relying on it getting run when we do an
ansible run, since that's intermittent and it's annoying
when you want to do an ansible run and it sits there for
hours creating disk images. This way we'll know they'll
get updated regularly and ansible runs should never get
blocked on image creation, though we still do it in the
ansible plays just in case (and for initial deployment).
This should now be safe, with the recent changes to make it
time out gracefully and run atomically. We also use withlock
to make sure we don't stack jobs.
this will only work with the new openqa package builds I just
did, but won't break anything with older ones. With a new enough
openQA package, it'll prevent the web UI from showing download
links for ISOs and HDD files.
I tweaked the playbook to not patch the templates for non-infra
deployments, but then forgot to make test loading work using
non-patched templates for non-infra...
necessary updates for openqa roles have gone stable for now, so
disable updates-testing usage (keep the plays around commented,
though, since we'll likely need them again in future). Also, a
bit more attempted support for non-infra use: make the monkey
patching of the repo URLs in the test templates only happen if
deployment_type is defined, actually respect the openqa_consumer
var (don't enable the job scheduling consumer unless it's truey)
and only enable any wiki reporting consumer if deployment_type
is defined.
rwmjones says the guestfs / rpm bug has been fixed (a new base
fedora-23 image has been uploaded which should avoid it, anyway)
so let's try turning disk image generation back on and see how
it flies.
I'm still kinda trying to make the openqa roles usable outside
of infra, so now I have a minute, let's do this: it makes the
static UID/GID for geekotest optional and configurable, instead
of hard coding it. For infra we set the value to 601, as we
are already using, in the openqa and openqa-stg group config.
this is a database value and there's no openQA API function to
set it, so we have to do it directly in the database...this
*should* work. I think. I should add equivalent functionality
for sqlite use as well, really...
there seems to be a bug in python2-guestfs which causes the
disk image with an updates.img file to be broken (the updates
image is only 4096 bytes long and incomplete). When built with
Python 3 the image seems to be fine. So, run the script with
python3 (as its hashbang says) and ensure necessary packages
are installed.
with openQA jobs being scheduled and wiki results reported
(well, when we get that working again) by fedmsg-hub consumers,
we have to let the fedmsg user read these files.
