This sort of seemed to work in staging (although there is some issue
with staging koji).
So, lets try it in production. It will make all the boostrap repo regens
a little slower, but it should get epel8 back working hopefully for now.
Signed-off-by: Kevin Fenzi <kevin@scrye.com>
lets try setting the opt to download filelists for the bootstrap chroot.
This is needed for epel8 because the deps for dnf need to resolve
/usr/libexec/platform-python and cannot due to f40 dnf not downloading
filelists by default anymore.
This will affect all bootstrap chroot creation, but it seems the
simpilest solution without changes in koji.
Signed-off-by: Kevin Fenzi <kevin@scrye.com>
This removes osbs and allmost all it's associated playbooks and files.
It served long and well, but we no longer need it.
flatpaks are building with a koji-flatpak plugin.
base/minimal/toolbox containers are building with kiwi.
We aren't building any other containers right now, and we did they could
be added to kiwi.
This is the end of an era... I look with nostolga on
ansible-ansible-openshift-ansible (a role to setup ansible on a control
host and run it from our ansible).
Good bye osbs!
Signed-off-by: Kevin Fenzi <kevin@scrye.com>
We need to still setup a bug tag and target for this (which needs old
mock chroot) and a group, but this enables the plugin and allows us to
download blueprints.
Signed-off-by: Kevin Fenzi <kevin@scrye.com>
We want to move (well, really re-install) all these over on the new lpar
in rdu. This will have much higher stats and be in general faster by
both network and cpu. Hopefully all these will replace all the old
boston ones.
We may need to break these up some more into smaller vm's if the number
isn't able to keep up ok. We can adjust after things are all working.
Signed-off-by: Kevin Fenzi <kevin@scrye.com>
Lets move this vm over to rdu, and set it up as a new varnish cache.
This way we can test 01 doing builds before moving others.
Signed-off-by: Kevin Fenzi <kevin@scrye.com>
Just enable the osbuild plugin on all the hosts, and we will narrow
things down with the channel/policy setup.
Signed-off-by: Kevin Fenzi <kevin@scrye.com>
Right now we just run the cron to update osbuildapi ip on buildhw-x86 in
prod, but we need to run it on the osbuild channel builders in staging
too. Fixes ticket 11575.
Signed-off-by: Kevin Fenzi <kevin@scrye.com>
We tried to override the machine scope, but that didn't work in this
case, so instead lets tell mock to pass a new core limit to
systemd-nspawn and see if that gets us cores.
Signed-off-by: Kevin Fenzi <kevin@scrye.com>
We want to get core dumps from these builders in order to try and track
down livemedia failures in python with SIGILL.
https://bugzilla.redhat.com/show_bug.cgi?id=2247319
We just enable this on those builders for now and we set it so it is the
limit for the systemd-nspawn containers that mock uses.
Signed-off-by: Kevin Fenzi <kevin@scrye.com>
We added local site config to make sure there was a /dev/fuse in the
mock nspawn container in order to get ostree oci image building working
correctly. Now in mock 5.x, /dev/fuse is automatically setup/added, so
we need to drop our local config so it doesn't conflict with the
internal upstream version now.
See: https://github.com/rpm-software-management/mock/issues/1188
Signed-off-by: Kevin Fenzi <kevin@scrye.com>
Right now builders are checking in every 20s, but that puts a lot of
load on the db server. Having them check in every 30s should ease that
some. Might increase it higher as well.
Signed-off-by: Kevin Fenzi <kevin@scrye.com>
Just for staging for now:
- enable 'podman-login' role for the buildvm group, so built Flatpaks
can be pushed to the skopeo registry.
- add the koji-flatpak hub plugin to koji_hub role
- add the koji-flatpak builder plugin to the koji_builder role
and configure it.
Signed-off-by: Owen W. Taylor <otaylor@fishsoup.net>
Currently, I (Stephen Smoogen) do not have the time to work on Fedora
system administration items. However, I get a lot of email and people
see my email address in various places to ping me for working on
things. I feel it would be better to remove myself from those places
and let Fedora Infrastructure add someone else to replace me when it
is possible to do so.
Signed-off-by: Stephen Smoogen <ssmoogen@redhat.com>
identity.api.openshift.com was shut down several hours ago. The plugin now
needs to use sso.redhat.com instead.
This commit adjusts the token URL and the script that pokes holes in the
firewall for selected domains.
We want to keep using dnf4 for koji builds, even after dnf5 is default
in rawhide. Once it's ready to do building as well, we can revert/drop
this. See https://pagure.io/releng/issue/11446
Signed-off-by: Kevin Fenzi <kevin@scrye.com>
Right now all builders ask the hub for jobs every 15s.
Bump that out a bit to see if it lowers the pressure on the db server.
Signed-off-by: Kevin Fenzi <kevin@scrye.com>
Moving all SCM policies previously defined in each builder to
centralized hub configuration. From now on, any SCM policy change just
needs updating the hub config and reloading it. Builders need nor change
nor reload.
Related: https://pagure.io/fedora-infrastructure/issue/9728
Signed-off-by: Tomas Kopecek <tkopecek@redhat.com>
With dnf5 becoming default in rawhide, we need to make sure to override
things to use dnf4 for making buildroots for now until dnf5 is ready to
take that on.
Signed-off-by: Kevin Fenzi <kevin@scrye.com>
Add reload in so we can reload and have kojid finish any jobs and then
restart and also add a 60s restart backoff time.
Signed-off-by: Kevin Fenzi <kevin@scrye.com>
Right now we run a script on all builders once a minute to update the
api/auth ip's for osbuild. This has a number of problems:
* Sometimes osbuild jobs land on s390x builders that have no internet
access and hang or fail.
* Sometimes the update script hangs or takes a long time to run because
the builder is heavily loaded with builds, resulting in locking emails
to sysadmin-main folks.
So, in this commit we:
* make a new koji channel called 'osbuild' with all the buildhw-x86's in
it. They are usually not too overloaded and there are 16 of them so it
should be available all the time.
* Leave the cron job on all builders for now in case, but make them only
update once a day since they won't be getting jobs. If this works out
we can remove it entirely there.
* Make the buildhw-x86s only update every 5min. This opens a larger
window for it being wrong, but it's still pretty small and should
reduce the number of emails for stalled processes we get.
See https://pagure.io/fedora-infrastructure/issue/10982
Signed-off-by: Kevin Fenzi <kevin@scrye.com>
The awk helper responsible for extracting IP addresses from the resolvectl
call could handle only 2 of them.
It turns out that api.openshift.com now has 4 A records, therefore this method
became flakey: It added only 2 addresses to the IP set, so if the osbuild
plugin used one of the 2 ignored addresses, the call failed.
This commit solves it by introducing a different method of parsing the
resolvectl output:
We now use an ugly but working sed command that erases everything from the
line except for the IPv4 address. Therefore, I had to quote the echo before
the new sed command so it can get a proper multiline input. Also, I limited
resolvectl to just use IPv4 because the new script cannot handle IPv6
properly. This doesn't cause any harm because api.openshift.com isn't
actually accessible by IPv6. Sigh...
It seems that ppc64le and aarch64 are both quite slow during the
ELN Image install. The aarch64 is now finishing most of the time,
but the ppc64le build still times out.
Signed-off-by: Stephen Gallagher <sgallagh@redhat.com>
The ostree_installer job needs additional capabilities over the normal
mock systemd-spawn ones. So, add that here and since the
ostree_installer jobs are all runroot jobs we can isolate it to just the
runroot builders.
Signed-off-by: Kevin Fenzi <kevin@scrye.com>
