... it is down on staging. So the whole role/messaging/base is broken,
and we can not depend on it:
TASK [rabbit/user : Create the user in RabbitMQ] ***********************
Tuesday 11 August 2020 05:37:34 +0000 (0:00:00.121) 0:05:22.244 *
Tuesday 11 August 2020 05:37:34 +0000 (0:00:00.121) 0:05:22.243 *
[WARNING]: Unhandled error in Python interpreter discovery for host
rabbitmq01.stg.iad2.fedoraproject.org: Failed to connect to the host via
ssh: ssh: connect to host rabbitmq01.stg.iad2.fedoraproject.org port 22:
Connection timed out
fatal: [copr-be-dev.aws.fedoraproject.org]: UNREACHABLE! => {"changed":
false, "msg": "Data could not be sent to remote host
\"rabbitmq01.stg.iad2.fedoraproject.org\". Make sure this host can be
reached over ssh: ssh: connect to host rab
bitmq01.stg.iad2.fedoraproject.org port 22: Connection timed out\r\n",
"unreachable": true}
So there's now a new toggle named "copr_messaging", turned on only for
production copr instance.
This still had all the phx2 local servers, adjust them to the new iad2
ips and see if that helps the wiki stop thowing so many 503's.
Signed-off-by: Kevin Fenzi <kevin@scrye.com>
For the same reason as the last commit. We don't have staging back yet
and so we can't get certs for them.
Signed-off-by: Kevin Fenzi <kevin@scrye.com>
We don't own fpaste.org, in the past the owners just pointed it to our
proxies and we got a cert for it. However, it's still pointing to the
old phx2 proxies right now. So, drop it until it's fixed to point to the
new ones, or given to us as a domain.
Signed-off-by: Kevin Fenzi <kevin@scrye.com>
Doing this makes letsencrypt try and get a cert for it, but it's not
setup to do that correctly. Wait until stg is back.
Signed-off-by: Kevin Fenzi <kevin@scrye.com>
I think the lack of ! on the osbs rule meant that nothing ever went to
the cdn. This increases load on the real registry a lot.
Also, we are using varnish here, but lets try and just go via haproxy.
varnish might be having problems keeping all the 404s in memory/cache.
The cdn thing should help that, but since we have cloudfront I don't
think we also need to use varnish here.
Signed-off-by: Kevin Fenzi <kevin@scrye.com>
One of the 2 patches we were carrying for kojira was merged upstream in
1.22. The other wasn't yet, so we need to keep it around for now.
But we need to merge in the other 1.22 changes or auth fails.
Hopefully this rebases to 1.22
Signed-off-by: Kevin Fenzi <kevin@scrye.com>
Upstream switched from python-requests-kerberos to
python-requests-gssapi and dropped some options.
Unfortunately, these cause kojid / kojira to not even start if they are
present, so we have to remove them to make things happy.
Signed-off-by: Kevin Fenzi <kevin@scrye.com>
Seems if you use a %20 for a space in the Redirect url it will just pass
that on and not urldecode it. So, try and just \ the space and see if
that works better.
Signed-off-by: Kevin Fenzi <kevin@scrye.com>
db-koji01 has been running with this since before the mass rebuild, and
it seems to make it have a higher load, but process faster and without
stalling when doing backups or when long/bad koji-gc queries for old
versions of texlive hit it.
Signed-off-by: Kevin Fenzi <kevin@scrye.com>
This causes:
Aug 05 23:04:40 bvmhost-x86-06.iad2.fedoraproject.org multipathd[3266]:
uevent trigger error
Aug 05 23:04:40 bvmhost-x86-06.iad2.fedoraproject.org multipathd[3266]:
parse error in feature string "no_path_retry queue"
So, just drop it.
Signed-off-by: Kevin Fenzi <kevin@scrye.com>
The s390x boxes have been really slow of late and have i/o issues.
Lets make sure that we only use x86 and ppc64le hosts for this so to
avoid moving large srpms around.
Signed-off-by: Kevin Fenzi <kevin@scrye.com>
We may need to revert this if it causes issues, but try and enable
bootstrap mode again. Newer koji handles GC on these repos now and
it would help us out with a number of issues we are going to hit soon.
Signed-off-by: Kevin Fenzi <kevin@scrye.com>
