The proxies in the new fedora-isolated vlan are in rdu3 and have rdu3
domain in their hostname, but they aren't strictly in the rdu3
datacenter for purposes of access. They do not have acls to directly
talk to backend applications from that vlan.
Signed-off-by: Kevin Fenzi <kevin@scrye.com>
This avoids the tcp timeout problem totally from what I can tell.
Just switch it for now as we continue to work on the underlying problem.
This does mean that we don't use varnish, but apache is able to
keep up ok so far.
Signed-off-by: Kevin Fenzi <kevin@scrye.com>
openqa uses apache load balancer now, and doesn't use haproxy at all.
Clean up some things that current haproxy warns about on start.
Signed-off-by: Kevin Fenzi <kevin@scrye.com>
After some troubleshooting I was finally able to fix the OpenID
authentication on staging. These are the changes I ended up deploying to fix
the remaining issues.
Signed-off-by: Michal Konecny <mkonecny@redhat.com>
This seems to be a similar case to the kojipkgs one, where we see from
time to time timeouts from proxies to pkgs01.
If it's a health check, haproxy will mark the backend down.
If it's a user request they will get a timeout and a 503 back.
This will help mitigate the second problem and retry those.
Signed-off-by: Kevin Fenzi <kevin@scrye.com>
We are having problems with connections sometimes hanging from proxies
to kojipkgs. Lets try and mitigate that at the haproxy level and
hopefully improve things while we try and figure out what the underlying
cause is.
This should retry connections that failed for any 'retryable' output
(including timeout) and also it should try a _different_ backend than
the one that returned the error. This will not eliminate errors, but
should reduce them.
Signed-off-by: Kevin Fenzi <kevin@scrye.com>
To be redirected to openid server during the authentication let's set a
cookie for it and match against that.
This was tested and it's working, but ipsilon is doing something with
the requests and the cookie is gone after redirect.
Signed-off-by: Michal Konecny <mkonecny@redhat.com>
When checking if the server has openid capabilities we are checking for
openid_identifier, let's redirect that to openid backend as well.
Signed-off-by: Michal Konecny <mkonecny@redhat.com>
The paths are too similar and /openidc ended up being routed to wrong
ipsilon server, let's add specific rule for OIDC as well.
Signed-off-by: Michal Konecny <mkonecny@redhat.com>
HAProxy had incorrect IPA certificate for staging. I'm not sure how that
even worked, but the issue was revealed when the IAD2 machines were
removed from cluster.
Prior to 38d138e this condition existed with 'iad2' instead of
'rdu3'. @abompard took it out entirely, but that was wrong, it
makes the external proxies include this block. We need to put the
condition back with the correct data center name.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
The IPA cert doesn't change when we move datacenters, because we
just replicate across. So it shouldn't have the datacenter in the
name. This should fix haproxy deployment (it was broken because
we didn't have an 'rdu3' file).
Signed-off-by: Adam Williamson <awilliam@redhat.com>
This will break things in not rdu3, but we are moving staging tomorrow
anyhow. So, just running this on rdu3 staging for now.
Signed-off-by: Kevin Fenzi <kevin@scrye.com>
Added host vars for all the control plane vm's and bootstrap node.
Set latest version for downloading and setting things up.
Setup haproxy in rdu3 prod to load balance the ocp api and internal api.
Signed-off-by: Kevin Fenzi <kevin@scrye.com>
This will unify all the handlers to use first uppercase letter for
ansible-lint to stop complaining.
I went through all `notify:` occurrences and fixed them by running
```
set TEXT "text_to_replace"; set REPLACEMENT "replacement_text"; git grep
-rlz "$TEXT" . | xargs -0 sed -i "s/$TEXT/$REPLACEMENT/g"
```
Then I went through all the changes and removed the ones that wasn't
expected to be changed.
Fixes https://pagure.io/fedora-infrastructure/issue/12391
Signed-off-by: Michal Konecny <mkonecny@redhat.com>
fix 1900 failures of the following case issue:
`name[casing]: All names should start with an uppercase letter.`
Signed-off-by: Ryan Lerch <rlerch@redhat.com>
