Commit Graph

247 Commits

Author SHA1 Message Date
Kevin Fenzi
91e9a5627d httpd / botblocking: fix syntax on bot rewrite
These have to be in "s in order to do a string comparison, since
they were not, they were never matching anything. ;(

Fix them all up, and also block a few more repos on pagure that are
getting heavily crawled.

Signed-off-by: Kevin Fenzi <kevin@scrye.com>
2025-05-13 11:39:25 -07:00
Kevin Fenzi
3a9f0cbfbb Revert "proxies: re-enable koji endpoints"
This reverts commit 5a72566502.

The bots are back, close these endpoints. :(
2025-04-26 12:24:59 -07:00
Kevin Fenzi
5a72566502 proxies: re-enable koji endpoints
Signed-off-by: Kevin Fenzi <kevin@scrye.com>
2025-04-21 14:17:31 -07:00
Kevin Fenzi
002afafc5f proxy01/10 / koji: block some kojiweb queries that bots are hitting
Bots are htiting these at a massive level.
Since these cause koji to do db queries it basically swamps it and it
stops processing at all.

Hopefully with enough of these 403's the bots will go away.

Signed-off-by: Kevin Fenzi <kevin@scrye.com>
2025-04-16 18:40:24 -07:00
Kevin Fenzi
af2cadbda9 wiki: 404 a list of bots
Bots are hitting the wiki pretty hard and we don't particularly
care about indexing it anymore, as most real docs should have moved
to docs.fedoraproject.org. Also, many of these bots ignore robots.txt
or do other things we don't want.

Signed-off-by: Kevin Fenzi <kevin@scrye.com>
2025-03-19 20:53:05 +00:00
Michal Konecny
2ec055db6f Use first uppercase letter for all handlers
This will unify all the handlers to use first uppercase letter for
ansible-lint to stop complaining.

I went through all `notify:` occurrences and fixed them by running
```
set TEXT "text_to_replace"; set REPLACEMENT "replacement_text"; git grep
-rlz "$TEXT" . | xargs -0 sed -i "s/$TEXT/$REPLACEMENT/g"
```

Then I went through all the changes and removed the ones that wasn't
expected to be changed.

Fixes https://pagure.io/fedora-infrastructure/issue/12391

Signed-off-by: Michal Konecny <mkonecny@redhat.com>
2025-02-10 20:31:49 +00:00
Mattia Verga
2b4de7b85d another try on registry redirection
Signed-off-by: Mattia Verga <mattia.verga@tiscali.it>
2025-01-20 23:18:55 +00:00
Kevin Fenzi
96a5107afa registry: just test in staging for now
Signed-off-by: Kevin Fenzi <kevin@scrye.com>
2025-01-17 10:27:44 -08:00
Kevin Fenzi
719417e668 registry: handle stg redirect too
Signed-off-by: Kevin Fenzi <kevin@scrye.com>
2025-01-17 10:18:24 -08:00
Mattia Verga
926b12b2a5 Move registry redirects rules to reverseproxy 2025-01-17 17:52:24 +00:00
Ryan Lerch
47c68f478d ansiblelint fixes - fqcn[action-core] - template to ansible.builtin.template
Replaces references to template: with ansible.builtin.template

Signed-off-by: Ryan Lerch <rlerch@redhat.com>
2025-01-15 11:30:29 +10:00
Ryan Lerch
89f6f1fc32 Fix majority of remaining yamllint warnings and errors
Signed-off-by: Ryan Lerch <rlerch@redhat.com>
2024-11-28 17:31:45 +10:00
Kevin Fenzi
4e52328281 proxies / id: just disable this mess for now and revisit later
Signed-off-by: Kevin Fenzi <kevin@scrye.com>
2024-08-20 14:00:40 -07:00
Nick Bebout
7d0bc1dc62 Add my work IP to allowlist for ipa 2024-08-20 14:32:39 -05:00
Kevin Fenzi
51b344c104 proxies / id: clean up syntax and also allow all internal ips in iad2
Signed-off-by: Kevin Fenzi <kevin@scrye.com>
2024-08-20 12:14:29 -07:00
Kevin Fenzi
9457e0e58e proxies / ipa / api: close access to ui
Users don't need to use this and so restrict it to admins by ip for now.
Down the road we should be able to do this much better once we can set a
policy for access here.

Signed-off-by: Kevin Fenzi <kevin@scrye.com>
2024-08-20 11:44:15 -07:00
Kevin Fenzi
5a371a1049 mirrormanager: redirect mirrors.fedoraproject.org to new mirrormanager
The mirrormanager application moved over to
mirrormanager.fedoraproject.org from
admin.fedoraproject.org/mirrormanager. So we need to change this
redirect to point things to the new place.

Signed-off-by: Kevin Fenzi <kevin@scrye.com>
2024-07-03 14:22:14 -07:00
Frank Ch. Eigler
d4a36c7c74 debuginfod: add /metadata* reverse proxy URL prefix
Upcoming versions of elfutils/debuginfod support a new webapi URL
prefix.
2024-06-07 14:06:58 -04:00
Aurélien Bompard
f8bf7b83ad Mirrormanager: adjust the filename
Signed-off-by: Aurélien Bompard <aurelien@bompard.org>
2024-05-29 10:04:24 +02:00
Kevin Fenzi
c84b99223c osbs: raise a glass for it's service
This removes osbs and allmost all it's associated playbooks and files.

It served long and well, but we no longer need it.
flatpaks are building with a koji-flatpak plugin.
base/minimal/toolbox containers are building with kiwi.
We aren't building any other containers right now, and we did they could
be added to kiwi.

This is the end of an era... I look with nostolga on
ansible-ansible-openshift-ansible (a role to setup ansible on a control
host and run it from our ansible).

Good bye osbs!

Signed-off-by: Kevin Fenzi <kevin@scrye.com>
2024-03-28 12:52:07 -07:00
Leo Puvilland
e05a7b6bb1 flatpak-cache: fix httpd not using the squid proxy correctly...
Signed-off-by: Leo Puvilland <leo@craftcat.dev>
2024-03-06 14:40:56 -08:00
Leo Puvilland
6a37e3b87e flatpak-cache: add ProxyRemote to apache configuration
Signed-off-by: Leo Puvilland <leo@craftcat.dev>
2024-03-04 14:19:44 -08:00
Nick Bebout
d1c563acc2 Revert "Change the httpd proxy stuff to point to ipa02 also"
This reverts commit 65cf67f584.
2024-01-25 11:23:03 -06:00
Nick Bebout
65cf67f584 Change the httpd proxy stuff to point to ipa02 also 2024-01-25 09:06:44 -06:00
Kevin Fenzi
5d24117ce8 proxies / ipa / ui: missing trailing /
Signed-off-by: Kevin Fenzi <kevin@scrye.com>
2024-01-24 15:27:01 -08:00
Kevin Fenzi
62279be3e5 proxies / ipa / ui: fix another typo
Signed-off-by: Kevin Fenzi <kevin@scrye.com>
2024-01-24 15:21:59 -08:00
Kevin Fenzi
ec09a8d302 proxies / ipa / ui: fix a typo
Signed-off-by: Kevin Fenzi <kevin@scrye.com>
2024-01-24 15:20:59 -08:00
Kevin Fenzi
3996374ea0 proxies / ipa / ui: try and edit the referrer for new ipa
New ipa checks the referrer to avoid CSRF issues.
We need to have the proxy edit requests for the right internal hostname
for it to be able to work.

Signed-off-by: Kevin Fenzi <kevin@scrye.com>
2024-01-24 15:19:14 -08:00
Kevin Fenzi
6abea25563 proxies: registry: let ppc64le builders directly access registry
f39 adds flatpaks for ppc64le, so we need to allow ppc64le builders to
access the registry directly so they can install flatpaks in the ostree
install images. Without this they try and get them from the cdn and the
builder firewall blocks them and it times out and the image fails to
compose.

Signed-off-by: Kevin Fenzi <kevin@scrye.com>
2023-10-31 13:46:57 -07:00
Kevin Fenzi
a381bf52e0 proxies / registry: don't use cdn for aarch64 builders either
Right now we only exclude the builders on 10.3.169 from using the
registry cdn (ie, the x86_86 builders), but we also make aarch64
containers/images and we should exclude it too ( 10.3.170.x ).
This might fix a weird compose failure we have been getting on
aarch64 ostree installer images.

Signed-off-by: Kevin Fenzi <kevin@scrye.com>
2023-04-17 11:46:10 -07:00
Kevin Fenzi
531eee95d3 proxies / reverseproxy: try and fix template to be correct.
Signed-off-by: Kevin Fenzi <kevin@scrye.com>
2022-12-03 20:39:13 -08:00
Kevin Fenzi
3c5ae163f9 proxy: simplify reverseproxy template
Signed-off-by: Kevin Fenzi <kevin@scrye.com>
2022-12-03 20:29:36 -08:00
Kevin Fenzi
b1b0923af0 openshift 3.11: raise a glass!
Our openshift 3.11 cluster(s) served us long and well.

Now we have everything finally moved to the openshift 4 clusters (fas2
was the last holdout). We can finally retire this. :)

🎉🥂

Signed-off-by: Kevin Fenzi <kevin@scrye.com>
2022-11-28 13:08:01 -08:00
Kevin Fenzi
91ccdea73f proxies: Reach ocp4 cluster in prod over vpn instead of sending 421
I think I handled all the special cases here already.
We want to switch non iad2 proxies to reach the oco4 cluster over it's
vpn now that it has one. This should allow us to still keep ipv6
available for applications and not have to change dns for moving from
ocp3 cluster anymore. Will roll this out slowly to one proxy then
another, then the rest if it all looks ok.

Signed-off-by: Kevin Fenzi <kevin@scrye.com>
2022-06-13 10:08:48 -07:00
Kevin Fenzi
80ef5e47df proxies / reverseproxypass: try and only 421 ocp4 non iad2 proxies
Signed-off-by: Kevin Fenzi <kevin@scrye.com>
2022-06-05 13:44:02 -07:00
Kevin Fenzi
42bc4ea41a proxies / reverseproxy: try just always setting balancer_members and dropping the set facts
Signed-off-by: Kevin Fenzi <kevin@scrye.com>
2022-06-05 11:47:17 -07:00
Kevin Fenzi
7e90378b30 proxies / reverseproxy: Try and clean up logic and whitespace.
Signed-off-by: Kevin Fenzi <kevin@scrye.com>
2022-06-05 11:24:24 -07:00
Kevin Fenzi
0e1d9f6ea7 proxies / reverseproxy: try some different logic
Signed-off-by: Kevin Fenzi <kevin@scrye.com>
2022-06-05 09:30:58 -07:00
Kevin Fenzi
dc6b7697c1 proxies / reverseproxy: unset balancer_members after each template so it can get set right for the next one
Signed-off-by: Kevin Fenzi <kevin@scrye.com>
2022-06-04 21:05:37 -07:00
Kevin Fenzi
e2deffab4e proxies / reverseproxy: consolidate non openshift-iad2 case
Signed-off-by: Kevin Fenzi <kevin@scrye.com>
2022-06-04 14:51:33 -07:00
Kevin Fenzi
03fe998142 proxies / reverseproxy: also special case the old resultsdbapi
Signed-off-by: Kevin Fenzi <kevin@scrye.com>
2022-06-04 14:25:32 -07:00
Kevin Fenzi
77bdc05883 proxies / reverseproxy: work around taskotron with its own config
Signed-off-by: Kevin Fenzi <kevin@scrye.com>
2022-06-04 14:04:07 -07:00
Kevin Fenzi
6d99b43608 proxies / reverspassproxy: fix loop
Signed-off-by: Kevin Fenzi <kevin@scrye.com>
2022-06-03 20:51:46 -07:00
Kevin Fenzi
6a0727c9e6 proxies / openqa: rework conf file to avoid issue with openqa and its websockets
Signed-off-by: Kevin Fenzi <kevin@scrye.com>
2022-06-03 20:15:36 -07:00
Kevin Fenzi
7455b3fdb1 proxies / reverseproxy: fix missing )
Signed-off-by: Kevin Fenzi <kevin@scrye.com>
2022-06-03 14:22:13 -07:00
Kevin Fenzi
52a8936914 proxies: rework reversepassproxy and mark all ocp4 / ocp3 apps
The ocp3 cluster is reachable/available via the vpn, so any proxy can
reach it.

The ocp4 cluster is (at least for now) only reachable/available from the
iad2 proxies (proxy01/proxy10).

There's a firefox bug that causes it to reuse h2 connections, and in
some cases try and request something of a non iad2 proxy that it can't
reach. To work around this in those cases we need to send a 421 back to
the client so it doesn't do that.

This moves that logic into the template so all ocp4: true hosts do this
by default. Also, we default the balancer nodes so we only have to
change them in one place if we remove/add a compute node.

Finally, we mark all the ocp3 apps with 'ocp4: false' so we know what
they are and can move them more easily.

Signed-off-by: Kevin Fenzi <kevin@scrye.com>
2022-06-03 13:14:28 -07:00
Kevin Fenzi
0302040f3c proxies / reverseproxy / blockerbugs: have proxies return 421 for non iad2 access.
This is the fun firefox h2 connection reuse bug. blockerbugs is only in
iad2, so if firefox tries to reuse a connection to another proxy for it,
just send it a 421 so it knows thats bad on it.

Signed-off-by: Kevin Fenzi <kevin@scrye.com>
2022-05-25 17:29:37 -07:00
Kevin Fenzi
6d1e07d599 proxies / reverseproxy / openqa: do this in a more sustainable way
Signed-off-by: Kevin Fenzi <kevin@scrye.com>
2022-05-11 15:45:14 -07:00
Kevin Fenzi
8d38f818e7 proxies / reverseproxy / openqa: drop unneeded conditional that is causing a syntax error
Signed-off-by: Kevin Fenzi <kevin@scrye.com>
2022-05-11 15:31:26 -07:00
Kevin Fenzi
63adb316a7 proxies / reverseproxy / openqa: make openqa on non iad2 proxies send a 421
Due to http/2 connection reuse bugs, sometimes firefox will decide to
'reuse' a connection to fedoraproject.org for openqa.fedoraproject.org
(since they both have the same tls cert), but openqa is only available
from the 2 iad2 proxies, not all of them. This results in a 503 timeout
and it just not loading. This should make those reused connections get a
421 from proxies and reconnect to the proper ips. (we hope)

Signed-off-by: Kevin Fenzi <kevin@scrye.com>
2022-05-11 13:10:43 -07:00