Commit Graph

460 Commits

Author SHA1 Message Date
Kevin Fenzi
9a22870dd7 candidate-registry: set network to 10.16 instead of the old iad2 10.3
This apache config is needed in order to allow auth to upload to the
candidate registry. Without it, skopeo just gives a perm denied.
Anytime the datacenter networks change this will also need updated.

Signed-off-by: Kevin Fenzi <kevin@scrye.com>
2025-07-29 15:28:51 -07:00
Kevin Fenzi
f32e35dc41 koji: restore some web endpoints now, uncomment if load becomes too bad
Signed-off-by: Kevin Fenzi <kevin@scrye.com>
2025-07-11 11:16:09 -07:00
Nils Philippsen
6c85fda0c9 Mass remove/replace iad2 -> rdu3, 10.3. -> 10.16.
Signed-off-by: Nils Philippsen <nils@redhat.com>
2025-07-03 20:05:02 +02:00
Kevin Fenzi
ac0abd5853 proxies: switch openqa stg to use rdu3 and add a tag for easily setting it up
Signed-off-by: Kevin Fenzi <kevin@scrye.com>
2025-06-30 17:40:44 -07:00
Kevin Fenzi
fc396674c9 proxies: ipa web ui, missed a iad2
Signed-off-by: Kevin Fenzi <kevin@scrye.com>
2025-06-30 10:15:19 -07:00
Kevin Fenzi
4edce82deb proxies: send ipa web ui to rdu3
Signed-off-by: Kevin Fenzi <kevin@scrye.com>
2025-06-30 10:11:28 -07:00
Kevin Fenzi
7b0eeff7e6 proxies / kojipkgs / nagios: enable in rdu3
Signed-off-by: Kevin Fenzi <kevin@scrye.com>
2025-06-26 15:39:59 -07:00
Kevin Fenzi
126e66c736 proxies: also enable proxy for src.stg/src in rdu3
Signed-off-by: Kevin Fenzi <kevin@scrye.com>
2025-06-26 14:36:15 -07:00
Kevin Fenzi
e670096cd4 proxies / reverspassproxy: setup rdu3 non openshift to work (for riscv koji)
Signed-off-by: Kevin Fenzi <kevin@scrye.com>
2025-06-26 13:30:35 -07:00
Kevin Fenzi
8c64f2ee8a koschei: also drop bots on koschei
Signed-off-by: Kevin Fenzi <kevin@scrye.com>
2025-06-18 13:53:35 -07:00
Kevin Fenzi
4ccfef4a5b bodhi: move bot section to the right place
Signed-off-by: Kevin Fenzi <kevin@scrye.com>
2025-06-18 13:24:43 -07:00
Kevin Fenzi
6880ca537b proxies: bodhi: try and block all the bots going to bodhi
Signed-off-by: Kevin Fenzi <kevin@scrye.com>
2025-06-18 12:45:14 -07:00
Kevin Fenzi
a6c096617b proxies: openshift rdu3
So, we renamed the cluster with the ocp-rdu3 name, so drop all this
special handling. All the proxies should be able to reach it by that
name and via the vpn endpoints it has.

Signed-off-by: Kevin Fenzi <kevin@scrye.com>
2025-06-16 11:10:08 -07:00
Kevin Fenzi
880e83dd43 proxies / rdu3: fix ca filename
Signed-off-by: Kevin Fenzi <kevin@scrye.com>
2025-06-14 14:51:21 -07:00
Kevin Fenzi
e7e6db8d8d proxies: rename ocp4-rdu3 to ocp4_rdu3 to hopefully get jinja2 to leave it alone
Signed-off-by: Kevin Fenzi <kevin@scrye.com>
2025-06-14 14:30:00 -07:00
Kevin Fenzi
e1132667d3 proxies: add a ocp-rdu3 site/endpoint to get to the new rdu3 prod openshift
We can't use easily the existing hostname/site, as that goes to the
current iad2 cluster, so setup a -rdu3 version for now.
After we switch we can drop this and repoint the main one to the new
cluster.

Hopefully this all works and does the right thing.

Signed-off-by: Kevin Fenzi <kevin@scrye.com>
2025-06-14 12:35:20 -07:00
Kevin Fenzi
b1c844e6d3 proxies / koji: block buildroot and rpminfo for now as scrapers are beating things up on them
Signed-off-by: Kevin Fenzi <kevin@scrye.com>
2025-06-02 07:42:37 -07:00
Kevin Fenzi
d42014c740 httpd: set a few conf files as optional for initial installs
Due to ordering, these config files don't get copied over before the
first time httpd gets restarted. Setting them as optional should allow
initial runs to work better.

Signed-off-by: Kevin Fenzi <kevin@scrye.com>
2025-05-28 19:49:32 -07:00
Kevin Fenzi
91e9a5627d httpd / botblocking: fix syntax on bot rewrite
These have to be in "s in order to do a string comparison, since
they were not, they were never matching anything. ;(

Fix them all up, and also block a few more repos on pagure that are
getting heavily crawled.

Signed-off-by: Kevin Fenzi <kevin@scrye.com>
2025-05-13 11:39:25 -07:00
Kevin Fenzi
3a9f0cbfbb Revert "proxies: re-enable koji endpoints"
This reverts commit 5a72566502.

The bots are back, close these endpoints. :(
2025-04-26 12:24:59 -07:00
Kevin Fenzi
5a72566502 proxies: re-enable koji endpoints
Signed-off-by: Kevin Fenzi <kevin@scrye.com>
2025-04-21 14:17:31 -07:00
Kevin Fenzi
002afafc5f proxy01/10 / koji: block some kojiweb queries that bots are hitting
Bots are htiting these at a massive level.
Since these cause koji to do db queries it basically swamps it and it
stops processing at all.

Hopefully with enough of these 403's the bots will go away.

Signed-off-by: Kevin Fenzi <kevin@scrye.com>
2025-04-16 18:40:24 -07:00
Kevin Fenzi
af2cadbda9 wiki: 404 a list of bots
Bots are hitting the wiki pretty hard and we don't particularly
care about indexing it anymore, as most real docs should have moved
to docs.fedoraproject.org. Also, many of these bots ignore robots.txt
or do other things we don't want.

Signed-off-by: Kevin Fenzi <kevin@scrye.com>
2025-03-19 20:53:05 +00:00
Michal Konecny
2ec055db6f Use first uppercase letter for all handlers
This will unify all the handlers to use first uppercase letter for
ansible-lint to stop complaining.

I went through all `notify:` occurrences and fixed them by running
```
set TEXT "text_to_replace"; set REPLACEMENT "replacement_text"; git grep
-rlz "$TEXT" . | xargs -0 sed -i "s/$TEXT/$REPLACEMENT/g"
```

Then I went through all the changes and removed the ones that wasn't
expected to be changed.

Fixes https://pagure.io/fedora-infrastructure/issue/12391

Signed-off-by: Michal Konecny <mkonecny@redhat.com>
2025-02-10 20:31:49 +00:00
Mattia Verga
2b4de7b85d another try on registry redirection
Signed-off-by: Mattia Verga <mattia.verga@tiscali.it>
2025-01-20 23:18:55 +00:00
Kevin Fenzi
96a5107afa registry: just test in staging for now
Signed-off-by: Kevin Fenzi <kevin@scrye.com>
2025-01-17 10:27:44 -08:00
Kevin Fenzi
719417e668 registry: handle stg redirect too
Signed-off-by: Kevin Fenzi <kevin@scrye.com>
2025-01-17 10:18:24 -08:00
Mattia Verga
926b12b2a5 Move registry redirects rules to reverseproxy 2025-01-17 17:52:24 +00:00
Ryan Lerch
47c68f478d ansiblelint fixes - fqcn[action-core] - template to ansible.builtin.template
Replaces references to template: with ansible.builtin.template

Signed-off-by: Ryan Lerch <rlerch@redhat.com>
2025-01-15 11:30:29 +10:00
Ryan Lerch
25391e95b7 ansiblelint fixes - fqcn[action-core] - package to ansible.builtin.package
Replaces many references to  package: with ansible.builtin.package

Signed-off-by: Ryan Lerch <rlerch@redhat.com>
2025-01-15 11:28:00 +10:00
Ryan Lerch
6a3816dfdc ansiblelint fixes-- fqcn[action-core] - copy to ansible.builtin.copy
Replaces many references to 'copy' with ansible.builtin.copy

Signed-off-by: Ryan Lerch <rlerch@redhat.com>
2025-01-15 10:43:31 +10:00
Ryan Lerch
62952df107 ansiblelint fixes-- fqcn[action-core] - file to ansible.builtin.file
Replaces many references to  file: with ansible.builtin.file

Signed-off-by: Ryan Lerch <rlerch@redhat.com>
2025-01-15 10:41:52 +10:00
Ryan Lerch
691adee6ee Fix name[casing] ansible-lint issues
fix 1900 failures of the following case issue:

`name[casing]: All names should start with an uppercase letter.`

Signed-off-by: Ryan Lerch <rlerch@redhat.com>
2025-01-14 20:20:07 +10:00
Kevin Fenzi
f085833c90 proxies / redirectmatch: revert trailing slash on redirectmatch, breaks codecs
Signed-off-by: Kevin Fenzi <kevin@scrye.com>
2025-01-06 19:17:42 -08:00
Kevin Fenzi
acded5f1be proxies / redirects: add trailing / to prevent unintended redirects
Without this you can get the redirect to redirect to another host.
Not a super big problem, but definitely not intended.
Hopefully just adding a tailing / will not break anything.

Signed-off-by: Kevin Fenzi <kevin@scrye.com>
2025-01-06 09:57:46 -08:00
Ryan Lerch
89f6f1fc32 Fix majority of remaining yamllint warnings and errors
Signed-off-by: Ryan Lerch <rlerch@redhat.com>
2024-11-28 17:31:45 +10:00
Kevin Fenzi
c04dd50fbf certificates: move to new 2024 wildcard fedoraproject.org cert
The old one expires in about 57 days, but might as well just renew it
early and avoid problems later.

Signed-off-by: Kevin Fenzi <kevin@scrye.com>
2024-10-03 16:50:01 -07:00
Mikolaj Izdebski
bb4d2c2456 Koschei: Add custom robots.txt
Related: fedora-infrastructure#12180
2024-09-13 22:15:14 +02:00
Kevin Fenzi
4e52328281 proxies / id: just disable this mess for now and revisit later
Signed-off-by: Kevin Fenzi <kevin@scrye.com>
2024-08-20 14:00:40 -07:00
Nick Bebout
7d0bc1dc62 Add my work IP to allowlist for ipa 2024-08-20 14:32:39 -05:00
Kevin Fenzi
51b344c104 proxies / id: clean up syntax and also allow all internal ips in iad2
Signed-off-by: Kevin Fenzi <kevin@scrye.com>
2024-08-20 12:14:29 -07:00
Kevin Fenzi
9457e0e58e proxies / ipa / api: close access to ui
Users don't need to use this and so restrict it to admins by ip for now.
Down the road we should be able to do this much better once we can set a
policy for access here.

Signed-off-by: Kevin Fenzi <kevin@scrye.com>
2024-08-20 11:44:15 -07:00
Kevin Fenzi
f40fef9316 resultsdb: crawlers go away
Signed-off-by: Kevin Fenzi <kevin@scrye.com>
2024-07-19 10:45:02 -07:00
Stephen Smoogen
7e426dbf37 Add more spiders which do not seem to honour robots.txt
I went through the last couple of logs afer the first round of 'turn
off the spiders' went out. I looked at the areas which the /robots.txt
disregard and then looked for the bots which ignored it and still
looked up stuff in 'accounts'. This may cut down CPU spikes as these
are looking at dynamic data which can 'blow' things up.

It might be good to add similar tooling to pagure and src since they
seem to be hit a lot in the logs also.

Signed-off-by: Stephen Smoogen <ssmoogen@redhat.com>
2024-07-09 09:18:56 +00:00
Kevin Fenzi
5a371a1049 mirrormanager: redirect mirrors.fedoraproject.org to new mirrormanager
The mirrormanager application moved over to
mirrormanager.fedoraproject.org from
admin.fedoraproject.org/mirrormanager. So we need to change this
redirect to point things to the new place.

Signed-off-by: Kevin Fenzi <kevin@scrye.com>
2024-07-03 14:22:14 -07:00
Kevin Fenzi
6c2f16eb9a src: adjust robots.txt and fix formatting
Signed-off-by: Kevin Fenzi <kevin@scrye.com>
2024-07-03 10:00:01 -07:00
Kevin Fenzi
2140cf2080 mailman: adjust the right robots.txt and fix formatting
Signed-off-by: Kevin Fenzi <kevin@scrye.com>
2024-07-03 09:58:18 -07:00
Kevin Fenzi
59829605c5 src.fedoraproject.org: adjust robots.txt
Signed-off-by: Kevin Fenzi <kevin@scrye.com>
2024-06-28 12:15:16 -07:00
Frank Ch. Eigler
d4a36c7c74 debuginfod: add /metadata* reverse proxy URL prefix
Upcoming versions of elfutils/debuginfod support a new webapi URL
prefix.
2024-06-07 14:06:58 -04:00
Aurélien Bompard
f8bf7b83ad Mirrormanager: adjust the filename
Signed-off-by: Aurélien Bompard <aurelien@bompard.org>
2024-05-29 10:04:24 +02:00