WarlockFish/fedora-infra_ansible
1
0
Fork 0
mirror of https://pagure.io/fedora-infra/ansible.git synced 2026-03-19 19:46:38 +08:00
Code Issues Packages Projects Releases Wiki Activity

Compare commits

base: WarlockFish:0d56b527a6ce27e428d0538052ad4454e67eb7c8
Branches Tags
WarlockFish:main WarlockFish:letsencrypt WarlockFish:rawhide-greenwave WarlockFish:openqa WarlockFish:secarch WarlockFish:darkserver WarlockFish:openvpn_handler WarlockFish:easyfix WarlockFish:denyhosts
...
compare: WarlockFish:e5d26fea609fce8f48e4e98031754ccd2947b334
Branches Tags
WarlockFish:main WarlockFish:letsencrypt WarlockFish:rawhide-greenwave WarlockFish:openqa WarlockFish:secarch WarlockFish:darkserver WarlockFish:openvpn_handler WarlockFish:easyfix WarlockFish:denyhosts

2 Commits
0d56b527a6 ... e5d26fea60

Author SHA1 Message Date
Kevin Fenzi
e5d26fea60 proxies: allow POST for forge 
Some folks were getting posts failing against forge.
See https://forge.fedoraproject.org/forge/forge/issues/401

So, lets just explicitly allow POSTs through anubis as
this should be fine for normal people and should not be something
that scrapers normally do.

Signed-off-by: Kevin Fenzi <kevin@scrye.com>
 2026-02-12 16:57:26 -08:00
Kevin Fenzi
fefbc356f0 download: block infinite crawler looping 
The /pub/alt/virtio-win directory had in it some 11 year old things.
One was a readme noting that it moved 11 years ago and nothing was
still here. The others were links to .

The scrapers, being as dumb as posts followed all those links over and
over again to the tune of millions per day.
I removed the links, but of course they were still trying, so
lets be a bit more aggressive and just 403 them all.

Signed-off-by: Kevin Fenzi <kevin@scrye.com>
 2026-02-12 16:54:16 -08:00
2 changed files with 8 additions and 0 deletions
Expand all files Collapse all files

3
roles/anubis/templates/policies.yaml.j2
View File

@@ -15,6 +15,9 @@ bots:
- name: bodhi and badges rss feeds - name: bodhi and badges rss feeds
path_regex: /rss/ path_regex: /rss/
action: ALLOW action: ALLOW
- name: allow POSTs
expression: method == "POST"
action: ALLOW
- name: allow ostree - name: allow ostree
path_regex: ^/ostree path_regex: ^/ostree
action: ALLOW action: ALLOW

5
roles/download/templates/httpd/dl.fedoraproject.org.conf
View File

@@ -32,6 +32,11 @@
SSLProtocol {{ ssl_protocols }} SSLProtocol {{ ssl_protocols }}
SSLCipherSuite {{ ssl_ciphers }} SSLCipherSuite {{ ssl_ciphers }}
RewriteEngine On
RewriteCond %{REQUEST_URI} ^/pub/alt/virtio-win/.*$
RewriteRule .* - [F]
# proxy all requests to anubis after ssl termination # proxy all requests to anubis after ssl termination
RequestHeader set "X-Real-Ip" expr=%{REMOTE_ADDR} RequestHeader set "X-Real-Ip" expr=%{REMOTE_ADDR}