WarlockFish/fedora-infra_ansible
1
0
Fork 0
mirror of https://pagure.io/fedora-infra/ansible.git synced 2026-03-19 19:46:38 +08:00
Code Issues Packages Projects Releases Wiki Activity

download: block infinite crawler looping

Browse Source 
The /pub/alt/virtio-win directory had in it some 11 year old things.
One was a readme noting that it moved 11 years ago and nothing was
still here. The others were links to .

The scrapers, being as dumb as posts followed all those links over and
over again to the tune of millions per day.
I removed the links, but of course they were still trying, so
lets be a bit more aggressive and just 403 them all.

Signed-off-by: Kevin Fenzi <kevin@scrye.com>
This commit is contained in:
Kevin Fenzi
2026-02-12 16:54:16 -08:00
parent 0d56b527a6
commit fefbc356f0
1 changed files with 5 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

5
roles/download/templates/httpd/dl.fedoraproject.org.conf
View File

@@ -32,6 +32,11 @@
SSLProtocol {{ ssl_protocols }}
SSLCipherSuite {{ ssl_ciphers }}
RewriteEngine On
RewriteCond %{REQUEST_URI} ^/pub/alt/virtio-win/.*$
RewriteRule .* - [F]
# proxy all requests to anubis after ssl termination
RequestHeader set "X-Real-Ip" expr=%{REMOTE_ADDR}