Commit Graph

10233 Commits

Author SHA1 Message Date
Miroslav Suchý
23ff808fe3 copr: add x86_64 reserved powerful builders 2026-02-16 11:26:43 +01:00
Pavel Raiskup
d075c470de copr-hv: enable p09 01 2026-02-16 09:59:37 +01:00
Pavel Raiskup
d2b458f041 copr-hv: copy config from 02 to vmhost-p09-copr01 2026-02-16 08:27:45 +01:00
Pavel Raiskup
5fa5dd69b1 copr-hv: fix p09_01 host 2026-02-16 07:13:14 +01:00
Kevin Fenzi
c4bdfcc897 proxies: block a ip that was hitting release-monitoring.org a lot
This ip had hit release-monitoring.org like 5,000,000 times in the
course of a few hours and swamped it's web pod.

Lets block it for now and see if anyone complains.
If this is you: please add some rate limiting.

Signed-off-by: Kevin Fenzi <kevin@scrye.com>
2026-02-13 08:48:01 -08:00
Kevin Fenzi
7e6d17307a storinator01: update mac addresses
Signed-off-by: Kevin Fenzi <kevin@scrye.com>
2026-02-10 10:22:38 -08:00
Kevin Fenzi
24ea94601d vmhost-x86-copr01/02/03: stop removing nftables
These hosts are rhel10 now and removing nftables takes out the entire
libvirt stack as it doesn't support iptables anymore.

This results in base removing libvirt and the hypervisor role
re-installing it every playbook run. It also means the network doesn't
work on guests at all.

Signed-off-by: Kevin Fenzi <kevin@scrye.com>
2026-02-09 15:08:28 -08:00
Kevin Fenzi
faff0d9e0f vmhost-p09-copr01: not encrypted yet, needs reinstall
Signed-off-by: Kevin Fenzi <kevin@scrye.com>
2026-02-09 13:34:35 -08:00
Kevin Fenzi
991273d7f1 copr_hypervisors: enable nbde on all of them
The x86 ones are now in rdu3 and reinstalled with rhel10.
All the power9 ones are in rdu3 and reinstalled.

So, we should just enable nbde on all of them.

Signed-off-by: Kevin Fenzi <kevin@scrye.com>
2026-02-09 11:30:46 -08:00
Greg Sutcliffe
1324b1e72a Add more CPU to proxy11 to help with httpd alerts
Signed-off-by: Greg Sutcliffe <fedora@emeraldreverie.org>
2026-02-06 14:03:41 +00:00
Kevin Fenzi
46ceb38264 vmhost-x86-copr02: update mac addresses
Signed-off-by: Kevin Fenzi <kevin@scrye.com>
2026-01-30 20:26:15 -08:00
Kevin Fenzi
c86adb0115 vmhost-x86-copr01: update mac addesses
Signed-off-by: Kevin Fenzi <kevin@scrye.com>
2026-01-30 13:59:30 -08:00
Kevin Fenzi
ab01301f5c inventory: update to reflect machines that moved from f42 to f43
Signed-off-by: Kevin Fenzi <kevin@scrye.com>
2026-01-30 11:36:47 -08:00
Kevin Fenzi
0b261fc507 inventory: re-enable proxy05
We still aren't able to get to mgmt on this host, but it's up and
operating normally, so we might as well use it for now.

If it goes down we can remove it again.

Signed-off-by: Kevin Fenzi <kevin@scrye.com>
2026-01-30 10:37:59 -08:00
Kevin Fenzi
5091fd4373 ocp-rdu3: retire this host/proxy/cert now that we are moved
There's no need to keep ocp-rdu3 around anymore, we only used
it when we were moving datacenters last year.

Signed-off-by: Kevin Fenzi <kevin@scrye.com>
2026-01-30 10:35:10 -08:00
James Antill
0dd9a4f2c4 proxies: Renewed stg.fp.o wildcard cert
Signed-off-by: James Antill <james@fedoraproject.org>
2026-01-29 15:30:33 -05:00
David Kirwan
ec59933a54 adding os-control01 to backups01 management
Signed-off-by: David Kirwan <davidkirwanirl@gmail.com>
2026-01-28 13:49:54 +00:00
Kevin Fenzi
25df4da94f koji01.stg: move to f43
Signed-off-by: Kevin Fenzi <kevin@scrye.com>
2026-01-27 17:28:27 -08:00
Adam Williamson
f07a93576f openqa: drop a stray - from a variable definition
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2026-01-26 15:42:09 -08:00
Adam Williamson
ae2cd3530b roles/openqa/server: drop OpenID auth support
We've been using OAuth2 for prod and stg for some time now, so
let's clean this up.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2026-01-26 15:40:50 -08:00
Greg Sutcliffe
40cc20185a Zabbix: fix more hosts/groups for apache-status
Signed-off-by: Greg Sutcliffe <fedora@emeraldreverie.org>
2026-01-26 14:18:01 +00:00
Greg Sutcliffe
a72e5b2fbf Zabbix: fix different handling of /apache-status on proxy hosts
Seems like the proxies don't want to handle port 80 nicely, I get
errors in Zabbix for them using localhost:80/apache-status (which
works elsewhere, like sundries). However using https/443 seems to
work, so we'll do that instead.

Signed-off-by: Greg Sutcliffe <fedora@emeraldreverie.org>
2026-01-26 11:39:48 +00:00
Kevin Fenzi
27a81038b9 proxies: block a bunch of networks from an abusive asn
This asn is hammering koji links.

Signed-off-by: Kevin Fenzi <kevin@scrye.com>
2026-01-25 10:32:48 -08:00
Kevin Fenzi
896ab6360b download: block subnet thats using rclone against master mirrors
Signed-off-by: Kevin Fenzi <kevin@scrye.com>
2026-01-22 10:35:44 -08:00
Kevin Fenzi
a754144f19 Update infra pagure.io links to forge.fp.o (WIP)
This should update all the references we have to
https://pagure.io/fedora-infrastructure to the
new https://forge.fedoraproject.org/infra/tickets/ area.

Do not merge this before the migration on tuesday.

Signed-off-by: Kevin Fenzi <kevin@scrye.com>
2026-01-20 14:39:40 -08:00
Jakub Kadlcik
7d56f15586 copr-be: upgrade production builders to F43
Fix https://github.com/fedora-copr/copr/issues/4001

I updated the `copr_builder_images.hypervisor.x86_64` even though the image is
not uploaded there yet because the HV is currently down.

Somebody needs to run this when it gets back online:

    STAMP=$(date -I) \
    ARCHES=x86_64 \
    TARGETS=libvirt \
    copr-upload-builder-images /var/lib/copr/public_html/images/2026-01-14/
2026-01-20 15:50:20 +01:00
Jakub Kadlcik
82245389e8 copr-be-dev: upgrade builder HV ppc64le images to F43 2026-01-20 11:32:44 +01:00
Kevin Fenzi
bad6a2db89 storinator01: add mac addresses and fix ip var
Signed-off-by: Kevin Fenzi <kevin@scrye.com>
2026-01-19 12:44:01 -08:00
Michal Konecny
5809288d1e Remove proxy05 from mirrorlist proxies
The proxy05 is unavailable for last few days, let's remove it from
mirrorlist_proxies till the situation is resolved.

This will fix mirrorlist-statistics cronjob. See
https://pagure.io/fedora-infrastructure/issue/12993 for more info.

Signed-off-by: Michal Konecny <mkonecny@redhat.com>
2026-01-19 13:37:35 +01:00
Kevin Fenzi
5298fcbd4b proxy03/14: fix larger memory sizes
Signed-off-by: Kevin Fenzi <kevin@scrye.com>
2026-01-15 11:41:26 -08:00
Jakub Kadlcik
f68255f1ae copr-be-dev: upgrade builder images to F43
Except for HV because of https://github.com/fedora-copr/copr/issues/4040
2026-01-15 14:50:12 +01:00
Greg Sutcliffe
93ed0457e0 Nagios: remove first batch of services
This removes the known-good things we've had in Zabbix for a while -
RAID, disk space, processes, and mail queue. It also removes swap which
we've decided we don't need.

Also includes some FS overrides on the Zabbix side so the relevant
NFS mounts get monitored on the OCI, and pkgs hosts, as per Nagios had.

Signed-off-by: Greg Sutcliffe <fedora@emeraldreverie.org>
2026-01-14 10:25:15 +00:00
Jiri Kyjovsky
88102722cb copr-anubis: allow copr-related user agents to pass without challenge 2026-01-12 09:00:32 +00:00
Jiri Kyjovsky
3595436104 copr-anubis: loosen policies (go through defaults), allow essentials 2026-01-08 16:40:34 +01:00
Greg Sutcliffe
5765dd09b5 Zabbix: report warnings to #noc:fpo Matrix room
Signed-off-by: Greg Sutcliffe <fedora@emeraldreverie.org>
2026-01-08 13:56:38 +00:00
Greg Sutcliffe
85ac8eeab7 Zabbix: raise warning levels for openqa disk usage
Signed-off-by: Greg Sutcliffe <fedora@emeraldreverie.org>
2026-01-08 13:55:20 +00:00
Jiri Kyjovsky
d79d6ca487 copr/anubis: fix configuration on frontend to protect only /coprs (same for distgit)
Anubis was accidentally enabled for all traffic (/) instead of just
the /coprs/ web UI. This caused unnecessary bot challenges for API
clients, dnf/yum, and other automated tools.

Use Anubis BASE_PREFIX to cleanly protect only specific endpoints:
- Frontend: /coprs/ (web UI)
- Dist-git: /{{ cgit_uri }}/ (package browser)

https://anubis.techaro.lol/docs/admin/installation#using-base-prefix
2026-01-07 13:58:11 +01:00
Miroslav Suchý
1becdd9e81 bump up number of reserved instances 2026-01-06 12:55:31 +01:00
Kevin Fenzi
ee5c732c8c people01: also increase memory to 32g
Signed-off-by: Kevin Fenzi <kevin@scrye.com>
2025-12-30 09:34:45 -08:00
Kevin Fenzi
f717aeb615 people01: bump number of cpus to 16
Signed-off-by: Kevin Fenzi <kevin@scrye.com>
2025-12-30 09:31:39 -08:00
Adam Williamson
37e2d34eb6 proxies: block IP attempting SQL injections on openQA (prod)
My staging effort seems to have worked, so let's do prod too.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2025-12-30 08:58:11 -08:00
Adam Williamson
3e8c446877 proxies: block IP attempting SQL injections on openQA
I hope I got this right. This IP is trying Little Bobby Tables
attacks on openQA and it's making the servers crash.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2025-12-30 08:46:28 -08:00
James Antill
c127602949 Remove rest of f41-test.
Signed-off-by: James Antill <james@and.org>
2025-12-26 12:46:03 -05:00
James Antill
efa0061ca5 Remove f41-test from inventory.
Signed-off-by: James Antill <james@and.org>
2025-12-25 08:35:47 -05:00
Greg Sutcliffe
9fae8c7d2d Zabbix: First try of the CMDB-in-Zabbix-Inventory idea (#12817)
Signed-off-by: Greg Sutcliffe <fedora@emeraldreverie.org>
2025-12-22 15:09:07 +00:00
Pavel Raiskup
cd95aa0095 copr-be: try to use x86 copr04 2025-12-20 15:54:12 +01:00
James Antill
c68ec66f3e Move prox01.stg to F43.
Signed-off-by: James Antill <james@and.org>
2025-12-19 14:39:12 -05:00
Greg Sutcliffe
149047b50d Copr: fix IP hostvars on vmhost-x86-copr04 - again
Signed-off-by: Greg Sutcliffe <fedora@emeraldreverie.org>
2025-12-19 11:51:57 +00:00
Greg Sutcliffe
f639e26c12 Copr: fix IP hostvars on vmhost-x86-copr04
Signed-off-by: Greg Sutcliffe <fedora@emeraldreverie.org>
2025-12-19 11:43:49 +00:00
David Kirwan
1a18ff41fd communishift: mark project communishift-fedora-coreos-ai-helpers do not
clean up.

Signed-off-by: David Kirwan <davidkirwanirl@gmail.com>
2025-12-18 16:09:42 +00:00