Commit Graph

4348 Commits

Author SHA1 Message Date
Greg Sutcliffe
04dcafe578 Zabbix: ignore vnet* on builder hosts
Signed-off-by: Greg Sutcliffe <fedora@emeraldreverie.org>
2025-07-31 14:57:04 +01:00
Greg Sutcliffe
a1b2783a67 Zabbix: ensure existence of correct hostgroups
also applies the correct hostgroups when creating/updating hosts

Signed-off-by: Greg Sutcliffe <fedora@emeraldreverie.org>
2025-07-31 12:51:43 +01:00
Greg Sutcliffe
922fec19e2 Zabbix: adjust read/write alerts for Pagure & host
Signed-off-by: Greg Sutcliffe <fedora@emeraldreverie.org>
2025-07-31 10:39:55 +01:00
Kevin Fenzi
8e4ecd355a smtp-mm: 2gb memory is too low anymore
Signed-off-by: Kevin Fenzi <kevin@scrye.com>
2025-07-30 16:07:05 -07:00
Greg Sutcliffe
322e1a78e4 Zabbix: ignore vnet* on colo_virt
Signed-off-by: Greg Sutcliffe <fedora@emeraldreverie.org>
2025-07-30 21:01:35 +01:00
Greg Sutcliffe
ce30505b5c Zabbix: lower threshold for stg proxy swap
Signed-off-by: Greg Sutcliffe <fedora@emeraldreverie.org>
2025-07-30 17:06:01 +01:00
Greg Sutcliffe
1479c0ca27 Zabbix: Point colo hosts at Zabbix rdu3 via VPN
Signed-off-by: Greg Sutcliffe <fedora@emeraldreverie.org>
2025-07-30 11:54:31 +01:00
Greg Sutcliffe
66e2129002 Zabbix: Use VPN DNS name for zabbix on pagure*
Signed-off-by: Greg Sutcliffe <fedora@emeraldreverie.org>
2025-07-30 10:50:46 +01:00
Greg Sutcliffe
d98ce7802b Zabbix: Put pagure-stg01 into prod zabbix (via vpn)
Signed-off-by: Greg Sutcliffe <fedora@emeraldreverie.org>
2025-07-30 10:26:20 +01:00
Pavel Raiskup
e6cf44da56 copr: promote tested images to production 2025-07-28 19:50:02 +02:00
Pavel Raiskup
17122b9409 copr: re-uploaded aarch64 image
With the correct --arch aarch64 metadata.
2025-07-28 19:22:53 +02:00
Pavel Raiskup
c542c03f44 copr-dev: try a new set of builder images
Relates: https://github.com/fedora-copr/copr/pull/3803
2025-07-28 14:20:01 +02:00
Greg Sutcliffe
e72d2b062b Communishift: add missing name atttribute to communishift-admins
Signed-off-by: Greg Sutcliffe <fedora@emeraldreverie.org>
2025-07-28 10:49:03 +01:00
Greg Sutcliffe
93c5faa6c1 Communishift: two new projects for Discourse and Jitsi
See Pagure tickets 12661 and 12615

Signed-off-by: Greg Sutcliffe <fedora@emeraldreverie.org>
2025-07-28 09:43:35 +01:00
Kevin Fenzi
c6ed99beb9 nagios: drop some trailing .s on entries that confuses nagios http_check plugin
Signed-off-by: Kevin Fenzi <kevin@scrye.com>
2025-07-24 13:36:52 -07:00
Kevin Fenzi
3a4aea0cd9 pagure: add another network to blocklist
Signed-off-by: Kevin Fenzi <kevin@scrye.com>
2025-07-24 13:06:59 -07:00
Greg Sutcliffe
43d29fc0bf Added buildhw-x86-13.rdu3.fedoraproject.org - in the other places
Signed-off-by: Greg Sutcliffe <fedora@emeraldreverie.org>
2025-07-24 15:58:13 +01:00
Kevin Fenzi
1b43d4160c nagios: fix duplicate mgmt host 01 that should have been 02
Signed-off-by: Kevin Fenzi <kevin@scrye.com>
2025-07-23 17:01:07 -07:00
Kevin Fenzi
b87f7de156 pagure: add another network thats hitting the api very hard
Signed-off-by: Kevin Fenzi <kevin@scrye.com>
2025-07-23 13:13:30 -07:00
Kevin Fenzi
a3dd45c855 buildvm: all these are f42 already, just adjust the more generic groups files
Signed-off-by: Kevin Fenzi <kevin@scrye.com>
2025-07-21 17:31:11 -07:00
Kevin Fenzi
b44c28e08d inventory: add some buildhw's to inventory/nagios
We want to monitor the a64 and x86 buildhw devices too.

Signed-off-by: Kevin Fenzi <kevin@scrye.com>
2025-07-21 14:36:43 -07:00
Aurélien Bompard
a51c0ea353 RabbitMQ: setup sending the queue metrix to CentOS
Signed-off-by: Aurélien Bompard <aurelien@bompard.org>
2025-07-21 15:23:42 +02:00
Michal Konecny
163446f4dc [ipa] Add ipa machines to VPN
The IPA machines are currently not reachable through VPN. This is
because they are missing firewall rules for VPN as they need to
have vpn variable set to include them.
2025-07-18 10:44:39 +02:00
Greg Sutcliffe
7f877e95ee Zabbix: Add prod Matrix room ID - take2
Signed-off-by: Greg Sutcliffe <fedora@emeraldreverie.org>
2025-07-17 17:16:13 +01:00
Greg Sutcliffe
32f290bf88 Zabbix: Add prod Matrix room ID
Signed-off-by: Greg Sutcliffe <fedora@emeraldreverie.org>
2025-07-17 17:14:20 +01:00
Kevin Fenzi
97dab9dcaf iscsi_client: readd role, apply to power10 host and switch guests to use it
This re-adds a iscsi_client role we had in iad2 back in in rdu3.
When then apply it to bvmhost-p10-01 to login and use a iscsi lun from
the rdu3 netapp. We then move the buildvm-ppc64le vm's to use this iscsi
volume instead of local storage.

As we reinstall those builders they will use the iscsi volume.

Signed-off-by: Kevin Fenzi <kevin@scrye.com>
2025-07-16 15:19:56 -07:00
Michal Konecny
7ff5ac563e [mailman3] Remove the authentication options 2
Remove the authentication options also from group vars.
2025-07-14 15:41:49 +02:00
Greg Sutcliffe
0d71c0bce0 Nagios: remove http check on p10 mgmt interface
Signed-off-by: Greg Sutcliffe <fedora@emeraldreverie.org>
2025-07-11 20:06:33 +00:00
Kevin Fenzi
233ec96688 inventory: drop non existant machines
These are various machines that are not yet deployed, or no longer exist
in rdu3 (though they did in iad2). This should clean up nagios
a fair bit and when/if we redeploy these we can add them back in.

Signed-off-by: Kevin Fenzi <kevin@scrye.com>
2025-07-09 10:26:51 -07:00
Kevin Fenzi
3a9405cd77 bvirthost: increase default number of processes before alerting
Signed-off-by: Kevin Fenzi <kevin@scrye.com>
2025-07-06 09:37:40 -07:00
Nils Philippsen
6c85fda0c9 Mass remove/replace iad2 -> rdu3, 10.3. -> 10.16.
Signed-off-by: Nils Philippsen <nils@redhat.com>
2025-07-03 20:05:02 +02:00
Kevin Fenzi
c0180dc19e proxies: drop worker06.vpn as we do not have a 06 anymore
Signed-off-by: Kevin Fenzi <kevin@scrye.com>
2025-07-03 09:48:57 -07:00
Adam Williamson
5e737c675c openqa: disable ppc64le on lab for now
We don't have any workers. We may turn this back on later, or...
not.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2025-07-02 19:54:44 -07:00
Adam Williamson
9d931214ea Revert "openQA: rename openvswitch bridge device to avoid conflict"
This reverts commit 4dc01bc892 and
a follow-up commit. I'm having trouble getting things to work
and want to see if it works if we go back to having the openQA
bridge be br0, and rename the bridge used for the system's bonded
network connection to something else instead.
2025-07-02 17:25:18 -07:00
Kevin Fenzi
90ed0a38e0 pkgs: change the pagure user to uid 1000 for suexec, block in sssd
The pagure user needs to be uid 1000 because suexec won't let users with
uid under that suexec. ;(

Also, filter pagure user out in sssd so we get the local user.

Signed-off-by: Kevin Fenzi <kevin@scrye.com>
2025-07-02 15:25:17 -07:00
Kevin Fenzi
70ee9cda84 pkgs: set ipa_host_group_desc or ipa playbook errors
failed: [pkgs01.rdu3.fedoraproject.org -> ipa01.rdu3.fedoraproject.org] (item=ipa_host_group_desc) => {"ansible_loop_var": "item", "changed": false, "item": "ipa_host_group_desc", "msg": "`ipa_host_group_desc` is not defined"}

Signed-off-by: Kevin Fenzi <kevin@scrye.com>
2025-07-02 14:52:48 -07:00
Kevin Fenzi
160cbd7932 inventory: switch everything back to mtu 1500
We have been hitting lots of weird problems going accross vlans in rdu3
with mtu 9000. For now and to stablize things, lets just switch
everything back to 1500. We can revisit this down the road, but stablity
is better than a few % of overhead.

Signed-off-by: Kevin Fenzi <kevin@scrye.com>
2025-07-02 13:26:11 -07:00
Kevin Fenzi
07b5336e55 nftables: rework for s390x builders, rip out iad2
Signed-off-by: Kevin Fenzi <kevin@scrye.com>
2025-07-02 12:40:06 -07:00
Kevin Fenzi
c5fea2e61c buildvm_s390x: actually use the new kickstart
Signed-off-by: Kevin Fenzi <kevin@scrye.com>
2025-07-02 10:43:54 -07:00
Adam Williamson
38982adb55 Try and complete openQA prod and lab deployments to RDU3
OK, I think we're ready to try this now...

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2025-07-02 09:42:35 -07:00
Kevin Fenzi
8c770773bd buildvm-s390x: move to f42 and fix dns config
Signed-off-by: Kevin Fenzi <kevin@scrye.com>
2025-07-02 09:34:33 -07:00
Adam Williamson
4dc01bc892 openQA: rename openvswitch bridge device to avoid conflict
On the new rdu3 worker hosts, br0 already exists and is the main
system 'interface' (it's a bridge on two bonded physical interfaces
connected to different switches, to make networking upgrades
easier). So we can't call our openvswitch bridge 'br0' any more.
Let's try calling it 'openqabr0' and see if anything explodes.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2025-06-30 16:14:01 -07:00
Adam Williamson
a8dfee88ab openqa: update db host to rdu3
We're not doing prod *quite* yet, but since it's down now and you
can't really run plays in iad2 any more, not worth splitting this
up temporarily.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2025-06-30 15:10:23 -07:00
Michal Konecny
0328532987 [mailman3] Remove IAD2
There are a lot of things still pointing to IAD2, let's redirect them to RDU3.
2025-06-30 20:27:53 +02:00
Kevin Fenzi
d831a03bef group_vars/all: change some defaults over to rdu3
Signed-off-by: Kevin Fenzi <kevin@scrye.com>
2025-06-30 10:26:55 -07:00
Kevin Fenzi
cc03b32da8 inventory: switch to rdu3 ipa servers by default
Signed-off-by: Kevin Fenzi <kevin@scrye.com>
2025-06-30 10:10:21 -07:00
Adam Williamson
63bc9b50ca Replace openqa-lab01.iad2 with openqa-lab01.rdu3
I think this should be everything. Also trimmed the extremely
generous resources allocated to the VM so they match the ones
used for the prod VM, as that's been working fine.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
2025-06-30 08:19:05 -07:00
Kevin Fenzi
46e93ae29b proxies: block a few more nets
Signed-off-by: Kevin Fenzi <kevin@scrye.com>
2025-06-28 17:07:23 -07:00
Kevin Fenzi
a2dad69ed9 buildvm_aarch64_rdu3: set local rdu3 ipa server as default
Signed-off-by: Kevin Fenzi <kevin@scrye.com>
2025-06-28 15:29:40 -07:00
Kevin Fenzi
d2598f422c buildvm-ppc64le.rdu3: oddly, aarch64 kernel does not boot on ppc64le?
Signed-off-by: Kevin Fenzi <kevin@scrye.com>
2025-06-28 13:07:36 -07:00