Commit Graph

826 Commits

Author SHA1 Message Date
Kevin Fenzi
92867b50a2 base / crypto-policies: move fedora instances back to DEFAULT
We changed this to DEFAULT:FEDORA32 a while back because the certs for
the old totpcgi sudo needed it to work. Now thats all gone and we are
100% on ipa and sssd, this should no longer be needed.

Signed-off-by: Kevin Fenzi <kevin@scrye.com>
2022-06-06 14:10:20 -07:00
Kevin Fenzi
17959a301c base: no more docker anywhere, drop this
We aren't using docker anywhere anymore, so we don't need to uselessly
try and restart it all the time.

Signed-off-by: Kevin Fenzi <kevin@scrye.com>
2022-06-02 15:14:54 -07:00
Kevin Fenzi
33403e750e bastion: readd may tls for all non rh sites
Signed-off-by: Kevin Fenzi <kevin@scrye.com>
2022-05-24 17:46:04 -07:00
Kevin Fenzi
caf5f130c8 bastion / postfix: clean up tls_policy duplicates
Signed-off-by: Kevin Fenzi <kevin@scrye.com>
2022-05-24 12:33:26 -07:00
Kevin Fenzi
a1af68173d bastion / postfix: disable tls_resue for now
Something is broken with smtp_tls_connection_reuse = yes, so disable it
for now. Also, setup a tls_policy map file and tell it to not use tls
for mx2.redhat.com. The normal smtp connection reuse works just fine, so
this will keep mail flowing until we can one day figure out why tls
connection reuse is busted.

Signed-off-by: Kevin Fenzi <kevin@scrye.com>
2022-05-24 12:24:30 -07:00
Kevin Fenzi
81771937c2 bastion / postfix: need to set session cache for tls connections and timeout
Signed-off-by: Kevin Fenzi <kevin@scrye.com>
2022-05-24 09:13:08 -07:00
Kevin Fenzi
105cad26dc postfix / main.cf: fix releng and vpn config and drop phx2
We fixed the config in a PR from aheath1992 for most of the machines,
but we need to fix vpn (proxies in particular) and releng boxes now.
Also, while we are here, lets drop the phx2 file since it's not used
anymore.

Signed-off-by: Kevin Fenzi <kevin@scrye.com>
2022-05-23 16:43:01 -07:00
Kevin Fenzi
b1395e9088 bastion / postfix: re-add missing smtp_connection_cache_destinations
Without this it wasn't caching tls connections and was going over the
small limit redhat.com mx had. Hopefully this gets mail flowing again.

Signed-off-by: Kevin Fenzi <kevin@scrye.com>
2022-05-23 08:44:36 -07:00
Andrew Heath
32764a7f3a Testing https://access.redhat.com/solutions/5420401 on postfix systems 2022-05-17 00:29:25 +00:00
Kevin Fenzi
ba294ed5c6 postfix / gateway: fix tlsproxy to actually work
So, without this tlsproxy wasn't working and no connection reuse was
happening. With it, it seems to be processing away nicely and reusing
connections.

Signed-off-by: Kevin Fenzi <kevin@scrye.com>
2022-04-26 12:31:48 -07:00
Kevin Fenzi
6d3114e3bd base / postfix / bastion/gateway: fix up email flow to @redhat.com
Recently, redhat.com changed internal MX servers. The new servers are
have rate limits on incoming emails from one ip and admins there don't
want to add a bunch of exceptions, so we need to adjust our end to not
flood connections to them. Currently, connections burst up to 100 (the
smtp postfix default) which goes over their limits and causes the
internal MX to reject emails from us for a while.

So, this change:

* Adds some domains to fast_flush. This allows us to use postqueue -s
  domain to flush emails to a particular domain.

* Changes the smtp limit to 40. This is under the redhat.com limit.

* Has ansible actually install the master.cf.gateway on bastion servers.
  Currently they were using the stock/default one.

* Enables the tlsproxy service, which is actually needed to get that tls
  reuse working.

After these changes, we keep few connections to the redhat.com mx open,
but we reuse them and send more emails over existing connections. No
'too many connection emails' have happened since the changes.
The queue slowly seems to be processing down.

Since this was causing an outage of email, I have already applied these
things to bastion01, but I'd like to make sure we match up to whats in
ansible.

Signed-off-by: Kevin Fenzi <kevin@scrye.com>
2022-04-24 16:59:35 +00:00
Kevin Fenzi
affac369ab postfix / main.cf / gateway: enable smtp_tls_connection_reuse
Right now we are hitting redhat.com mx server connection limits.
This might be because we are starting too many new connections at once.
Enabling this should reduce the new connections by reusing existing
ones.

Signed-off-by: Kevin Fenzi <kevin@scrye.com>
2022-04-20 16:07:40 -07:00
Kevin Fenzi
44caedc87d bastion: switch to mx2 for redhat.com emails
It seems this is a load balancer that we should be using according to
the internal ticket.

Signed-off-by: Kevin Fenzi <kevin@scrye.com>
2022-04-05 07:58:45 -07:00
Kevin Fenzi
20b3db2561 bastion: switch email gateway for redhat.com emails
The old ext-mx seems no more, switch to new gateway.

Signed-off-by: Kevin Fenzi <kevin@scrye.com>
2022-04-04 16:20:27 -07:00
Kevin Fenzi
fbb4e4691f Adjust the list of hosts that do not use linux-system-roles/network
In addition to the cloud_aws group, we want to exclude ibiblio virthosts
as they use bonding and thats a more complex setup. Someday we should
get it working here.

Signed-off-by: Kevin Fenzi <kevin@scrye.com>
2022-01-26 14:09:43 -08:00
Kevin Fenzi
eb991fa9c1 base / big network cleanup
Everything should now be using linux-system-roles/network, so we drop
our hacky nmcli calls and everything that referred to them, including
exclude variables. Also, lets just let NM handle resolv.conf so it's not
wrong all the time on reboots.

Signed-off-by: Kevin Fenzi <kevin@scrye.com>
2022-01-26 12:14:23 -08:00
Kevin Fenzi
f824885753 base / linux-system-roles: the group is cloud_aws
Signed-off-by: Kevin Fenzi <kevin@scrye.com>
2022-01-25 15:16:05 -08:00
Kevin Fenzi
f6a8b121f7 base / linux-system-roles: double quote the when line
Signed-off-by: Kevin Fenzi <kevin@scrye.com>
2022-01-25 15:10:19 -08:00
Kevin Fenzi
796f59fa1b base / linux-system-roles: try and reword when
Signed-off-by: Kevin Fenzi <kevin@scrye.com>
2022-01-25 15:07:45 -08:00
Kevin Fenzi
ec42acdcab base / linux-system-roles: Clean up conditional
Signed-off-by: Kevin Fenzi <kevin@scrye.com>
2022-01-25 15:02:04 -08:00
Kevin Fenzi
6ebe03a470 base: linux-system-roles: cloud datacenter to be wider
Signed-off-by: Kevin Fenzi <kevin@scrye.com>
2022-01-25 14:52:13 -08:00
Kevin Fenzi
d1b419b257 base: linux-system-roles are not used in cloud/aws
Signed-off-by: Kevin Fenzi <kevin@scrye.com>
2022-01-25 14:45:57 -08:00
Pavel Raiskup
408a990a24 base: try s/dnf/package/ module
TASK [base : Ensure iptables is installed] ************************************
Wednesday 05 January 2022  10:49:01 +0000 (0:00:00.251)       0:01:17.816 *****
Wednesday 05 January 2022  10:49:01 +0000 (0:00:00.251)       0:01:17.815 *****
fatal: [noc02.fedoraproject.org]: FAILED! => {"changed": false, "cmd":
"dnf install -y python2-dnf", "msg": "[Errno 2] No such file or directory", "rc": 2}
2022-01-05 13:03:37 +01:00
Mark O Brien
148f8307d7 base: ensure iptables is installed
Signed-off-by: Mark O Brien <markobri@redhat.com>
2021-12-21 17:06:32 +00:00
T.C. Williams
995f499a6d Adjust rsyslog imjournal rate-limiting settings to prevent dropped messages under load. fedora-infrastructure-9512 2021-11-19 00:30:32 +00:00
Eddie Jennings, Jr
6ef496d56a Reconfigure IPv6
Reconfigure IPv6

Reconfigure IPv6

Reconfigure IPv6

Reconfigure IPv6

Reconfigure IPv6

Reconfigure IPv6

Reconfigure IPv6

Configure IPv6

Reconfigure IPv6

Reconfigure IPv6

Reconfigure IPv6

Reconfigure IPv6

Reconfigure IPv6

Reconfigure IPv6

Reconfigure IPv6

Reconfigure IPv6

Configure IPv6

Update IPv6 address for noc02 rule

Update IPv6 address in confg for noc02 address change

Update IPv6 address for proxy04

Update IPv6 address for torrent02
2021-11-08 22:56:05 +00:00
Nils Philippsen
c782eceae1 Move syncHttpLogs.sh into web-data-analysis role
Signed-off-by: Nils Philippsen <nils@redhat.com>
2021-07-23 13:06:23 +02:00
Adam Saleh
eb42244c28 Remove the comment, timeouts should be sufficient. 2021-07-22 21:34:00 +02:00
Adam Saleh
be40017d00 Change the syncHttp timeout to 2h max, comment out the misbehaving proxy09 2021-07-22 21:34:00 +02:00
Adam Saleh
721e3e9864 Add 3 times retry and 30m+30m rsync timeout to syncHttpLogs 2021-07-22 21:34:00 +02:00
Kevin Fenzi
5d6f460b42 buildvm_s390x: the z/vm s390x instances use enc900 as network interface
Signed-off-by: Kevin Fenzi <kevin@scrye.com>
2021-07-07 13:43:22 -07:00
Kevin Fenzi
ad91eae586 base: adjust grouping to give the desired result in when for NM tweaking
Signed-off-by: Kevin Fenzi <kevin@scrye.com>
2021-07-07 12:56:46 -07:00
Pierre-Yves Chibon
2b46beed2c fedocal: drop everything related to fedocal in VM
Signed-off-by: Pierre-Yves Chibon <pingou@pingoured.fr>
2021-06-23 17:30:27 +02:00
Kevin Fenzi
ee49c53f08 base / iptables: Adjust iptables on all vpn hosts to drop less secure traffic
Most of our vpn hosts are on a 192.168.1.0/24 network.
However we have a small number on a 'less secure' 'less trusted' subnet:
192.168.100.0/24. This change adds in logic to:
* on log01, allow rsyslog from 192.168.100.x hosts
* on ipa servers, allow ipa ports for 192.168.100.x hosts
* then reject everything else.
This will make sure 192.168.100.x hosts can only hit ssh and the two
above items, otherwise all vpn hosts will reject their traffic. This
should add a bit of security to having those hosts on the vpn.

Signed-off-by: Kevin Fenzi <kevin@scrye.com>
2021-05-04 21:15:05 +00:00
Kevin Fenzi
6a2bff7af9 base / ifcfg: make sure and set staging to use stg.iad2.fedoraproject.org domain
This should only affect stg hosts.
We had set all of iad2 the same, prod and stg both.
We need to make sure stg resolves to stg hosts first.
This worked somewhat until now because we replace the resolv.conf on stg
hosts, but without this they are borken right after boot and until we
replace the resolv.conf and restart httpd or other services.

Signed-off-by: Kevin Fenzi <kevin@scrye.com>
2021-04-21 12:13:06 -07:00
Kevin Fenzi
dc201cf3d6 linux system roles: add tag
Signed-off-by: Kevin Fenzi <kevin@scrye.com>
2021-03-31 11:35:25 -07:00
Kevin Fenzi
fc2db16120 iptables / kojibuilder: add some more ports needed by ipa-clients
Signed-off-by: Kevin Fenzi <kevin@scrye.com>
2021-03-29 15:20:00 -07:00
Kevin Fenzi
6bf8552e7f base / iptables / kojibuilder: add ipa ports for koji builder ipa clients
Note that this will not yet work, it needs the RHIT firewall between
vlans opened on these ports first, but after that this is needed to
allow them to use those ports.

Signed-off-by: Kevin Fenzi <kevin@scrye.com>
2021-03-25 10:10:55 -07:00
Kevin Fenzi
55cc28c3b1 base / iptables: Add rules to block staging networks on prod hosts
In IAD2 the prod and stg hosts are on different VLANs, so we thought we
didn't need this. However, we are still seeing some odd mixing of prod
and stg fedmsgs, so likely some fedmsg port has become enabled accross
all the VLANS. In any case this should do no harm, it just adds 2
subnets on all prod hosts to block staging, except for a small number of
staging_friendly hosts (in the staging_friendly ansible group).

Signed-off-by: Kevin Fenzi <kevin@scrye.com>
2021-03-19 20:08:05 +00:00
Stephen Smoogen
ec8c05f116 Fix tasks/postfix.yaml and roles/basic/tasks/postfix.yaml to match
Found the reason that the definitions I had put were not
working. There were two different ones and i was looking at the wrong
one. Put the two tasks with the same logic so things should work no
matter which one is run.
2021-02-17 19:17:50 -05:00
Aurélien Bompard
00e8e4eb25 Don't get a keytab on IPA itself
Signed-off-by: Aurélien Bompard <aurelien@bompard.org>
2021-02-11 12:34:22 +01:00
Kevin Fenzi
acbf9e21f7 base / crypto-policy: always run, even in check mode
We need to always run these even in check mode, because they register
things used in the last one of them. So, this could change this in check
mode if we modify it. Be careful!

Signed-off-by: Kevin Fenzi <kevin@scrye.com>
2021-02-10 15:13:19 -08:00
Pavel Raiskup
a2d65ff508 Move devel servers to elastic IPs, too 2021-01-18 14:58:58 +01:00
Mark O'Brien
8c00c6840b iptables change wasnt needed 2020-11-24 17:35:33 +00:00
Mark O'Brien
a426b0e240 no iptables on Centos 8 2020-11-24 17:26:29 +00:00
Pavel Raiskup
f14b897b1f copr: point playbooks to new set of (elastic) IPs 2020-11-13 10:05:21 +01:00
Jakub Kadlcik
e043b62c5a reprovisioning copr-fe-dev from a new instance 2020-11-11 14:12:15 +01:00
Kevin Fenzi
e5606578de base: try changing f33 crypto-policies to a less open version to get 2fa working
LEGACY allows all kinds of old junk, lets try and just
enable the things that FEDORA32 allowed.

Signed-off-by: Kevin Fenzi <kevin@scrye.com>
2020-11-03 14:10:39 -08:00
Adam Williamson
e14052db33 ok debug stuff did its job goodbye
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-10-30 13:14:54 -07:00
Adam Williamson
ea70d16680 gah stupid tags
Signed-off-by: Adam Williamson <awilliam@redhat.com>
2020-10-30 13:12:33 -07:00