Commit Graph

512 Commits

Author SHA1 Message Date
Kevin Fenzi
c12a1cb27d pagure: drop fedora-infra forks for now as crawlers are hammering the crap out of pagure over them
Signed-off-by: Kevin Fenzi <kevin@scrye.com>
2025-06-02 07:43:30 -07:00
Kevin Fenzi
836d79193c pagure: update ansible for latest blocks from ai
Update the latest project blocks that were added.

Also, extend the bot block to docs.pagure.io, which was
being hammered by scrapers.

Signed-off-by: Kevin Fenzi <kevin@scrye.com>
2025-05-30 10:02:24 -07:00
Kevin Fenzi
b55d66cc51 Revert "pagure: block some more forks and blame/history from ai scrapers"
This reverts commit 1c7ecb6bcf.

Lets see if the bots have moved on.
2025-05-26 08:32:37 -07:00
Kevin Fenzi
1c7ecb6bcf pagure: block some more forks and blame/history from ai scrapers
Signed-off-by: Kevin Fenzi <kevin@scrye.com>
2025-05-25 21:47:39 -07:00
Kevin Fenzi
f67d942dbb pagure: http.logrotate is a template now
Signed-off-by: Kevin Fenzi <kevin@scrye.com>
2025-05-24 21:21:27 -07:00
Kevin Fenzi
91e9a5627d httpd / botblocking: fix syntax on bot rewrite
These have to be in "s in order to do a string comparison, since
they were not, they were never matching anything. ;(

Fix them all up, and also block a few more repos on pagure that are
getting heavily crawled.

Signed-off-by: Kevin Fenzi <kevin@scrye.com>
2025-05-13 11:39:25 -07:00
Michal Konecny
a0046b5b4b [distgit] Fixing ansible lint errors
This commit is fixing ansible lint errors for distgit role.

It also introduces two new addition to skip list as the structure of our
ansible repository doesn't adhere to ansible standards.

The errors that will be now skipped are:
- role-name[path] - we have plenty of roles that have sub-roles inside
  them and we need to access them
- var-naming[no-role-prefix] - variables for roles are not usually
  prefixed correctly in our repository and forcing people to change that
  will introduce more issues than what it solves
2025-04-30 10:07:35 +00:00
Kevin Fenzi
4bba87381d pagure: block some old mirror copies of big projects
Signed-off-by: Kevin Fenzi <kevin@scrye.com>
2025-04-16 10:22:35 -07:00
Ryan Lerch
be6e3b0161 pagure: close pagure.io off to new projects
Signed-off-by: Ryan Lerch <rlerch@redhat.com>
2025-04-15 01:40:19 +00:00
Aurélien Bompard
d884a0f8ba Use the combined RabbitMQ CA cert in the clients
Signed-off-by: Aurélien Bompard <aurelien@bompard.org>
2025-04-11 15:15:45 +02:00
Ryan Lerch
654260ceff [pagure stg] disable new projects from being created
Signed-off-by: Ryan Lerch <rlerch@redhat.com>
2025-04-03 18:50:47 +10:00
Michal Konecny
1459e74b5a [pagure] Tweak the performance of httpd server
Today the pagure started to be unresponsive. After few hours of
debugging we narrowed the issue to worker limit. We tweaked the value to
something that should be more responsive and it helped.

Here is the PR to make the change permanent.

Thanks @gwmngilfen and @nphillip for help with that.
2025-03-26 14:03:34 +00:00
Michal Konecny
6b0a66cb37 [fedora-messaging] Adding topic_prefix back
Removing the topic_prefix from fedora messaging config files was a bit
premature. So let's put them back in place.

Signed-off-by: Michal Konecny <mkonecny@redhat.com>
2025-02-14 12:40:12 +01:00
Michal Konecny
6428f8f772 Sunset github2fedmsg and fedmsg
This commit is removing all the fedmsg related stuff from ansible
repository.

Signed-off-by: Michal Konecny <mkonecny@redhat.com>
2025-02-13 10:08:51 +00:00
Michal Konecny
2ec055db6f Use first uppercase letter for all handlers
This will unify all the handlers to use first uppercase letter for
ansible-lint to stop complaining.

I went through all `notify:` occurrences and fixed them by running
```
set TEXT "text_to_replace"; set REPLACEMENT "replacement_text"; git grep
-rlz "$TEXT" . | xargs -0 sed -i "s/$TEXT/$REPLACEMENT/g"
```

Then I went through all the changes and removed the ones that wasn't
expected to be changed.

Fixes https://pagure.io/fedora-infrastructure/issue/12391

Signed-off-by: Michal Konecny <mkonecny@redhat.com>
2025-02-10 20:31:49 +00:00
Michal Konecny
7b58dfdce8 Remove fedmsg and github2fedmsg from staging
The messaging bridges openshift project and github2fedmsg VM were
already removed in staging. This is to clean the ansible playbooks.

I will create a separate one for production after this one is merged.

Signed-off-by: Michal Konecny <mkonecny@redhat.com>
2025-02-04 09:13:40 +01:00
Kevin Fenzi
a6c874f45e pagure: fix overzellous shell call that should be a variable
Signed-off-by: Kevin Fenzi <kevin@scrye.com>
2025-01-24 20:18:38 -08:00
Ryan Lerch
47c68f478d ansiblelint fixes - fqcn[action-core] - template to ansible.builtin.template
Replaces references to template: with ansible.builtin.template

Signed-off-by: Ryan Lerch <rlerch@redhat.com>
2025-01-15 11:30:29 +10:00
Ryan Lerch
3c41882bb0 ansiblelint fixes - fqcn[action-core] - shell to ansible.builtin.shell
Replaces references to shell: with ansible.builtin.shell

Signed-off-by: Ryan Lerch <rlerch@redhat.com>
2025-01-15 11:29:10 +10:00
Ryan Lerch
25391e95b7 ansiblelint fixes - fqcn[action-core] - package to ansible.builtin.package
Replaces many references to  package: with ansible.builtin.package

Signed-off-by: Ryan Lerch <rlerch@redhat.com>
2025-01-15 11:28:00 +10:00
Ryan Lerch
462176464b ansiblelint fixes-- fqcn[action-core] - command to ansible.builtin.command
Replaces many references to  command: with ansible.builtin.command

Signed-off-by: Ryan Lerch <rlerch@redhat.com>
2025-01-15 11:26:47 +10:00
Ryan Lerch
6a3816dfdc ansiblelint fixes-- fqcn[action-core] - copy to ansible.builtin.copy
Replaces many references to 'copy' with ansible.builtin.copy

Signed-off-by: Ryan Lerch <rlerch@redhat.com>
2025-01-15 10:43:31 +10:00
Ryan Lerch
62952df107 ansiblelint fixes-- fqcn[action-core] - file to ansible.builtin.file
Replaces many references to  file: with ansible.builtin.file

Signed-off-by: Ryan Lerch <rlerch@redhat.com>
2025-01-15 10:41:52 +10:00
Ryan Lerch
691adee6ee Fix name[casing] ansible-lint issues
fix 1900 failures of the following case issue:

`name[casing]: All names should start with an uppercase letter.`

Signed-off-by: Ryan Lerch <rlerch@redhat.com>
2025-01-14 20:20:07 +10:00
Ryan Lerch
89f6f1fc32 Fix majority of remaining yamllint warnings and errors
Signed-off-by: Ryan Lerch <rlerch@redhat.com>
2024-11-28 17:31:45 +10:00
Aurélien Bompard
e479cb5600 Pagure: fix the user's full name with OIDC
Signed-off-by: Aurélien Bompard <aurelien@bompard.org>
2024-11-14 07:44:51 +01:00
Aurélien Bompard
901c843706 Prepare dist-git for OIDC
Signed-off-by: Aurélien Bompard <aurelien@bompard.org>
2024-11-13 09:53:50 +01:00
Aurélien Bompard
297d92b7b0 Switch Pagure to OIDC in prod
Signed-off-by: Aurélien Bompard <aurelien@bompard.org>
2024-11-13 09:24:48 +01:00
Jiri Podivin
f513e7cbcd Linting python scripts
Signed-off-by: Jiri Podivin <jpodivin@redhat.com>
2024-09-18 19:57:29 +00:00
Kevin Fenzi
8925937958 pagure: Drop incorrect robots.txt entry
Signed-off-by: Kevin Fenzi <kevin@scrye.com>
2024-07-13 10:21:28 -07:00
Stephen Smoogen
969bbfcf2a Add blockers to dl.fedoraproject.org
Looked at logs of servers being hit by the 'non-responsive' bots and
the following were hit heavily every day multiple times a day:

100006 nagios.fedoraproject.org-access.log
102150 koschei.fedoraproject.org-access.log
162296 lists.fedoraproject.org-access.log
495776 fedoraproject.org-access.log
850471 dl.fedoraproject.org-access.log

Added bloks to dl.fedoraproject to try and lower its hit rate. Others
need review from people who know their internals more.

Signed-off-by: Stephen Smoogen <ssmoogen@redhat.com>
2024-07-09 09:18:56 +00:00
Nils Philippsen
205e2c9ed5 distgit/pagure: Set git directories as safe
This was fixed previously for pagure.io in the context of
paguremirroring. Turns out, it affects all kinds of git operations, so
document and move accordingly.

Fixes: releng#12181
Fixes: fedora-infrastructure#12010

Signed-off-by: Nils Philippsen <nils@redhat.com>
2024-06-27 11:21:19 +02:00
Kevin Fenzi
c991e41638 pagure: allow https push on pagure.io too
Signed-off-by: Kevin Fenzi <kevin@scrye.com>
2024-06-17 15:09:03 -07:00
Michal Konecny
abe14b6489 [pagure] Change the folder permission for pagure
To create a new log file the as paguremirroring user we need to add write
permissions to /var/log/pagure/ folder as well. This is correctly set for
distgit/pagure role, but not for pagure itself.

Signed-off-by: Michal Konecny <mkonecny@redhat.com>
2024-06-04 11:52:42 +02:00
Michal Konecny
c897c9c197 [pagure] Add tags to acl fix
Signed-off-by: Michal Konecny <mkonecny@redhat.com>
2024-06-04 10:28:39 +02:00
Kevin Fenzi
4701fd8783 pagure: fix typo
Signed-off-by: Kevin Fenzi <kevin@scrye.com>
2024-06-03 14:13:40 -07:00
Michal Konecny
c5b14f29fa [pagure] Add default ACLs to /var/log/pagure
This is a permanent fix for https://pagure.io/fedora-infrastructure/issue/11957

Signed-off-by: Michal Konecny <mkonecny@redhat.com>
2024-06-03 21:10:36 +00:00
Kevin Fenzi
4a3da58a9e pagure: drop Bytespider crawler
This crawler is hitting pagure.io really hard.
Sent it a 403 and block in robots.txt

Signed-off-by: Kevin Fenzi <kevin@scrye.com>
2024-05-01 12:19:43 -07:00
Kevin Fenzi
ec49ea3083 pagure: increase crawl delay to reduce load
This was causing pagure to not be very responsive.
Lets try increasing the delay to reduce load.

Signed-off-by: Kevin Fenzi <kevin@scrye.com>
2024-02-28 14:04:02 -08:00
Kevin Fenzi
c471f29867 pagure: try increasing the number of processes for pagure.io
Signed-off-by: Kevin Fenzi <kevin@scrye.com>
2024-02-05 14:06:40 -08:00
Michal Konecny
17f02d725a [Pagure] Enable safe directories for production
See https://pagure.io/fedora-infrastructure/issue/11330 for more details.

Signed-off-by: Michal Konecny <mkonecny@redhat.com>
2023-09-21 17:19:38 +02:00
Michal Konecny
ba7424bdf2 [Pagure] The value needs to be string
Signed-off-by: Michal Konecny <mkonecny@redhat.com>
2023-09-06 10:17:33 +02:00
Michal Konecny
3c9cd1ad34 [Pagure] Set git directories as safe system-wide
This will solve https://pagure.io/fedora-infrastructure/issue/11330

Signed-off-by: Michal Konecny <mkonecny@redhat.com>
2023-09-06 10:12:02 +02:00
Michal Konecny
4c2dc64958 [Pagure] Disable OIDC on production for now
Limiting the number of processes to 1 caused unexpected error on production
instance. For more info see https://pagure.io/fedora-infrastructure/issue/10372#comment-868823

Signed-off-by: Michal Konecny <mkonecny@redhat.com>
2023-08-14 12:30:45 +02:00
Michal Konecny
88f7258c7b [Pagure] Enable OIDC on production
As OIDC authentication works without issue on staging let's enable it on
production as well.

Signed-off-by: Michal Konecny <mkonecny@redhat.com>
2023-08-14 11:25:39 +02:00
Aurélien Bompard
44ce99733e Pagure: only use a single apache process
Signed-off-by: Aurélien Bompard <aurelien@bompard.org>
2023-08-11 12:29:10 +00:00
Michal Konecny
a4ddf54509 [Pagure] Limit the httplib2 task by tags
Signed-off-by: Michal Konecny <mkonecny@redhat.com>
2023-08-11 12:04:09 +02:00
Aurélien Bompard
1e26cf9246 Some more fixes to get Pagure to work with OIDC on staging
- patch the httplib2 library to avoid hardcoding TLSv1
- set the missing configuration variables in `pagure.cfg` (they have no
  defaults)
- set the password for the future production version of
  `client_secrets.json`

Also note that in the private ansible repo, the Pagure client
configuration in Ipsilon was fixed: the `token_endpoint_auth_method`
variable was set to `"client_secret_post"`.

Signed-off-by: Aurélien Bompard <aurelien@bompard.org>
2023-08-11 11:39:03 +02:00
Michal Konecny
a05113b48d [Pagure] Fix the client_secrets.json
The `env_suffix` variable doesn't work in case of pagure. So we need to do it
different way.

Signed-off-by: Michal Konecny <mkonecny@redhat.com>
2023-08-11 11:36:38 +02:00
Michal Konecny
d737ecb396 [Pagure] Disable OIDC for staging pagure
The OIDC is not working yet on staging pagure and the work is being done in
https://pagure.io/fedora-infra/ansible/pull-request/1514

Signed-off-by: Michal Konecny <mkonecny@redhat.com>
2023-07-28 14:14:52 +02:00