WarlockFish/fedora-infra_ansible
1
0
Fork 0
mirror of https://pagure.io/fedora-infra/ansible.git synced 2026-05-02 14:30:45 +08:00
Code Issues Packages Projects Releases Wiki Activity

Reduce number of parallel crawlers from 38 to 35

Browse Source 
To avoid regular nagios memory warnings the number of parallel crawlers
is reduced from 38 to 35 to reduce the required memory.
This commit is contained in:
Adrian Reber
2015-05-31 08:11:04 +00:00
parent 492db6878a
commit af9462caab
1 changed files with 2 additions and 2 deletions
Download Patch File Download Diff File Expand all files Collapse all files

4
roles/mirrormanager/crawler/files/crawler.cron
View File

@@ -1,8 +1,8 @@
# run the crawler twice a day
# logs sent to /var/log/mirrormanager/crawler.log and crawl/* by default
# 32GB of RAM is not enough for 75 threads, 38 seems to work so far
# 32GB of RAM is not enough for 75 threads, 35 seems to work so far
#
# [ "`hostname -s`" == "mm-crawler02" ] && sleep 3h is used to start the crawl
# later on the second crawler to reduce the number of parallel accesses to
# the database
0 */12 * * * mirrormanager [ "`hostname -s`" == "mm-crawler02" ] && sleep 3h; /usr/bin/mm2_crawler --timeout-minutes 180 --threads 38 `/usr/local/bin/run_crawler.sh 2` > /dev/null 2>&1
0 */12 * * * mirrormanager [ "`hostname -s`" == "mm-crawler02" ] && sleep 3h; /usr/bin/mm2_crawler --timeout-minutes 180 --threads 35 `/usr/local/bin/run_crawler.sh 2` > /dev/null 2>&1