f8a5720535
This should automate running the "countme" scripts every day to parse new log data and publish updated totals. Here's what I've added to the ansible role: * install package deps for `mirrors-countme` * make "countme" user with home /srv/countme * clone 'prod' branch of https://pagure.io/mirrors-countme to /srv/countme * if changed: pip install /srv/countme/mirrors-countme * make web subdir /var/www/html/csv-reports/countme * make local data dir /var/lib/countme * install `countme-update.sh` to /usr/local/bin * install `countme-update.cron` to /etc/cron.d * runs /usr/local/bin/countme-update.sh daily, as user `countme` That should make sure `countme-update.sh` runs every day. That script works like this: 1. Run `countme-update-rawdb.sh` * parse new mirrors.fp.o logs in /var/log/hosts/proxy* * write data to /var/lib/countme/raw.db 2. Run `countme-update-totals.sh` * parse raw data from /var/lib/countme/raw.db * write updated totals to /var/lib/countme/totals.{db,csv} 3. Track changes in updated totals * set up /var/lib/countme as git repo (if needed) * commit new `totals.csv` (if changed) 4. Make updated totals public * Copy totals.{db,csv} to /var/www/html/csv-reports/countme For safety's sake, I've tried to set up everything so it runs as the `countme` user rather than running everything as `root`. This might be an unnecessary complication but it seemed like the right thing to do. Similarly, keeping totals.csv in a git repo isn't _required_, but it seemed like a good idea to keep historical records in case we want/need to change the counting algorithm or something. I checked the YAML with ansible-lint and tested that all the scripts work as expected when run as `wwoods`, so unless I've missed something this should do the trick.
Fedora Infrastructure
Welcome! This is the Fedora Infrastructure Ansible Pagure project.
Pull requests and forks can be made against this repository hosted at https://pagure.io/fedora-infra/ansible
This repository is also mirrored for production runs to https://infrastructure.fedoraproject.org/infra/ansible/ but this is the working repository where changes are made.
If you would like to help out with Fedora Infrastructure, see:
- https://fedoraproject.org/wiki/Infrastructure/GettingStarted
- https://fedoraproject.org/wiki/Infrastructure_Apprentice
Ansible repository/structure
files - files and templates for use in playbooks/tasks
- subdirs for specific tasks/dirs highly recommended
inventory - where the inventory and additional vars is stored
- All files in this directory in ini format
- added together for total inventory
group_vars:
- per group variables set here in a file per group
host_vars:
- per host variables set here in a file per host
library - library of custom local ansible modules
playbooks - collections of plays we want to run on systems
groups: groups of hosts configured from one playbook.
hosts: playbooks for single hosts.
manual: playbooks that are only run manually by an admin as needed.
tasks - snippets of tasks that should be included in plays
roles - specific roles to be use in playbooks.
Each role has it's own files/templates/vars
filter_plugins - Jinja filters
master.yml - This is the master playbook, consisting of all
current group and host playbooks. Note that the
daily cron doesn't run this, it runs even over
playbooks that are not yet included in master.
This playbook is usefull for making changes over
multiple groups/hosts usually with -t (tag).
Paths
The public path on batcave01 (our control host) for everything is /srv/web/infra/ansible
The private path on batcave01 (our control host) (which is sysadmin-main accessible only)
is /srv/private/ansible
In general to run any ansible playbook you will want to run:
sudo -i ansible-playbook /path/to/playbook.yml
(On batcave01, our control host)
Scheduled check-diff
Every night a cron job runs over all playbooks under playbooks/{groups}{hosts}
with ansible --check --diff. A report from this is sent to sysadmin-logs.
In the ideal state this report would be empty.
Idempotency
All playbooks should be idempotent. Ie, if run once they should bring the machine(s) to the desired state, and if run again N times after that they should make 0 changes (because the machine(s) are in the desired state). Please make sure your playbooks are idempotent.
Can be run anytime
When a playbook or change is checked into ansible you should assume that it could be run at ANY TIME. Always make sure the checked in state is the desired state. Always test changes when they land so they don't surprise you later.