From 174a4b66cff72d505798976de93538c6f0478b84 Mon Sep 17 00:00:00 2001 From: Ralph Bean Date: Thu, 27 Oct 2016 16:17:25 -0400 Subject: [PATCH 1/2] Add a HACKING.rst doc alongside README.rst. --- HACKING.rst | 102 ++++++++++++++++++++++++++++++++++++++++++++++++++++ README.rst | 88 +++------------------------------------------ 2 files changed, 107 insertions(+), 83 deletions(-) create mode 100644 HACKING.rst diff --git a/HACKING.rst b/HACKING.rst new file mode 100644 index 00000000..d20850e0 --- /dev/null +++ b/HACKING.rst @@ -0,0 +1,102 @@ +Development +=========== + +We have two mechanisms for quickly setting up a development environment. `docker-compose` and `vagrant`. + +In order to to setup a development environment, it is required that you have +your FAS (Fedora Account System) certificates generated and located in your +home directory. For more information on these certificates, visit the `Koji +documentation `_. + +Docker +------ + +You can use docker containers for development. Here's a guide how to setup +`docker `_ and +`docker-compose `_ +(it's just a `dnf install` away). + +After your docker engine is set up and running, and docker-compose installed +you can start whole system with a single command:: + + $ docker-compose up + +That will start a number of services in containers, including the `frontend` +and the backend `scheduler`. You can submit a local test build with the +`submit-build.sh` script, which should submit an HTTP POST to the frontend, +requesting a build. + +You may want to wipe your local development database from time to time to try +something starting from scratch. Try the following commands, and you should +have a fresh environment:: + + $ rm module_build_service.db + $ docker-compose down -v && docker-compose up + +If things get really screwy and your container won't start properly, best thing +to do is to rebuild the environment from scratch:: + + $ docker-compose down -v + $ docker-compose build --no-cache --pull + +First command will stop and remove all containers and volumes and second +command will pull latest base image and perform a clean build without cache. + +Vagrant +------- + +Once your environment is setup, run (depending on your OS, you may need to run it with sudo):: + + $ vagrant up + +This will start module_build_service's frontend (API) and scheduler. To access the frontend, visit the following URL:: + + https://127.0.0.1:5000/module-build-service/1/module-builds/ + +At any point you may enter the guest VM with:: + + $ vagrant ssh + +To start the frontend manually, run the following inside the guest:: + + $ cd /opt/module_build_service/src + $ python manage.py runssl --debug + +To start the scheduler manually, run the following inside the guest:: + + $ cd /opt/module_build_service/src + $ python module_build_service_daemon.py + +Alternatively, you can restart the Vagrant guest, which inherently starts/restarts the frontend and the scheduler with:: + + $ vagrant reload + +Logging +------ + +If you're running module_build_service from scm then the DevConfiguration from config.py which contains LOG_LEVEL=debug should get applied. If you're having trouble just change LOG_LEVEL in BaseConfiguration. +See more about it in module_build_service/__init__.py config.from_object() + + +fedmsg Signing for Development +------------------------------ + +In order to enable fedmsg signing in development, you will need to follow a series of steps. +Note that this will conflict with signed messages from a different CA that are on the message bus, so this may cause unexpected results. + +Generate the CA, the certificate to be used by fedmsg, and the CRL with:: + + $ python manage.py gendevfedmsgcert + +Setup Apache to host the CRL:: + + $ dnf install httpd && systemctl enable httpd && systemctl start httpd + $ mkdir -p /var/www/html/crl + $ ln -s /opt/module_build_service/pki/ca.crl /var/www/html/crl/ca.crl + $ ln -s /opt/module_build_service/pki/ca.crt /var/www/html/crl/ca.crt + +Create a directory to house the fedmsg cache:: + + $ mkdir -p /etc/pki/fedmsg + +Then uncomment the fedmsg signing configuration in fedmsg.d/module_build_service.py. diff --git a/README.rst b/README.rst index bc6e008a..64cc124e 100644 --- a/README.rst +++ b/README.rst @@ -14,6 +14,11 @@ tasks: - Emitting bus messages about all state changes so that other infrastructure services can pick up the work. +Development +=========== + +For help on setting up a development environment, see `HACKING.rst`. + Client-side API =============== @@ -378,86 +383,3 @@ E.g. ``"scmurl": "git://pkgs.stg.fedoraproject.org/modules/testmodule.git?#020ea The toplevel directory containing the trees for each architecture of a module. This field is only present when a module finished building, i.e. with the states 'done' or 'ready'. - - -Development -=========== -``Logging`` ----------- - -If you're running module_build_service from scm then the DevConfiguration from config.py which contains LOG_LEVEL=debug should get applied. If you're having trouble just change LOG_LEVEL in BaseConfiguration. -See more about it in module_build_service/__init__.py config.from_object() - - -``Docker`` ----------- - -You can use docker containers for development. Here's a guide how to setup `docker `_ and `docker-compose `_. - -After your docker engine is set up and running, and docker-compose installed you can start whole system with a single command:: - - $ sudo docker-compose up - -It may happen that you will run into issues and the container won't start properly. Best thing to do in that case is to rebuild the environment from scratch:: - - $ sudo docker-compose down -v - $ sudo docker-compose build --no-cache --pull - -First command will stop and remove all containers and volumes and second command will pull latest base image and perform a clean build without cache. - - -``Vagrant`` ------------ - -In order to to setup a development environment using Vagrant, it is required that you have your FAS (Fedora Account System) certificates generated and located in your home directory. -For more information on these certificates, visit the `Koji documentation `_. - -Once your environment is setup, run (depending on your OS, you may need to run it with sudo):: - - $ vagrant up - -This will start module_build_service's frontend (API) and scheduler. To access the frontend, visit the following URL:: - - https://127.0.0.1:5000/module-build-service/1/module-builds/ - -At any point you may enter the guest VM with:: - - $ vagrant ssh - -To start the frontend manually, run the following inside the guest:: - - $ cd /opt/module_build_service/src - $ python manage.py runssl --debug - -To start the scheduler manually, run the following inside the guest:: - - $ cd /opt/module_build_service/src - $ python module_build_service_daemon.py - -Alternatively, you can restart the Vagrant guest, which inherently starts/restarts the frontend and the scheduler with:: - - $ vagrant reload - - -``fedmsg Signing for Development`` ----------------------------------- - -In order to enable fedmsg signing in development, you will need to follow a series of steps. -Note that this will conflict with signed messages from a different CA that are on the message bus, so this may cause unexpected results. - -Generate the CA, the certificate to be used by fedmsg, and the CRL with:: - - $ python manage.py gendevfedmsgcert - -Setup Apache to host the CRL:: - - $ dnf install httpd && systemctl enable httpd && systemctl start httpd - $ mkdir -p /var/www/html/crl - $ ln -s /opt/module_build_service/pki/ca.crl /var/www/html/crl/ca.crl - $ ln -s /opt/module_build_service/pki/ca.crt /var/www/html/crl/ca.crt - -Create a directory to house the fedmsg cache:: - - $ mkdir -p /etc/pki/fedmsg - -Then uncomment the fedmsg signing configuration in fedmsg.d/module_build_service.py. From 61a260d0209691635d88a5e603967e88a139952b Mon Sep 17 00:00:00 2001 From: Ralph Bean Date: Fri, 28 Oct 2016 11:51:39 -0400 Subject: [PATCH 2/2] Adjust CONTRIBUTING.rst doc based on PR feedback. --- HACKING.rst => CONTRIBUTING.rst | 25 +++++++++++++------------ 1 file changed, 13 insertions(+), 12 deletions(-) rename HACKING.rst => CONTRIBUTING.rst (75%) diff --git a/HACKING.rst b/CONTRIBUTING.rst similarity index 75% rename from HACKING.rst rename to CONTRIBUTING.rst index d20850e0..95976384 100644 --- a/HACKING.rst +++ b/CONTRIBUTING.rst @@ -11,13 +11,14 @@ documentation `_ and `docker-compose `_ -(it's just a `dnf install` away). +for Fedora users (it's just a `dnf install` away). Mac users should see `these +docs `_. -After your docker engine is set up and running, and docker-compose installed -you can start whole system with a single command:: +After your docker engine is set up and running and docker-compose is installed, +you can start the entire development environment with a single command:: $ docker-compose up @@ -26,21 +27,20 @@ and the backend `scheduler`. You can submit a local test build with the `submit-build.sh` script, which should submit an HTTP POST to the frontend, requesting a build. -You may want to wipe your local development database from time to time to try -something starting from scratch. Try the following commands, and you should -have a fresh environment:: +You may want to wipe your local development database from time to time. Try the +following commands, and you should have a fresh environment:: $ rm module_build_service.db $ docker-compose down -v && docker-compose up -If things get really screwy and your container won't start properly, best thing +If things get really screwy and your containers won't start properly, the best thing to do is to rebuild the environment from scratch:: $ docker-compose down -v $ docker-compose build --no-cache --pull -First command will stop and remove all containers and volumes and second -command will pull latest base image and perform a clean build without cache. +The first command will stop and remove all your containers and volumes and the second +command will pull the latest base image and perform a clean build without using the cache. Vagrant ------- @@ -74,8 +74,9 @@ Alternatively, you can restart the Vagrant guest, which inherently starts/restar Logging ------ -If you're running module_build_service from scm then the DevConfiguration from config.py which contains LOG_LEVEL=debug should get applied. If you're having trouble just change LOG_LEVEL in BaseConfiguration. -See more about it in module_build_service/__init__.py config.from_object() +If you're running module_build_service from scm, then the DevConfiguration from +`config.py` which contains `LOG_LEVEL=debug` should get applied. See more about +it in `module_build_service/__init__.py`, `config.from_object()`. fedmsg Signing for Development