Commit Graph

45 Commits

Author SHA1 Message Date
Matt Prahl
2e9e31eb56 Fix the 'process_waiting_module_builds' function in the scheduler 2016-12-15 16:27:16 -05:00
Ralph Bean
4b39eda44f Move the moksha.hub.main code out into its own main function again. 2016-12-15 16:27:16 -05:00
Ralph Bean
feb1079fba Remove unused work_queue. 2016-12-15 16:27:15 -05:00
Ralph Bean
250b9510bc Have the producer interact with the consumer queue appropriately. 2016-12-15 16:27:15 -05:00
Ralph Bean
f11caf32e5 Get the basic framework for local module builds working again with hub/consumer. 2016-12-15 16:27:15 -05:00
Matt Prahl
261549c8c2 Migrate scheduler to be run by fedmsg-hub 2016-12-15 16:27:15 -05:00
Filip Valder
e0240edee6 fix #264: failed 'failed module' handling due to missing koji tag 2016-12-15 16:16:24 -05:00
Ralph Bean
fcdf272eaa Merge #260 Cleanup in preparation for package review. 2016-12-12 20:41:27 +00:00
Ralph Bean
d13c7991ed Cleanup in preparation for package review.
I'm taking on #235.  This is all stuff that came up while doing a
cursory package review myself.

- The source files all state that they are MIT, but the specfile and
  repo claimed to be GPL.  I switched everything to MIT (even though I
  favor GPL.. the source was MIT first before it was incorrectly
  labelled GPL).
- There are various bugs and lint issues in the specfile fixed here.
- Notably, the systemd service file is missing from the 1.0.0 tarball on
  PyPI, so I'll need to do a new release after this.
2016-12-12 15:09:58 -05:00
Jan Kaluza
d06c13d973 Fix the bug when module build freezes when all the components in batch have been already built by builder. 2016-12-12 21:03:44 +01:00
Filip Valder
41f9830808 add docstring and Ctrl+C instruction 2016-12-12 11:43:29 +01:00
Filip Valder
60ec4552b9 prepare graceful_stop method for proper shutdown; support for intermediate Ctrl+C 2016-12-12 11:43:29 +01:00
Filip Valder
5d8a728d89 fix #211: stay attached to the main thread to keep control (foreground mode) 2016-12-12 11:43:29 +01:00
Filip Valder
adbdc29545 tidy up 2016-12-12 11:43:29 +01:00
Jakub Kadlčík
7c8349e50c Call finalize method on builder 2016-12-01 02:16:56 +01:00
Jakub Kadlčík
f9220a4470 Pretend that build was successful so that rida submits next batch 2016-12-01 02:16:56 +01:00
Jan Kaluza
1e4e1be91e Fix #105 - Add new cancel API call which can be used to cancel the module build. 2016-11-30 12:20:50 +01:00
Ralph Bean
27484f2c32 Merge #225 Fix #223, #224 - Use generated Mock config, define local RPM repository there and make it available in Mock chroot after batch is built. 2016-11-28 16:39:21 +00:00
Jan Kaluza
dd3eb61b53 Fix #223, #224 - Use generated Mock config, define local RPM repository there and make it available in Mock chroot after batch is built. 2016-11-25 12:15:32 +01:00
Jan Kaluza
77366b94ce Try to build testmodule.yaml as unit-test with fake builder backend. 2016-11-24 14:01:28 +01:00
Jan Kaluza
c719e1d0c1 Fix the buildroot_connect calls in repos and models and set 'system' to 'mock' when invoked from manage.py using build_module_locally 2016-11-22 10:54:38 +01:00
Ralph Bean
439721da99 Gather build groups from module profiles.
Fixes #181.

When initializing the buildroot for a module build, we used to set up
some build 'groups' for the tag: `build` and `srpm-build`.  These are
the lists of RPMs that koji is supposed to install into the buildroot
before anything else is done.  Crucial stuff goes here, like `git` in
the `srpm-build` group so that koji can clone the repo in the first
place.

We had those lists hardcoded before.  This list changes that to use the
`buildroot` and `srpm-buildroot` profiles of the modules which are our
dependencies (recursively).

This will allow people like @psabata and the base-runtime to make
changes to the build groups for the generational core and work around
their own problems, instead of having to ask us to expand that list.

There were a couple ways to do this:

- I could've cloned the SCM repos for all dependencies and gotten their
  profiles from the modulemd source there.  This seemed flimsy because
  we only want to depend on the profiles of modules that were *really*
  built.
- We could modify PDC to stuff the modulemd contents in there.  We
  already get some dep and tag info from PDC.  My thought here was that
  it would be too heavyweight to store every copy of the modulemd file
  in PDC for every build ever.  We already have it in MBS.
- Lastly, and this is what I did here, I just referred to MBS' own
  database to get the profiles.  This seems to work just fine.

One side-effect is that we need the build profiles from the manually
bootstrapped modules that were put together by hand, and were never
built in the MBS.  In order to work around that, I added an alembic
upgrade script which pre-populates the database with one fake
bootstrapped base-runtime module.  We can expand on that in the future
if we need.
2016-11-21 13:25:52 -05:00
Ralph Bean
e278ffa5a4 build.buildroot_task_id is not a thing.
... and it never was, as far as I can tell from the gitlog.
2016-11-21 13:25:52 -05:00
Ralph Bean
bf9e854046 Store the reason for failure. 2016-11-17 16:14:22 -05:00
Matt Prahl
244b77f54c Add component build throttling 2016-11-16 15:24:17 -05:00
Matt Prahl
67d07023c5 Rename function to start_build_batch for further flexibility in the future 2016-11-16 15:24:16 -05:00
Matt Prahl
62654f871f Fixes bug where the build tag was not returned 2016-11-16 15:24:16 -05:00
Matt Prahl
28d684331f Remove stale code from previous modulemd version 2016-11-16 15:24:16 -05:00
Jan Kaluza
2b7ebdfc6f Fix #172 - Set ComponentBuild.batch according to mmd.buildorder attribute 2016-11-16 12:41:06 +01:00
Ralph Bean
f9b2821f7f Some better logging around PDC stuff.
I spent a while debugging these interactions tonight and the log
statements definitely helped to figure it out.
2016-11-14 19:33:25 -05:00
Jan Kaluza
966a73251a Add support for modulemd-1.0 backward-incompatible changes like commit being replaced by ref 2016-11-14 15:32:51 +01:00
Jan Kaluza
7b9e4f7234 Initial version of mock backend 2016-11-10 09:57:00 +01:00
Jan Kaluza
8339e389be Fix out-of-memory caused by loading ModuleBuild for every ComponentBuild row from database. Now we lazyload it. 2016-11-03 10:17:03 +01:00
Jan Kaluza
d9916a1634 Fix #126 - remove Builder class 2016-11-01 14:44:42 +01:00
Ralph Bean
7ff97cbe43 Merge #147 Pass around more info on pre-built components 2016-10-31 15:14:38 +00:00
Matt Prahl
fdbdd6a2c2 Merge #145 Allow handlers to return extra work to the scheduler. 2016-10-31 13:25:06 +00:00
Ralph Bean
c2101df92b Add comment explaining what's going on here. 2016-10-31 09:06:43 -04:00
Jan Kaluza
6fd16587bc Ignore repository builds without the '-build' suffix. 2016-10-31 13:14:31 +01:00
Ralph Bean
99b0993fe0 Pass around more info on pre-built components so we can resume module builds. 2016-10-29 16:12:24 -04:00
Ralph Bean
b3ae06f824 Use the local session, not the Flask-managed one. 2016-10-29 16:00:11 -04:00
Ralph Bean
fdb735de44 Allow handlers to return extra work to the scheduler. 2016-10-29 15:55:06 -04:00
Courtney Pacheco
dbe26fcfb5 Adding code to handle componentless builds 2016-10-25 03:29:58 -04:00
Jakub Kadlčík
de796c2b73 Use Builder wrapper instead of KojiModuleBuilder 2016-10-24 20:46:07 +02:00
Jakub Kadlčík
57b0c8b86c Add CoprModuleBuilder skeleton 2016-10-24 20:46:07 +02:00
Matt Prahl
b4082dc551 Rename module from rida to module_build_service
Rename routes from /rida/1/module-builds/ to /module-build-service/1/module-builds/
2016-10-24 10:30:23 -04:00