Commit Graph

60 Commits

Author SHA1 Message Date
Ralph Bean
6eaf8d7863 Demote this log statement. Way too noisy in production. 2017-02-15 15:43:38 -05:00
Ralph Bean
d093c5eef3 Default needed for this to make sense. 2017-02-02 03:30:25 -05:00
Ralph Bean
e082ff1289 Careful with msg_id here too. 2017-02-01 11:07:21 -05:00
Ralph Bean
78bd24b62a Be careful with attribute access here.
I hit a traceback where msg was NoneType and therefore didn't have a
`msg_id` attribute, I therefore wasn't able to see the real exception
that got me there in the first place.
2017-01-31 12:37:50 -05:00
Filip Valder
abf37e98e1 in_memory messaging -> fallback to '*' topics 2017-01-12 19:57:34 +01:00
Filip Valder
34ef9ce5fd Add support for message topic prefixes + explicit services/categories (incl. code lints) 2017-01-12 19:30:34 +01:00
Jan Kaluza
dda47acab7 Fix #276 - Use PDC to resolve buildroot components instead of MBS database 2017-01-10 14:29:54 +01:00
Jan Kaluza
23802418af Fix tag_artifacts with concurrent build threshold, do not send repo-done in Mock backend after each build, but just after the tag_artifacts call. 2017-01-06 11:04:54 +01:00
Jan Kaluza
ab567803e8 Fix #41 - Tag builds only in the end of the badge to not regen repository all the time. 2017-01-05 15:24:46 +01:00
Jan Kaluza
174e657379 MBSConsumer: Handle initial messages in right order, override validate for non-fedmsg messaging backends. 2016-12-19 12:47:57 +01:00
Ralph Bean
3bf952cda6 Tell moksha to just listen to /dev/null for local builds. 2016-12-15 16:27:17 -05:00
Ralph Bean
775819e87c Bugfix to shutdown and restart for the test suite. 2016-12-15 16:27:17 -05:00
Ralph Bean
9d3b85426d Bring back the in-memory messaging backend. 2016-12-15 16:27:17 -05:00
Ralph Bean
21a902ab5e Some debugging on the stop_condition. 2016-12-15 16:27:16 -05:00
Ralph Bean
b2cf02080d A few bugfixes. 2016-12-15 16:27:16 -05:00
Matt Prahl
2e9e31eb56 Fix the 'process_waiting_module_builds' function in the scheduler 2016-12-15 16:27:16 -05:00
Ralph Bean
4b39eda44f Move the moksha.hub.main code out into its own main function again. 2016-12-15 16:27:16 -05:00
Ralph Bean
feb1079fba Remove unused work_queue. 2016-12-15 16:27:15 -05:00
Ralph Bean
250b9510bc Have the producer interact with the consumer queue appropriately. 2016-12-15 16:27:15 -05:00
Ralph Bean
f11caf32e5 Get the basic framework for local module builds working again with hub/consumer. 2016-12-15 16:27:15 -05:00
Matt Prahl
261549c8c2 Migrate scheduler to be run by fedmsg-hub 2016-12-15 16:27:15 -05:00
Filip Valder
e0240edee6 fix #264: failed 'failed module' handling due to missing koji tag 2016-12-15 16:16:24 -05:00
Ralph Bean
fcdf272eaa Merge #260 Cleanup in preparation for package review. 2016-12-12 20:41:27 +00:00
Ralph Bean
d13c7991ed Cleanup in preparation for package review.
I'm taking on #235.  This is all stuff that came up while doing a
cursory package review myself.

- The source files all state that they are MIT, but the specfile and
  repo claimed to be GPL.  I switched everything to MIT (even though I
  favor GPL.. the source was MIT first before it was incorrectly
  labelled GPL).
- There are various bugs and lint issues in the specfile fixed here.
- Notably, the systemd service file is missing from the 1.0.0 tarball on
  PyPI, so I'll need to do a new release after this.
2016-12-12 15:09:58 -05:00
Jan Kaluza
d06c13d973 Fix the bug when module build freezes when all the components in batch have been already built by builder. 2016-12-12 21:03:44 +01:00
Filip Valder
41f9830808 add docstring and Ctrl+C instruction 2016-12-12 11:43:29 +01:00
Filip Valder
60ec4552b9 prepare graceful_stop method for proper shutdown; support for intermediate Ctrl+C 2016-12-12 11:43:29 +01:00
Filip Valder
5d8a728d89 fix #211: stay attached to the main thread to keep control (foreground mode) 2016-12-12 11:43:29 +01:00
Filip Valder
adbdc29545 tidy up 2016-12-12 11:43:29 +01:00
Jakub Kadlčík
7c8349e50c Call finalize method on builder 2016-12-01 02:16:56 +01:00
Jakub Kadlčík
f9220a4470 Pretend that build was successful so that rida submits next batch 2016-12-01 02:16:56 +01:00
Jan Kaluza
1e4e1be91e Fix #105 - Add new cancel API call which can be used to cancel the module build. 2016-11-30 12:20:50 +01:00
Ralph Bean
27484f2c32 Merge #225 Fix #223, #224 - Use generated Mock config, define local RPM repository there and make it available in Mock chroot after batch is built. 2016-11-28 16:39:21 +00:00
Jan Kaluza
dd3eb61b53 Fix #223, #224 - Use generated Mock config, define local RPM repository there and make it available in Mock chroot after batch is built. 2016-11-25 12:15:32 +01:00
Jan Kaluza
77366b94ce Try to build testmodule.yaml as unit-test with fake builder backend. 2016-11-24 14:01:28 +01:00
Jan Kaluza
c719e1d0c1 Fix the buildroot_connect calls in repos and models and set 'system' to 'mock' when invoked from manage.py using build_module_locally 2016-11-22 10:54:38 +01:00
Ralph Bean
439721da99 Gather build groups from module profiles.
Fixes #181.

When initializing the buildroot for a module build, we used to set up
some build 'groups' for the tag: `build` and `srpm-build`.  These are
the lists of RPMs that koji is supposed to install into the buildroot
before anything else is done.  Crucial stuff goes here, like `git` in
the `srpm-build` group so that koji can clone the repo in the first
place.

We had those lists hardcoded before.  This list changes that to use the
`buildroot` and `srpm-buildroot` profiles of the modules which are our
dependencies (recursively).

This will allow people like @psabata and the base-runtime to make
changes to the build groups for the generational core and work around
their own problems, instead of having to ask us to expand that list.

There were a couple ways to do this:

- I could've cloned the SCM repos for all dependencies and gotten their
  profiles from the modulemd source there.  This seemed flimsy because
  we only want to depend on the profiles of modules that were *really*
  built.
- We could modify PDC to stuff the modulemd contents in there.  We
  already get some dep and tag info from PDC.  My thought here was that
  it would be too heavyweight to store every copy of the modulemd file
  in PDC for every build ever.  We already have it in MBS.
- Lastly, and this is what I did here, I just referred to MBS' own
  database to get the profiles.  This seems to work just fine.

One side-effect is that we need the build profiles from the manually
bootstrapped modules that were put together by hand, and were never
built in the MBS.  In order to work around that, I added an alembic
upgrade script which pre-populates the database with one fake
bootstrapped base-runtime module.  We can expand on that in the future
if we need.
2016-11-21 13:25:52 -05:00
Ralph Bean
e278ffa5a4 build.buildroot_task_id is not a thing.
... and it never was, as far as I can tell from the gitlog.
2016-11-21 13:25:52 -05:00
Ralph Bean
bf9e854046 Store the reason for failure. 2016-11-17 16:14:22 -05:00
Matt Prahl
244b77f54c Add component build throttling 2016-11-16 15:24:17 -05:00
Matt Prahl
67d07023c5 Rename function to start_build_batch for further flexibility in the future 2016-11-16 15:24:16 -05:00
Matt Prahl
62654f871f Fixes bug where the build tag was not returned 2016-11-16 15:24:16 -05:00
Matt Prahl
28d684331f Remove stale code from previous modulemd version 2016-11-16 15:24:16 -05:00
Jan Kaluza
2b7ebdfc6f Fix #172 - Set ComponentBuild.batch according to mmd.buildorder attribute 2016-11-16 12:41:06 +01:00
Ralph Bean
f9b2821f7f Some better logging around PDC stuff.
I spent a while debugging these interactions tonight and the log
statements definitely helped to figure it out.
2016-11-14 19:33:25 -05:00
Jan Kaluza
966a73251a Add support for modulemd-1.0 backward-incompatible changes like commit being replaced by ref 2016-11-14 15:32:51 +01:00
Jan Kaluza
7b9e4f7234 Initial version of mock backend 2016-11-10 09:57:00 +01:00
Jan Kaluza
8339e389be Fix out-of-memory caused by loading ModuleBuild for every ComponentBuild row from database. Now we lazyload it. 2016-11-03 10:17:03 +01:00
Jan Kaluza
d9916a1634 Fix #126 - remove Builder class 2016-11-01 14:44:42 +01:00
Ralph Bean
7ff97cbe43 Merge #147 Pass around more info on pre-built components 2016-10-31 15:14:38 +00:00