Commit Graph

1436 Commits

Author SHA1 Message Date
Martin Curlej
c139e4cb60 Added the ability to build modules with static contexts
This is a fix and feature in one. This patch enables to use static
contexts instead of module stream expansion. This fixes the issue with
upgrade path issue in dnf.

Signed-off-by: Martin Curlej <mcurlej@redhat.com>
2020-09-03 09:22:08 +02:00
Brendan Reilly
5c13368f77 Only get latest build of latest stream when determining requirements 2020-07-24 11:43:07 -04:00
jobrauer
8a5bf3a579 Move ComponentBuildTrace creation from before_commit to before_flush
Since ComponentBuildTrace(s) get created with db_session.commit() call,
is is not possible to commit more items in bulk if they already have been flushed.

Current unit-tests' setup can be significantly sped up if items can be quickly
flushed on the fly and bulk-commited only once at the end. Moreover in general it
seems more appropriate/safer to handle this in before_flush as any implicit
or accidental flush could cause new build traces not to be created at all. As flush
is implicitly called before every commit anyway, this change shouldn't pose any harm.
2020-07-13 05:55:45 +00:00
Brendan Reilly
169e5aa3b5 Do not login when getting final mmds 2020-06-23 14:09:52 -04:00
Brendan Reilly
db70560d2d Add API call to get final modulemds of builds 2020-06-22 11:31:05 -04:00
Brendan Reilly
e68f5c4c26 Use repo id instead of latest symlink
Fixes https://pagure.io/fm-orchestrator/issue/1581
2020-04-02 12:12:13 -04:00
Brendan Reilly
25cdfbbd22 Add the ability to limit arches 2020-04-01 17:02:51 +00:00
mprahl
244f29f25f Make record_module_build_arches idempotent
This function could get called multiple times if the init handler
runs more than once. This can happen if the build failed in the
init handler due to external infrastructure being down and the
user resumes their build.
2020-03-23 20:22:57 +00:00
mprahl
fee1dff8d0 Honor the MBS_CONFIG_SECTION environment variable when performing a local build 2020-03-17 17:08:27 -04:00
Merlin Mathesius
da51bebcd9 New 'module_stream' optional parameter
- Implement new optional parameter module_stream to allow a scratch module
  build's stream name to be set from the command line when also submitting a
  YAML modulemd file.
- Validate that module_name and module_stream parameters can only be specified
  along with a YAML modulemd file.
- Add tests to verify that module_stream sets the stream name correctly.
- Add tests to verify that module_name and module_stream are only allowed along
  with a YAML modulemd file.

Signed-off-by: Merlin Mathesius <mmathesi@redhat.com>
2020-03-10 14:08:29 +00:00
Merlin Mathesius
de2a776226 Skip git ref checks for rpm components with srpm overrides
Signed-off-by: Merlin Mathesius <mmathesi@redhat.com>
2020-03-10 14:02:11 +00:00
Mike McLean
6c9f61e465 Merge #1577 Check schedule date when adding new streams 2020-03-06 16:56:17 +00:00
Brendan Reilly
88597a7c92 Check schedule date when adding new streams 2020-03-05 16:28:42 -05:00
mprahl
57d02ac3ba Make the code that ignores messages consistent 2020-03-04 16:11:25 -05:00
mprahl
757191fded Bubble up IgnoreMessage exceptions in get_abstracted_event_info
This allows the MBSConsumer.consume method to log the message and return
without calling the validate_event method.
2020-03-04 16:11:25 -05:00
mprahl
d2b3eace3d Handle when a message parser returns None
mbs-messaging-umb is such a parser that returns None if a message
does not match the schema it is validating against.
2020-03-04 16:11:25 -05:00
Chenxiong Qi
1c0715662a Cleanup module_build_service/__init__.py
Most of the code are moved to dedicated subpackages, but some others
can't due to the cycle dependencies.

Signed-off-by: Chenxiong Qi <cqi@redhat.com>
2020-03-03 14:48:48 -05:00
Chenxiong Qi
1e5ed864c9 Avoid to handle a late init message
Signed-off-by: Chenxiong Qi <cqi@redhat.com>
2020-03-03 14:48:47 -05:00
Chenxiong Qi
81b36b1d6e Add indexes to ModuleBuild and ComponentBuild
Tests are updated accordingly as well.

Signed-off-by: Chenxiong Qi <cqi@redhat.com>
2020-03-03 14:48:47 -05:00
Chenxiong Qi
83b9e56f46 Remove workaround from handlers init and wait
Commit 98b1ac79 ensures the message is sent after data changes are
committed into database. Hence, it is doable to remove these two
workarounds.

Signed-off-by: Chenxiong Qi <cqi@redhat.com>
2020-03-03 14:48:47 -05:00
Chenxiong Qi
b70c632a70 Send module build state change message after commit to database
In MBS, there are two cases to send a message when a module build moves
to a new state. One is to create a new module build, with
ModuleBuild.create particularly, when user submit a module build.
Another one is to transition a module build to a new state with
ModuleBuild.transition. This commit handles these two cases in a little
different ways.

For the former, existing code is refactored by moving the publish call
outside ModuleBuild.create.

For the latter, message is sent in a hook of SQLAlchemy ORM event
after_commit rather than immediately inside the ModuleBuild.transition.

Both of these changes ensure the message is sent after the changes are
committed into database successfully. Then, the backend can have
confidence that the database has the module build data when receive a
message.

Signed-off-by: Chenxiong Qi <cqi@redhat.com>
2020-03-03 14:48:47 -05:00
Chenxiong Qi
d068ff06c4 Use anonymous koji session to list build RPMs
Signed-off-by: Chenxiong Qi <cqi@redhat.com>
2020-03-03 14:48:47 -05:00
mprahl
f1f92a4d25 Use an in-memory SQLite database to run the tests
This change was lost from the following PR during a rebase:
https://pagure.io/fm-orchestrator/pull-request/1511
2020-03-03 14:48:47 -05:00
mprahl
27972706f0 Remove the dependency on Flask-SQLAlchemy in DBResolver.py 2020-03-03 14:48:47 -05:00
mprahl
81f2bffdda Rearrange the imports to meet the style guide
This also includes `from __future__ import absolute_import`
in every file so that the imports are consistent in Python 2 and 3.
The Python 2 tests fail without this.
2020-03-03 14:48:47 -05:00
mprahl
c10a6769ed Move scm.py to common/scm.py 2020-03-03 14:48:47 -05:00
mprahl
4b32612002 Move route.py to scheduler/route.py 2020-03-03 14:48:47 -05:00
mprahl
c7c650a4f0 Move logger.py to common/logger.py 2020-03-03 14:48:47 -05:00
mprahl
7a42d89ba7 Move proxy.py to web/proxy.py 2020-03-03 14:48:47 -05:00
mprahl
9ddb35b8d3 Move models.py to common/models.py and views.py to web/views.py 2020-03-03 14:48:47 -05:00
mprahl
e6b1372d5d Move views.py to web/views.py 2020-03-03 14:48:47 -05:00
mprahl
d9f32bff9d Move errors.py to common/errors.py 2020-03-03 14:48:47 -05:00
mprahl
78feb5abc7 Move monitor.py to common/monitor.py 2020-03-03 14:48:47 -05:00
mprahl
f6bdaf59bf Move mmd_resolver.py to web/mmd_resolver.py 2020-03-03 14:48:47 -05:00
mprahl
18dd43926c Move messaging.py to common/messaging.py 2020-03-03 14:48:47 -05:00
mprahl
b16cb88bc3 Move db_session.py to scheduler/db_session.py 2020-03-03 14:48:47 -05:00
mprahl
98b54d211e Move config.py to common/config.py 2020-03-03 14:48:47 -05:00
mprahl
5671ae0827 Move backports.py to web/backports.py 2020-03-03 14:48:47 -05:00
mprahl
988f50a7f6 Move auth.py to web/auth.py 2020-03-03 14:48:47 -05:00
mprahl
96b0c85927 Split utils/submit.py
This moves the code used by the backend and API to common/submit.py,
the code used just by the API to web/submit.py, and the code used
just by the backend to scheduler/submit.py.
2020-03-03 14:48:47 -05:00
mprahl
85e3a2c496 Move utils/reuse.py to scheduler/reuse.py 2020-03-03 14:48:47 -05:00
mprahl
63776f3c01 Move utils/views.py to web/views.py
In the next commit, this will be merged into web/utils.py.
2020-03-03 14:48:47 -05:00
mprahl
a1dfeeb7e3 Move utils/ursine.py to scheduler/ursine.py 2020-03-03 14:48:47 -05:00
mprahl
2ce2e33431 Move utils/request_utils.py to common/request_utils.py 2020-03-03 14:48:47 -05:00
mprahl
d3d1f2d3dd Split utils/mse.py
This moves the code used by the backend and API to common/resolve.py
and moves the code used just by the API to web/mse.py.
2020-03-03 14:48:47 -05:00
mprahl
b0a27081f4 Split utils/general.py
This puts backend specific code in either the builder or scheduler
subpackage. This puts API specific code in the new web subpackage.
Lastly, any code shared between the API and backend is placed in the
common subpackage.
2020-03-03 14:48:47 -05:00
mprahl
1ab8aabd75 Remove outdated TODO comments in builder/base.py 2020-03-03 14:48:47 -05:00
mprahl
84901c9ba5 Move utils/greenwave.py to scheduler/greenwave.py 2020-03-03 14:48:47 -05:00
mprahl
81ab984b1f Move utils/batches.py to scheduler/batches.py 2020-03-03 14:48:47 -05:00
Qixiang Wan
938a3b9557 Ignore koji_build_change event without task_id 2020-03-03 14:48:47 -05:00