46 Commits

Author SHA1 Message Date
Brendan Reilly
d4c213d678 Resolve issues reported by bandit 2022-07-21 09:09:12 -04:00
Chenxiong Qi
1c0715662a Cleanup module_build_service/__init__.py
Most of the code are moved to dedicated subpackages, but some others
can't due to the cycle dependencies.

Signed-off-by: Chenxiong Qi <cqi@redhat.com>
2020-03-03 14:48:48 -05:00
Chenxiong Qi
81b36b1d6e Add indexes to ModuleBuild and ComponentBuild
Tests are updated accordingly as well.

Signed-off-by: Chenxiong Qi <cqi@redhat.com>
2020-03-03 14:48:47 -05:00
mprahl
9ddb35b8d3 Move models.py to common/models.py and views.py to web/views.py 2020-03-03 14:48:47 -05:00
mprahl
540e2ceac3 Fix a fork in the database migration scripts 2019-10-03 16:29:47 -04:00
Valerij Maljulin
689d949a48 buildonly support
This fixes #1307

Signed-off-by: Valerij Maljulin <vmaljuli@redhat.com>
2019-10-03 16:20:04 +02:00
Jan Kaluza
fc39df9487 Store the build_context without base modules in a database.
For KojiResolver, we need to be able to find out the module to reuse
components from based only on the non-base-module buildrequires. The
base module buildrequirement will be defined by the Koji tag inheritance.

For this, we need to store build_context computed without the base
modules.
2019-10-02 07:16:55 +02:00
mprahl
a6bf9f88dd Allow components to be reused from module builds even if the buildrequires commit hashes changed for the changed-and-after rebuild strategy
This behavior was not documented, and it was confusing to users since module builds
in a stream should always have a compatible API.
2019-08-05 09:23:44 -04:00
mprahl
84e3f6f437 Add a missing foreign key constraint that was missed in 40b2c7d988d7_add_reused_module_id_column.py 2019-08-02 09:33:16 -04:00
mprahl
971aade159 Add SQLite support for the downgrade function in 40b2c7d988d7_add_reused_module_id_column.py 2019-08-02 09:32:41 -04:00
mprahl
f13b7308e1 Add a missing column in 0b00036c540f_add_log_messages_table.py 2019-08-02 09:32:41 -04:00
mprahl
4f3692c711 Fix a flake8 error in 0b00036c540f_add_log_messages_table.py 2019-08-02 09:04:45 -04:00
Martin Curlej
c5d484fb81 Added an REST endpoint to display log messages
The issue is that users don't get feedback from MBS about why a
component was not reused. There was added logic which enables to
store log messages in the database and can be viewed through the
REST api of MBS.

Ticket-ID: #1284

Signed-off-by: Martin Curlej <mcurlej@redhat.com>
2019-07-31 13:32:33 +02:00
Martin Curlej
ed24ca870a Module builds now remember what module they reused for building.
There was a race condition, when 2 builds where build in the same time.
There was an issue that after one has finished the other reused the new build
for reuse and not the one it started with.

Ticket-ID: FACTORY-3862

Signed-off-by: Martin Curlej <mcurlej@redhat.com>
2019-06-25 11:09:33 +02:00
Jan Kaluza
bf0bcaff57 Take the list of arches for -build Koji tag from buildrequired modules.
Currently, we are using just `conf.arches` and `conf.base_module_arches`
to define the list of arches for which the RPMs in a submitted module are
built. This is not enough, because RCM needs to generate modules based
on the base modules which should use different arches.

This commit changes the MBS to take the list of arches from the buildrequired
module build. It checks the buildrequires for "privileged" module or base
module and if it finds such module, it queries the Koji to find out the list
of arches to set for the module.

The "privileged" module is a module which can override base module arches
or disttag. Previously, these modules have been defined by
`allowed_disttag_marking_module_names` config option. In this commit,
this has been renamed to `allowed_privileged_module_names`.

The list of arches are stored per module build in new table represented
by ModuleArch class and are m:n mapped to ModuleBuild.
2019-06-07 13:16:31 +02:00
mprahl
84dc82c4a0 Add a migration that adds a missing uniqueness constraint that was defined in the model
The uniqueness constraint was added in 599c881714, but it
was not part of the migration.
2019-05-15 11:14:05 -04:00
mprahl
14098cea08 Migrate to libmodulemd v2
This also moves the methods load_mmd and load_mmd_file to
module_build_service.utils.general.

This also removes some MSE unit tests with a mix of positive and
negative streams since this is not supported in libmodulemd v2. The
user will be presented with a syntax error if they try to submit
such a modulemd file.
2019-05-13 13:40:37 -04:00
mprahl
00daedccfd Allow the virtual streams of a base module to be queryable in the database and API 2019-04-25 13:15:27 -04:00
Chenxiong Qi
7c993f9165 Handle streams in base module stream
A base module's stream (the platform for RHEL) could have Z-stream suffix, e.g.
el8.0.0.z, this patch handles this Z-stream suffix and other potential streams
by returning the stream version as a float with configured suffix value. For
example, el8.1.0.z would be parsed as 080100.1. Note that, the 0.1 is totally
configured in config and it actually could be any value according to concrete
cases in practice.

Config STREAM_SUFFIXES is enabled in TestConfiguration so that tests depending
on the return value from ModuleBuild.get_stream_version are covered.

Part fixture of test TestMMDResolver.test_solve_virtual_streams is updated by
adding Z-stream suffix to platform:el8.2.0 in order to ensure this patch does
not break the MMD resolver.

Addresses FACTORY-4307

Signed-off-by: Chenxiong Qi <cqi@redhat.com>
2019-04-23 11:21:46 +08:00
Merlin Mathesius
152419f376 Module scratch build fixups per PR review feedback:
- Keep scratch module builds in the 'done' state.
- Make koji tagging for scratch modules unique so the same
  commit can be resubmitted.
- Use alternate prefix for scratch module build components so they can
  be identified later.
- Prevent scratch build components from being reused.
- Assorted code and comment cleanup.

Signed-off-by: Merlin Mathesius <mmathesi@redhat.com>
2019-03-01 10:27:04 -06:00
Merlin Mathesius
10e9dace9c Add database migration handling of new module_builds columns for module scratch builds.
Signed-off-by: Merlin Mathesius <mmathesi@redhat.com>
2019-03-01 10:27:04 -06:00
mprahl
ba3bef9f46 Don't assume the context is set on base modules in the 526fb7d445f7_module_buildrequires migration 2018-11-01 09:38:02 -04:00
mprahl
26547f707a Fix typo in the migration script 2018-10-30 12:16:56 -04:00
mprahl
599c881714 Add the ability to query by the base modules a module build buildrequires
Future use cases will require the ability to find compatible module builds
to buildrequire based on the base module the module used to build. This
commit adds an association table that will contain module build IDs
and the base module they buildrequire.

Addresses FACTORY-3353
2018-10-17 07:47:31 -04:00
mprahl
a78c564073 Revert "Store the component's build ID and use that to identify the build when acting on a tag message"
This reverts commit 9bd16beeef.
2018-08-07 10:32:50 -04:00
mprahl
9bd16beeef Store the component's build ID and use that to identify the build when acting on a tag message
This resolves an issue where the component name is different than the
package being tagged due to a macro such as those used for SCLs.
2018-08-03 20:45:00 -04:00
mprahl
8c83959f58 Fix the COPR cleanup database migration for SQLite 2018-08-03 11:28:11 -04:00
mprahl
36a9026ca8 Remove references to COPR 2018-06-28 10:00:06 -04:00
mprahl
5930dd0b3f Set the modulemd's context as the context in the DB to correct the current values 2018-05-29 10:06:53 -04:00
Qixiang Wan
d83e6897ca Change ModuleBuild.context to db column
1. Changed ModuleBuild's context property to db column, it's
non-nullable and default value is '00000000' to keep it consistent
with previous behaviour.

2. Changed ModuleBuild.contexts_from_mmd to return a tuple of
(ref_build_context, build_context, runtime_context, context).

3. Updated tests affected by this change.
2018-04-28 11:46:31 +08:00
Jan Kaluza
8d1d439737 Generate 'context' from hash based on buildrequires/requires stream. Reuse components only from module with the same stream_build_context. 2018-04-24 15:45:26 +02:00
Jan Kaluza
e5e9a24fce Allow adding bool to XMD. Fix flake8 issues. 2018-04-03 09:58:57 -04:00
mprahl
f0852d9009 Port to libmodulemd and support v2 modulemd without module stream expansion 2018-04-03 09:58:57 -04:00
mprahl
a67b53f5c3 Add a "context" field on component and module releases in Koji for uniqueness for when Module Stream Expansion is implemented 2018-01-16 10:36:09 -05:00
Jan Kaluza
7de120c17b Get the weight of Koji tasks and use it to sort the components in a batch instead of builds time. 2017-11-28 10:52:17 -05:00
mprahl
45cce653eb Add the ability to specify different rebuild methods 2017-11-01 15:39:43 -04:00
Jan Kaluza
ad6874b0e6 Fix #670 - Tag Content Generator Koji build to special tag based on the base module stream 2017-09-25 08:36:29 +02:00
Jan Kaluza
999baa2992 Wait for components to be tagged also in final tag before marking module as done. 2017-09-15 15:25:29 +02:00
Jan Kaluza
8bbe2d359f Trigger newRepo by MBS instead of waiting on Kojira in case when we have sucessfully tagged all the components 2017-04-06 17:19:35 +02:00
Filip Valder
13b7bcd03e Support for state tracing of modules and components
- DB migration scripts.
- Models: ComponentBuildTrace, ModuleBuildTrace.
- ModuleBuild.state_trace method for querying for a particular module's
state history.
- SQLAlchemy before commit session event handler for recording
module/component state changes.
- REST API verbose mode for getting state trace of a particular module.
- Tests use make_session, so that event handlers are in effect.
- Short info in README about verbose mode.
- Tests verifying whether state trace information about a module appears
in verbose mode.
- Other minor fixes (RidaBase -> MBSBase, PEP8...)
2017-02-28 16:15:43 +01:00
Jakub Kadlčík
5c52fce586 Update to the accurate down_revision 2017-02-23 20:55:51 +01:00
Jakub Kadlčík
92dce63091 Allow submitting optional parameters such as copr_owner and copr_project 2017-02-23 20:51:31 +01:00
Matt Prahl
385045d778 Reuse components from previous module builds 2017-02-21 14:22:30 -05:00
Ralph Bean
2a60d1ff20 Also, alembic. 2017-02-15 12:30:31 -05:00
Ralph Bean
d13c7991ed Cleanup in preparation for package review.
I'm taking on #235.  This is all stuff that came up while doing a
cursory package review myself.

- The source files all state that they are MIT, but the specfile and
  repo claimed to be GPL.  I switched everything to MIT (even though I
  favor GPL.. the source was MIT first before it was incorrectly
  labelled GPL).
- There are various bugs and lint issues in the specfile fixed here.
- Notably, the systemd service file is missing from the 1.0.0 tarball on
  PyPI, so I'll need to do a new release after this.
2016-12-12 15:09:58 -05:00
Filip Valder
7c4443fd3e this shall be accessible for module_build_service's product users in order to be able to upgrade underlying DB 2016-12-05 15:16:41 +01:00