Commit Graph

57 Commits

Author SHA1 Message Date
Matt Prahl
8d36d97a5d Merge #1085 Show expanded buildrequires in API output 2018-12-04 16:57:12 +00:00
Chenxiong Qi
1cd64d336a Fix some minor issues
* Fix failure test after rebasing on master branch.
* Fix some grammar issues.
* Only check stream collision modules on new created module build.
* Logging message properly.

Signed-off-by: Chenxiong Qi <cqi@redhat.com>
2018-11-27 11:06:34 +08:00
Chenxiong Qi
917c06ad0c Resolve stream collision with modules added to ursine content
This resolve the stream collision by adding specific RPMs to
module-build-macros SRPM as Conflicts.

For more information about module stream collision, please refer to
docstring in utils/ursine.py

Signed-off-by: Chenxiong Qi <cqi@redhat.com>
2018-11-27 11:06:34 +08:00
Chenxiong Qi
04b5a9a2a7 Show expanded buildrequires in API output
ModuleBuild.json now includes xmd/mbs/buildrequires in the output JSON
data. As a result, these APIs will show buildrequires by default.

* /module-builds/
* /module-builds/?verbose=True
* /module-builds/$build_id

Buildrequires is accessible like this:

    br_module_names = response_json['buildrequires'].keys()
    br_module_dict = response_json['buildrequires'][module name]

Fixes: FACTORY-2201

Signed-off-by: Chenxiong Qi <cqi@redhat.com>
2018-11-23 17:40:23 +08:00
Jan Kaluza
f2a236bc74 Pass buildrequired modules built against all compatible base module streams to MMDResolver.
Imagine we have "platform:f29.0.0" and "platform:f29.1.0" base modules.
We also have "DBI" module we want to build agaisnt "platform:f29.1.0".
This "DBI" module depends on "perl" module which is only build against
"platform:f29.0.0".

Currently, DBI build would fail to resolve the dependencies, because
it wouldn't find "perl" module, because it is built against different
platform stream.

This PR changes the MSE code to include buildrequired module builds built
against all the compatible platform streams.

It does so by introducing following changes:

- MSE code uses new get_base_module_mmds() method to find out all the
  compatible platform modules. This needed new methods in DBResolver
  and MBSResolver.
- For each buildrequired module defined by name:stream, the MSE code then
  finds particular NSVC built against each compatible platform module.

Side effect of these code changes is that every module now must buildrequire
some base module.
2018-10-26 14:02:36 +02:00
Chenxiong Qi
20de3d2e94 Make _make_module reusable
TestUtilsModuleStreamExpansion._make_module is moved to
tests/__init__.py so that other tests could reuse this to create a
module build.

Signed-off-by: Chenxiong Qi <cqi@redhat.com>
2018-10-19 15:58:47 +08:00
mprahl
599c881714 Add the ability to query by the base modules a module build buildrequires
Future use cases will require the ability to find compatible module builds
to buildrequire based on the base module the module used to build. This
commit adds an association table that will contain module build IDs
and the base module they buildrequire.

Addresses FACTORY-3353
2018-10-17 07:47:31 -04:00
mprahl
5197eb4912 Revert "Use the libmodulemd API properly when setting values on existing objects"
This reverts commit 8173040ea6. Since it didn't
provide any real benefit after talking with sgallagh, it seems like we'd better
off not introduce more risk to the next deployment.
2018-08-07 12:03:44 -04:00
mprahl
a78c564073 Revert "Store the component's build ID and use that to identify the build when acting on a tag message"
This reverts commit 9bd16beeef.
2018-08-07 10:32:50 -04:00
mprahl
9bd16beeef Store the component's build ID and use that to identify the build when acting on a tag message
This resolves an issue where the component name is different than the
package being tagged due to a macro such as those used for SCLs.
2018-08-03 20:45:00 -04:00
mprahl
8173040ea6 Use the libmodulemd API properly when setting values on existing objects
MBS has been using the libmodulemd API incorrectly by assuming that
methods like `get_rpm_components` return the actual object in memory
and not a copy. This has been true but wasn't something sgallagh
intended. We found this out after he had me test his new version of
libmodulemd. This PR removes those assumptions.
2018-07-17 09:00:42 -04:00
Qixiang Wan
d83e6897ca Change ModuleBuild.context to db column
1. Changed ModuleBuild's context property to db column, it's
non-nullable and default value is '00000000' to keep it consistent
with previous behaviour.

2. Changed ModuleBuild.contexts_from_mmd to return a tuple of
(ref_build_context, build_context, runtime_context, context).

3. Updated tests affected by this change.
2018-04-28 11:46:31 +08:00
Qixiang Wan
8b807a9fcd Generate koji tag from MBS and use informative name
koji now supports tags with max length of 256, we can use
more informative tag name instead of the hash one.

The new format of koji tag name is:

    module-<name>-<stream>-<version>-<context>

However when the generated tag's length > (256 - len('build')), we
fallback to the old way of name in hash format (module-<hash>).

In this change, koji tag is always generated from MBS itself, even
with pdc resolver.

FIXES: #918 #925
2018-04-26 22:04:16 +08:00
Jan Kaluza
8d1d439737 Generate 'context' from hash based on buildrequires/requires stream. Reuse components only from module with the same stream_build_context. 2018-04-24 15:45:26 +02:00
Jan Kaluza
6fc8c646de Add 'mbs-manager import_module'. 2018-04-03 09:58:57 -04:00
Jan Kaluza
e5e9a24fce Allow adding bool to XMD. Fix flake8 issues. 2018-04-03 09:58:57 -04:00
Jan Kaluza
31ddbe51a5 Add generate_expanded_mmds which returns list of MMDs which can be submitted as MSE builds and submit them. 2018-04-03 09:58:57 -04:00
mprahl
ee43b1bb7d Flake8 fixes 2018-04-03 09:58:57 -04:00
mprahl
eb0b2e1c38 Add the database resolver plugin 2018-04-03 09:58:57 -04:00
Jan Kaluza
1bbe8d69f1 Add ModuleBuild methods for getting list of streams, last build in a name:stream and all builds in name:stream:version. 2018-04-03 09:58:57 -04:00
mprahl
f0852d9009 Port to libmodulemd and support v2 modulemd without module stream expansion 2018-04-03 09:58:57 -04:00
Jan Kaluza
6a5d7267fd Save another 20s in tests. 2018-02-09 11:46:12 +01:00
Jan Kaluza
fb1463a036 Remove zeromq time.sleep(1) to make tests even faster. 2018-02-08 20:23:55 +01:00
Jan Kaluza
4001d3c488 Use in-memory SQLite database in tests to make them 40% faster. 2018-02-08 08:01:24 +01:00
mprahl
8f024e6b04 Remove the dependency on vcrpy in unit tests and modernize some of the test data 2018-02-05 22:17:18 -05:00
mprahl
e91d09f7ca Change the format of the unit tests to pytest 2018-01-31 16:34:21 -05:00
Filip Valder
5b4a8f6240 New: Resolver API for pluggable resolvers 2018-01-29 15:54:19 +01:00
Ralph Bean
7e6b54dd54 Address modules in the test suite with id=0.
It wasn't technically wrong before, just soft.
2018-01-18 14:49:27 -05:00
mprahl
a67b53f5c3 Add a "context" field on component and module releases in Koji for uniqueness for when Module Stream Expansion is implemented 2018-01-16 10:36:09 -05:00
mprahl
bb0dde9936 Only complete the module build when *all components are tagged* and a KojiRepoRegen message is received 2017-11-10 08:33:53 -05:00
mprahl
45cce653eb Add the ability to specify different rebuild methods 2017-11-01 15:39:43 -04:00
Martin Curlej
60b3d97c97 flake8 cleanup
Signed-off-by: Martin Curlej <mcurlej@redhat.com>

removed some noqa tags
2017-10-25 12:13:59 +02:00
Jan Kaluza
aa8a0574e4 Move vcrpy cache to gzipped tarball and extract it before tests execution. 2017-09-08 11:35:24 +02:00
Filip Valder
7ef2975984 PEP8 fixes 2017-07-17 18:29:10 +02:00
Jan Kaluza
b4c0f49e4b Fix get_reusable_component in case when database returns components in different order than what we have in MBS. 2017-06-05 15:03:59 +02:00
Stanislav Ochotnicky
acb59d23f1 Add an example mmd file into the test database
This will actually be used to test the content generator functionality
2017-05-04 15:45:42 +02:00
Nils Philippsen
ec2be36d6e pep8: don't surround '=' with spaces for keyword args 2017-04-21 12:54:06 +02:00
Jan Kaluža
27e4c3868a Merge #444 Mark module build as failed if one of the components fail 2017-03-17 14:20:31 +00:00
Jan Kaluza
5aa6360595 Mark module build as failed if one of the components fail 2017-03-17 15:14:41 +01:00
Jan Kaluza
a72a97754a Fix typo in test_reuse_component_init_data name, set SERVER_NAME config variable in config.py, return commit in MockedSCM.get_latest 2017-03-17 08:58:58 +01:00
Jan Kaluza
830b1dd111 Add tests for 'continue' part of start_next_batch and component resubmit 2017-03-16 11:20:51 +01:00
Filip Valder
13b7bcd03e Support for state tracing of modules and components
- DB migration scripts.
- Models: ComponentBuildTrace, ModuleBuildTrace.
- ModuleBuild.state_trace method for querying for a particular module's
state history.
- SQLAlchemy before commit session event handler for recording
module/component state changes.
- REST API verbose mode for getting state trace of a particular module.
- Tests use make_session, so that event handlers are in effect.
- Short info in README about verbose mode.
- Tests verifying whether state trace information about a module appears
in verbose mode.
- Other minor fixes (RidaBase -> MBSBase, PEP8...)
2017-02-28 16:15:43 +01:00
Matt Prahl
8ca616ec57 Add new tests for reusing components 2017-02-21 14:22:31 -05:00
Matt Prahl
8577d5f307 Fix and cleanup tests 2017-02-21 14:22:30 -05:00
Jan Kaluza
4acc1d69aa Use vcrpy for tests to simulate PDC responses. 2017-01-12 11:52:49 +01:00
Jan Kaluza
0f797c2917 Remove fake base-runtime module from DB and old resolve_profiles method because they are not used anymore. Fix the tests to work with remove fake base-runtime. 2017-01-10 15:58:26 +01:00
Jan Kaluza
dda47acab7 Fix #276 - Use PDC to resolve buildroot components instead of MBS database 2017-01-10 14:29:54 +01:00
Matt Prahl
61a724d0d9 Use the database in test_repo_done.py tests 2016-12-12 09:00:44 -05:00
Matt Prahl
cbf1f6acdc Insert baseruntime in test database 2016-12-12 09:00:44 -05:00
Filip Valder
febb8de4f6 tests and test.test_models module initialization updated 2016-11-21 22:53:32 +01:00