Commit Graph

33 Commits

Author SHA1 Message Date
mprahl
f7e7765686 Add the ability to override buildrequires and requires when submitting a module build
This will allow for tooling to automatically override certain buildrequires and requires
based on the branch name the modulemd is built form.

Addresses FACTORY-3414
2018-10-29 17:08:27 -04:00
Chenxiong Qi
ec28f9aa00 Remove unused parameter scm from submit_module_build
submit_module_build creates new module build and set parameter url to
ModuleBuild.scmurl. It looks scm.url could be passed to scmurl as well,
but scm is not used through out whole method. url is enough for the
purpose, hence scm is removed.

Signed-off-by: Chenxiong Qi <cqi@redhat.com>
2018-10-22 16:51:03 +08:00
Jan Kaluza
192eaae012 Support Virtual Streams in MMDResolver.
Modularity team needs to be able to define multiple streams for single "platform"
module. They need it to express that "platform:el8.1.0" also provides "platform:el8".

This needs changes in a way how MMDResolver resolves dependencies between modules.

For example, if we are building module against platform:el8.1.0, the MMDResolver must
not return buildrequired module built against platform > el8.1.0, but it can for example
return (cherry-pick) buildrequired module from platform:el8.0.0 if there is no such
module built for platform:el8.1.0.

The way how it is implemented is following:

  - MMDResolver reads list of virtual streams from xmd["mbs"]["virtual_streams"] when
    creating Solvable from MMD and adds additional Provides for these virtual streams.
    We expect these to be set mainly on base modules. The versions of such provides
    are based on "stream version" number, so we can compare them.
  - The base module ("platform") buildrequires of MMDs added to MMDResolver are overriden
    to mark particular platform "stream version". For example, if module "foo" buildrequires
    "platform:el8" in MMD file, but was in fact built against platform:el8.1.0, it will
    be treated as if it would really buildrequire "platform:el8.1.0". This is needed
    to not include buildrequired modules built against newer platform than what we want.
  - MMDResolver resolves all the valid combinations of buildrequires, but we are only
    interested in the one with latest versions of buildrequired module. Therefore the
    solve() method is changed to find out combinations which have the latest versions
    for each alternative name:stream buildrequires.
2018-10-18 15:06:04 +02:00
mprahl
599c881714 Add the ability to query by the base modules a module build buildrequires
Future use cases will require the ability to find compatible module builds
to buildrequire based on the base module the module used to build. This
commit adds an association table that will contain module build IDs
and the base module they buildrequire.

Addresses FACTORY-3353
2018-10-17 07:47:31 -04:00
mprahl
4f9fffff3b Move some of the logic in get_prefixed_version to a ModuleBuild static method
The logic used to parse the stream version will need to be reused elsewhere
to support the filtering and ordering of a base module stream version.
2018-10-16 10:26:39 -04:00
Valerij Maljulin
7f60db545a renaming koji_arches to arches
Signed-off-by: Valerij Maljulin <vmaljuli@redhat.com>
2018-10-15 15:29:50 +02:00
Valerij Maljulin
924a0951fb add architectures support for format_mmd 2018-10-15 15:29:50 +02:00
mprahl
117791ee16 Prefix the component disttag with the platform stream 2018-10-04 07:19:07 -04:00
mprahl
cfb75b4d0f Prefix the module version based on the platform it buildrequires 2018-10-04 07:19:07 -04:00
mprahl
314688f170 Correct the docstrings referring to the old modulemd type 2018-10-02 12:59:36 -04:00
Chenxiong Qi
520a979d3d Add singleton system_resolver
system_resolver is created based on loaded configuration, which could
avoid calls like `GenericResolver.create(conf)` repeatedly in the code.

However, if some cases need to create a specific resolver explicitly,
`GenericResolver.create` could be called with addition argument, for
example db or mbs is passed to argument backend in tests.

Signed-off-by: Chenxiong Qi <cqi@redhat.com>
2018-09-28 11:00:52 +08:00
Chenxiong Qi
2c58471630 Fix and reword some docstrings again
During I'm reading through the code base to learn MBS, I found out some
typos and minor issues in some other docstrings, and I also found some
docstrings with extra informative description could make it easier to
understand the code.

Signed-off-by: Chenxiong Qi <cqi@redhat.com>
2018-09-26 14:12:39 +08:00
Ralph Bean
cfbe031218 Fix bug breaking local builds.
This bug was reported by @nphilipp here https://bodhi.fedoraproject.org/updates/FEDORA-2018-c9c3a37d96

The issue is that the `record_filtered_rpms` method returns a new xmd
buildrequires block, but whole buildrequires entries were omitted in the return
value for local builds.  This one-liner includes them again.
2018-09-10 20:53:29 -04:00
Filip Valder
598347e1b6 Import module API 2018-09-07 17:12:01 +02:00
mprahl
9d40718895 Allow module components to use a git ref outside of the master branch 2018-09-06 15:04:24 -04:00
Ralph Bean
e7089d7be0 Rename this to be PDC-agnostic. 2018-09-04 09:51:56 -04:00
Ralph Bean
0aff640ef3 Optionally, block building modules with EOL streams.
Requested by @mboddu in:

- https://pagure.io/fm-orchestrator/issue/960
- https://pagure.io/modularity/issue/102
2018-08-31 15:23:04 -04:00
Jan Kaluza
56fe201c33 Fill in the 'filterer_rpms' in backend to not access Koji from frontend. 2018-08-21 07:58:52 +02:00
Ralph Bean
84762891f7 Close threadpools after use. 2018-08-15 12:47:18 -04:00
Ralph Bean
4279e947be Allow overriding the name and stream from scm repos.
If configured on the server side, disabled by default.
2018-08-13 14:08:36 -04:00
mprahl
5197eb4912 Revert "Use the libmodulemd API properly when setting values on existing objects"
This reverts commit 8173040ea6. Since it didn't
provide any real benefit after talking with sgallagh, it seems like we'd better
off not introduce more risk to the next deployment.
2018-08-07 12:03:44 -04:00
mprahl
8173040ea6 Use the libmodulemd API properly when setting values on existing objects
MBS has been using the libmodulemd API incorrectly by assuming that
methods like `get_rpm_components` return the actual object in memory
and not a copy. This has been true but wasn't something sgallagh
intended. We found this out after he had me test his new version of
libmodulemd. This PR removes those assumptions.
2018-07-17 09:00:42 -04:00
Martin Curlej
dfe7660519 Removed PDC dependency from MBS and replaced it with MBS prod instance.
Signed-off-by: Martin Curlej <mcurlej@redhat.com>

Removed pdc from comments.

Signed-off-by: Martin Curlej <mcurlej@redhat.com>

Adding missing files.

Signed-off-by: Martin Curlej <mcurlej@redhat.com>

Updated PR according to review.

Signed-off-by: Martin Curlej <mcurlej@redhat.com>

Local modules builds now

Signed-off-by: Martin Curlej <mcurlej@redhat.com>

Removed copr from config

Signed-off-by: Martin Curlej <mcurlej@redhat.com>

Fixed bugs

Signed-off-by: Martin Curlej <mcurlej@redhat.com>
2018-07-04 10:13:18 +02:00
Owen W. Taylor
19bf94f0ea Load 'context' for local builds
'context' needed to be stored into the in-memory database and then
propagated for builds added to a MockModuleBuild with --add-local-build.
2018-05-30 15:47:14 -04:00
Ralph Bean
95eead27f3 Merge #934 Log this exception. 2018-05-08 19:55:02 +00:00
Ralph Bean
12fcd91bcd Log this exception.
If libmodulemd returns some garbage, it will be nice to know what it is.
2018-05-08 15:49:51 -04:00
Ralph Bean
5a8764e8d9 py3: The dict-value objects returned by .values() doesn't support addition. 2018-05-08 14:18:00 -04:00
Qixiang Wan
d83e6897ca Change ModuleBuild.context to db column
1. Changed ModuleBuild's context property to db column, it's
non-nullable and default value is '00000000' to keep it consistent
with previous behaviour.

2. Changed ModuleBuild.contexts_from_mmd to return a tuple of
(ref_build_context, build_context, runtime_context, context).

3. Updated tests affected by this change.
2018-04-28 11:46:31 +08:00
Matt Prahl
8351713828 Merge #922 Generate 'context' from hash based on buildrequires/requires stream. Reuse components only from module with the same stream_build_context. 2018-04-24 22:50:02 +00:00
Jan Kaluza
8d1d439737 Generate 'context' from hash based on buildrequires/requires stream. Reuse components only from module with the same stream_build_context. 2018-04-24 15:45:26 +02:00
mprahl
d4d9360662 Rename a variable for clarity in load_local_builds 2018-04-04 11:12:08 -04:00
mprahl
19db295795 Fix flake8 errors 2018-04-03 09:58:57 -04:00
Jan Kaluza
9b6fd2ba39 Split utils.py to multiple files based on the methods use-case. 2018-04-03 09:58:57 -04:00