2064 Commits

Author SHA1 Message Date
mprahl
654a30129f Release v2.9.2 v2.9.2 2018-12-12 10:33:56 -05:00
Matt Prahl
f4af515dad Merge #1100 Fix handling of SRPMs in Content Generator builds when SRPM name and main package name are different. 2018-12-12 15:26:59 +00:00
Jan Kaluza
7e7f298381 Fix handling of SRPMs in Content Generator builds when SRPM name and main package name are different.
Current code presumes SRPM name always matches the RPM name built out of this SRPM
and only includes it together with the main package in this case. This is wrong
assumption, because usually there are multiple binary RPMs built from single SRPM.

This commit fixes that by including the SRPM NEVRA in `non_devel_source_rpms`
no matter what RPM name is.

The test RPMs are reworked in this commit to match the reality better - especially
the relations between SRPM and RPMs. The case with different SRPM name and RPM
name is also included in the reworked test - dhcp-libs binary RPM built from
dhcp SRPM.
2018-12-12 16:25:20 +01:00
mprahl
96f82443aa Remove the unused "owner" parameter from KojiModuleBuilder.get_session 2018-12-10 09:59:42 -05:00
mprahl
00693cbd00 Improve the readability in KojiModuleBuilder.get_session 2018-12-10 09:54:29 -05:00
Matt Prahl
692c34fceb Merge #1094 Use anonymous Koji session properly 2018-12-10 14:52:22 +00:00
Jan Kaluza
6f43411ea9 Fix circular dependency between module_build_service.__init__ and module_build_service.backports.
This fixes issues with calling 'python module_build_service/manage.py' directly
from the MBS git repo as part of testing local module builds without installing
MBS.

The issue is that 'import pkg_resources' for some reason tries to load
module_build_service.backports before the module_build_service.app actually
exists. This results in traceback saying that module_build_service.app does
not exist.

This commit fixes this by importing whole module_build_service in backports.py.
2018-12-10 12:05:51 +01:00
Chenxiong Qi
cbb8156311 Add tests for use of anonymous koji session
This patch also clean and rewrite some tests that are relative to use
anonymous koji session.

Signed-off-by: Chenxiong Qi <cqi@redhat.com>
2018-12-07 18:37:36 +08:00
Chenxiong Qi
55add5cfc0 Use anonymous Koji session properly
MBS calls some read-only Koji APIs which does not require to log into a
session. This patch makes it optional to choose whether to login a
session and use anonymous session properly to call those read-only APIs.

Signed-off-by: Chenxiong Qi <cqi@redhat.com>
2018-12-06 16:31:12 +08:00
mprahl
61d8b22fc3 Release v2.9.1 v2.9.1 2018-12-05 09:14:44 -05:00
mprahl
9604da1b22 Fix a flake8 error 2018-12-05 09:13:26 -05:00
Chenxiong Qi
5579a0e72b Init event handler handles module stream collision
Fixes: FACTORY-3616

Signed-off-by: Chenxiong Qi <cqi@redhat.com>
2018-12-05 18:25:14 +08:00
mprahl
676bef79f2 Release v2.9.0 v2.9.0 2018-12-04 13:46:56 -05:00
Matt Prahl
01f2571024 Merge #1087 Fix rebuild strategy name changed-and-after in README 2018-12-04 16:57:49 +00:00
Matt Prahl
8d36d97a5d Merge #1085 Show expanded buildrequires in API output 2018-12-04 16:57:12 +00:00
Chenxiong Qi
0c642a0944 Fix deprecation warnings from log.warn and inspect.getargspec
Signed-off-by: Chenxiong Qi <cqi@redhat.com>
2018-12-04 18:35:56 +08:00
mprahl
25b6a93a07 Make the devel Koji CG build optional using a config option 2018-12-03 12:00:30 -05:00
Petr Šabata
6abb585cdc Define ModularityLabel RPM header
We intend to stop using the DistTag header and replace it with
ModularityLabel.  Upstream RPM already supports it but it is not yet
available in Fedora.

Related Modularity ticket: https://pagure.io/modularity/issue/113

Related RHBZ: https://bugzilla.redhat.com/show_bug.cgi?id=1650286

For the time being, we need to keep both.

Signed-off-by: Petr Šabata <contyk@redhat.com>
2018-12-03 11:44:06 +01:00
Chenxiong Qi
2febab9c19 Fix rebuild strategy name changed-and-after in README
Signed-off-by: Chenxiong Qi <cqi@redhat.com>
2018-11-28 22:10:33 +08:00
mprahl
66a76f4fcd Don't clean up Koji build targets on base modules 2018-11-27 15:45:28 -05:00
Chenxiong Qi
41481afc75 Remove unuseful topurl from tests
Signed-off-by: Chenxiong Qi <cqi@redhat.com>
2018-11-27 11:06:34 +08:00
Chenxiong Qi
1cd64d336a Fix some minor issues
* Fix failure test after rebasing on master branch.
* Fix some grammar issues.
* Only check stream collision modules on new created module build.
* Logging message properly.

Signed-off-by: Chenxiong Qi <cqi@redhat.com>
2018-11-27 11:06:34 +08:00
Chenxiong Qi
d517a72de9 Add _get_module in DBResolver and fix _record_ursine_rpms
_record_ursine_rpms needs to get each collision module's koji_tag and
then get built RPMs from that koji_tag eventually. _get_module on each
individual resolver is called to get module metadata accordingly. So,
when running a local build, module metadata is got from remote MBS, and
for a MBS instance, connected database is queried.

Signed-off-by: Chenxiong Qi <cqi@redhat.com>
2018-11-27 11:06:34 +08:00
Chenxiong Qi
476c51e1f4 Log exception for catching custom exception in init handler
For tracking error easily when custom exception is raised, log exception
as well before transitioning to failed state.

Signed-off-by: Chenxiong Qi <cqi@redhat.com>
2018-11-27 11:06:34 +08:00
Chenxiong Qi
5d3ea762c6 Make external repo URL prefix configurable
Not both Fedora and internal Brew uses config topurl as the external
repo's URL prefix. Hence, make it configurable to fulfill this
difference.

Signed-off-by: Chenxiong Qi <cqi@redhat.com>
2018-11-27 11:06:34 +08:00
Chenxiong Qi
917c06ad0c Resolve stream collision with modules added to ursine content
This resolve the stream collision by adding specific RPMs to
module-build-macros SRPM as Conflicts.

For more information about module stream collision, please refer to
docstring in utils/ursine.py

Signed-off-by: Chenxiong Qi <cqi@redhat.com>
2018-11-27 11:06:34 +08:00
Chenxiong Qi
04b5a9a2a7 Show expanded buildrequires in API output
ModuleBuild.json now includes xmd/mbs/buildrequires in the output JSON
data. As a result, these APIs will show buildrequires by default.

* /module-builds/
* /module-builds/?verbose=True
* /module-builds/$build_id

Buildrequires is accessible like this:

    br_module_names = response_json['buildrequires'].keys()
    br_module_dict = response_json['buildrequires'][module name]

Fixes: FACTORY-2201

Signed-off-by: Chenxiong Qi <cqi@redhat.com>
2018-11-23 17:40:23 +08:00
Luiz Carvalho
086ed4a3f3 Remove dangling -debug* RPMs from final MMD
If an RPM is not included, its correspnding -debug* RPMs should also not
be included.

Also ensure that source RPMs are only ever added to -devel modules if
a binary RPM has been completely excluded from non-devel module.

Internal ref: FACTORY-3263

Signed-off-by: Luiz Carvalho <lucarval@redhat.com>
2018-11-21 13:56:20 -05:00
Jan Kaluza
fe88cffa21 Require non-devel module in -devel module. Clear API list and profiles for -devel. 2018-11-21 15:16:36 +01:00
Jan Kaluza
f5e3c81b0c Import original source modulemd file as stored in dist-git to CG build.
There is a need to ship the unchanged source file which was used to build
a module build from legal reasons. The KojiContentGenerator is extended
in this commit to fetch it from SCM URL using the `scm` module and later
attach it as `modulemd.src.txt` to Koji CG build.
2018-11-21 14:22:58 +01:00
Jan Kaluža
98aa2ea19f Merge #1080 Do not retreive rpm info if there is an empty rpm list 2018-11-21 10:10:25 +00:00
Chenxiong Qi
b5860fa039 Make it optional to tag build for CG import
Fixes FACTORY-3467

Signed-off-by: Chenxiong Qi <cqi@redhat.com>
2018-11-21 09:28:03 +08:00
Chenxiong Qi
42429cf043 Refactor _get_deps_and_tag
Signed-off-by: Chenxiong Qi <cqi@redhat.com>
2018-11-20 19:00:48 +08:00
Valerij Maljulin
f3c86541b6 Do not retreive rpm info if there is an empty rpm list.
Fixes #1078

Signed-off-by: Valerij Maljulin <vmaljuli@redhat.com>
2018-11-20 10:45:17 +01:00
Jan Kaluza
343d145180 Do not leak internal data into final MMD files attached to CG build.
This commit introduces KojiContentGenerator._sanitize_mmd method to:
- remove `repository` and `cache` from ComponentRPM in Modulemd.
- remove `mbs` section from `xmd`.

This is done to not leak internal build-only information to final
modulemd files.
2018-11-19 09:51:27 +01:00
Valerij Maljulin
8adce7593b set module build state to 'failed' imediately
Fixes #1009

Signed-off-by: Valerij Maljulin <vmaljuli@redhat.com>
2018-11-15 17:13:34 +01:00
Matt Prahl
290f19d477 Merge #1076 Update the README for OpenShift deployments 2018-11-13 17:24:22 +00:00
Hunor Csomortáni
8ec8a01006 Rename 'koji_arches' to 'arches' in OpenShift config
'koji_arches' was renamed to 'arches' in a previous commit. Update the
configuration in the OpenShift template accordingly.

Signed-off-by: Hunor Csomortáni <csomh@redhat.com>
2018-11-13 15:10:32 +01:00
Hunor Csomortáni
abda4dd8f2 Update the README for OpenShift deployments
The backend container images needs a build argument holding the UMB CA
certificate. Update the README with this information.

Signed-off-by: Hunor Csomortáni <csomh@redhat.com>
2018-11-13 12:39:44 +01:00
Hunor Csomortáni
cded35e33a Increase polling interval
Polling works in weird ways. It's much better to rely on messaging.

Signed-off-by: Hunor Csomortáni <csomh@redhat.com>
2018-11-12 17:56:16 +01:00
Hunor Csomortáni
76e5845632 Indicate 'git' in default repository URLs
Without this, Koji will not treat them as SCM URLs.

Signed-off-by: Hunor Csomortáni <csomh@redhat.com>
2018-11-12 17:56:16 +01:00
Hunor Csomortáni
64b8d8c169 Allow custom SCMs in OpenShift
The template is intended to be used for testing, so it makes sense to
allow submitting builds from custom SCM URLs.

Signed-off-by: Hunor Csomortáni <csomh@redhat.com>
2018-11-12 17:56:16 +01:00
Hunor Csomortáni
438fc1e531 Configure topic prefix in the OpenShift template
Signed-off-by: Hunor Csomortáni <csomh@redhat.com>
2018-11-12 17:56:16 +01:00
Hunor Csomortáni
849ffec729 Add module-build-macros to platform buildroots
Koji builds done for a module have to have this in order to set the
correct %dist tag, for example.

Signed-off-by: Hunor Csomortáni <csomh@redhat.com>
2018-11-12 17:56:16 +01:00
Hunor Csomortáni
236bc69938 Disable authentication in the OpenShift template
In testing contexts there auth providers are still missing. Disable auth
to allow anyone to submit module builds.

Signed-off-by: Hunor Csomortáni <csomh@redhat.com>
2018-11-12 17:56:16 +01:00
Hunor Csomortáni
f68d708f41 Add the test UMB server CA to the backend image
Signed-off-by: Hunor Csomortáni <csomh@redhat.com>
2018-11-12 17:55:54 +01:00
Hunor Csomortáni
d30c50cff0 Use stomppy 3.1.6 in the container image
Using the latest version would require rewriting some of the messaging
pieces.

Signed-off-by: Hunor Csomortáni <csomh@redhat.com>
2018-11-12 17:51:30 +01:00
Valerij Maljulin
6b7ea33d18 limit Koji build architectures
Fixes issue #869

Signed-off-by: Valerij Maljulin <vmaljuli@redhat.com>
2018-11-12 12:44:52 +01:00
Matt Prahl
f28766ea13 Merge #1070 Refactor SCM.get_latest 2018-11-07 14:47:08 +00:00
Matt Prahl
7fe4dc89fc Merge #1072 Minor fixes to wait handler and KojiModuleBuilder 2018-11-07 13:11:15 +00:00