Commit Graph

2085 Commits

Author SHA1 Message Date
Valerij Maljulin
aed51dfae1 Adding exception handling while doing import
This fixes #1068

Signed-off-by: Valerij Maljulin <vmaljuli@redhat.com>
2019-01-10 15:22:37 +01:00
Petr Šabata
ff1ed9926a Don't define DistTag anymore
Fedora 30 now includes rpm-4.14.2.1-4.fc30 with support for
ModularityLabel.  We're not using the DistTag anywhere anymore and it
should be safe to drop it at this point.

Signed-off-by: Petr Šabata <contyk@redhat.com>
2019-01-04 12:02:22 +01:00
Jan Kaluza
3c7f8b7006 Handle lost message from Koji informing MBS about Component being tagged into Koji tag.
Adds new meethod checking the "tagged" and "tagged_in_final" attributes of
"complete" ComponentBuilds in the current batch of module builds
in "building" state against the Koji.

In case the Koji shows the build as tagged/tagged_in_final,
the fake "tagged" message is added to work queue.
2019-01-02 09:33:42 +01:00
mprahl
3515919a94 Add a missing entry to the change log 2018-12-19 13:06:03 -05:00
mprahl
b579a30931 Release v2.11.1 v2.11.1 2018-12-19 09:09:38 -05:00
mprahl
e237676c7e Don't overwrite the state parameter in _nudge_module_builds_in_state 2018-12-19 09:06:59 -05:00
mprahl
fec1a8be92 Fix a log statement in the poller 2018-12-18 17:17:44 -05:00
mprahl
dd1baf1682 Release v2.11.0 v2.11.0 2018-12-18 16:25:26 -05:00
Matt Prahl
8bb50cfe16 Merge #1093 List package name in failed state reason 2018-12-18 21:19:14 +00:00
mprahl
25122cb53e Modify MBS to use a separate Kerberos context per thread so both threads can use the thread keyring to store the Kerberos cache
This commit includes the backport of the changes to `krb_login` in
https://pagure.io/koji/pull-request/1187. This change is required
for our separate threads to use a separate Kerberos context per thread.
2018-12-18 16:05:55 -05:00
Jan Kaluza
e17fdbdb5a Raise an exception in case getRPMHeaders return an empty dict or does not return the RPM license. 2018-12-18 16:28:47 +01:00
Jan Kaluža
f47cc39de9 Merge #1109 Fix creation of CG build without any RPM. 2018-12-18 09:38:28 +00:00
Chenxiong Qi
5c2b411309 List package name in failed state reason
Client tool like module-build-watch and `module-build --watch' could
output failed reason. However, original message "Some components failed
to buld" is not informative enough. Instead, list package names of
failed builds would be helpful.

Signed-off-by: Chenxiong Qi <cqi@redhat.com>
2018-12-18 16:09:16 +08:00
Jan Kaluza
bcfead0809 In case module is stuck in 'init' state for more than 10 minutes, send fake MBSModule msg.
This fixes issues when UMB message delivery from frontend to backend fails
for whatever reason...

We do the same thing for 'wait' state already, so this commit just extends
it to 'init' state too.
2018-12-18 08:06:31 +01:00
Jan Kaluža
cc303854ad Merge #1096 Non-zero return code for failing builds when building locally 2018-12-17 14:21:40 +00:00
Jan Kaluza
db116aee71 Fix creation of CG build without any RPM.
In case the module did not contain any RPM, _koji_rpms_in_tag still
tried to call the session.getRPMHeaders with an empty list. This results
in None being returned, which is something _koji_rpms_in_tag did
not count with and failed with traceback when trying to iterate
that None value.

In this commit, the _koji_rpms_in_tag returns an empty list early
if Koji tag does not contain any RPM. Therefore the
session.getRPMHeaders is not called at all in this case, because it
does not make any sense to get RPM Headers when Koji tag does not
contain any RPM.
2018-12-14 08:20:57 +01:00
mprahl
6f2866e25a Release v2.10.0 v2.10.0 2018-12-12 19:18:12 -05:00
Matt Prahl
78bb367e7a Merge #1103 Force the epoch on SRPM artifacts in the modulemd files uploaded to the CG build 2018-12-13 00:10:34 +00:00
mprahl
7f82275114 Force the use of a separte Kerberos cache per thread
When using a single Kerberos cache that is shared among threads,
Koji logins start failing because the cache gets corrupt. This uses
the Linux kernel keyring to store a Kerberos cache per MBS thread.

See https://web.mit.edu/kerberos/krb5-1.12/doc/basic/ccache_def.html
2018-12-12 19:04:00 -05:00
mprahl
c3a4b2b91f Force the epoch on SRPM artifacts in the modulemd files uploaded to the CG build 2018-12-12 16:14:07 -05:00
mprahl
654a30129f Release v2.9.2 v2.9.2 2018-12-12 10:33:56 -05:00
Matt Prahl
f4af515dad Merge #1100 Fix handling of SRPMs in Content Generator builds when SRPM name and main package name are different. 2018-12-12 15:26:59 +00:00
Jan Kaluza
7e7f298381 Fix handling of SRPMs in Content Generator builds when SRPM name and main package name are different.
Current code presumes SRPM name always matches the RPM name built out of this SRPM
and only includes it together with the main package in this case. This is wrong
assumption, because usually there are multiple binary RPMs built from single SRPM.

This commit fixes that by including the SRPM NEVRA in `non_devel_source_rpms`
no matter what RPM name is.

The test RPMs are reworked in this commit to match the reality better - especially
the relations between SRPM and RPMs. The case with different SRPM name and RPM
name is also included in the reworked test - dhcp-libs binary RPM built from
dhcp SRPM.
2018-12-12 16:25:20 +01:00
Valerij Maljulin
cadb0587c1 Non-zero return code for failing builds when building locally
Fixes #1063

Signed-off-by: Valerij Maljulin <vmaljuli@redhat.com>
2018-12-10 16:24:11 +01:00
mprahl
96f82443aa Remove the unused "owner" parameter from KojiModuleBuilder.get_session 2018-12-10 09:59:42 -05:00
mprahl
00693cbd00 Improve the readability in KojiModuleBuilder.get_session 2018-12-10 09:54:29 -05:00
Matt Prahl
692c34fceb Merge #1094 Use anonymous Koji session properly 2018-12-10 14:52:22 +00:00
Jan Kaluza
6f43411ea9 Fix circular dependency between module_build_service.__init__ and module_build_service.backports.
This fixes issues with calling 'python module_build_service/manage.py' directly
from the MBS git repo as part of testing local module builds without installing
MBS.

The issue is that 'import pkg_resources' for some reason tries to load
module_build_service.backports before the module_build_service.app actually
exists. This results in traceback saying that module_build_service.app does
not exist.

This commit fixes this by importing whole module_build_service in backports.py.
2018-12-10 12:05:51 +01:00
Chenxiong Qi
cbb8156311 Add tests for use of anonymous koji session
This patch also clean and rewrite some tests that are relative to use
anonymous koji session.

Signed-off-by: Chenxiong Qi <cqi@redhat.com>
2018-12-07 18:37:36 +08:00
Chenxiong Qi
55add5cfc0 Use anonymous Koji session properly
MBS calls some read-only Koji APIs which does not require to log into a
session. This patch makes it optional to choose whether to login a
session and use anonymous session properly to call those read-only APIs.

Signed-off-by: Chenxiong Qi <cqi@redhat.com>
2018-12-06 16:31:12 +08:00
mprahl
61d8b22fc3 Release v2.9.1 v2.9.1 2018-12-05 09:14:44 -05:00
mprahl
9604da1b22 Fix a flake8 error 2018-12-05 09:13:26 -05:00
Chenxiong Qi
5579a0e72b Init event handler handles module stream collision
Fixes: FACTORY-3616

Signed-off-by: Chenxiong Qi <cqi@redhat.com>
2018-12-05 18:25:14 +08:00
mprahl
676bef79f2 Release v2.9.0 v2.9.0 2018-12-04 13:46:56 -05:00
Matt Prahl
01f2571024 Merge #1087 Fix rebuild strategy name changed-and-after in README 2018-12-04 16:57:49 +00:00
Matt Prahl
8d36d97a5d Merge #1085 Show expanded buildrequires in API output 2018-12-04 16:57:12 +00:00
Chenxiong Qi
0c642a0944 Fix deprecation warnings from log.warn and inspect.getargspec
Signed-off-by: Chenxiong Qi <cqi@redhat.com>
2018-12-04 18:35:56 +08:00
mprahl
25b6a93a07 Make the devel Koji CG build optional using a config option 2018-12-03 12:00:30 -05:00
Petr Šabata
6abb585cdc Define ModularityLabel RPM header
We intend to stop using the DistTag header and replace it with
ModularityLabel.  Upstream RPM already supports it but it is not yet
available in Fedora.

Related Modularity ticket: https://pagure.io/modularity/issue/113

Related RHBZ: https://bugzilla.redhat.com/show_bug.cgi?id=1650286

For the time being, we need to keep both.

Signed-off-by: Petr Šabata <contyk@redhat.com>
2018-12-03 11:44:06 +01:00
Chenxiong Qi
2febab9c19 Fix rebuild strategy name changed-and-after in README
Signed-off-by: Chenxiong Qi <cqi@redhat.com>
2018-11-28 22:10:33 +08:00
mprahl
66a76f4fcd Don't clean up Koji build targets on base modules 2018-11-27 15:45:28 -05:00
Chenxiong Qi
41481afc75 Remove unuseful topurl from tests
Signed-off-by: Chenxiong Qi <cqi@redhat.com>
2018-11-27 11:06:34 +08:00
Chenxiong Qi
1cd64d336a Fix some minor issues
* Fix failure test after rebasing on master branch.
* Fix some grammar issues.
* Only check stream collision modules on new created module build.
* Logging message properly.

Signed-off-by: Chenxiong Qi <cqi@redhat.com>
2018-11-27 11:06:34 +08:00
Chenxiong Qi
d517a72de9 Add _get_module in DBResolver and fix _record_ursine_rpms
_record_ursine_rpms needs to get each collision module's koji_tag and
then get built RPMs from that koji_tag eventually. _get_module on each
individual resolver is called to get module metadata accordingly. So,
when running a local build, module metadata is got from remote MBS, and
for a MBS instance, connected database is queried.

Signed-off-by: Chenxiong Qi <cqi@redhat.com>
2018-11-27 11:06:34 +08:00
Chenxiong Qi
476c51e1f4 Log exception for catching custom exception in init handler
For tracking error easily when custom exception is raised, log exception
as well before transitioning to failed state.

Signed-off-by: Chenxiong Qi <cqi@redhat.com>
2018-11-27 11:06:34 +08:00
Chenxiong Qi
5d3ea762c6 Make external repo URL prefix configurable
Not both Fedora and internal Brew uses config topurl as the external
repo's URL prefix. Hence, make it configurable to fulfill this
difference.

Signed-off-by: Chenxiong Qi <cqi@redhat.com>
2018-11-27 11:06:34 +08:00
Chenxiong Qi
917c06ad0c Resolve stream collision with modules added to ursine content
This resolve the stream collision by adding specific RPMs to
module-build-macros SRPM as Conflicts.

For more information about module stream collision, please refer to
docstring in utils/ursine.py

Signed-off-by: Chenxiong Qi <cqi@redhat.com>
2018-11-27 11:06:34 +08:00
Chenxiong Qi
04b5a9a2a7 Show expanded buildrequires in API output
ModuleBuild.json now includes xmd/mbs/buildrequires in the output JSON
data. As a result, these APIs will show buildrequires by default.

* /module-builds/
* /module-builds/?verbose=True
* /module-builds/$build_id

Buildrequires is accessible like this:

    br_module_names = response_json['buildrequires'].keys()
    br_module_dict = response_json['buildrequires'][module name]

Fixes: FACTORY-2201

Signed-off-by: Chenxiong Qi <cqi@redhat.com>
2018-11-23 17:40:23 +08:00
Luiz Carvalho
086ed4a3f3 Remove dangling -debug* RPMs from final MMD
If an RPM is not included, its correspnding -debug* RPMs should also not
be included.

Also ensure that source RPMs are only ever added to -devel modules if
a binary RPM has been completely excluded from non-devel module.

Internal ref: FACTORY-3263

Signed-off-by: Luiz Carvalho <lucarval@redhat.com>
2018-11-21 13:56:20 -05:00
Jan Kaluza
fe88cffa21 Require non-devel module in -devel module. Clear API list and profiles for -devel. 2018-11-21 15:16:36 +01:00