2055 Commits

Author SHA1 Message Date
mprahl
61d8b22fc3 Release v2.9.1 v2.9.1 2018-12-05 09:14:44 -05:00
mprahl
9604da1b22 Fix a flake8 error 2018-12-05 09:13:26 -05:00
Chenxiong Qi
5579a0e72b Init event handler handles module stream collision
Fixes: FACTORY-3616

Signed-off-by: Chenxiong Qi <cqi@redhat.com>
2018-12-05 18:25:14 +08:00
mprahl
676bef79f2 Release v2.9.0 v2.9.0 2018-12-04 13:46:56 -05:00
Matt Prahl
01f2571024 Merge #1087 Fix rebuild strategy name changed-and-after in README 2018-12-04 16:57:49 +00:00
Matt Prahl
8d36d97a5d Merge #1085 Show expanded buildrequires in API output 2018-12-04 16:57:12 +00:00
Chenxiong Qi
0c642a0944 Fix deprecation warnings from log.warn and inspect.getargspec
Signed-off-by: Chenxiong Qi <cqi@redhat.com>
2018-12-04 18:35:56 +08:00
mprahl
25b6a93a07 Make the devel Koji CG build optional using a config option 2018-12-03 12:00:30 -05:00
Petr Šabata
6abb585cdc Define ModularityLabel RPM header
We intend to stop using the DistTag header and replace it with
ModularityLabel.  Upstream RPM already supports it but it is not yet
available in Fedora.

Related Modularity ticket: https://pagure.io/modularity/issue/113

Related RHBZ: https://bugzilla.redhat.com/show_bug.cgi?id=1650286

For the time being, we need to keep both.

Signed-off-by: Petr Šabata <contyk@redhat.com>
2018-12-03 11:44:06 +01:00
Chenxiong Qi
2febab9c19 Fix rebuild strategy name changed-and-after in README
Signed-off-by: Chenxiong Qi <cqi@redhat.com>
2018-11-28 22:10:33 +08:00
mprahl
66a76f4fcd Don't clean up Koji build targets on base modules 2018-11-27 15:45:28 -05:00
Chenxiong Qi
41481afc75 Remove unuseful topurl from tests
Signed-off-by: Chenxiong Qi <cqi@redhat.com>
2018-11-27 11:06:34 +08:00
Chenxiong Qi
1cd64d336a Fix some minor issues
* Fix failure test after rebasing on master branch.
* Fix some grammar issues.
* Only check stream collision modules on new created module build.
* Logging message properly.

Signed-off-by: Chenxiong Qi <cqi@redhat.com>
2018-11-27 11:06:34 +08:00
Chenxiong Qi
d517a72de9 Add _get_module in DBResolver and fix _record_ursine_rpms
_record_ursine_rpms needs to get each collision module's koji_tag and
then get built RPMs from that koji_tag eventually. _get_module on each
individual resolver is called to get module metadata accordingly. So,
when running a local build, module metadata is got from remote MBS, and
for a MBS instance, connected database is queried.

Signed-off-by: Chenxiong Qi <cqi@redhat.com>
2018-11-27 11:06:34 +08:00
Chenxiong Qi
476c51e1f4 Log exception for catching custom exception in init handler
For tracking error easily when custom exception is raised, log exception
as well before transitioning to failed state.

Signed-off-by: Chenxiong Qi <cqi@redhat.com>
2018-11-27 11:06:34 +08:00
Chenxiong Qi
5d3ea762c6 Make external repo URL prefix configurable
Not both Fedora and internal Brew uses config topurl as the external
repo's URL prefix. Hence, make it configurable to fulfill this
difference.

Signed-off-by: Chenxiong Qi <cqi@redhat.com>
2018-11-27 11:06:34 +08:00
Chenxiong Qi
917c06ad0c Resolve stream collision with modules added to ursine content
This resolve the stream collision by adding specific RPMs to
module-build-macros SRPM as Conflicts.

For more information about module stream collision, please refer to
docstring in utils/ursine.py

Signed-off-by: Chenxiong Qi <cqi@redhat.com>
2018-11-27 11:06:34 +08:00
Chenxiong Qi
04b5a9a2a7 Show expanded buildrequires in API output
ModuleBuild.json now includes xmd/mbs/buildrequires in the output JSON
data. As a result, these APIs will show buildrequires by default.

* /module-builds/
* /module-builds/?verbose=True
* /module-builds/$build_id

Buildrequires is accessible like this:

    br_module_names = response_json['buildrequires'].keys()
    br_module_dict = response_json['buildrequires'][module name]

Fixes: FACTORY-2201

Signed-off-by: Chenxiong Qi <cqi@redhat.com>
2018-11-23 17:40:23 +08:00
Luiz Carvalho
086ed4a3f3 Remove dangling -debug* RPMs from final MMD
If an RPM is not included, its correspnding -debug* RPMs should also not
be included.

Also ensure that source RPMs are only ever added to -devel modules if
a binary RPM has been completely excluded from non-devel module.

Internal ref: FACTORY-3263

Signed-off-by: Luiz Carvalho <lucarval@redhat.com>
2018-11-21 13:56:20 -05:00
Jan Kaluza
fe88cffa21 Require non-devel module in -devel module. Clear API list and profiles for -devel. 2018-11-21 15:16:36 +01:00
Jan Kaluza
f5e3c81b0c Import original source modulemd file as stored in dist-git to CG build.
There is a need to ship the unchanged source file which was used to build
a module build from legal reasons. The KojiContentGenerator is extended
in this commit to fetch it from SCM URL using the `scm` module and later
attach it as `modulemd.src.txt` to Koji CG build.
2018-11-21 14:22:58 +01:00
Jan Kaluža
98aa2ea19f Merge #1080 Do not retreive rpm info if there is an empty rpm list 2018-11-21 10:10:25 +00:00
Chenxiong Qi
b5860fa039 Make it optional to tag build for CG import
Fixes FACTORY-3467

Signed-off-by: Chenxiong Qi <cqi@redhat.com>
2018-11-21 09:28:03 +08:00
Chenxiong Qi
42429cf043 Refactor _get_deps_and_tag
Signed-off-by: Chenxiong Qi <cqi@redhat.com>
2018-11-20 19:00:48 +08:00
Valerij Maljulin
f3c86541b6 Do not retreive rpm info if there is an empty rpm list.
Fixes #1078

Signed-off-by: Valerij Maljulin <vmaljuli@redhat.com>
2018-11-20 10:45:17 +01:00
Jan Kaluza
343d145180 Do not leak internal data into final MMD files attached to CG build.
This commit introduces KojiContentGenerator._sanitize_mmd method to:
- remove `repository` and `cache` from ComponentRPM in Modulemd.
- remove `mbs` section from `xmd`.

This is done to not leak internal build-only information to final
modulemd files.
2018-11-19 09:51:27 +01:00
Valerij Maljulin
8adce7593b set module build state to 'failed' imediately
Fixes #1009

Signed-off-by: Valerij Maljulin <vmaljuli@redhat.com>
2018-11-15 17:13:34 +01:00
Matt Prahl
290f19d477 Merge #1076 Update the README for OpenShift deployments 2018-11-13 17:24:22 +00:00
Hunor Csomortáni
8ec8a01006 Rename 'koji_arches' to 'arches' in OpenShift config
'koji_arches' was renamed to 'arches' in a previous commit. Update the
configuration in the OpenShift template accordingly.

Signed-off-by: Hunor Csomortáni <csomh@redhat.com>
2018-11-13 15:10:32 +01:00
Hunor Csomortáni
abda4dd8f2 Update the README for OpenShift deployments
The backend container images needs a build argument holding the UMB CA
certificate. Update the README with this information.

Signed-off-by: Hunor Csomortáni <csomh@redhat.com>
2018-11-13 12:39:44 +01:00
Hunor Csomortáni
cded35e33a Increase polling interval
Polling works in weird ways. It's much better to rely on messaging.

Signed-off-by: Hunor Csomortáni <csomh@redhat.com>
2018-11-12 17:56:16 +01:00
Hunor Csomortáni
76e5845632 Indicate 'git' in default repository URLs
Without this, Koji will not treat them as SCM URLs.

Signed-off-by: Hunor Csomortáni <csomh@redhat.com>
2018-11-12 17:56:16 +01:00
Hunor Csomortáni
64b8d8c169 Allow custom SCMs in OpenShift
The template is intended to be used for testing, so it makes sense to
allow submitting builds from custom SCM URLs.

Signed-off-by: Hunor Csomortáni <csomh@redhat.com>
2018-11-12 17:56:16 +01:00
Hunor Csomortáni
438fc1e531 Configure topic prefix in the OpenShift template
Signed-off-by: Hunor Csomortáni <csomh@redhat.com>
2018-11-12 17:56:16 +01:00
Hunor Csomortáni
849ffec729 Add module-build-macros to platform buildroots
Koji builds done for a module have to have this in order to set the
correct %dist tag, for example.

Signed-off-by: Hunor Csomortáni <csomh@redhat.com>
2018-11-12 17:56:16 +01:00
Hunor Csomortáni
236bc69938 Disable authentication in the OpenShift template
In testing contexts there auth providers are still missing. Disable auth
to allow anyone to submit module builds.

Signed-off-by: Hunor Csomortáni <csomh@redhat.com>
2018-11-12 17:56:16 +01:00
Hunor Csomortáni
f68d708f41 Add the test UMB server CA to the backend image
Signed-off-by: Hunor Csomortáni <csomh@redhat.com>
2018-11-12 17:55:54 +01:00
Hunor Csomortáni
d30c50cff0 Use stomppy 3.1.6 in the container image
Using the latest version would require rewriting some of the messaging
pieces.

Signed-off-by: Hunor Csomortáni <csomh@redhat.com>
2018-11-12 17:51:30 +01:00
Valerij Maljulin
6b7ea33d18 limit Koji build architectures
Fixes issue #869

Signed-off-by: Valerij Maljulin <vmaljuli@redhat.com>
2018-11-12 12:44:52 +01:00
Matt Prahl
f28766ea13 Merge #1070 Refactor SCM.get_latest 2018-11-07 14:47:08 +00:00
Matt Prahl
7fe4dc89fc Merge #1072 Minor fixes to wait handler and KojiModuleBuilder 2018-11-07 13:11:15 +00:00
Chenxiong Qi
a1efbaae66 Fix out-dated comment to KojiModuleBuilder.module_build_tag
Signed-off-by: Chenxiong Qi <cqi@redhat.com>
2018-11-07 17:35:42 +08:00
Chenxiong Qi
1ef30f642e Replace name dependencies with dep_koji_tags in wait handler
The value referenced by variable dependencies is a list of koji_tags.
So, it would be more straightforward to use an alternative name like
dep_koji_tags than dependencies which is too general.

Signed-off-by: Chenxiong Qi <cqi@redhat.com>
2018-11-07 17:31:40 +08:00
Chenxiong Qi
bc682ff15d No need set initial value to tag and dependencies in wait handler
Both tag and dependencies are set directly by calling private method
_get_deps_and_tag. There is no need to set initial value for them
separately.

Signed-off-by: Chenxiong Qi <cqi@redhat.com>
2018-11-07 17:09:21 +08:00
Chenxiong Qi
d6407ac5fb Make directory /etc/module-build-service instead of link
rhpkg could be installed inside Vagrant machine to run MBS with internal
infra. rhpkg installs config file into /etc/module-build-service, which
could fail with following error if /etc/module-build-service is a link
to /opt/module_build_service/conf.

error: unpacking of archive failed on file
/etc/module-build-service/redhat.py;5be25ac4: cpio: chown

Signed-off-by: Chenxiong Qi <cqi@redhat.com>
2018-11-07 12:04:22 +08:00
Chenxiong Qi
8aab2688e1 Fix incorrect username appended to mock group
Signed-off-by: Chenxiong Qi <cqi@redhat.com>
2018-11-07 12:04:13 +08:00
Chenxiong Qi
b577b0f9e1 Refactor SCM.get_latest
git-ls-remote accepts paramter refs to just return refs for specific
ones. This refactor uses this parameter to avoid handling full list of
refs from remote repository.

In original code, option --exit-code is passed to git-ls-remote, which
would cause command return exit code 2 when no ref is found from remote
repository. SCM._run raises an error if that happens, but the raised
error is not handled. This refactor catches this error to ensure the
original behavior happens, that is if a ref is not found, treat it as a
commit hash and call SCM.get_full_commit_hash.

Signed-off-by: Chenxiong Qi <cqi@redhat.com>
2018-11-07 11:02:40 +08:00
Matt Prahl
8356848bec Merge #1069 Remove unused variable local_modules from format_mmd 2018-11-06 14:03:47 +00:00
Matt Prahl
fb95b6b6cd Merge #1066 Refine Vagrantfile 2018-11-06 13:29:14 +00:00
Chenxiong Qi
879d0daca0 Detect transitive stream collision
For an explanation of transitive stream collision, please refer to the
source code docstring.

When such collision is detected, error is raised with proper error
message.

Signed-off-by: Chenxiong Qi <cqi@redhat.com>
2018-11-06 20:27:43 +08:00