Commit Graph

78 Commits

Author SHA1 Message Date
Valerij Maljulin
687a78242e Change xmlrpclib importing way
Signed-off-by: Valerij Maljulin <vmaljuli@redhat.com>
2019-02-25 16:56:30 +01:00
Valerij Maljulin
eafa93037f Make finalization before changing state to done
Signed-off-by: Valerij Maljulin <vmaljuli@redhat.com>
2019-02-15 15:19:59 +01:00
Petr Šabata
ff1ed9926a Don't define DistTag anymore
Fedora 30 now includes rpm-4.14.2.1-4.fc30 with support for
ModularityLabel.  We're not using the DistTag anywhere anymore and it
should be safe to drop it at this point.

Signed-off-by: Petr Šabata <contyk@redhat.com>
2019-01-04 12:02:22 +01:00
mprahl
25122cb53e Modify MBS to use a separate Kerberos context per thread so both threads can use the thread keyring to store the Kerberos cache
This commit includes the backport of the changes to `krb_login` in
https://pagure.io/koji/pull-request/1187. This change is required
for our separate threads to use a separate Kerberos context per thread.
2018-12-18 16:05:55 -05:00
mprahl
7f82275114 Force the use of a separte Kerberos cache per thread
When using a single Kerberos cache that is shared among threads,
Koji logins start failing because the cache gets corrupt. This uses
the Linux kernel keyring to store a Kerberos cache per MBS thread.

See https://web.mit.edu/kerberos/krb5-1.12/doc/basic/ccache_def.html
2018-12-12 19:04:00 -05:00
mprahl
96f82443aa Remove the unused "owner" parameter from KojiModuleBuilder.get_session 2018-12-10 09:59:42 -05:00
mprahl
00693cbd00 Improve the readability in KojiModuleBuilder.get_session 2018-12-10 09:54:29 -05:00
Chenxiong Qi
55add5cfc0 Use anonymous Koji session properly
MBS calls some read-only Koji APIs which does not require to log into a
session. This patch makes it optional to choose whether to login a
session and use anonymous session properly to call those read-only APIs.

Signed-off-by: Chenxiong Qi <cqi@redhat.com>
2018-12-06 16:31:12 +08:00
Chenxiong Qi
0c642a0944 Fix deprecation warnings from log.warn and inspect.getargspec
Signed-off-by: Chenxiong Qi <cqi@redhat.com>
2018-12-04 18:35:56 +08:00
mprahl
25b6a93a07 Make the devel Koji CG build optional using a config option 2018-12-03 12:00:30 -05:00
Petr Šabata
6abb585cdc Define ModularityLabel RPM header
We intend to stop using the DistTag header and replace it with
ModularityLabel.  Upstream RPM already supports it but it is not yet
available in Fedora.

Related Modularity ticket: https://pagure.io/modularity/issue/113

Related RHBZ: https://bugzilla.redhat.com/show_bug.cgi?id=1650286

For the time being, we need to keep both.

Signed-off-by: Petr Šabata <contyk@redhat.com>
2018-12-03 11:44:06 +01:00
Chenxiong Qi
917c06ad0c Resolve stream collision with modules added to ursine content
This resolve the stream collision by adding specific RPMs to
module-build-macros SRPM as Conflicts.

For more information about module stream collision, please refer to
docstring in utils/ursine.py

Signed-off-by: Chenxiong Qi <cqi@redhat.com>
2018-11-27 11:06:34 +08:00
Valerij Maljulin
6b7ea33d18 limit Koji build architectures
Fixes issue #869

Signed-off-by: Valerij Maljulin <vmaljuli@redhat.com>
2018-11-12 12:44:52 +01:00
Chenxiong Qi
a1efbaae66 Fix out-dated comment to KojiModuleBuilder.module_build_tag
Signed-off-by: Chenxiong Qi <cqi@redhat.com>
2018-11-07 17:35:42 +08:00
Jan Kaluza
13ab18425a Import -devel CG build with RPMs which are filtered out of the current real CG build.
For some modules, modularity-wg wants to be able to ship the "-devel" modules containing
the RPMs which are normally filtered out from the module.

This PR generates second Koji CG module with -devel suffix in a name with final modulemd
files containing the filtered out RPMs.
2018-10-30 13:49:12 +01:00
Chenxiong Qi
a2ee9b4918 Fix a typo in comment
Signed-off-by: Chenxiong Qi <cqi@redhat.com>
2018-10-20 19:48:30 +08:00
Valerij Maljulin
8ee7168017 get_reusable_component now checks the architecture
Signed-off-by: Valerij Maljulin <vmaljuli@redhat.com>
2018-10-16 13:44:18 +02:00
Valerij Maljulin
7f60db545a renaming koji_arches to arches
Signed-off-by: Valerij Maljulin <vmaljuli@redhat.com>
2018-10-15 15:29:50 +02:00
Chenxiong Qi
2c58471630 Fix and reword some docstrings again
During I'm reading through the code base to learn MBS, I found out some
typos and minor issues in some other docstrings, and I also found some
docstrings with extra informative description could make it easier to
understand the code.

Signed-off-by: Chenxiong Qi <cqi@redhat.com>
2018-09-26 14:12:39 +08:00
Jan Kaluza
56fe201c33 Fill in the 'filterer_rpms' in backend to not access Koji from frontend. 2018-08-21 07:58:52 +02:00
Jan Kaluza
f0b35be067 Add BASE_MODULE_KOJI_ARCHES to override default Koji arches. 2018-08-14 13:18:56 +02:00
mprahl
d54422adeb Don't filter RPMs of reused components from a module that depends on itself
If a module has filters set and it buildrequires itself, issues occur
during component reuse. In that event, the filtered RPMs from the
previous module build will not be available in the buildroot because the NVRs
in the filters will match the RPMs from the reused component.

Co-authored-by: Jan Kaluza <jkaluza@redhat.com>
2018-08-13 11:13:40 -04:00
Jan Kaluza
6d2e4bbffa Fix filters generation 2018-08-09 15:16:13 +02:00
mprahl
f08135352a Use the NVR instead of the component name when responding to a tag message
This addresses an issue when a module contains builds that have different
names in their NVR than the component name in MBS. This happens with
SCLs for instance. MBS wouldn't be able to find the component to mark
as tagged in the database since it was searching by the wrong name, so the
build would just stall. This uses the NVR now to find the correct build
in the database.
2018-08-07 11:54:28 -04:00
mprahl
042b677a01 Raise an exception if the build is not returned from Koji when checking if the buildroot is ready 2018-08-07 10:33:14 -04:00
mprahl
a78c564073 Revert "Store the component's build ID and use that to identify the build when acting on a tag message"
This reverts commit 9bd16beeef.
2018-08-07 10:32:50 -04:00
mprahl
9bd16beeef Store the component's build ID and use that to identify the build when acting on a tag message
This resolves an issue where the component name is different than the
package being tagged due to a macro such as those used for SCLs.
2018-08-03 20:45:00 -04:00
Matt Prahl
e45863e8f4 Merge #967 Define %{_module_context} and %{disttag} 2018-06-29 15:28:20 +00:00
Mikolaj Izdebski
0d29633da9 Make repo_include_all setting configurable in xmd
Previously MBS configured all Koji build tags with
repo_include_all=True extra option.  But for some modules it was
desired to be able to have tags with repo_include_all=False.

Fixes #957
2018-06-29 17:06:58 +02:00
Petr Šabata
323880999b Define %{_module_context} and %{disttag}
No reason to hide module context from the consumers, plus it's useful to
keep it in the DISTTAG tag (not to be confused with the %{dist} tag) for
tracking.

The %{disttag} feature will do nothing until RPM supports it.  This is
planned in 4.14.2.

See https://bugzilla.redhat.com/show_bug.cgi?id=1596192.

Signed-off-by: Petr Šabata <contyk@redhat.com>
2018-06-29 16:36:01 +02:00
Matt Prahl
6e075951b8 Merge #962 Support the modulemd buildopts.rpms.whitelist option 2018-06-27 11:42:49 +00:00
mprahl
74e4c99984 Support the modulemd buildopts.rpms.whitelist option 2018-06-27 07:05:06 -04:00
Martin Curlej
0aa9f014ce Added search for modules by binary rpm.
Signed-off-by: Martin Curlej <mcurlej@redhat.com>

Updated PR according to review.

Signed-off-by: Martin Curlej <mcurlej@redhat.com>

Searching by multiple koji tags + tests

Signed-off-by: Martin Curlej <mcurlej@redhat.com>
2018-06-26 15:52:37 +02:00
Ralph Bean
6981449853 Fix local builds.
I discovered that local builds have been broken by recent (good) changes in
dnf.  At the end of every batch, we regenerate the local repo with createrepo
and we also call modifyrepo to include the modulemd file as we progress.  This
was added in #467.

What we really want, is for the modulemd file to be present at the *end* of the
build.

Recently, dnf started respecting the modulemd file natively, such that builds
we built in batch0 would not show up in batch1.  They would be present in the
repo, but they would be marked as belonging to a module which was not enabled,
and so could not be pulled in.  Every module build would fail because
module-srpm-macros was in a disabled module (the module being built at the
time).

This change makes it so that the module metadata is only added to the repo at
the very end of the build.  I moved it into a `finalize` method on the builder
which the copr builder was using, and for symmetry's sake I moved the koji
content generator code to the same method on that builder.

There was a circular import issue to solve between the koji module builder and
the koji content generator modules that generated a larger diff, but is mostly
cosmetic and mock changes.
2018-06-25 17:05:22 -04:00
Owen W. Taylor
827546dc9a module-build-macros: Use /etc instead of %_sysconfdir
Commit 42aef7d5d2 moved the macros installed by module-build-macros
from %_rpmconfdir (which is unaffected by %_prefix) to %_sysconfdir/rpm
(which follows %_prefix). This interacts badly with the Flatpak module
macros which change %_prefix to /app - when flatpak-rpm-macros is
in the buildroot while module-build-macros is built, module-build-macros
installs its macros to /app and they aren't found when building the
rest of the packages in the module.

Hardcoding /etc rather than using %_sysconfdir fixes this.
2018-05-31 17:52:04 -04:00
mprahl
15a7e322a6 Fallback to the old Koji tag format for the target when the tag name is too long 2018-05-08 09:11:22 -04:00
Jan Kaluza
9236133cd2 Fix wrong rebase of #919. 2018-04-13 13:07:00 +02:00
Jan Kaluza
c1c95fc963 Allow defining list of packages which are blocked in the -build tag until they are built in a module. 2018-04-13 12:57:27 +02:00
Jan Kaluza
5d91c97386 Allow defining list of packages which are blocked in the -build tag until they are built in a module. 2018-04-12 09:45:57 +02:00
mprahl
f0852d9009 Port to libmodulemd and support v2 modulemd without module stream expansion 2018-04-03 09:58:57 -04:00
Yash Nanavati
1d3fd2af64 Minor changes towards Python 3 Compatability 2018-02-28 09:01:51 -05:00
mprahl
df594da8a5 Remove the Koji proxyuser functionality 2018-01-30 11:49:03 -05:00
mprahl
bf80ce4537 Don't traceback on failed task cancellations and log those exceptions 2018-01-22 15:20:41 -05:00
mprahl
a67b53f5c3 Add a "context" field on component and module releases in Koji for uniqueness for when Module Stream Expansion is implemented 2018-01-16 10:36:09 -05:00
mprahl
a54045e518 Don't run KojiModuleBuilder.buildroot_connect in the stale module builds poller 2017-12-11 13:27:30 -05:00
Jan Kaluza
7de120c17b Get the weight of Koji tasks and use it to sort the components in a batch instead of builds time. 2017-11-28 10:52:17 -05:00
Nils Philippsen
3f746582eb honor custom RPM %_sourcedir
If the user redefines %_sourcedir in ~/.rpmmacros, `mbs-build local`
will fail trying to build module-build-macros otherwise because rpmbuild
will look for sources -- macros.modules -- in the wrong place.
2017-11-28 13:49:06 +01:00
mprahl
30ef85af01 Fix "instant complete" builds and reusing all components 2017-11-27 16:20:28 -05:00
mprahl
cf44455ff2 Add a missing docstring to 'untag_artifacts' 2017-11-27 16:20:28 -05:00
mprahl
ef5dc64f2b Transition stale failed builds to the "garbage" state and untag their components 2017-11-17 12:07:51 -05:00