* Add docstring
* Call dict.setdefault to simplify the code a little which constructs the map
from package name to built RPMs' NVRs.
Signed-off-by: Chenxiong Qi <cqi@redhat.com>
This fixes module builds which sets the `buildopts` for something else
than the RPM whitelist.
Before this commit, when `buildopts` was set, the RPM whitelist was
always taken from the buildopts even if it was not defined there.
This resulted in empty `rpm_whitelist` and therefore no pkglist set
in Koji tag.
In this commit, MBS uses whitelist from `buildopts` only if it is
really set by the module maintainer.
Currently, we are using just `conf.arches` and `conf.base_module_arches`
to define the list of arches for which the RPMs in a submitted module are
built. This is not enough, because RCM needs to generate modules based
on the base modules which should use different arches.
This commit changes the MBS to take the list of arches from the buildrequired
module build. It checks the buildrequires for "privileged" module or base
module and if it finds such module, it queries the Koji to find out the list
of arches to set for the module.
The "privileged" module is a module which can override base module arches
or disttag. Previously, these modules have been defined by
`allowed_disttag_marking_module_names` config option. In this commit,
this has been renamed to `allowed_privileged_module_names`.
The list of arches are stored per module build in new table represented
by ModuleArch class and are m:n mapped to ModuleBuild.
This also moves the methods load_mmd and load_mmd_file to
module_build_service.utils.general.
This also removes some MSE unit tests with a mix of positive and
negative streams since this is not supported in libmodulemd v2. The
user will be presented with a syntax error if they try to submit
such a modulemd file.
Previously MockModuleBuilder was checking the module state to see if
it should run a final createrepo, but since eafa93037f, finalize() is
called before changing the module state; add an explicit boolean to
GenericBuilder.finalize() to avoid worrying about ordering.
There are following changes introduced in this commit:
- The `koji_tag` of module builds imported from the local repositories
is now in `repofile:///etc/yum.repos.d/some.repo` format to store the
repository from which the module was imported to local MBS DB.
- The `koji_tag` of fake base module is set to empty `repofile://`
and in `MockModuleBuilder` the `conf.base_module_repofiles` list
is used as source for the repositories defining platform. We can't
simply use single repository, because there might be fedora.repo
and fedora-update.repo and so on.
- The list of default .repo files for platform are passed using the
`-r` switch in `build_module_locally` `mbs-manager` command.
- The LocalResolver (subclass of DBResolver) is added which is used
to resolve the build dependencies when building modules offline
locally.
- The `MockModuleBuilder` enables the buildrequired modules and
repositories from which they come in the mock config.
With this commit, it is possible to build testmodule locally
without any external infra.
The Koji tag extra options used to be hard-coded and to change them,
we had to release new MBS version.
We do not change them often, but right now fedora-infra is requesting
to use new `mock.new_chroot` option and we need to release new MBS
because of that.
This commit makes such changes easier in the future.
Currently, the Koji tags have properly set the scrmod- prefix, but
the build target still sets module- prefix even for scratch builds.
In this commit, build target has the scrmod- prefix too.
- Keep scratch module builds in the 'done' state.
- Make koji tagging for scratch modules unique so the same
commit can be resubmitted.
- Use alternate prefix for scratch module build components so they can
be identified later.
- Prevent scratch build components from being reused.
- Assorted code and comment cleanup.
Signed-off-by: Merlin Mathesius <mmathesi@redhat.com>
Fedora 30 now includes rpm-4.14.2.1-4.fc30 with support for
ModularityLabel. We're not using the DistTag anywhere anymore and it
should be safe to drop it at this point.
Signed-off-by: Petr Šabata <contyk@redhat.com>
This commit includes the backport of the changes to `krb_login` in
https://pagure.io/koji/pull-request/1187. This change is required
for our separate threads to use a separate Kerberos context per thread.
When using a single Kerberos cache that is shared among threads,
Koji logins start failing because the cache gets corrupt. This uses
the Linux kernel keyring to store a Kerberos cache per MBS thread.
See https://web.mit.edu/kerberos/krb5-1.12/doc/basic/ccache_def.html
MBS calls some read-only Koji APIs which does not require to log into a
session. This patch makes it optional to choose whether to login a
session and use anonymous session properly to call those read-only APIs.
Signed-off-by: Chenxiong Qi <cqi@redhat.com>
This resolve the stream collision by adding specific RPMs to
module-build-macros SRPM as Conflicts.
For more information about module stream collision, please refer to
docstring in utils/ursine.py
Signed-off-by: Chenxiong Qi <cqi@redhat.com>
For some modules, modularity-wg wants to be able to ship the "-devel" modules containing
the RPMs which are normally filtered out from the module.
This PR generates second Koji CG module with -devel suffix in a name with final modulemd
files containing the filtered out RPMs.
During I'm reading through the code base to learn MBS, I found out some
typos and minor issues in some other docstrings, and I also found some
docstrings with extra informative description could make it easier to
understand the code.
Signed-off-by: Chenxiong Qi <cqi@redhat.com>
If a module has filters set and it buildrequires itself, issues occur
during component reuse. In that event, the filtered RPMs from the
previous module build will not be available in the buildroot because the NVRs
in the filters will match the RPMs from the reused component.
Co-authored-by: Jan Kaluza <jkaluza@redhat.com>
This addresses an issue when a module contains builds that have different
names in their NVR than the component name in MBS. This happens with
SCLs for instance. MBS wouldn't be able to find the component to mark
as tagged in the database since it was searching by the wrong name, so the
build would just stall. This uses the NVR now to find the correct build
in the database.
Previously MBS configured all Koji build tags with
repo_include_all=True extra option. But for some modules it was
desired to be able to have tags with repo_include_all=False.
Fixes#957
No reason to hide module context from the consumers, plus it's useful to
keep it in the DISTTAG tag (not to be confused with the %{dist} tag) for
tracking.
The %{disttag} feature will do nothing until RPM supports it. This is
planned in 4.14.2.
See https://bugzilla.redhat.com/show_bug.cgi?id=1596192.
Signed-off-by: Petr Šabata <contyk@redhat.com>
Signed-off-by: Martin Curlej <mcurlej@redhat.com>
Updated PR according to review.
Signed-off-by: Martin Curlej <mcurlej@redhat.com>
Searching by multiple koji tags + tests
Signed-off-by: Martin Curlej <mcurlej@redhat.com>
