When using default modules, this feature will add conflicts to
module-build-macros for every RPM in a buildrequired base module
that overlaps with RPMs in the buildrequired modules. This will
prevent them from being available in the buildroot, and thus
ensure that the RPMs from the buildrequired modules (non-base
modules) are used even if they have a lower NVR.
A base module can set xmd.mbs.default_modules_url, which contains a
URL to a list of modules in the format of name:stream separated by
new lines. When a module buildrequires this base module, the list
of default modules are added as buildrequires of the module automatically
unless there are conflicting streams or the default module is not
in the MBS database.
Currently, we are using just `conf.arches` and `conf.base_module_arches`
to define the list of arches for which the RPMs in a submitted module are
built. This is not enough, because RCM needs to generate modules based
on the base modules which should use different arches.
This commit changes the MBS to take the list of arches from the buildrequired
module build. It checks the buildrequires for "privileged" module or base
module and if it finds such module, it queries the Koji to find out the list
of arches to set for the module.
The "privileged" module is a module which can override base module arches
or disttag. Previously, these modules have been defined by
`allowed_disttag_marking_module_names` config option. In this commit,
this has been renamed to `allowed_privileged_module_names`.
The list of arches are stored per module build in new table represented
by ModuleArch class and are m:n mapped to ModuleBuild.
This also moves the methods load_mmd and load_mmd_file to
module_build_service.utils.general.
This also removes some MSE unit tests with a mix of positive and
negative streams since this is not supported in libmodulemd v2. The
user will be presented with a syntax error if they try to submit
such a modulemd file.
Previously MockModuleBuilder was checking the module state to see if
it should run a final createrepo, but since eafa93037f, finalize() is
called before changing the module state; add an explicit boolean to
GenericBuilder.finalize() to avoid worrying about ordering.
There are following changes introduced in this commit:
- The `koji_tag` of module builds imported from the local repositories
is now in `repofile:///etc/yum.repos.d/some.repo` format to store the
repository from which the module was imported to local MBS DB.
- The `koji_tag` of fake base module is set to empty `repofile://`
and in `MockModuleBuilder` the `conf.base_module_repofiles` list
is used as source for the repositories defining platform. We can't
simply use single repository, because there might be fedora.repo
and fedora-update.repo and so on.
- The list of default .repo files for platform are passed using the
`-r` switch in `build_module_locally` `mbs-manager` command.
- The LocalResolver (subclass of DBResolver) is added which is used
to resolve the build dependencies when building modules offline
locally.
- The `MockModuleBuilder` enables the buildrequired modules and
repositories from which they come in the mock config.
With this commit, it is possible to build testmodule locally
without any external infra.
The `KojiContentGenerator.finalize()` needs the `ModuleBuild.time_completed`.
We currently set `time_completed` once the module build transitions
into `done` state. But we have moved the `KojiContentGenerator` call
to end of `build` state, so right now it is called before the `time_completed`
is set. This leads to traceback.
In this commit, the `time_completed` is set before the `KojiContentGenerator`
call, so it is defined properly.
- Keep scratch module builds in the 'done' state.
- Make koji tagging for scratch modules unique so the same
commit can be resubmitted.
- Use alternate prefix for scratch module build components so they can
be identified later.
- Prevent scratch build components from being reused.
- Assorted code and comment cleanup.
Signed-off-by: Merlin Mathesius <mmathesi@redhat.com>
We encountered an issue where start_next_batch_build was called twice on a
module build and it caused the module build to fail because Brew failed
one of the components since it was submitted twice. This will help us narrow
down the issue if it happens again.
This commit introduces new to_text_type helper method and calls it
for return value of all mmd.dumps() calls. That way, we always
end up with proper unicode string represntation on both python
major versions.
This commit also adds unicode character to description of all
the yaml files we use in the tests so we can be sure MBS can
handle unicode characters properly.
This might be temporary fix, depending on the result of discussion
at https://github.com/fedora-modularity/libmodulemd/issues/184.
Adds new meethod checking the "tagged" and "tagged_in_final" attributes of
"complete" ComponentBuilds in the current batch of module builds
in "building" state against the Koji.
In case the Koji shows the build as tagged/tagged_in_final,
the fake "tagged" message is added to work queue.
Client tool like module-build-watch and `module-build --watch' could
output failed reason. However, original message "Some components failed
to buld" is not informative enough. Instead, list package names of
failed builds would be helpful.
Signed-off-by: Chenxiong Qi <cqi@redhat.com>
This fixes issues when UMB message delivery from frontend to backend fails
for whatever reason...
We do the same thing for 'wait' state already, so this commit just extends
it to 'init' state too.
MBS calls some read-only Koji APIs which does not require to log into a
session. This patch makes it optional to choose whether to login a
session and use anonymous session properly to call those read-only APIs.
Signed-off-by: Chenxiong Qi <cqi@redhat.com>
For tracking error easily when custom exception is raised, log exception
as well before transitioning to failed state.
Signed-off-by: Chenxiong Qi <cqi@redhat.com>
The value referenced by variable dependencies is a list of koji_tags.
So, it would be more straightforward to use an alternative name like
dep_koji_tags than dependencies which is too general.
Signed-off-by: Chenxiong Qi <cqi@redhat.com>
Both tag and dependencies are set directly by calling private method
_get_deps_and_tag. There is no need to set initial value for them
separately.
Signed-off-by: Chenxiong Qi <cqi@redhat.com>
system_resolver is created based on loaded configuration, which could
avoid calls like `GenericResolver.create(conf)` repeatedly in the code.
However, if some cases need to create a specific resolver explicitly,
`GenericResolver.create` could be called with addition argument, for
example db or mbs is passed to argument backend in tests.
Signed-off-by: Chenxiong Qi <cqi@redhat.com>
