This seems to be better behaviour than simply rejecting the module build
completely. MBS still shows warning in the log that it cannot find
any compatible module, but the build continues with the base module
requested in the submitted modulemd.
When NUM_CONCURRENT_BUILDS is 0, then multi-threading is disabled
when submitting builds to Koji. This is not acceptable, so this
commit makes the number of threads configurable.
There was a race condition, when 2 builds where build in the same time.
There was an issue that after one has finished the other reused the new build
for reuse and not the one it started with.
Ticket-ID: FACTORY-3862
Signed-off-by: Martin Curlej <mcurlej@redhat.com>
This commit fixes issue with following situation:
Module `foo` has been built with `buildrequires: platform: [f28]`
and `requires: platform: []`. It can therefore be used as a buildrequirement
on any platform stream. But if you want to build module `app` which
buildrequires `foo` on `platform:f30`, the MBS won't pull-in the `foo`
module build, because MBS currently limits the modules by the platform
they have been built for.
This commit adds new config option to allow including modules built
against any platform stream.
To do this, we need to use the same database session in import_mmd as in
ModuleBuild.get_buildrequired_base_modules, otherwise, the returned
ModuleBuild objects are in a detached state.
Currently, we are using just `conf.arches` and `conf.base_module_arches`
to define the list of arches for which the RPMs in a submitted module are
built. This is not enough, because RCM needs to generate modules based
on the base modules which should use different arches.
This commit changes the MBS to take the list of arches from the buildrequired
module build. It checks the buildrequires for "privileged" module or base
module and if it finds such module, it queries the Koji to find out the list
of arches to set for the module.
The "privileged" module is a module which can override base module arches
or disttag. Previously, these modules have been defined by
`allowed_disttag_marking_module_names` config option. In this commit,
this has been renamed to `allowed_privileged_module_names`.
The list of arches are stored per module build in new table represented
by ModuleArch class and are m:n mapped to ModuleBuild.
In certain use-cases, a module's buildrequires may remain the same
in the modulemd, but a different support stream of the buildrequired
base module should be used. For example, since RHEL 8.0.0 is GA, any
modules that buildrequire platform:el8.0.0 should buildrequire
platform:el8.0.0z instead.
This also moves the methods load_mmd and load_mmd_file to
module_build_service.utils.general.
This also removes some MSE unit tests with a mix of positive and
negative streams since this is not supported in libmodulemd v2. The
user will be presented with a syntax error if they try to submit
such a modulemd file.
GenericResolver.extract_modulemd is not removed, but deprecated. Call of it
will result in a deprecation message printed. Any new code should call
load_mmd.
Signed-off-by: Chenxiong Qi <cqi@redhat.com>
A base module's stream (the platform for RHEL) could have Z-stream suffix, e.g.
el8.0.0.z, this patch handles this Z-stream suffix and other potential streams
by returning the stream version as a float with configured suffix value. For
example, el8.1.0.z would be parsed as 080100.1. Note that, the 0.1 is totally
configured in config and it actually could be any value according to concrete
cases in practice.
Config STREAM_SUFFIXES is enabled in TestConfiguration so that tests depending
on the return value from ModuleBuild.get_stream_version are covered.
Part fixture of test TestMMDResolver.test_solve_virtual_streams is updated by
adding Z-stream suffix to platform:el8.2.0 in order to ensure this patch does
not break the MMD resolver.
Addresses FACTORY-4307
Signed-off-by: Chenxiong Qi <cqi@redhat.com>
The original description of argument stream is inconsistent with the value
passed to function _expand_mse_streams. The actual passed-in value is
Modulemd.SimpleSet. To fix this inconsistency, just pass list of streams, each
of them is a string representing a stream.
Signed-off-by: Chenxiong Qi <cqi@redhat.com>
Currently, the PLATFORM_ID is parsed from the `/etc/os-release`. This
is good default value, but sometimes you want to build module locally
against the different platform stream.
For example building on platform:f29 against the platform:f30 modules. In
that case, we need to be able to override the host PLATFORM_ID and
set it manually chosen value.
When importing modules for offline local builds from local repositories,
the XMD section does not have to be set at all - it gets removed during
the compose and is also MBS specific.
We need to be able to import such MMDs using the `import_mmd` method
in order to make --offline local builds working.
This commit adds new `check_buildrequires` bool kwarg in `import_mmd`
method to disable `xmd["buildrequires"]` checks to fix this.
If base module name:stream is not found, consider `stream` as virtual stream
and return the latest (the one with highest stream_version) base module with
that virtual stream.
These module builds will basically act as metadata-only module builds.
This will be more useful as additional features stem from these types
of builds.
There are following changes introduced in this commit:
- The `koji_tag` of module builds imported from the local repositories
is now in `repofile:///etc/yum.repos.d/some.repo` format to store the
repository from which the module was imported to local MBS DB.
- The `koji_tag` of fake base module is set to empty `repofile://`
and in `MockModuleBuilder` the `conf.base_module_repofiles` list
is used as source for the repositories defining platform. We can't
simply use single repository, because there might be fedora.repo
and fedora-update.repo and so on.
- The list of default .repo files for platform are passed using the
`-r` switch in `build_module_locally` `mbs-manager` command.
- The LocalResolver (subclass of DBResolver) is added which is used
to resolve the build dependencies when building modules offline
locally.
- The `MockModuleBuilder` enables the buildrequired modules and
repositories from which they come in the mock config.
With this commit, it is possible to build testmodule locally
without any external infra.
