A base module can set xmd.mbs.default_modules_url, which contains a
URL to a list of modules in the format of name:stream separated by
new lines. When a module buildrequires this base module, the list
of default modules are added as buildrequires of the module automatically
unless there are conflicting streams or the default module is not
in the MBS database.
Most of the issues are caused by the use of SQLAlchemy database session. Some
inline comments describe the issues in detail.
Signed-off-by: Chenxiong Qi <cqi@redhat.com>
There was a race condition, when 2 builds where build in the same time.
There was an issue that after one has finished the other reused the new build
for reuse and not the one it started with.
Ticket-ID: FACTORY-3862
Signed-off-by: Martin Curlej <mcurlej@redhat.com>
This commit fixes issue with following situation:
Module `foo` has been built with `buildrequires: platform: [f28]`
and `requires: platform: []`. It can therefore be used as a buildrequirement
on any platform stream. But if you want to build module `app` which
buildrequires `foo` on `platform:f30`, the MBS won't pull-in the `foo`
module build, because MBS currently limits the modules by the platform
they have been built for.
This commit adds new config option to allow including modules built
against any platform stream.
This fixes module builds which sets the `buildopts` for something else
than the RPM whitelist.
Before this commit, when `buildopts` was set, the RPM whitelist was
always taken from the buildopts even if it was not defined there.
This resulted in empty `rpm_whitelist` and therefore no pkglist set
in Koji tag.
In this commit, MBS uses whitelist from `buildopts` only if it is
really set by the module maintainer.
Currently, we are using just `conf.arches` and `conf.base_module_arches`
to define the list of arches for which the RPMs in a submitted module are
built. This is not enough, because RCM needs to generate modules based
on the base modules which should use different arches.
This commit changes the MBS to take the list of arches from the buildrequired
module build. It checks the buildrequires for "privileged" module or base
module and if it finds such module, it queries the Koji to find out the list
of arches to set for the module.
The "privileged" module is a module which can override base module arches
or disttag. Previously, these modules have been defined by
`allowed_disttag_marking_module_names` config option. In this commit,
this has been renamed to `allowed_privileged_module_names`.
The list of arches are stored per module build in new table represented
by ModuleArch class and are m:n mapped to ModuleBuild.
In certain use-cases, a module's buildrequires may remain the same
in the modulemd, but a different support stream of the buildrequired
base module should be used. For example, since RHEL 8.0.0 is GA, any
modules that buildrequire platform:el8.0.0 should buildrequire
platform:el8.0.0z instead.
The context column was added in d83e6897ca. The
migration set `nullable=False`, but the model kept the default of `nullable=True`.
This caused `mbs-manager db migrate` to create a migration for setting the context
collumn to `nullable=True`.
This also moves the methods load_mmd and load_mmd_file to
module_build_service.utils.general.
This also removes some MSE unit tests with a mix of positive and
negative streams since this is not supported in libmodulemd v2. The
user will be presented with a syntax error if they try to submit
such a modulemd file.
This will prevent the need to call `SCM.get_latest` in the constructor,
since not all SCM objects need the commit to the branch. It also fixes
the situation where a component's git repo doesn't have a "master" branch.
See https://pagure.io/fm-orchestrator/issue/1224
When `default_streams` is set, the current code overwrites `stream`
kwarg in the `for` loop handling the `default_streams`.
In this commit, the `stream` kwarg is not overwritten.
Our current code has following issues with -debuginfo/-debugsource handling:
- The foo-debuginfo is included in the MMD only when foo is included there,
but some special packages contain custom -debuginfo sub-packages like
foo-common-debuginfo without matching foo-common sub-package. With our
current code, the foo-common-debuginfo is never included in the final MMD.
- The foo-debugsource is included in the MMD only when foo.src.rpm,
but some special packages contain custom -debuginfo sub-package.
This commit changes the handling of -debuginfo/-debugsource like this:
- The RPMs to include in the final MMD are evaluated in particular order now.
At first we evaluate non-debug RPMs and then debug RPMs.
- When handling the foo-debuginfo/foo-debugsource RPM, we include it in
final MMD only in one of these cases:
- The "foo" is included in the MMD file (it means it is not filtered out).
- The "foo" package does not exist at all (it means only foo-debuginfo exists
and we need to include this package unless filtered out) and in the same time
the SRPM from which this -debuginfo/-debugsource RPM has been built is included
in a final MMD (it means that there is at least some package from this build
included - this handles case when only foo.src.rpm and foo-debugsource.rpm
would be included in a final MMD, which would be wrong.)
We also respect filters here, so it is possible to explicitely filter out also
-debuginfo/-debugsource packages.
GenericResolver.extract_modulemd is not removed, but deprecated. Call of it
will result in a deprecation message printed. Any new code should call
load_mmd.
Signed-off-by: Chenxiong Qi <cqi@redhat.com>
This patch proposes another code style to create module builds and associated
component builds. The major purpose to make this refactor is to make it easier
to follow up the lines of code and understand the test data and the
relationship between module builds and component builds.
Signed-off-by: Chenxiong Qi <cqi@redhat.com>
A base module's stream (the platform for RHEL) could have Z-stream suffix, e.g.
el8.0.0.z, this patch handles this Z-stream suffix and other potential streams
by returning the stream version as a float with configured suffix value. For
example, el8.1.0.z would be parsed as 080100.1. Note that, the 0.1 is totally
configured in config and it actually could be any value according to concrete
cases in practice.
Config STREAM_SUFFIXES is enabled in TestConfiguration so that tests depending
on the return value from ModuleBuild.get_stream_version are covered.
Part fixture of test TestMMDResolver.test_solve_virtual_streams is updated by
adding Z-stream suffix to platform:el8.2.0 in order to ensure this patch does
not break the MMD resolver.
Addresses FACTORY-4307
Signed-off-by: Chenxiong Qi <cqi@redhat.com>
Currently, the PLATFORM_ID is parsed from the `/etc/os-release`. This
is good default value, but sometimes you want to build module locally
against the different platform stream.
For example building on platform:f29 against the platform:f30 modules. In
that case, we need to be able to override the host PLATFORM_ID and
set it manually chosen value.
When importing modules for offline local builds from local repositories,
the XMD section does not have to be set at all - it gets removed during
the compose and is also MBS specific.
We need to be able to import such MMDs using the `import_mmd` method
in order to make --offline local builds working.
This commit adds new `check_buildrequires` bool kwarg in `import_mmd`
method to disable `xmd["buildrequires"]` checks to fix this.
