Please note that this patch does not change the use of database session
in MBS. So, in the frontend, the database session is still managed by
Flask-SQLAlchemy, that is the db.session. And the backend, running event
handlers, has its own database session created from SQLAclehmy session
API directly.
This patch aims to reduce the number of scoped_session created when call
original function make_db_session. For technical detailed information,
please refer to SQLAlchemy documentation Contextual/Thread-local
Sessions.
As a result, a global scoped_session is accessible from the
code running inside backend, both the event handlers and functions
called from handlers. The library code shared by frontend and backend,
like resolvers, has no change.
Similarly, db.session is only used to recreate database for every test.
Signed-off-by: Chenxiong Qi <cqi@redhat.com>
This will result in the error actually being in the state reason
of the failed module build instead of "An unknown error occurred
while validating the modulemd".
This is the first step to have some tests, that we could run against an
MBS instance, to check that it's functionally correct. Ultimately, these
will replace the test scripts (`contrib/test-*`).
This doesn't really do anything yet, but I would like to make sure that
everyone is on the same page regarding how this will be set up.
Signed-off-by: Hunor Csomortáni <csomh@redhat.com>
Query Koji for the real stream name of each module and keep only those matching
requested `stream`.
This needs to be done, because MBS stores the stream name in the "version" field in Koji,
but the "version" field cannot contain "-" character. Therefore MBS replaces all "-"
with "_". This makes it impossible to reconstruct the original stream name from the
"version" field.
We therefore need to ask for real original stream name here and filter out modules based
on this real stream name.
* A significant performance improvement is to avoid repeating call of
mmd.get_xmd() in a looping.
* Function is refactored to avoid call of mmd_to_str twice for a module
build.
* Test using this fixture already has a setup_method to clean database,
so no need to call clean_database again.
* Calculate build context directly without calculating other contexts
together.
Signed-off-by: Chenxiong Qi <cqi@redhat.com>
If KojiResolver is enabled for buildrequired base module and
MBSResolver is used, then `MBSResolver.get_buildrequired_modulemds`
will use KojiResolver to get the list of buildrequired module builds.
Otherwise it uses the current behavior.
To implement this, the `KojiResolver.get_buildrequired_modules` was
split into two methods:
- `get_buildrequired_koji_builds` returns buildrequired Koji builds.
- `get_buildrequired_modules` calls `get_buildrequired_koji_builds`
and finds the corresponding ModuleBuilds in MBS DB.
In this commit, when component reuse code finds out that the base module uses
KojiResolver, it uses the `KojiResolver.get_buildrequired_modules` method
to find out possible modules to reuse and limits the original query just
by the IDs of these modules.
In order to do that, this commit splits the original
`KojiResolver.get_buildrequired_modulemds` into two methods:
- The `get_buildrequired_modules` returning the ModuleBuilds.
- The `get_buildrequired_modulemds` calling the `get_buildrequired_modules`
and returning modulemd metadata.
This also removes the outdated comments around authorship of each
file. If there is still interest in this information, one can just
look at the git history.
For KojiResolver, we need to be able to find out the module to reuse
components from based only on the non-base-module buildrequires. The
base module buildrequirement will be defined by the Koji tag inheritance.
For this, we need to store build_context computed without the base
modules.
For KojiResolver, this method returns always an empty list. The compatible modules are
defined by the Koji tag inheritance, so there is no need to find out the compatible
base modules on MBS side.
This makes `mse.get_base_module_mmds` to ignore virtual streams and just use
the input base module as the only module without finding the compatible
base modules.
This commit:
- Adds KojiResolver class and KojiResolver tests.
- Changes the GenericResolver and its subclasses to pass base_module_mmds
instead of base_module_nsvc to get_buildrequired_modulemds. This is needed,
because KojiResolver needs to access XMD section of base module.
- Implements KojiResolver.get_buildrequired_modulemds to ask Koji for list of
modules tagged in the Koji tag and return their modulemds.
PR #1331 made the assumption that the Ursa Major ursine RPMs were at
xmd["mbs"]["ursine_rpms"], but they are actually at
xmd["mbs"]["buildrequires"]["platform"]["ursine_rpms"]. This commit
handles the ursine RPMs generated by handle_collisions_with_base_module_rpms
separately since the base module the RPMs came from are not tracked in
that function.
Update test test_build_module_locally_set_stream accordingly by not
mocking database session object in order to test against the real
SQLAlchemy Session scope.
Resolves: RHBZ#1752075
Signed-off-by: Chenxiong Qi <cqi@redhat.com>
Currently, we generate `Conflicts` for ursine RPMs conflicting with
modular RPMs only when Ursa Prime is used for the base module. This
commit changes it, so these Conflicts are generated everytime.
The reason is that modular RPMs should always be preferred in the
buildroot over the ursine RPMs no matter what is their NVR. So far,
this has been guarded on Koji side by using external repos, but
we need to move away from external repo or at least use "bare"
merge mode which basically means we won't get this feature for free
from Koji.
The reason why we need to move away from external repos or use "bare"
merge mode is that without this, the Koji removes RPMs sharing the same
name but different version/release from the buildroot and only keeps
the latest one. This is an issue in situation when you need two
versions of single RPM in a buildroot comming from two modules.
This removes support for default_modules_url in the Platform XMD and
gets the list of default name:stream combinations to include in the buildroot
from https://pagure.io/releng/fedora-module-defaults.
Addresses #1402
Before this commit, the base modules used in the `get_reusable_module` have
not been sorted and therefore when `get_reusable_module` tried to find out
the reusable module built against some base module, it could find a module
built against some old version of base module.
This could lead to situation when MBS tried to reuse components from quite
old module despite the fact that newer module build existed.
This commit fixes this by sorting the base modules by stream_version,
so MBS always tries to get the reusable module built against the latest
base module.
