mirror of
https://pagure.io/fm-orchestrator.git
synced 2026-02-03 05:03:43 +08:00
530136a522
This was written with the help of Jan Kaluža. Co-authored-by: Jan Kaluža <jkaluza@redhat.com>
217 lines
12 KiB
ReStructuredText
217 lines
12 KiB
ReStructuredText
How Dependency Resolution Works in MBS
|
||
======================================
|
||
|
||
#. Evaluate any buildrequire or require overrides submitted manually by the user.
|
||
|
||
#. If there is none, the MBS configuration of ``BR_STREAM_OVERRIDE_MODULE`` and
|
||
``BR_STREAM_OVERRIDE_REGEXES`` are used to parse the branch name, to see if there are any
|
||
buildrequires that should be overridden.
|
||
|
||
#. Perform module stream expansion.
|
||
|
||
#. If a buildrequire is an empty list, then MBS gets all the latest versions of every available
|
||
stream in all contexts.
|
||
|
||
#. If a buildrequire contains only streams with ``-`` prefix, the list of streams are treated as
|
||
a blacklist.
|
||
|
||
#. For example, if the buildrequire is ``platform: ["-f29"]``, then the module is built for
|
||
all the streams of ``platform`` except the ``f29`` one.
|
||
|
||
#. Get all the compatible latest buildrequires and recursively the requires of the buildrequires.
|
||
|
||
#. Compatibility is determined by finding the compatible ``platforms`` that the module is
|
||
buildrequiring. For example, if you buildrequire ``platform: el8.1.0``, and you buildrequire
|
||
module ``foo`` that was built against ``platform: el8.0.0``, then that module ``foo`` is
|
||
compatible as a buildrequire. The reason is that all modules that have been built with a lower
|
||
``platform`` of the same major version, are considered compatible. These ``platform`` modules,
|
||
must also all provide the same virtual stream. In the case of RHEL, that is a virtual stream
|
||
of ``el8``.
|
||
|
||
#. If there are multiple sets of dependencies in the ``dependencies`` list, then these are treated
|
||
like all the possible valid combinations, and MBS will build the input module for each
|
||
combination if the dependency resolution succeeds for that combination.
|
||
|
||
#. Resolve all the possible combinations of buildrequires.
|
||
|
||
#. For each resolved buildrequire combination, a modulemd file is generated based on the original
|
||
modulemd, but with the buildrequires/requires changed based on the resolved combinations.
|
||
|
||
#. If the buildrequire and require streams are the same on the original modulemd for a
|
||
particular module, then set the required stream on that module to the same as the
|
||
buildrequired stream. For examples, see the table below.
|
||
|
||
#. In the event that you buildrequire two streams of two different modules, four module builds
|
||
are generated.
|
||
|
||
#. MBS records the resulting buildrequires (and the recursive requires of the buildrequires) in
|
||
the ``xmd`` section of the modulemd.
|
||
|
||
#. MBS records the context in the modulemd file since the buildrequires have been determined.
|
||
|
||
#. Each generated modulemd file is submitted as a module build.
|
||
|
||
|
||
Examples of buildrequire and require changes from resolved combinations
|
||
-----------------------------------------------------------------------
|
||
|
||
+--------------------------+-------------------------+----------------------------+-----------------------+
|
||
| Buildrequires | Requires | Resulting Build #1 | Resulting Build #2 |
|
||
+==========================+=========================+============================+=======================+
|
||
| .. code:: yaml | .. code:: yaml | .. code:: yaml | .. code:: yaml |
|
||
| | | | |
|
||
| platform: [f29, f30] | platform: [f29, f30] | buildrequires: | buildrequires: |
|
||
| | | - platform: [f29] | - platform: [f30] |
|
||
| | | requires: | requires: |
|
||
| | | - platform: [f29] | - platform: [f30] |
|
||
+--------------------------+-------------------------+----------------------------+-----------------------+
|
||
| .. code:: yaml | .. code:: yaml | .. code:: yaml | .. code:: yaml |
|
||
| | | | |
|
||
| platform: [f29, f30] | platform: [f30] | buildrequires: | buildrequires: |
|
||
| | | - platform: [f29] | - platform: [f30] |
|
||
| | | requires: | requires: |
|
||
| | | - platform: [f30] | - platform: [f30] |
|
||
+--------------------------+-------------------------+----------------------------+-----------------------+
|
||
| .. code:: yaml | .. code:: yaml | .. code:: yaml | |
|
||
| | | | |
|
||
| platform: [f30] | platform: [f29, f30] | buildrequires: | |
|
||
| | | - platform: [f30] | |
|
||
| | | requires: | |
|
||
| | | - platform: [f29, f30] | |
|
||
+--------------------------+-------------------------+----------------------------+-----------------------+
|
||
| .. code:: yaml | .. code:: yaml | .. code:: yaml | |
|
||
| | | | |
|
||
| platform: [f29] | platform: [f29] | buildrequires: | |
|
||
| | | - platform: [f29] | |
|
||
| | | requires: | |
|
||
| | | - platform: [f29] | |
|
||
+--------------------------+-------------------------+----------------------------+-----------------------+
|
||
|
||
|
||
How libsolv Works in MBS
|
||
========================
|
||
|
||
Libsolv Terms
|
||
-------------
|
||
|
||
- **Pool** - the main object that represents the libsolv context that all "solvables" get added to
|
||
it.
|
||
- **Dep** - an object used for dependency resolution metadata on a string. For example, a ``Dep``
|
||
object might be "platform:el8" (module name and stream), or it might just be a version of a
|
||
module, such as "80000". With these ``Dep`` objects, we can use operators, such as ``REL_EQ``,
|
||
``REL_OR``, and ``REL_GT`` to define the relation between ``Dep`` objects. For example, the
|
||
"platform:el8" ``Dep`` object would have a ``REL_EQ`` relationship with the "80000" ``Dep``
|
||
object. After creating this relationship, a new ``Dep`` object is returned, with that
|
||
relationship. That ``Dep`` object can then be used in the ``add_deparray`` method, which provides
|
||
a relationship from this ``Dep`` object to a solvable.
|
||
- **Solvable** - an installable artifact with properties such as name, version, release, and arch
|
||
that is created in a repo in the pool. Usually, a solvable represents an RPM, but in the case of
|
||
MBS, it represents a module.
|
||
|
||
- **Requires** - the ``Dep`` objects that this solvable requires to be available in the repo when
|
||
the solvable is installed.
|
||
- **Provides** - the ``Dep`` objects that this solvable provides. For example,
|
||
"platform:el8:0:c1" (NSVC), would also provide "platform:el8" and "platform:el8 = 0"
|
||
(``REL_EQ`` relationship), this way a solvable can require ``platform:el8`` and not the whole
|
||
NSVC.
|
||
- **Conflicts** - the ``Dep`` objects that represent a solvable that cannot be installed when this
|
||
solvable is installed. For example, two modules of the same name but different stream, cannot be
|
||
installed at the same time, so a "foo:bar1" conflicts with "foo:bar2" and vice-versa.
|
||
- **Repo** - a collection of solvables.
|
||
- **Job** - what to do with the solvables in the pool. In MBS, this involves stating the solvable we
|
||
want to install (the module being built), and then the solvables that are preferred in the
|
||
solution, which override the default behavior of libsolv. For example, if the "platform:f28" and
|
||
"platform:f29" solvables are both in the pool (due to Module Stream Expansion), MBS will create
|
||
two sets of jobs, the first which favors "platform:f28", and the second which favors
|
||
"platform:f29". This way, if possible, the dependencies are determined for both platforms.
|
||
- **Solver** - executes the jobs, and finds the best solution for the given jobs based on the
|
||
solvables in the pool.
|
||
- **Transaction** - this describes the solution from the solver execution. In MBS, this is always
|
||
about installing the solvable that represents the module being built.
|
||
|
||
|
||
How It's Used
|
||
-------------
|
||
|
||
There are two repos initialized in the constructor the ``MMDResolver`` class: ``build`` and
|
||
``available``. The ``build`` repo contains solvable objects that are created to represent the input
|
||
module to resolve. The ``available`` repo contains the solvable objects of the possible build
|
||
dependencies of the input module.
|
||
|
||
There are two main methods: ``add_modules`` and ``solve``.
|
||
|
||
add_modules
|
||
~~~~~~~~~~~
|
||
|
||
#. Gets the NSVC of the input module.
|
||
|
||
#. If the context is set, then it’s treated as a dependency.
|
||
|
||
#. A solvable object is created in the ``available`` repo with the name, version, and
|
||
architecture (hard-coded to "x86_64" since libsolv requires an architecture, but MBS is
|
||
architecture agnostic for dependency resolution).
|
||
|
||
#. Fill in the ``Provides`` for the module by creating a ``Dep`` object.
|
||
|
||
#. If it’s not a base module, it provides:
|
||
|
||
#. ``module(foo)``
|
||
#. ``module(foo:stream) = 2019``
|
||
|
||
#. The version is just used to find the latest version by libsolv.
|
||
|
||
#. If it's a base module, it provides:
|
||
|
||
#. ``module(platform)``
|
||
|
||
#. ``module(platform:el8.0.0) = 3``
|
||
|
||
#. This shouldn't be defined if a stream version is set (see #1334).
|
||
|
||
#. ``module(platform:el8.0.0) = 80000``
|
||
|
||
#. ``80000`` is the "stream version" and not the version of the module.
|
||
|
||
#. For each virtual stream if there is a "stream version":
|
||
|
||
#. ``module(platform:virtual_stream) = stream_version``
|
||
|
||
#. Fills in the ``Requires``.
|
||
|
||
#. ``_deps2reqs`` is called, which translates the elements of the dependencies array in the
|
||
modulemd to a libsolv ``Dep`` object. So for a simple example, with the input
|
||
``deps = [{'gtk': ['1'], 'foo': ['1']}]``, the resulting ``solv.Dep`` expression will be
|
||
``((module(gtk) with module(gtk:1)) and (module(foo) with module(foo:1)))``.
|
||
|
||
#. Fills in the ``Conflicts``.
|
||
|
||
#. This is so that modules of the same stream cannot both be used. For example,
|
||
``module(bar:1)`` will conflict with ``module(bar)``, meaning any other module that also
|
||
provides ``module(bar)``.
|
||
|
||
#. If the context is not set, it’s treated as the input module.
|
||
|
||
#. For each buildrequires/requires pair, a solvable object is created in the ``build`` repo with
|
||
the name, version, and architecture (always ``src``).
|
||
|
||
#. The context of the solvable is the index of the buildrequires/requires pair. This is later
|
||
used by the MSE code to distinguish the buildrequires/requires pair, and then which to keep in
|
||
the final modulemd and which to remove.
|
||
|
||
#. The requires are filled in by ``_deps2reqs``, as described above.
|
||
|
||
|
||
solve method
|
||
~~~~~~~~~~~~
|
||
|
||
#. The input modulemd is of the input module.
|
||
|
||
#. For each solvable in the ``build`` repo:
|
||
|
||
#. Create a libsolv job to "install" the module.
|
||
|
||
#. Iterate over all possible combinations of streams without trying to parallel install any
|
||
module. As part of this iteration, this is done by telling libsolv to favor that combination.
|
||
If what libsolv resolves is the same combination that was favored, we know it’s a valid
|
||
combination.
|