modules/fm-orchestrator
1
0
Fork 0
mirror of https://pagure.io/fm-orchestrator.git synced 2026-04-04 19:28:49 +08:00
Code Issues Actions Packages Projects Releases Wiki Activity

Add documentation about dependency resolution

Browse Source 
This was written with the help of Jan Kaluža.

Co-authored-by: Jan Kaluža <jkaluza@redhat.com>
This commit is contained in:
mprahl
2019-08-02 15:46:09 -04:00
parent 49465fc761
commit 530136a522
1 changed files with 176 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

176
docs/DEPENDENCY_RESOLUTION.rst
View File

@@ -1,3 +1,93 @@
How Dependency Resolution Works in MBS
======================================
#. Evaluate any buildrequire or require overrides submitted manually by the user.
#. If there is none, the MBS configuration of ``BR_STREAM_OVERRIDE_MODULE`` and
``BR_STREAM_OVERRIDE_REGEXES`` are used to parse the branch name, to see if there are any
buildrequires that should be overridden.
#. Perform module stream expansion.
#. If a buildrequire is an empty list, then MBS gets all the latest versions of every available
stream in all contexts.
#. If a buildrequire contains only streams with ``-`` prefix, the list of streams are treated as
a blacklist.
#. For example, if the buildrequire is ``platform: ["-f29"]``, then the module is built for
all the streams of ``platform`` except the ``f29`` one.
#. Get all the compatible latest buildrequires and recursively the requires of the buildrequires.
#. Compatibility is determined by finding the compatible ``platforms`` that the module is
buildrequiring. For example, if you buildrequire ``platform: el8.1.0``, and you buildrequire
module ``foo`` that was built against ``platform: el8.0.0``, then that module ``foo`` is
compatible as a buildrequire. The reason is that all modules that have been built with a lower
``platform`` of the same major version, are considered compatible. These ``platform`` modules,
must also all provide the same virtual stream. In the case of RHEL, that is a virtual stream
of ``el8``.
#. If there are multiple sets of dependencies in the ``dependencies`` list, then these are treated
like all the possible valid combinations, and MBS will build the input module for each
combination if the dependency resolution succeeds for that combination.
#. Resolve all the possible combinations of buildrequires.
#. For each resolved buildrequire combination, a modulemd file is generated based on the original
modulemd, but with the buildrequires/requires changed based on the resolved combinations.
#. If the buildrequire and require streams are the same on the original modulemd for a
particular module, then set the required stream on that module to the same as the
buildrequired stream. For examples, see the table below.
#. In the event that you buildrequire two streams of two different modules, four module builds
are generated.
#. MBS records the resulting buildrequires (and the recursive requires of the buildrequires) in
the ``xmd`` section of the modulemd.
#. MBS records the context in the modulemd file since the buildrequires have been determined.
#. Each generated modulemd file is submitted as a module build.
Examples of buildrequire and require changes from resolved combinations
-----------------------------------------------------------------------
+--------------------------+-------------------------+----------------------------+-----------------------+
| Buildrequires | Requires | Resulting Build #1 | Resulting Build #2 |
+==========================+=========================+============================+=======================+
| .. code:: yaml | .. code:: yaml | .. code:: yaml | .. code:: yaml |
| | | | |
| platform: [f29, f30] | platform: [f29, f30] | buildrequires: | buildrequires: |
| | | - platform: [f29] | - platform: [f30] |
| | | requires: | requires: |
| | | - platform: [f29] | - platform: [f30] |
+--------------------------+-------------------------+----------------------------+-----------------------+
| .. code:: yaml | .. code:: yaml | .. code:: yaml | .. code:: yaml |
| | | | |
| platform: [f29, f30] | platform: [f30] | buildrequires: | buildrequires: |
| | | - platform: [f29] | - platform: [f30] |
| | | requires: | requires: |
| | | - platform: [f30] | - platform: [f30] |
+--------------------------+-------------------------+----------------------------+-----------------------+
| .. code:: yaml | .. code:: yaml | .. code:: yaml | |
| | | | |
| platform: [f30] | platform: [f29, f30] | buildrequires: | |
| | | - platform: [f30] | |
| | | requires: | |
| | | - platform: [f29, f30] | |
+--------------------------+-------------------------+----------------------------+-----------------------+
| .. code:: yaml | .. code:: yaml | .. code:: yaml | |
| | | | |
| platform: [f29] | platform: [f29] | buildrequires: | |
| | | - platform: [f29] | |
| | | requires: | |
| | | - platform: [f29] | |
+--------------------------+-------------------------+----------------------------+-----------------------+
How libsolv Works in MBS
========================
@@ -38,3 +128,89 @@ Libsolv Terms
solvables in the pool.
- **Transaction** - this describes the solution from the solver execution. In MBS, this is always
about installing the solvable that represents the module being built.
How It's Used
-------------
There are two repos initialized in the constructor the ``MMDResolver`` class: ``build`` and
``available``. The ``build`` repo contains solvable objects that are created to represent the input
module to resolve. The ``available`` repo contains the solvable objects of the possible build
dependencies of the input module.
There are two main methods: ``add_modules`` and ``solve``.
add_modules
~~~~~~~~~~~
#. Gets the NSVC of the input module.
#. If the context is set, then it’s treated as a dependency.
#. A solvable object is created in the ``available`` repo with the name, version, and
architecture (hard-coded to "x86_64" since libsolv requires an architecture, but MBS is
architecture agnostic for dependency resolution).
#. Fill in the ``Provides`` for the module by creating a ``Dep`` object.
#. If it’s not a base module, it provides:
#. ``module(foo)``
#. ``module(foo:stream) = 2019``
#. The version is just used to find the latest version by libsolv.
#. If it's a base module, it provides:
#. ``module(platform)``
#. ``module(platform:el8.0.0) = 3``
#. This shouldn't be defined if a stream version is set (see #1334).
#. ``module(platform:el8.0.0) = 80000``
#. ``80000`` is the "stream version" and not the version of the module.
#. For each virtual stream if there is a "stream version":
#. ``module(platform:virtual_stream) = stream_version``
#. Fills in the ``Requires``.
#. ``_deps2reqs`` is called, which translates the elements of the dependencies array in the
modulemd to a libsolv ``Dep`` object. So for a simple example, with the input
``deps = [{'gtk': ['1'], 'foo': ['1']}]``, the resulting ``solv.Dep`` expression will be
``((module(gtk) with module(gtk:1)) and (module(foo) with module(foo:1)))``.
#. Fills in the ``Conflicts``.
#. This is so that modules of the same stream cannot both be used. For example,
``module(bar:1)`` will conflict with ``module(bar)``, meaning any other module that also
provides ``module(bar)``.
#. If the context is not set, it’s treated as the input module.
#. For each buildrequires/requires pair, a solvable object is created in the ``build`` repo with
the name, version, and architecture (always ``src``).
#. The context of the solvable is the index of the buildrequires/requires pair. This is later
used by the MSE code to distinguish the buildrequires/requires pair, and then which to keep in
the final modulemd and which to remove.
#. The requires are filled in by ``_deps2reqs``, as described above.
solve method
~~~~~~~~~~~~
#. The input modulemd is of the input module.
#. For each solvable in the ``build`` repo:
#. Create a libsolv job to "install" the module.
#. Iterate over all possible combinations of streams without trying to parallel install any
module. As part of this iteration, this is done by telling libsolv to favor that combination.
If what libsolv resolves is the same combination that was favored, we know it’s a valid
combination.