We use module builds as an intermediate build for building flatpaked
applications on Fedora. As Flatpaks in Fedora are officially supported
only on aarch64 and x86_64 we wanted to limit the builds just to these
architectures to save Fedora resources. We were able to do with commit
https://src.fedoraproject.org/modules/flatpak-common/c/65a01f which
works perfectly in koji/mbs, but doesn't work when run locally as the
build fails with following errors and exceptions:
info: Getting tag for flatpak-common:f36:3620220516070452
info: Start to handle flatpak-common:f36:3620220516070452:cab77b58 which is in init state.
Traceback (most recent call last):
File "/usr/lib/python3.10/site-packages/module_build_service/scheduler/handlers/modules.py", line 182, in init
record_module_build_arches(mmd, build)
File "/usr/lib/python3.10/site-packages/module_build_service/scheduler/submit.py", line 150, in record_module_build_arches
arches = get_build_arches(mmd, conf)
File "/usr/lib/python3.10/site-packages/module_build_service/scheduler/submit.py", line 95, in get_build_arches
new_arches = _check_buildopts_arches(mmd, arches)
File "/usr/lib/python3.10/site-packages/module_build_service/scheduler/submit.py", line 131, in _check_buildopts_arches
print(arches not in unsupported_arches, file=sys.stderr)
TypeError: unhashable type: 'list'
info: State transition: 'init' -> 'failed', <ModuleBuild flatpak-common, id=2, stream=f36, version=3620220516070452, scratch=False, state 'failed', batch 0, state_reason 'An unknown error occurred while validating the modulemd'>
warning: Note that retrieved module state 4 doesn't match message module state 'failed'
Traceback (most recent call last):
File "/usr/bin/mbs-manager", line 33, in <module>
sys.exit(load_entry_point('module-build-service==3.6.1', 'console_scripts', 'mbs-manager')())
File "/usr/lib/python3.10/site-packages/click/core.py", line 1137, in __call__
return self.main(*args, **kwargs)
File "/usr/lib/python3.10/site-packages/flask/cli.py", line 596, in main
return super().main(*args, **kwargs)
File "/usr/lib/python3.10/site-packages/click/core.py", line 1062, in main
rv = self.invoke(ctx)
File "/usr/lib/python3.10/site-packages/click/core.py", line 1668, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/usr/lib/python3.10/site-packages/click/core.py", line 1404, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/usr/lib/python3.10/site-packages/click/core.py", line 763, in invoke
return __callback(*args, **kwargs)
File "/usr/lib/python3.10/site-packages/click/decorators.py", line 26, in new_func
return f(get_current_context(), *args, **kwargs)
File "/usr/lib/python3.10/site-packages/flask/cli.py", line 440, in decorator
return __ctx.invoke(f, *args, **kwargs)
File "/usr/lib/python3.10/site-packages/click/core.py", line 763, in invoke
return __callback(*args, **kwargs)
File "/usr/lib/python3.10/site-packages/module_build_service/manage.py", line 171, in build_module_locally
module_build_service.scheduler.local.main(module_build_ids)
File "/usr/lib/python3.10/site-packages/module_build_service/scheduler/local.py", line 57, in main
raise_for_failed_build(module_build_ids)
File "/usr/lib/python3.10/site-packages/module_build_service/scheduler/local.py", line 39, in raise_for_failed_build
raise ValueError("Local module build failed.")
ValueError: Local module build failed.
The problem is that the code as it's now will fail to proceed if it will
detect any unsupported architecture - in this case the aarch64 even
tough the local x86_64 is supported. The check should be redone so if
there's an unsupported architecture detected, we should check that isn't
not the local one and proceed with the build, otherwise fail (the local
hardware doesn't support any of the specified architectures).
There is a need to rebuild the module builds done in CentOS 9 Stream
internally in MBS to include them in RHEL. This is currenly a hard task,
because the RPM components included in a module are usually
taken from HEAD of the branch defined by their `ref` value.
For the rebuild task, it means we would have to ensure that the HEAD
of all RPM components points to right commit hash right before we start
rebuilding CentOS 9 Stream module in internal MBS. This is very hard
and fragile thing to do, especially if there are two different modules
using the RPM component from the same branch. This is prone to race
condition and makes the rebuilds quite complex and in some cases
not possible to do without force pushes to RPM component repositories
which is not acceptable by internal dist-git policy.
This commit fixes it by allowing overriding the commit hash while
submitting the module build. This helps in the mentioned situation,
because we can keep internal RPM components branches in 1:1 sync with
CentOS 9 Stream branches and HEAD can always point to the same commit
in both internal and CentOS 9 Stream repositories.
When the module rebuild is submitted in internal MBS,
we can use this new feature to override the `ref` for each RPM component
so it points to particular commit and the requirement for HEAD to point
to this commit is no longer there.
The `ref` is overriden only internally in MBS (but it is recorded in logs
and in XMD section), so the input modulemd file is not altered. This is
the same logic as used for other overrides (`buildrequire_overrides` or
`side_tag`).
This does not bring any security problem, because it is already possible
to use commit hash in `ref`, so the package maintainer can already change
the commit hash to any particular commit by using this `ref` value.
Signed-off-by: Jan Kaluza <jkaluza@redhat.com>
This function could get called multiple times if the init handler
runs more than once. This can happen if the build failed in the
init handler due to external infrastructure being down and the
user resumes their build.
This also includes `from __future__ import absolute_import`
in every file so that the imports are consistent in Python 2 and 3.
The Python 2 tests fail without this.
This moves the code used by the backend and API to common/submit.py,
the code used just by the API to web/submit.py, and the code used
just by the backend to scheduler/submit.py.
