In case there is lot of components in a module build or there are some
networking issues and we need to retry the "git clone" commands, the
`format_mmd` method can take long time.
If it takes more than 10 minutes, the poller can produce fake event,
because it seems the module build is stuck. This is wrong, because
it would lead to another unexpected init handler call.
In this commit, the `format_mmd` updates the `ModuleBuild.time_modified`
method regularly to prevent poller from sending the unexpected fake
init message.
This is required for monitoring use-cases, where we can have a Kerberos principal for a
service account but no associated account in LDAP to check group membership.
This commit introduces new to_text_type helper method and calls it
for return value of all mmd.dumps() calls. That way, we always
end up with proper unicode string represntation on both python
major versions.
This commit also adds unicode character to description of all
the yaml files we use in the tests so we can be sure MBS can
handle unicode characters properly.
This might be temporary fix, depending on the result of discussion
at https://github.com/fedora-modularity/libmodulemd/issues/184.
This reverts commit ae79b711d8.
This breaks MBS with python3 and fixes only single occurence of this
issue. We want to fix this in libmodulemd or find a better way how
to fix this to work on both python2 and python3.
We have libmodulemd PR open to address this issue on libmodulemd
level: https://github.com/fedora-modularity/libmodulemd/issues/184.
We always set the "context" to DEFAULT_MODULE_CONTEXT from the historical
reasons - the context was not stored in the database before and we stored
just the build_context/runtime_context. But this is no longer true for
some time.
In this commit, the context is respected and stored in the database when
importing module using the `import_mmd` method. If the context is not set
in the imported MMD, the DEFAULT_MODULE_CONTEXT is used.
This is needed for Fedora branching, but it is generally useful.
For example, there is a module buildrequiring "platform:[]". In the time
this module has been built, only platform:f29 existed, so it has been built
just against platform:f29.
After a while, the platform:f30 is released and the maintainer needs
to rebuild the module against platform:f30. Right now, he needs to create
new commit in the module and submit the build, but this will result in useless
rebuild of the module also against platform:f29.
In this commit, MBS allows to resubmit the module build in a case
there are new MSE builds to build. MBS will send all the module builds
back to the user - so the existing builds will be already marked as
"ready" and the newly submitted builds will have the "init" state in
the REST API response.
However, in case when there are no new MSE builds to build, MBS still
sends back the Conflict error as it used to. This is done for backwards
compatibility and also to not confuse the users in case when no new build
has been submitted.
With this command, admins can retire module builds that should no longer
be used as a dependency for other module builds.
Fixes#1021
Signed-off-by: Luiz Carvalho <lucarval@redhat.com>
Due to a logic error, if record_filtered_rpms() was called on a module which
already had filtered_rpms set, the buildrequires xmd information was discarded.
This broke all local builds.
When encoutering a Windows end of line (^M), io.open and open in Python 3
will convert those to UNIX end of lines by default. When reading logs
to compute the checksum, it's important those new lines aren't converted,
to ensure the checksum is correct. This caused issues in Fedora staging
because when cloning down a repo, the repoSpanner output had Windows end
of lines, and this would end up in build.log. The solution is to just read
it as binary so that Python doesn't perform these conversions.
Fedora 30 now includes rpm-4.14.2.1-4.fc30 with support for
ModularityLabel. We're not using the DistTag anywhere anymore and it
should be safe to drop it at this point.
Signed-off-by: Petr Šabata <contyk@redhat.com>
Adds new meethod checking the "tagged" and "tagged_in_final" attributes of
"complete" ComponentBuilds in the current batch of module builds
in "building" state against the Koji.
In case the Koji shows the build as tagged/tagged_in_final,
the fake "tagged" message is added to work queue.
This commit includes the backport of the changes to `krb_login` in
https://pagure.io/koji/pull-request/1187. This change is required
for our separate threads to use a separate Kerberos context per thread.
Client tool like module-build-watch and `module-build --watch' could
output failed reason. However, original message "Some components failed
to buld" is not informative enough. Instead, list package names of
failed builds would be helpful.
Signed-off-by: Chenxiong Qi <cqi@redhat.com>
This fixes issues when UMB message delivery from frontend to backend fails
for whatever reason...
We do the same thing for 'wait' state already, so this commit just extends
it to 'init' state too.
In case the module did not contain any RPM, _koji_rpms_in_tag still
tried to call the session.getRPMHeaders with an empty list. This results
in None being returned, which is something _koji_rpms_in_tag did
not count with and failed with traceback when trying to iterate
that None value.
In this commit, the _koji_rpms_in_tag returns an empty list early
if Koji tag does not contain any RPM. Therefore the
session.getRPMHeaders is not called at all in this case, because it
does not make any sense to get RPM Headers when Koji tag does not
contain any RPM.
When using a single Kerberos cache that is shared among threads,
Koji logins start failing because the cache gets corrupt. This uses
the Linux kernel keyring to store a Kerberos cache per MBS thread.
See https://web.mit.edu/kerberos/krb5-1.12/doc/basic/ccache_def.html
Current code presumes SRPM name always matches the RPM name built out of this SRPM
and only includes it together with the main package in this case. This is wrong
assumption, because usually there are multiple binary RPMs built from single SRPM.
This commit fixes that by including the SRPM NEVRA in `non_devel_source_rpms`
no matter what RPM name is.
The test RPMs are reworked in this commit to match the reality better - especially
the relations between SRPM and RPMs. The case with different SRPM name and RPM
name is also included in the reworked test - dhcp-libs binary RPM built from
dhcp SRPM.
This fixes issues with calling 'python module_build_service/manage.py' directly
from the MBS git repo as part of testing local module builds without installing
MBS.
The issue is that 'import pkg_resources' for some reason tries to load
module_build_service.backports before the module_build_service.app actually
exists. This results in traceback saying that module_build_service.app does
not exist.
This commit fixes this by importing whole module_build_service in backports.py.
