Previously MockModuleBuilder was checking the module state to see if
it should run a final createrepo, but since eafa93037f, finalize() is
called before changing the module state; add an explicit boolean to
GenericBuilder.finalize() to avoid worrying about ordering.
There are following changes introduced in this commit:
- The `koji_tag` of module builds imported from the local repositories
is now in `repofile:///etc/yum.repos.d/some.repo` format to store the
repository from which the module was imported to local MBS DB.
- The `koji_tag` of fake base module is set to empty `repofile://`
and in `MockModuleBuilder` the `conf.base_module_repofiles` list
is used as source for the repositories defining platform. We can't
simply use single repository, because there might be fedora.repo
and fedora-update.repo and so on.
- The list of default .repo files for platform are passed using the
`-r` switch in `build_module_locally` `mbs-manager` command.
- The LocalResolver (subclass of DBResolver) is added which is used
to resolve the build dependencies when building modules offline
locally.
- The `MockModuleBuilder` enables the buildrequired modules and
repositories from which they come in the mock config.
With this commit, it is possible to build testmodule locally
without any external infra.
The `KojiContentGenerator.finalize()` needs the `ModuleBuild.time_completed`.
We currently set `time_completed` once the module build transitions
into `done` state. But we have moved the `KojiContentGenerator` call
to end of `build` state, so right now it is called before the `time_completed`
is set. This leads to traceback.
In this commit, the `time_completed` is set before the `KojiContentGenerator`
call, so it is defined properly.
- Keep scratch module builds in the 'done' state.
- Make koji tagging for scratch modules unique so the same
commit can be resubmitted.
- Use alternate prefix for scratch module build components so they can
be identified later.
- Prevent scratch build components from being reused.
- Assorted code and comment cleanup.
Signed-off-by: Merlin Mathesius <mmathesi@redhat.com>
We encountered an issue where start_next_batch_build was called twice on a
module build and it caused the module build to fail because Brew failed
one of the components since it was submitted twice. This will help us narrow
down the issue if it happens again.
This commit introduces new to_text_type helper method and calls it
for return value of all mmd.dumps() calls. That way, we always
end up with proper unicode string represntation on both python
major versions.
This commit also adds unicode character to description of all
the yaml files we use in the tests so we can be sure MBS can
handle unicode characters properly.
This might be temporary fix, depending on the result of discussion
at https://github.com/fedora-modularity/libmodulemd/issues/184.
Adds new meethod checking the "tagged" and "tagged_in_final" attributes of
"complete" ComponentBuilds in the current batch of module builds
in "building" state against the Koji.
In case the Koji shows the build as tagged/tagged_in_final,
the fake "tagged" message is added to work queue.
Client tool like module-build-watch and `module-build --watch' could
output failed reason. However, original message "Some components failed
to buld" is not informative enough. Instead, list package names of
failed builds would be helpful.
Signed-off-by: Chenxiong Qi <cqi@redhat.com>
This fixes issues when UMB message delivery from frontend to backend fails
for whatever reason...
We do the same thing for 'wait' state already, so this commit just extends
it to 'init' state too.
MBS calls some read-only Koji APIs which does not require to log into a
session. This patch makes it optional to choose whether to login a
session and use anonymous session properly to call those read-only APIs.
Signed-off-by: Chenxiong Qi <cqi@redhat.com>
For tracking error easily when custom exception is raised, log exception
as well before transitioning to failed state.
Signed-off-by: Chenxiong Qi <cqi@redhat.com>
The value referenced by variable dependencies is a list of koji_tags.
So, it would be more straightforward to use an alternative name like
dep_koji_tags than dependencies which is too general.
Signed-off-by: Chenxiong Qi <cqi@redhat.com>
Both tag and dependencies are set directly by calling private method
_get_deps_and_tag. There is no need to set initial value for them
separately.
Signed-off-by: Chenxiong Qi <cqi@redhat.com>
system_resolver is created based on loaded configuration, which could
avoid calls like `GenericResolver.create(conf)` repeatedly in the code.
However, if some cases need to create a specific resolver explicitly,
`GenericResolver.create` could be called with addition argument, for
example db or mbs is passed to argument backend in tests.
Signed-off-by: Chenxiong Qi <cqi@redhat.com>
This addresses an issue when a module contains builds that have different
names in their NVR than the component name in MBS. This happens with
SCLs for instance. MBS wouldn't be able to find the component to mark
as tagged in the database since it was searching by the wrong name, so the
build would just stall. This uses the NVR now to find the correct build
in the database.
I discovered that local builds have been broken by recent (good) changes in
dnf. At the end of every batch, we regenerate the local repo with createrepo
and we also call modifyrepo to include the modulemd file as we progress. This
was added in #467.
What we really want, is for the modulemd file to be present at the *end* of the
build.
Recently, dnf started respecting the modulemd file natively, such that builds
we built in batch0 would not show up in batch1. They would be present in the
repo, but they would be marked as belonging to a module which was not enabled,
and so could not be pulled in. Every module build would fail because
module-srpm-macros was in a disabled module (the module being built at the
time).
This change makes it so that the module metadata is only added to the repo at
the very end of the build. I moved it into a `finalize` method on the builder
which the copr builder was using, and for symmetry's sake I moved the koji
content generator code to the same method on that builder.
There was a circular import issue to solve between the koji module builder and
the koji content generator modules that generated a larger diff, but is mostly
cosmetic and mock changes.
koji now supports tags with max length of 256, we can use
more informative tag name instead of the hash one.
The new format of koji tag name is:
module-<name>-<stream>-<version>-<context>
However when the generated tag's length > (256 - len('build')), we
fallback to the old way of name in hash format (module-<hash>).
In this change, koji tag is always generated from MBS itself, even
with pdc resolver.
FIXES: #918#925
