This also moves the methods load_mmd and load_mmd_file to
module_build_service.utils.general.
This also removes some MSE unit tests with a mix of positive and
negative streams since this is not supported in libmodulemd v2. The
user will be presented with a syntax error if they try to submit
such a modulemd file.
Previously MockModuleBuilder was checking the module state to see if
it should run a final createrepo, but since eafa93037f, finalize() is
called before changing the module state; add an explicit boolean to
GenericBuilder.finalize() to avoid worrying about ordering.
This is needed for offline local builds to build a component which is
stored on local git repository.
This PR also adds OfflineLocalBuildConfiguration configuration class
for offline local builds to set the RESOLVER.
There are following changes introduced in this commit:
- The `koji_tag` of module builds imported from the local repositories
is now in `repofile:///etc/yum.repos.d/some.repo` format to store the
repository from which the module was imported to local MBS DB.
- The `koji_tag` of fake base module is set to empty `repofile://`
and in `MockModuleBuilder` the `conf.base_module_repofiles` list
is used as source for the repositories defining platform. We can't
simply use single repository, because there might be fedora.repo
and fedora-update.repo and so on.
- The list of default .repo files for platform are passed using the
`-r` switch in `build_module_locally` `mbs-manager` command.
- The LocalResolver (subclass of DBResolver) is added which is used
to resolve the build dependencies when building modules offline
locally.
- The `MockModuleBuilder` enables the buildrequired modules and
repositories from which they come in the mock config.
With this commit, it is possible to build testmodule locally
without any external infra.
- Keep scratch module builds in the 'done' state.
- Make koji tagging for scratch modules unique so the same
commit can be resubmitted.
- Use alternate prefix for scratch module build components so they can
be identified later.
- Prevent scratch build components from being reused.
- Assorted code and comment cleanup.
Signed-off-by: Merlin Mathesius <mmathesi@redhat.com>
MBS calls some read-only Koji APIs which does not require to log into a
session. This patch makes it optional to choose whether to login a
session and use anonymous session properly to call those read-only APIs.
Signed-off-by: Chenxiong Qi <cqi@redhat.com>
The build steps assume that the the repository for the build is non-modular,
so after the final createrepo is run with include_module_yaml=True, it's
impossible to repeat a build steps. Running mock -c <failed_config> build is
quite useful for debugging a failed build (even though it's hard to find the
right config) - skipping the final createrepo step for failed module builds
makes it work as expected.
First, rename LocalBuilder to SRPMBuilder. Reading the manpage for mock tells
me that the `--rebuild` option is specifically for rebuilding a given srpm.
The class was just named poorly (imo).
Second, change the type detection. The bug I'm trying to fix is if you try to
pass a string for the `repository:` in your modulemd like
`file:///home/user/repos/libfoo/`. The old logic here would assume that it is
an srpm since it didn't start with "git" or "http", but that's not correct. We
have people who want to experiment with building modules with components that
are not publicly accessible over the network - which are only local. This
change allows that.
I discovered that local builds have been broken by recent (good) changes in
dnf. At the end of every batch, we regenerate the local repo with createrepo
and we also call modifyrepo to include the modulemd file as we progress. This
was added in #467.
What we really want, is for the modulemd file to be present at the *end* of the
build.
Recently, dnf started respecting the modulemd file natively, such that builds
we built in batch0 would not show up in batch1. They would be present in the
repo, but they would be marked as belonging to a module which was not enabled,
and so could not be pulled in. Every module build would fail because
module-srpm-macros was in a disabled module (the module being built at the
time).
This change makes it so that the module metadata is only added to the repo at
the very end of the build. I moved it into a `finalize` method on the builder
which the copr builder was using, and for symmetry's sake I moved the koji
content generator code to the same method on that builder.
There was a circular import issue to solve between the koji module builder and
the koji content generator modules that generated a larger diff, but is mostly
cosmetic and mock changes.
The built RPM artifacts were being added to the components list (the
RPMs to build) instead of the proper artifacts section. Also, they
need to be in NEVRA format, so use 'rpm -qf' to query the epochs from
the built artifacts.
Multiple build threads would try to write to the same distgit-clone-wrapper
location, which could result in ETXTBSY when one thread tried to execute
a script that another was writing to or vice-versa.
Instead of using a wrapper script, use a sh -c invocation for distgit_get.
https://pagure.io/fm-orchestrator/issue/592
There's no good reason to have artifacts in both the thread-specific results directory
and in the main results directory, so just move them across instead of copying them.
https://pagure.io/fm-orchestrator/issue/591
createrepo_c recursively looks for RPMs in the target directory and
subdirectories, meaning that it was finding RPMs in the thread-specific
directories. Pass in an explicit list of RPMs at the toplevel.
https://pagure.io/fm-orchestrator/issue/591
Both pungi and dnf use the new multidocument format, with the
old one not really being supported by anything at this point.
Let's be compatible.
This change requires modulemd-1.2.0+.
Signed-off-by: Petr Šabata <contyk@redhat.com>
