Set a consistent set of options on Jenkins pipeline jobs. This includes timestamps,
reasonable timeouts, saving only the last 10 builds (to avoid filling up the disk
of the Jenkins master), disabling concurrent builds (to avoid hitting quota limits),
and skipping the default checkout (all jobs that need the source call checkout()
explicitly).
Previously, the integration test pipeline was configured to always use the tests present on the
master branch. When running the pre-merge pipeline for a PR, this would result in potentially
out-dated tests being run. This change runs the tests from the current branch, allowing the code
and the tests to evolve together.
This also disables the default checkout of the master branch, which is unnecessary when calling
checkout() from the pipeline.
The old way performed a `DISTINCT (module_builds.id)` on the original query passed in
to ModuleBuild._add_virtual_streams_filter, but this caused issues when the original
query was ordered by something other than ID on Postgres databases. This new approach
uses a subquery to filter that module builds with the desired virtual streams, and then
joins this subquery to the original query.
The context column was added in d83e6897ca. The
migration set `nullable=False`, but the model kept the default of `nullable=True`.
This caused `mbs-manager db migrate` to create a migration for setting the context
collumn to `nullable=True`.
This also moves the methods load_mmd and load_mmd_file to
module_build_service.utils.general.
This also removes some MSE unit tests with a mix of positive and
negative streams since this is not supported in libmodulemd v2. The
user will be presented with a syntax error if they try to submit
such a modulemd file.
This will prevent the need to call `SCM.get_latest` in the constructor,
since not all SCM objects need the commit to the branch. It also fixes
the situation where a component's git repo doesn't have a "master" branch.
See https://pagure.io/fm-orchestrator/issue/1224
This change introduces a set of Jenkins pipelines for building MBS
images and running integration tests against Koji using those images.
These pipelines are directly based on the WaiverDB pipeline work:
https://pagure.io/waiverdb/blob/master/f/openshift
The results of those tests are used to provide feedback to Pagure PRs
and to promote images through a series of environments, which may be
used to implement a continuous deployment process.
The current test cases, written in Groovy, are:
- module-build-init: initate a module build and check that tags
and targets in Koji are created correctly
- module-build-cgimport: build an empty module and ensure that
results are imported correctly into Koji, using the CGImport
interface
When `default_streams` is set, the current code overwrites `stream`
kwarg in the `for` loop handling the `default_streams`.
In this commit, the `stream` kwarg is not overwritten.
Our current code has following issues with -debuginfo/-debugsource handling:
- The foo-debuginfo is included in the MMD only when foo is included there,
but some special packages contain custom -debuginfo sub-packages like
foo-common-debuginfo without matching foo-common sub-package. With our
current code, the foo-common-debuginfo is never included in the final MMD.
- The foo-debugsource is included in the MMD only when foo.src.rpm,
but some special packages contain custom -debuginfo sub-package.
This commit changes the handling of -debuginfo/-debugsource like this:
- The RPMs to include in the final MMD are evaluated in particular order now.
At first we evaluate non-debug RPMs and then debug RPMs.
- When handling the foo-debuginfo/foo-debugsource RPM, we include it in
final MMD only in one of these cases:
- The "foo" is included in the MMD file (it means it is not filtered out).
- The "foo" package does not exist at all (it means only foo-debuginfo exists
and we need to include this package unless filtered out) and in the same time
the SRPM from which this -debuginfo/-debugsource RPM has been built is included
in a final MMD (it means that there is at least some package from this build
included - this handles case when only foo.src.rpm and foo-debugsource.rpm
would be included in a final MMD, which would be wrong.)
We also respect filters here, so it is possible to explicitely filter out also
-debuginfo/-debugsource packages.
