Removing the requirement to build the rpm as part of the image build simplifies the
build process and reduces fragility introduced by trying to keep the rpm build and
image builds in sync.
No implementations of MBS are using Greenwave, and there are no current plans
to do so. Koji Resolver will be sufficient for any usecase dependent on gating.
New pipeline-as-a-service Koji is going to use Kerberos instead SSL.
Added dist-git to services since new koji worker in
pipeline-as-a-service is integrated with dist-git.
Changing package_manager from dnf to yum which is configured in koji
from pipeline-as-a-service.
Reduce number of jobs from 20 to 12 by merging them and using
templating to avoid code duplicity.
Jenkinsfile are directly put into jobs as a script - not pulled from
repository - avoiding issue with not clean workspace.
Secrets are taken from openshift using service account intead of
refering them in environment which caused that secret had to exist and
could be empty.
Job execution is straitforward - one jobs is repsonsible for each
process, only triggered jobs are c3iaas request and
pipeline-as-a-service job, everything else is managed by one job.
This change deploys a KDC using the c3i-library (https://pagure.io/c3i-library) and configures MBS
to require Kerberos authentication for submitting builds to the frontend.
With the use of C3IaaS, resource contention in the main OpenShift project will
be dramatically reduced. This allows jobs to run in paralle, increasing throughput
and reducing lag time for feedback on changes.
The C3I-as-a-Service system (https://pagure.io/c3i-library/blob/master/f/c3iaas)
enables creation of new OpenShift projects, which will be deleted after a
configurable period of time, and are suitable for building and testing proposed
changes. The MBS-Koji integration tests now make use of C3IaaS, to reduce resource
contention and avoid false failures due to quota limitations.
Only checkout exactly the code we need to run the tests. This should help work around
errors and timeouts (early EOF/index-pack failed) from Pagure. Jenkins seems to require
a minimum of depth=2 or it fails when walking the revision history.
Using a shallow clone is safe for PRs because we're always building from the head of
the PR branch (OpenShift doesn't have a way to specify a non-head commit on a PR branch).
When building from the master branch we use depth=10 to avoid a race condition in case
multiple commits have been pushed in quick succession. The integration test job is
launched using the exact revision of the code that was used to build the images, to
ensure the results are valid.
Sometimes it is necessary recreate the Jenkins PR polling job, or
cleanup old runs. This can cause Jenkins to lose the record of which
PRs it has already seen, and retest old PRs which are long closed.
This change causes the polling job to check the state of the PR, and
ignore it if it isn't Open. Testing PRs which are in any other state
doesn't provide any value.
Set a consistent set of options on Jenkins pipeline jobs. This includes timestamps,
reasonable timeouts, saving only the last 10 builds (to avoid filling up the disk
of the Jenkins master), disabling concurrent builds (to avoid hitting quota limits),
and skipping the default checkout (all jobs that need the source call checkout()
explicitly).
Previously, the integration test pipeline was configured to always use the tests present on the
master branch. When running the pre-merge pipeline for a PR, this would result in potentially
out-dated tests being run. This change runs the tests from the current branch, allowing the code
and the tests to evolve together.
This also disables the default checkout of the master branch, which is unnecessary when calling
checkout() from the pipeline.
This change introduces a set of Jenkins pipelines for building MBS
images and running integration tests against Koji using those images.
These pipelines are directly based on the WaiverDB pipeline work:
https://pagure.io/waiverdb/blob/master/f/openshift
The results of those tests are used to provide feedback to Pagure PRs
and to promote images through a series of environments, which may be
used to implement a continuous deployment process.
The current test cases, written in Groovy, are:
- module-build-init: initate a module build and check that tags
and targets in Koji are created correctly
- module-build-cgimport: build an empty module and ensure that
results are imported correctly into Koji, using the CGImport
interface
'koji_arches' was renamed to 'arches' in a previous commit. Update the
configuration in the OpenShift template accordingly.
Signed-off-by: Hunor Csomortáni <csomh@redhat.com>
