Commit Graph

130 Commits

Author SHA1 Message Date
Valerij Maljulin
1b486b1625 Greenwave query class and configuration update
Signed-off-by: Valerij Maljulin <vmaljuli@redhat.com>
2019-04-29 18:15:23 +02:00
mprahl
66c3f82160 Format the coding style across the codebase using "black" and manual tweaks
The main benefit of this commit is that the use of double quotes
is now consistent.
2019-04-26 00:32:13 -04:00
Chenxiong Qi
7c993f9165 Handle streams in base module stream
A base module's stream (the platform for RHEL) could have Z-stream suffix, e.g.
el8.0.0.z, this patch handles this Z-stream suffix and other potential streams
by returning the stream version as a float with configured suffix value. For
example, el8.1.0.z would be parsed as 080100.1. Note that, the 0.1 is totally
configured in config and it actually could be any value according to concrete
cases in practice.

Config STREAM_SUFFIXES is enabled in TestConfiguration so that tests depending
on the return value from ModuleBuild.get_stream_version are covered.

Part fixture of test TestMMDResolver.test_solve_virtual_streams is updated by
adding Z-stream suffix to platform:el8.2.0 in order to ensure this patch does
not break the MMD resolver.

Addresses FACTORY-4307

Signed-off-by: Chenxiong Qi <cqi@redhat.com>
2019-04-23 11:21:46 +08:00
mprahl
83bc780a23 Add backwards-compatibility for cloning local repos with the artifact name and not the full path
This addresses #1211
2019-04-11 10:23:12 -04:00
mprahl
d6d0c870fc Disable Greenwave gating by default 2019-04-04 13:16:33 -04:00
mprahl
9512a49631 Allow whitelisted buildrequires with xmd.mbs.disttag_marking set to influnece the disttag 2019-04-04 09:12:27 -04:00
Chenxiong Qi
0ee801877b Move module build to ready from done according to Greenwave
Signed-off-by: Chenxiong Qi <cqi@redhat.com>
2019-04-02 16:11:43 +08:00
Chenxiong Qi
3992a1f1ac Fix syntax error when checking if --offline is specified in CLI
Signed-off-by: Chenxiong Qi <cqi@redhat.com>
2019-04-02 08:33:20 +08:00
Jan Kaluza
360a8f3b84 Add support for local component repository using file://.
This is needed for offline local builds to build a component which is
stored on local git repository.

This PR also adds OfflineLocalBuildConfiguration configuration class
for offline local builds to set the RESOLVER.
2019-04-01 13:13:08 +00:00
Jan Kaluza
67a5a9d1b0 Allow building module in --offline module with dependencies from local repositories.
There are following changes introduced in this commit:

- The `koji_tag` of module builds imported from the local repositories
  is now in `repofile:///etc/yum.repos.d/some.repo` format to store the
  repository from which the module was imported to local MBS DB.
- The `koji_tag` of fake base module is set to empty `repofile://`
  and in `MockModuleBuilder` the `conf.base_module_repofiles` list
  is used as source for the repositories defining platform. We can't
  simply use single repository, because there might be fedora.repo
  and fedora-update.repo and so on.
- The list of default .repo files for platform are passed using the
  `-r` switch in `build_module_locally` `mbs-manager` command.
- The LocalResolver (subclass of DBResolver) is added which is used
  to resolve the build dependencies when building modules offline
  locally.
- The `MockModuleBuilder` enables the buildrequired modules and
  repositories from which they come in the mock config.

With this commit, it is possible to build testmodule locally
without any external infra.
2019-04-01 13:13:08 +00:00
Jan Kaluza
e43d979bc8 Import modules found in the local repositories to MBS database for Offline local builds.
This is the first PR in many for Offline local builds. This PR:

- Adds --offline flag to build_module_locally mbs-manager command to enable
  offline local builds.
- If this flag is used, new `import_builds_from_local_dnf_repos` method is
  called which uses DNF API to get all the available installable modulemd
  files and imports each module into MBS local SQLite database.
- It also adds fake "platform:stream" module based on the /etc/os-release,
  so the buildrequirements of the imported modules are satisfied.

The idea here is that in upcoming commits, I will create LocalResolver
which will be similar to DBResolver with some extra rules to resolve
local module builds. This new LocalResolver will still be based on
the models.ModuleBuild methods and therefore we need the modules
imported in database.
2019-04-01 13:13:08 +00:00
mprahl
a8a8913bbc Add the ability to override a buildrequired module stream based on a module's branch
This moves the functionality that was in rpkg to MBS. See the following
PRs for more context:
https://pagure.io/rpkg/pull-request/390
https://pagure.io/rpkg/pull-request/420
2019-03-19 16:45:00 -04:00
Jan Kaluza
765e640129 Allow setting the Koji tag extra options using the conf.koji_tag_extra_opts.
The Koji tag extra options used to be hard-coded and to change them,
we had to release new MBS version.

We do not change them often, but right now fedora-infra is requesting
to use new `mock.new_chroot` option and we need to release new MBS
because of that.

This commit makes such changes easier in the future.
2019-03-18 17:27:57 +00:00
Merlin Mathesius
dd950857ec Update koji tagging for scratch modules.
Signed-off-by: Merlin Mathesius <mmathesi@redhat.com>
2019-03-01 10:27:04 -06:00
mprahl
776330a2bb Add the allowed_users configuration for service accounts to bypass the group membership check
This is required for monitoring use-cases, where we can have a Kerberos principal for a
service account but no associated account in LDAP to check group membership.
2019-02-14 11:11:18 -05:00
mprahl
c95f1897f4 Fix the default distgits config value to match the default value for RPMS_DEFAULT_REPOSITORY
This was missed as part of PR#1120, where the default dist-git URLs were changed.
2019-01-17 09:07:22 -05:00
mprahl
9f9a025958 Use the https protocol to clone dist-git repos instead of the git protocol
You can read about this at:
https://fedoraproject.org/wiki/Infrastructure/HTTPS-commits

The bug was reported at:
https://bugzilla.redhat.com/show_bug.cgi?id=1666257
2019-01-16 19:00:25 +00:00
mprahl
7f82275114 Force the use of a separte Kerberos cache per thread
When using a single Kerberos cache that is shared among threads,
Koji logins start failing because the cache gets corrupt. This uses
the Linux kernel keyring to store a Kerberos cache per MBS thread.

See https://web.mit.edu/kerberos/krb5-1.12/doc/basic/ccache_def.html
2018-12-12 19:04:00 -05:00
mprahl
25b6a93a07 Make the devel Koji CG build optional using a config option 2018-12-03 12:00:30 -05:00
Chenxiong Qi
5d3ea762c6 Make external repo URL prefix configurable
Not both Fedora and internal Brew uses config topurl as the external
repo's URL prefix. Hence, make it configurable to fulfill this
difference.

Signed-off-by: Chenxiong Qi <cqi@redhat.com>
2018-11-27 11:06:34 +08:00
Chenxiong Qi
b5860fa039 Make it optional to tag build for CG import
Fixes FACTORY-3467

Signed-off-by: Chenxiong Qi <cqi@redhat.com>
2018-11-21 09:28:03 +08:00
Valerij Maljulin
6b7ea33d18 limit Koji build architectures
Fixes issue #869

Signed-off-by: Valerij Maljulin <vmaljuli@redhat.com>
2018-11-12 12:44:52 +01:00
Valerij Maljulin
7f60db545a renaming koji_arches to arches
Signed-off-by: Valerij Maljulin <vmaljuli@redhat.com>
2018-10-15 15:29:50 +02:00
mprahl
54c1ed7166 Change the config base_module_names to be a list instead of a set to allow ordering by the admin 2018-10-04 07:19:07 -04:00
mprahl
ac400fafcf Set the default base_module_names config option to be platform 2018-10-03 10:11:34 -04:00
Filip Valder
598347e1b6 Import module API 2018-09-07 17:12:01 +02:00
Ralph Bean
e7089d7be0 Rename this to be PDC-agnostic. 2018-09-04 09:51:56 -04:00
Ralph Bean
0aff640ef3 Optionally, block building modules with EOL streams.
Requested by @mboddu in:

- https://pagure.io/fm-orchestrator/issue/960
- https://pagure.io/modularity/issue/102
2018-08-31 15:23:04 -04:00
Martin Curlej
5a3d3d99d3 ISSUE-991: Added a method to the producer that will move stuck builds into 'failed' state
Signed-off-by: Martin Curlej <mcurlej@redhat.com>
2018-08-30 16:02:24 +02:00
Matt Prahl
09edb4cba8 Merge #990 Allow overriding the name and stream from scm repos. 2018-08-14 13:03:52 +00:00
Jan Kaluza
f0b35be067 Add BASE_MODULE_KOJI_ARCHES to override default Koji arches. 2018-08-14 13:18:56 +02:00
Ralph Bean
4279e947be Allow overriding the name and stream from scm repos.
If configured on the server side, disabled by default.
2018-08-13 14:08:36 -04:00
Martin Curlej
dfe7660519 Removed PDC dependency from MBS and replaced it with MBS prod instance.
Signed-off-by: Martin Curlej <mcurlej@redhat.com>

Removed pdc from comments.

Signed-off-by: Martin Curlej <mcurlej@redhat.com>

Adding missing files.

Signed-off-by: Martin Curlej <mcurlej@redhat.com>

Updated PR according to review.

Signed-off-by: Martin Curlej <mcurlej@redhat.com>

Local modules builds now

Signed-off-by: Martin Curlej <mcurlej@redhat.com>

Removed copr from config

Signed-off-by: Martin Curlej <mcurlej@redhat.com>

Fixed bugs

Signed-off-by: Martin Curlej <mcurlej@redhat.com>
2018-07-04 10:13:18 +02:00
mprahl
36a9026ca8 Remove references to COPR 2018-06-28 10:00:06 -04:00
mprahl
19db295795 Fix flake8 errors 2018-04-03 09:58:57 -04:00
mprahl
eb0b2e1c38 Add the database resolver plugin 2018-04-03 09:58:57 -04:00
mprahl
b7998d4402 Remove the need for nose and solely rely on pytest 2018-02-05 22:11:31 -05:00
mprahl
df594da8a5 Remove the Koji proxyuser functionality 2018-01-30 11:49:03 -05:00
Filip Valder
5b4a8f6240 New: Resolver API for pluggable resolvers 2018-01-29 15:54:19 +01:00
Ralph Bean
c21f1d34a3 Faster SCM tests.
Fixes #832.

The slow tests there are hobbled by this `retry(...)` decorator with hardcoded
timeouts.  We want those to be relatively long in prod and relatively short in
test and dev.
2018-01-15 16:30:52 -05:00
mprahl
ef5dc64f2b Transition stale failed builds to the "garbage" state and untag their components 2017-11-17 12:07:51 -05:00
Martin Curlej
c76c5dcf15 mbs-build now builds modules only from yaml files 2017-11-10 12:33:42 -05:00
Martin Curlej
03cff80e4a Streamlined cmd line options for local builds
Signed-off-by: Martin Curlej <mcurlej@redhat.com>
2017-11-10 12:33:42 -05:00
mprahl
08dc070152 Fix line length in config.py 2017-11-09 08:47:45 -05:00
Jakub Kadlcik
f098e6e3b7 Make the module build log name configurable 2017-11-07 00:04:14 +01:00
mprahl
089a0c3db4 Fix typo 2017-11-02 08:37:59 -04:00
mprahl
45cce653eb Add the ability to specify different rebuild methods 2017-11-01 15:39:43 -04:00
Martin Curlej
60b3d97c97 flake8 cleanup
Signed-off-by: Martin Curlej <mcurlej@redhat.com>

removed some noqa tags
2017-10-25 12:13:59 +02:00
Ralph Bean
0e9ccb1895 Remove old SSL config options.
We used to do x509 authn a long time ago.  We have since stopped doing
that, but we kept all of thise confusing SSL/TLS support stuff.

This removes all that, and fixes #685.
2017-10-04 11:37:17 +02:00
Jan Kaluza
ad6874b0e6 Fix #670 - Tag Content Generator Koji build to special tag based on the base module stream 2017-09-25 08:36:29 +02:00