Commit Graph

124 Commits

Author SHA1 Message Date
Chenxiong Qi
0ee801877b Move module build to ready from done according to Greenwave
Signed-off-by: Chenxiong Qi <cqi@redhat.com>
2019-04-02 16:11:43 +08:00
Chenxiong Qi
3992a1f1ac Fix syntax error when checking if --offline is specified in CLI
Signed-off-by: Chenxiong Qi <cqi@redhat.com>
2019-04-02 08:33:20 +08:00
Jan Kaluza
360a8f3b84 Add support for local component repository using file://.
This is needed for offline local builds to build a component which is
stored on local git repository.

This PR also adds OfflineLocalBuildConfiguration configuration class
for offline local builds to set the RESOLVER.
2019-04-01 13:13:08 +00:00
Jan Kaluza
67a5a9d1b0 Allow building module in --offline module with dependencies from local repositories.
There are following changes introduced in this commit:

- The `koji_tag` of module builds imported from the local repositories
  is now in `repofile:///etc/yum.repos.d/some.repo` format to store the
  repository from which the module was imported to local MBS DB.
- The `koji_tag` of fake base module is set to empty `repofile://`
  and in `MockModuleBuilder` the `conf.base_module_repofiles` list
  is used as source for the repositories defining platform. We can't
  simply use single repository, because there might be fedora.repo
  and fedora-update.repo and so on.
- The list of default .repo files for platform are passed using the
  `-r` switch in `build_module_locally` `mbs-manager` command.
- The LocalResolver (subclass of DBResolver) is added which is used
  to resolve the build dependencies when building modules offline
  locally.
- The `MockModuleBuilder` enables the buildrequired modules and
  repositories from which they come in the mock config.

With this commit, it is possible to build testmodule locally
without any external infra.
2019-04-01 13:13:08 +00:00
Jan Kaluza
e43d979bc8 Import modules found in the local repositories to MBS database for Offline local builds.
This is the first PR in many for Offline local builds. This PR:

- Adds --offline flag to build_module_locally mbs-manager command to enable
  offline local builds.
- If this flag is used, new `import_builds_from_local_dnf_repos` method is
  called which uses DNF API to get all the available installable modulemd
  files and imports each module into MBS local SQLite database.
- It also adds fake "platform:stream" module based on the /etc/os-release,
  so the buildrequirements of the imported modules are satisfied.

The idea here is that in upcoming commits, I will create LocalResolver
which will be similar to DBResolver with some extra rules to resolve
local module builds. This new LocalResolver will still be based on
the models.ModuleBuild methods and therefore we need the modules
imported in database.
2019-04-01 13:13:08 +00:00
mprahl
a8a8913bbc Add the ability to override a buildrequired module stream based on a module's branch
This moves the functionality that was in rpkg to MBS. See the following
PRs for more context:
https://pagure.io/rpkg/pull-request/390
https://pagure.io/rpkg/pull-request/420
2019-03-19 16:45:00 -04:00
Jan Kaluza
765e640129 Allow setting the Koji tag extra options using the conf.koji_tag_extra_opts.
The Koji tag extra options used to be hard-coded and to change them,
we had to release new MBS version.

We do not change them often, but right now fedora-infra is requesting
to use new `mock.new_chroot` option and we need to release new MBS
because of that.

This commit makes such changes easier in the future.
2019-03-18 17:27:57 +00:00
Merlin Mathesius
dd950857ec Update koji tagging for scratch modules.
Signed-off-by: Merlin Mathesius <mmathesi@redhat.com>
2019-03-01 10:27:04 -06:00
mprahl
776330a2bb Add the allowed_users configuration for service accounts to bypass the group membership check
This is required for monitoring use-cases, where we can have a Kerberos principal for a
service account but no associated account in LDAP to check group membership.
2019-02-14 11:11:18 -05:00
mprahl
c95f1897f4 Fix the default distgits config value to match the default value for RPMS_DEFAULT_REPOSITORY
This was missed as part of PR#1120, where the default dist-git URLs were changed.
2019-01-17 09:07:22 -05:00
mprahl
9f9a025958 Use the https protocol to clone dist-git repos instead of the git protocol
You can read about this at:
https://fedoraproject.org/wiki/Infrastructure/HTTPS-commits

The bug was reported at:
https://bugzilla.redhat.com/show_bug.cgi?id=1666257
2019-01-16 19:00:25 +00:00
mprahl
7f82275114 Force the use of a separte Kerberos cache per thread
When using a single Kerberos cache that is shared among threads,
Koji logins start failing because the cache gets corrupt. This uses
the Linux kernel keyring to store a Kerberos cache per MBS thread.

See https://web.mit.edu/kerberos/krb5-1.12/doc/basic/ccache_def.html
2018-12-12 19:04:00 -05:00
mprahl
25b6a93a07 Make the devel Koji CG build optional using a config option 2018-12-03 12:00:30 -05:00
Chenxiong Qi
5d3ea762c6 Make external repo URL prefix configurable
Not both Fedora and internal Brew uses config topurl as the external
repo's URL prefix. Hence, make it configurable to fulfill this
difference.

Signed-off-by: Chenxiong Qi <cqi@redhat.com>
2018-11-27 11:06:34 +08:00
Chenxiong Qi
b5860fa039 Make it optional to tag build for CG import
Fixes FACTORY-3467

Signed-off-by: Chenxiong Qi <cqi@redhat.com>
2018-11-21 09:28:03 +08:00
Valerij Maljulin
6b7ea33d18 limit Koji build architectures
Fixes issue #869

Signed-off-by: Valerij Maljulin <vmaljuli@redhat.com>
2018-11-12 12:44:52 +01:00
Valerij Maljulin
7f60db545a renaming koji_arches to arches
Signed-off-by: Valerij Maljulin <vmaljuli@redhat.com>
2018-10-15 15:29:50 +02:00
mprahl
54c1ed7166 Change the config base_module_names to be a list instead of a set to allow ordering by the admin 2018-10-04 07:19:07 -04:00
mprahl
ac400fafcf Set the default base_module_names config option to be platform 2018-10-03 10:11:34 -04:00
Filip Valder
598347e1b6 Import module API 2018-09-07 17:12:01 +02:00
Ralph Bean
e7089d7be0 Rename this to be PDC-agnostic. 2018-09-04 09:51:56 -04:00
Ralph Bean
0aff640ef3 Optionally, block building modules with EOL streams.
Requested by @mboddu in:

- https://pagure.io/fm-orchestrator/issue/960
- https://pagure.io/modularity/issue/102
2018-08-31 15:23:04 -04:00
Martin Curlej
5a3d3d99d3 ISSUE-991: Added a method to the producer that will move stuck builds into 'failed' state
Signed-off-by: Martin Curlej <mcurlej@redhat.com>
2018-08-30 16:02:24 +02:00
Matt Prahl
09edb4cba8 Merge #990 Allow overriding the name and stream from scm repos. 2018-08-14 13:03:52 +00:00
Jan Kaluza
f0b35be067 Add BASE_MODULE_KOJI_ARCHES to override default Koji arches. 2018-08-14 13:18:56 +02:00
Ralph Bean
4279e947be Allow overriding the name and stream from scm repos.
If configured on the server side, disabled by default.
2018-08-13 14:08:36 -04:00
Martin Curlej
dfe7660519 Removed PDC dependency from MBS and replaced it with MBS prod instance.
Signed-off-by: Martin Curlej <mcurlej@redhat.com>

Removed pdc from comments.

Signed-off-by: Martin Curlej <mcurlej@redhat.com>

Adding missing files.

Signed-off-by: Martin Curlej <mcurlej@redhat.com>

Updated PR according to review.

Signed-off-by: Martin Curlej <mcurlej@redhat.com>

Local modules builds now

Signed-off-by: Martin Curlej <mcurlej@redhat.com>

Removed copr from config

Signed-off-by: Martin Curlej <mcurlej@redhat.com>

Fixed bugs

Signed-off-by: Martin Curlej <mcurlej@redhat.com>
2018-07-04 10:13:18 +02:00
mprahl
36a9026ca8 Remove references to COPR 2018-06-28 10:00:06 -04:00
mprahl
19db295795 Fix flake8 errors 2018-04-03 09:58:57 -04:00
mprahl
eb0b2e1c38 Add the database resolver plugin 2018-04-03 09:58:57 -04:00
mprahl
b7998d4402 Remove the need for nose and solely rely on pytest 2018-02-05 22:11:31 -05:00
mprahl
df594da8a5 Remove the Koji proxyuser functionality 2018-01-30 11:49:03 -05:00
Filip Valder
5b4a8f6240 New: Resolver API for pluggable resolvers 2018-01-29 15:54:19 +01:00
Ralph Bean
c21f1d34a3 Faster SCM tests.
Fixes #832.

The slow tests there are hobbled by this `retry(...)` decorator with hardcoded
timeouts.  We want those to be relatively long in prod and relatively short in
test and dev.
2018-01-15 16:30:52 -05:00
mprahl
ef5dc64f2b Transition stale failed builds to the "garbage" state and untag their components 2017-11-17 12:07:51 -05:00
Martin Curlej
c76c5dcf15 mbs-build now builds modules only from yaml files 2017-11-10 12:33:42 -05:00
Martin Curlej
03cff80e4a Streamlined cmd line options for local builds
Signed-off-by: Martin Curlej <mcurlej@redhat.com>
2017-11-10 12:33:42 -05:00
mprahl
08dc070152 Fix line length in config.py 2017-11-09 08:47:45 -05:00
Jakub Kadlcik
f098e6e3b7 Make the module build log name configurable 2017-11-07 00:04:14 +01:00
mprahl
089a0c3db4 Fix typo 2017-11-02 08:37:59 -04:00
mprahl
45cce653eb Add the ability to specify different rebuild methods 2017-11-01 15:39:43 -04:00
Martin Curlej
60b3d97c97 flake8 cleanup
Signed-off-by: Martin Curlej <mcurlej@redhat.com>

removed some noqa tags
2017-10-25 12:13:59 +02:00
Ralph Bean
0e9ccb1895 Remove old SSL config options.
We used to do x509 authn a long time ago.  We have since stopped doing
that, but we kept all of thise confusing SSL/TLS support stuff.

This removes all that, and fixes #685.
2017-10-04 11:37:17 +02:00
Jan Kaluza
ad6874b0e6 Fix #670 - Tag Content Generator Koji build to special tag based on the base module stream 2017-09-25 08:36:29 +02:00
Ralph Bean
5add97648d Clearer error message. 2017-09-19 13:23:59 -04:00
Ralph Bean
5e08d2ffe4 Introduce pluggable backends.
This allows an operator to write a custom messaging backend to handle
whatever transport and format.
2017-09-19 13:13:40 -04:00
Ralph Bean
72d1c83735 Complain early. 2017-09-15 13:03:15 -04:00
Ralph Bean
fa8ac2001d Merge #681 Add Kerberos + LDAP Authentication 2017-09-15 16:10:22 +00:00
mprahl
0f6d7a55c5 Add Kerberos + LDAP authentication support 2017-09-15 12:02:57 -04:00
mprahl
79a79fdcf2 Support calling the tests by using pytest instead of just py.test 2017-09-15 09:21:59 -04:00