1360 Commits

Author SHA1 Message Date
Mikolaj Izdebski
2d1b056508 Add workaround for DNF fail-safe mechanism
Fixes #1525
2020-02-04 14:09:06 -05:00
Matt Prahl
66a2362a90 Merge #1552 Fix the provides of base modules 2020-01-16 19:38:55 +00:00
Chenxiong Qi
3d771aac10 Fix a typo
Signed-off-by: Chenxiong Qi <cqi@redhat.com>
2020-01-10 16:08:33 +08:00
Matt Prahl
fef0ffaf53 Merge #1563 use gssapi if correct python-requests-kerberos is available 2020-01-09 16:38:00 +00:00
Brendan Reilly
c349946718 use gssapi if correct python-requests-kerberos is available 2020-01-09 10:01:36 -05:00
Brendan Reilly
df319e14ce Fix the provides of base modules when it has a stream version 2020-01-07 11:33:56 -05:00
Mike McLean
6b85a31e0d close dnf base instance after we're done with it 2019-12-20 09:55:09 -05:00
Mike McLean
1fae917fc3 error if repo metadata fails to load 2019-12-20 09:55:09 -05:00
Mike McLean
0d66e24a05 make dnf timeout configurable 2019-12-20 09:55:09 -05:00
Matt Prahl
f4ab801b3a Merge #1535 Only allow cancelling module builds in the init, wait, and build states 2019-12-06 14:02:13 +00:00
mprahl
9d946af461 Only allow cancelling module builds in the init, wait, and build states 2019-12-04 13:50:52 -05:00
mprahl
3051596166 Remove the koji.ClientSession backport
See https://pagure.io/koji/pull-request/1187 for context.
2019-12-02 12:07:40 -05:00
Matt Prahl
f487da140b Merge #1524 Gracefully handle builds without a koji tag 2019-11-22 16:26:12 +00:00
Luiz Carvalho
618eb3a3c6 Gracefully handle builds without a koji tag
MBS will iterate through all the builds in buildrequires to determine
the expected list of arches on the associated Koji tag. In some cases,
these builds do not have a Koji tag. They should be skipped for this
operation.

Signed-off-by: Luiz Carvalho <lucarval@redhat.com>
2019-11-22 10:53:40 -05:00
Christopher O'Brien
12acc9242d allow koji tags to be created with a configurable permission. Fixes #1478 2019-11-22 07:41:16 -05:00
Chenxiong Qi
87f4f3e546 Use distro.linux_distribution instead
platform.linux_distribution is removed since Python 3.8.

Signed-off-by: Chenxiong Qi <cqi@redhat.com>
2019-11-18 10:41:41 +08:00
Mike McLean
11df325e5c Merge #1475 allow reuse of added components in some cases 2019-11-13 13:25:58 +00:00
Jan Kaluža
8dc3eed766 Merge #1504 Correct function name spelling 2019-11-13 07:51:28 +00:00
Chenxiong Qi
fb0cd95427 Fix call of at_concurrent_component_threshold in producer
Signed-off-by: Chenxiong Qi <cqi@redhat.com>
2019-11-13 11:28:35 +08:00
Merlin Mathesius
8807389222 Correct function name spelling
Signed-off-by: Merlin Mathesius <mmathesi@redhat.com>
2019-11-12 12:12:39 -06:00
Mike McLean
f5692d0f3a Adjust unnecessarily strict reuse check
Fixes https://pagure.io/fm-orchestrator/issue/1298
2019-11-11 18:07:13 -05:00
mprahl
fc7ecb145f Configure the MBS backend code to use the correct SQLite database during local builds 2019-11-11 11:12:45 -05:00
mprahl
1160d47e7a Stop adding repo entries for empty tags for local builds
DNF fails on Fedora 31 when there is a repo entry to an empty directory.
2019-11-07 11:18:50 -05:00
mprahl
be641dee57 Set skip_if_unavailable=True on localrepo to avoid module-build-macros failures for local builds
When module-build-macros is built, the localrepo hasn't been generated yet.
This has caused errors on Fedora 31.
2019-11-07 11:18:38 -05:00
Chenxiong Qi
f24cd4222f Make db_session singleton
Please note that this patch does not change the use of database session
in MBS. So, in the frontend, the database session is still managed by
Flask-SQLAlchemy, that is the db.session. And the backend, running event
handlers, has its own database session created from SQLAclehmy session
API directly.

This patch aims to reduce the number of scoped_session created when call
original function make_db_session. For technical detailed information,
please refer to SQLAlchemy documentation Contextual/Thread-local
Sessions.

As a result, a global scoped_session is accessible from the
code running inside backend, both the event handlers and functions
called from handlers. The library code shared by frontend and backend,
like resolvers, has no change.

Similarly, db.session is only used to recreate database for every test.

Signed-off-by: Chenxiong Qi <cqi@redhat.com>
2019-11-07 11:06:40 +08:00
mprahl
eebefd0689 Set module_hotfixes on repos created for local builds to allow using modular packages in non-modular repos
This fixes an issue that causes module-build-macros to fail during a local
build when a module buildrequires a non-base module.
2019-11-06 09:13:32 -05:00
Chenxiong Qi
28f089bf3d Merge #1491 Remove unused _dummy_context_mgr 2019-11-06 02:17:53 +00:00
mprahl
8dc20c1731 Remove unused Kerberos configuration options
This is some leftover code that was missed in 9fd3731ff7.
2019-11-05 15:08:27 -05:00
mprahl
3b85cecf6a Remove unused _dummy_context_mgr 2019-11-05 08:48:40 -05:00
Jan Kaluža
84398834e4 Merge #1481 Search for default modules built also against compatible base modules. 2019-11-05 07:41:42 +00:00
Jan Kaluza
96ca7fd00b Search for default modules built also against compatible base modules. 2019-11-05 07:03:40 +01:00
Matt Prahl
4eee019791 Merge #1486 update container base images to Fedora 31 2019-11-04 20:27:39 +00:00
Mike Bonnet
e582126106 use the C locale for generating the date string for the specfile changelog
This produces exactly the same output as the en_US locale, and is guaranteed to be supported
in all environments.
2019-11-01 16:50:56 -07:00
mprahl
f95b665904 Use RuntimeError instead of ValueError when the retrieval of default modules fails
This will result in the error actually being in the state reason
of the failed module build instead of "An unknown error occurred
while validating the modulemd".
2019-11-01 14:49:10 -04:00
Chenxiong Qi
1c645f5a04 Fix log.exception
log.exception() without a message will fail and module build cannot move
to failed state.

Signed-off-by: Chenxiong Qi <cqi@redhat.com>
2019-11-01 21:33:08 +08:00
Chenxiong Qi
1ab259ba77 Fix state for determining if continue building components in current batch
The original code before changing to is_unbuilt is to check if any
component builds in current batch has state None. This patch fixes that.

Signed-off-by: Chenxiong Qi <cqi@redhat.com>
2019-10-30 09:26:07 +08:00
Chenxiong Qi
cab9cb8ee3 Merge #1479 Minor fixes to consume and process_message 2019-10-30 01:24:38 +00:00
Chenxiong Qi
251239621d Refactor consume and process_message
A new method _map_message is added for converting a message object to
corresponding event handler and module build. This is used to shorten the
process_message method.

process_message is refactored so that:

* when handler is NO_OP, just return as earlier as possible because setting it
  make no sense to set MBSConsumer.current_module_build_id back and forth.
* Re-raise error thrown from handler execution so that the caller is able to
  get a chance to collect monitoring metric inside except clause handling
  Exception. Otherwise, no failed metric is collected. This is the major
  problem this patch is to fix.
* Ensure MBSConsumer.current_module_build_id is set back to None.

In consumer method, no traceback is logged inside except clause catching
Exception. process_method does that.

Signed-off-by: Chenxiong Qi <cqi@redhat.com>
2019-10-29 22:22:10 +08:00
Chenxiong Qi
fa227698ef Fix wrong value passed to ModuleBuild.transition state argument
state argument expects an int value represeting a module build state
rather than a string value.

Signed-off-by: Chenxiong Qi <cqi@redhat.com>
2019-10-29 22:19:46 +08:00
Chenxiong Qi
4bebd13872 Fix wrong argument name in docstring and assert its value
Signed-off-by: Chenxiong Qi <cqi@redhat.com>
2019-10-29 22:19:46 +08:00
Chenxiong Qi
3bdbdaee20 Remove unnecessary code from process_message
It is not necessary to initialize the build variable before following
if-elif-else branch.

After the if-elif-else branch, if no build is found, process_message
just returns immediately. So, no need to check if build is None during
handling error raised from handler call.

Signed-off-by: Chenxiong Qi <cqi@redhat.com>
2019-10-28 20:49:52 +08:00
Chenxiong Qi
d1026c0df7 Do not drop realm from user's kerberos principal name
Save the realm and just keep the variable there for any further possible
use.

Signed-off-by: Chenxiong Qi <cqi@redhat.com>
2019-10-25 13:46:54 +08:00
Mike McLean
620263c34b combine common code 2019-10-18 09:44:45 -04:00
Mike McLean
ad6afecc69 also fix build.extra 2019-10-18 09:44:45 -04:00
Mike McLean
8581b9d02f adjust name for devel builds in modulemd.txt 2019-10-18 09:44:45 -04:00
Chenxiong Qi
9fd3731ff7 Remove KerberosAuthenticate and get kerberos username from REMOTE_USER
Signed-off-by: Chenxiong Qi <cqi@redhat.com>
2019-10-17 10:20:59 +08:00
Jan Kaluza
eeb65c97da KojiResolver: Filter module builds based on the real stream name.
Query Koji for the real stream name of each module and keep only those matching
requested `stream`.

This needs to be done, because MBS stores the stream name in the "version" field in Koji,
but the "version" field cannot contain "-" character. Therefore MBS replaces all "-"
with "_". This makes it impossible to reconstruct the original stream name from the
"version" field.

We therefore need to ask for real original stream name here and filter out modules based
on this real stream name.
2019-10-14 14:13:34 +02:00
mprahl
27f4d71392 Only use default modules that were built with the same base module 2019-10-08 11:38:39 -04:00
Jan Kaluza
339c1ef98d Add KojiResolver to config.SUPPORTED_RESOLVERS. 2019-10-07 12:52:13 +02:00
Jan Kaluza
5ffdeb4649 Fallback to DBResolver in KojiResolver.get_compatible_base_module_modulemds.
In case KojiResolver is not enabled for the base module, fallback to
DBResolver in `KojiResolver.get_compatible_base_module_modulemds`.
2019-10-04 15:22:35 +02:00