Commit Graph

2559 Commits

Author SHA1 Message Date
Mike McLean
dbcb0f30d1 drop unnecessary parentheses 2019-10-18 09:44:45 -04:00
Mike McLean
5e854f503d adjust unit test 2019-10-18 09:44:45 -04:00
Mike McLean
620263c34b combine common code 2019-10-18 09:44:45 -04:00
Mike McLean
ad6afecc69 also fix build.extra 2019-10-18 09:44:45 -04:00
Mike McLean
8a83a17c53 update devel case assertion 2019-10-18 09:44:45 -04:00
Mike McLean
8581b9d02f adjust name for devel builds in modulemd.txt 2019-10-18 09:44:45 -04:00
Mike McLean
72621277f3 Merge #1465 run-unittests.sh: docker pull needed images 2019-10-17 15:13:55 +00:00
Mike McLean
1b02a45331 no need to update database image 2019-10-17 11:06:20 -04:00
Chenxiong Qi
9fd3731ff7 Remove KerberosAuthenticate and get kerberos username from REMOTE_USER
Signed-off-by: Chenxiong Qi <cqi@redhat.com>
2019-10-17 10:20:59 +08:00
Mike McLean
120681806b update script options in CONTRIBUTING doc 2019-10-15 11:17:51 -04:00
Mike McLean
0d8af77ce4 run-unittests.sh: docker pull needed images
Related: https://pagure.io/fm-orchestrator/issue/1463
2019-10-15 11:14:10 -04:00
Jan Kaluza
eeb65c97da KojiResolver: Filter module builds based on the real stream name.
Query Koji for the real stream name of each module and keep only those matching
requested `stream`.

This needs to be done, because MBS stores the stream name in the "version" field in Koji,
but the "version" field cannot contain "-" character. Therefore MBS replaces all "-"
with "_". This makes it impossible to reconstruct the original stream name from the
"version" field.

We therefore need to ask for real original stream name here and filter out modules based
on this real stream name.
2019-10-14 14:13:34 +02:00
Jan Kaluža
578d6f1e33 Merge #1461 Add KojiResolver doc. 2019-10-11 07:36:29 +00:00
Jan Kaluza
3fedb14483 Add KojiResolver doc. 2019-10-11 09:34:19 +02:00
Matt Prahl
74bf638cdc Merge #1458 Only use default modules that were built with the same base module 2019-10-09 12:44:52 +00:00
Jan Kaluža
514e4c115a Merge #1457 Make reuse_shared_userspace_init_data faster 2019-10-09 05:24:00 +00:00
mprahl
27f4d71392 Only use default modules that were built with the same base module 2019-10-08 11:38:39 -04:00
Chenxiong Qi
800f606b83 Make reuse_shared_userspace_init_data faster
* A significant performance improvement is to avoid repeating call of
  mmd.get_xmd() in a looping.
* Function is refactored to avoid call of mmd_to_str twice for a module
  build.
* Test using this fixture already has a setup_method to clean database,
  so no need to call clean_database again.
* Calculate build context directly without calculating other contexts
  together.

Signed-off-by: Chenxiong Qi <cqi@redhat.com>
2019-10-08 11:30:50 +08:00
Jan Kaluza
339c1ef98d Add KojiResolver to config.SUPPORTED_RESOLVERS. 2019-10-07 12:52:13 +02:00
Jan Kaluza
5ffdeb4649 Fallback to DBResolver in KojiResolver.get_compatible_base_module_modulemds.
In case KojiResolver is not enabled for the base module, fallback to
DBResolver in `KojiResolver.get_compatible_base_module_modulemds`.
2019-10-04 15:22:35 +02:00
Jan Kaluža
9e38493112 Merge #1452 Support KojiResolver in MBSResolver. 2019-10-04 06:45:39 +00:00
Jan Kaluza
72d2c3daf4 Support KojiResolver in MBSResolver.
If KojiResolver is enabled for buildrequired base module and
MBSResolver is used, then `MBSResolver.get_buildrequired_modulemds`
will use KojiResolver to get the list of buildrequired module builds.

Otherwise it uses the current behavior.

To implement this, the `KojiResolver.get_buildrequired_modules` was
split into two methods:

- `get_buildrequired_koji_builds` returns buildrequired Koji builds.
- `get_buildrequired_modules` calls `get_buildrequired_koji_builds`
  and finds the corresponding ModuleBuilds in MBS DB.
2019-10-04 07:43:11 +02:00
Mike McLean
e9946fcd00 Release v2.29.1 v2.29.1 2019-10-03 16:37:02 -04:00
mprahl
540e2ceac3 Fix a fork in the database migration scripts 2019-10-03 16:29:47 -04:00
Mike McLean
bbe10fd2da Release 2.29.0 v2.29.0 2019-10-03 15:10:54 -04:00
Valerij Maljulin
2c0291987f Raise Unathorized exception instead of general one whenever OIDC auth fails
This fixes #1147
2019-10-03 17:02:33 +02:00
Valerij Maljulin
689d949a48 buildonly support
This fixes #1307

Signed-off-by: Valerij Maljulin <vmaljuli@redhat.com>
2019-10-03 16:20:04 +02:00
Jan Kaluža
6e34e50c18 Merge #1451 Add support for KojiResolver in component reuse code. 2019-10-03 13:16:00 +00:00
Jan Kaluza
486dc39898 Add support for KojiResolver in component reuse code.
In this commit, when component reuse code finds out that the base module uses
KojiResolver, it uses the `KojiResolver.get_buildrequired_modules` method
to find out possible modules to reuse and limits the original query just
by the IDs of these modules.

In order to do that, this commit splits the original
`KojiResolver.get_buildrequired_modulemds` into two methods:

- The `get_buildrequired_modules` returning the ModuleBuilds.
- The `get_buildrequired_modulemds` calling the `get_buildrequired_modules`
  and returning modulemd metadata.
2019-10-03 15:14:29 +02:00
mprahl
8c6cfb702d Use small license headers in the Python files
This also removes the outdated comments around authorship of each
file. If there is still interest in this information, one can just
look at the git history.
2019-10-03 08:47:24 -04:00
Matt Prahl
df295516e3 Merge #1449 Use double quotes instead of single quotes to match the style guide 2019-10-02 13:57:36 +00:00
Matt Prahl
73e2d1a6f7 Merge #1427 Allow running an individual test using a relative path 2019-10-02 12:09:30 +00:00
mprahl
f60b292c9c Use double quotes instead of single quotes to match the style guide 2019-10-02 07:42:20 -04:00
mprahl
164f592c1d Make the DNF minrate setting configurable when loading repos 2019-10-02 07:37:12 -04:00
Matt Prahl
2796b9e6a1 Merge #1448 Load the DNF repos in parallel 2019-10-02 11:33:21 +00:00
Jan Kaluza
fc39df9487 Store the build_context without base modules in a database.
For KojiResolver, we need to be able to find out the module to reuse
components from based only on the non-base-module buildrequires. The
base module buildrequirement will be defined by the Koji tag inheritance.

For this, we need to store build_context computed without the base
modules.
2019-10-02 07:16:55 +02:00
mprahl
8d76977434 Load the DNF repos in parallel
In production, loading each repo can be quite slow. This approach
loads the repos in parallel.
2019-10-01 15:29:00 -04:00
Matt Prahl
d4b7a019c7 Merge #1444 Return an empty list in KojiResolver.get_compatible_base_module_modulemds. 2019-09-30 15:12:31 +00:00
Matt Prahl
db7696eafc Merge #1439 Disable handling collisions on local builds due to RHBZ#1693683 2019-09-30 14:58:40 +00:00
Jan Kaluza
a28df5edb7 Return an empty list in KojiResolver.get_compatible_base_module_modulemds.
For KojiResolver, this method returns always an empty list. The compatible modules are
defined by the Koji tag inheritance, so there is no need to find out the compatible
base modules on MBS side.

This makes `mse.get_base_module_mmds` to ignore virtual streams and just use
the input base module as the only module without finding the compatible
base modules.
2019-09-30 09:04:21 +02:00
Jan Kaluža
f6c2a88cc0 Merge #1435 Add initial code for KojiResolver class. 2019-09-30 06:25:55 +00:00
Jan Kaluza
a7540452cd Add initial code for KojiResolver class.
This commit:

- Adds KojiResolver class and KojiResolver tests.
- Changes the GenericResolver and its subclasses to pass base_module_mmds
  instead of base_module_nsvc to get_buildrequired_modulemds. This is needed,
  because KojiResolver needs to access XMD section of base module.
- Implements KojiResolver.get_buildrequired_modulemds to ask Koji for list of
  modules tagged in the Koji tag and return their modulemds.
2019-09-30 07:56:07 +02:00
Stephen Gallagher
8e3ebe5df2 Doc improvements for virtual modules
Signed-off-by: Stephen Gallagher <sgallagh@redhat.com>
2019-09-27 08:48:05 -04:00
mprahl
6b2e5be93a Disable handling collisions on local builds due to RHBZ#1693683 2019-09-26 11:43:30 -04:00
mprahl
453d871e03 Make the log message in execute_cmd easier to read for local builds 2019-09-25 17:30:21 -04:00
Jan Kaluza
1385a1dea3 default_modules: Convert arch to canon_arch.
Convert arch to canon_arch. This handles cases where Koji "i686" arch is mapped to
"i386" when generating RPM repository.
2019-09-23 13:18:38 +02:00
Jan Kaluža
3d5dd91cf7 Merge #1433 __all__ should be in type list[str] 2019-09-23 05:41:43 +00:00
Chenxiong Qi
8e54f05428 __all__ should be in type list[str]
Signed-off-by: Chenxiong Qi <cqi@redhat.com>
2019-09-20 21:47:04 +08:00
Luiz Carvalho
97be1c4737 Document how to generate DB migration scripts
Signed-off-by: Luiz Carvalho <lucarval@redhat.com>
2019-09-19 16:26:26 -04:00
mprahl
550b266961 Release v2.28.2 v2.28.2 2019-09-19 10:22:01 -04:00