Commit Graph

773 Commits

Author SHA1 Message Date
ejegrova
f61f7afbdd Add integration test for reuse components 2019-12-09 15:18:11 +01:00
Mariana Ulaieva
06974ef202 Reuse all components test
Implement the integration test for the Reuse All components Scenario.
2019-12-09 14:43:38 +01:00
Hunor Csomortáni
eb86e52c29 Merge #1538 Various test improvements 2019-12-09 12:59:40 +00:00
Matt Prahl
f4ab801b3a Merge #1535 Only allow cancelling module builds in the init, wait, and build states 2019-12-06 14:02:13 +00:00
Hunor Csomortáni
35c1097920 Tests: increase koji task timeout to 5 minutes
When running multiple test builds, the 1 minute timeout was often not
enough.

Signed-off-by: Hunor Csomortáni <csomh@redhat.com>
2019-12-06 12:33:44 +01:00
Hunor Csomortáni
b2485a6150 Tests: fix koji.PathInfo
This should use "topurl". Remove the "weburl" config as it's not needed
anymore.

Signed-off-by: Hunor Csomortáni <csomh@redhat.com>
2019-12-06 12:33:44 +01:00
Hunor Csomortáni
343f1b9818 Tests: separate starting and watching the builds
This makes the tests more explicit, but also prepares the ground to
enable canceling test builds during session teardown in the future.

Signed-off-by: Hunor Csomortáni <csomh@redhat.com>
2019-12-06 12:33:44 +01:00
Hunor Csomortáni
72bb5a2943 Tests: bump the commit before builds
Whenever the tests start a "normal" (non-scratch) build, the commit
needs to be bumped.

Signed-off-by: Hunor Csomortáni <csomh@redhat.com>
2019-12-06 12:33:44 +01:00
Hunor Csomortáni
cfb3549e79 Tests: clone modules with the packaging utility
This is how packagers do. Git URL configuration can also be removed.

Also: redirect command outputs, so that pytest can capture them.

Signed-off-by: Hunor Csomortáni <csomh@redhat.com>
2019-12-06 12:33:44 +01:00
mprahl
9d946af461 Only allow cancelling module builds in the init, wait, and build states 2019-12-04 13:50:52 -05:00
mprahl
3051596166 Remove the koji.ClientSession backport
See https://pagure.io/koji/pull-request/1187 for context.
2019-12-02 12:07:40 -05:00
Mariana Ulaieva
4c8a92cb93 Integration test for the Cancel And Resume
Implement the integration test for the Cancel and Resume scenario.
2019-11-29 13:45:40 +01:00
Hunor Csomortáni
99fc977614 Merge #1515 Integration test for normal build 2019-11-26 15:50:52 +00:00
ejegrova
1d25b81d4c Add integration test for normal build 2019-11-26 16:48:32 +01:00
Luiz Carvalho
618eb3a3c6 Gracefully handle builds without a koji tag
MBS will iterate through all the builds in buildrequires to determine
the expected list of arches on the associated Koji tag. In some cases,
these builds do not have a Koji tag. They should be skipped for this
operation.

Signed-off-by: Luiz Carvalho <lucarval@redhat.com>
2019-11-22 10:53:40 -05:00
Luiz Carvalho
9204941d82 Increase get_module_build_arches test coverage
Signed-off-by: Luiz Carvalho <lucarval@redhat.com>
2019-11-22 10:17:24 -05:00
Hunor Csomortáni
0f0e809467 Tests: allow running integration tests in parallel
This is done in order to speed up test execution.

Signed-off-by: Hunor Csomortáni <csomh@redhat.com>
2019-11-20 16:47:12 +01:00
Hunor Csomortáni
b41b6b9cc1 Tests: allow reusing an existing module build
Module builds take a long time to run, which can be a pain to wait for
during integration test development.

Instead of requiring developers to locally tweak the test code to be
able to reuse module builds, allow specifying the build ID of the module
build to be reused in test.env.yaml .

Signed-off-by: Hunor Csomortáni <csomh@redhat.com>
2019-11-20 14:27:33 +01:00
Hunor Csomortáni
b05ae2fe9a Merge #1499 Integration test for failed build 2019-11-19 16:30:54 +00:00
Mariana Ulaieva
6095ecc5fd Integration test for failed build
Implement the integration test for the failed build scenario.
2019-11-19 13:04:02 +01:00
Chenxiong Qi
87f4f3e546 Use distro.linux_distribution instead
platform.linux_distribution is removed since Python 3.8.

Signed-off-by: Chenxiong Qi <cqi@redhat.com>
2019-11-18 10:41:41 +08:00
Chenxiong Qi
2101f38e63 Run tests in in-memory SQLite db
Signed-off-by: Chenxiong Qi <cqi@redhat.com>
2019-11-14 15:20:21 +08:00
Chenxiong Qi
d962834614 Remove removed db_session fixture from tests
Signed-off-by: Chenxiong Qi <cqi@redhat.com>
2019-11-14 10:52:57 +08:00
Mike McLean
f5692d0f3a Adjust unnecessarily strict reuse check
Fixes https://pagure.io/fm-orchestrator/issue/1298
2019-11-11 18:07:13 -05:00
Hunor Csomortáni
f5bf0d7252 Add integration test to check scratch module builds
Signed-off-by: Hunor Csomortáni <csomh@redhat.com>
2019-11-07 14:30:53 +01:00
Chenxiong Qi
f24cd4222f Make db_session singleton
Please note that this patch does not change the use of database session
in MBS. So, in the frontend, the database session is still managed by
Flask-SQLAlchemy, that is the db.session. And the backend, running event
handlers, has its own database session created from SQLAclehmy session
API directly.

This patch aims to reduce the number of scoped_session created when call
original function make_db_session. For technical detailed information,
please refer to SQLAlchemy documentation Contextual/Thread-local
Sessions.

As a result, a global scoped_session is accessible from the
code running inside backend, both the event handlers and functions
called from handlers. The library code shared by frontend and backend,
like resolvers, has no change.

Similarly, db.session is only used to recreate database for every test.

Signed-off-by: Chenxiong Qi <cqi@redhat.com>
2019-11-07 11:06:40 +08:00
Jan Kaluža
84398834e4 Merge #1481 Search for default modules built also against compatible base modules. 2019-11-05 07:41:42 +00:00
Jan Kaluza
96ca7fd00b Search for default modules built also against compatible base modules. 2019-11-05 07:03:40 +01:00
mprahl
f95b665904 Use RuntimeError instead of ValueError when the retrieval of default modules fails
This will result in the error actually being in the state reason
of the failed module build instead of "An unknown error occurred
while validating the modulemd".
2019-11-01 14:49:10 -04:00
mprahl
ef0ab348bc Remove commented out code in test_utils.py 2019-10-30 15:29:45 -04:00
Chenxiong Qi
d1026c0df7 Do not drop realm from user's kerberos principal name
Save the realm and just keep the variable there for any further possible
use.

Signed-off-by: Chenxiong Qi <cqi@redhat.com>
2019-10-25 13:46:54 +08:00
Hunor Csomortáni
0066c79dae Bootstrap integration tests
This is the first step to have some tests, that we could run against an
MBS instance, to check that it's functionally correct. Ultimately, these
will replace the test scripts (`contrib/test-*`).

This doesn't really do anything yet, but I would like to make sure that
everyone is on the same page regarding how this will be set up.

Signed-off-by: Hunor Csomortáni <csomh@redhat.com>
2019-10-24 10:16:16 +02:00
Mike McLean
107df94776 fix unit test 2019-10-18 09:44:45 -04:00
Mike McLean
dbcb0f30d1 drop unnecessary parentheses 2019-10-18 09:44:45 -04:00
Mike McLean
5e854f503d adjust unit test 2019-10-18 09:44:45 -04:00
Mike McLean
8a83a17c53 update devel case assertion 2019-10-18 09:44:45 -04:00
Chenxiong Qi
9fd3731ff7 Remove KerberosAuthenticate and get kerberos username from REMOTE_USER
Signed-off-by: Chenxiong Qi <cqi@redhat.com>
2019-10-17 10:20:59 +08:00
Jan Kaluza
eeb65c97da KojiResolver: Filter module builds based on the real stream name.
Query Koji for the real stream name of each module and keep only those matching
requested `stream`.

This needs to be done, because MBS stores the stream name in the "version" field in Koji,
but the "version" field cannot contain "-" character. Therefore MBS replaces all "-"
with "_". This makes it impossible to reconstruct the original stream name from the
"version" field.

We therefore need to ask for real original stream name here and filter out modules based
on this real stream name.
2019-10-14 14:13:34 +02:00
Matt Prahl
74bf638cdc Merge #1458 Only use default modules that were built with the same base module 2019-10-09 12:44:52 +00:00
Jan Kaluža
514e4c115a Merge #1457 Make reuse_shared_userspace_init_data faster 2019-10-09 05:24:00 +00:00
mprahl
27f4d71392 Only use default modules that were built with the same base module 2019-10-08 11:38:39 -04:00
Chenxiong Qi
800f606b83 Make reuse_shared_userspace_init_data faster
* A significant performance improvement is to avoid repeating call of
  mmd.get_xmd() in a looping.
* Function is refactored to avoid call of mmd_to_str twice for a module
  build.
* Test using this fixture already has a setup_method to clean database,
  so no need to call clean_database again.
* Calculate build context directly without calculating other contexts
  together.

Signed-off-by: Chenxiong Qi <cqi@redhat.com>
2019-10-08 11:30:50 +08:00
Jan Kaluza
339c1ef98d Add KojiResolver to config.SUPPORTED_RESOLVERS. 2019-10-07 12:52:13 +02:00
Jan Kaluza
5ffdeb4649 Fallback to DBResolver in KojiResolver.get_compatible_base_module_modulemds.
In case KojiResolver is not enabled for the base module, fallback to
DBResolver in `KojiResolver.get_compatible_base_module_modulemds`.
2019-10-04 15:22:35 +02:00
Jan Kaluža
9e38493112 Merge #1452 Support KojiResolver in MBSResolver. 2019-10-04 06:45:39 +00:00
Jan Kaluza
72d2c3daf4 Support KojiResolver in MBSResolver.
If KojiResolver is enabled for buildrequired base module and
MBSResolver is used, then `MBSResolver.get_buildrequired_modulemds`
will use KojiResolver to get the list of buildrequired module builds.

Otherwise it uses the current behavior.

To implement this, the `KojiResolver.get_buildrequired_modules` was
split into two methods:

- `get_buildrequired_koji_builds` returns buildrequired Koji builds.
- `get_buildrequired_modules` calls `get_buildrequired_koji_builds`
  and finds the corresponding ModuleBuilds in MBS DB.
2019-10-04 07:43:11 +02:00
Valerij Maljulin
2c0291987f Raise Unathorized exception instead of general one whenever OIDC auth fails
This fixes #1147
2019-10-03 17:02:33 +02:00
Valerij Maljulin
689d949a48 buildonly support
This fixes #1307

Signed-off-by: Valerij Maljulin <vmaljuli@redhat.com>
2019-10-03 16:20:04 +02:00
Jan Kaluža
6e34e50c18 Merge #1451 Add support for KojiResolver in component reuse code. 2019-10-03 13:16:00 +00:00
Jan Kaluza
486dc39898 Add support for KojiResolver in component reuse code.
In this commit, when component reuse code finds out that the base module uses
KojiResolver, it uses the `KojiResolver.get_buildrequired_modules` method
to find out possible modules to reuse and limits the original query just
by the IDs of these modules.

In order to do that, this commit splits the original
`KojiResolver.get_buildrequired_modulemds` into two methods:

- The `get_buildrequired_modules` returning the ModuleBuilds.
- The `get_buildrequired_modulemds` calling the `get_buildrequired_modules`
  and returning modulemd metadata.
2019-10-03 15:14:29 +02:00