Commit Graph

2670 Commits

Author SHA1 Message Date
Chenxiong Qi
fa697a950d Add celery app instance with base config
This patch allows to schedule tasks and launch workers with basic
config. To launch a worker:

    celery -A module_build_service.celery_app -l info

Signed-off-by: Chenxiong Qi <cqi@redhat.com>
2020-01-16 14:39:45 -05:00
Chenxiong Qi
5f4ef94103 Pass event info arguments to event handler directly
This patch drops message objects, defined by class BaseMessage and its
subclasses, and pass event info arguments to event handler directly.
Different event handler requires different arguments to handle a kind of
specific event. The event info is parsed from the raw message received
from message bus.

Signed-off-by: Chenxiong Qi <cqi@redhat.com>
2020-01-16 14:39:45 -05:00
Chenxiong Qi
e1581ac831 Remove config argument from event handlers
For the purpose of migrating to Celery to run event handler inside a
worker, Config object is not serializable. And from the usage of config
argument, every event handler can just access module_build_service.conf
directly. This removal would make the migration easier.

Signed-off-by: Chenxiong Qi <cqi@redhat.com>
2020-01-16 14:39:45 -05:00
Chenxiong Qi
9c76f27d78 Access configured messaging backend directly
The accessible configured messaging backend is sigleton. This patch make
it possible to access the configured backend and avoid accessing a
"private" variable from module messaging.

Signed-off-by: Chenxiong Qi <cqi@redhat.com>
2020-01-16 14:39:45 -05:00
Chenxiong Qi
807acc2fb6 Clear up module_build_service.messaging
Message classes and FedmsgMessageParser are moved into dedicated Python module
under scheduler/ directory.

FedmsgMessageParser is decoupled from messaging.py by initializing a parser
object with known fedmsg services. This decouple avoids cycle import between
parser.py and messaging.py.

Signed-off-by: Chenxiong Qi <cqi@redhat.com>
2020-01-16 14:39:45 -05:00
Chenxiong Qi
fca0e354c1 Remove unused argument arches from add_default_modules
Signed-off-by: Chenxiong Qi <cqi@redhat.com>
2020-01-16 14:39:45 -05:00
Matt Prahl
66a2362a90 Merge #1552 Fix the provides of base modules 2020-01-16 19:38:55 +00:00
Brendan Reilly
c2561da405 Added unit tests for add_base_module_provides 2020-01-16 14:26:04 -05:00
Hunor Csomortáni
5f7442f8c1 Tests: Create a fixture for scenario configuration
Until now, to access the configuration of a scenario, the full path in
test_env had to be specified.

This might be cumbersome and error prone.

Create a scenario fixture, to make it easier to access the same
configuration.

Instead of

    test_env["testdata"]["my_scenario"]["my_config"]

one can use the following:

    scenario["my_config"]

Signed-off-by: Hunor Csomortáni <csomh@redhat.com>
2020-01-16 10:29:14 +01:00
Luiz Carvalho
351eb671f6 Merge #1546 No components integration test 2020-01-10 14:06:44 +00:00
Mariana Ulaieva
c35fe26e73 No components integration test
Implement the Module With No Components integration test.
2020-01-10 14:38:13 +01:00
Chenxiong Qi
3d771aac10 Fix a typo
Signed-off-by: Chenxiong Qi <cqi@redhat.com>
2020-01-10 16:08:33 +08:00
Matt Prahl
fef0ffaf53 Merge #1563 use gssapi if correct python-requests-kerberos is available 2020-01-09 16:38:00 +00:00
Brendan Reilly
c349946718 use gssapi if correct python-requests-kerberos is available 2020-01-09 10:01:36 -05:00
Brendan Reilly
df319e14ce Fix the provides of base modules when it has a stream version 2020-01-07 11:33:56 -05:00
Mike McLean
6b85a31e0d close dnf base instance after we're done with it 2019-12-20 09:55:09 -05:00
Mike McLean
1fae917fc3 error if repo metadata fails to load 2019-12-20 09:55:09 -05:00
Mike McLean
0d66e24a05 make dnf timeout configurable 2019-12-20 09:55:09 -05:00
Mike McLean
36ca88115e Release 2.31.0 v2.31.0 2019-12-19 15:11:22 -05:00
Matt Prahl
7c91d7e2e1 Merge #1545 Use jinja templates to provide 'full-jobs' 2019-12-12 15:05:14 +00:00
Michal Kovarik
a97f6ce9ff Use jinja templates to provide 'full-jobs'
Reduce number of jobs from 20 to 12 by merging them and using
templating to avoid code duplicity.
Jenkinsfile are directly put into jobs as a script - not pulled from
repository - avoiding issue with not clean workspace.
Secrets are taken from openshift using service account intead of
refering them in environment which caused that secret had to exist and
could be empty.
Job execution is straitforward - one jobs is repsonsible for each
process, only triggered jobs are c3iaas request and
pipeline-as-a-service job, everything else is managed by one job.
2019-12-11 12:30:59 +01:00
ejegrova
f61f7afbdd Add integration test for reuse components 2019-12-09 15:18:11 +01:00
Mariana Ulaieva
06974ef202 Reuse all components test
Implement the integration test for the Reuse All components Scenario.
2019-12-09 14:43:38 +01:00
Hunor Csomortáni
eb86e52c29 Merge #1538 Various test improvements 2019-12-09 12:59:40 +00:00
Matt Prahl
f4ab801b3a Merge #1535 Only allow cancelling module builds in the init, wait, and build states 2019-12-06 14:02:13 +00:00
Hunor Csomortáni
35c1097920 Tests: increase koji task timeout to 5 minutes
When running multiple test builds, the 1 minute timeout was often not
enough.

Signed-off-by: Hunor Csomortáni <csomh@redhat.com>
2019-12-06 12:33:44 +01:00
Hunor Csomortáni
b2485a6150 Tests: fix koji.PathInfo
This should use "topurl". Remove the "weburl" config as it's not needed
anymore.

Signed-off-by: Hunor Csomortáni <csomh@redhat.com>
2019-12-06 12:33:44 +01:00
Hunor Csomortáni
343f1b9818 Tests: separate starting and watching the builds
This makes the tests more explicit, but also prepares the ground to
enable canceling test builds during session teardown in the future.

Signed-off-by: Hunor Csomortáni <csomh@redhat.com>
2019-12-06 12:33:44 +01:00
Hunor Csomortáni
72bb5a2943 Tests: bump the commit before builds
Whenever the tests start a "normal" (non-scratch) build, the commit
needs to be bumped.

Signed-off-by: Hunor Csomortáni <csomh@redhat.com>
2019-12-06 12:33:44 +01:00
Hunor Csomortáni
cfb3549e79 Tests: clone modules with the packaging utility
This is how packagers do. Git URL configuration can also be removed.

Also: redirect command outputs, so that pytest can capture them.

Signed-off-by: Hunor Csomortáni <csomh@redhat.com>
2019-12-06 12:33:44 +01:00
Hunor Csomortáni
30215a1ac5 Tests: improve reporting when running with tox
Generate a short report at the end of each test run and a self-contained
HTML report, too.

Signed-off-by: Hunor Csomortáni <csomh@redhat.com>
2019-12-06 12:33:44 +01:00
Matt Prahl
19f0e9ad83 Merge #1529 Update tests to use pipeline-as-a-service 2019-12-05 17:49:43 +00:00
mprahl
9d946af461 Only allow cancelling module builds in the init, wait, and build states 2019-12-04 13:50:52 -05:00
Michal Kovarik
0f057696c8 update tests 2019-12-03 10:15:11 +01:00
Michal Kovarik
e40f5b9735 Integrate pipeline-as-a-code for build 2019-12-03 10:14:20 +01:00
Michal Kovarik
8ba97efce7 Move pagure functions to c3i-library 2019-12-03 07:31:28 +01:00
mprahl
3051596166 Remove the koji.ClientSession backport
See https://pagure.io/koji/pull-request/1187 for context.
2019-12-02 12:07:40 -05:00
Mariana Ulaieva
4c8a92cb93 Integration test for the Cancel And Resume
Implement the integration test for the Cancel and Resume scenario.
2019-11-29 13:45:40 +01:00
Hunor Csomortáni
99fc977614 Merge #1515 Integration test for normal build 2019-11-26 15:50:52 +00:00
ejegrova
1d25b81d4c Add integration test for normal build 2019-11-26 16:48:32 +01:00
Matt Prahl
f487da140b Merge #1524 Gracefully handle builds without a koji tag 2019-11-22 16:26:12 +00:00
Luiz Carvalho
618eb3a3c6 Gracefully handle builds without a koji tag
MBS will iterate through all the builds in buildrequires to determine
the expected list of arches on the associated Koji tag. In some cases,
these builds do not have a Koji tag. They should be skipped for this
operation.

Signed-off-by: Luiz Carvalho <lucarval@redhat.com>
2019-11-22 10:53:40 -05:00
Luiz Carvalho
9204941d82 Increase get_module_build_arches test coverage
Signed-off-by: Luiz Carvalho <lucarval@redhat.com>
2019-11-22 10:17:24 -05:00
Christopher O'Brien
12acc9242d allow koji tags to be created with a configurable permission. Fixes #1478 2019-11-22 07:41:16 -05:00
Hunor Csomortáni
0f0e809467 Tests: allow running integration tests in parallel
This is done in order to speed up test execution.

Signed-off-by: Hunor Csomortáni <csomh@redhat.com>
2019-11-20 16:47:12 +01:00
Hunor Csomortáni
b41b6b9cc1 Tests: allow reusing an existing module build
Module builds take a long time to run, which can be a pain to wait for
during integration test development.

Instead of requiring developers to locally tweak the test code to be
able to reuse module builds, allow specifying the build ID of the module
build to be reused in test.env.yaml .

Signed-off-by: Hunor Csomortáni <csomh@redhat.com>
2019-11-20 14:27:33 +01:00
Mike McLean
3435b7d1a6 Release 2.30.4 v2.30.4 2019-11-19 16:07:49 -05:00
Hunor Csomortáni
b05ae2fe9a Merge #1499 Integration test for failed build 2019-11-19 16:30:54 +00:00
Mariana Ulaieva
6095ecc5fd Integration test for failed build
Implement the integration test for the failed build scenario.
2019-11-19 13:04:02 +01:00
Chenxiong Qi
87f4f3e546 Use distro.linux_distribution instead
platform.linux_distribution is removed since Python 3.8.

Signed-off-by: Chenxiong Qi <cqi@redhat.com>
2019-11-18 10:41:41 +08:00