This will show log like "State transition: init -> wait, ...", which is
much straightforward than showing state number "state 1->2".
Signed-off-by: Chenxiong Qi <cqi@redhat.com>
This patch separates the use of database session in different MBS components
and do not mix them together.
In general, MBS components could be separated as the REST API (implemented
based on Flask) and non-REST API including the backend build workflow
(implemented as a fedmsg consumer on top of fedmsg-hub and running
independently) and library shared by them. As a result, there are two kind of
database session used in MBS, one is created and managed by Flask-SQLAlchemy,
and another one is created from SQLAclhemy Session API directly. The goal of
this patch is to make ensure session object is used properly in the right
place.
All the changes follow these rules:
* REST API related code uses the session object db.session created and
managed by Flask-SQLAlchemy.
* Non-REST API related code uses the session object created with SQLAlchemy
Session API. Function make_db_session does that.
* Shared code does not created a new session object as much as possible.
Instead, it accepts an argument db_session.
The first two rules are applicable to tests as well.
Major changes:
* Switch tests back to run with a file-based SQLite database.
* make_session is renamed to make_db_session and SQLAlchemy connection pool
options are applied for PostgreSQL backend.
* Frontend Flask related code uses db.session
* Shared code by REST API and backend build workflow accepts SQLAlchemy session
object as an argument. For example, resolver class is constructed with a
database session, and some functions accepts an argument for database session.
* Build workflow related code use session object returned from make_db_session
and ensure db.session is not used.
* Only tests for views use db.session, and other tests use db_session fixture
to access database.
* All argument name session, that is for database access, are renamed to
db_session.
* Functions model_tests_init_data, reuse_component_init_data and
reuse_shared_userspace_init_data, which creates fixture data for
tests, are converted into pytest fixtures from original function
called inside setup_method or a test method. The reason of this
conversion is to use fixture ``db_session`` rather than create a
new one. That would also benefit the whole test suite to reduce the
number of SQLAlchemy session objects.
Signed-off-by: Chenxiong Qi <cqi@redhat.com>
This just simplifies the code. It doesn't change the functionality
at all. The tests had to be changed because it assumed that the
xmd.mbs.buildrequires section would be filled out for the input
module which isn't true.
When using default modules, this feature will add conflicts to
module-build-macros for every RPM in a buildrequired base module
that overlaps with RPMs in the buildrequired modules. This will
prevent them from being available in the buildroot, and thus
ensure that the RPMs from the buildrequired modules (non-base
modules) are used even if they have a lower NVR.
This seems to be better behaviour than simply rejecting the module build
completely. MBS still shows warning in the log that it cannot find
any compatible module, but the build continues with the base module
requested in the submitted modulemd.
A base module can set xmd.mbs.default_modules_url, which contains a
URL to a list of modules in the format of name:stream separated by
new lines. When a module buildrequires this base module, the list
of default modules are added as buildrequires of the module automatically
unless there are conflicting streams or the default module is not
in the MBS database.
Only checkout exactly the code we need to run the tests. This should help work around
errors and timeouts (early EOF/index-pack failed) from Pagure. Jenkins seems to require
a minimum of depth=2 or it fails when walking the revision history.
Using a shallow clone is safe for PRs because we're always building from the head of
the PR branch (OpenShift doesn't have a way to specify a non-head commit on a PR branch).
When building from the master branch we use depth=10 to avoid a race condition in case
multiple commits have been pushed in quick succession. The integration test job is
launched using the exact revision of the code that was used to build the images, to
ensure the results are valid.
Sometimes it is necessary recreate the Jenkins PR polling job, or
cleanup old runs. This can cause Jenkins to lose the record of which
PRs it has already seen, and retest old PRs which are long closed.
This change causes the polling job to check the state of the PR, and
ignore it if it isn't Open. Testing PRs which are in any other state
doesn't provide any value.
When NUM_CONCURRENT_BUILDS is 0, then multi-threading is disabled
when submitting builds to Koji. This is not acceptable, so this
commit makes the number of threads configurable.
Most of the issues are caused by the use of SQLAlchemy database session. Some
inline comments describe the issues in detail.
Signed-off-by: Chenxiong Qi <cqi@redhat.com>
