Please note that this patch does not change the use of database session
in MBS. So, in the frontend, the database session is still managed by
Flask-SQLAlchemy, that is the db.session. And the backend, running event
handlers, has its own database session created from SQLAclehmy session
API directly.
This patch aims to reduce the number of scoped_session created when call
original function make_db_session. For technical detailed information,
please refer to SQLAlchemy documentation Contextual/Thread-local
Sessions.
As a result, a global scoped_session is accessible from the
code running inside backend, both the event handlers and functions
called from handlers. The library code shared by frontend and backend,
like resolvers, has no change.
Similarly, db.session is only used to recreate database for every test.
Signed-off-by: Chenxiong Qi <cqi@redhat.com>
A new method _map_message is added for converting a message object to
corresponding event handler and module build. This is used to shorten the
process_message method.
process_message is refactored so that:
* when handler is NO_OP, just return as earlier as possible because setting it
make no sense to set MBSConsumer.current_module_build_id back and forth.
* Re-raise error thrown from handler execution so that the caller is able to
get a chance to collect monitoring metric inside except clause handling
Exception. Otherwise, no failed metric is collected. This is the major
problem this patch is to fix.
* Ensure MBSConsumer.current_module_build_id is set back to None.
In consumer method, no traceback is logged inside except clause catching
Exception. process_method does that.
Signed-off-by: Chenxiong Qi <cqi@redhat.com>
It is not necessary to initialize the build variable before following
if-elif-else branch.
After the if-elif-else branch, if no build is found, process_message
just returns immediately. So, no need to check if build is None during
handling error raised from handler call.
Signed-off-by: Chenxiong Qi <cqi@redhat.com>
This also removes the outdated comments around authorship of each
file. If there is still interest in this information, one can just
look at the git history.
This patch separates the use of database session in different MBS components
and do not mix them together.
In general, MBS components could be separated as the REST API (implemented
based on Flask) and non-REST API including the backend build workflow
(implemented as a fedmsg consumer on top of fedmsg-hub and running
independently) and library shared by them. As a result, there are two kind of
database session used in MBS, one is created and managed by Flask-SQLAlchemy,
and another one is created from SQLAclhemy Session API directly. The goal of
this patch is to make ensure session object is used properly in the right
place.
All the changes follow these rules:
* REST API related code uses the session object db.session created and
managed by Flask-SQLAlchemy.
* Non-REST API related code uses the session object created with SQLAlchemy
Session API. Function make_db_session does that.
* Shared code does not created a new session object as much as possible.
Instead, it accepts an argument db_session.
The first two rules are applicable to tests as well.
Major changes:
* Switch tests back to run with a file-based SQLite database.
* make_session is renamed to make_db_session and SQLAlchemy connection pool
options are applied for PostgreSQL backend.
* Frontend Flask related code uses db.session
* Shared code by REST API and backend build workflow accepts SQLAlchemy session
object as an argument. For example, resolver class is constructed with a
database session, and some functions accepts an argument for database session.
* Build workflow related code use session object returned from make_db_session
and ensure db.session is not used.
* Only tests for views use db.session, and other tests use db_session fixture
to access database.
* All argument name session, that is for database access, are renamed to
db_session.
* Functions model_tests_init_data, reuse_component_init_data and
reuse_shared_userspace_init_data, which creates fixture data for
tests, are converted into pytest fixtures from original function
called inside setup_method or a test method. The reason of this
conversion is to use fixture ``db_session`` rather than create a
new one. That would also benefit the whole test suite to reduce the
number of SQLAlchemy session objects.
Signed-off-by: Chenxiong Qi <cqi@redhat.com>
Today, we ignore koji messages with a None `task_id` in two ways. First
with a conditional, and then with a second check in the message
`__init__` method. This is a belt-and-suspenders approach.
For other reasons in the UMB messaging plugin, we'd like to put this
check in just one place and use a special exception in the initializer
instead of in a conditional beforehand.
I have hotfixed this in production on mbs-backend01::
Traceback (most recent call last):
File "module_build_service/scheduler/consumer.py", line 134, in consume
self.process_message(session, msg)
File "module_build_service/scheduler/consumer.py", line 224, in process_message
for event in further_work:
UnboundLocalError: local variable 'further_work' referenced before assignment
I hit a traceback where msg was NoneType and therefore didn't have a
`msg_id` attribute, I therefore wasn't able to see the real exception
that got me there in the first place.
