16 Commits

Author SHA1 Message Date
jobrauer
c584d84b76 JIRA: RHELBLD-257,RHELBLD-310 - refactor clean_database
Replace clean_database calls with cheaper truncate operation.
Remove duplicate calls of clean_database.
Extract clean_database/init_data into fixtures.
2020-05-15 16:06:42 +02:00
mprahl
81f2bffdda Rearrange the imports to meet the style guide
This also includes `from __future__ import absolute_import`
in every file so that the imports are consistent in Python 2 and 3.
The Python 2 tests fail without this.
2020-03-03 14:48:47 -05:00
mprahl
9ddb35b8d3 Move models.py to common/models.py and views.py to web/views.py 2020-03-03 14:48:47 -05:00
mprahl
b16cb88bc3 Move db_session.py to scheduler/db_session.py 2020-03-03 14:48:47 -05:00
Chenxiong Qi
f24cd4222f Make db_session singleton
Please note that this patch does not change the use of database session
in MBS. So, in the frontend, the database session is still managed by
Flask-SQLAlchemy, that is the db.session. And the backend, running event
handlers, has its own database session created from SQLAclehmy session
API directly.

This patch aims to reduce the number of scoped_session created when call
original function make_db_session. For technical detailed information,
please refer to SQLAlchemy documentation Contextual/Thread-local
Sessions.

As a result, a global scoped_session is accessible from the
code running inside backend, both the event handlers and functions
called from handlers. The library code shared by frontend and backend,
like resolvers, has no change.

Similarly, db.session is only used to recreate database for every test.

Signed-off-by: Chenxiong Qi <cqi@redhat.com>
2019-11-07 11:06:40 +08:00
mprahl
8c6cfb702d Use small license headers in the Python files
This also removes the outdated comments around authorship of each
file. If there is still interest in this information, one can just
look at the git history.
2019-10-03 08:47:24 -04:00
Chenxiong Qi
3878affa41 Separate use of database sessions
This patch separates the use of database session in different MBS components
and do not mix them together.

In general, MBS components could be separated as the REST API (implemented
based on Flask) and non-REST API including the backend build workflow
(implemented as a fedmsg consumer on top of fedmsg-hub and running
independently) and library shared by them. As a result, there are two kind of
database session used in MBS, one is created and managed by Flask-SQLAlchemy,
and another one is created from SQLAclhemy Session API directly. The goal of
this patch is to make ensure session object is used properly in the right
place.

All the changes follow these rules:

* REST API related code uses the session object db.session created and
  managed by Flask-SQLAlchemy.
* Non-REST API related code uses the session object created with SQLAlchemy
  Session API. Function make_db_session does that.
* Shared code does not created a new session object as much as possible.
  Instead, it accepts an argument db_session.

The first two rules are applicable to tests as well.

Major changes:

* Switch tests back to run with a file-based SQLite database.
* make_session is renamed to make_db_session and SQLAlchemy connection pool
  options are applied for PostgreSQL backend.
* Frontend Flask related code uses db.session
* Shared code by REST API and backend build workflow accepts SQLAlchemy session
  object as an argument. For example, resolver class is constructed with a
  database session, and some functions accepts an argument for database session.
* Build workflow related code use session object returned from make_db_session
  and ensure db.session is not used.
* Only tests for views use db.session, and other tests use db_session fixture
  to access database.
* All argument name session, that is for database access, are renamed to
  db_session.
* Functions model_tests_init_data, reuse_component_init_data and
  reuse_shared_userspace_init_data, which creates fixture data for
  tests, are converted into pytest fixtures from original function
  called inside setup_method or a test method. The reason of this
  conversion is to use fixture ``db_session`` rather than create a
  new one. That would also benefit the whole test suite to reduce the
  number of SQLAlchemy session objects.

Signed-off-by: Chenxiong Qi <cqi@redhat.com>
2019-07-18 21:26:50 +08:00
mprahl
66c3f82160 Format the coding style across the codebase using "black" and manual tweaks
The main benefit of this commit is that the use of double quotes
is now consistent.
2019-04-26 00:32:13 -04:00
Chenxiong Qi
520a979d3d Add singleton system_resolver
system_resolver is created based on loaded configuration, which could
avoid calls like `GenericResolver.create(conf)` repeatedly in the code.

However, if some cases need to create a specific resolver explicitly,
`GenericResolver.create` could be called with addition argument, for
example db or mbs is passed to argument backend in tests.

Signed-off-by: Chenxiong Qi <cqi@redhat.com>
2018-09-28 11:00:52 +08:00
Martin Curlej
dfe7660519 Removed PDC dependency from MBS and replaced it with MBS prod instance.
Signed-off-by: Martin Curlej <mcurlej@redhat.com>

Removed pdc from comments.

Signed-off-by: Martin Curlej <mcurlej@redhat.com>

Adding missing files.

Signed-off-by: Martin Curlej <mcurlej@redhat.com>

Updated PR according to review.

Signed-off-by: Martin Curlej <mcurlej@redhat.com>

Local modules builds now

Signed-off-by: Martin Curlej <mcurlej@redhat.com>

Removed copr from config

Signed-off-by: Martin Curlej <mcurlej@redhat.com>

Fixed bugs

Signed-off-by: Martin Curlej <mcurlej@redhat.com>
2018-07-04 10:13:18 +02:00
Jan Kaluza
6a5d7267fd Save another 20s in tests. 2018-02-09 11:46:12 +01:00
mprahl
e91d09f7ca Change the format of the unit tests to pytest 2018-01-31 16:34:21 -05:00
Filip Valder
5b4a8f6240 New: Resolver API for pluggable resolvers 2018-01-29 15:54:19 +01:00
Jan Kaluza
7de120c17b Get the weight of Koji tasks and use it to sort the components in a batch instead of builds time. 2017-11-28 10:52:17 -05:00
mprahl
a8421568f9 Fix Flake8 errors 2017-11-08 08:45:58 -05:00
Jan Kaluza
b56f56a486 Use dogpile.cache to cache the default_buildroot_groups result 2017-10-26 14:03:23 +02:00