No implementations of MBS are using Greenwave, and there are no current plans
to do so. Koji Resolver will be sufficient for any usecase dependent on gating.
There is a need to rebuild the module builds done in CentOS 9 Stream
internally in MBS to include them in RHEL. This is currenly a hard task,
because the RPM components included in a module are usually
taken from HEAD of the branch defined by their `ref` value.
For the rebuild task, it means we would have to ensure that the HEAD
of all RPM components points to right commit hash right before we start
rebuilding CentOS 9 Stream module in internal MBS. This is very hard
and fragile thing to do, especially if there are two different modules
using the RPM component from the same branch. This is prone to race
condition and makes the rebuilds quite complex and in some cases
not possible to do without force pushes to RPM component repositories
which is not acceptable by internal dist-git policy.
This commit fixes it by allowing overriding the commit hash while
submitting the module build. This helps in the mentioned situation,
because we can keep internal RPM components branches in 1:1 sync with
CentOS 9 Stream branches and HEAD can always point to the same commit
in both internal and CentOS 9 Stream repositories.
When the module rebuild is submitted in internal MBS,
we can use this new feature to override the `ref` for each RPM component
so it points to particular commit and the requirement for HEAD to point
to this commit is no longer there.
The `ref` is overriden only internally in MBS (but it is recorded in logs
and in XMD section), so the input modulemd file is not altered. This is
the same logic as used for other overrides (`buildrequire_overrides` or
`side_tag`).
This does not bring any security problem, because it is already possible
to use commit hash in `ref`, so the package maintainer can already change
the commit hash to any particular commit by using this `ref` value.
Signed-off-by: Jan Kaluza <jkaluza@redhat.com>
This function could get called multiple times if the init handler
runs more than once. This can happen if the build failed in the
init handler due to external infrastructure being down and the
user resumes their build.
In MBS, there are two cases to send a message when a module build moves
to a new state. One is to create a new module build, with
ModuleBuild.create particularly, when user submit a module build.
Another one is to transition a module build to a new state with
ModuleBuild.transition. This commit handles these two cases in a little
different ways.
For the former, existing code is refactored by moving the publish call
outside ModuleBuild.create.
For the latter, message is sent in a hook of SQLAlchemy ORM event
after_commit rather than immediately inside the ModuleBuild.transition.
Both of these changes ensure the message is sent after the changes are
committed into database successfully. Then, the backend can have
confidence that the database has the module build data when receive a
message.
Signed-off-by: Chenxiong Qi <cqi@redhat.com>
This also includes `from __future__ import absolute_import`
in every file so that the imports are consistent in Python 2 and 3.
The Python 2 tests fail without this.
This moves the code used by the backend and API to common/submit.py,
the code used just by the API to web/submit.py, and the code used
just by the backend to scheduler/submit.py.
This puts backend specific code in either the builder or scheduler
subpackage. This puts API specific code in the new web subpackage.
Lastly, any code shared between the API and backend is placed in the
common subpackage.
This merges the configuration from conf/config.py to
module_build_service/config.py. This also greatly simplifies the logic
in `init_config`. Additionally, `init_config` is no longer aware of
Flask. This will allow us to eventually break up the configuration
between the API and the backend.
The following handler arguments are not used at all:
1. `build_id` in handlers/components.py:build_task_finalize
2. `build_name` in handlers/tags.py:tagged
Poller methods within original class MBSProducer become module level
functions and are registered as Celery periodic tasks.
Code logging the size of fedmsg-hub queue are removed from log_summary.
process_open_component_builds is still kept there and not converted to a
periodic task.
There are some small refactor:
* do not format string in logging method call.
* reformat some lines of code doing SQLAlchemy database query to make
them more readable.
Signed-off-by: Chenxiong Qi <cqi@redhat.com>
This patch drops message objects, defined by class BaseMessage and its
subclasses, and pass event info arguments to event handler directly.
Different event handler requires different arguments to handle a kind of
specific event. The event info is parsed from the raw message received
from message bus.
Signed-off-by: Chenxiong Qi <cqi@redhat.com>
For the purpose of migrating to Celery to run event handler inside a
worker, Config object is not serializable. And from the usage of config
argument, every event handler can just access module_build_service.conf
directly. This removal would make the migration easier.
Signed-off-by: Chenxiong Qi <cqi@redhat.com>
Message classes and FedmsgMessageParser are moved into dedicated Python module
under scheduler/ directory.
FedmsgMessageParser is decoupled from messaging.py by initializing a parser
object with known fedmsg services. This decouple avoids cycle import between
parser.py and messaging.py.
Signed-off-by: Chenxiong Qi <cqi@redhat.com>
Please note that this patch does not change the use of database session
in MBS. So, in the frontend, the database session is still managed by
Flask-SQLAlchemy, that is the db.session. And the backend, running event
handlers, has its own database session created from SQLAclehmy session
API directly.
This patch aims to reduce the number of scoped_session created when call
original function make_db_session. For technical detailed information,
please refer to SQLAlchemy documentation Contextual/Thread-local
Sessions.
As a result, a global scoped_session is accessible from the
code running inside backend, both the event handlers and functions
called from handlers. The library code shared by frontend and backend,
like resolvers, has no change.
Similarly, db.session is only used to recreate database for every test.
Signed-off-by: Chenxiong Qi <cqi@redhat.com>
This will result in the error actually being in the state reason
of the failed module build instead of "An unknown error occurred
while validating the modulemd".
This also removes the outdated comments around authorship of each
file. If there is still interest in this information, one can just
look at the git history.
