Commit Graph

2182 Commits

Author SHA1 Message Date
mprahl
ff251b14db Release v2.18.1 v2.18.1 2019-03-20 14:25:47 -04:00
mprahl
c65e1b0a53 Log the exception when an invalid modulemd is encountered
This will help us determine why a modulemd couldn't be parsed
other than formatting issues. An example of this is if the database
session becomes stale, the exception is not due to the modulemd being
invalid.
2019-03-20 13:43:24 -04:00
mprahl
40ce635164 Keep the database session alive when checking the base module modulemd 2019-03-20 13:42:28 -04:00
mprahl
803d5409c1 Remove semicolons from virtual modules documentation 2019-03-20 12:04:15 -04:00
Jan Kaluza
1631a1ba4f Release 2.18.0 v2.18.0 2019-03-20 09:22:42 +01:00
mprahl
a8a8913bbc Add the ability to override a buildrequired module stream based on a module's branch
This moves the functionality that was in rpkg to MBS. See the following
PRs for more context:
https://pagure.io/rpkg/pull-request/390
https://pagure.io/rpkg/pull-request/420
2019-03-19 16:45:00 -04:00
mprahl
998084b441 Don't default the module name to "unnamed" on a direct modulemd submission
The module name should either be set in the modulemd file or provided
by the `module_name` parameter.
2019-03-19 08:31:33 -04:00
mprahl
275e7e4509 Remove the concept of required and optional parameters
Since the required parameters vary based on if the modulemd
comes from SCM or a direct submission, the concept of optional
parameters doesn't really apply.
2019-03-19 08:02:39 -04:00
mprahl
6d61a59b13 Clean up the valid API parameters
The MBS submission API endpoint should not accept every parameter
that is also a column on the ModuleBuild table. There are two
reasons for this. The first is that a user should be notified if
the supplied parameter is invalid, whereas it could get silently
ignored. The second reason is that a nefarious user could pass
in specially crafted API parameters causing MBS to do something
unexpected or undesired.
2019-03-19 08:01:23 -04:00
Merlin Mathesius
7b7b07211e Add/improve documentation for scratch module build requests,
particularly for the new 'modulemd' and 'module_name' parameters.

Signed-off-by: Merlin Mathesius <mmathesi@redhat.com>
2019-03-19 11:56:47 +00:00
Merlin Mathesius
aaf9eaa6ec Use log.exception when reporting JSON parsing exceptions.
Signed-off-by: Merlin Mathesius <mmathesi@redhat.com>
2019-03-19 11:56:47 +00:00
Merlin Mathesius
a46a4ec470 Accept modulemd for scratch module builds as a parameter in the submitted JSON.
Signed-off-by: Merlin Mathesius <mmathesi@redhat.com>
2019-03-19 11:56:47 +00:00
mprahl
64053e7fed Document virtual platforms 2019-03-19 11:40:20 +00:00
mprahl
3d3da4def1 Fix RHBZ#1686473 2019-03-18 13:30:04 -04:00
Jan Kaluza
765e640129 Allow setting the Koji tag extra options using the conf.koji_tag_extra_opts.
The Koji tag extra options used to be hard-coded and to change them,
we had to release new MBS version.

We do not change them often, but right now fedora-infra is requesting
to use new `mock.new_chroot` option and we need to release new MBS
because of that.

This commit makes such changes easier in the future.
2019-03-18 17:27:57 +00:00
mprahl
40e534ff9d Don't allow a dash in the value that will be used for an RPM disttag
When importing a base module, we must ensure the value that will be
used in the RPM disttags doesn't contain a dash since a dash isn't
allowed in the release field of the NVR.
2019-03-15 13:14:25 -04:00
mprahl
6bb41e786c Add the ability to override the base module marking used in the RPM disttags
MBS uses the base module's stream that was buildrequired by the module
in the RPM disttags for that module build. The stream name may not be
ideal for all situations, so now this is customizable by setting the
xmd['mbs']['disttag_marking'] in the base module's modulemd.
2019-03-15 13:14:25 -04:00
mprahl
63c7a35e4b Instruct the user to delete the container after the test is run 2019-03-15 12:06:51 -04:00
Jan Kaluža
2f70945ba7 Merge #1166 Ignore some SQLAlchemy DeprecationWarnings 2019-03-15 06:47:30 +00:00
Jan Kaluza
9a6646f27c Use the scrmod prefix also for build targets for scratch builds.
Currently, the Koji tags have properly set the scrmod- prefix, but
the build target still sets module- prefix even for scratch builds.

In this commit, build target has the scrmod- prefix too.
2019-03-14 13:10:47 +01:00
Valerij Maljulin
b39f92b820 Categorize log messages
This fixes #1064

Signed-off-by: Valerij Maljulin <vmaljuli@redhat.com>
2019-03-13 13:38:08 +01:00
Filip Valder
14f1b582c9 Ignore some SQLAlchemy DeprecationWarnings 2019-03-12 22:00:53 +01:00
mprahl
b570948484 Don't allow building modules with the same name as a base module (e.g. platform)
A user managed to build a module called "platform", which stopped dependency
resolution from working. This should stop that from happening again.
2019-03-12 14:58:28 -04:00
Jan Kaluza
57d5548105 Fix conversion to bytes in Python3 in tests. 2019-03-12 18:33:27 +00:00
Jan Kaluza
9778812ab5 Version 2.17.0 v2.17.0 2019-03-12 07:51:18 +01:00
Merlin Mathesius
244d873130 Always allow submitting yaml for scratch module builds.
Signed-off-by: Merlin Mathesius <mmathesi@redhat.com>
2019-03-08 11:58:46 -06:00
Jan Kaluža
b38feb5deb Merge #1160 Fix the way how KojiContentGenerator computes filesize. 2019-03-07 15:44:43 +00:00
Jan Kaluža
293e74c3a5 Merge #1158 Fix config name MODULES_ALLOW_SCRATCH in docs 2019-03-07 14:58:56 +00:00
Jan Kaluza
04082047fe Fix the way how KojiContentGenerator computes filesize.
The current code reads the data, converts them to unicode string and
then uses the `len()` of that string as filesize. This is wrong,
because Koji expects filesize to be really number of bytes, not number
of characters.

Therefore, in this commit, the filesize is computed from raw data (bytes).
2019-03-07 15:29:16 +01:00
Jan Kaluža
1a79f4d21e Merge #1159 Allow auth.get_user() method to be called multiple times. 2019-03-07 13:20:23 +00:00
Jan Kaluza
c5a9a1b8af Allow auth.get_user() method to be called multiple times.
The `ImportModuleAPI` calls the `auth.get_user()` which auths the
user using Kerberos. the `ImportModuleAPI` later calls `SCMHandler`
which in its `__init__` method calls the `auth.get_user()` again.
This leads to traceback in GSSAPI, because the user is already
authed.

This commit fixes this by caching the auth results in `flask.g`,
which is reset after each request based on the Note in
http://flask.pocoo.org/docs/1.0/appcontext/#storing-data.

This commit also marks mutual auth as OPTIONAL in `mbs-cli`,
because MBS server currently does not do mutual auth.
2019-03-07 09:37:14 +01:00
Chenxiong Qi
ea56685667 Fix config name MODULES_ALLOW_SCRATCH in docs
Signed-off-by: Chenxiong Qi <cqi@redhat.com>
2019-03-06 14:38:37 +08:00
Jan Kaluza
eb89205813 bump 2019-03-04 11:14:20 +01:00
Jan Kaluza
40e00f8aa8 Release 2.16.0 v2.16.0 2019-03-04 10:13:33 +01:00
Jan Kaluza
ac0bd8ceb8 Set time_completed before calling KojiContentGenerator.
The `KojiContentGenerator.finalize()` needs the `ModuleBuild.time_completed`.

We currently set `time_completed` once the module build transitions
into `done` state. But we have moved the `KojiContentGenerator` call
to end of `build` state, so right now it is called before the `time_completed`
is set. This leads to traceback.

In this commit, the `time_completed` is set before the `KojiContentGenerator`
call, so it is defined properly.
2019-03-04 10:03:47 +01:00
Merlin Mathesius
152419f376 Module scratch build fixups per PR review feedback:
- Keep scratch module builds in the 'done' state.
- Make koji tagging for scratch modules unique so the same
  commit can be resubmitted.
- Use alternate prefix for scratch module build components so they can
  be identified later.
- Prevent scratch build components from being reused.
- Assorted code and comment cleanup.

Signed-off-by: Merlin Mathesius <mmathesi@redhat.com>
2019-03-01 10:27:04 -06:00
Merlin Mathesius
6daa4d3776 Revise command line parsing for local module builds to require SRPMs to be
specified individually using multiple '--srpm SRPM' options.

Signed-off-by: Merlin Mathesius <mmathesi@redhat.com>
2019-03-01 10:27:04 -06:00
Merlin Mathesius
a43d684859 Updates to handle including custom SRPMs for scratch module builds.
Signed-off-by: Merlin Mathesius <mmathesi@redhat.com>
2019-03-01 10:27:04 -06:00
Merlin Mathesius
dd950857ec Update koji tagging for scratch modules.
Signed-off-by: Merlin Mathesius <mmathesi@redhat.com>
2019-03-01 10:27:04 -06:00
Merlin Mathesius
7794950802 Add preliminary module scratch build details to documentation.
Signed-off-by: Merlin Mathesius <mmathesi@redhat.com>
2019-03-01 10:27:04 -06:00
Merlin Mathesius
838feb4ecd Add scratch module build submit tests.
Signed-off-by: Merlin Mathesius <mmathesi@redhat.com>
2019-03-01 10:27:04 -06:00
Merlin Mathesius
10e9dace9c Add database migration handling of new module_builds columns for module scratch builds.
Signed-off-by: Merlin Mathesius <mmathesi@redhat.com>
2019-03-01 10:27:04 -06:00
Merlin Mathesius
2bf7813446 Add 'scratch' and 'srpms' attributes to data model for module scratch builds,
and update tests accordingly.

Signed-off-by: Merlin Mathesius <mmathesi@redhat.com>
2019-03-01 10:27:04 -06:00
Merlin Mathesius
f999d1b8fc Import json.dumps and json.loads to avoid future namespace conflicts.
Signed-off-by: Merlin Mathesius <mmathesi@redhat.com>
2019-03-01 10:27:04 -06:00
Merlin Mathesius
f2e7a8db07 Add web request options 'scratch' and 'srpms' for module scratch builds.
Signed-off-by: Merlin Mathesius <mmathesi@redhat.com>
2019-03-01 10:27:04 -06:00
Merlin Mathesius
fcac146230 Separate multipart/form-data logic from YAML handling logic.
Signed-off-by: Merlin Mathesius <mmathesi@redhat.com>
2019-03-01 10:27:04 -06:00
Jan Kaluza
1258294984 Release 2.15.0 v2.15.0 2019-03-01 15:18:23 +01:00
Jan Kaluža
93ce2da037 Merge #1154 Add simple mbs-cli client tool. 2019-03-01 12:44:45 +00:00
Jan Kaluza
d5d64f8a8a Add simple mbs-cli client tool.
The goal of this client is to allow simple access to endpoints we use
to manage MBS - like "import" module or "retire" module.

It is not intended to contain tasks available in pyrpkg.

So far only "import" module is implemented, because that's the only
command we have REST API for.
2019-03-01 12:34:03 +01:00
mprahl
8cfb7c9f99 Add a log statement to know when the poller processes a paused module build
We encountered an issue where start_next_batch_build was called twice on a
module build and it caused the module build to fail because Brew failed
one of the components since it was submitted twice. This will help us narrow
down the issue if it happens again.
2019-02-28 14:46:46 -05:00