2125 Commits

Author SHA1 Message Date
mprahl
89ccef5e42 Release v2.14.0 v2.14.0 2019-02-14 13:39:32 -05:00
mprahl
776330a2bb Add the allowed_users configuration for service accounts to bypass the group membership check
This is required for monitoring use-cases, where we can have a Kerberos principal for a
service account but no associated account in LDAP to check group membership.
2019-02-14 11:11:18 -05:00
Jan Kaluza
40f23b65e4 Fix the handling of modulemd files with unicode characters.
This commit introduces new to_text_type helper method and calls it
for return value of all mmd.dumps() calls. That way, we always
end up with proper unicode string represntation on both python
major versions.

This commit also adds unicode character to description of all
the yaml files we use in the tests so we can be sure MBS can
handle unicode characters properly.

This might be temporary fix, depending on the result of discussion
at https://github.com/fedora-modularity/libmodulemd/issues/184.
2019-02-14 13:42:47 +00:00
Jan Kaluza
7052ea0a11 Make init handler idempotent.
This fixes the issue when module build is cancelled in init state.
2019-02-14 14:18:54 +01:00
mprahl
486d644d68 Update the module submission API documentation 2019-02-12 15:55:01 -05:00
Filip Valder
38c0ea6147 Basic monitoring w/-o MBS-specific metrics 2019-02-12 16:08:41 +00:00
mprahl
afd95f3043 Don't use more-itertools 6.0.0 in Python 2 tests 2019-02-12 15:04:16 +00:00
Jan Kaluza
73d71e11b8 Revert "KojiContentGenerator: Convert MMD to UTF-8"
This reverts commit ae79b711d8.

This breaks MBS with python3 and fixes only single occurence of this
issue. We want to fix this in libmodulemd or find a better way how
to fix this to work on both python2 and python3.

We have libmodulemd PR open to address this issue on libmodulemd
level: https://github.com/fedora-modularity/libmodulemd/issues/184.
2019-02-12 15:21:04 +01:00
Igor Gnatenko
ae79b711d8 KojiContentGenerator: Convert MMD to UTF-8
Signed-off-by: Igor Gnatenko <ignatenkobrain@fedoraproject.org>
2019-02-12 08:21:05 +01:00
Jan Kaluza
130911bb49 Do not retry on 'git ls-remote' failure, but instead fallback to 'git clone'. 2019-02-11 15:27:07 +01:00
Mike Bonnet
0b3d7809f9 test pull requests under Python 3 as well 2019-02-08 17:43:43 +00:00
Mike Bonnet
b4d3c51219 don't assign the exception to a variable to make flake8 happy 2019-02-08 17:43:43 +00:00
Mike Bonnet
8479014de2 reindent to 4 characters to make flake8 happy 2019-02-08 17:43:43 +00:00
Mike Bonnet
e01b3fd077 handle different string representations of set() 2019-02-08 17:43:43 +00:00
Mike Bonnet
dcd38db5e0 hide the missing krbV module 2019-02-08 17:43:43 +00:00
Mike Bonnet
0f76fd5591 fix unicode/str/bytes inconsistencies 2019-02-08 17:43:42 +00:00
Mike Bonnet
a44e1fe4b0 fix cases where dict views could not be used in place of lists 2019-02-08 17:43:42 +00:00
Mike Bonnet
6e62cf6867 libmodulemd does not require the branch value to be bytes 2019-02-06 16:56:33 -08:00
mprahl
fce77b48dc Release v2.13.1 v2.13.1 2019-01-28 09:14:52 -05:00
Jan Kaluza
642c8406ca Allow setting context in the imported MMD file.
We always set the "context" to DEFAULT_MODULE_CONTEXT from the historical
reasons - the context was not stored in the database before and we stored
just the build_context/runtime_context. But this is no longer true for
some time.

In this commit, the context is respected and stored in the database when
importing module using the `import_mmd` method. If the context is not set
in the imported MMD, the DEFAULT_MODULE_CONTEXT is used.
2019-01-28 12:16:07 +01:00
Jan Kaluza
26d707cb8e Allow resubmiting the very same module build in case it results in new MSE build.
This is needed for Fedora branching, but it is generally useful.

For example, there is a module buildrequiring "platform:[]". In the time
this module has been built, only platform:f29 existed, so it has been built
just against platform:f29.

After a while, the platform:f30 is released and the maintainer needs
to rebuild the module against platform:f30. Right now, he needs to create
new commit in the module and submit the build, but this will result in useless
rebuild of the module also against platform:f29.

In this commit, MBS allows to resubmit the module build in a case
there are new MSE builds to build. MBS will send all the module builds
back to the user - so the existing builds will be already marked as
"ready" and the newly submitted builds will have the "init" state in
the REST API response.

However, in case when there are no new MSE builds to build, MBS still
sends back the Conflict error as it used to. This is done for backwards
compatibility and also to not confuse the users in case when no new build
has been submitted.
2019-01-28 10:53:37 +01:00
mprahl
6817102506 Release v2.13.0 v2.13.0 2019-01-24 13:23:13 -05:00
mprahl
25cda8d2bd Shutdown fedmsg-hub when SQLAlchemy can no longer resolve DNS records 2019-01-24 10:48:28 -05:00
Luiz Carvalho
5921c109f7 Add retire command to mbs-manager
With this command, admins can retire module builds that should no longer
be used as a dependency for other module builds.

Fixes #1021

Signed-off-by: Luiz Carvalho <lucarval@redhat.com>
2019-01-18 10:58:26 -05:00
mprahl
a94ad0a20b Release v2.12.2 v2.12.2 2019-01-17 09:37:53 -05:00
mprahl
5b0cb105f8 Add a missing entry to the change log 2019-01-17 09:35:20 -05:00
mprahl
c95f1897f4 Fix the default distgits config value to match the default value for RPMS_DEFAULT_REPOSITORY
This was missed as part of PR#1120, where the default dist-git URLs were changed.
2019-01-17 09:07:22 -05:00
Matt Prahl
812b43821e Merge #1122 Pin the python-ldap3 version in the unit tests while RHBZ#1664859 gets resolved 2019-01-16 21:02:43 +00:00
mprahl
522db2743a Release v2.12.1 v2.12.1 2019-01-16 16:00:01 -05:00
mprahl
fed906f9b4 Pin the python-ldap3 version in the unit tests while RHBZ#1664859 gets resolved 2019-01-16 15:49:24 -05:00
mprahl
9f9a025958 Use the https protocol to clone dist-git repos instead of the git protocol
You can read about this at:
https://fedoraproject.org/wiki/Infrastructure/HTTPS-commits

The bug was reported at:
https://bugzilla.redhat.com/show_bug.cgi?id=1666257
2019-01-16 19:00:25 +00:00
Owen W. Taylor
31f4aee548 record_filtered_rpms: Don't discard buildrequires if filtered_rpms already is found
Due to a logic error, if record_filtered_rpms() was called on a module which
already had filtered_rpms set, the buildrequires xmd information was discarded.
This broke all local builds.
2019-01-16 12:30:44 -05:00
mprahl
82c9b993f0 Release v2.12.0 v2.12.0 2019-01-14 08:47:46 -05:00
mprahl
67ebe16f72 Read the files to be hashed as binary to ensure end of lines are not converted
When encoutering a Windows end of line (^M), io.open and open in Python 3
will convert those to UNIX end of lines by default. When reading logs
to compute the checksum, it's important those new lines aren't converted,
to ensure the checksum is correct. This caused issues in Fedora staging
because when cloning down a repo, the repoSpanner output had Windows end
of lines, and this would end up in build.log. The solution is to just read
it as binary so that Python doesn't perform these conversions.
2019-01-11 16:24:18 -05:00
Matt Prahl
64abb202db Merge #1118 Removing tests for _check_upload as long as function was separated from pull-request 2019-01-11 13:52:37 +00:00
Valerij Maljulin
020315fa4e Removing tests for _check_upload as long as function was separated from pull-request
Signed-off-by: Valerij Maljulin <vmaljuli@redhat.com>
2019-01-11 14:43:25 +01:00
Lukas Holecek
3ebc67979e Omit extra git-checkout while cloning repo
Signed-off-by: Lukas Holecek <hluk@email.cz>
2019-01-11 14:17:32 +01:00
Merlin Mathesius
9974d48709 Correct reStructuredText markup in docs
Signed-off-by: Merlin Mathesius <mmathesi@redhat.com>
2019-01-10 15:33:15 -06:00
Valerij Maljulin
436e0bef3f Fix for the test test_all_builds_in_batch_fail
Signed-off-by: Valerij Maljulin <vmaljuli@redhat.com>
2019-01-10 15:22:37 +01:00
Valerij Maljulin
51a9892398 Unit tests for koji_import
Signed-off-by: Valerij Maljulin <vmaljuli@redhat.com>
2019-01-10 15:22:37 +01:00
Valerij Maljulin
aed51dfae1 Adding exception handling while doing import
This fixes #1068

Signed-off-by: Valerij Maljulin <vmaljuli@redhat.com>
2019-01-10 15:22:37 +01:00
Petr Šabata
ff1ed9926a Don't define DistTag anymore
Fedora 30 now includes rpm-4.14.2.1-4.fc30 with support for
ModularityLabel.  We're not using the DistTag anywhere anymore and it
should be safe to drop it at this point.

Signed-off-by: Petr Šabata <contyk@redhat.com>
2019-01-04 12:02:22 +01:00
Jan Kaluza
3c7f8b7006 Handle lost message from Koji informing MBS about Component being tagged into Koji tag.
Adds new meethod checking the "tagged" and "tagged_in_final" attributes of
"complete" ComponentBuilds in the current batch of module builds
in "building" state against the Koji.

In case the Koji shows the build as tagged/tagged_in_final,
the fake "tagged" message is added to work queue.
2019-01-02 09:33:42 +01:00
mprahl
3515919a94 Add a missing entry to the change log 2018-12-19 13:06:03 -05:00
mprahl
b579a30931 Release v2.11.1 v2.11.1 2018-12-19 09:09:38 -05:00
mprahl
e237676c7e Don't overwrite the state parameter in _nudge_module_builds_in_state 2018-12-19 09:06:59 -05:00
mprahl
fec1a8be92 Fix a log statement in the poller 2018-12-18 17:17:44 -05:00
mprahl
dd1baf1682 Release v2.11.0 v2.11.0 2018-12-18 16:25:26 -05:00
Matt Prahl
8bb50cfe16 Merge #1093 List package name in failed state reason 2018-12-18 21:19:14 +00:00
mprahl
25122cb53e Modify MBS to use a separate Kerberos context per thread so both threads can use the thread keyring to store the Kerberos cache
This commit includes the backport of the changes to `krb_login` in
https://pagure.io/koji/pull-request/1187. This change is required
for our separate threads to use a separate Kerberos context per thread.
2018-12-18 16:05:55 -05:00