Commit Graph

915 Commits

Author SHA1 Message Date
mprahl
8a54b26fd7 Expose the NVR of component builds in the API to help future debugging 2018-08-07 10:33:14 -04:00
mprahl
a78c564073 Revert "Store the component's build ID and use that to identify the build when acting on a tag message"
This reverts commit 9bd16beeef.
2018-08-07 10:32:50 -04:00
mprahl
9bd16beeef Store the component's build ID and use that to identify the build when acting on a tag message
This resolves an issue where the component name is different than the
package being tagged due to a macro such as those used for SCLs.
2018-08-03 20:45:00 -04:00
mprahl
8c83959f58 Fix the COPR cleanup database migration for SQLite 2018-08-03 11:28:11 -04:00
mprahl
8173040ea6 Use the libmodulemd API properly when setting values on existing objects
MBS has been using the libmodulemd API incorrectly by assuming that
methods like `get_rpm_components` return the actual object in memory
and not a copy. This has been true but wasn't something sgallagh
intended. We found this out after he had me test his new version of
libmodulemd. This PR removes those assumptions.
2018-07-17 09:00:42 -04:00
Martin Curlej
dfe7660519 Removed PDC dependency from MBS and replaced it with MBS prod instance.
Signed-off-by: Martin Curlej <mcurlej@redhat.com>

Removed pdc from comments.

Signed-off-by: Martin Curlej <mcurlej@redhat.com>

Adding missing files.

Signed-off-by: Martin Curlej <mcurlej@redhat.com>

Updated PR according to review.

Signed-off-by: Martin Curlej <mcurlej@redhat.com>

Local modules builds now

Signed-off-by: Martin Curlej <mcurlej@redhat.com>

Removed copr from config

Signed-off-by: Martin Curlej <mcurlej@redhat.com>

Fixed bugs

Signed-off-by: Martin Curlej <mcurlej@redhat.com>
2018-07-04 10:13:18 +02:00
Matt Prahl
e45863e8f4 Merge #967 Define %{_module_context} and %{disttag} 2018-06-29 15:28:20 +00:00
Mikolaj Izdebski
0d29633da9 Make repo_include_all setting configurable in xmd
Previously MBS configured all Koji build tags with
repo_include_all=True extra option.  But for some modules it was
desired to be able to have tags with repo_include_all=False.

Fixes #957
2018-06-29 17:06:58 +02:00
Petr Šabata
323880999b Define %{_module_context} and %{disttag}
No reason to hide module context from the consumers, plus it's useful to
keep it in the DISTTAG tag (not to be confused with the %{dist} tag) for
tracking.

The %{disttag} feature will do nothing until RPM supports it.  This is
planned in 4.14.2.

See https://bugzilla.redhat.com/show_bug.cgi?id=1596192.

Signed-off-by: Petr Šabata <contyk@redhat.com>
2018-06-29 16:36:01 +02:00
mprahl
36a9026ca8 Remove references to COPR 2018-06-28 10:00:06 -04:00
mprahl
eb2643703f Fix a syntax error that caused the Koji session to not be returned when the Koji CG is running 2018-06-27 10:44:41 -04:00
Matt Prahl
6e075951b8 Merge #962 Support the modulemd buildopts.rpms.whitelist option 2018-06-27 11:42:49 +00:00
mprahl
74e4c99984 Support the modulemd buildopts.rpms.whitelist option 2018-06-27 07:05:06 -04:00
Martin Curlej
0aa9f014ce Added search for modules by binary rpm.
Signed-off-by: Martin Curlej <mcurlej@redhat.com>

Updated PR according to review.

Signed-off-by: Martin Curlej <mcurlej@redhat.com>

Searching by multiple koji tags + tests

Signed-off-by: Martin Curlej <mcurlej@redhat.com>
2018-06-26 15:52:37 +02:00
Ralph Bean
6981449853 Fix local builds.
I discovered that local builds have been broken by recent (good) changes in
dnf.  At the end of every batch, we regenerate the local repo with createrepo
and we also call modifyrepo to include the modulemd file as we progress.  This
was added in #467.

What we really want, is for the modulemd file to be present at the *end* of the
build.

Recently, dnf started respecting the modulemd file natively, such that builds
we built in batch0 would not show up in batch1.  They would be present in the
repo, but they would be marked as belonging to a module which was not enabled,
and so could not be pulled in.  Every module build would fail because
module-srpm-macros was in a disabled module (the module being built at the
time).

This change makes it so that the module metadata is only added to the repo at
the very end of the build.  I moved it into a `finalize` method on the builder
which the copr builder was using, and for symmetry's sake I moved the koji
content generator code to the same method on that builder.

There was a circular import issue to solve between the koji module builder and
the koji content generator modules that generated a larger diff, but is mostly
cosmetic and mock changes.
2018-06-25 17:05:22 -04:00
Mikolaj Izdebski
31d92b1484 Fix MSE for self-buildrequiring modules
We can't rely on matching name-stream to distinguish which module is
currently being built as modules can buildrequire older versions of
the same module-stream.

Fixes #954
2018-06-18 14:25:44 +02:00
Matt Prahl
5e12152acd Merge #952 Get buildrequired modules for koji tag inheritance using NSVC, not just NSV. 2018-06-15 00:38:46 +00:00
Matt Prahl
764c467c1d Merge #950 New: Support querying of modules/components with multiple state value filtering 2018-06-15 00:36:33 +00:00
Jan Kaluza
ddbeb0aca7 Get buildrequired modules for koji tag inheritance using NSVC, not just NSV. 2018-06-14 11:37:52 +02:00
Filip Valder
f1fc7ed467 New: Support querying of modules/components with multiple state value filtering 2018-06-13 09:42:38 +02:00
Martin Curlej
e2e804b1d6 Added the ability to search mbs builds by the whole NSVC string.
Signed-off-by: Martin Curlej <mcurlej@redhat.com>

Found about the magical properties of zip.

Signed-off-by: Martin Curlej <mcurlej@redhat.com>
2018-06-13 09:30:52 +02:00
Owen W. Taylor
827546dc9a module-build-macros: Use /etc instead of %_sysconfdir
Commit 42aef7d5d2 moved the macros installed by module-build-macros
from %_rpmconfdir (which is unaffected by %_prefix) to %_sysconfdir/rpm
(which follows %_prefix). This interacts badly with the Flatpak module
macros which change %_prefix to /app - when flatpak-rpm-macros is
in the buildroot while module-build-macros is built, module-build-macros
installs its macros to /app and they aren't found when building the
rest of the packages in the module.

Hardcoding /etc rather than using %_sysconfdir fixes this.
2018-05-31 17:52:04 -04:00
mprahl
2e6589ae41 Check the libmodulemd version instead of modulemd 2018-05-31 14:32:06 -04:00
Owen W. Taylor
6fd3a2586d PDCResolver: always look for local modules.
Modules that might have been added to a Mock build with --add-local-build
were only checked in one place, but there were two other places where data
was loaded directly from the PDC.
2018-05-30 15:47:14 -04:00
Owen W. Taylor
19bf94f0ea Load 'context' for local builds
'context' needed to be stored into the in-memory database and then
propagated for builds added to a MockModuleBuild with --add-local-build.
2018-05-30 15:47:14 -04:00
Owen W. Taylor
ddee4f840b MockModuleBuilder: add artifacts to artifacts not components
The built RPM artifacts were being added to the components list (the
RPMs to build) instead of the proper artifacts section. Also, they
need to be in NEVRA format, so use 'rpm -qf' to query the epochs from
the built artifacts.
2018-05-30 15:47:14 -04:00
mprahl
5930dd0b3f Set the modulemd's context as the context in the DB to correct the current values 2018-05-29 10:06:53 -04:00
mprahl
574c3eae92 Query by the context directly 2018-05-21 13:33:39 -04:00
mprahl
aa98eb1cc2 Don't determine the buildroot_groups when cleaning up stale module builds 2018-05-14 13:55:56 -04:00
mprahl
b0728b1080 Make the published messages smaller 2018-05-10 11:49:02 -04:00
mprahl
00e46c205b Show siblings and component_builds in the standard modules API 2018-05-10 11:48:00 -04:00
Ralph Bean
95eead27f3 Merge #934 Log this exception. 2018-05-08 19:55:02 +00:00
Ralph Bean
12fcd91bcd Log this exception.
If libmodulemd returns some garbage, it will be nice to know what it is.
2018-05-08 15:49:51 -04:00
Matt Prahl
3a4fd53e7a Merge #930 Fallback to the old Koji tag format for the target name when the tag name is too long 2018-05-08 19:48:56 +00:00
Ralph Bean
5a8764e8d9 py3: The dict-value objects returned by .values() doesn't support addition. 2018-05-08 14:18:00 -04:00
Ralph Bean
d0ae77c324 py3: Unicode strings must be encoded to bytes before hashing. 2018-05-08 14:17:36 -04:00
mprahl
15a7e322a6 Fallback to the old Koji tag format for the target when the tag name is too long 2018-05-08 09:11:22 -04:00
Qixiang Wan
d83e6897ca Change ModuleBuild.context to db column
1. Changed ModuleBuild's context property to db column, it's
non-nullable and default value is '00000000' to keep it consistent
with previous behaviour.

2. Changed ModuleBuild.contexts_from_mmd to return a tuple of
(ref_build_context, build_context, runtime_context, context).

3. Updated tests affected by this change.
2018-04-28 11:46:31 +08:00
Qixiang Wan
8b807a9fcd Generate koji tag from MBS and use informative name
koji now supports tags with max length of 256, we can use
more informative tag name instead of the hash one.

The new format of koji tag name is:

    module-<name>-<stream>-<version>-<context>

However when the generated tag's length > (256 - len('build')), we
fallback to the old way of name in hash format (module-<hash>).

In this change, koji tag is always generated from MBS itself, even
with pdc resolver.

FIXES: #918 #925
2018-04-26 22:04:16 +08:00
Matt Prahl
8351713828 Merge #922 Generate 'context' from hash based on buildrequires/requires stream. Reuse components only from module with the same stream_build_context. 2018-04-24 22:50:02 +00:00
Jan Kaluza
8d1d439737 Generate 'context' from hash based on buildrequires/requires stream. Reuse components only from module with the same stream_build_context. 2018-04-24 15:45:26 +02:00
Jan Kaluza
9236133cd2 Fix wrong rebase of #919. 2018-04-13 13:07:00 +02:00
Jan Kaluza
c1c95fc963 Allow defining list of packages which are blocked in the -build tag until they are built in a module. 2018-04-13 12:57:27 +02:00
Jan Kaluza
5d91c97386 Allow defining list of packages which are blocked in the -build tag until they are built in a module. 2018-04-12 09:45:57 +02:00
Jan Kaluza
836fb9db10 Keep the 'module_name:[]' in requires unexpanded in generate_expanded_mmds. 2018-04-06 10:48:59 +02:00
mprahl
05f34598eb Surface the error to the user when a module dependency can't be found 2018-04-05 14:37:08 -04:00
mprahl
aa990d3de3 Use six instead of future 2018-04-05 11:32:21 -04:00
mprahl
c2824138d3 Fix a bug in MSE when a module requires a module that isn't also a buildrequire 2018-04-05 10:58:29 -04:00
Jan Kaluža
4b9a5fb551 Merge #904 Rename a variable for clarity in load_local_builds 2018-04-05 12:13:38 +00:00
Jan Kaluza
65513afd1a Share the same in the dist tag between all MSE builds. 2018-04-05 10:42:56 +02:00