Commit Graph

1173 Commits

Author SHA1 Message Date
mprahl
37e4c718d9 Simplify format_mmd since scm.commit is always set or returns an exception 2019-05-06 15:13:24 +00:00
mprahl
f4dd91e9ec Add an efficiency improvement to SCM.get_full_commit_hash
Basically, if the commit has already been resolved from the branch,
the full commit hash is already cached, so just return that.
2019-05-06 15:13:24 +00:00
mprahl
3a22204255 Clean up a comment in SCM.get_latest 2019-05-06 15:13:24 +00:00
mprahl
c98ac3e4c3 Remove unnecessary getters and setters in the SCM class 2019-05-06 15:13:24 +00:00
mprahl
771fad6d50 Resolve the commit in the SCM.commit property method instead of the constructor
This will prevent the need to call `SCM.get_latest` in the constructor,
since not all SCM objects need the commit to the branch. It also fixes
the situation where a component's git repo doesn't have a "master" branch.

See https://pagure.io/fm-orchestrator/issue/1224
2019-05-06 15:13:24 +00:00
mprahl
02f00d93c1 Add debug log statements so that SCM problems are easier to debug in the future 2019-05-06 15:13:23 +00:00
mprahl
1885a8b11f Improve the readability of SCM.get_latest 2019-05-06 15:13:23 +00:00
mprahl
6187ce4a05 Update the SCM.__init__ docstring 2019-05-06 15:13:23 +00:00
mprahl
b75c3dd76e Remove SCM.is_available since it's not used 2019-05-06 15:13:23 +00:00
mprahl
3c4faf0cc0 Remove checking if the commit is set in SCM.checkout since it's always required 2019-05-06 15:13:23 +00:00
Jan Kaluza
c5d000e900 Do not overwrite stream kwarg in build_module_locally.
When `default_streams` is set, the current code overwrites `stream`
kwarg in the `for` loop handling the `default_streams`.

In this commit, the `stream` kwarg is not overwritten.
2019-04-30 08:42:59 +02:00
Valerij Maljulin
1b486b1625 Greenwave query class and configuration update
Signed-off-by: Valerij Maljulin <vmaljuli@redhat.com>
2019-04-29 18:15:23 +02:00
Jan Kaluža
f4553dd3c2 Merge #1237 [v2] Fix -debuginfo/-debugsource packages handling in KojiContentGenerator. 2019-04-26 13:24:36 +00:00
Jan Kaluza
5d346f8dd3 Fix -debuginfo/-debugsource packages handling in KojiContentGenerator.
Our current code has following issues with -debuginfo/-debugsource handling:

- The foo-debuginfo is included in the MMD only when foo is included there,
  but some special packages contain custom -debuginfo sub-packages like
  foo-common-debuginfo without matching foo-common sub-package. With our
  current code, the foo-common-debuginfo is never included in the final MMD.
- The foo-debugsource is included in the MMD only when foo.src.rpm,
  but some special packages contain custom -debuginfo sub-package.

This commit changes the handling of -debuginfo/-debugsource like this:

- The RPMs to include in the final MMD are evaluated in particular order now.
  At first we evaluate non-debug RPMs and then debug RPMs.
- When handling the foo-debuginfo/foo-debugsource RPM, we include it in
  final MMD only in one of these cases:
  - The "foo" is included in the MMD file (it means it is not filtered out).
  - The "foo" package does not exist at all (it means only foo-debuginfo exists
    and we need to include this package unless filtered out) and in the same time
    the SRPM from which this -debuginfo/-debugsource RPM has been built is included
    in a final MMD (it means that there is at least some package from this build
    included - this handles case when only foo.src.rpm and foo-debugsource.rpm
    would be included in a final MMD, which would be wrong.)

We also respect filters here, so it is possible to explicitely filter out also
-debuginfo/-debugsource packages.
2019-04-26 15:18:50 +02:00
mprahl
60160a2903 Remove extract_modulemd since it's no longer used in the code 2019-04-26 08:02:35 -04:00
mprahl
a642b5ddc0 Catch invalid modulemd exceptions using UnprocessableEntity instead of Exception 2019-04-26 08:01:50 -04:00
mprahl
6c8d48280d Use load_mmd_file instead of load_mmd where possible 2019-04-26 08:00:54 -04:00
mprahl
66c3f82160 Format the coding style across the codebase using "black" and manual tweaks
The main benefit of this commit is that the use of double quotes
is now consistent.
2019-04-26 00:32:13 -04:00
mprahl
559f0dd922 Use dedent when defining the module-build-macros specfile 2019-04-25 15:53:56 -04:00
mprahl
2fda8cdbfd Expose the stream version of a module in the API 2019-04-25 13:15:27 -04:00
mprahl
7b8947f660 Allow buildrequring a virtual stream of a base module 2019-04-25 13:15:27 -04:00
mprahl
88078a555b Define a generic error to use in MBSResolver 2019-04-25 13:15:27 -04:00
mprahl
00daedccfd Allow the virtual streams of a base module to be queryable in the database and API 2019-04-25 13:15:27 -04:00
mprahl
16091b7d86 Revert "Allow getting the latest stream of base module."
This reverts commit bc4a019e7d.

The approach taken causes MBS to submit module builds for every platform
stream, which is incorrect.
2019-04-25 13:04:18 -04:00
mprahl
d0aea40788 Support the stream_version_lte filter in the API 2019-04-25 08:12:49 -04:00
Chenxiong Qi
1146bb3043 Reuse function load_mmd
GenericResolver.extract_modulemd is not removed, but deprecated. Call of it
will result in a deprecation message printed. Any new code should call
load_mmd.

Signed-off-by: Chenxiong Qi <cqi@redhat.com>
2019-04-25 17:05:43 +08:00
mprahl
0217496d1f Refactor the API ordering to accept multiple order keyword arguments of the same direction
This also improves how a column is determined to be valid for ordering.
2019-04-23 08:37:12 -04:00
mprahl
3a7786c01b Make _get_module consistent across resolvers
This also adds additional code in the event a module is not
returned.
2019-04-23 08:26:14 -04:00
Matt Prahl
e72538b8b1 Merge #1219 Handle streams in base module stream 2019-04-23 12:24:36 +00:00
Chenxiong Qi
7c993f9165 Handle streams in base module stream
A base module's stream (the platform for RHEL) could have Z-stream suffix, e.g.
el8.0.0.z, this patch handles this Z-stream suffix and other potential streams
by returning the stream version as a float with configured suffix value. For
example, el8.1.0.z would be parsed as 080100.1. Note that, the 0.1 is totally
configured in config and it actually could be any value according to concrete
cases in practice.

Config STREAM_SUFFIXES is enabled in TestConfiguration so that tests depending
on the return value from ModuleBuild.get_stream_version are covered.

Part fixture of test TestMMDResolver.test_solve_virtual_streams is updated by
adding Z-stream suffix to platform:el8.2.0 in order to ensure this patch does
not break the MMD resolver.

Addresses FACTORY-4307

Signed-off-by: Chenxiong Qi <cqi@redhat.com>
2019-04-23 11:21:46 +08:00
Chenxiong Qi
642c37526e Fix typo in MMDResolver._deps2reqs docstring
Signed-off-by: Chenxiong Qi <cqi@redhat.com>
2019-04-23 10:39:46 +08:00
Chenxiong Qi
f3d61b8259 Pass list of stream values to _expand_mse_streams
The original description of argument stream is inconsistent with the value
passed to function _expand_mse_streams. The actual passed-in value is
Modulemd.SimpleSet. To fix this inconsistency, just pass list of streams, each
of them is a string representing a stream.

Signed-off-by: Chenxiong Qi <cqi@redhat.com>
2019-04-22 20:18:58 +08:00
Matt Prahl
4858e64d96 Merge #1210 Allow passing --platform-id (-p) to build_module_locally. 2019-04-11 17:19:22 +00:00
mprahl
83bc780a23 Add backwards-compatibility for cloning local repos with the artifact name and not the full path
This addresses #1211
2019-04-11 10:23:12 -04:00
Jan Kaluza
40ed1cbfdc Allow passing --platform-id (-p) to build_module_locally.
Currently, the PLATFORM_ID is parsed from the `/etc/os-release`. This
is good default value, but sometimes you want to build module locally
against the different platform stream.

For example building on platform:f29 against the platform:f30 modules. In
that case, we need to be able to override the host PLATFORM_ID and
set it manually chosen value.
2019-04-11 14:45:15 +02:00
Jan Kaluza
3cfefd7e0b Allow importing MMD without xmd["buildrequires"].
When importing modules for offline local builds from local repositories,
the XMD section does not have to be set at all - it gets removed during
the compose and is also MBS specific.

We need to be able to import such MMDs using the `import_mmd` method
in order to make --offline local builds working.

This commit adds new `check_buildrequires` bool kwarg in `import_mmd`
method to disable `xmd["buildrequires"]` checks to fix this.
2019-04-11 06:45:53 +02:00
Valerij Maljulin
e564edc808 Build counters
Signed-off-by: Valerij Maljulin <vmaljuli@redhat.com>
2019-04-09 15:44:45 +02:00
Jan Kaluza
bc4a019e7d Allow getting the latest stream of base module.
If base module name:stream is not found, consider `stream` as virtual stream
and return the latest (the one with highest stream_version) base module with
that virtual stream.
2019-04-04 17:21:49 +00:00
mprahl
f348def71f Fix a log message 2019-04-04 13:16:33 -04:00
mprahl
d6d0c870fc Disable Greenwave gating by default 2019-04-04 13:16:33 -04:00
mprahl
9512a49631 Allow whitelisted buildrequires with xmd.mbs.disttag_marking set to influnece the disttag 2019-04-04 09:12:27 -04:00
mprahl
e2b2da7708 Move the module name validation to validate_mmd 2019-04-04 09:12:27 -04:00
mprahl
1570db8a7e Don't allow a user to set the xmd.mbs field in their modulemd 2019-04-04 09:12:27 -04:00
mprahl
f1480feec2 Allow importing modules without a Koji tag
These module builds will basically act as metadata-only module builds.
This will be more useful as additional features stem from these types
of builds.
2019-04-04 09:09:38 -04:00
Jan Kaluza
f6ecc72fc8 Revert "Fix tests."
This reverts commit 7a975ac0ff.
2019-04-04 07:03:10 +02:00
Jan Kaluza
7a975ac0ff Fix tests. 2019-04-04 07:02:38 +02:00
Merlin Mathesius
06e903c3c1 Fixes and test improvements based on review feedback.
Signed-off-by: Merlin Mathesius <mmathesi@redhat.com>
2019-04-03 18:15:59 +00:00
Merlin Mathesius
cd76a1dca5 Update scratch context uniqueness suffix character from '.' to '_'.
Signed-off-by: Merlin Mathesius <mmathesi@redhat.com>
2019-04-03 18:15:59 +00:00
Merlin Mathesius
663b7fc4d0 Allow resubmitting the same NSV for scratch module builds.
Signed-off-by: Merlin Mathesius <mmathesi@redhat.com>
2019-04-03 18:15:59 +00:00
Owen W. Taylor
da57146bf2 GenericBuilder: Add a boolean 'succeeded' parameter to finalize
Previously MockModuleBuilder was checking the module state to see if
it should run a final createrepo, but since eafa93037f, finalize() is
called before changing the module state; add an explicit boolean to
GenericBuilder.finalize() to avoid worrying about ordering.
2019-04-03 18:12:57 +00:00