Commit Graph

3035 Commits

Author SHA1 Message Date
Brendan Reilly
ae004e86a0 Merge #1766 batches.py: Fix handling of "changed-and-after" 2023-03-07 21:10:16 +00:00
Owen W. Taylor
4f3c60c808 batches.py: Fix handling of "changed-and-after"
all_reused_in_prev_batch should only check components in the previous
batch, not components in all batches including future batches. This
was accidentally regressed by some code refactoring in c36bd7ebac.
2023-03-07 21:08:55 +00:00
Brendan Reilly
a518bee114 Merge #1764 local builds: improve handling of build.log 2023-03-07 18:36:43 +00:00
Owen W. Taylor
a276a33969 LocalBuildConfiguration: log to <build_dir>/build.log not <build_dir>/build-2.log
Since the build log is already within the a build-specific directory, we
don't need to put the build ID (which ends up always being "2") into the
build log filename.
2023-03-07 18:33:30 +00:00
Owen W. Taylor
1c8e2a07d8 Local builds: buffer up initial logs and replay to the module build file
Logging during a build that occurs before the build directory is created
used to be logged to the console, but not retained in the build log
file. This made referring to the build log file confusing. Solve this
by buffering logs in memory until the log file is created and replaying
them.

A little bit of hackery is needed to avoid saving dangling references to
libsolv objects.
2023-03-07 18:33:30 +00:00
Brendan Reilly
295847e672 Merge #1780 Allow krb ccache to be configured 2023-02-27 17:59:35 +00:00
jmicanek
8709c632b0 Allow krb ccache to be configured
Access to the kernel keyring may be restricted in containers. Allow users to specify non keyring ccache for kerberos.
2023-02-27 18:35:53 +01:00
Brendan Reilly
f58e07df2a Merge #1778 switch to using gssapi_login for koji 2023-02-22 15:41:26 +00:00
Mike McLean
f6dcf6c346 switch to using gssapi_login for koji
Fixes https://pagure.io/fm-orchestrator/issue/1777

koji recently dropped this old call
https://pagure.io/koji/pull-request/3599

Previously, koji was ignoring the ctx arg and wrapping gssapi_login,
so this change is completely equivalent to the old behavior.
2023-02-22 10:22:53 -05:00
Brendan Reilly
c76f541bca Merge #1762 Increase test coverage of code used in local builds: 2023-02-21 16:49:15 +00:00
Owen W. Taylor
88c138084c Test more code paths in create_local_repo_from_koji_tag 2023-02-21 16:47:24 +00:00
Owen W. Taylor
8ab293dc9e test_start_next_batch_build_repo_building: actually try to start the batch 2023-02-21 16:47:24 +00:00
Brendan Reilly
c76bc32380 Merge #1761 local.py: Pass state to modules_failed_handler() correctly 2023-02-16 18:33:20 +00:00
Owen W. Taylor
2e373d2fe0 local.py: Pass state to modules_failed_handler() correctly
Handlers take a state integer rather than a string
2023-02-16 18:10:00 +00:00
Brendan Reilly
4e4fe47e87 Merge #1681 Improve handling of configuration file 2023-02-15 19:16:56 +00:00
Owen W. Taylor
b6c11f83d8 Improve handling of configuration file
* Allow MBS_CONFIG_FILE="" to entirely suppress loading any configuration
   file (useful for running tests and avoiding loading a system-wide
   configuration file.)
 * When loading the configuration file:
   * If the default configuration file path doesn't exist, silently fall back
     to the default configuration
   * For any other OSError, print the exact error
   * Let any other exception throw through, to allow people to debug their
     configuration file
2023-01-23 18:39:25 -05:00
Brendan Reilly
ab41f01209 Merge #1769 Remove unnecessary requirements 2023-01-06 18:14:42 +00:00
Brendan Reilly
f4625e56c3 Remove unnecessary requirements 2022-12-21 10:16:50 -05:00
Brendan Reilly
75abfad341 Merge #1767 Use click instead of flask script 2022-12-20 20:27:50 +00:00
Brendan Reilly
543e2a013f Match dockerfile deps with Fedora requirements 2022-12-20 11:44:47 -05:00
Brendan Reilly
6b43180229 Replace deprecated getargspec with getfullargspec 2022-12-19 13:45:46 -05:00
Brendan Reilly
ac2a338786 Add workaround for upgradedb entrypoint 2022-12-15 15:11:47 -05:00
Brendan Reilly
60692b0c6a Fix C3I tests 2022-12-15 10:19:35 -05:00
Brendan Reilly
f12fc9c969 Use click instead of flask script 2022-12-13 13:31:56 -05:00
Brendan Reilly
e4dba5e242 Release 3.8.0 3.8.0 2022-12-07 13:06:00 -05:00
Mike McLean
407b93f9af Merge #1760 use branch name as default component ref 2022-11-22 14:40:09 +00:00
Mike McLean
a014f20f15 handle default_ref recursively for module components 2022-11-21 15:17:30 -05:00
Mike McLean
9287297943 fix unit test 2022-11-08 19:23:33 -05:00
Mike McLean
46f4f449a8 partial work 2022-11-08 14:39:24 -05:00
Mike McLean
1236fed8b8 bugfix 2022-11-08 14:39:24 -05:00
Mike McLean
79648cf188 use submitted branch as default ref for components 2022-11-08 14:39:24 -05:00
Mike McLean
d1180871de fix unit test 2022-11-08 14:39:24 -05:00
Mike McLean
879e3befae combine code for noting params in xmd 2022-11-08 14:39:24 -05:00
Brendan Reilly
1ff8f66c02 Merge #1743 Allow siblings in the garbage state 2022-11-03 14:56:39 +00:00
Brendan Reilly
9460b56f69 Release 3.7.0 3.7.0 2022-10-28 13:48:16 -04:00
Brendan Reilly
84b4199e7a Release 3.7.0 2022-10-28 13:37:53 -04:00
Brendan Reilly
bcb62004ca Merge #1747 Architecture limited module builds fail to build locally 2022-10-11 13:33:55 +00:00
Brendan Reilly
18cbd2d022 Merge #1758 Add delay to missed new repo check 2022-09-21 16:44:07 +00:00
Brendan Reilly
79e3eae43e Add delay to missed new repo check 2022-09-02 12:01:34 -04:00
Mike McLean
7f6f58401d Merge #1757 install mbs in the container images using pip 2022-08-24 21:54:07 +00:00
Mike Bonnet
ecf7da7aa3 install mbs in the container images using pip
Removing the requirement to build the rpm as part of the image build simplifies the
build process and reduces fragility introduced by trying to keep the rpm build and
image builds in sync.
2022-08-24 12:03:08 -07:00
Brendan Reilly
d676e06396 Merge #1745 Resolve issues reported by bandit 2022-07-21 13:10:10 +00:00
Brendan Reilly
d4c213d678 Resolve issues reported by bandit 2022-07-21 09:09:12 -04:00
Brendan Reilly
5066752004 Merge #1753 Pass connect args to sqlalchemy 2022-07-12 20:23:39 +00:00
Brendan Reilly
fedbfc37e1 Pass connect args to sqlalchemy 2022-06-22 13:59:44 -04:00
Brendan Reilly
3111c70f8a Merge #1744 Also run bandit during py3 unit tests 2022-05-25 15:57:17 +00:00
Brendan Reilly
3231d3584c Merge #1675 Don't traceback on failed builds 2022-05-19 18:30:20 +00:00
Brendan Reilly
687191681d Merge #1673 Improve exception and SIGINT handling for tests/test_build 2022-05-16 18:33:59 +00:00
Tomas Popela
31839917bc Architecture limited module builds fail to build locally
We use module builds as an intermediate build for building flatpaked
applications on Fedora. As Flatpaks in Fedora are officially supported
only on aarch64 and x86_64 we wanted to limit the builds just to these
architectures to save Fedora resources. We were able to do with commit
https://src.fedoraproject.org/modules/flatpak-common/c/65a01f which
works perfectly in koji/mbs, but doesn't work when run locally as the
build fails with following errors and exceptions:

info: Getting tag for flatpak-common:f36:3620220516070452
info: Start to handle flatpak-common:f36:3620220516070452:cab77b58 which is in init state.
Traceback (most recent call last):
  File "/usr/lib/python3.10/site-packages/module_build_service/scheduler/handlers/modules.py", line 182, in init
    record_module_build_arches(mmd, build)
  File "/usr/lib/python3.10/site-packages/module_build_service/scheduler/submit.py", line 150, in record_module_build_arches
    arches = get_build_arches(mmd, conf)
  File "/usr/lib/python3.10/site-packages/module_build_service/scheduler/submit.py", line 95, in get_build_arches
    new_arches = _check_buildopts_arches(mmd, arches)
  File "/usr/lib/python3.10/site-packages/module_build_service/scheduler/submit.py", line 131, in _check_buildopts_arches
    print(arches not in unsupported_arches, file=sys.stderr)
TypeError: unhashable type: 'list'
info: State transition: 'init' -> 'failed', <ModuleBuild flatpak-common, id=2, stream=f36, version=3620220516070452, scratch=False, state 'failed', batch 0, state_reason 'An unknown error occurred while validating the modulemd'>
warning: Note that retrieved module state 4 doesn't match message module state 'failed'
Traceback (most recent call last):
  File "/usr/bin/mbs-manager", line 33, in <module>
    sys.exit(load_entry_point('module-build-service==3.6.1', 'console_scripts', 'mbs-manager')())
  File "/usr/lib/python3.10/site-packages/click/core.py", line 1137, in __call__
    return self.main(*args, **kwargs)
  File "/usr/lib/python3.10/site-packages/flask/cli.py", line 596, in main
    return super().main(*args, **kwargs)
  File "/usr/lib/python3.10/site-packages/click/core.py", line 1062, in main
    rv = self.invoke(ctx)
  File "/usr/lib/python3.10/site-packages/click/core.py", line 1668, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/usr/lib/python3.10/site-packages/click/core.py", line 1404, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/usr/lib/python3.10/site-packages/click/core.py", line 763, in invoke
    return __callback(*args, **kwargs)
  File "/usr/lib/python3.10/site-packages/click/decorators.py", line 26, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "/usr/lib/python3.10/site-packages/flask/cli.py", line 440, in decorator
    return __ctx.invoke(f, *args, **kwargs)
  File "/usr/lib/python3.10/site-packages/click/core.py", line 763, in invoke
    return __callback(*args, **kwargs)
  File "/usr/lib/python3.10/site-packages/module_build_service/manage.py", line 171, in build_module_locally
    module_build_service.scheduler.local.main(module_build_ids)
  File "/usr/lib/python3.10/site-packages/module_build_service/scheduler/local.py", line 57, in main
    raise_for_failed_build(module_build_ids)
  File "/usr/lib/python3.10/site-packages/module_build_service/scheduler/local.py", line 39, in raise_for_failed_build
    raise ValueError("Local module build failed.")
ValueError: Local module build failed.

The problem is that the code as it's now will fail to proceed if it will
detect any unsupported architecture - in this case the aarch64 even
tough the local x86_64 is supported. The check should be redone so if
there's an unsupported architecture detected, we should check that isn't
not the local one and proceed with the build, otherwise fail (the local
hardware doesn't support any of the specified architectures).
2022-05-16 14:18:22 +02:00
Brendan Reilly
17819cf6a6 Merge #1671 Avoid deep recursion when handling a large queue of events 2022-05-13 16:56:00 +00:00