Recover from errors with completed builds.

The base-runtime build from today had two of its 180 builds that
succeeded, but MBS failed to notice (due to an unhealthy way that I
killed it and restarted it earlier in the day).  The result was that the
component builds were done, but the module stayed stuck in the build
state even though its work was over.

This change alters the poller (which checks for builds that failed in
the buildSRPMFromSCM step which doesn't emit a message for our
consumer), to additionally check for builds that succeeded but which we
didn't notice.
This commit is contained in:
Ralph Bean
2017-03-07 21:18:10 -05:00
parent c0cf9da84d
commit c4b0f618c6

View File

@@ -77,21 +77,24 @@ class MBSProducer(PollingProducer):
.format(component_build.task_id))
task_info = koji_session.getTaskInfo(component_build.task_id)
dead_states = (
koji.TASK_STATES['CANCELED'],
koji.TASK_STATES['FAILED'],
)
state_mapping = {
# Cancelled and failed builds should be marked as failed.
koji.TASK_STATES['CANCELED']: koji.BUILD_STATES['FAILED'],
koji.TASK_STATES['FAILED']: koji.BUILD_STATES['FAILED'],
# Completed tasks should be marked as complete.
koji.TASK_STATES['CLOSED']: koji.BUILD_STATES['COMPLETE'],
}
log.info(' task {0!r} is in state {1!r}'.format(
component_build.task_id, task_info['state']))
if task_info['state'] in dead_states:
if task_info['state'] in state_mapping:
# Fake a fedmsg message on our internal queue
msg = module_build_service.messaging.KojiBuildChange(
msg_id='a faked internal message',
build_id=component_build.task_id,
task_id=component_build.task_id,
build_name=component_build.package,
build_new_state=koji.BUILD_STATES['FAILED'],
build_new_state=state_mapping[task_info['state']],
build_release=None,
build_version=None
)