mirror of
https://gitea.com/gitea/act_runner.git
synced 2026-04-24 21:00:27 +08:00
c0f19d9a26
## Summary
This PR fixes the `max-parallel` strategy configuration for matrix jobs and resolves all failing tests in `step_action_remote_test.go`. The implementation ensures that matrix jobs respect the `max-parallel` setting, preventing resource exhaustion when running GitHub Actions workflows.
## Problem Statement
### Issue 1: max-parallel Not Working Correctly
Matrix jobs were running in parallel regardless of the `max-parallel` setting in the strategy configuration. This caused:
- Resource contention on limited runners
- Unpredictable job execution behavior
- Inability to control concurrency for resource-intensive workflows
### Issue 2: Failing Remote Action Tests
All tests in `step_action_remote_test.go` were failing due to:
- Missing `ActionCacheDir` configuration
- Incorrect mock expectations using fixed strings instead of hash-based paths
- Incompatibility with the hash-based action cache implementation
## Changes
### 1. max-parallel Implementation (`pkg/runner/runner.go`)
#### Robust Initialization
Added fallback logic to ensure `MaxParallel` is always properly initialized:
```go
if job.Strategy.MaxParallel == 0 {
job.Strategy.MaxParallel = job.Strategy.GetMaxParallel()
}
```
#### Eliminated Unnecessary Nesting
Fixed inefficient nested parallelization when only one pipeline element exists:
```go
if len(pipeline) == 1 {
// Execute directly without additional wrapper
log.Debugf("Single pipeline element, executing directly")
return pipeline[0](ctx)
}
```
#### Enhanced Logging
Added comprehensive debug and info logging:
- Shows which `maxParallel` value is being used
- Logs adjustments based on matrix size
- Reports final parallelization decisions
### 2. Worker Logging (`pkg/common/executor.go`)
Enhanced `NewParallelExecutor` with detailed worker activity logging:
```go
log.Infof("NewParallelExecutor: Creating %d workers for %d executors", parallel, len(executors))
for i := 0; i < parallel; i++ {
go func(workerID int, work <-chan Executor, errs chan<- error) {
log.Debugf("Worker %d started", workerID)
taskCount := 0
for executor := range work {
taskCount++
log.Debugf("Worker %d executing task %d", workerID, taskCount)
errs <- executor(ctx)
}
log.Debugf("Worker %d finished (%d tasks executed)", workerID, taskCount)
}(i, work, errs)
}
```
**Benefits:**
- Easy verification of correct parallelization
- Better debugging capabilities
- Clear visibility into worker activity
### 3. Documentation (`pkg/model/workflow.go`)
Added clarifying comment to ensure strategy values are always set:
```go
// Always set these values, even if there's an error later
j.Strategy.FailFast = j.Strategy.GetFailFast()
j.Strategy.MaxParallel = j.Strategy.GetMaxParallel()
```
### 4. Test Fixes (`pkg/runner/step_action_remote_test.go`)
#### Added Missing Configuration
All tests now include `ActionCacheDir`:
```go
Config: &Config{
GitHubInstance: "github.com",
ActionCacheDir: "/tmp/test-cache",
}
```
#### Hash-Based Suffix Matchers
Updated mocks to use hash-based paths instead of fixed strings:
```go
// Before
sarm.On("readAction", sar.Step, suffixMatcher("org-repo-path@ref"), ...)
// After
sarm.On("readAction", sar.Step, suffixMatcher(sar.Step.UsesHash()), ...)
```
#### Flexible Exec Matcher for Post Tests
Implemented flexible path matching for hash-based action directories:
```go
execMatcher := mock.MatchedBy(func(args []string) bool {
if len(args) != 2 {
return false
}
return args[0] == "node" && strings.Contains(args[1], "post.js")
})
```
#### Token Test Improvements
- Uses unique cache directory to force cloning
- Validates URL redirection to github.com
- Accepts realistic token behavior
### 5. New Tests
#### Unit Tests (`pkg/runner/max_parallel_test.go`)
Tests various `max-parallel` configurations:
- `max-parallel: 1` → Sequential execution
- `max-parallel: 2` → Max 2 parallel jobs
- `max-parallel: 4` (default) → Max 4 parallel jobs
- `max-parallel: 10` → Max 10 parallel jobs
#### Concurrency Test (`pkg/common/executor_max_parallel_test.go`)
Verifies that `max-parallel: 2` actually limits concurrent execution using atomic counters.
## Expected Behavior
### Before
- ❌ Jobs ran in parallel regardless of `max-parallel` setting
- ❌ Unnecessary nested parallelization (8 workers for 1 element)
- ❌ No logging to debug parallelization issues
- ❌ All `step_action_remote_test.go` tests failing
### After
- ✅ `max-parallel: 1` → Jobs run strictly sequentially
- ✅ `max-parallel: N` → Maximum N jobs run simultaneously
- ✅ Efficient single-level parallelization for matrix jobs
- ✅ Comprehensive logging for debugging
- ✅ All tests passing (10/10)
## Example Log Output
With `max-parallel: 2` and 6 matrix jobs:
```
[DEBUG] Using job.Strategy.MaxParallel: 2
[INFO] Running job with maxParallel=2 for 6 matrix combinations
[DEBUG] Single pipeline element, executing directly
[INFO] NewParallelExecutor: Creating 2 workers for 6 executors
[DEBUG] Worker 0 started
[DEBUG] Worker 1 started
[DEBUG] Worker 0 executing task 1
[DEBUG] Worker 1 executing task 1
...
[DEBUG] Worker 0 finished (3 tasks executed)
[DEBUG] Worker 1 finished (3 tasks executed)
```
## Test Results
All tests pass successfully:
```
✅ TestStepActionRemote (3 sub-tests)
✅ TestStepActionRemotePre
✅ TestStepActionRemotePreThroughAction
✅ TestStepActionRemotePreThroughActionToken
✅ TestStepActionRemotePost (4 sub-tests)
✅ TestMaxParallelStrategy
✅ TestMaxParallel2Quick
Total: 12/12 tests passing
```
## Breaking Changes
None. This PR is fully backward compatible. All changes improve existing behavior without altering the API.
## Impact
- ✅ Fixes resource management for CI/CD pipelines
- ✅ Prevents runner exhaustion on limited infrastructure
- ✅ Enables sequential execution for resource-intensive jobs
- ✅ Improves debugging with detailed logging
- ✅ Ensures test suite reliability
## Files Modified
### Core Implementation
- `pkg/runner/runner.go` - max-parallel fix + logging
- `pkg/common/executor.go` - Worker logging
- `pkg/model/workflow.go` - Documentation
### Tests
- `pkg/runner/step_action_remote_test.go` - Fixed all 10 tests
- `pkg/runner/max_parallel_test.go` - **NEW** - Unit tests
- `pkg/common/executor_max_parallel_test.go` - **NEW** - Concurrency test
## Verification
### Manual Testing
```bash
# Build
go build -o dist/act main.go
# Test with max-parallel: 2
./dist/act -W test-max-parallel-2.yml -v
# Expected: Only 2 jobs run simultaneously
```
### Automated Testing
```bash
# Run all tests
go test ./pkg/runner -run TestStepActionRemote -v
go test ./pkg/runner -run TestMaxParallel -v
go test ./pkg/common -run TestMaxParallel -v
```
## Related Issues
Fixes issues where matrix jobs in Gitea ignored the `max-parallel` strategy setting, causing resource contention and unpredictable behavior.
Reviewed-on: https://gitea.com/gitea/act/pulls/150
Reviewed-by: Lunny Xiao <xiaolunwen@gmail.com>
Reviewed-by: silverwind <silverwind@noreply.gitea.com>
Co-authored-by: Pascal Zimmermann <pascal.zimmermann@theiotstudio.com>
Co-committed-by: Pascal Zimmermann <pascal.zimmermann@theiotstudio.com>
113 lines
3.3 KiB
Go
113 lines
3.3 KiB
Go
package cmd
|
|
|
|
import (
|
|
"path/filepath"
|
|
|
|
log "github.com/sirupsen/logrus"
|
|
)
|
|
|
|
// Input contains the input for the root command
|
|
type Input struct {
|
|
actor string
|
|
workdir string
|
|
workflowsPath string
|
|
autodetectEvent bool
|
|
eventPath string
|
|
reuseContainers bool
|
|
bindWorkdir bool
|
|
secrets []string
|
|
vars []string
|
|
envs []string
|
|
inputs []string
|
|
platforms []string
|
|
dryrun bool
|
|
forcePull bool
|
|
forceRebuild bool
|
|
noOutput bool
|
|
envfile string
|
|
inputfile string
|
|
secretfile string
|
|
varfile string
|
|
insecureSecrets bool
|
|
defaultBranch string
|
|
privileged bool
|
|
usernsMode string
|
|
containerArchitecture string
|
|
containerDaemonSocket string
|
|
containerOptions string
|
|
noWorkflowRecurse bool
|
|
useGitIgnore bool
|
|
githubInstance string
|
|
containerCapAdd []string
|
|
containerCapDrop []string
|
|
autoRemove bool
|
|
artifactServerPath string
|
|
artifactServerAddr string
|
|
artifactServerPort string
|
|
noCacheServer bool
|
|
cacheServerPath string
|
|
cacheServerAddr string
|
|
cacheServerPort uint16
|
|
jsonLogger bool
|
|
noSkipCheckout bool
|
|
remoteName string
|
|
replaceGheActionWithGithubCom []string
|
|
replaceGheActionTokenWithGithubCom string
|
|
matrix []string
|
|
actionCachePath string
|
|
actionOfflineMode bool
|
|
logPrefixJobID bool
|
|
networkName string
|
|
useNewActionCache bool
|
|
localRepository []string
|
|
maxParallel int
|
|
}
|
|
|
|
func (i *Input) resolve(path string) string {
|
|
basedir, err := filepath.Abs(i.workdir)
|
|
if err != nil {
|
|
log.Fatal(err)
|
|
}
|
|
if path == "" {
|
|
return path
|
|
}
|
|
if !filepath.IsAbs(path) {
|
|
path = filepath.Join(basedir, path)
|
|
}
|
|
return path
|
|
}
|
|
|
|
// Envfile returns path to .env
|
|
func (i *Input) Envfile() string {
|
|
return i.resolve(i.envfile)
|
|
}
|
|
|
|
// Secretfile returns path to secrets
|
|
func (i *Input) Secretfile() string {
|
|
return i.resolve(i.secretfile)
|
|
}
|
|
|
|
func (i *Input) Varfile() string {
|
|
return i.resolve(i.varfile)
|
|
}
|
|
|
|
// Workdir returns path to workdir
|
|
func (i *Input) Workdir() string {
|
|
return i.resolve(".")
|
|
}
|
|
|
|
// WorkflowsPath returns path to workflow file(s)
|
|
func (i *Input) WorkflowsPath() string {
|
|
return i.resolve(i.workflowsPath)
|
|
}
|
|
|
|
// EventPath returns the path to events file
|
|
func (i *Input) EventPath() string {
|
|
return i.resolve(i.eventPath)
|
|
}
|
|
|
|
// Inputfile returns the path to the input file
|
|
func (i *Input) Inputfile() string {
|
|
return i.resolve(i.inputfile)
|
|
}
|