mirror of https://gitea.com/gitea/act_runner.git synced 2026-04-24 21:00:27 +08:00

Files

Pascal Zimmermann 15dd63a839 feat: Add support for Dynamic Matrix Strategies with Job Outputs (#149 )

# Matrix Strategy: Support for Scalar Values and Template Expressions

## 🎯 Overview

This Pull Request implements support for **scalar values and template expressions** in matrix strategies. Previously, every matrix value had to be declared as a YAML array — a bare scalar like `os: ubuntu-latest` or `version: ${{ fromJSON(...) }}` caused silent failures. Now, scalars are automatically wrapped into single-element arrays, enabling the `${{ fromJSON(...) }}` pattern that is commonly used in GitHub Actions workflows for dynamic matrix generation.

## ⚠️ Semantic Changes

Two subtle behavior changes are introduced that are worth understanding:

### 1. Scalar matrix values now produce a 1-element matrix expansion

**Before this PR:** A scalar value like

```yaml
strategy:
  matrix:
    os: ubuntu-latest   # no brackets
```

failed to decode as `map[string][]interface{}`, so `Matrix()` returned `nil`. The job ran **once, without any matrix context** (`matrix.os` was undefined).

**After this PR:** The same input is wrapped to `["ubuntu-latest"]`. The job runs **one matrix iteration** with `matrix.os = "ubuntu-latest"` set.

> **Impact on existing workflows:** Workflows that accidentally used bare scalars instead of arrays will now run with a matrix context where they previously did not. The functional result is usually the same, but `matrix.*` variables are now populated. This is considered an improvement, but is a semantic change.

---

### 2. Unevaluated template expressions fall back to a literal string value

The actual resolution of `${{ fromJSON(needs.job1.outputs.versions) }}` into an array happens **before** `Matrix()` is called — in `EvaluateYamlNode` inside `pkg/runner/runner.go`. This PR does not change that evaluation path.

If evaluation succeeds → the YAML node already contains a proper array → `Matrix()` decodes it normally.

If evaluation **fails or is skipped** (e.g., expression not yet resolved) → the scalar string is now wrapped as `["${{ fromJSON(...) }}"]` → the job runs **one iteration with the literal unexpanded string** as the matrix value.

> **Note:** The test `TestJobMatrix/matrix_with_template_expression` covers this fallback path, not the successful resolution path. The literal-wrap behavior is the fallback, not the primary mechanism for template expression support.

---

## 🔑 Key Changes

### 1. Flexible Matrix Decoding (`pkg/model/workflow.go`)

#### New Helper Function: `normalizeMatrixValue`
```go
func normalizeMatrixValue(key string, val interface{}) ([]interface{}, error)
```
- Converts a single matrix value to the expected array format
- Handles three cases:
  1. **Arrays**: Pass through unchanged
  2. **Scalars**: Wrap in single-element array (including template expressions)
  3. **Invalid types** (maps, etc.): Return error to detect misconfigurations early

**Supported scalar types:**
- `string` — including unevaluated template expressions
- `int`, `float64` — numeric values
- `bool` — boolean values
- `nil` — null values

#### Enhanced `Matrix()` Method

**Before:**
```go
func (j *Job) Matrix() map[string][]interface{} {
    if j.Strategy.RawMatrix.Kind == yaml.MappingNode {
        var val map[string][]interface{}
        if !decodeNode(j.Strategy.RawMatrix, &val) {
            return nil
        }
        return val
    }
    return nil
}
```

**After:**
```go
func (j *Job) Matrix() map[string][]interface{} {
    if j.Strategy == nil || j.Strategy.RawMatrix.Kind != yaml.MappingNode {
        return nil
    }

    // First decode to flexible map[string]interface{} to handle scalars and arrays
    var flexVal map[string]interface{}
    err := j.Strategy.RawMatrix.Decode(&flexVal)
    if err != nil {
        // If that fails, try the strict array format
        var val map[string][]interface{}
        if !decodeNode(j.Strategy.RawMatrix, &val) {
            return nil
        }
        return val
    }

    // Convert flexible format to expected format with validation
    val := make(map[string][]interface{})
    for k, v := range flexVal {
        normalized, err := normalizeMatrixValue(k, v)
        if err != nil {
            log.Errorf("matrix validation error: %v", err)
            return nil
        }
        val[k] = normalized
    }
    return val
}
```

**Key improvements:**
- Handles unevaluated template expressions as scalar strings
- Automatic conversion to array format for consistent processing
- Validation to catch nested maps and invalid configurations
- Fallback to strict array format decoding for backward compatibility
- Early nil check for strategy to prevent nil pointer panics

---

## 🎁 Use Cases

### Example 1: Template Expression in Matrix (primary motivation)
```yaml
jobs:
  setup:
    runs-on: ubuntu-latest
    outputs:
      versions: ${{ steps.version-map.outputs.versions }}
    steps:
      - id: version-map
        run: echo "versions=[\"1.17\",\"1.18\",\"1.19\"]" >> $GITHUB_OUTPUT

  test:
    needs: setup
    strategy:
      matrix:
        # EvaluateYamlNode resolves this to an array before Matrix() is called
        version: ${{ fromJSON(needs.setup.outputs.versions) }}
    runs-on: ubuntu-latest
    steps:
      - run: echo "Testing version ${{ matrix.version }}"
```

### Example 2: Mixed Scalar and Array Values
```yaml
jobs:
  build:
    strategy:
      matrix:
        os: [ubuntu-latest, windows-latest]  # Array — unchanged
        go-version: 1.19                      # Scalar — wrapped to [1.19]
        node: [14, 16, 18]                    # Array — unchanged
    runs-on: ${{ matrix.os }}
```

### Example 3: Single Value Matrix
```yaml
jobs:
  deploy:
    strategy:
      matrix:
        environment: production  # Scalar — wrapped to ["production"]
    runs-on: ubuntu-latest
```

---

## ✅ Benefits

- ✨ **Feature Parity with GitHub Actions**: Supports template expressions in matrix strategies
- 🔒 **Robust Validation**: Detects misconfigured matrices (nested maps) early
- 🔄 **Backward Compatible**: Existing array-based workflows continue to work
- 📝 **Clear Error Messages**: Helpful validation messages for debugging
- 🛡️ **Type Safety**: Handles all YAML scalar types correctly

---

## 🚀 Breaking Changes

**Technically none in a strictly backward-incompatible sense**, but two semantic changes exist (see [⚠️ Semantic Changes](#️-semantic-changes-read-before-merging) above):

1. Scalar matrix values now iterate (previously they caused a `nil` matrix / no iteration)
2. Unevaluated template strings now produce a literal iteration instead of skipping

Both changes are improvements in practice, but downstream users should be aware.

---

## 📚 Related Work

- Related Gitea Server PR: [go-gitea/gitea#36564](https://github.com/go-gitea/gitea/pull/36564)
- Enables dynamic matrix generation from job outputs
- Supports advanced CI/CD patterns used in GitHub Actions

## 🔗 References

- [GitHub Actions: Matrix strategies](https://docs.github.com/en/actions/using-jobs/using-a-matrix-for-your-jobs)
- [GitHub Actions: Expression syntax](https://docs.github.com/en/actions/learn-github-actions/expressions)
- [GitHub Actions: `fromJSON` in matrix context](https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions#jobsjob_idstrategymatrix)

Reviewed-on: https://gitea.com/gitea/act/pulls/149
Reviewed-by: silverwind <2021+silverwind@noreply.gitea.com>
Co-authored-by: Pascal Zimmermann <pascal.zimmermann@theiotstudio.com>
Co-committed-by: Pascal Zimmermann <pascal.zimmermann@theiotstudio.com>

2026-04-19 22:36:34 +00:00

5.5 KiB

Raw Blame History

Forking rules

This is a custom fork of nektos/act, for the purpose of serving act_runner.

It cannot be used as command line tool anymore, but only as a library.

It's a soft fork, which means that it will track the latest release of nektos/act.

Branches:

main: default branch, contains custom changes, based on the latest release(not the latest of the master branch) of nektos/act.
nektos/master: mirror for the master branch of nektos/act.

Tags:

nektos/vX.Y.Z: mirror for vX.Y.Z of nektos/act.
vX.YZ.*: based on nektos/vX.Y.Z, contains custom changes.
- Examples:
  - nektos/v0.2.23 -> v0.223.*
  - nektos/v0.3.1 -> v0.301.*, not ~~v0.31.*~~
  - nektos/v0.10.1 -> v0.1001.*, not ~~v0.101.*~~
  - nektos/v0.3.100 -> not ~~v0.3100.*~~, I don't think it's really going to happen, if it does, we can find a way to handle it.

Gitea-specific changes

Matrix strategy: scalar values and template expressions

This fork extends the matrix strategy parser workflow.go to accept bare scalar YAML values in addition to arrays, and to handle unevaluated template expressions gracefully.

Scalar wrapping

A matrix key written without brackets is automatically promoted to a single-element array:

strategy:
  matrix:
    go-version: 1.21        # treated as [1.21]
    os: ubuntu-latest       # treated as ["ubuntu-latest"]

Note

Previously such a value caused the matrix decoding to fail and the job ran without a matrix context (matrix.* variables were undefined). Now the job runs one matrix iteration with the scalar as the value. Existing workflows that used scalars by accident may see a difference in which matrix variables are populated.

Template expression support (${{ fromJSON(...) }})

Template expressions in the matrix are resolved by EvaluateYamlNode (pkg/runner/runner.go) before Matrix() is called. When successful, the expression is replaced by a proper YAML sequence and the matrix expands normally.

If the expression cannot be resolved (e.g., the necessary context is not yet available), the literal string is wrapped as a one-element array, and the job runs once with the unexpanded string as the matrix value (graceful degradation).

Overview

"Think globally, act locally"

Run your GitHub Actions locally! Why would you want to do this? Two reasons:

Fast Feedback - Rather than having to commit/push every time you want to test out the changes you are making to your .github/workflows/ files (or for any changes to embedded GitHub actions), you can use act to run the actions locally. The environment variables and filesystem are all configured to match what GitHub provides.
Local Task Runner - I love make. However, I also hate repeating myself. With act, you can use the GitHub Actions defined in your .github/workflows/ to replace your Makefile!

How Does It Work?

When you run act it reads in your GitHub Actions from .github/workflows/ and determines the set of actions that need to be run. It uses the Docker API to either pull or build the necessary images, as defined in your workflow files and finally determines the execution path based on the dependencies that were defined. Once it has the execution path, it then uses the Docker API to run containers for each action based on the images prepared earlier. The environment variables and filesystem are all configured to match what GitHub provides.

Let's see it in action with a sample repo!

Act User Guide

Please look at the act user guide for more documentation.

Support

Need help? Ask on Gitter!

Contributing

Want to contribute to act? Awesome! Check out the contributing guidelines to get involved.

Manually building from source

Install Go tools 1.20+ - (https://golang.org/doc/install)
Clone this repo git clone git@github.com:nektos/act.git
Run unit tests with make test
Build and install: make install

5.5 KiB Raw Blame History