Compare commits

..

51 Commits

Author SHA1 Message Date
silverwind
0d84ec6dab fix: remove the job network even when container removal overruns (#1318)
Job teardown ran every step on one shared minute. When removing the job container took longer, the network removal ran on an expired context and the network stayed behind until the orphan reclaim, which only takes networks older than `workdir_cleanup_age`, 24h by default. All that time it held a subnet of the daemon's address pool.

The job and service containers are now removed in parallel, and the docker proxy sweep, the network and the volumes get a fresh budget after them. The idle reclaim takes unused networks and volumes once they are 10 minutes old, and network removal treats a network that is already gone as removed.

Reviewed-on: https://gitea.com/gitea/runner/pulls/1318
Reviewed-by: bircni <bircni@icloud.com>
Co-authored-by: silverwind <2021+silverwind@noreply.gitea.com>
2026-10-15 14:15:41 +00:00
silverwind
b769a9765f perf: download actions as archives once per job (#1316)
Each step refreshed its own clone of an action, so a composite using one action repository ten times fetched it ten times per job, and a single network error failed the job.

Like actions/runner, a job now downloads each action repository and ref once, as an archive extracted into a directory per commit. Each action still gets its own copy in the job container. This restores the per-job dedupe of https://gitea.com/gitea/runner/pulls/1178, which only reached a since-deleted branch.

How a download works:

1. The action URL is taken after `insteadOf` rewriting, as git would use it.
2. The archive comes from `<url>/archive/<ref>.tar.gz`, or from codeload.github.com for github.com, as for actions/runner.
3. A cached tag or branch is revalidated with its ETag, which GitHub and Gitea 28+ answer with a 304. A cached commit pin needs no request.
4. Transient failures (5xx, 408, 429 honouring `Retry-After`, dropped transfers) are retried three times with a 10 to 30 second backoff.
5. Git takes over, cloning and exporting the commit into the same directory layout, when the URL is not HTTP(S), the ref is a short commit, the host serves no usable archive, the host cannot be connected to, or the retries run out.

Median of 3 runs, git (shallow clone, then fetch) against archives, on a cold and a warm cache:

| Action | git cold | git warm | archive cold | archive warm |
|---|---|---|---|---|
| github.com actions/checkout@v4 | 937ms | 726ms | 286ms | 171ms |
| github.com actions/cache@v4 | 1006ms | 801ms | 516ms | 161ms |
| github.com actions/setup-node@v4 | 1046ms | 786ms | 463ms | 156ms |
| gitea.com actions/checkout@v4 | 2007ms | 1165ms | 667ms | 222ms |

The action cache layout changes, so existing caches download once more.

*Written by Claude.*

Fixes: https://gitea.com/gitea/runner/issues/1220
Reviewed-on: https://gitea.com/gitea/runner/pulls/1316
Reviewed-by: bircni <bircni@icloud.com>
Co-authored-by: silverwind <me@silverwind.io>
2026-10-15 12:49:37 +00:00
Lunny Xiao
9c4c66e70f enhance(report): add runner.log_report_level for the uploaded runner messages (#1310)
The runner's own messages between steps (not step output) were uploaded to the job log at `log.level`, so raising `log.level` for local debugging also changed what workflow authors see in Gitea, and lowering it hid messages from them.

Add `runner.log_report_level` to set that independently. Unset, it still follows `log.level`, so existing setups behave as before. An invalid value is rejected when the config loads.

Assisted-by: Codet
Assisted-by: Claude Code:claude-opus-5-5
Co-authored-by: silverwind <me@silverwind.io>
Co-authored-by: silverwind <2021+silverwind@noreply.gitea.com>
Reviewed-on: https://gitea.com/gitea/runner/pulls/1310
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
2026-10-14 14:15:26 +00:00
silverwind
42858c61df fix: stop pruning the cached action ref on refresh (#1312)
The action cache refresh fetched with `--prune` and the unqualified refspec `+<ref>:refs/runner/<hash>`. Git cannot map that ref back to a remote ref, so it deleted it on every other refresh. The first job after a branch or tag moved then failed with `cannot lock ref`, and a moved tag in a full clone could silently run its old commit. `--no-prune` fixes this and also overrides an inherited `fetch.prune`.

Fixes: https://gitea.com/gitea/runner/issues/1311
Reviewed-on: https://gitea.com/gitea/runner/pulls/1312
Reviewed-by: Lunny Xiao <xiaolunwen@gmail.com>
Reviewed-by: Zettat123 <39446+zettat123@noreply.gitea.com>
2026-10-14 13:15:01 +00:00
Lunny Xiao
470969dc74 chore(deps): update actionslib to v1.4.1 (#1313)
actionslib v1.4.1 no longer logs through logrus, so the workflow model no longer needs the logrus bridge. The bridge stays for docker/cli and moby/go-archive, which still log through logrus's global logger; the comment now says so.

actionslib now logs through slog's global logger, which the runner did not capture. A new slog bridge forwards it to the process-wide logger, so its messages keep the runner's level, format and secret masking.

Assisted-by: Codet
Co-authored-by: silverwind <me@silverwind.io>
Co-authored-by: bircni <bircni@icloud.com>
Reviewed-on: https://gitea.com/gitea/runner/pulls/1313
Reviewed-by: silverwind <2021+silverwind@noreply.gitea.com>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
2026-10-14 12:09:48 +00:00
silverwind
be7ddcd15a fix: give each docker proxy its own job label key (#1314)
A runner nested behind another runner's docker proxy, as in gitea.com CI, had its `com.gitea.runner.job` label overwritten by the outer proxy, so its job cleanup removed nothing. This failed `TestDockerProxyWithDaemon` depending on the CI host, for example on https://gitea.com/gitea/runner/pulls/1312.

Each proxy now sets `com.gitea.runner.job.<job>` and cleans up by that key, so nested proxies keep each other's. This replaces the `com.gitea.runner.job=<job>` label.

Reviewed-on: https://gitea.com/gitea/runner/pulls/1314
Reviewed-by: bircni <bircni@icloud.com>
Co-authored-by: silverwind <me@silverwind.io>
2026-10-14 08:00:17 +00:00
Lunny Xiao
c3c647ef83 test: use t.Context() instead of context.Background() (#1306)
The test context is cancelled when the test ends, so anything still holding it stops instead of leaking into the next test. Uses inside `t.Cleanup`, `defer` and `go func` blocks are left alone, as the context is already cancelled there.

Stacked on https://gitea.com/gitea/runner/pulls/1305.

Assisted-by: Codet
Co-authored-by: bircni <bircni@icloud.com>
Reviewed-on: https://gitea.com/gitea/runner/pulls/1306
Reviewed-by: bircni <bircni@icloud.com>
2026-10-12 21:11:17 +00:00
bircni
160e18e705 enhance: report the steps of a task as they run (#1293)
Gitea derives the steps of a job from the workflow file, so the `pre` and `post` steps of actions only show up inside "Set up job" and "Complete job", see https://github.com/go-gitea/gitea/issues/24604.

The runner now also reports the steps as it runs them (https://gitea.com/gitea/actions-proto-def/pulls/19): "Set up job", a "Pre" and "Post" step for each action that has one, the workflow steps with their evaluated names, and "Complete job", as actions/runner shows them. The per-step state Gitea reads today is unchanged, and the job log only moves "Complete job name" and "Matrix:" ahead of the pre steps, so older Gitea versions keep showing everything as before. Job hooks and containers stay part of "Set up job" and "Complete job".

The `replace` of `gitea.dev/actionslib` is swapped for the release once the proto change is merged.

Reviewed-on: https://gitea.com/gitea/runner/pulls/1293
Reviewed-by: Zettat123 <39446+zettat123@noreply.gitea.com>
Co-authored-by: bircni <bircni@icloud.com>
2026-10-12 20:35:02 +00:00
silverwind
20c8b4abc4 refactor: replace go-git with the git CLI for gitignore filtering (#1309)
Replace go-git in `exec --use-gitignore` filtering with `git ls-files`. The git CLI is already required for action clones since https://gitea.com/gitea/runner/pulls/1277, so this drops `go-git` and 14 other modules, and filtering now follows git's own ignore rules.

Also stop deleting a remote action's `.gitignore` before copying it. Tracked files are never filtered, so the deletion only made copied actions differ from GitHub's action tarballs.

Reviewed-on: https://gitea.com/gitea/runner/pulls/1309
Reviewed-by: bircni <bircni@icloud.com>
Co-authored-by: silverwind <me@silverwind.io>
2026-10-12 18:13:41 +00:00
Renovate Bot
ccb9afd7a5 chore(deps): update dependencies (#1308)
This PR contains the following updates:

| Package | Type | Update | Change | Pending | [Age](https://docs.renovatebot.com/merge-confidence/) | [Confidence](https://docs.renovatebot.com/merge-confidence/) |
|---|---|---|---|---|---|---|
| [actions/setup-node](https://github.com/actions/setup-node) ([changelog](8207627860..949feb2413)) | action | digest | `8207627` → `949feb2` |  |  |  |
| [github.com/go-git/go-git/v5](https://github.com/go-git/go-git) | require | patch | `v5.19.2` → `v5.19.3` |  | ![age](https://developer.mend.io/api/mc/badges/age/go/github.com%2fgo-git%2fgo-git%2fv5/v5.19.3?slim=true) | ![confidence](https://developer.mend.io/api/mc/badges/confidence/go/github.com%2fgo-git%2fgo-git%2fv5/v5.19.2/v5.19.3?slim=true) |
| [github.com/moby/moby/api](https://github.com/moby/moby) | require | patch | `v1.56.0` → `v1.56.1` |  | ![age](https://developer.mend.io/api/mc/badges/age/go/github.com%2fmoby%2fmoby%2fapi/v1.56.1?slim=true) | ![confidence](https://developer.mend.io/api/mc/badges/confidence/go/github.com%2fmoby%2fmoby%2fapi/v1.56.0/v1.56.1?slim=true) |
| [github.com/moby/moby/client](https://github.com/moby/moby) | require | patch | `v0.6.0` → `v0.6.1` | `v0.6.2` | ![age](https://developer.mend.io/api/mc/badges/age/go/github.com%2fmoby%2fmoby%2fclient/v0.6.1?slim=true) | ![confidence](https://developer.mend.io/api/mc/badges/confidence/go/github.com%2fmoby%2fmoby%2fclient/v0.6.0/v0.6.1?slim=true) |
| [go.opentelemetry.io/otel](https://github.com/open-telemetry/opentelemetry-go) | require | minor | `v1.46.0` → `v1.47.0` |  | ![age](https://developer.mend.io/api/mc/badges/age/go/go.opentelemetry.io%2fotel/v1.47.0?slim=true) | ![confidence](https://developer.mend.io/api/mc/badges/confidence/go/go.opentelemetry.io%2fotel/v1.46.0/v1.47.0?slim=true) |
| [go.opentelemetry.io/otel/exporters/otlp/otlptrace](https://github.com/open-telemetry/opentelemetry-go) | require | minor | `v1.46.0` → `v1.47.0` |  | ![age](https://developer.mend.io/api/mc/badges/age/go/go.opentelemetry.io%2fotel%2fexporters%2fotlp%2fotlptrace/v1.47.0?slim=true) | ![confidence](https://developer.mend.io/api/mc/badges/confidence/go/go.opentelemetry.io%2fotel%2fexporters%2fotlp%2fotlptrace/v1.46.0/v1.47.0?slim=true) |
| [go.opentelemetry.io/otel/sdk](https://github.com/open-telemetry/opentelemetry-go) | require | minor | `v1.46.0` → `v1.47.0` |  | ![age](https://developer.mend.io/api/mc/badges/age/go/go.opentelemetry.io%2fotel%2fsdk/v1.47.0?slim=true) | ![confidence](https://developer.mend.io/api/mc/badges/confidence/go/go.opentelemetry.io%2fotel%2fsdk/v1.46.0/v1.47.0?slim=true) |
| [go.opentelemetry.io/otel/trace](https://github.com/open-telemetry/opentelemetry-go) | require | minor | `v1.46.0` → `v1.47.0` |  | ![age](https://developer.mend.io/api/mc/badges/age/go/go.opentelemetry.io%2fotel%2ftrace/v1.47.0?slim=true) | ![confidence](https://developer.mend.io/api/mc/badges/confidence/go/go.opentelemetry.io%2fotel%2ftrace/v1.46.0/v1.47.0?slim=true) |
| [go.opentelemetry.io/proto/otlp](https://github.com/open-telemetry/opentelemetry-proto-go) | require | patch | `v1.11.0` → `v1.11.1` |  | ![age](https://developer.mend.io/api/mc/badges/age/go/go.opentelemetry.io%2fproto%2fotlp/v1.11.1?slim=true) | ![confidence](https://developer.mend.io/api/mc/badges/confidence/go/go.opentelemetry.io%2fproto%2fotlp/v1.11.0/v1.11.1?slim=true) |
| [golang.org/x/sync](https://pkg.go.dev/golang.org/x/sync) | require | minor | [`v0.23.0` → `v0.24.0`](https://cs.opensource.google/go/x/sync/+/refs/tags/v0.23.0...refs/tags/v0.24.0) |  | ![age](https://developer.mend.io/api/mc/badges/age/go/golang.org%2fx%2fsync/v0.24.0?slim=true) | ![confidence](https://developer.mend.io/api/mc/badges/confidence/go/golang.org%2fx%2fsync/v0.23.0/v0.24.0?slim=true) |
| [golang.org/x/sys](https://pkg.go.dev/golang.org/x/sys) | require | minor | [`v0.48.0` → `v0.49.0`](https://cs.opensource.google/go/x/sys/+/refs/tags/v0.48.0...refs/tags/v0.49.0) |  | ![age](https://developer.mend.io/api/mc/badges/age/go/golang.org%2fx%2fsys/v0.49.0?slim=true) | ![confidence](https://developer.mend.io/api/mc/badges/confidence/go/golang.org%2fx%2fsys/v0.48.0/v0.49.0?slim=true) |
| [node](https://github.com/nodejs/node) |  | digest | `0e0ff40` → `d6aa754` |  |  |  |

---

### Release Notes

<details>
<summary>go-git/go-git (github.com/go-git/go-git/v5)</summary>

### [`v5.19.3`](https://github.com/go-git/go-git/releases/tag/v5.19.3)

[Compare Source](https://github.com/go-git/go-git/compare/v5.19.2...v5.19.3)

#### What's Changed

- \[v5] git: repository, Ensure commondir is closed. by [@&#8203;arcaven](https://github.com/arcaven) in [#&#8203;2285](https://github.com/go-git/go-git/pull/2285)
- config: read repositoryformatversion in unmarshalCore by [@&#8203;c-tonneslan](https://github.com/c-tonneslan) in [#&#8203;2167](https://github.com/go-git/go-git/pull/2167)
- build: Update module github.com/go-git/go-git/v5 to v5.19.2 \[SECURITY] (releases/v5.x) by [@&#8203;go-git-renovate](https://github.com/go-git-renovate)\[bot] in [#&#8203;2304](https://github.com/go-git/go-git/pull/2304)
- \[v5] plumbing: revlist, Skip path validation in the object walk by [@&#8203;r0h1tb](https://github.com/r0h1tb) in [#&#8203;2305](https://github.com/go-git/go-git/pull/2305)
- build: Update module golang.org/x/crypto to v0.55.0 \[SECURITY] (releases/v5.x) by [@&#8203;go-git-renovate](https://github.com/go-git-renovate)\[bot] in [#&#8203;2348](https://github.com/go-git/go-git/pull/2348)
- \[v5] plumbing: transport/http, Contain credentials across redirects by [@&#8203;hiddeco](https://github.com/hiddeco) in [#&#8203;2358](https://github.com/go-git/go-git/pull/2358)
- build: Fix workflows for Go 1.27 (v5) by [@&#8203;pjbgf](https://github.com/pjbgf) in [#&#8203;2364](https://github.com/go-git/go-git/pull/2364)
- plumbing: format/packfile, Stop running TestChecksumMismatch and TestMalformedPack in parallel by [@&#8203;akashchamp](https://github.com/akashchamp) in [#&#8203;2421](https://github.com/go-git/go-git/pull/2421)
- build: Update module golang.org/x/crypto to v0.56.0 \[SECURITY] (releases/v5.x) by [@&#8203;go-git-renovate](https://github.com/go-git-renovate)\[bot] in [#&#8203;2357](https://github.com/go-git/go-git/pull/2357)
- plumbing: format/packfile, Break cyclic delta chains instead of recur… by [@&#8203;jgoosens](https://github.com/jgoosens) in [#&#8203;2340](https://github.com/go-git/go-git/pull/2340)
- build: Drop Go 1.25 by [@&#8203;pjbgf](https://github.com/pjbgf) in [#&#8203;2451](https://github.com/go-git/go-git/pull/2451)
- build: Update module github.com/go-git/go-billy/v5 to v5.9.2 (releases/v5.x) by [@&#8203;go-git-renovate](https://github.com/go-git-renovate)\[bot] in [#&#8203;2467](https://github.com/go-git/go-git/pull/2467)
- build: Bump pjbgf/sha1cd to v0.7.0 by [@&#8203;pjbgf](https://github.com/pjbgf) in [#&#8203;2468](https://github.com/go-git/go-git/pull/2468)

#### New Contributors

- [@&#8203;arcaven](https://github.com/arcaven) made their first contribution in [#&#8203;2285](https://github.com/go-git/go-git/pull/2285)
- [@&#8203;c-tonneslan](https://github.com/c-tonneslan) made their first contribution in [#&#8203;2167](https://github.com/go-git/go-git/pull/2167)
- [@&#8203;r0h1tb](https://github.com/r0h1tb) made their first contribution in [#&#8203;2305](https://github.com/go-git/go-git/pull/2305)
- [@&#8203;akashchamp](https://github.com/akashchamp) made their first contribution in [#&#8203;2421](https://github.com/go-git/go-git/pull/2421)
- [@&#8203;jgoosens](https://github.com/jgoosens) made their first contribution in [#&#8203;2340](https://github.com/go-git/go-git/pull/2340)

**Full Changelog**: <https://github.com/go-git/go-git/compare/v5.19.2...v5.19.3>

</details>

<details>
<summary>moby/moby (github.com/moby/moby/client)</summary>

### [`v0.6.1`](https://github.com/moby/moby/compare/v0.6.0...v0.6.1)

[Compare Source](https://github.com/moby/moby/compare/v0.6.0...v0.6.1)

</details>

<details>
<summary>open-telemetry/opentelemetry-go (go.opentelemetry.io/otel)</summary>

### [`v1.47.0`](https://github.com/open-telemetry/opentelemetry-go/releases/tag/v1.47.0): /v0.69.0/v0.23.0/v0.1.0

[Compare Source](https://github.com/open-telemetry/opentelemetry-go/compare/v1.46.0...v1.47.0)

This release contains the first stable release of the OpenTelemetry Go Logs API and SDK.
Our project stability guarantees now apply to the following modules:

- `go.opentelemetry.io/otel/log`
- `go.opentelemetry.io/otel/sdk/log`

See our [versioning policy](https://github.com/open-telemetry/opentelemetry-go/blob/main/VERSIONING.md) for more information about these stability guarantees.

##### Added

- Add `String` method for `KeyValue` type in `go.opentelemetry.io/otel/attribute`. ([#&#8203;8205](https://github.com/open-telemetry/opentelemetry-go/issues/8205))
- Add `String` method for `Set` type in `go.opentelemetry.io/otel/attribute`. ([#&#8203;8347](https://github.com/open-telemetry/opentelemetry-go/issues/8347))
- Add `WithMaxExportBatchSize` to `go.opentelemetry.io/otel/sdk/metric` to configure the maximum export batch size for `PeriodicReader`. ([#&#8203;8960](https://github.com/open-telemetry/opentelemetry-go/issues/8960))
- Add `WithAttributeValueDepthLimit`, `DefaultAttributeValueDepthLimit`, and `SpanLimits.AttributeValueDepthLimit` in `go.opentelemetry.io/otel/sdk/trace` to configure the maximum depth of span, event, link, and instrumentation scope attribute values. ([#&#8203;8938](https://github.com/open-telemetry/opentelemetry-go/issues/8938))
- Add `WithAttributeValueDepthLimit` in `go.opentelemetry.io/otel/sdk/log` to configure the maximum depth of log record and instrumentation scope attribute values. ([#&#8203;8938](https://github.com/open-telemetry/opentelemetry-go/issues/8938))
- Add `WithMaxResponseSize` to `go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracehttp`,
  `go.opentelemetry.io/otel/exporters/otlp/otlpmetric/otlpmetrichttp`, and
  `go.opentelemetry.io/otel/exporters/otlp/otlplog/otlploghttp`. ([#&#8203;8941](https://github.com/open-telemetry/opentelemetry-go/issues/8941))
- Add experimental `ProbabilitySampler` in `go.opentelemetry.io/otel/sdk/trace/x` that conforms to the [OpenTelemetry specification's threshold-based sampling algorithm](https://opentelemetry.io/docs/specs/otel/trace/sdk/#probabilitysampler). ([#&#8203;8123](https://github.com/open-telemetry/opentelemetry-go/issues/8123))
- Add experimental `*Binder` extension interfaces in `go.opentelemetry.io/otel/metric/x` for instruments that support binding attributes ahead of time. ([#&#8203;8760](https://github.com/open-telemetry/opentelemetry-go/issues/8760))
- Add the experimental `Finisher` synchronous metric instrument extension interface to `go.opentelemetry.io/otel/metric/x`. ([#&#8203;8906](https://github.com/open-telemetry/opentelemetry-go/issues/8906))

##### Changed

- Use the [OpenTelemetry Attribute Collection representation for non-OTLP protocols](https://opentelemetry.io/docs/specs/otel/common/#attribute-collection-representation-for-non-otlp) for metric data diffs in `go.opentelemetry.io/otel/sdk/metric/metricdata/metricdatatest`. ([#&#8203;8993](https://github.com/open-telemetry/opentelemetry-go/issues/8993))
- Apply a maximum depth of 64 by default to span, event, link, log record, and instrumentation scope attribute values in `go.opentelemetry.io/otel/sdk/trace` and `go.opentelemetry.io/otel/sdk/log`. ([#&#8203;8938](https://github.com/open-telemetry/opentelemetry-go/issues/8938))
- Reduce allocations when applying attribute value length limits in `go.opentelemetry.io/otel/sdk/trace` and `go.opentelemetry.io/otel/sdk/log` by rebuilding composite values only when truncation is needed. ([#&#8203;8912](https://github.com/open-telemetry/opentelemetry-go/issues/8912))
- Decode `traceparent` using a hex lookup table in `go.opentelemetry.io/otel/propagation`, which rejects the specification-disallowed upper-case characters without a separate scan of the header. ([#&#8203;8739](https://github.com/open-telemetry/opentelemetry-go/issues/8739))
- Decode all characters at constant indices in `TraceIDFromHex` and `SpanIDFromHex` in `go.opentelemetry.io/otel/trace` to eliminate bounds checks. ([#&#8203;8740](https://github.com/open-telemetry/opentelemetry-go/issues/8740))

##### Fixed

- Count valid U+FFFD replacement characters toward attribute value length limits in `go.opentelemetry.io/otel/trace`, `go.opentelemetry.io/otel/sdk/trace`, and `go.opentelemetry.io/otel/sdk/log`.
- Truncate string attribute values to empty strings when the configured length limit is zero, including malformed UTF-8, in `go.opentelemetry.io/otel/trace`, `go.opentelemetry.io/otel/sdk/trace`, and `go.opentelemetry.io/otel/sdk/log`. ([#&#8203;9054](https://github.com/open-telemetry/opentelemetry-go/issues/9054))
- Fix a data race in `NewPeriodicReader` where `r.inst` was assigned after the background goroutine was launched in `go.opentelemetry.io/otel/sdk/metric`. ([#&#8203;9028](https://github.com/open-telemetry/opentelemetry-go/issues/9028))
- Prevent precision loss for large `int64` values in `Sum`, `Histogram`, and `ExponentialHistogram` aggregations in `go.opentelemetry.io/otel/sdk/metric`. ([#&#8203;8981](https://github.com/open-telemetry/opentelemetry-go/issues/8981))
- Ensure `grpc.DialOption` values passed via `WithDialOption` take precedence over conflicting internally-computed defaults in
  `go.opentelemetry.io/otel/exporters/otlp/otlplog/otlploggrpc`,
  `go.opentelemetry.io/otel/exporters/otlp/otlpmetric/otlpmetricgrpc`, and
  `go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracegrpc`. ([#&#8203;8805](https://github.com/open-telemetry/opentelemetry-go/issues/8805), [#&#8203;8834](https://github.com/open-telemetry/opentelemetry-go/issues/8834), [#&#8203;8836](https://github.com/open-telemetry/opentelemetry-go/issues/8836))
- Treat overflowing `OTEL_METRIC_EXPORT_INTERVAL` and `OTEL_METRIC_EXPORT_TIMEOUT` values as invalid in `go.opentelemetry.io/otel/sdk/metric`. ([#&#8203;8800](https://github.com/open-telemetry/opentelemetry-go/issues/8800))
- Prevent a panic in `NewFixedSizeReservoir` and `FixedSizeReservoirProvider` when given a negative size in `go.opentelemetry.io/otel/sdk/metric/exemplar`; negative sizes are now clamped to zero, consistent with a size of zero. ([#&#8203;8832](https://github.com/open-telemetry/opentelemetry-go/issues/8832))
- Preserve exponential histogram `ZeroThreshold` during OTLP metric export in `go.opentelemetry.io/otel/exporters/otlp/otlpmetric`. ([#&#8203;8801](https://github.com/open-telemetry/opentelemetry-go/issues/8801))
- Ensure metric helpers in `go.opentelemetry.io/otel/semconv/v1.32.0`, `go.opentelemetry.io/otel/semconv/v1.33.0`, and `go.opentelemetry.io/otel/semconv/v1.34.0` record measurements only once when no attributes are provided. ([#&#8203;8849](https://github.com/open-telemetry/opentelemetry-go/issues/8849))
- Treat empty `OTEL_TRACES_SAMPLER` and `OTEL_TRACES_SAMPLER_ARG` values as unset in `go.opentelemetry.io/otel/sdk/trace`. ([#&#8203;8873](https://github.com/open-telemetry/opentelemetry-go/issues/8873))
- Normalize global `OTEL_EXPORTER_OTLP_ENDPOINT` path joining in `go.opentelemetry.io/otel/exporters/otlp/otlplog/otlploghttp` so trailing-slash base URLs do not produce double-slash log export paths. ([#&#8203;8864](https://github.com/open-telemetry/opentelemetry-go/issues/8864))
- Prevent a deadlock when formatting an error passed to `RecordError` calls back into the span in `go.opentelemetry.io/otel/sdk/trace`. ([#&#8203;8815](https://github.com/open-telemetry/opentelemetry-go/issues/8815))
- Prevent `SimpleSpanProcessor.Shutdown` from panicking when constructed with a nil exporter in `go.opentelemetry.io/otel/sdk/trace`. ([#&#8203;8844](https://github.com/open-telemetry/opentelemetry-go/issues/8844))
- Propagate invalid exponential histogram scale errors to Prometheus exporter self-observability metrics in `go.opentelemetry.io/otel/exporters/prometheus`. ([#&#8203;8839](https://github.com/open-telemetry/opentelemetry-go/issues/8839))
- Retain instrument advisory attributes configured with `go.opentelemetry.io/otel/metric/x.WithDefaultAttributes` when a matching View in `go.opentelemetry.io/otel/sdk/metric` does not specify an attribute filter. ([#&#8203;8859](https://github.com/open-telemetry/opentelemetry-go/issues/8859))
- Ignore HTTP URL paths when building OTLP/gRPC metric exporter targets from `OTEL_EXPORTER_OTLP_ENDPOINT` and `OTEL_EXPORTER_OTLP_METRICS_ENDPOINT`, while preserving `unix://` and `unix-abstract://` targets; treat `unix-abstract://` endpoints as insecure in `go.opentelemetry.io/otel/exporters/otlp/otlpmetric/otlpmetricgrpc` and `go.opentelemetry.io/otel/exporters/otlp/otlpmetric/otlpmetrichttp`. ([#&#8203;8862](https://github.com/open-telemetry/opentelemetry-go/issues/8862))
- Ignore HTTP(S) paths when deriving gRPC trace exporter endpoints from `OTEL_EXPORTER_OTLP_ENDPOINT` and `OTEL_EXPORTER_OTLP_TRACES_ENDPOINT` in `go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracegrpc`. ([#&#8203;8852](https://github.com/open-telemetry/opentelemetry-go/issues/8852))
- Bound memory used while parsing dense W3C property metadata by storing it compactly and materializing `Property` values on demand in `go.opentelemetry.io/otel/baggage` and `go.opentelemetry.io/otel/propagation`.

##### Removed

- Remove the experimental `OTEL_GO_X_METRIC_EXPORT_BATCH_SIZE` environment variable in `go.opentelemetry.io/otel/sdk/metric`.
  Use the `WithMaxExportBatchSize` option instead. ([#&#8203;8960](https://github.com/open-telemetry/opentelemetry-go/issues/8960))

##### What's Changed

- Prevent panic in FixedSizeReservoir for negative sizes by [@&#8203;chiliec](https://github.com/chiliec) in [#&#8203;8832](https://github.com/open-telemetry/opentelemetry-go/pull/8832)
- fix(metric): reject overflowing export durations by [@&#8203;maithilijoshi20](https://github.com/maithilijoshi20) in [#&#8203;8800](https://github.com/open-telemetry/opentelemetry-go/pull/8800)
- chore(deps): update module github.com/gofrs/flock to v0.13.1 by [@&#8203;renovate](https://github.com/renovate)\[bot] in [#&#8203;8885](https://github.com/open-telemetry/opentelemetry-go/pull/8885)
- chore(deps): update module github.com/4meepo/tagalign to v1.4.4 by [@&#8203;renovate](https://github.com/renovate)\[bot] in [#&#8203;8900](https://github.com/open-telemetry/opentelemetry-go/pull/8900)
- fix(deps): update googleapis to [`18b4a75`](https://github.com/open-telemetry/opentelemetry-go/commit/18b4a75) by [@&#8203;renovate](https://github.com/renovate)\[bot] in [#&#8203;8907](https://github.com/open-telemetry/opentelemetry-go/pull/8907)
- fix(deps): update module github.com/prometheus/client\_model to v0.6.3 by [@&#8203;renovate](https://github.com/renovate)\[bot] in [#&#8203;8908](https://github.com/open-telemetry/opentelemetry-go/pull/8908)
- chore(deps): update module github.com/prometheus/procfs to v0.22.0 by [@&#8203;renovate](https://github.com/renovate)\[bot] in [#&#8203;8894](https://github.com/open-telemetry/opentelemetry-go/pull/8894)
- chore(deps): update module github.com/godoc-lint/godoc-lint to v0.11.4 by [@&#8203;renovate](https://github.com/renovate)\[bot] in [#&#8203;8897](https://github.com/open-telemetry/opentelemetry-go/pull/8897)
- chore(deps): update github.com/charmbracelet/ultraviolet digest to [`d60c349`](https://github.com/open-telemetry/opentelemetry-go/commit/d60c349) by [@&#8203;renovate](https://github.com/renovate)\[bot] in [#&#8203;8909](https://github.com/open-telemetry/opentelemetry-go/pull/8909)
- fix(deps): update module github.com/prometheus/common to v0.71.0 by [@&#8203;renovate](https://github.com/renovate)\[bot] in [#&#8203;8911](https://github.com/open-telemetry/opentelemetry-go/pull/8911)
- chore(deps): update golang.org/x/telemetry digest to [`86cb573`](https://github.com/open-telemetry/opentelemetry-go/commit/86cb573) by [@&#8203;renovate](https://github.com/renovate)\[bot] in [#&#8203;8895](https://github.com/open-telemetry/opentelemetry-go/pull/8895)
- fix(deps): update github.com/opentracing-contrib/go-grpc/test digest to [`7bb06b0`](https://github.com/open-telemetry/opentelemetry-go/commit/7bb06b0) by [@&#8203;renovate](https://github.com/renovate)\[bot] in [#&#8203;8902](https://github.com/open-telemetry/opentelemetry-go/pull/8902)
- chore(deps): update module github.com/timonwong/loggercheck to v0.12.0 by [@&#8203;renovate](https://github.com/renovate)\[bot] in [#&#8203;8898](https://github.com/open-telemetry/opentelemetry-go/pull/8898)
- \[otlptracegrpc] Grpc dialoption override bug by [@&#8203;ps-mir](https://github.com/ps-mir) in [#&#8203;8805](https://github.com/open-telemetry/opentelemetry-go/pull/8805)
- otlpmetricgrpc: fix preference for user supplied dialOptions by [@&#8203;ps-mir](https://github.com/ps-mir) in [#&#8203;8834](https://github.com/open-telemetry/opentelemetry-go/pull/8834)
- Do not panic on nil exporter in SimpleSpanProcessor.Shutdown by [@&#8203;ShreeyashPatil2708](https://github.com/ShreeyashPatil2708) in [#&#8203;8844](https://github.com/open-telemetry/opentelemetry-go/pull/8844)
- \[chore] Add doc comments to exported internal symbols (1/N) by [@&#8203;om7057](https://github.com/om7057) in [#&#8203;8752](https://github.com/open-telemetry/opentelemetry-go/pull/8752)
- \[chore] Add doc comments to exported internal symbols (3/N) by [@&#8203;om7057](https://github.com/om7057) in [#&#8203;8869](https://github.com/open-telemetry/opentelemetry-go/pull/8869)
- \[chore] docs: mark Logs as release candidate by [@&#8203;pellared](https://github.com/pellared) in [#&#8203;8910](https://github.com/open-telemetry/opentelemetry-go/pull/8910)
- chore(deps): update module github.com/mattn/go-runewidth to v0.0.29 by [@&#8203;renovate](https://github.com/renovate)\[bot] in [#&#8203;8917](https://github.com/open-telemetry/opentelemetry-go/pull/8917)
- chore(deps): update github.com/charmbracelet/ultraviolet digest to [`af8eda3`](https://github.com/open-telemetry/opentelemetry-go/commit/af8eda3) by [@&#8203;renovate](https://github.com/renovate)\[bot] in [#&#8203;8915](https://github.com/open-telemetry/opentelemetry-go/pull/8915)
- chore(deps): update github.com/golangci/rowserrcheck digest to [`241134d`](https://github.com/open-telemetry/opentelemetry-go/commit/241134d) by [@&#8203;renovate](https://github.com/renovate)\[bot] in [#&#8203;8913](https://github.com/open-telemetry/opentelemetry-go/pull/8913)
- fix(deps): update module go.opentelemetry.io/collector/pdata to v1.66.0 by [@&#8203;renovate](https://github.com/renovate)\[bot] in [#&#8203;8918](https://github.com/open-telemetry/opentelemetry-go/pull/8918)
- chore(deps): update otel/weaver docker tag to v0.26.1 by [@&#8203;renovate](https://github.com/renovate)\[bot] in [#&#8203;8922](https://github.com/open-telemetry/opentelemetry-go/pull/8922)
- chore(deps): update open-telemetry/shared-workflows action to v0.13.0 by [@&#8203;renovate](https://github.com/renovate)\[bot] in [#&#8203;8921](https://github.com/open-telemetry/opentelemetry-go/pull/8921)
- chore(deps): update github.com/charmbracelet/ultraviolet digest to [`043ea8a`](https://github.com/open-telemetry/opentelemetry-go/commit/043ea8a) by [@&#8203;renovate](https://github.com/renovate)\[bot] in [#&#8203;8919](https://github.com/open-telemetry/opentelemetry-go/pull/8919)
- fix(otlploggrpc): honor custom dial option precedence by [@&#8203;jstar0](https://github.com/jstar0) in [#&#8203;8836](https://github.com/open-telemetry/opentelemetry-go/pull/8836)
- Define bound instrument API in metric/x by [@&#8203;dashpole](https://github.com/dashpole) in [#&#8203;8760](https://github.com/open-telemetry/opentelemetry-go/pull/8760)
- fix(prometheus): report invalid histogram scale errors by [@&#8203;jstar0](https://github.com/jstar0) in [#&#8203;8839](https://github.com/open-telemetry/opentelemetry-go/pull/8839)
- Add experimental metric Finisher interface by [@&#8203;MrAlias](https://github.com/MrAlias) in [#&#8203;8906](https://github.com/open-telemetry/opentelemetry-go/pull/8906)
- chore(deps): update github.com/charmbracelet/ultraviolet digest to [`ae99b73`](https://github.com/open-telemetry/opentelemetry-go/commit/ae99b73) by [@&#8203;renovate](https://github.com/renovate)\[bot] in [#&#8203;8923](https://github.com/open-telemetry/opentelemetry-go/pull/8923)
- chore(deps): update golang.org/x by [@&#8203;renovate](https://github.com/renovate)\[bot] in [#&#8203;8920](https://github.com/open-telemetry/opentelemetry-go/pull/8920)
- sdk/trace/x: Add experimental `ProbabilitySampler` by [@&#8203;yuanyuanzhao3](https://github.com/yuanyuanzhao3) in [#&#8203;8123](https://github.com/open-telemetry/opentelemetry-go/pull/8123)
- sdk/trace/x: Adding GoDev badge. by [@&#8203;yuanyuanzhao3](https://github.com/yuanyuanzhao3) in [#&#8203;8927](https://github.com/open-telemetry/opentelemetry-go/pull/8927)
- perf(propagation): use a hex lookup table to decode traceparent by [@&#8203;itssaharsh](https://github.com/itssaharsh) in [#&#8203;8739](https://github.com/open-telemetry/opentelemetry-go/pull/8739)
- chore(deps): update github.com/charmbracelet/ultraviolet digest to [`0277a17`](https://github.com/open-telemetry/opentelemetry-go/commit/0277a17) by [@&#8203;renovate](https://github.com/renovate)\[bot] in [#&#8203;8931](https://github.com/open-telemetry/opentelemetry-go/pull/8931)
- fix(deps): update googleapis to [`f8649dd`](https://github.com/open-telemetry/opentelemetry-go/commit/f8649dd) by [@&#8203;renovate](https://github.com/renovate)\[bot] in [#&#8203;8929](https://github.com/open-telemetry/opentelemetry-go/pull/8929)
- chore(deps): update module github.com/dlclark/regexp2/v2 to v2.8.0 by [@&#8203;renovate](https://github.com/renovate)\[bot] in [#&#8203;8930](https://github.com/open-telemetry/opentelemetry-go/pull/8930)
- \[chore] Add doc comments to exported internal symbols (5/N) by [@&#8203;om7057](https://github.com/om7057) in [#&#8203;8934](https://github.com/open-telemetry/opentelemetry-go/pull/8934)
- \[chore] Add doc comments to exported internal symbols (4/N) by [@&#8203;om7057](https://github.com/om7057) in [#&#8203;8926](https://github.com/open-telemetry/opentelemetry-go/pull/8926)
- sdk: optimize composite attribute truncation by [@&#8203;pellared](https://github.com/pellared) in [#&#8203;8912](https://github.com/open-telemetry/opentelemetry-go/pull/8912)
- chore(deps): update module github.com/go-critic/go-critic to v0.15.0 by [@&#8203;renovate](https://github.com/renovate)\[bot] in [#&#8203;8924](https://github.com/open-telemetry/opentelemetry-go/pull/8924)
- chore(deps): update module go.augendre.info/fatcontext to v0.10.1 by [@&#8203;renovate](https://github.com/renovate)\[bot] in [#&#8203;8939](https://github.com/open-telemetry/opentelemetry-go/pull/8939)
- chore(deps): update module mvdan.cc/gofumpt to v0.12.0 by [@&#8203;renovate](https://github.com/renovate)\[bot] in [#&#8203;8935](https://github.com/open-telemetry/opentelemetry-go/pull/8935)
- chore(deps): update module dev.gaijin.team/go/exhaustruct/v5 to v5.2.0 by [@&#8203;renovate](https://github.com/renovate)\[bot] in [#&#8203;8928](https://github.com/open-telemetry/opentelemetry-go/pull/8928)
- fix(deps): update golang.org/x by [@&#8203;renovate](https://github.com/renovate)\[bot] in [#&#8203;8936](https://github.com/open-telemetry/opentelemetry-go/pull/8936)
- chore(deps): update module github.com/ghostiam/protogetter to v1.0.1 by [@&#8203;renovate](https://github.com/renovate)\[bot] in [#&#8203;8940](https://github.com/open-telemetry/opentelemetry-go/pull/8940)
- fix(trace): avoid reentrant RecordError deadlock by [@&#8203;jstar0](https://github.com/jstar0) in [#&#8203;8815](https://github.com/open-telemetry/opentelemetry-go/pull/8815)
- \[chore] Add doc comments to exported internal symbols (6/6), remove exclusion by [@&#8203;om7057](https://github.com/om7057) in [#&#8203;8937](https://github.com/open-telemetry/opentelemetry-go/pull/8937)
- otlploghttp: normalize global endpoint path joining by [@&#8203;RedZapdos123](https://github.com/RedZapdos123) in [#&#8203;8864](https://github.com/open-telemetry/opentelemetry-go/pull/8864)
- sdk/trace: treat empty sampler env values as unset by [@&#8203;RedZapdos123](https://github.com/RedZapdos123) in [#&#8203;8873](https://github.com/open-telemetry/opentelemetry-go/pull/8873)
- Run full benchmarks weekly by [@&#8203;XSAM](https://github.com/XSAM) in [#&#8203;8932](https://github.com/open-telemetry/opentelemetry-go/pull/8932)
- fix(semconv): avoid duplicate metric recording without attributes by [@&#8203;jstar0](https://github.com/jstar0) in [#&#8203;8849](https://github.com/open-telemetry/opentelemetry-go/pull/8849)
- chore(deps): update module github.com/mattn/go-runewidth to v0.0.30 by [@&#8203;renovate](https://github.com/renovate)\[bot] in [#&#8203;8946](https://github.com/open-telemetry/opentelemetry-go/pull/8946)
- chore(deps): update ubuntu:26.04 docker digest to [`e5a4d62`](https://github.com/open-telemetry/opentelemetry-go/commit/e5a4d62) by [@&#8203;renovate](https://github.com/renovate)\[bot] in [#&#8203;8945](https://github.com/open-telemetry/opentelemetry-go/pull/8945)
- chore(deps): update github/codeql-action action to v4.38.0 by [@&#8203;renovate](https://github.com/renovate)\[bot] in [#&#8203;8944](https://github.com/open-telemetry/opentelemetry-go/pull/8944)
- chore(deps): update golang.org/x/telemetry digest to [`55ac43c`](https://github.com/open-telemetry/opentelemetry-go/commit/55ac43c) by [@&#8203;renovate](https://github.com/renovate)\[bot] in [#&#8203;8943](https://github.com/open-telemetry/opentelemetry-go/pull/8943)
- otlptracegrpc: fix env endpoint URL parsing by [@&#8203;RedZapdos123](https://github.com/RedZapdos123) in [#&#8203;8852](https://github.com/open-telemetry/opentelemetry-go/pull/8852)
- otlpmetricgrpc: preserve gRPC env endpoint targets by [@&#8203;RedZapdos123](https://github.com/RedZapdos123) in [#&#8203;8862](https://github.com/open-telemetry/opentelemetry-go/pull/8862)
- chore(deps): update module github.com/sourcegraph/go-diff to v0.9.0 by [@&#8203;renovate](https://github.com/renovate)\[bot] in [#&#8203;8947](https://github.com/open-telemetry/opentelemetry-go/pull/8947)
- feat(otlphttp): configure maximum response size by [@&#8203;pellared](https://github.com/pellared) in [#&#8203;8941](https://github.com/open-telemetry/opentelemetry-go/pull/8941)
- fix(otlpmetric): preserve exponential histogram zero threshold by [@&#8203;maithilijoshi20](https://github.com/maithilijoshi20) in [#&#8203;8801](https://github.com/open-telemetry/opentelemetry-go/pull/8801)
- chore(deps): update golang.org/x/telemetry digest to [`15ceca2`](https://github.com/open-telemetry/opentelemetry-go/commit/15ceca2) by [@&#8203;renovate](https://github.com/renovate)\[bot] in [#&#8203;8948](https://github.com/open-telemetry/opentelemetry-go/pull/8948)
- chore(deps): update github.com/charmbracelet/ultraviolet digest to [`6c9e17d`](https://github.com/open-telemetry/opentelemetry-go/commit/6c9e17d) by [@&#8203;renovate](https://github.com/renovate)\[bot] in [#&#8203;8950](https://github.com/open-telemetry/opentelemetry-go/pull/8950)
- chore(deps): update module github.com/dave/dst to v0.28.0 by [@&#8203;renovate](https://github.com/renovate)\[bot] in [#&#8203;8949](https://github.com/open-telemetry/opentelemetry-go/pull/8949)
- fix(deps): update googleapis to [`f61a6ca`](https://github.com/open-telemetry/opentelemetry-go/commit/f61a6ca) by [@&#8203;renovate](https://github.com/renovate)\[bot] in [#&#8203;8951](https://github.com/open-telemetry/opentelemetry-go/pull/8951)
- chore(deps): update module github.com/nishanths/exhaustive to v0.13.0 by [@&#8203;renovate](https://github.com/renovate)\[bot] in [#&#8203;8953](https://github.com/open-telemetry/opentelemetry-go/pull/8953)
- chore(deps): update module github.com/xo/terminfo to v1.2.0 by [@&#8203;renovate](https://github.com/renovate)\[bot] in [#&#8203;8954](https://github.com/open-telemetry/opentelemetry-go/pull/8954)
- sdk/log: add queue\_full shutdown regression test by [@&#8203;RedZapdos123](https://github.com/RedZapdos123) in [#&#8203;8754](https://github.com/open-telemetry/opentelemetry-go/pull/8754)
- internal/tools/verifyreadmes: skip hidden dot-directories on Windows by [@&#8203;RedZapdos123](https://github.com/RedZapdos123) in [#&#8203;8729](https://github.com/open-telemetry/opentelemetry-go/pull/8729)
- sdk/metric: add internal finish-aware Sum lifecycle by [@&#8203;MrAlias](https://github.com/MrAlias) in [#&#8203;8925](https://github.com/open-telemetry/opentelemetry-go/pull/8925)
- sdk: limit nested attribute value depth by [@&#8203;pellared](https://github.com/pellared) in [#&#8203;8938](https://github.com/open-telemetry/opentelemetry-go/pull/8938)
- chore(deps): update codecov/codecov-action action to v7.1.0 by [@&#8203;renovate](https://github.com/renovate)\[bot] in [#&#8203;8966](https://github.com/open-telemetry/opentelemetry-go/pull/8966)
- fix(deps): update module go.opentelemetry.io/collector/pdata to v1.67.0 by [@&#8203;renovate](https://github.com/renovate)\[bot] in [#&#8203;8965](https://github.com/open-telemetry/opentelemetry-go/pull/8965)
- chore(deps): update open-telemetry/shared-workflows action to v0.14.0 by [@&#8203;renovate](https://github.com/renovate)\[bot] in [#&#8203;8961](https://github.com/open-telemetry/opentelemetry-go/pull/8961)
- Fix Alex Kats emeritus mention by [@&#8203;dmathieu](https://github.com/dmathieu) in [#&#8203;8933](https://github.com/open-telemetry/opentelemetry-go/pull/8933)
- sdk/metric: stabilize max batch size in PeriodicReader by [@&#8203;dashpole](https://github.com/dashpole) in [#&#8203;8960](https://github.com/open-telemetry/opentelemetry-go/pull/8960)
- chore(deps): update ubuntu:26.04 docker digest to [`61ebaa5`](https://github.com/open-telemetry/opentelemetry-go/commit/61ebaa5) by [@&#8203;renovate](https://github.com/renovate)\[bot] in [#&#8203;8968](https://github.com/open-telemetry/opentelemetry-go/pull/8968)
- sdk/metric: Retain advisory attributes when View has no AttributeFilter by [@&#8203;dashpole](https://github.com/dashpole) in [#&#8203;8859](https://github.com/open-telemetry/opentelemetry-go/pull/8859)
- chore(deps): update module github.com/ldez/tagliatelle to v0.8.0 by [@&#8203;renovate](https://github.com/renovate)\[bot] in [#&#8203;8970](https://github.com/open-telemetry/opentelemetry-go/pull/8970)
- sdk/metric: Document nil AttributeFilter behavior by [@&#8203;pellared](https://github.com/pellared) in [#&#8203;8971](https://github.com/open-telemetry/opentelemetry-go/pull/8971)
- chore: fix gofumpt extra-rules warning by [@&#8203;dmathieu](https://github.com/dmathieu) in [#&#8203;8942](https://github.com/open-telemetry/opentelemetry-go/pull/8942)
- chore(deps): update codecov/codecov-action action to v7.1.1 by [@&#8203;renovate](https://github.com/renovate)\[bot] in [#&#8203;8975](https://github.com/open-telemetry/opentelemetry-go/pull/8975)
- chore(deps): update module github.com/ldez/gomoddirectives to v0.10.0 by [@&#8203;renovate](https://github.com/renovate)\[bot] in [#&#8203;8974](https://github.com/open-telemetry/opentelemetry-go/pull/8974)
- sdk/trace/x: deduplicate tracestate combination error report in ProbabilitySampler by [@&#8203;alex131125](https://github.com/alex131125) in [#&#8203;8969](https://github.com/open-telemetry/opentelemetry-go/pull/8969)
- \[chore]: fix article and word typos in doc comments by [@&#8203;simpleqt](https://github.com/simpleqt) in [#&#8203;8972](https://github.com/open-telemetry/opentelemetry-go/pull/8972)
- \[chore]: fix two doc comments that name the wrong method by [@&#8203;simpleqt](https://github.com/simpleqt) in [#&#8203;8973](https://github.com/open-telemetry/opentelemetry-go/pull/8973)
- Update googleapis to [`eeb232e`](https://github.com/open-telemetry/opentelemetry-go/commit/eeb232e) by [@&#8203;renovate](https://github.com/renovate)\[bot] in [#&#8203;8983](https://github.com/open-telemetry/opentelemetry-go/pull/8983)
- sdk/metric: extract generic hotColdMap for delta aggregators by [@&#8203;dashpole](https://github.com/dashpole) in [#&#8203;8756](https://github.com/open-telemetry/opentelemetry-go/pull/8756)
- Update github/codeql-action action to v4.38.1 by [@&#8203;renovate](https://github.com/renovate)\[bot] in [#&#8203;8987](https://github.com/open-telemetry/opentelemetry-go/pull/8987)
- Update googleapis to [`cecb647`](https://github.com/open-telemetry/opentelemetry-go/commit/cecb647) by [@&#8203;renovate](https://github.com/renovate)\[bot] in [#&#8203;8986](https://github.com/open-telemetry/opentelemetry-go/pull/8986)
- \[chore]: remove CLAUDE.md in favor of AGENTS.md by [@&#8203;jarrensj](https://github.com/jarrensj) in [#&#8203;8988](https://github.com/open-telemetry/opentelemetry-go/pull/8988)
- test(internal/tools/verifyreadmes): add unit tests for verifyReadme by [@&#8203;marcschaeferger](https://github.com/marcschaeferger) in [#&#8203;8066](https://github.com/open-telemetry/opentelemetry-go/pull/8066)
- perf(trace): unroll hex decoders to drop bounds checks by [@&#8203;itssaharsh](https://github.com/itssaharsh) in [#&#8203;8740](https://github.com/open-telemetry/opentelemetry-go/pull/8740)
- sdk/metric: add canonical attribute identity by [@&#8203;MrAlias](https://github.com/MrAlias) in [#&#8203;8963](https://github.com/open-telemetry/opentelemetry-go/pull/8963)
- attribute: add Set.String by [@&#8203;iyiola-dev](https://github.com/iyiola-dev) in [#&#8203;8347](https://github.com/open-telemetry/opentelemetry-go/pull/8347)
- attribute: add String method for KeyValue by [@&#8203;pellared](https://github.com/pellared) in [#&#8203;8205](https://github.com/open-telemetry/opentelemetry-go/pull/8205)
- Update open-telemetry/shared-workflows action to v0.15.0 by [@&#8203;renovate](https://github.com/renovate)\[bot] in [#&#8203;8992](https://github.com/open-telemetry/opentelemetry-go/pull/8992)
- Update googleapis to [`b142276`](https://github.com/open-telemetry/opentelemetry-go/commit/b142276) by [@&#8203;renovate](https://github.com/renovate)\[bot] in [#&#8203;8991](https://github.com/open-telemetry/opentelemetry-go/pull/8991)
- Update golang.org/x/telemetry digest to [`bdcd072`](https://github.com/open-telemetry/opentelemetry-go/commit/bdcd072) by [@&#8203;renovate](https://github.com/renovate)\[bot] in [#&#8203;8990](https://github.com/open-telemetry/opentelemetry-go/pull/8990)
- Update module google.golang.org/grpc to v1.84.0 by [@&#8203;renovate](https://github.com/renovate)\[bot] in [#&#8203;8982](https://github.com/open-telemetry/opentelemetry-go/pull/8982)
- Update open-telemetry/shared-workflows action to v0.16.0 by [@&#8203;renovate](https://github.com/renovate)\[bot] in [#&#8203;8995](https://github.com/open-telemetry/opentelemetry-go/pull/8995)
- Update github.com/opentracing-contrib/go-grpc/test digest to [`c4b3818`](https://github.com/open-telemetry/opentelemetry-go/commit/c4b3818) by [@&#8203;renovate](https://github.com/renovate)\[bot] in [#&#8203;8994](https://github.com/open-telemetry/opentelemetry-go/pull/8994)
- Revert "Update module google.golang.org/grpc to v1.84.0 ([#&#8203;8982](https://github.com/open-telemetry/opentelemetry-go/issues/8982))" by [@&#8203;dashpole](https://github.com/dashpole) in [#&#8203;8997](https://github.com/open-telemetry/opentelemetry-go/pull/8997)
- metricdatatest: use attribute collection formatting by [@&#8203;pellared](https://github.com/pellared) in [#&#8203;8993](https://github.com/open-telemetry/opentelemetry-go/pull/8993)
- Update github.com/charmbracelet/ultraviolet digest to [`4e49372`](https://github.com/open-telemetry/opentelemetry-go/commit/4e49372) by [@&#8203;renovate](https://github.com/renovate)\[bot] in [#&#8203;8998](https://github.com/open-telemetry/opentelemetry-go/pull/8998)
- fix(sdk/metric): preserve int64 precision in sum and histogram aggregations by [@&#8203;dashpole](https://github.com/dashpole) in [#&#8203;8981](https://github.com/open-telemetry/opentelemetry-go/pull/8981)
- Update golang.org/x/telemetry digest to [`c6096d2`](https://github.com/open-telemetry/opentelemetry-go/commit/c6096d2) by [@&#8203;renovate](https://github.com/renovate)\[bot] in [#&#8203;9000](https://github.com/open-telemetry/opentelemetry-go/pull/9000)
- Fix OSS-Fuzz issues in FuzzHashKVs, FuzzGetBin, and FuzzSpanIDFromHex by [@&#8203;dashpole](https://github.com/dashpole) in [#&#8203;8996](https://github.com/open-telemetry/opentelemetry-go/pull/8996)
- Update github.com/timakin/bodyclose digest to [`98ded9a`](https://github.com/open-telemetry/opentelemetry-go/commit/98ded9a) by [@&#8203;renovate](https://github.com/renovate)\[bot] in [#&#8203;9003](https://github.com/open-telemetry/opentelemetry-go/pull/9003)
- Update module github.com/ProtonMail/go-crypto to v1.5.0 by [@&#8203;renovate](https://github.com/renovate)\[bot] in [#&#8203;9004](https://github.com/open-telemetry/opentelemetry-go/pull/9004)
- Update module github.com/ProtonMail/go-crypto to v1.5.1 by [@&#8203;renovate](https://github.com/renovate)\[bot] in [#&#8203;9021](https://github.com/open-telemetry/opentelemetry-go/pull/9021)
- Update golang.org/x/telemetry digest to [`9662f6f`](https://github.com/open-telemetry/opentelemetry-go/commit/9662f6f) by [@&#8203;renovate](https://github.com/renovate)\[bot] in [#&#8203;9020](https://github.com/open-telemetry/opentelemetry-go/pull/9020)
- Update module github.com/mgechev/revive to v1.17.0 by [@&#8203;renovate](https://github.com/renovate)\[bot] in [#&#8203;9022](https://github.com/open-telemetry/opentelemetry-go/pull/9022)
- Add ps-mir as an approver by [@&#8203;MrAlias](https://github.com/MrAlias) in [#&#8203;9018](https://github.com/open-telemetry/opentelemetry-go/pull/9018)
- Add Marc Schäfer as a project Triager by [@&#8203;MrAlias](https://github.com/MrAlias) in [#&#8203;9019](https://github.com/open-telemetry/opentelemetry-go/pull/9019)
- Update golang.org/x/telemetry digest to [`ed294f9`](https://github.com/open-telemetry/opentelemetry-go/commit/ed294f9) by [@&#8203;renovate](https://github.com/renovate)\[bot] in [#&#8203;9027](https://github.com/open-telemetry/opentelemetry-go/pull/9027)
- Update github/codeql-action action to v4.38.2 by [@&#8203;renovate](https://github.com/renovate)\[bot] in [#&#8203;9025](https://github.com/open-telemetry/opentelemetry-go/pull/9025)
- Update module github.com/golangci/golangci-lint/v2 to v2.14.0 by [@&#8203;renovate](https://github.com/renovate)\[bot] in [#&#8203;9026](https://github.com/open-telemetry/opentelemetry-go/pull/9026)
- Update github.com/charmbracelet/ultraviolet digest to [`b78653f`](https://github.com/open-telemetry/opentelemetry-go/commit/b78653f) by [@&#8203;renovate](https://github.com/renovate)\[bot] in [#&#8203;9036](https://github.com/open-telemetry/opentelemetry-go/pull/9036)
- Update github.com/charmbracelet/ultraviolet digest to [`bbf040a`](https://github.com/open-telemetry/opentelemetry-go/commit/bbf040a) by [@&#8203;renovate](https://github.com/renovate)\[bot] in [#&#8203;9037](https://github.com/open-telemetry/opentelemetry-go/pull/9037)
- Update module github.com/golangci/golines to v0.16.0 by [@&#8203;renovate](https://github.com/renovate)\[bot] in [#&#8203;9043](https://github.com/open-telemetry/opentelemetry-go/pull/9043)
- Update module github.com/prometheus/common to v0.72.0 by [@&#8203;renovate](https://github.com/renovate)\[bot] in [#&#8203;9044](https://github.com/open-telemetry/opentelemetry-go/pull/9044)
- Update module go.opentelemetry.io/collector/pdata to v1.68.0 by [@&#8203;renovate](https://github.com/renovate)\[bot] in [#&#8203;9046](https://github.com/open-telemetry/opentelemetry-go/pull/9046)
- Update module github.com/uudashr/iface to v1.5.2 by [@&#8203;renovate](https://github.com/renovate)\[bot] in [#&#8203;9042](https://github.com/open-telemetry/opentelemetry-go/pull/9042)
- Update module github.com/ProtonMail/go-crypto to v1.5.2 by [@&#8203;renovate](https://github.com/renovate)\[bot] in [#&#8203;9040](https://github.com/open-telemetry/opentelemetry-go/pull/9040)
- Update open-telemetry/shared-workflows action to v0.17.0 by [@&#8203;renovate](https://github.com/renovate)\[bot] in [#&#8203;9047](https://github.com/open-telemetry/opentelemetry-go/pull/9047)
- Update github.com/golangci/canonicalheader digest to [`e18e5c5`](https://github.com/open-telemetry/opentelemetry-go/commit/e18e5c5) by [@&#8203;renovate](https://github.com/renovate)\[bot] in [#&#8203;9039](https://github.com/open-telemetry/opentelemetry-go/pull/9039)
- Update googleapis to [`8a89bd6`](https://github.com/open-telemetry/opentelemetry-go/commit/8a89bd6) by [@&#8203;renovate](https://github.com/renovate)\[bot] in [#&#8203;9041](https://github.com/open-telemetry/opentelemetry-go/pull/9041)
- Update github.com/charmbracelet/ultraviolet digest to [`666ce5e`](https://github.com/open-telemetry/opentelemetry-go/commit/666ce5e) by [@&#8203;renovate](https://github.com/renovate)\[bot] in [#&#8203;9051](https://github.com/open-telemetry/opentelemetry-go/pull/9051)
- Remove retired Go Report Card badge from README by [@&#8203;kijimad](https://github.com/kijimad) in [#&#8203;9050](https://github.com/open-telemetry/opentelemetry-go/pull/9050)
- fix(sdk/metric): Fix data race with initialization of NewPeriodicReader by [@&#8203;CTrando](https://github.com/CTrando) in [#&#8203;9028](https://github.com/open-telemetry/opentelemetry-go/pull/9028)
- Update module github.com/dlclark/regexp2/v2 to v2.8.1 by [@&#8203;renovate](https://github.com/renovate)\[bot] in [#&#8203;9061](https://github.com/open-telemetry/opentelemetry-go/pull/9061)
- Update github.com/charmbracelet/ultraviolet digest to [`270558f`](https://github.com/open-telemetry/opentelemetry-go/commit/270558f) by [@&#8203;renovate](https://github.com/renovate)\[bot] in [#&#8203;9060](https://github.com/open-telemetry/opentelemetry-go/pull/9060)
- chore: Replace deprecated gofumpt extra-rules with extra options by [@&#8203;kijimad](https://github.com/kijimad) in [#&#8203;9062](https://github.com/open-telemetry/opentelemetry-go/pull/9062)
- Update module github.com/pjbgf/sha1cd to v0.7.0 by [@&#8203;renovate](https://github.com/renovate)\[bot] in [#&#8203;9064](https://github.com/open-telemetry/opentelemetry-go/pull/9064)
- Update github.com/golangci/rowserrcheck digest to [`f772fe6`](https://github.com/open-telemetry/opentelemetry-go/commit/f772fe6) by [@&#8203;renovate](https://github.com/renovate)\[bot] in [#&#8203;9066](https://github.com/open-telemetry/opentelemetry-go/pull/9066)
- Update github.com/charmbracelet/ultraviolet digest to [`8786532`](https://github.com/open-telemetry/opentelemetry-go/commit/8786532) by [@&#8203;renovate](https://github.com/renovate)\[bot] in [#&#8203;9065](https://github.com/open-telemetry/opentelemetry-go/pull/9065)
- Fix zero attribute value length for malformed UTF-8 by [@&#8203;pellared](https://github.com/pellared) in [#&#8203;9054](https://github.com/open-telemetry/opentelemetry-go/pull/9054)
- Update module github.com/uudashr/iface to v1.5.3 by [@&#8203;renovate](https://github.com/renovate)\[bot] in [#&#8203;9067](https://github.com/open-telemetry/opentelemetry-go/pull/9067)
- Update ubuntu:26.04 Docker digest to [`88a381d`](https://github.com/open-telemetry/opentelemetry-go/commit/88a381d) by [@&#8203;renovate](https://github.com/renovate)\[bot] in [#&#8203;9069](https://github.com/open-telemetry/opentelemetry-go/pull/9069)
- Release 1.47.0/0.69.0/0.23.0/0.1.0 by [@&#8203;pellared](https://github.com/pellared) in [#&#8203;9024](https://github.com/open-telemetry/opentelemetry-go/pull/9024)

##### New Contributors

- [@&#8203;chiliec](https://github.com/chiliec) made their first contribution in [#&#8203;8832](https://github.com/open-telemetry/opentelemetry-go/pull/8832)
- [@&#8203;maithilijoshi20](https://github.com/maithilijoshi20) made their first contribution in [#&#8203;8800](https://github.com/open-telemetry/opentelemetry-go/pull/8800)
- [@&#8203;ps-mir](https://github.com/ps-mir) made their first contribution in [#&#8203;8805](https://github.com/open-telemetry/opentelemetry-go/pull/8805)
- [@&#8203;ShreeyashPatil2708](https://github.com/ShreeyashPatil2708) made their first contribution in [#&#8203;8844](https://github.com/open-telemetry/opentelemetry-go/pull/8844)
- [@&#8203;alex131125](https://github.com/alex131125) made their first contribution in [#&#8203;8969](https://github.com/open-telemetry/opentelemetry-go/pull/8969)
- [@&#8203;simpleqt](https://github.com/simpleqt) made their first contribution in [#&#8203;8972](https://github.com/open-telemetry/opentelemetry-go/pull/8972)
- [@&#8203;jarrensj](https://github.com/jarrensj) made their first contribution in [#&#8203;8988](https://github.com/open-telemetry/opentelemetry-go/pull/8988)
- [@&#8203;iyiola-dev](https://github.com/iyiola-dev) made their first contribution in [#&#8203;8347](https://github.com/open-telemetry/opentelemetry-go/pull/8347)
- [@&#8203;kijimad](https://github.com/kijimad) made their first contribution in [#&#8203;9050](https://github.com/open-telemetry/opentelemetry-go/pull/9050)
- [@&#8203;CTrando](https://github.com/CTrando) made their first contribution in [#&#8203;9028](https://github.com/open-telemetry/opentelemetry-go/pull/9028)

**Full Changelog**: <https://github.com/open-telemetry/opentelemetry-go/compare/v1.47.0-rc.1...v1.47.0>

</details>

<details>
<summary>open-telemetry/opentelemetry-proto-go (go.opentelemetry.io/proto/otlp)</summary>

### [`v1.11.1`](https://github.com/open-telemetry/opentelemetry-proto-go/releases/tag/v1.11.1): /v0.4.1

[Compare Source](https://github.com/open-telemetry/opentelemetry-proto-go/compare/v1.11.0...v1.11.1)

Generated Go code for the [`v1.11.1`][otlp] version of the OTLP.

[otlp]: https://github.com/open-telemetry/opentelemetry-proto/releases/tag/v1.11.1

#### What's Changed

- chore(deps): update module github.com/go-git/go-billy/v5 to v5.9.1 by [@&#8203;renovate](https://github.com/renovate)\[bot] in [#&#8203;664](https://github.com/open-telemetry/opentelemetry-proto-go/pull/664)
- chore(deps): update ossf/scorecard-action action to v2.4.4 by [@&#8203;renovate](https://github.com/renovate)\[bot] in [#&#8203;663](https://github.com/open-telemetry/opentelemetry-proto-go/pull/663)
- chore(deps): update googleapis to [`3fe39f3`](https://github.com/open-telemetry/opentelemetry-proto-go/commit/3fe39f3) by [@&#8203;renovate](https://github.com/renovate)\[bot] in [#&#8203;662](https://github.com/open-telemetry/opentelemetry-proto-go/pull/662)
- chore(deps): update googleapis to [`b2f2020`](https://github.com/open-telemetry/opentelemetry-proto-go/commit/b2f2020) by [@&#8203;renovate](https://github.com/renovate)\[bot] in [#&#8203;665](https://github.com/open-telemetry/opentelemetry-proto-go/pull/665)
- chore(deps): update module go.yaml.in/yaml/v3 to v3.0.5 by [@&#8203;renovate](https://github.com/renovate)\[bot] in [#&#8203;666](https://github.com/open-telemetry/opentelemetry-proto-go/pull/666)
- chore(deps): update googleapis to [`6c54ddd`](https://github.com/open-telemetry/opentelemetry-proto-go/commit/6c54ddd) by [@&#8203;renovate](https://github.com/renovate)\[bot] in [#&#8203;667](https://github.com/open-telemetry/opentelemetry-proto-go/pull/667)
- chore(deps): update google.golang.org/genproto/googleapis/api digest to [`8efbd57`](https://github.com/open-telemetry/opentelemetry-proto-go/commit/8efbd57) by [@&#8203;renovate](https://github.com/renovate)\[bot] in [#&#8203;670](https://github.com/open-telemetry/opentelemetry-proto-go/pull/670)
- chore(deps): update module github.com/go-git/go-git/v5 to v5.19.2 by [@&#8203;renovate](https://github.com/renovate)\[bot] in [#&#8203;669](https://github.com/open-telemetry/opentelemetry-proto-go/pull/669)
- chore(deps): update google.golang.org/genproto/googleapis/rpc digest to [`8efbd57`](https://github.com/open-telemetry/opentelemetry-proto-go/commit/8efbd57) by [@&#8203;renovate](https://github.com/renovate)\[bot] in [#&#8203;671](https://github.com/open-telemetry/opentelemetry-proto-go/pull/671)
- chore(deps): update github/codeql-action action to v4.37.4 by [@&#8203;renovate](https://github.com/renovate)\[bot] in [#&#8203;673](https://github.com/open-telemetry/opentelemetry-proto-go/pull/673)
- fix(deps): update module google.golang.org/grpc to v1.83.0 by [@&#8203;renovate](https://github.com/renovate)\[bot] in [#&#8203;672](https://github.com/open-telemetry/opentelemetry-proto-go/pull/672)
- chore: Change Renovate configuration to best practices by [@&#8203;thompson-tomo](https://github.com/thompson-tomo) in [#&#8203;674](https://github.com/open-telemetry/opentelemetry-proto-go/pull/674)
- chore(deps): pin dependencies by [@&#8203;renovate](https://github.com/renovate)\[bot] in [#&#8203;677](https://github.com/open-telemetry/opentelemetry-proto-go/pull/677)
- chore(deps): update github/codeql-action action to v4.37.5 by [@&#8203;renovate](https://github.com/renovate)\[bot] in [#&#8203;676](https://github.com/open-telemetry/opentelemetry-proto-go/pull/676)
- chore(deps): update googleapis to [`6ac0973`](https://github.com/open-telemetry/opentelemetry-proto-go/commit/6ac0973) by [@&#8203;renovate](https://github.com/renovate)\[bot] in [#&#8203;678](https://github.com/open-telemetry/opentelemetry-proto-go/pull/678)
- chore(deps): update github/codeql-action action to v4.37.6 by [@&#8203;renovate](https://github.com/renovate)\[bot] in [#&#8203;679](https://github.com/open-telemetry/opentelemetry-proto-go/pull/679)
- ci: add shared security scanning workflow by [@&#8203;opentelemetrybot](https://github.com/opentelemetrybot) in [#&#8203;675](https://github.com/open-telemetry/opentelemetry-proto-go/pull/675)
- chore(deps): update open-telemetry/shared-workflows action to v0.5.0 by [@&#8203;renovate](https://github.com/renovate)\[bot] in [#&#8203;682](https://github.com/open-telemetry/opentelemetry-proto-go/pull/682)
- chore(deps): update module github.com/cloudflare/circl to v1.6.5 by [@&#8203;renovate](https://github.com/renovate)\[bot] in [#&#8203;681](https://github.com/open-telemetry/opentelemetry-proto-go/pull/681)
- fix(deps): update module github.com/grpc-ecosystem/grpc-gateway/v2 to v2.30.0 by [@&#8203;renovate](https://github.com/renovate)\[bot] in [#&#8203;680](https://github.com/open-telemetry/opentelemetry-proto-go/pull/680)
- chore(deps): update googleapis to [`c8921c7`](https://github.com/open-telemetry/opentelemetry-proto-go/commit/c8921c7) by [@&#8203;renovate](https://github.com/renovate)\[bot] in [#&#8203;684](https://github.com/open-telemetry/opentelemetry-proto-go/pull/684)
- chore(deps): update open-telemetry/shared-workflows action to v0.6.0 by [@&#8203;renovate](https://github.com/renovate)\[bot] in [#&#8203;683](https://github.com/open-telemetry/opentelemetry-proto-go/pull/683)
- chore(deps): update googleapis to [`ec0a776`](https://github.com/open-telemetry/opentelemetry-proto-go/commit/ec0a776) by [@&#8203;renovate](https://github.com/renovate)\[bot] in [#&#8203;685](https://github.com/open-telemetry/opentelemetry-proto-go/pull/685)
- fix(deps): update module google.golang.org/protobuf to v1.36.12 by [@&#8203;renovate](https://github.com/renovate)\[bot] in [#&#8203;686](https://github.com/open-telemetry/opentelemetry-proto-go/pull/686)
- chore(deps): update module golang.org/x/mod to v0.39.0 by [@&#8203;renovate](https://github.com/renovate)\[bot] in [#&#8203;687](https://github.com/open-telemetry/opentelemetry-proto-go/pull/687)
- ci: use shared OSSF Scorecard workflow by [@&#8203;opentelemetrybot](https://github.com/opentelemetrybot) in [#&#8203;688](https://github.com/open-telemetry/opentelemetry-proto-go/pull/688)
- chore(deps): update all golang.org/x packages by [@&#8203;renovate](https://github.com/renovate)\[bot] in [#&#8203;689](https://github.com/open-telemetry/opentelemetry-proto-go/pull/689)
- chore(deps): update otel/build-protobuf docker tag to v0.26.0 by [@&#8203;renovate](https://github.com/renovate)\[bot] in [#&#8203;668](https://github.com/open-telemetry/opentelemetry-proto-go/pull/668)
- chore(deps): update module golang.org/x/mod to v0.40.0 by [@&#8203;renovate](https://github.com/renovate)\[bot] in [#&#8203;690](https://github.com/open-telemetry/opentelemetry-proto-go/pull/690)
- chore(deps): update open-telemetry/shared-workflows action to v0.7.0 by [@&#8203;renovate](https://github.com/renovate)\[bot] in [#&#8203;691](https://github.com/open-telemetry/opentelemetry-proto-go/pull/691)
- chore(deps): update googleapis to [`ac3dfec`](https://github.com/open-telemetry/opentelemetry-proto-go/commit/ac3dfec) by [@&#8203;renovate](https://github.com/renovate)\[bot] in [#&#8203;692](https://github.com/open-telemetry/opentelemetry-proto-go/pull/692)
- chore(deps): update googleapis to [`1b09341`](https://github.com/open-telemetry/opentelemetry-proto-go/commit/1b09341) by [@&#8203;renovate](https://github.com/renovate)\[bot] in [#&#8203;693](https://github.com/open-telemetry/opentelemetry-proto-go/pull/693)
- fix(deps): update module google.golang.org/grpc to v1.83.1 by [@&#8203;renovate](https://github.com/renovate)\[bot] in [#&#8203;694](https://github.com/open-telemetry/opentelemetry-proto-go/pull/694)
- chore(deps): update googleapis to [`08b0e42`](https://github.com/open-telemetry/opentelemetry-proto-go/commit/08b0e42) by [@&#8203;renovate](https://github.com/renovate)\[bot] in [#&#8203;695](https://github.com/open-telemetry/opentelemetry-proto-go/pull/695)
- chore(deps): update open-telemetry/shared-workflows action to v0.8.0 by [@&#8203;renovate](https://github.com/renovate)\[bot] in [#&#8203;696](https://github.com/open-telemetry/opentelemetry-proto-go/pull/696)
- chore(deps): update open-telemetry/shared-workflows action to v0.10.0 by [@&#8203;renovate](https://github.com/renovate)\[bot] in [#&#8203;697](https://github.com/open-telemetry/opentelemetry-proto-go/pull/697)
- chore(deps): update open-telemetry/shared-workflows action to v0.11.0 by [@&#8203;renovate](https://github.com/renovate)\[bot] in [#&#8203;698](https://github.com/open-telemetry/opentelemetry-proto-go/pull/698)
- chore(deps): update googleapis to [`da73d73`](https://github.com/open-telemetry/opentelemetry-proto-go/commit/da73d73) by [@&#8203;renovate](https://github.com/renovate)\[bot] in [#&#8203;700](https://github.com/open-telemetry/opentelemetry-proto-go/pull/700)
- fix(deps): update module google.golang.org/grpc to v1.83.2 by [@&#8203;renovate](https://github.com/renovate)\[bot] in [#&#8203;699](https://github.com/open-telemetry/opentelemetry-proto-go/pull/699)
- chore(deps): update module github.com/skeema/knownhosts to v1.3.3 by [@&#8203;renovate](https://github.com/renovate)\[bot] in [#&#8203;702](https://github.com/open-telemetry/opentelemetry-proto-go/pull/702)
- chore(deps): update open-telemetry/shared-workflows action to v0.12.0 by [@&#8203;renovate](https://github.com/renovate)\[bot] in [#&#8203;701](https://github.com/open-telemetry/opentelemetry-proto-go/pull/701)
- chore(deps): update googleapis to [`18b4a75`](https://github.com/open-telemetry/opentelemetry-proto-go/commit/18b4a75) by [@&#8203;renovate](https://github.com/renovate)\[bot] in [#&#8203;703](https://github.com/open-telemetry/opentelemetry-proto-go/pull/703)
- chore(deps): update open-telemetry/shared-workflows action to v0.13.0 by [@&#8203;renovate](https://github.com/renovate)\[bot] in [#&#8203;704](https://github.com/open-telemetry/opentelemetry-proto-go/pull/704)
- chore(deps): update googleapis to [`f8649dd`](https://github.com/open-telemetry/opentelemetry-proto-go/commit/f8649dd) by [@&#8203;renovate](https://github.com/renovate)\[bot] in [#&#8203;706](https://github.com/open-telemetry/opentelemetry-proto-go/pull/706)
- chore(deps): update googleapis to [`f61a6ca`](https://github.com/open-telemetry/opentelemetry-proto-go/commit/f61a6ca) by [@&#8203;renovate](https://github.com/renovate)\[bot] in [#&#8203;708](https://github.com/open-telemetry/opentelemetry-proto-go/pull/708)
- Make slim OTLP descriptors coexist with canonical OTLP by [@&#8203;pellared](https://github.com/pellared) in [#&#8203;707](https://github.com/open-telemetry/opentelemetry-proto-go/pull/707)
- chore(deps): update open-telemetry/shared-workflows action to v0.14.0 by [@&#8203;renovate](https://github.com/renovate)\[bot] in [#&#8203;710](https://github.com/open-telemetry/opentelemetry-proto-go/pull/710)
- Drop support for Go 1.25 by [@&#8203;pellared](https://github.com/pellared) in [#&#8203;711](https://github.com/open-telemetry/opentelemetry-proto-go/pull/711)
- chore(deps): update all golang.org/x packages by [@&#8203;renovate](https://github.com/renovate)\[bot] in [#&#8203;709](https://github.com/open-telemetry/opentelemetry-proto-go/pull/709)
- chore(deps): update googleapis to [`eeb232e`](https://github.com/open-telemetry/opentelemetry-proto-go/commit/eeb232e) by [@&#8203;renovate](https://github.com/renovate)\[bot] in [#&#8203;713](https://github.com/open-telemetry/opentelemetry-proto-go/pull/713)
- fix(deps): update module google.golang.org/grpc to v1.84.0 by [@&#8203;renovate](https://github.com/renovate)\[bot] in [#&#8203;712](https://github.com/open-telemetry/opentelemetry-proto-go/pull/712)
- chore(deps): update googleapis to [`cecb647`](https://github.com/open-telemetry/opentelemetry-proto-go/commit/cecb647) by [@&#8203;renovate](https://github.com/renovate)\[bot] in [#&#8203;714](https://github.com/open-telemetry/opentelemetry-proto-go/pull/714)
- chore(deps): update googleapis to [`b142276`](https://github.com/open-telemetry/opentelemetry-proto-go/commit/b142276) by [@&#8203;renovate](https://github.com/renovate)\[bot] in [#&#8203;715](https://github.com/open-telemetry/opentelemetry-proto-go/pull/715)
- chore(deps): update open-telemetry/shared-workflows action to v0.15.0 by [@&#8203;renovate](https://github.com/renovate)\[bot] in [#&#8203;716](https://github.com/open-telemetry/opentelemetry-proto-go/pull/716)
- chore(deps): update open-telemetry/shared-workflows action to v0.16.0 by [@&#8203;renovate](https://github.com/renovate)\[bot] in [#&#8203;717](https://github.com/open-telemetry/opentelemetry-proto-go/pull/717)
- chore(deps): update module github.com/protonmail/go-crypto to v1.5.0 by [@&#8203;renovate](https://github.com/renovate)\[bot] in [#&#8203;718](https://github.com/open-telemetry/opentelemetry-proto-go/pull/718)
- chore(deps): update module github.com/protonmail/go-crypto to v1.5.1 by [@&#8203;renovate](https://github.com/renovate)\[bot] in [#&#8203;719](https://github.com/open-telemetry/opentelemetry-proto-go/pull/719)
- fix(deps): update module github.com/grpc-ecosystem/grpc-gateway/v2 to v2.31.0 by [@&#8203;renovate](https://github.com/renovate)\[bot] in [#&#8203;720](https://github.com/open-telemetry/opentelemetry-proto-go/pull/720)
- chore(deps): update googleapis to [`8a89bd6`](https://github.com/open-telemetry/opentelemetry-proto-go/commit/8a89bd6) by [@&#8203;renovate](https://github.com/renovate)\[bot] in [#&#8203;723](https://github.com/open-telemetry/opentelemetry-proto-go/pull/723)
- chore(deps): update open-telemetry/shared-workflows action to v0.17.0 by [@&#8203;renovate](https://github.com/renovate)\[bot] in [#&#8203;722](https://github.com/open-telemetry/opentelemetry-proto-go/pull/722)
- chore(deps): update module github.com/protonmail/go-crypto to v1.5.2 by [@&#8203;renovate](https://github.com/renovate)\[bot] in [#&#8203;721](https://github.com/open-telemetry/opentelemetry-proto-go/pull/721)
- Release v1.11.1 by [@&#8203;pellared](https://github.com/pellared) in [#&#8203;724](https://github.com/open-telemetry/opentelemetry-proto-go/pull/724)

#### New Contributors

- [@&#8203;thompson-tomo](https://github.com/thompson-tomo) made their first contribution in [#&#8203;674](https://github.com/open-telemetry/opentelemetry-proto-go/pull/674)

**Full Changelog**: <https://github.com/open-telemetry/opentelemetry-proto-go/compare/v1.11.0...v1.11.1>

</details>

---

### Configuration

📅 **Schedule**: (UTC)

- Branch creation
  - Between 12:00 AM and 03:59 AM, only on Monday (`* 0-3 * * 1`)
- Automerge
  - At any time (no schedule defined)

🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied.

♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.

👻 **Immortal**: This PR will be recreated if closed unmerged. Get [config help](https://github.com/renovatebot/renovate/discussions) if that's undesired.

---

 - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box

---

This PR has been generated by [Mend Renovate](https://github.com/renovatebot/renovate).
<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiI0My4xOTEuMiIsInVwZGF0ZWRJblZlciI6IjQzLjE5MS4yIiwidGFyZ2V0QnJhbmNoIjoibWFpbiIsImxhYmVscyI6W119-->

---------

Co-authored-by: silverwind <2021+silverwind@noreply.gitea.com>
Reviewed-on: https://gitea.com/gitea/runner/pulls/1308
Reviewed-by: silverwind <2021+silverwind@noreply.gitea.com>
Co-authored-by: Renovate Bot <renovate-bot@gitea.com>
2026-10-12 15:36:28 +00:00
Lunny Xiao
49e34813ae docs: describe the logging package (#1305)
Drop the logrus references from the docs and the config example, and add a Logging section to `DEVELOPMENT.md` on how to log and test logging.

Stacked on https://gitea.com/gitea/runner/pulls/1304.

Assisted-by: Codet
Co-authored-by: bircni <bircni@icloud.com>
Reviewed-on: https://gitea.com/gitea/runner/pulls/1305
Reviewed-by: bircni <bircni@icloud.com>
2026-10-12 06:52:14 +00:00
Lunny Xiao
e0ef637805 refactor(logging): replace the logrus backend with log/slog (#1304)
Log through `log/slog`, with a console `slog.Handler` that writes the text the runner has always written (sorted `key=value` fields on a plain stream, a colored prefix on a terminal), so nothing changes for users. I compared its output byte for byte with logrus's `TextFormatter` over plain, colored, timestamped and caller cases.

Why not `slog.TextHandler`: its output differs from what users and their log parsers see today, and `ReplaceAttr` cannot close the gap.
- level is `WARN` where logrus wrote `warning`, and time has milliseconds and no quotes
- fields keep call order instead of being sorted
- the caller is one `source=` key, where the runner writes `func=` and `file=` (reshaped by `callPrettyfier`), and `ReplaceAttr` returns a single attribute
- no colored prefix on a terminal
- no hook to rewrite a whole formatted line, which the process-wide secret masking relies on

Libraries that log through logrus's global logger (the workflow model, the Docker client) are forwarded to the process-wide logger, so they keep following its level, format and secret masking. This is why logrus stays in `go.mod`.

Stacked on https://gitea.com/gitea/runner/pulls/1303.

Assisted-by: Codet
Co-authored-by: bircni <bircni@icloud.com>
Reviewed-on: https://gitea.com/gitea/runner/pulls/1304
Reviewed-by: bircni <bircni@icloud.com>
2026-10-12 06:36:55 +00:00
Lunny Xiao
8502663d0e chore(lint): forbid importing logrus outside the logging package (#1303)
Add a `depguard` rule so everything logs through `act/common/logging`, which keeps the backend swappable.

Stacked on https://gitea.com/gitea/runner/pulls/1302.

Assisted-by: Codet
Co-authored-by: bircni <bircni@icloud.com>
Reviewed-on: https://gitea.com/gitea/runner/pulls/1303
Reviewed-by: Zettat123 <39446+zettat123@noreply.gitea.com>
2026-10-12 03:39:48 +00:00
Lunny Xiao
4e8037a93b refactor(logging): use the logging package in internal (#1302)
Move the log calls in `internal` onto `logging`, and add what the process-wide logger setup and its tests need: `SetTextFormat` with a line filter for secret masking (replacing `report.MaskingFormatter`), `SetReportCaller`, `SetDefault` and `logtest.SetDefault`. After this only the `logging` package imports logrus.

Stacked on https://gitea.com/gitea/runner/pulls/1301.

Assisted-by: Codet
Co-authored-by: bircni <bircni@icloud.com>
Reviewed-on: https://gitea.com/gitea/runner/pulls/1302
Reviewed-by: bircni <bircni@icloud.com>
2026-10-12 00:19:01 +00:00
Lunny Xiao
76110557ab refactor(logging): use the logging package for the remaining act log calls (#1301)
Move the global logrus calls left in `act` onto `logging`, and add `SetOutput` and `Output` for tests that capture the process logger. Behavior is unchanged.

Stacked on https://gitea.com/gitea/runner/pulls/1300.

Assisted-by: Codet
Co-authored-by: bircni <bircni@icloud.com>
Reviewed-on: https://gitea.com/gitea/runner/pulls/1301
Reviewed-by: bircni <bircni@icloud.com>
2026-10-11 23:55:02 +00:00
Lunny Xiao
3bc5cc03f8 test(logging): add logtest.NewLogger (#1300)
Return a `Logger` and its recording `Handler` together, so tests no longer need the two-line setup.

Stacked on https://gitea.com/gitea/runner/pulls/1295.

Assisted-by: Codet
Co-authored-by: bircni <bircni@icloud.com>
Reviewed-on: https://gitea.com/gitea/runner/pulls/1300
Reviewed-by: bircni <bircni@icloud.com>
2026-10-11 23:47:50 +00:00
Lunny Xiao
de961b91eb refactor(logging): add a logging abstraction and move the job logger onto it (#1295)
Add `act/common/logging`: a printf-style `Logger` on a slog-shaped `Handler`, with a logrus-backed handler as the only backend, plus a `logtest` handler for tests. Move the job logger onto it: `common.Logger` returns `*logging.Logger`, the job log formatters and the `Reporter` hook become handlers, and composite step IDs travel in the context.

Logging is tightly coupled to logrus types (hooks, formatters, `Entry.Data`). Going through our own interface first lets the remaining call sites move over with unchanged behavior, and lets the backend be swapped for `log/slog` in one place afterwards. The global process logger and its masking formatter stay on logrus for now.

Assisted-by: Codet
Co-authored-by: bircni <bircni@icloud.com>
Reviewed-on: https://gitea.com/gitea/runner/pulls/1295
Reviewed-by: bircni <bircni@icloud.com>
2026-10-11 23:26:00 +00:00
silverwind
ad9eeb668a fix: job container, daemon startup, action cache and exec fixes (#1294)
- Keep `GITHUB_ENV`, `GITHUB_OUTPUT`, `GITHUB_STATE` and `GITHUB_PATH` when a job times out or is cancelled, so post steps still get their state
- Retry the startup Declare while Gitea is unreachable instead of exiting, still failing fast on rejected registrations, auth and misconfiguration
- Reuse a local image when a platform variant such as `linux/arm/v7` is requested instead of pulling again
- Read `PATH` from the created job container so cross-architecture jobs get tool paths for the right architecture
- Extract action copies relative to the act volume so they never cross an image's `/var/run` symlink, which rootless Podman rejects
- Keep the job container's entrypoint when container options set `--entrypoint`, warning instead of appending it to the keep-alive command
- Lock action cache entries across runner processes that share a cache directory
- `exec`: omit `GITHUB_SERVER_URL` and `GITHUB_API_URL` without `--gitea-instance` instead of exporting a bare `https://`, and read owner and repository from any origin remote

Reviewed-on: https://gitea.com/gitea/runner/pulls/1294
Reviewed-by: bircni <bircni@icloud.com>
Co-authored-by: silverwind <me@silverwind.io>
2026-10-10 18:11:03 +00:00
Renovate Bot
b741335222 fix(deps): update module golang.org/x/net to v0.60.0 [security] (#1292)
This PR contains the following updates:

| Package | Change | [Age](https://docs.renovatebot.com/merge-confidence/) | [Confidence](https://docs.renovatebot.com/merge-confidence/) |
|---|---|---|---|
| [golang.org/x/net](https://pkg.go.dev/golang.org/x/net) | [`v0.59.0` → `v0.60.0`](https://cs.opensource.google/go/x/net/+/refs/tags/v0.59.0...refs/tags/v0.60.0) | ![age](https://developer.mend.io/api/mc/badges/age/go/golang.org%2fx%2fnet/v0.60.0?slim=true) | ![confidence](https://developer.mend.io/api/mc/badges/confidence/go/golang.org%2fx%2fnet/v0.59.0/v0.60.0?slim=true) |

---

### HTTP/2 server memory exhaustion due to Trailer headers in net/http
[CVE-2026-78659](https://nvd.nist.gov/vuln/detail/CVE-2026-78659) / [GO-2026-6603](https://pkg.go.dev/vuln/GO-2026-6603)

<details>
<summary>More information</summary>

#### Details
When "Trailer" headers are sent by a client, the HTTP server internally uses the header values to populate the Request.Trailer map passed to the server handler. Because Request.Trailer is a map, each entry incurs memory overhead. For HTTP/2 servers, a malicious client can exploit this by sending a "Trailer" header that declares a large number of fields, causing the server to allocate a disproportionate amount of memory while bypassing Server.MaxHeaderValueCount and Server.MaxHeaderBytes limits. This exploit is not applicable for HTTP/1 servers, which do not support multiplexing a large number of requests over one TCP connection, and whose Server.MaxHeaderBytes are calculated differently.

#### Severity
Unknown

#### References
- [https://go.dev/cl/847185](https://go.dev/cl/847185)
- [https://go.dev/cl/847314](https://go.dev/cl/847314)
- [https://go.dev/issue/81857](https://go.dev/issue/81857)
- [https://groups.google.com/g/golang-announce/c/U2fTuyDJznI](https://groups.google.com/g/golang-announce/c/U2fTuyDJznI)
- [https://groups.google.com/g/golang-announce/c/ZPwCyRUuGBs](https://groups.google.com/g/golang-announce/c/ZPwCyRUuGBs)

This data is provided by [OSV](https://osv.dev/vulnerability/GO-2026-6603) and the [Go Vulnerability Database](https://github.com/golang/vulndb) ([CC-BY 4.0](https://github.com/golang/vulndb#license)).
</details>

---

### HTTP/2 transport accepts malformed framing-related headers in net/http
[CVE-2026-78660](https://nvd.nist.gov/vuln/detail/CVE-2026-78660) / [GO-2026-6610](https://pkg.go.dev/vuln/GO-2026-6610)

<details>
<summary>More information</summary>

#### Details
Historically, we have been rather lax about malformed framing-related headers in our HTTP/2 implementation, as they cannot interfere with HTTP/2 framing. However, this makes it possible for our HTTP/2 implementation to forward responses containing such headers to an HTTP/1 client when acting as a reverse proxy. If the HTTP/1 client also does not behave strictly enough, this can result in response smuggling.

#### Severity
Unknown

#### References
- [https://go.dev/cl/835145](https://go.dev/cl/835145)
- [https://go.dev/cl/836385](https://go.dev/cl/836385)
- [https://go.dev/issue/81115](https://go.dev/issue/81115)
- [https://groups.google.com/g/golang-announce/c/U2fTuyDJznI](https://groups.google.com/g/golang-announce/c/U2fTuyDJznI)

This data is provided by [OSV](https://osv.dev/vulnerability/GO-2026-6610) and the [Go Vulnerability Database](https://github.com/golang/vulndb) ([CC-BY 4.0](https://github.com/golang/vulndb#license)).
</details>

---

### Excessive CPU consumption from repeated initial window changes in net/http
[CVE-2026-78669](https://nvd.nist.gov/vuln/detail/CVE-2026-78669) / [GO-2026-6611](https://pkg.go.dev/vuln/GO-2026-6611)

<details>
<summary>More information</summary>

#### Details
A malicious HTTP/2 peer can cause excessive CPU consumption in the client or server by opening a large number of streams and then sending many small SETTINGS frames containing SETTINGS_INITIAL_WINDOW_SIZE values.

#### Severity
Unknown

#### References
- [https://go.dev/cl/847186](https://go.dev/cl/847186)
- [https://go.dev/cl/847308](https://go.dev/cl/847308)
- [https://go.dev/issue/81742](https://go.dev/issue/81742)
- [https://groups.google.com/g/golang-announce/c/U2fTuyDJznI](https://groups.google.com/g/golang-announce/c/U2fTuyDJznI)
- [https://groups.google.com/g/golang-announce/c/ZPwCyRUuGBs](https://groups.google.com/g/golang-announce/c/ZPwCyRUuGBs)

This data is provided by [OSV](https://osv.dev/vulnerability/GO-2026-6611) and the [Go Vulnerability Database](https://github.com/golang/vulndb) ([CC-BY 4.0](https://github.com/golang/vulndb#license)).
</details>

---

### Double flow control refund on HTTP/2 server streams in net/http
[CVE-2026-78663](https://nvd.nist.gov/vuln/detail/CVE-2026-78663) / [GO-2026-6612](https://pkg.go.dev/vuln/GO-2026-6612)

<details>
<summary>More information</summary>

#### Details
The HTTP/2 server can refund connection-level flow control twice for the same data: Once when a client resets a stream (refunding data for any sent-but-unread portion of the stream), and again when a request handler reads the buffered data. A malicious client can exploit this to bypass the configured connection-level flow control limit (MaxReceiveBufferPerConnection). Total buffered data is still limited by the concurrent stream limit and stream-level flow control.

#### Severity
Unknown

#### References
- [https://go.dev/cl/847187](https://go.dev/cl/847187)
- [https://go.dev/cl/847310](https://go.dev/cl/847310)
- [https://go.dev/issue/81743](https://go.dev/issue/81743)
- [https://groups.google.com/g/golang-announce/c/U2fTuyDJznI](https://groups.google.com/g/golang-announce/c/U2fTuyDJznI)
- [https://groups.google.com/g/golang-announce/c/ZPwCyRUuGBs](https://groups.google.com/g/golang-announce/c/ZPwCyRUuGBs)

This data is provided by [OSV](https://osv.dev/vulnerability/GO-2026-6612) and the [Go Vulnerability Database](https://github.com/golang/vulndb) ([CC-BY 4.0](https://github.com/golang/vulndb#license)).
</details>

---

### HTTP/2 server crash due to HPACK encoder race in net/http
[CVE-2026-97032](https://nvd.nist.gov/vuln/detail/CVE-2026-97032) / [GO-2026-6617](https://pkg.go.dev/vuln/GO-2026-6617)

<details>
<summary>More information</summary>

#### Details
HTTP/2 servers could end up crashing due to inadvertently modifying its HPACK encoder concurrently. This happens because the server modifies the HPACK encoder from two goroutines without synchronization: one uses the encoder to encode a HEADERS frame as part of a response sent to a client and the other modifies the encoder's table size when handling a SETTINGS frame containing SETTINGS_HEADER_TABLE_SIZE that a client sends. A malicious client can repeatedly send a request while changing the header table size to crash the server.

#### Severity
Unknown

#### References
- [https://go.dev/cl/847188](https://go.dev/cl/847188)
- [https://go.dev/cl/847313](https://go.dev/cl/847313)
- [https://go.dev/issue/81867](https://go.dev/issue/81867)
- [https://groups.google.com/g/golang-announce/c/U2fTuyDJznI](https://groups.google.com/g/golang-announce/c/U2fTuyDJznI)
- [https://groups.google.com/g/golang-announce/c/ZPwCyRUuGBs](https://groups.google.com/g/golang-announce/c/ZPwCyRUuGBs)

This data is provided by [OSV](https://osv.dev/vulnerability/GO-2026-6617) and the [Go Vulnerability Database](https://github.com/golang/vulndb) ([CC-BY 4.0](https://github.com/golang/vulndb#license)).
</details>

---

### Configuration

📅 **Schedule**: (UTC)

- Branch creation
  - ""
- Automerge
  - At any time (no schedule defined)

🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied.

♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.

🔕 **Ignore**: Close this PR and you won't be reminded about this update again.

---

 - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box

---

This PR has been generated by [Mend Renovate](https://github.com/renovatebot/renovate).
<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiI0My4xOTEuMiIsInVwZGF0ZWRJblZlciI6IjQzLjE5MS4yIiwidGFyZ2V0QnJhbmNoIjoibWFpbiIsImxhYmVscyI6W119-->

---------

Co-authored-by: silverwind <2021+silverwind@noreply.gitea.com>
Reviewed-on: https://gitea.com/gitea/runner/pulls/1292
Reviewed-by: silverwind <2021+silverwind@noreply.gitea.com>
Co-authored-by: Renovate Bot <renovate-bot@gitea.com>
2026-10-10 03:45:46 +00:00
Renovate Bot
20143c9558 fix(deps): update go toolchain directive to v1.27.2 [security] (#1291)
This PR contains the following updates:

| Package | Type | Update | Change |
|---|---|---|---|
| [go](https://go.dev/) ([source](https://github.com/golang/go)) | toolchain | patch | `1.27.1` → `1.27.2` |

---

### Reset context tracking on consecutive template expressions in html/template
[CVE-2026-94448](https://nvd.nist.gov/vuln/detail/CVE-2026-94448) / [GO-2026-6599](https://pkg.go.dev/vuln/GO-2026-6599)

<details>
<summary>More information</summary>

#### Details
When a JavaScript template literal contains consecutive expressions, the context tracking state was not properly reset upon entering a new expression.

We now ensure that template-literal expression entries correctly reset context variables so all subsequent regular expression literals are accurately recognized and escaped.

#### Severity
Unknown

#### References
- [https://go.dev/cl/839866](https://go.dev/cl/839866)
- [https://go.dev/issue/81821](https://go.dev/issue/81821)
- [https://groups.google.com/g/golang-announce/c/U2fTuyDJznI](https://groups.google.com/g/golang-announce/c/U2fTuyDJznI)

This data is provided by [OSV](https://osv.dev/vulnerability/GO-2026-6599) and the [Go Vulnerability Database](https://github.com/golang/vulndb) ([CC-BY 4.0](https://github.com/golang/vulndb#license)).
</details>

---

### Recognize yield as regexp preceder keyword in html/template
[CVE-2026-97030](https://nvd.nist.gov/vuln/detail/CVE-2026-97030) / [GO-2026-6600](https://pkg.go.dev/vuln/GO-2026-6600)

<details>
<summary>More information</summary>

#### Details
A trusted template author may have previously written a valid template wherein the use of the 'yield' keyword would not be correctly escaped.

We now ensure that valid keyword uses are escaped and non-keyword uses are not escaped.

#### Severity
Unknown

#### References
- [https://go.dev/cl/840925](https://go.dev/cl/840925)
- [https://go.dev/issue/81823](https://go.dev/issue/81823)
- [https://groups.google.com/g/golang-announce/c/U2fTuyDJznI](https://groups.google.com/g/golang-announce/c/U2fTuyDJznI)

This data is provided by [OSV](https://osv.dev/vulnerability/GO-2026-6600) and the [Go Vulnerability Database](https://github.com/golang/vulndb) ([CC-BY 4.0](https://github.com/golang/vulndb#license)).
</details>

---

### HTTP/2 server memory exhaustion due to Trailer headers in net/http
[CVE-2026-78659](https://nvd.nist.gov/vuln/detail/CVE-2026-78659) / [GO-2026-6603](https://pkg.go.dev/vuln/GO-2026-6603)

<details>
<summary>More information</summary>

#### Details
When "Trailer" headers are sent by a client, the HTTP server internally uses the header values to populate the Request.Trailer map passed to the server handler. Because Request.Trailer is a map, each entry incurs memory overhead. For HTTP/2 servers, a malicious client can exploit this by sending a "Trailer" header that declares a large number of fields, causing the server to allocate a disproportionate amount of memory while bypassing Server.MaxHeaderValueCount and Server.MaxHeaderBytes limits. This exploit is not applicable for HTTP/1 servers, which do not support multiplexing a large number of requests over one TCP connection, and whose Server.MaxHeaderBytes are calculated differently.

#### Severity
Unknown

#### References
- [https://go.dev/cl/847185](https://go.dev/cl/847185)
- [https://go.dev/cl/847314](https://go.dev/cl/847314)
- [https://go.dev/issue/81857](https://go.dev/issue/81857)
- [https://groups.google.com/g/golang-announce/c/U2fTuyDJznI](https://groups.google.com/g/golang-announce/c/U2fTuyDJznI)
- [https://groups.google.com/g/golang-announce/c/ZPwCyRUuGBs](https://groups.google.com/g/golang-announce/c/ZPwCyRUuGBs)

This data is provided by [OSV](https://osv.dev/vulnerability/GO-2026-6603) and the [Go Vulnerability Database](https://github.com/golang/vulndb) ([CC-BY 4.0](https://github.com/golang/vulndb#license)).
</details>

---

### Root.Mkdir(All) can follow junctions out of the root on Windows in os
[CVE-2026-56857](https://nvd.nist.gov/vuln/detail/CVE-2026-56857) / [GO-2026-6604](https://pkg.go.dev/vuln/GO-2026-6604)

<details>
<summary>More information</summary>

#### Details
On Windows, when the target of Root.Mkdir or Root.MkdirAll is a junction pointing to an empty location, the operation can create a directory at the junction target even when that target is located outside the root. This only applies to operations where the last path component is a junction (path/to/junction, but not path/junction/target).

#### Severity
Unknown

#### References
- [https://go.dev/cl/847305](https://go.dev/cl/847305)
- [https://go.dev/issue/81739](https://go.dev/issue/81739)
- [https://groups.google.com/g/golang-announce/c/U2fTuyDJznI](https://groups.google.com/g/golang-announce/c/U2fTuyDJznI)

This data is provided by [OSV](https://osv.dev/vulnerability/GO-2026-6604) and the [Go Vulnerability Database](https://github.com/golang/vulndb) ([CC-BY 4.0](https://github.com/golang/vulndb#license)).
</details>

---

### HTTP/1 client connection desynchronization after CONNECT rejection in net/http
[CVE-2026-56866](https://nvd.nist.gov/vuln/detail/CVE-2026-56866) / [GO-2026-6605](https://pkg.go.dev/vuln/GO-2026-6605)

<details>
<summary>More information</summary>

#### Details
When http.Transport sends an HTTP/1 CONNECT request with a non-empty Request.Body, it writes the body directly to the connection without framing after the request headers. If the server rejects the CONNECT request with a non-2xx keep-alive response, Transport returns the connection to the idle pool. Because CONNECT requests do not have a request body, the server may interpret the trailing body bytes as a subsequent pipelined HTTP/1.1 request on the connection, leaving the pooled connection desynchronized and causing the next caller that reuses it to read the response to the injected request. In reverse proxies (including httputil.ReverseProxy) that forward CONNECT requests through a shared Transport, this can lead to cross-user response poisoning.

#### Severity
Unknown

#### References
- [https://go.dev/cl/847306](https://go.dev/cl/847306)
- [https://go.dev/issue/81740](https://go.dev/issue/81740)
- [https://groups.google.com/g/golang-announce/c/U2fTuyDJznI](https://groups.google.com/g/golang-announce/c/U2fTuyDJznI)

This data is provided by [OSV](https://osv.dev/vulnerability/GO-2026-6605) and the [Go Vulnerability Database](https://github.com/golang/vulndb) ([CC-BY 4.0](https://github.com/golang/vulndb#license)).
</details>

---

### Reject malformed ECH outer extension references in crypto/tls
[CVE-2026-97031](https://nvd.nist.gov/vuln/detail/CVE-2026-97031) / [GO-2026-6607](https://pkg.go.dev/vuln/GO-2026-6607)

<details>
<summary>More information</summary>

#### Details
Multiple ECH outer extension references are not permitted under RFC 9849; previously, a client could send a well-crafted packet that could trigger memory exhaustion in the server process by specifying multiple references.

We now reject these as malformed and curb the memory amplification vector as a result.

#### Severity
Unknown

#### References
- [https://go.dev/cl/847312](https://go.dev/cl/847312)
- [https://go.dev/issue/81855](https://go.dev/issue/81855)
- [https://groups.google.com/g/golang-announce/c/U2fTuyDJznI](https://groups.google.com/g/golang-announce/c/U2fTuyDJznI)

This data is provided by [OSV](https://osv.dev/vulnerability/GO-2026-6607) and the [Go Vulnerability Database](https://github.com/golang/vulndb) ([CC-BY 4.0](https://github.com/golang/vulndb#license)).
</details>

---

### Memory limit bypass when parsing MIME headers in net/textproto, mime/multipart
[CVE-2026-94440](https://nvd.nist.gov/vuln/detail/CVE-2026-94440) / [GO-2026-6608](https://pkg.go.dev/vuln/GO-2026-6608)

<details>
<summary>More information</summary>

#### Details
Parsing a multipart form can bypass memory limits and read an arbitrarily long line into memory when the remaining limit at the start of a part is less than 400 bytes.

#### Severity
Unknown

#### References
- [https://go.dev/cl/847307](https://go.dev/cl/847307)
- [https://go.dev/issue/81741](https://go.dev/issue/81741)
- [https://groups.google.com/g/golang-announce/c/U2fTuyDJznI](https://groups.google.com/g/golang-announce/c/U2fTuyDJznI)

This data is provided by [OSV](https://osv.dev/vulnerability/GO-2026-6608) and the [Go Vulnerability Database](https://github.com/golang/vulndb) ([CC-BY 4.0](https://github.com/golang/vulndb#license)).
</details>

---

### Lack of limit on size of parsed Range headers in net/http
[CVE-2026-78667](https://nvd.nist.gov/vuln/detail/CVE-2026-78667) / [GO-2026-6609](https://pkg.go.dev/vuln/GO-2026-6609)

<details>
<summary>More information</summary>

#### Details
When parsing a Range header containing a large number of small ranges, FileServer(FS), ServeContent, and ServeFile(FS) can consume an excessive amount of CPU.

#### Severity
Unknown

#### References
- [https://go.dev/cl/847309](https://go.dev/cl/847309)
- [https://go.dev/issue/81858](https://go.dev/issue/81858)
- [https://groups.google.com/g/golang-announce/c/U2fTuyDJznI](https://groups.google.com/g/golang-announce/c/U2fTuyDJznI)

This data is provided by [OSV](https://osv.dev/vulnerability/GO-2026-6609) and the [Go Vulnerability Database](https://github.com/golang/vulndb) ([CC-BY 4.0](https://github.com/golang/vulndb#license)).
</details>

---

### HTTP/2 transport accepts malformed framing-related headers in net/http
[CVE-2026-78660](https://nvd.nist.gov/vuln/detail/CVE-2026-78660) / [GO-2026-6610](https://pkg.go.dev/vuln/GO-2026-6610)

<details>
<summary>More information</summary>

#### Details
Historically, we have been rather lax about malformed framing-related headers in our HTTP/2 implementation, as they cannot interfere with HTTP/2 framing. However, this makes it possible for our HTTP/2 implementation to forward responses containing such headers to an HTTP/1 client when acting as a reverse proxy. If the HTTP/1 client also does not behave strictly enough, this can result in response smuggling.

#### Severity
Unknown

#### References
- [https://go.dev/cl/835145](https://go.dev/cl/835145)
- [https://go.dev/cl/836385](https://go.dev/cl/836385)
- [https://go.dev/issue/81115](https://go.dev/issue/81115)
- [https://groups.google.com/g/golang-announce/c/U2fTuyDJznI](https://groups.google.com/g/golang-announce/c/U2fTuyDJznI)

This data is provided by [OSV](https://osv.dev/vulnerability/GO-2026-6610) and the [Go Vulnerability Database](https://github.com/golang/vulndb) ([CC-BY 4.0](https://github.com/golang/vulndb#license)).
</details>

---

### Excessive CPU consumption from repeated initial window changes in net/http
[CVE-2026-78669](https://nvd.nist.gov/vuln/detail/CVE-2026-78669) / [GO-2026-6611](https://pkg.go.dev/vuln/GO-2026-6611)

<details>
<summary>More information</summary>

#### Details
A malicious HTTP/2 peer can cause excessive CPU consumption in the client or server by opening a large number of streams and then sending many small SETTINGS frames containing SETTINGS_INITIAL_WINDOW_SIZE values.

#### Severity
Unknown

#### References
- [https://go.dev/cl/847186](https://go.dev/cl/847186)
- [https://go.dev/cl/847308](https://go.dev/cl/847308)
- [https://go.dev/issue/81742](https://go.dev/issue/81742)
- [https://groups.google.com/g/golang-announce/c/U2fTuyDJznI](https://groups.google.com/g/golang-announce/c/U2fTuyDJznI)
- [https://groups.google.com/g/golang-announce/c/ZPwCyRUuGBs](https://groups.google.com/g/golang-announce/c/ZPwCyRUuGBs)

This data is provided by [OSV](https://osv.dev/vulnerability/GO-2026-6611) and the [Go Vulnerability Database](https://github.com/golang/vulndb) ([CC-BY 4.0](https://github.com/golang/vulndb#license)).
</details>

---

### Double flow control refund on HTTP/2 server streams in net/http
[CVE-2026-78663](https://nvd.nist.gov/vuln/detail/CVE-2026-78663) / [GO-2026-6612](https://pkg.go.dev/vuln/GO-2026-6612)

<details>
<summary>More information</summary>

#### Details
The HTTP/2 server can refund connection-level flow control twice for the same data: Once when a client resets a stream (refunding data for any sent-but-unread portion of the stream), and again when a request handler reads the buffered data. A malicious client can exploit this to bypass the configured connection-level flow control limit (MaxReceiveBufferPerConnection). Total buffered data is still limited by the concurrent stream limit and stream-level flow control.

#### Severity
Unknown

#### References
- [https://go.dev/cl/847187](https://go.dev/cl/847187)
- [https://go.dev/cl/847310](https://go.dev/cl/847310)
- [https://go.dev/issue/81743](https://go.dev/issue/81743)
- [https://groups.google.com/g/golang-announce/c/U2fTuyDJznI](https://groups.google.com/g/golang-announce/c/U2fTuyDJznI)
- [https://groups.google.com/g/golang-announce/c/ZPwCyRUuGBs](https://groups.google.com/g/golang-announce/c/ZPwCyRUuGBs)

This data is provided by [OSV](https://osv.dev/vulnerability/GO-2026-6612) and the [Go Vulnerability Database](https://github.com/golang/vulndb) ([CC-BY 4.0](https://github.com/golang/vulndb#license)).
</details>

---

### HTTP/1 server connection desynchronization after 2xx CONNECT response in net/http
[CVE-2026-94439](https://nvd.nist.gov/vuln/detail/CVE-2026-94439) / [GO-2026-6613](https://pkg.go.dev/vuln/GO-2026-6613)

<details>
<summary>More information</summary>

#### Details
When an HTTP server handler sends a 2xx response to an HTTP/1 CONNECT request and returns without hijacking the connection, the server improperly continues to read and serve requests from the connection. Since a 2xx response to an HTTP/1 CONNECT converts the connection into a tunnel, the server should not treat the connection as continuing to contain HTTP.

The impact of this misbehavior is mostly limited to potential request smuggling, where an intermediate proxy considers the data on the connection to be tunneled and the server considers it to be HTTP.

#### Severity
Unknown

#### References
- [https://go.dev/cl/847311](https://go.dev/cl/847311)
- [https://go.dev/issue/81744](https://go.dev/issue/81744)
- [https://groups.google.com/g/golang-announce/c/U2fTuyDJznI](https://groups.google.com/g/golang-announce/c/U2fTuyDJznI)

This data is provided by [OSV](https://osv.dev/vulnerability/GO-2026-6613) and the [Go Vulnerability Database](https://github.com/golang/vulndb) ([CC-BY 4.0](https://github.com/golang/vulndb#license)).
</details>

---

### HTTP/2 server crash due to HPACK encoder race in net/http
[CVE-2026-97032](https://nvd.nist.gov/vuln/detail/CVE-2026-97032) / [GO-2026-6617](https://pkg.go.dev/vuln/GO-2026-6617)

<details>
<summary>More information</summary>

#### Details
HTTP/2 servers could end up crashing due to inadvertently modifying its HPACK encoder concurrently. This happens because the server modifies the HPACK encoder from two goroutines without synchronization: one uses the encoder to encode a HEADERS frame as part of a response sent to a client and the other modifies the encoder's table size when handling a SETTINGS frame containing SETTINGS_HEADER_TABLE_SIZE that a client sends. A malicious client can repeatedly send a request while changing the header table size to crash the server.

#### Severity
Unknown

#### References
- [https://go.dev/cl/847188](https://go.dev/cl/847188)
- [https://go.dev/cl/847313](https://go.dev/cl/847313)
- [https://go.dev/issue/81867](https://go.dev/issue/81867)
- [https://groups.google.com/g/golang-announce/c/U2fTuyDJznI](https://groups.google.com/g/golang-announce/c/U2fTuyDJznI)
- [https://groups.google.com/g/golang-announce/c/ZPwCyRUuGBs](https://groups.google.com/g/golang-announce/c/ZPwCyRUuGBs)

This data is provided by [OSV](https://osv.dev/vulnerability/GO-2026-6617) and the [Go Vulnerability Database](https://github.com/golang/vulndb) ([CC-BY 4.0](https://github.com/golang/vulndb#license)).
</details>

---

### Configuration

📅 **Schedule**: (UTC)

- Branch creation
  - ""
- Automerge
  - At any time (no schedule defined)

🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied.

♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.

🔕 **Ignore**: Close this PR and you won't be reminded about this update again.

---

 - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box

---

This PR has been generated by [Mend Renovate](https://github.com/renovatebot/renovate).
<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiI0My4xOTEuMiIsInVwZGF0ZWRJblZlciI6IjQzLjE5MS4yIiwidGFyZ2V0QnJhbmNoIjoibWFpbiIsImxhYmVscyI6W119-->

Reviewed-on: https://gitea.com/gitea/runner/pulls/1291
Reviewed-by: silverwind <2021+silverwind@noreply.gitea.com>
Co-authored-by: Renovate Bot <renovate-bot@gitea.com>
2026-10-10 03:29:46 +00:00
wmTJc9IK0Q
8b14a38d4a enhance!: send OTLP metrics under OpenTelemetry semantic conventions, add runner state (#1282)
Sends the OTLP metrics under [OpenTelemetry semantic convention](https://opentelemetry.io/docs/specs/semconv/) names where the runner's data matches the definition, like `cicd.worker.count`, `cicd.system.errors` and the `go.*` and `process.*` runtime metrics, and under `gitea.runner.*` otherwise. OTel dashboards then work without remapping, as they already do for the traces from https://gitea.com/gitea/runner/pulls/1206.

Adds a `gitea_runner_state` gauge (`busy`, `idle`, `unavailable`), and `/readyz` now fails once the runner shuts down.

**Breaking**

1. OTLP metric names, units, types and attribute values change, and the info and start time metrics move to resource attributes.
1. Prometheus `gitea_runner_client_errors_total` gains a `code` label, and `gitea_runner_job_total` counts jobs stopped by `runner.timeout` as `timeout` instead of `failure`.

---------

Co-authored-by: silverwind <2021+silverwind@noreply.gitea.com>
Co-authored-by: silverwind <me@silverwind.io>
Co-authored-by: bircni <bircni@icloud.com>
Reviewed-on: https://gitea.com/gitea/runner/pulls/1282
Reviewed-by: silverwind <2021+silverwind@noreply.gitea.com>
Co-authored-by: wmTJc9IK0Q <231276+wmtjc9ik0q@noreply.gitea.com>
2026-10-08 18:01:36 +00:00
bircni
179e435cd0 enhance(config): add container.sweep option (#1275)
The Docker proxy labels every container, network and volume a job creates and removes them when the job ends, which breaks jobs that deploy long-running services to the runner's own Docker daemon. Document that behavior and add `container.sweep` (default `true`) so operators who deploy to the same daemon can opt out. Kept resources stay labeled with `com.gitea.runner.job`.

Fixes: https://gitea.com/gitea/runner/issues/1243
Co-authored-by: silverwind <me@silverwind.io>
Reviewed-on: https://gitea.com/gitea/runner/pulls/1275
Reviewed-by: silverwind <2021+silverwind@noreply.gitea.com>
Co-authored-by: bircni <bircni@icloud.com>
2026-10-08 17:53:10 +00:00
bircni
f963e9cfe8 enhance: support client certificates for mutual TLS (#1273)
Add `runner.client_cert_file` and `runner.client_key_file` for Gitea instances that require mutual TLS. The runner presents the certificate on its RPCs, job summary uploads, and the action, reusable workflow and `builtin:checkout` clones from the instance. The files are read on use, so renewed certificates apply without a restart. Job steps do not get them. An incomplete or mismatched pair fails startup.

Clones now decide the token, client certificate and `runner.insecure` with one host check, which also gives reusable workflow clones the TLS settings they lacked. `runner.insecure` still only covers the registered address.

Fixes: https://gitea.com/gitea/runner/issues/321
Co-authored-by: silverwind <2021+silverwind@noreply.gitea.com>
Co-authored-by: silverwind <me@silverwind.io>
Reviewed-on: https://gitea.com/gitea/runner/pulls/1273
Reviewed-by: silverwind <2021+silverwind@noreply.gitea.com>
Co-authored-by: bircni <bircni@icloud.com>
2026-10-08 17:48:31 +00:00
bircni
e5aa9add02 fix: apply the daemon's mtu to job networks (#1270)
Copy the `mtu` of Docker's default bridge network to job networks, since Docker applies the daemon's `mtu` setting only to the default bridge. Docker's default of 1500 is not copied, so `default-network-opts` in the daemon config keeps applying.

Closes https://gitea.com/gitea/runner/issues/232

---------

Co-authored-by: silverwind <2021+silverwind@noreply.gitea.com>
Co-authored-by: silverwind <me@silverwind.io>
Reviewed-on: https://gitea.com/gitea/runner/pulls/1270
Reviewed-by: silverwind <2021+silverwind@noreply.gitea.com>
Co-authored-by: bircni <bircni@icloud.com>
2026-10-08 17:39:54 +00:00
bircni
329115b148 enhance(config): expand environment values (#1274)
Config values can reference the runner's environment as `${NAME}`, for example `capacity: ${RUNNER_CAPACITY}`, so container deployments can configure the runner without templating the file. An unset `NAME` fails the load instead of silently becoming empty, and `$${NAME}` writes a literal `${NAME}`. Keys are never expanded.

Breaking: an existing value containing `${NAME}`, such as a shell command in a Kubernetes `pod_template`, now needs `$${NAME}`.

Closes https://gitea.com/gitea/runner/issues/551
Co-authored-by: silverwind <2021+silverwind@noreply.gitea.com>
Co-authored-by: silverwind <me@silverwind.io>
Reviewed-on: https://gitea.com/gitea/runner/pulls/1274
Reviewed-by: silverwind <2021+silverwind@noreply.gitea.com>
Co-authored-by: bircni <bircni@icloud.com>
2026-10-08 17:31:00 +00:00
bircni
ff30f39b1b enhance(runner): add extra HTTP headers (#1269)
Add `runner.extra_headers` for reverse proxies that require request headers on the runner's API requests to Gitea, like registration, polling and log and status reports.

Headers the runner sets itself (authentication, User-Agent, protocol headers) stay authoritative. Extra headers are dropped when a redirect changes the hostname or leaves HTTPS, while the runner's own authentication headers follow redirects as before. Invalid or case-insensitively duplicated header names are rejected at config load.

Fixes: https://gitea.com/gitea/runner/issues/171
Co-authored-by: silverwind <me@silverwind.io>
Reviewed-on: https://gitea.com/gitea/runner/pulls/1269
Co-authored-by: bircni <bircni@icloud.com>
2026-10-08 17:22:37 +00:00
bircni
3802feb3fe fix: enforce read-only volume restrictions (#1272)
Allow `valid_volumes` entries ending in `:ro` to permit only read-only mounts, so a workflow can no longer mount writable a host path the runner only shares read-only, like a certificate store. `**:ro` allows any volume read-only. An entry without `:ro` that matches too still allows writable mounts.

A named volume allowed only through `:ro` is mounted with `nocopy`, because Docker otherwise fills an empty volume from the image even when it is mounted read-only.

Fixes: https://gitea.com/gitea/runner/issues/407
Co-authored-by: silverwind <me@silverwind.io>
Reviewed-on: https://gitea.com/gitea/runner/pulls/1272
Reviewed-by: silverwind <2021+silverwind@noreply.gitea.com>
Co-authored-by: bircni <bircni@icloud.com>
2026-10-08 17:15:41 +00:00
bircni
27e9b4de87 fix!: use the git CLI for action clones and make builtin:checkout match actions/checkout (#1277)
The runner now uses the `git` command line instead of go-git to download actions and reusable workflows. go-git had recurring bugs and does not support SHA-256 repositories, and full clones of actions are several times faster.

`builtin:checkout` now runs `git` where the job runs and behaves like `actions/checkout`:

1. A branch is checked out as a local branch tracking `origin`, tag and pull request runs keep their ref, and a triggering tag that moved fails the step.
1. Later steps can fetch and push with the persisted token or SSH key.
1. It supports the `ssh-*`, `lfs`, `submodules`, `sparse-checkout`, `filter`, `fetch-tags`, `clean`, `persist-credentials`, `show-progress` and `set-safe-directory` inputs and the `ref` and `commit` outputs.
1. SHA-256 repositories work for actions, reusable workflows and `builtin:checkout`.

Git keeps the token off its command line, ignores `GIT_DIR` and similar variables inherited by the runner, and keeps action files LF regardless of `core.autocrlf`. A cancelled job lets git remove its lock files.

This is a breaking change: the runner needs Git 2.34.1 or newer on its `PATH`, and so do job images that use `builtin:checkout`. The official runner images include it.

Fixes: https://gitea.com/gitea/runner/issues/1077
Fixes: https://gitea.com/gitea/runner/issues/1267
Fixes: https://gitea.com/gitea/runner/issues/1279
Fixes: https://gitea.com/gitea/runner/issues/1284
Fixes: https://gitea.com/gitea/runner/issues/1285
Co-authored-by: silverwind <me@silverwind.io>
Co-authored-by: silverwind <2021+silverwind@noreply.gitea.com>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
Reviewed-on: https://gitea.com/gitea/runner/pulls/1277
Reviewed-by: Lunny Xiao <xiaolunwen@gmail.com>
Reviewed-by: silverwind <2021+silverwind@noreply.gitea.com>
Co-authored-by: bircni <bircni@icloud.com>
2026-10-08 17:08:29 +00:00
bircni
4bedadef9d fix: reject workflows with jobs lacking runs-on (#1271)
Fail the workflow before any of its jobs run when a regular job omits `runs-on` or leaves it empty, as GitHub does, instead of running the job in the fallback image. Gitea rejects such workflows itself since https://github.com/go-gitea/gitea/pull/39480. This covers released Gitea versions, which hand a job without labels to any runner, and `exec`. Reusable workflow jobs and the stepless placeholders the runner builds for a job's `needs` are exempt.

Closes https://gitea.com/gitea/runner/issues/836

---------

Co-authored-by: silverwind <me@silverwind.io>
Reviewed-on: https://gitea.com/gitea/runner/pulls/1271
Reviewed-by: silverwind <2021+silverwind@noreply.gitea.com>
2026-10-08 09:48:07 +00:00
bircni
83b63250a3 fix: strip log-driver and OOM options when privileged is off (#1283)
When privileged mode is disabled, reset `--log-driver`, `--log-opt`, `--oom-score-adj` and `--oom-kill-disable` from workflow `container.options`. Log drivers run in the host daemon, so a workflow could make it connect to any address, and `--oom-score-adj -1000` exempts a job from the host OOM killer.

Breaking: on unprivileged runners, workflows can no longer set these options, e.g. `--log-driver none` or `--log-opt max-size=10m`. They are dropped with a warning and the job still runs. Runner-level `container.options` and privileged runners are unaffected.

Fixes: https://github.com/go-gitea/gitea/security/advisories/GHSA-cgx2-cw7h-xgcx
Fixes: https://github.com/go-gitea/gitea/security/advisories/GHSA-c988-q5xx-2cff
Co-authored-by: silverwind <2021+silverwind@noreply.gitea.com>
Co-authored-by: silverwind <me@silverwind.io>
Reviewed-on: https://gitea.com/gitea/runner/pulls/1283
Reviewed-by: silverwind <2021+silverwind@noreply.gitea.com>
2026-10-07 19:04:04 +00:00
silverwind
2e59141ae9 fix: allow tmpfs mounts in container options (#1289)
Support tmpfs in container `options` via `--mount type=tmpfs,...`, like GitHub does. The `valid_volumes` check silently dropped these mounts because they have no source. A tmpfs is backed by memory and exposes nothing from the host, same as `--tmpfs`, which already passes. Every other mount still goes through the allowlist.

Both forms now work:

```yaml
services:
  mysql:
    image: mysql:8.4
    options: --tmpfs /var/lib/mysql # already worked
```

```yaml
services:
  mysql:
    image: mysql:8.4
    options: --mount type=tmpfs,destination=/var/lib/mysql # was dropped, now works
```

Reviewed-on: https://gitea.com/gitea/runner/pulls/1289
Reviewed-by: bircni <bircni@icloud.com>
Co-authored-by: silverwind <me@silverwind.io>
2026-10-06 19:19:51 +00:00
Drew Inglis
020f0d2861 fix: extract copied directories at their destination in Kubernetes pods (#1286)
The Kubernetes backend extracted CopyDir archives at "/" with entries like
"var/run/act/actions/<sha>/...". Ubuntu's patched tar refuses to extract
through an absolute symlink, failing with "Invalid cross-device link" on
images that link /var/run to /run, so actions never reached the job
container. Extract at the destination with relative entries instead, like
Copy already does.

This is a similar issue to #1128, but for k8s instead of Docker.

I built the image and deployed it to my k8s cluster to verify the fix.

---------

Co-authored-by: bircni <bircni@icloud.com>
Co-authored-by: silverwind <me@silverwind.io>
Reviewed-on: https://gitea.com/gitea/runner/pulls/1286
Reviewed-by: silverwind <2021+silverwind@noreply.gitea.com>
Co-authored-by: Drew Inglis <drew@drewinglis.com>
2026-10-05 11:05:17 +00:00
Renovate Bot
711b72735b chore(deps): update dependencies (#1288)
Co-authored-by: bircni <bircni@icloud.com>
Co-authored-by: silverwind <me@silverwind.io>
Reviewed-on: https://gitea.com/gitea/runner/pulls/1288
Reviewed-by: bircni <bircni@icloud.com>
Co-authored-by: Renovate Bot <renovate-bot@gitea.com>
2026-10-05 04:43:34 +00:00
silverwind
436d0d2942 fix: recognize Gitea's 401 reply for an unregistered runner (#1287)
Since https://github.com/go-gitea/gitea/pull/39578 (backported to v28), Gitea answers an unregistered runner with a Connect `Unauthenticated` error instead of a gRPC status wrapped as `Unknown`. `IsRegistrationRejected` missed it, so an ephemeral runner kept its spent registration file and failed on every restart. This also fails the `gitea/gitea:main-nightly` e2e job.

Match on the `unregistered runner` message suffix, which both shapes share. The code can't distinguish Gitea from a proxy, because a proxy's 401 with a JSON body also decodes as a wire `Unauthenticated` error.

Written by Claude Code.

Reviewed-on: https://gitea.com/gitea/runner/pulls/1287
Reviewed-by: bircni <bircni@icloud.com>
Co-authored-by: silverwind <me@silverwind.io>
2026-10-05 03:34:00 +00:00
silverwind
a284a62d96 chore(deps): update gitea.dev/actionslib to v1.3.0 (#1278)
Updates to [actionslib v1.3.0](https://gitea.com/gitea/actionslib/releases/tag/v1.3.0). Matrix include and exclude entries now match combinations with expression equality like GitHub, so an include of `os: LINUX` extends the `os: linux` combination instead of adding a new one. No code changes are needed.

Reviewed-on: https://gitea.com/gitea/runner/pulls/1278
Reviewed-by: Lunny Xiao <xiaolunwen@gmail.com>
Co-authored-by: silverwind <me@silverwind.io>
2026-10-02 18:53:13 +00:00
bircni
2b1e64e414 ci: Skip builds on push on main (#1276)
These builds were already tested on pr merge so why run again?

Reviewed-on: https://gitea.com/gitea/runner/pulls/1276
Reviewed-by: silverwind <2021+silverwind@noreply.gitea.com>
2026-10-01 21:21:38 +00:00
silverwind
ddee2e054a fix: align Docker actions, job env and host mode with GitHub (#1268)
- Docker actions evaluate `runs.args` and `runs.env` with their own `inputs`, so a same-named `workflow_dispatch` input no longer replaces the step's `with:` value
- Dockerfile action images are tagged per step and job and removed afterwards, so changed Dockerfiles rebuild and concurrent jobs never share an image
- Workflow and job `env` are evaluated before job and service container credentials and service `env` use them
- Docker actions in host jobs get the host workspace and file-command directory mounted, and no longer inherit the runner's own `PATH`
- Cache server address detection accepts a single interface address

Fixes: https://gitea.com/gitea/runner/issues/390
Fixes: https://gitea.com/gitea/runner/issues/406
Fixes: https://gitea.com/gitea/runner/issues/578
Related: https://gitea.com/gitea/runner/issues/64
Reviewed-on: https://gitea.com/gitea/runner/pulls/1268
Reviewed-by: bircni <bircni@icloud.com>
Co-authored-by: silverwind <me@silverwind.io>
2026-10-01 19:45:13 +00:00
silverwind
629e062f86 fix: drop stalled HTTP/2 connections in action clones (#1266)
go-git sends all concurrent action clones from a host over one HTTP/2 connection, and Go sends no health-check pings by default. When that connection stalls, every clone on it hangs until the job timeout and cancellation can't free them, as in https://gitea.com/gitea/runner/actions/runs/1003885/jobs/1361593.

Ping idle connections so a stalled one is dropped after about 45s.

---------

Co-authored-by: bircni <bircni@icloud.com>
Reviewed-on: https://gitea.com/gitea/runner/pulls/1266
Reviewed-by: bircni <bircni@icloud.com>
Co-authored-by: silverwind <me@silverwind.io>
2026-10-01 16:48:19 +00:00
bircni
73e9552f06 fix: fail fast in register and honor GITEA_MAX_REG_ATTEMPTS (#1197)
`gitea-runner register` pinged the instance in an unbounded loop, so a permanent failure like an untrusted certificate or a wrong URL never returned. It now pings once and returns the error, like other one-shot commands.

The image entrypoint keeps retrying registration indefinitely by default, like the daemon, so a container without a restart policy still recovers once Gitea is reachable. `GITEA_MAX_REG_ATTEMPTS` had no effect because its counter was incremented outside the loop. When set, it now caps the attempts, after which the entrypoint exits non-zero.

Fixes: https://gitea.com/gitea/runner/issues/1195
Co-authored-by: silverwind <me@silverwind.io>
Reviewed-on: https://gitea.com/gitea/runner/pulls/1197
Reviewed-by: Lunny Xiao <xiaolunwen@gmail.com>
Reviewed-by: silverwind <2021+silverwind@noreply.gitea.com>
Co-authored-by: bircni <bircni@icloud.com>
2026-10-01 16:01:14 +00:00
silverwind
05162998eb fix: dind image readiness, exit codes and shutdown order (#1265)
Fixes https://gitea.com/gitea/runner/issues/1263

- Wait for dockerd without a check limit. s6 readiness gave up after 7 checks, so the runner never started on slow hosts, also seen in https://gitea.com/gitea/runner/issues/712, https://gitea.com/gitea/runner/issues/652 and https://gitea.com/gitea/runner/issues/758.
- Let the runner run the docker service's readiness check itself instead of reading s6 status, which can be missing at startup or stale after an unclean stop.
- Exit the container with the first failing service's exit code instead of always 0, so `restart: on-failure` works. A requested stop still exits 0.
- Stop the runner before dockerd, so running jobs can finish within `shutdown_timeout`. The stop grace period has to cover that drain.
- Check the supervision exit codes in `make test-dind`.

---------

Co-authored-by: bircni <bircni@icloud.com>
Reviewed-on: https://gitea.com/gitea/runner/pulls/1265
Reviewed-by: bircni <bircni@icloud.com>
2026-10-01 13:22:25 +00:00
wmTJc9IK0Q
568f89d186 enhance: export the Prometheus metrics over OpenTelemetry (#1248)
Push the runner's Prometheus metrics to the OTLP endpoint next to the job traces, so both land in the same backend.

The `/metrics` registry is converted as the [Prometheus compatibility spec](https://opentelemetry.io/docs/specs/otel/compatibility/prometheus_and_openmetrics/) describes and sent through the existing gRPC-free client, so names stay unchanged and no second metrics stack is linked. Metrics are on by default and pushed every interval and once on shutdown, so ephemeral runners report too.

New standard variables, each `OTEL_EXPORTER_OTLP_*` one with `TRACES` and `METRICS` variants:

- `OTEL_METRICS_EXPORTER`, `none` turns metrics off
- `OTEL_METRIC_EXPORT_INTERVAL`, `OTEL_METRIC_EXPORT_TIMEOUT`
- `OTEL_EXPORTER_OTLP_TIMEOUT`, `OTEL_EXPORTER_OTLP_COMPRESSION`
- `OTEL_EXPORTER_OTLP_CERTIFICATE`, `OTEL_EXPORTER_OTLP_CLIENT_CERTIFICATE`, `OTEL_EXPORTER_OTLP_CLIENT_KEY`

---------

Co-authored-by: silverwind <me@silverwind.io>
Co-authored-by: silverwind <2021+silverwind@noreply.gitea.com>
Co-authored-by: bircni <bircni@icloud.com>
Reviewed-on: https://gitea.com/gitea/runner/pulls/1248
Reviewed-by: silverwind <2021+silverwind@noreply.gitea.com>
Reviewed-by: bircni <bircni@icloud.com>
Co-authored-by: wmTJc9IK0Q <231276+wmtjc9ik0q@noreply.gitea.com>
2026-10-01 12:55:16 +00:00
bircni
d93b7d278a fix: clear consumed ephemeral registration state (#1256)
Ephemeral registrations are single-use, but the daemon kept `.runner` after the task, so a restarted container failed `Declare` with `unregistered runner` forever.

An ephemeral runner now removes its registration file as soon as it fetches a task, so a container killed mid-job registers afresh on restart. It also removes the file when Gitea answers `unregistered runner` at startup or while polling. A proxy's 401 does not count. Persistent runners keep their file, because Gitea sends the same reply for a token mismatch.

Closes https://gitea.com/gitea/runner/issues/818

---------

Co-authored-by: silverwind <me@silverwind.io>
Reviewed-on: https://gitea.com/gitea/runner/pulls/1256
Reviewed-by: silverwind <2021+silverwind@noreply.gitea.com>
2026-10-01 09:23:45 +00:00
silverwind
0cedd98051 feat: add Kubernetes backend (#1260)
Adds a Kubernetes backend. A `name:kubernetes://image` label runs the job as one pod through the Kubernetes API, with services as containers of the same pod. Step env is exported by a script sent over exec stdin, keeping it out of the pod spec and process arguments, and service env goes into a per-job Secret. Docker actions and container options are rejected. See `docs/kubernetes.md`.

Operators shape job pods with `kubernetes.pod_template`, with per-label `pod_templates` merged in `runs-on` order and a `$services` entry for service containers. The idle cleanup removes pods and Secrets a runner left behind.

The client calls the Kubernetes API directly and runs step commands over `coder/websocket` (same dependency used in gitea) instead of pulling in the heavyweight `kubernetes/client-go`. Includes an e2e test against k3s.

Replaces: https://gitea.com/gitea/runner/pulls/1000
Fixes: https://gitea.com/gitea/runner/issues/31

Assisted-by: Claude Code:claude-opus-5-5
Co-authored-by: Zettat123 <zettat123@gmail.com>
Co-authored-by: bircni <bircni@icloud.com>
Reviewed-on: https://gitea.com/gitea/runner/pulls/1260
Reviewed-by: Lunny Xiao <xiaolunwen@gmail.com>
Reviewed-by: Zettat123 <39446+zettat123@noreply.gitea.com>
Co-authored-by: silverwind <me@silverwind.io>
2026-10-01 06:59:15 +00:00
bircni
07ca4bfa7f enhance: log task, job and run identifiers when a task starts (#1257)
Replace the `task N repo is ...` line with a structured `task started` entry carrying `task_id`, `job_id`, `run_id`, `run_number`, `repository` and the workflow's `job` key, so external supervisors can match runner activity to the job in Gitea. `job_id` is read from the runtime token's claims, tokens and secrets are never logged.

Closes https://gitea.com/gitea/runner/issues/752

---------

Co-authored-by: silverwind <2021+silverwind@noreply.gitea.com>
Co-authored-by: silverwind <me@silverwind.io>
Reviewed-on: https://gitea.com/gitea/runner/pulls/1257
Reviewed-by: silverwind <2021+silverwind@noreply.gitea.com>
2026-09-30 19:42:52 +00:00
bircni
78610b01f6 docs: recommend DinD for container jobs (#1258)
Make Docker-in-Docker the recommended and first-shown setup. Mounting the host's Docker socket gives every job root-equivalent access to the host, while DinD gives jobs their own daemon. The default `latest` tag is unchanged.

Also shorten the image flavour docs to what users need to pick an image, and correct that `dind-rootless` needs `--privileged` too.

Closes https://gitea.com/gitea/runner/issues/1227

---------

Co-authored-by: silverwind <2021+silverwind@noreply.gitea.com>
Co-authored-by: silverwind <me@silverwind.io>
Reviewed-on: https://gitea.com/gitea/runner/pulls/1258
Reviewed-by: silverwind <2021+silverwind@noreply.gitea.com>
Co-authored-by: bircni <bircni@icloud.com>
2026-09-30 18:27:11 +00:00
bircni
4a8ee4ce57 fix: resolve container sleep through PATH (#1255)
Resolve the job container's sleep command through PATH so images with nonstandard binary locations, including Nix, can run jobs.

Closes https://gitea.com/gitea/runner/issues/808

---------

Co-authored-by: silverwind <me@silverwind.io>
Reviewed-on: https://gitea.com/gitea/runner/pulls/1255
Reviewed-by: silverwind <2021+silverwind@noreply.gitea.com>
2026-09-29 18:44:57 +00:00
bircni
bf03f36d5a fix: honor configured Docker registration file (#1259)
The Docker entrypoint always checked `.runner` in `/data` and ignored `runner.file` from `CONFIG_FILE`. When `runner.file` pointed elsewhere, it re-registered on every start, or skipped registration because of a stale `.runner`. It now reads the path from a new `config runner-file` subcommand, which resolves `runner.file` the same way as `register` and `daemon`.

Closes https://gitea.com/gitea/runner/issues/585

---------

Co-authored-by: silverwind <me@silverwind.io>
Reviewed-on: https://gitea.com/gitea/runner/pulls/1259
Reviewed-by: silverwind <2021+silverwind@noreply.gitea.com>
2026-09-29 18:38:35 +00:00
Renovate Bot
5a38a3d0c4 chore(deps): update dependencies (#1254)
This PR contains the following updates:

| Package | Change | [Age](https://docs.renovatebot.com/merge-confidence/) | [Confidence](https://docs.renovatebot.com/merge-confidence/) | Type | Update |
|---|---|---|---|---|---|
| [github.com/bmatcuk/doublestar/v4](https://github.com/bmatcuk/doublestar) | `v4.10.0` → `v4.10.2` | ![age](https://developer.mend.io/api/mc/badges/age/go/github.com%2fbmatcuk%2fdoublestar%2fv4/v4.10.2?slim=true) | ![confidence](https://developer.mend.io/api/mc/badges/confidence/go/github.com%2fbmatcuk%2fdoublestar%2fv4/v4.10.0/v4.10.2?slim=true) | require | patch |
| [github.com/ulikunitz/xz](https://github.com/ulikunitz/xz) | `v0.5.16` → `v0.5.17` | ![age](https://developer.mend.io/api/mc/badges/age/go/github.com%2fulikunitz%2fxz/v0.5.17?slim=true) | ![confidence](https://developer.mend.io/api/mc/badges/confidence/go/github.com%2fulikunitz%2fxz/v0.5.16/v0.5.17?slim=true) |  | patch |
| nginx | `62ff208` → `df221db` |  |  |  | digest |

---

### Release Notes

<details>
<summary>bmatcuk/doublestar (github.com/bmatcuk/doublestar/v4)</summary>

### [`v4.10.2`](https://github.com/bmatcuk/doublestar/releases/tag/v4.10.2): Better support for Windows "Junctions"

[Compare Source](https://github.com/bmatcuk/doublestar/compare/v4.10.1...v4.10.2)

Has improved support for reading "junctions" in Windows (ie, Windows' version of symlinks).

Thanks [@&#8203;x0Lazarus](https://github.com/x0Lazarus) for the PR!

#### What's Changed

- Fix Windows junction traversal with Go 1.23 and later by [@&#8203;x0Lazarus](https://github.com/x0Lazarus) in [#&#8203;117](https://github.com/bmatcuk/doublestar/pull/117)

#### New Contributors

- [@&#8203;x0Lazarus](https://github.com/x0Lazarus) made their first contribution in [#&#8203;117](https://github.com/bmatcuk/doublestar/pull/117)

**Full Changelog**: <https://github.com/bmatcuk/doublestar/compare/v4.10.1...v4.10.2>

### [`v4.10.1`](https://github.com/bmatcuk/doublestar/releases/tag/v4.10.1): Fixed issue with alternatives and character classes

[Compare Source](https://github.com/bmatcuk/doublestar/compare/v4.10.0...v4.10.1)

There was an issue where commas or ending curly brackets inside a character class would cause code looking for the next alternative (commas) or the end of alternatives (ending curly brackets) to erroneously match those characters inside the character classes.

Thanks to [@&#8203;sidsri14](https://github.com/sidsri14) for the fix!

#### What's Changed

- fix brace alternation scanning with character classes by [@&#8203;sidsri14](https://github.com/sidsri14) in [#&#8203;115](https://github.com/bmatcuk/doublestar/pull/115)

#### New Contributors

- [@&#8203;sidsri14](https://github.com/sidsri14) made their first contribution in [#&#8203;115](https://github.com/bmatcuk/doublestar/pull/115)

**Full Changelog**: <https://github.com/bmatcuk/doublestar/compare/v4.10.0...v4.10.1>

</details>

<details>
<summary>ulikunitz/xz (github.com/ulikunitz/xz)</summary>

### [`v0.5.17`](https://github.com/ulikunitz/xz/compare/v0.5.16...v0.5.17)

[Compare Source](https://github.com/ulikunitz/xz/compare/v0.5.16...v0.5.17)

</details>

---

### Configuration

📅 **Schedule**: (UTC)

- Branch creation
  - Between 12:00 AM and 03:59 AM, only on Monday (`* 0-3 * * 1`)
- Automerge
  - At any time (no schedule defined)

🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied.

♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.

👻 **Immortal**: This PR will be recreated if closed unmerged. Get [config help](https://github.com/renovatebot/renovate/discussions) if that's undesired.

---

 - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box

---

This PR has been generated by [Mend Renovate](https://github.com/renovatebot/renovate).
<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiI0My4xOTEuMiIsInVwZGF0ZWRJblZlciI6IjQzLjE5MS4yIiwidGFyZ2V0QnJhbmNoIjoibWFpbiIsImxhYmVscyI6W119-->

---------

Co-authored-by: silverwind <2021+silverwind@noreply.gitea.com>
Reviewed-on: https://gitea.com/gitea/runner/pulls/1254
Reviewed-by: silverwind <2021+silverwind@noreply.gitea.com>
Co-authored-by: Renovate Bot <renovate-bot@gitea.com>
2026-09-28 09:49:36 +00:00
silverwind
6618e67645 fix: apply a job container without image to the runs-on image (#1252)
Fixes https://gitea.com/gitea/runner/issues/1251, a regression from https://gitea.com/gitea/runner/pulls/1233.

A job `container:` without an `image` key configures the runs-on image again, as in 3.x, so its `volumes`, `options`, `credentials` and `env` apply. An image that evaluates to empty still skips the container, as on GitHub.

---------

Co-authored-by: bircni <bircni@icloud.com>
Reviewed-on: https://gitea.com/gitea/runner/pulls/1252
Reviewed-by: Lunny Xiao <xiaolunwen@gmail.com>
Reviewed-by: bircni <bircni@icloud.com>
Co-authored-by: silverwind <me@silverwind.io>
2026-09-28 09:43:48 +00:00
silverwind
2b599fae7f fix: misc bug fixed and improvements (#1253)
1. Host mode is selected only through `runs-on` again, as in upstream act.
1. A workflow volume that `valid_volumes` drops no longer displaces the runner's own socket, workspace, tool cache or `/var/run/act` mount, so `/var/run/docker.sock:/var/run/docker.sock` works on hosts whose daemon socket lives elsewhere.
1. `docker://` is stripped from job and service images, as on GitHub.
1. Each created service's log is printed at job end, and service log dumps no longer run workflow commands, as on GitHub.
1. A `--privileged` dropped because privileged mode is off now logs a warning.
1. README: the proxy variables change per step through `env:` or for the rest of a job through `$GITHUB_ENV`, `container.env` does not override them.

Reviewed-on: https://gitea.com/gitea/runner/pulls/1253
Reviewed-by: Zettat123 <39446+zettat123@noreply.gitea.com>
Co-authored-by: silverwind <me@silverwind.io>
2026-09-28 05:53:41 +00:00
Zettat123
990ebaa2ac fix: restore the caller's event and inputs in reusable workflow jobs (#1250)
Fixes https://github.com/go-gitea/gitea/issues/38705

Gitea side: https://github.com/go-gitea/gitea/pull/39452

Jobs of a called workflow currently see `gitea.event_name == "workflow_call"`, and their `gitea.event.inputs` is replaced by the caller's `with:`. On GitHub they keep the caller's event, and their `inputs` are the run's `workflow_dispatch` inputs overlaid with the caller's `with:`.

For compatibility with older runners, Gitea keeps sending that form and adds a `gitea_workflow_call` key with the original trigger event name and inputs of the run, and the `inputs` Gitea resolved for the job. When the key is present, the runner restores `gitea.event_name` and `gitea.event.inputs` from it, and uses the resolved `inputs` for the job's expressions and the setup log.

Without that key (older Gitea, or jobs outside called workflows), nothing changes.

Reviewed-on: https://gitea.com/gitea/runner/pulls/1250
Reviewed-by: bircni <bircni@icloud.com>
Reviewed-by: silverwind <2021+silverwind@noreply.gitea.com>
Co-authored-by: Zettat123 <zettat123@gmail.com>
2026-09-27 16:13:14 +00:00
180 changed files with 9981 additions and 3453 deletions

View File

@@ -19,7 +19,7 @@ jobs:
timeout-minutes: 5
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7
- uses: actions/setup-node@820762786026740c76f36085b0efc47a31fe5020 # v7
- uses: actions/setup-node@949feb2413d6458794dcd2491c4babbbce0c15c1 # v7
with:
node-version: 24
- run: make lint-pr-title

View File

@@ -1,9 +1,7 @@
name: checks
on:
push:
branches:
- main
pull_request:
workflow_dispatch:
env:
# The runner image ships a stale docker.io login; point docker at an empty config so
@@ -30,8 +28,8 @@ jobs:
# daemon retains them between runs, so this is usually a fast manifest re-check.
- name: pre-pull test images
env:
TEST_JOB_IMAGE: node:24-bookworm-slim@sha256:0e0ff40c39bc087845bfb27465a0df4ea419520094bc35842ff83dd8cbe6f9b6 # renovate: datasource=docker
TEST_SERVICE_IMAGE: nginx:alpine@sha256:62ff2089abf5a9ed33bd232895bef5e22f7bb4b200675cec49a5ebc48e3d4ac8 # renovate: datasource=docker
TEST_JOB_IMAGE: node:24-bookworm-slim@sha256:d6aa754f16b3197301076f047b5def2f02ea1dbbc2ca920407d46d7ec7f87b20 # renovate: datasource=docker
TEST_SERVICE_IMAGE: nginx:alpine@sha256:df221db836e1754089190208cee7eeda94f233197056426eda74a43ab1abeac2 # renovate: datasource=docker
run: |
for image in "$TEST_JOB_IMAGE" "$TEST_SERVICE_IMAGE"; do
for attempt in 1 2 3; do

View File

@@ -39,6 +39,15 @@ linters:
deny:
- pkg: golang.org/x/exp
desc: it's experimental and unreliable
logging:
files:
- $all
- "!**/act/common/logging/**"
deny:
- pkg: github.com/sirupsen/logrus
desc: log through gitea.com/gitea/runner/act/common/logging
- pkg: log/slog
desc: log through gitea.com/gitea/runner/act/common/logging
nolintlint:
allow-unused: false
require-explanation: true

View File

@@ -31,6 +31,26 @@ depending on the prefix:
- Command *properties* also escape `%3A` and `%2C`, which the UI never decodes, so the reporter
decodes exactly those two when folding a location into an annotation.
## Logging
Log through `act/common/logging`; the linter rejects importing `log/slog` or logrus elsewhere.
A job's logger comes from `common.Logger(ctx)`; code with no job uses the package functions
(`logging.Infof`, `logging.With`). Both take printf-style messages, and `With("key", value)` adds
fields.
- The process-wide logger masks the secrets of every live task, the job logger masks its own.
Going around either leaks them.
- The job logger's output can reach the job log workflow authors see, the process-wide logger's
stays on the runner host, so keep operator-only detail (host paths, internal addresses) off the
job logger.
- Libraries that log through logrus's or slog's global logger are forwarded to the process-wide
one, so they follow its level and masking too.
- That forwarding only works while logrus's global logger is left as `logging` configures it. A
dependency that calls `logrus.SetOutput` or `SetLevel` breaks it, so check
`logging.LogrusBridgeIntact()` when adding or upgrading one.
- Tests read what was logged with `logtest.NewLogger()`, or `logtest.SetDefault(t)` for the
package functions.
## Built-in actions
Each lives in `internal/pkg/action/<name>` and is registered in `builtinActions` in

View File

@@ -6,12 +6,13 @@ SHASUM ?= shasum -a 256
HAS_GO = $(shell hash $(GO) > /dev/null 2>&1 && echo "GO" || echo "NOGO" )
XGO_PACKAGE ?= src.techknowlogick.com/xgo@v1.9.0 # renovate: datasource=go
XGO_VERSION := go-1.27.x
GXZ_PACKAGE ?= github.com/ulikunitz/xz/cmd/gxz@v0.5.16 # renovate: datasource=go
GXZ_PACKAGE ?= github.com/ulikunitz/xz/cmd/gxz@v0.5.17 # renovate: datasource=go
PODMAN_TEST_IMAGE ?= quay.io/containers/podman:v5.8.4-immutable@sha256:90405e3c6a86f33c5a6f8dbdfa8e02a98143a8ba4337b84bdaec5336d2ef008a # renovate: datasource=docker
E2E_JOB_IMAGE ?= node:24-bookworm@sha256:64af3819f9275802414d7cdc38c27e9d82bd564dec4d4da87d008255d36c63b4 # renovate: datasource=docker
E2E_JOB_IMAGE ?= docker.gitea.com/runner-images:ubuntu-latest@sha256:0e62e56b382ebf485bff1e51a05cbe27c708545a2e91425467fd31eb3e249521 # renovate: datasource=docker
SERVICE_IMAGE ?= nginx:1.31.4-alpine@sha256:db35bfc6b2951e7f8a72db5db120288c127ffaeeb4a6d4b95a26fead017d5913 # renovate: datasource=docker
E2E_GITEA_IMAGE ?= gitea/gitea:main-nightly
E2E_K3S_IMAGE ?= rancher/k3s:v1.36.4-k3s1@sha256:edad48e12bf81c3a09ac1c05c0c0ffaaa22145980b989d6fae84543a76b83657 # renovate: datasource=docker
LINUX_ARCHS ?= linux/amd64,linux/arm64
DARWIN_ARCHS ?= darwin-12/amd64,darwin-12/arm64
@@ -23,7 +24,7 @@ DOCKER_TAG ?= nightly
DOCKER_REF := $(DOCKER_IMAGE):$(DOCKER_TAG)
DOCKER_ROOTLESS_REF := $(DOCKER_IMAGE):$(DOCKER_TAG)-dind-rootless
GOLANGCI_LINT_PACKAGE ?= github.com/golangci/golangci-lint/v2/cmd/golangci-lint@v2.13.2 # renovate: datasource=go
GOLANGCI_LINT_PACKAGE ?= github.com/golangci/golangci-lint/v2/cmd/golangci-lint@v2.14.0 # renovate: datasource=go
GOVULNCHECK_PACKAGE ?= golang.org/x/vuln/cmd/govulncheck@v1.8.0 # renovate: datasource=go
GOTEST_FLAGS ?= -race -timeout 20m -parallel 8
@@ -170,7 +171,7 @@ E2E_CONCURRENCY ?= 8
.PHONY: test-e2e
test-e2e: ## run Gitea compatibility tests against E2E_GITEA_IMAGE
@E2E_CONCURRENCY=$(E2E_CONCURRENCY) E2E_GITEA_IMAGE=$(E2E_GITEA_IMAGE) E2E_JOB_IMAGE=$(E2E_JOB_IMAGE) GO=$(GO) SERVICE_IMAGE=$(SERVICE_IMAGE) ./tools/test-e2e.sh
@E2E_CONCURRENCY=$(E2E_CONCURRENCY) E2E_GITEA_IMAGE=$(E2E_GITEA_IMAGE) E2E_K3S_IMAGE=$(E2E_K3S_IMAGE) E2E_JOB_IMAGE=$(E2E_JOB_IMAGE) GO=$(GO) SERVICE_IMAGE=$(SERVICE_IMAGE) ./tools/test-e2e.sh
.PHONY: install
install: $(GOFILES) ## install the runner binary via `go install`

View File

@@ -4,7 +4,19 @@
### Prerequisites
Docker Engine Community version is required for docker mode. To install Docker CE, follow the official [install instructions](https://docs.docker.com/engine/install/).
On the runner:
- `git` 2.34.1 or newer on `PATH`, to clone reusable workflows, and actions whose host serves no archive of them.
- `docker` or `podman` with `podman.socket` enabled, for docker mode, Docker actions and `docker://` steps. To install Docker CE, follow the official [install instructions](https://docs.docker.com/engine/install/).
Where steps run, in the job image or on the runner in host mode:
- `bash` or `sh` for `run` steps, `pwsh` or `powershell` on a Windows host.
- `node` for JavaScript actions and `hashFiles()`.
- `git` 2.34.1 or newer for [`builtin:checkout`](docs/builtin-actions.md).
- `git-lfs` and `ssh` for `builtin:checkout`, only when its `lfs` or `ssh-key` input is used.
Kubernetes mode needs [a few more tools](docs/kubernetes.md) in job images.
### Download pre-built binary
@@ -79,53 +91,24 @@ If the registry succeed, it will run immediately. Next time, you could run the r
### Run with docker
```bash
docker run -e GITEA_INSTANCE_URL=https://your_gitea.com -e GITEA_RUNNER_REGISTRATION_TOKEN=<your_token> -v /var/run/docker.sock:/var/run/docker.sock --name my_runner gitea/runner:nightly
```
Mount a volume on `/data` if you want the registration file and optional config to survive container recreation (see [scripts/run.sh](scripts/run.sh)).
> **`/data` does not hold the image cache.** It is the runner's working directory and contains only the `.runner` registration file and, optionally, your config file. Images pulled for jobs live in the *Docker daemon's* data root, which for the `dind` flavours is inside the container (`/var/lib/docker`, or `/home/rootless/.local/share/docker` for `dind-rootless`). To keep the image cache across restarts, give that path its own volume as well — otherwise every new container re-pulls the job images. With the `basic` flavour the images live on whichever daemon you point the runner at, so there is nothing extra to persist.
### Image flavours
The image is published in three flavours, all built from the single multi-stage [Dockerfile](Dockerfile) in this repository. They differ only in how a Docker daemon is made available to the jobs the runner executes; the `gitea-runner` binary inside them is identical.
| Tag | Build target | Base image | Docker daemon | Process supervisor | Runs as |
| --- | --- | --- | --- | --- | --- |
| `latest` (and `<version>`) | `basic` | `alpine` | none — uses an external daemon you provide | [`tini`](https://github.com/krallin/tini) | `root` |
| `latest-dind` | `dind` | `docker:dind` | bundled, started inside the container | [`s6`](https://skarnet.org/software/s6/) | `root` (privileged) |
| `latest-dind-rootless` | `dind-rootless` | `docker:dind-rootless` | bundled, started rootless inside the container | [`s6`](https://skarnet.org/software/s6/) | `rootless` (UID 1000) |
#### `latest` — basic
The default flavour ships only the runner on a minimal Alpine base. It contains **no Docker daemon of its own**: jobs that use `docker://` images need a daemon supplied from outside the container, typically by bind-mounting the host's socket:
```bash
docker run -e GITEA_INSTANCE_URL=https://your_gitea.com -e GITEA_RUNNER_REGISTRATION_TOKEN=<your_token> \
-v /var/run/docker.sock:/var/run/docker.sock --name my_runner gitea/runner:latest
```
`tini` is the entrypoint (it reaps zombie processes), and it just runs [`scripts/run.sh`](scripts/run.sh), which registers the runner on first start and then execs `gitea-runner daemon`. This flavour does not need `--privileged`. The trade-off is that jobs share the host's daemon, so they can see other containers and images on that daemon.
#### `latest-dind` — Docker-in-Docker
This flavour is based on the official `docker:dind` image and bundles its own Docker daemon, so it needs no external socket — only the `--privileged` flag that Docker-in-Docker requires:
The recommended Docker-in-Docker image runs its own Docker daemon for jobs. Jobs never get the host's Docker socket, which is equivalent to root access on the host.
```bash
docker run --privileged -e GITEA_INSTANCE_URL=https://your_gitea.com -e GITEA_RUNNER_REGISTRATION_TOKEN=<your_token> \
--name my_runner gitea/runner:latest-dind
-v "$PWD/data:/data" -v runner-docker:/var/lib/docker --name my_runner gitea/runner:nightly-dind
```
Two processes have to run side by side here (the Docker daemon and the runner), so the entrypoint is the [`s6`](https://skarnet.org/software/s6/) supervision tree under [`scripts/s6`](scripts/s6) instead of `tini`. `s6` starts `dockerd`, and the runner service waits for the daemon to come up (`s6-svwait`) before launching [`run.sh`](scripts/run.sh). Each container has a private daemon isolated from the host's, at the cost of running privileged.
The volumes keep the runner's registration and the job image cache across container recreation.
#### `latest-dind-rootless` — rootless Docker-in-Docker
### Image flavours
Same idea as `dind`, but built on `docker:dind-rootless` so the bundled daemon and the runner run as an unprivileged user (`rootless`, UID 1000) rather than `root`. `DOCKER_HOST` is preset to `unix:///run/user/1000/docker.sock` so the runner talks to the rootless daemon. This reduces the blast radius compared to the privileged `dind` flavour, but rootless Docker carries the usual rootless limitations (networking, cgroups, storage drivers, and some operations that need additional host configuration such as `/etc/subuid` / `/etc/subgid` mappings and unprivileged user-namespace support).
| Tag | Docker daemon for jobs |
| --- | --- |
| `latest-dind` (recommended) | Bundled, needs `--privileged`. |
| `latest-dind-rootless` | Bundled, runs as a non-root user (UID 1000) for stronger isolation. Needs `--privileged` and [rootless Docker](https://docs.docker.com/engine/security/rootless/) support on the host. Keeps its image cache in `/home/rootless/.local/share/docker`. |
| `latest` | None. Mount the host's socket with `-v /var/run/docker.sock:/var/run/docker.sock`, or set `DOCKER_HOST`. |
> **The UID is fixed at 1000.** It comes from the `rootless` user baked into the upstream `docker:dind-rootless` base image, and the bundled daemon always listens on `/run/user/1000/docker.sock` inside the container, so running this flavour as a different user (`--user 1001`) does not work. If you need the runner to talk to a *host* rootless daemon that runs under some other UID, use the `basic` flavour instead and bind-mount that daemon's socket (see [examples/vm/rootless-docker.md](examples/vm/rootless-docker.md)); pointing `DOCKER_HOST` at a host socket from inside `dind-rootless` will not work. Changing the UID otherwise means rebuilding the image from a base with a different `rootless` user.
> **Note on Podman:** these images target the Docker daemon. The bundled `dind`/`dind-rootless` daemons are `dockerd`, not Podman, and the `basic` flavour expects a Docker-compatible socket. Running them under rootless Podman is not a supported configuration, though pointing the `basic` flavour at a Podman socket that emulates the Docker API may work for some workloads.
Versioned and `nightly` tags carry the same suffixes, for example `nightly-dind`. The images target Docker, not Podman.
### Configuration
@@ -186,7 +169,7 @@ Of the [runner images](https://gitea.com/gitea/runner-images), the `-full` flavo
#### Environment variables
Earlier releases let a few environment variables (`GITEA_DEBUG`, `GITEA_TRACE`, `GITEA_RUNNER_CAPACITY`, `GITEA_RUNNER_FILE`, `GITEA_RUNNER_ENVIRON`, `GITEA_RUNNER_ENV_FILE`) override parts of the config. They are gone, use the YAML file for all settings. The Docker images still read their own variables, such as `RUNNER_STATE_FILE`, see [scripts/run.sh](scripts/run.sh) and the container documentation below.
Earlier releases let a few environment variables (`GITEA_DEBUG`, `GITEA_TRACE`, `GITEA_RUNNER_CAPACITY`, `GITEA_RUNNER_FILE`, `GITEA_RUNNER_ENVIRON`, `GITEA_RUNNER_ENV_FILE`) override parts of the config. They are gone, use the YAML file for all settings. Config values can read the runner's environment as `${NAME}`, for example `capacity: ${RUNNER_CAPACITY}`. An unset `NAME` fails the load, and `$${NAME}` is a literal `${NAME}`. The Docker images still read their own variables, such as `RUNNER_STATE_FILE`, see [scripts/run.sh](scripts/run.sh) and the container documentation below.
### Labels
@@ -201,10 +184,10 @@ A label is written as:
| Part | Meaning |
| --- | --- |
| `name` | The name a workflow refers to in `runs-on`, e.g. `ubuntu-latest`. |
| `schema` | Either `docker` or `host`. Defaults to `host` when omitted. |
| `args` | Only used by the `docker` schema: the image to run the job in. |
| `schema` | `docker`, `kubernetes` or `host`. Defaults to `host` when omitted. |
| `args` | Only used by the `docker` and `kubernetes` schemas: the image to run the job in. |
Two schemas are supported:
Three schemas are supported:
- **`docker://<image>`** — the job runs inside a container created from `<image>`:
@@ -212,6 +195,12 @@ Two schemas are supported:
ubuntu-latest:docker://docker.gitea.com/runner-images:ubuntu-latest
```
- **`kubernetes://<image>`** — the job runs as a Kubernetes pod created from `<image>`, see **[docs/kubernetes.md](docs/kubernetes.md)**:
```text
ubuntu-latest:kubernetes://docker.gitea.com/runner-images:ubuntu-latest
```
- **`host`** — the job's steps run directly on the machine the runner is on, using the tools installed there:
```text
@@ -226,9 +215,9 @@ ubuntu-latest:docker://docker.gitea.com/runner-images:ubuntu-latest,macos:host
a workflow with `runs-on: ubuntu-latest` is executed in the `runner-images:ubuntu-latest` container, and one with `runs-on: macos` is executed directly on the host.
Names may themselves contain a colon (for example `pool:e57e18d4-10d4-406f-93bf-60f127221bdd`); only `host` and `docker` are treated as schemas.
Names may themselves contain a colon, for example `pool:e57e18d4-10d4-406f-93bf-60f127221bdd`. Only `host`, `docker` and `kubernetes` are treated as schemas.
If a job's `runs-on` matches none of the runner's labels, or sets no `runs-on` at all, it still runs: in `runner.default_image` where docker is available, on the host where it is not. Images maintained for this purpose are listed at [gitea/runner-images](https://gitea.com/gitea/runner-images).
Jobs run on `runner.default_image` when no runner label matches their `runs-on`: as a pod on runners with `kubernetes` labels, otherwise in a container, or on the host when docker is unavailable. Suitable images are listed at [gitea/runner-images](https://gitea.com/gitea/runner-images).
Labels are chosen at registration time (`--labels`, or the interactive prompt) and can be changed afterwards by editing `runner.labels` in the config file, or in the Gitea UI under the runner's settings.
@@ -275,7 +264,7 @@ volumes:
- ${GITEA_DOCKER_WORKSPACE:-.}/data:/app/data
```
Linux container jobs use a Docker proxy when socket sharing and permissions allow it. Other setups use the daemon socket directly. The proxy removes containers, networks and volumes after post steps and the completed hook. Named volumes created through it are job-scoped. To retain resources, mount the daemon socket explicitly in `container.options`. Host jobs use their existing Docker access, where `.` works.
Linux container jobs use a Docker proxy when socket sharing and permissions allow it. Other setups use the daemon socket directly. Host jobs use their existing Docker access, where `.` works. The proxy removes the containers, networks and volumes a job creates, `docker compose` services included, after post steps and the completed hook. Set `container.sweep: false` when jobs deploy services to the runner's Docker daemon.
#### Proxy
@@ -298,7 +287,7 @@ These hosts are added to `no_proxy` for jobs, so they are always reached directl
Gitea is not added. Add it to `no_proxy` yourself if it should be reached directly.
To change a value for one job, set it in a step's `env:` or in the job's `container.env`. Setting it at workflow or job level has no effect. To change it for the whole runner, set it in `runner.envs`. A `no_proxy` set there is added to the list above instead of replacing it.
For one step, set it in the step's `env:`, for the rest of a job, write it to `$GITHUB_ENV`. Setting it at workflow or job level or in `container.env` has no effect. To change it for the whole runner, set it in `runner.envs`. A `no_proxy` set there is added to the list above instead of replacing it.
Images are pulled by the Docker daemon, which needs its own proxy setting. In the `dind` images the daemon runs in the same container and reads the variables above. For any other daemon, see [the Docker documentation](https://docs.docker.com/engine/daemon/proxy/). The runner logs a warning at startup if it has a proxy and the daemon does not.
@@ -379,7 +368,8 @@ While the runner is idle it cleans up after earlier jobs:
- when `container.bind_workdir` is enabled, stale task workspace directories older than `runner.workdir_cleanup_age` are removed (default: `24h`; set `0` to disable)
- only purely numeric subdirectories under `container.workdir_parent` are treated as task workspaces and may be removed
- cleanup assumes `container.workdir_parent` is not shared across multiple runners
- on runners that use docker, per-job networks left behind by jobs the runner did not live to tear down are removed, identified by the `com.gitea.runner.uuid` label carrying this runner's uuid
- on runners that use docker, per-job networks and volumes left behind by jobs the runner did not live to tear down are removed once unused and 10 minutes old, identified by the `com.gitea.runner.uuid` label carrying this runner's uuid
- on runners with `kubernetes` labels, job pods with that label older than `runner.workdir_cleanup_age` are removed the same way, together with their Secrets
- cleanup runs every `runner.idle_cleanup_interval` (default: `10m`; set `0` to disable), and setting either knob to `0` disables all of the above
#### Post-task script (`runner.post_task_script`)
@@ -404,7 +394,7 @@ See **[docs/job-hooks.md](docs/job-hooks.md)** for the execution order, environm
#### OpenTelemetry (`OTEL_*`)
Set `OTEL_EXPORTER_OTLP_ENDPOINT` in the runner's environment to send a trace of every job to an OpenTelemetry collector. See **[docs/telemetry.md](docs/telemetry.md)**.
Set `OTEL_EXPORTER_OTLP_ENDPOINT` in the runner's environment to send a trace of every job, and the runner's metrics, to an OpenTelemetry collector. See **[docs/telemetry.md](docs/telemetry.md)**.
#### Local job logs (`log.job.dir`)

View File

@@ -15,7 +15,6 @@ import (
"encoding/json/v2"
"errors"
"fmt"
"io"
"net"
"net/http"
"net/http/httputil"
@@ -30,10 +29,10 @@ import (
"time"
"gitea.com/gitea/runner/act/common"
"gitea.com/gitea/runner/act/common/logging"
"gitea.com/gitea/runner/internal/pkg/disk"
"github.com/julienschmidt/httprouter"
"github.com/sirupsen/logrus"
"github.com/timshannon/bolthold"
"go.etcd.io/bbolt"
)
@@ -85,7 +84,7 @@ type Handler struct {
listener net.Listener
port int
server *http.Server
logger logrus.FieldLogger
logger *logging.Logger
gcing atomic.Bool
gcAt time.Time
@@ -133,7 +132,7 @@ type Options struct {
InternalSecret string
Policy Policy
Logger logrus.FieldLogger
Logger *logging.Logger
}
// StartHandler starts an embedded cache or forwards cache requests to Upstream.
@@ -147,11 +146,9 @@ func StartHandler(opts Options) (*Handler, error) {
}
if logger == nil {
discard := logrus.New()
discard.Out = io.Discard
logger = discard
logger = logging.New(logging.DiscardHandler)
}
logger = logger.WithField("module", "artifactcache")
logger = logger.With("module", "artifactcache")
h.logger = logger
if opts.OutboundIP != "" {

View File

@@ -230,7 +230,8 @@ func Serve(ctx context.Context, artifactPath, addr, port string) context.CancelF
go func() {
logger.Infof("Start server on http://%s", server.Addr)
if err := server.ListenAndServe(); err != nil && err != http.ErrServerClosed {
logger.Fatal(err)
logger.Error(err)
os.Exit(1)
}
}()

View File

@@ -4,15 +4,15 @@
package common
import (
"context"
"errors"
"testing"
"github.com/sirupsen/logrus"
"gitea.com/gitea/runner/act/common/logging"
"gitea.com/gitea/runner/act/common/logging/logtest"
)
func TestDryrunContext(t *testing.T) {
ctx := context.Background()
ctx := t.Context()
if Dryrun(ctx) {
t.Fatal("plain context should not be dryrun")
}
@@ -25,7 +25,7 @@ func TestDryrunContext(t *testing.T) {
}
func TestJobErrorContainer(t *testing.T) {
ctx := context.Background()
ctx := t.Context()
err := errors.New("job failed")
SetJobError(ctx, err)
@@ -40,34 +40,27 @@ func TestJobErrorContainer(t *testing.T) {
}
}
func TestLoggerAndHookContext(t *testing.T) {
ctx := context.Background()
if Logger(ctx) != logrus.StandardLogger() {
t.Fatal("plain context should use standard logger")
func TestLoggerAndHandlerContext(t *testing.T) {
ctx := t.Context()
if Logger(ctx) != logging.Default() {
t.Fatal("plain context should use the default logger")
}
if LoggerHook(ctx) != nil {
t.Fatal("plain context should not have a logger hook")
if LoggerHandler(ctx) != nil {
t.Fatal("plain context should not have a logger handler")
}
if StepIDs(ctx) != nil {
t.Fatal("plain context should not have step IDs")
}
logger := logrus.New()
logger := logging.New(logtest.New())
ctx = WithLogger(ctx, logger)
if Logger(ctx) != logger {
t.Fatal("WithLogger should set logger")
}
hook := testHook{}
ctx = WithLoggerHook(ctx, hook)
if LoggerHook(ctx) != hook {
t.Fatal("WithLoggerHook should set hook")
handler := logtest.New()
ctx = WithLoggerHandler(ctx, handler)
if LoggerHandler(ctx) != handler {
t.Fatal("WithLoggerHandler should set handler")
}
}
type testHook struct{}
func (testHook) Levels() []logrus.Level {
return logrus.AllLevels
}
func (testHook) Fire(*logrus.Entry) error {
return nil
}

View File

@@ -9,7 +9,7 @@ import (
"fmt"
"runtime/debug"
log "github.com/sirupsen/logrus"
"gitea.com/gitea/runner/act/common/logging"
)
// Executor define contract for the steps of a workflow
@@ -74,33 +74,33 @@ func NewParallelExecutor(parallel int, executors ...Executor) Executor {
errs := make(chan error, len(executors))
if 1 > parallel {
log.Debugf("Parallel tasks (%d) below minimum, setting to 1", parallel)
logging.Debugf("Parallel tasks (%d) below minimum, setting to 1", parallel)
parallel = 1
}
log.Infof("NewParallelExecutor: Creating %d workers for %d executors", parallel, len(executors))
logging.Infof("NewParallelExecutor: Creating %d workers for %d executors", parallel, len(executors))
for i := 0; i < parallel; i++ {
go func(workerID int, work <-chan Executor, errs chan<- error) {
log.Debugf("Worker %d started", workerID)
logging.Debugf("Worker %d started", workerID)
taskCount := 0
for executor := range work {
taskCount++
log.Debugf("Worker %d executing task %d", workerID, taskCount)
logging.Debugf("Worker %d executing task %d", workerID, taskCount)
// Recover from panics in executors to avoid crashing the worker
// goroutine which would leave the runner process hung.
// https://gitea.com/gitea/runner/issues/371
errs <- func() (err error) {
defer func() {
if r := recover(); r != nil {
log.Errorf("panic in executor: %v\n%s", r, debug.Stack())
logging.Errorf("panic in executor: %v\n%s", r, debug.Stack())
err = fmt.Errorf("panic: %v", r)
}
}()
return executor(ctx)
}()
}
log.Debugf("Worker %d finished (%d tasks executed)", workerID, taskCount)
logging.Debugf("Worker %d finished (%d tasks executed)", workerID, taskCount)
}(i, work, errs)
}

View File

@@ -20,7 +20,7 @@ import (
func TestNewWorkflow(t *testing.T) {
assert := assert.New(t)
ctx := context.Background()
ctx := t.Context()
// empty
emptyWorkflow := NewPipelineExecutor()
@@ -72,7 +72,7 @@ func concurrencyProbe(wantActive int32) (exec Executor, count, maxActive *atomic
}
func TestNewParallelExecutor(t *testing.T) {
ctx := context.Background()
ctx := t.Context()
exec, count, maxActive := concurrencyProbe(2)
require.NoError(t, NewParallelExecutor(2, exec, exec, exec)(ctx))
@@ -89,10 +89,10 @@ func TestNewParallelExecutor(t *testing.T) {
func TestNewParallelExecutorEmpty(t *testing.T) {
assert := assert.New(t)
ctx := context.Background()
ctx := t.Context()
require.NoError(t, NewParallelExecutor(2)(ctx))
canceledCtx, cancel := context.WithCancel(context.Background())
canceledCtx, cancel := context.WithCancel(t.Context())
cancel()
err := NewParallelExecutor(2)(canceledCtx)
@@ -102,7 +102,7 @@ func TestNewParallelExecutorEmpty(t *testing.T) {
func TestNewParallelExecutorFailed(t *testing.T) {
assert := assert.New(t)
ctx, cancel := context.WithCancel(context.Background())
ctx, cancel := context.WithCancel(t.Context())
cancel()
count := 0
@@ -118,7 +118,7 @@ func TestNewParallelExecutorFailed(t *testing.T) {
func TestNewParallelExecutorCanceled(t *testing.T) {
assert := assert.New(t)
ctx, cancel := context.WithCancel(context.Background())
ctx, cancel := context.WithCancel(t.Context())
cancel()
errExpected := errors.New("fake error")
@@ -150,12 +150,12 @@ func TestNewParallelExecutorRunsRemainingAfterFailure(t *testing.T) {
}
}
require.Error(t, NewParallelExecutor(2, executors...)(context.Background()))
require.Error(t, NewParallelExecutor(2, executors...)(t.Context()))
assert.Equal(t, int32(4), successCount.Load(), "a failing executor must not stop the others")
}
func TestExecutorConditionalsAndFinally(t *testing.T) {
ctx := context.Background()
ctx := t.Context()
var calls []string
record := func(name string) Executor {
return func(ctx context.Context) error {
@@ -180,7 +180,7 @@ func TestExecutorFinallyReturnsFinallyErrorWithOriginal(t *testing.T) {
mainErr := errors.New("main failed")
finalErr := errors.New("cleanup failed")
err := NewErrorExecutor(mainErr).Finally(NewErrorExecutor(finalErr))(context.Background())
err := NewErrorExecutor(mainErr).Finally(NewErrorExecutor(finalErr))(t.Context())
require.Error(t, err)
if !strings.Contains(err.Error(), "cleanup failed") || !strings.Contains(err.Error(), "main failed") {
t.Fatalf("finally error = %q, want both cleanup and original error", err)

View File

@@ -0,0 +1,238 @@
// Copyright 2026 The Gitea Authors. All rights reserved.
// SPDX-License-Identifier: MIT
package git
import (
"archive/tar"
"bytes"
"compress/gzip"
"context"
"errors"
"io"
"math/rand/v2"
"net"
"net/http"
"net/url"
"os"
"path/filepath"
"runtime"
"strconv"
"strings"
"time"
"gitea.com/gitea/runner/act/common"
)
var archiveRetryDelay = func(retryAfter string) time.Duration { // as actions/runner backs off
if seconds, err := strconv.Atoi(retryAfter); err == nil {
return min(max(time.Duration(seconds)*time.Second, 10*time.Second), 10*time.Minute)
}
return time.Duration(10+rand.IntN(21)) * time.Second
}
func FetchAction(ctx context.Context, input NewGitCloneExecutorInput) (string, string, error) { // input.Dir is one clone URL and ref's, holding a directory per commit
unlock, err := AcquireCloneLock(ctx, input.Dir)
if err != nil {
return "", "", err
}
defer unlock()
if err := os.MkdirAll(input.Dir, 0o755); err != nil {
return "", "", err
}
tree := func(sha string) string { return filepath.Join(input.Dir, sha) }
record := filepath.Join(input.Dir, "ref")
data, _ := os.ReadFile(record)
sha, etag, _ := strings.Cut(string(data), "\n")
if !isFullObjectID(sha) || !exists(tree(sha)) {
sha, etag = "", ""
} else if input.OfflineMode || isFullObjectID(input.Ref) {
return tree(sha), sha, nil
}
newSha, newEtag, err := downloadArchive(ctx, input, sha, etag)
if err != nil && ctx.Err() == nil {
common.Logger(ctx).Debugf("Cloning %s@%s: %v", input.URL, input.Ref, err)
newEtag = ""
newSha, err = exportClone(ctx, input)
}
if err != nil {
return "", "", err
}
if newSha != sha || newEtag != etag {
if err := os.WriteFile(record, []byte(newSha+"\n"+newEtag), 0o600); err != nil {
return "", "", err
}
}
return tree(newSha), newSha, nil
}
func downloadArchive(ctx context.Context, input NewGitCloneExecutorInput, sha, etag string) (string, string, error) {
rewritten, err := gitOutput(ctx, "", "ls-remote", "--get-url", input.URL)
if err != nil {
return "", "", err
}
source, err := url.Parse(rewritten)
if err != nil || source.Scheme != "http" && source.Scheme != "https" || hexRefRegex.MatchString(input.Ref) && !isFullObjectID(input.Ref) { // only git tells a short commit from a branch
return "", "", errors.New("no archive")
}
archive := source.JoinPath("archive", input.Ref+".tar.gz")
if source.Host == "github.com" {
source.Host = "codeload.github.com"
archive = source.JoinPath("tar.gz", input.Ref)
}
client := &http.Client{Transport: input.Transport, Timeout: 5 * time.Minute, CheckRedirect: func(request *http.Request, via []*http.Request) error {
if len(via) >= 10 {
return errors.New("stopped after 10 redirects")
}
if request.URL.Host != via[0].URL.Host || request.URL.Scheme != via[0].URL.Scheme { // the token stays on its origin
request.Header.Del("Authorization")
}
return nil
}}
for attempt := 1; ; attempt++ {
newSha, newEtag, retryAfter, err := downloadArchiveOnce(ctx, client, archive.String(), input.Token, sha, etag, input.Dir)
if retryAfter == 0 || attempt == 3 || ctx.Err() != nil {
return newSha, newEtag, err
}
common.Logger(ctx).Warnf("Failed to download action '%s'. Error: %v. Back off %s before retry.", archive.Redacted(), err, retryAfter)
select {
case <-ctx.Done():
return "", "", ctx.Err()
case <-time.After(retryAfter):
}
}
}
func downloadArchiveOnce(ctx context.Context, client *http.Client, archive, token, sha, etag, dir string) (string, string, time.Duration, error) {
request, err := http.NewRequestWithContext(ctx, http.MethodGet, archive, nil)
if err != nil {
return "", "", 0, err
}
if etag != "" {
request.Header.Set("If-None-Match", etag)
}
if token != "" {
request.SetBasicAuth("token", token)
}
response, err := client.Do(request)
if err != nil {
var opErr *net.OpError
if errors.As(err, &opErr) && opErr.Op == "dial" {
return "", "", 0, err
}
return "", "", archiveRetryDelay(""), err
}
defer response.Body.Close()
switch status := response.StatusCode; {
case status == http.StatusNotModified && sha != "":
return sha, etag, 0, nil
case status == http.StatusOK:
newEtag := response.Header.Get("ETag")
gz, err := gzip.NewReader(response.Body)
if err == nil {
sha, err = extractTree(gz, dir, !strings.HasSuffix(newEtag, `-false"`)) // Gitea's mark of an unprefixed archive
}
var netErr net.Error
if errors.Is(err, io.ErrUnexpectedEOF) || errors.As(err, &netErr) {
return "", "", archiveRetryDelay(""), err
}
return sha, newEtag, 0, err
case status == http.StatusRequestTimeout || status == http.StatusTooManyRequests || status >= http.StatusInternalServerError:
return "", "", archiveRetryDelay(response.Header.Get("Retry-After")), errors.New(response.Status)
default:
return "", "", 0, errors.New(response.Status)
}
}
func exportClone(ctx context.Context, input NewGitCloneExecutorInput) (string, error) {
clone := input
clone.Dir = filepath.Join(input.Dir, "git")
sha, _, err := resolveClone(ctx, clone)
if err != nil || exists(filepath.Join(input.Dir, sha)) {
return sha, err
}
archive, err := runGit(ctx, clone.Dir, nil, "archive", "--format=tar", sha)
if err != nil {
return "", err
}
_, err = extractTree(bytes.NewReader(archive), input.Dir, false)
return sha, err
}
func extractTree(archive io.Reader, dir string, strip bool) (string, error) { // into dir/<commit of the pax header>, strip dropping the directory wrapping the files
temp, err := os.MkdirTemp(dir, "tmp-")
if err != nil {
return "", err
}
defer os.RemoveAll(temp)
root, err := os.OpenRoot(temp)
if err != nil {
return "", err
}
defer root.Close()
sha := ""
reader := tar.NewReader(archive)
for {
header, err := reader.Next()
if errors.Is(err, io.EOF) {
break
}
if err != nil {
return "", err
}
if header.Typeflag == tar.TypeXGlobalHeader {
if sha = header.PAXRecords["comment"]; isFullObjectID(sha) && exists(filepath.Join(dir, sha)) {
return sha, nil
}
continue
}
name := header.Name
if strip {
_, name, _ = strings.Cut(name, "/")
}
if name = strings.TrimSuffix(name, "/"); name == "" {
continue
}
switch header.Typeflag {
case tar.TypeDir:
err = root.MkdirAll(name, 0o755)
case tar.TypeReg:
err = writeTreeFile(root, name, header.Mode, reader)
case tar.TypeSymlink:
if err = root.Symlink(header.Linkname, name); err != nil && runtime.GOOS == "windows" {
err = writeTreeFile(root, name, 0, strings.NewReader(header.Linkname))
}
}
if err != nil {
return "", err
}
}
if !isFullObjectID(sha) {
return "", errors.New("the archive names no commit")
}
if err := root.Close(); err != nil {
return "", err
}
return sha, os.Rename(temp, filepath.Join(dir, sha))
}
func writeTreeFile(root *os.Root, name string, mode int64, content io.Reader) error {
perm := os.FileMode(0o644)
if mode&0o111 != 0 {
perm = 0o755
}
file, err := root.OpenFile(name, os.O_CREATE|os.O_EXCL|os.O_WRONLY, perm)
if err != nil {
return err
}
_, err = io.Copy(file, content)
return errors.Join(err, file.Close())
}
func exists(name string) bool {
_, err := os.Stat(name)
return err == nil
}

View File

@@ -0,0 +1,225 @@
// Copyright 2026 The Gitea Authors. All rights reserved.
// SPDX-License-Identifier: MIT
package git
import (
"archive/tar"
"bytes"
"encoding/base64"
"fmt"
"net/http"
"net/http/cgi"
"net/http/httptest"
"os"
"os/exec"
"path/filepath"
"strings"
"sync"
"testing"
"time"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
func TestFetchAction(t *testing.T) {
original := archiveRetryDelay
archiveRetryDelay = func(_retryAfter string) time.Duration { return time.Nanosecond }
t.Cleanup(func() { archiveRetryDelay = original })
root := t.TempDir()
remote := filepath.Join(root, "org", "action")
require.NoError(t, gitCmd("init", "--initial-branch=main", remote))
require.NoError(t, os.WriteFile(filepath.Join(remote, "action.yml"), []byte("name: action\n"), 0o644))
require.NoError(t, os.WriteFile(filepath.Join(remote, "run.sh"), []byte("#!/bin/sh\n"), 0o755))
require.NoError(t, os.Symlink("action.yml", filepath.Join(remote, "link")))
require.NoError(t, gitCmd("-C", remote, "add", "."))
require.NoError(t, gitCmd("-C", remote, "commit", "-m", "initial"))
require.NoError(t, gitCmd("-C", remote, "tag", "v1"))
gitPath, err := exec.LookPath("git")
require.NoError(t, err)
backend := &cgi.Handler{Path: gitPath, Args: []string{"http-backend"}, Env: []string{"GIT_PROJECT_ROOT=" + root, "GIT_HTTP_EXPORT_ALL=1"}}
var mu sync.Mutex
var archiveRequests []http.Header
var mirror *httptest.Server
requests, archives, failures := 0, "", 0
handler := http.HandlerFunc(func(writer http.ResponseWriter, request *http.Request) {
mu.Lock()
defer mu.Unlock()
requests++
ref, ok := strings.CutPrefix(request.URL.Path, "/org/action/archive/")
if !ok {
backend.ServeHTTP(writer, request)
return
}
archiveRequests = append(archiveRequests, request.Header.Clone())
switch {
case failures > 0:
failures--
writer.WriteHeader(http.StatusServiceUnavailable)
return
case archives == "none":
http.NotFound(writer, request)
return
case archives == "unreachable":
http.Redirect(writer, request, "http://127.0.0.1:1"+request.URL.Path, http.StatusFound)
return
case archives == "redirect" && request.Host != mirror.Listener.Addr().String():
http.Redirect(writer, request, mirror.URL+request.URL.Path, http.StatusFound)
return
}
sha, _ := gitOutput(request.Context(), remote, "rev-parse", strings.TrimSuffix(ref, ".tar.gz"))
etag := fmt.Sprintf(`W/"%s-tar.gz-%t"`, sha, archives != "unprefixed")
if request.Header.Get("If-None-Match") == etag {
writer.WriteHeader(http.StatusNotModified)
return
}
args := []string{"archive", "--format=tar.gz", sha}
if archives != "unprefixed" {
args = append(args, "--prefix=action/")
}
archive, _ := runGit(request.Context(), remote, nil, args...)
writer.Header().Set("ETag", etag)
_, _ = writer.Write(archive)
})
server := httptest.NewServer(handler)
mirror = httptest.NewServer(handler)
t.Cleanup(server.Close)
t.Cleanup(mirror.Close)
serve := func(mode string, failing int) {
mu.Lock()
defer mu.Unlock()
requests, archives, failures, archiveRequests = 0, mode, failing, nil
}
served := func() (int, []http.Header) {
mu.Lock()
defer mu.Unlock()
return requests, append([]http.Header(nil), archiveRequests...)
}
newInput := func(ref string) NewGitCloneExecutorInput {
return NewGitCloneExecutorInput{URL: server.URL + "/org/action", Ref: ref, Dir: t.TempDir(), Token: "secret"}
}
t.Run("downloads, revalidates and follows a moved tag", func(t *testing.T) {
serve("prefixed", 0)
input := newInput("v1")
tree, sha, err := FetchAction(t.Context(), input)
require.NoError(t, err)
assert.Equal(t, gitRevParse(t, remote, "v1"), sha)
assert.Equal(t, filepath.Join(input.Dir, sha), tree)
assert.NoDirExists(t, filepath.Join(tree, ".git"))
info, err := os.Stat(filepath.Join(tree, "run.sh"))
require.NoError(t, err)
assert.Equal(t, os.FileMode(0o755), info.Mode().Perm())
link, err := os.Readlink(filepath.Join(tree, "link"))
require.NoError(t, err)
assert.Equal(t, "action.yml", link)
again, _, err := FetchAction(t.Context(), input)
require.NoError(t, err)
assert.Equal(t, tree, again)
require.NoError(t, gitCmd("-C", remote, "commit", "--allow-empty", "-m", "moved"))
require.NoError(t, gitCmd("-C", remote, "tag", "--force", "v1"))
moved, movedSha, err := FetchAction(t.Context(), input)
require.NoError(t, err)
assert.Equal(t, gitRevParse(t, remote, "v1"), movedSha)
assert.NotEqual(t, tree, moved)
assert.FileExists(t, filepath.Join(tree, "action.yml"))
input.OfflineMode = true
_, offlineSha, err := FetchAction(t.Context(), input)
require.NoError(t, err)
assert.Equal(t, movedSha, offlineSha)
requests, headers := served()
assert.Equal(t, 3, requests)
require.Len(t, headers, 3)
assert.Equal(t, "Basic "+base64.StdEncoding.EncodeToString([]byte("token:secret")), headers[0].Get("Authorization"))
assert.Empty(t, headers[0].Get("If-None-Match"))
assert.Equal(t, fmt.Sprintf(`W/"%s-tar.gz-true"`, sha), headers[1].Get("If-None-Match"))
assert.Equal(t, headers[1].Get("If-None-Match"), headers[2].Get("If-None-Match"))
})
t.Run("downloads a pinned commit once and rejects a short one", func(t *testing.T) {
serve("prefixed", 0)
sha := gitRevParse(t, remote, "main")
input := newInput(sha)
for range 2 {
tree, got, err := FetchAction(t.Context(), input)
require.NoError(t, err)
assert.Equal(t, sha, got)
assert.FileExists(t, filepath.Join(tree, "action.yml"))
}
_, _, err := FetchAction(t.Context(), newInput(sha[:7]))
require.ErrorIs(t, err, ErrShortRef)
_, headers := served()
assert.Len(t, headers, 1)
assert.NoDirExists(t, filepath.Join(input.Dir, "git"))
})
t.Run("retries a failing download and drops the token on a redirect to another origin", func(t *testing.T) {
serve("redirect", 2)
input := newInput("main")
_, sha, err := FetchAction(t.Context(), input)
require.NoError(t, err)
assert.Equal(t, gitRevParse(t, remote, "main"), sha)
assert.NoDirExists(t, filepath.Join(input.Dir, "git"))
_, headers := served()
require.Len(t, headers, 4)
assert.NotEmpty(t, headers[2].Get("Authorization"))
assert.Empty(t, headers[3].Get("Authorization"))
})
t.Run("keeps an unprefixed archive whole, downloads where insteadOf points and exports a clone without an archive or from an unreachable host", func(t *testing.T) {
for _, mode := range []string{"unprefixed", "none", "unreachable"} {
serve(mode, 0)
tree, sha, err := FetchAction(t.Context(), newInput("main"))
require.NoError(t, err, mode)
assert.Equal(t, gitRevParse(t, remote, "main"), sha, mode)
assert.FileExists(t, filepath.Join(tree, "action.yml"), mode)
assert.NoDirExists(t, filepath.Join(tree, ".git"), mode)
_, headers := served()
assert.Len(t, headers, 1, mode)
}
serve("prefixed", 0)
config := filepath.Join(t.TempDir(), "gitconfig")
require.NoError(t, os.WriteFile(config, []byte("[url \""+server.URL+"/org/\"]\n\tinsteadOf = https://example.invalid/org/\n"), 0o644))
t.Setenv("GIT_CONFIG_GLOBAL", config)
input := newInput("main")
input.URL = "https://example.invalid/org/action"
_, sha, err := FetchAction(t.Context(), input)
require.NoError(t, err)
assert.Equal(t, gitRevParse(t, remote, "main"), sha)
_, headers := served()
assert.Len(t, headers, 1)
assert.NoDirExists(t, filepath.Join(input.Dir, "git"))
})
}
func TestExtractTreeConfinesEntries(t *testing.T) {
outside := t.TempDir()
for name, entries := range map[string][]*tar.Header{
"parent path": {{Name: "../escape", Typeflag: tar.TypeReg}},
"through a symlink": {{Name: "out", Typeflag: tar.TypeSymlink, Linkname: outside}, {Name: "out/escape", Typeflag: tar.TypeReg}},
} {
t.Run(name, func(t *testing.T) {
var archive bytes.Buffer
writer := tar.NewWriter(&archive)
require.NoError(t, writer.WriteHeader(&tar.Header{Typeflag: tar.TypeXGlobalHeader, PAXRecords: map[string]string{"comment": strings.Repeat("a", 40)}}))
for _, entry := range entries {
require.NoError(t, writer.WriteHeader(entry))
}
require.NoError(t, writer.Close())
dir := t.TempDir()
_, err := extractTree(&archive, dir, false)
require.Error(t, err)
assert.NoFileExists(t, filepath.Join(dir, "escape"))
assert.NoFileExists(t, filepath.Join(outside, "escape"))
})
}
}

View File

@@ -6,44 +6,67 @@ package git
import (
"context"
"crypto/sha256"
"encoding/base64"
"errors"
"fmt"
"io"
"net/http"
"net/url"
"os"
"os/exec"
"path"
"path/filepath"
"regexp"
"runtime"
"slices"
"strconv"
"strings"
"sync"
"time"
"gitea.com/gitea/runner/act/common"
"gitea.com/gitea/runner/act/common/logging"
"gitea.com/gitea/runner/internal/pkg/lock"
"github.com/go-git/go-git/v5"
"github.com/go-git/go-git/v5/config"
"github.com/go-git/go-git/v5/plumbing"
"github.com/go-git/go-git/v5/plumbing/storer"
"github.com/go-git/go-git/v5/plumbing/transport/http"
"github.com/mattn/go-isatty"
log "github.com/sirupsen/logrus"
)
var (
codeCommitHTTPRegex = regexp.MustCompile(`^https?://git-codecommit\.(.+)\.amazonaws.com/v1/repos/(.+)$`)
codeCommitSSHRegex = regexp.MustCompile(`ssh://git-codecommit\.(.+)\.amazonaws.com/v1/repos/(.+)$`)
githubHTTPRegex = regexp.MustCompile(`^https?://.*github.com.*/(.+)/(.+?)(?:.git)?$`)
githubSSHRegex = regexp.MustCompile(`github.com[:/](.+)/(.+?)(?:.git)?$`)
hexRefRegex = regexp.MustCompile(`^[0-9a-fA-F]+$`)
LocalEnvVars = []string{ // `git rev-parse --local-env-vars`, inherited ones point git at another repository
"GIT_ALTERNATE_OBJECT_DIRECTORIES", "GIT_CONFIG", "GIT_CONFIG_PARAMETERS", "GIT_CONFIG_COUNT", "GIT_OBJECT_DIRECTORY",
"GIT_DIR", "GIT_WORK_TREE", "GIT_IMPLICIT_WORK_TREE", "GIT_GRAFT_FILE", "GIT_INDEX_FILE", "GIT_NO_REPLACE_OBJECTS",
"GIT_REPLACE_REF_BASE", "GIT_PREFIX", "GIT_SHALLOW_FILE", "GIT_COMMON_DIR",
}
cloneLocks lock.Keyed[string] // key: clone target directory
ErrShortRef = errors.New("short SHA references are not supported")
)
// AcquireCloneLock returns an unlock function after locking the per-directory mutex for dir.
// Only concurrent operations targeting the same directory are serialized; clones into different directories run in parallel.
// Callers reading files inside dir (e.g. tarring a checked-out action into a job container) must hold this lock too,
// otherwise a concurrent NewGitCloneExecutor on the same dir can mutate the worktree mid-read.
func AcquireCloneLock(dir string) func() {
return cloneLocks.Lock(dir)
// AcquireCloneLock serializes access to dir across runner processes, or within this one where no lock file can be created. Readers of dir must hold it too.
func AcquireCloneLock(ctx context.Context, dir string) (func(), error) {
dir = filepath.Clean(dir)
if runtime.GOOS == "plan9" {
return cloneLocks.Lock(dir), nil
}
if err := os.MkdirAll(filepath.Dir(dir), 0o755); err != nil {
return nil, err
}
for {
release, err := lock.TryLock(dir)
if err == nil {
return func() { _ = release() }, nil
}
if !errors.Is(err, lock.ErrLocked) {
return cloneLocks.Lock(dir), nil
}
select {
case <-ctx.Done():
return nil, ctx.Err()
case <-time.After(50 * time.Millisecond):
}
}
}
type Error struct {
@@ -51,196 +74,97 @@ type Error struct {
commit string
}
func (e *Error) Error() string {
return e.err.Error()
}
func (e *Error) Unwrap() error {
return e.err
}
func (e *Error) Commit() string {
return e.commit
}
// goGitMu serializes go-git repository access across the process. go-git is not safe for
// concurrent use of the same repository (even read access decodes packfiles into shared
// state), so parallel jobs inspecting the shared workdir repo race without this. The guarded
// operations are fast local reads; gitea runs one job per process, so the lock is effectively
// uncontended in production.
var goGitMu sync.Mutex
func (e *Error) Error() string { return e.err.Error() }
func (e *Error) Unwrap() error { return e.err }
func (e *Error) Commit() string { return e.commit }
// FindGitRevision get the current git revision
func FindGitRevision(ctx context.Context, file string) (shortSha, sha string, err error) {
goGitMu.Lock()
defer goGitMu.Unlock()
return findGitRevision(ctx, file)
}
func findGitRevision(ctx context.Context, file string) (shortSha, sha string, err error) {
func FindGitRevision(ctx context.Context, file string) (shortSHA, sha string, err error) {
logger := common.Logger(ctx)
gitDir, err := git.PlainOpenWithOptions(
file,
&git.PlainOpenOptions{
DetectDotGit: true,
EnableDotGitCommonDir: true,
},
)
sha, err = gitOutput(ctx, file, "rev-parse", "--verify", "HEAD^{commit}")
if err != nil {
logger.WithError(err).Error("path", file, "not located inside a git repository")
return "", "", err
}
head, err := gitDir.Reference(plumbing.HEAD, true)
if err != nil {
return "", "", err
if len(sha) < 7 {
return "", "", errors.New("HEAD could not be resolved")
}
if head.Hash().IsZero() {
return "", "", errors.New("HEAD sha1 could not be resolved")
}
hash := head.Hash().String()
logger.Debugf("Found revision: %s", hash)
return hash[:7], strings.TrimSpace(hash), nil
logger.Debugf("Found revision: %s", sha)
return sha[:7], sha, nil
}
// FindGitRef get the current git ref
func FindGitRef(ctx context.Context, file string) (string, error) {
goGitMu.Lock()
defer goGitMu.Unlock()
logger := common.Logger(ctx)
logger.Debugf("Loading revision from git directory")
_, ref, err := findGitRevision(ctx, file)
_, revision, err := FindGitRevision(ctx, file)
if err != nil {
return "", err
}
logger.Debugf("HEAD points to '%s'", revision)
logger.Debugf("HEAD points to '%s'", ref)
// Prefer the git library to iterate over the references and find a matching tag or branch.
refTag := ""
refBranch := ""
repo, err := git.PlainOpenWithOptions(
file,
&git.PlainOpenOptions{
DetectDotGit: true,
EnableDotGitCommonDir: true,
},
)
refs, err := gitOutput(ctx, file, "for-each-ref", "--points-at=HEAD", "--format=%(refname)", "refs/tags", "refs/heads")
if err != nil {
return "", err
}
iter, err := repo.References()
if err != nil {
return "", err
}
// find the reference that matches the revision's has
err = iter.ForEach(func(r *plumbing.Reference) error {
/* tags and branches will have the same hash
* when a user checks out a tag, it is not mentioned explicitly
* in the go-git package, we must identify the revision
* then check if any tag matches that revision,
* if so then we checked out a tag
* else we look for branches and if matches,
* it means we checked out a branch
*
* If a branches matches first we must continue and check all tags (all references)
* in case we match with a tag later in the interation
*/
if r.Hash().String() == ref {
if r.Name().IsTag() {
refTag = r.Name().String()
}
if r.Name().IsBranch() {
refBranch = r.Name().String()
}
var tag, branch string
for ref := range strings.FieldsSeq(refs) {
switch {
case strings.HasPrefix(ref, "refs/tags/"):
tag = ref
case strings.HasPrefix(ref, "refs/heads/"):
branch = ref
}
// we found what we where looking for
if refTag != "" && refBranch != "" {
return storer.ErrStop
}
return nil
})
if err != nil {
return "", err
}
// order matters here see above comment.
if refTag != "" {
return refTag, nil
if tag != "" {
return tag, nil
}
if refBranch != "" {
return refBranch, nil
if branch != "" {
return branch, nil
}
return "", fmt.Errorf("failed to identify reference (tag/branch) for the checked-out revision '%s'", ref)
return "", fmt.Errorf("failed to identify reference (tag/branch) for the checked-out revision '%s'", revision)
}
// FindGithubRepo get the repo
func FindGithubRepo(ctx context.Context, file, githubInstance string) (string, error) {
goGitMu.Lock()
defer goGitMu.Unlock()
url, err := findGitRemoteURL(ctx, file, "origin")
func FindGithubRepo(ctx context.Context, file string) (string, error) {
url, err := findGitRemoteURL(ctx, file)
if err != nil {
return "", err
}
_, slug := findGitSlug(url, githubInstance)
return slug, nil
return findGitSlug(url)
}
func findGitRemoteURL(_ context.Context, file, remoteName string) (string, error) {
repo, err := git.PlainOpenWithOptions(
file,
&git.PlainOpenOptions{
DetectDotGit: true,
EnableDotGitCommonDir: true,
},
)
func findGitRemoteURL(ctx context.Context, file string) (string, error) {
urls, err := gitOutput(ctx, file, "config", "--get-all", "remote.origin.url") // `remote get-url` applies insteadOf
if err != nil {
return "", err
}
remote, err := repo.Remote(remoteName)
if err != nil {
return "", err
if urls == "" {
return "", errors.New("remote 'origin' exists but has no URL")
}
if len(remote.Config().URLs) < 1 {
return "", fmt.Errorf("remote '%s' exists but has no URL", remoteName)
}
return remote.Config().URLs[0], nil
url, _, _ := strings.Cut(urls, "\n")
return url, nil
}
func findGitSlug(url, githubInstance string) (string, string) {
if matches := codeCommitHTTPRegex.FindStringSubmatch(url); matches != nil {
return "CodeCommit", matches[2]
} else if matches := codeCommitSSHRegex.FindStringSubmatch(url); matches != nil {
return "CodeCommit", matches[2]
} else if matches := githubHTTPRegex.FindStringSubmatch(url); matches != nil {
return "GitHub", fmt.Sprintf("%s/%s", matches[1], matches[2])
} else if matches := githubSSHRegex.FindStringSubmatch(url); matches != nil {
return "GitHub", fmt.Sprintf("%s/%s", matches[1], matches[2])
} else if githubInstance != "github.com" {
gheHTTPRegex := regexp.MustCompile(fmt.Sprintf(`^https?://%s/(.+)/(.+?)(?:.git)?$`, githubInstance))
gheSSHRegex := regexp.MustCompile(githubInstance + "[:/](.+)/(.+?)(?:.git)?$")
if matches := gheHTTPRegex.FindStringSubmatch(url); matches != nil {
return "GitHubEnterprise", fmt.Sprintf("%s/%s", matches[1], matches[2])
} else if matches := gheSSHRegex.FindStringSubmatch(url); matches != nil {
return "GitHubEnterprise", fmt.Sprintf("%s/%s", matches[1], matches[2])
}
func findGitSlug(remoteURL string) (string, error) {
if matches := codeCommitHTTPRegex.FindStringSubmatch(remoteURL); matches != nil {
return matches[2], nil
} else if matches := codeCommitSSHRegex.FindStringSubmatch(remoteURL); matches != nil {
return matches[2], nil
}
return "", url
if host, repoPath, ok := strings.Cut(remoteURL, ":"); ok && filepath.VolumeName(remoteURL) == "" && !strings.Contains(host, "/") && !strings.HasPrefix(repoPath, "//") {
remoteURL = "ssh://" + host + "/" + repoPath // scp-like [user@]host:path
}
parsed, err := url.Parse(remoteURL)
if err != nil {
return "", errors.New("cannot parse the origin remote URL")
}
owner, repo := path.Split(strings.TrimSuffix(strings.Trim(parsed.Path, "/"), ".git"))
owner = path.Base(owner)
if parsed.Host == "" || repo == "" || owner == "." || owner == "/" {
return "", errors.New("cannot determine owner and repository from the origin remote URL")
}
return owner + "/" + repo, nil
}
// NewGitCloneExecutorInput the input for the NewGitCloneExecutor
@@ -255,214 +179,179 @@ type NewGitCloneExecutorInput struct {
// 0 for full clone.
Depth int
// Quiet drops the informational clone line to debug level, for callers that log their own
// download summary (the setup section's action report).
Quiet bool
// For Gitea
InsecureSkipTLS bool
InsecureSkipTLS bool
ClientCertFile, ClientKeyFile string // PEM, absolute as git runs in other directories
Transport http.RoundTripper // for FetchAction's archive downloads, nil for http.DefaultTransport
}
// CloneIfRequired returns the repository and a boolean indicating whether an existing local clone was reused.
func CloneIfRequired(ctx context.Context, refName plumbing.ReferenceName, input NewGitCloneExecutorInput, logger log.FieldLogger) (*git.Repository, bool, error) {
r, err := git.PlainOpen(input.Dir)
// CloneIfRequired reports whether an existing local clone was reused.
func CloneIfRequired(ctx context.Context, input NewGitCloneExecutorInput, logger *logging.Logger) (bool, error) {
var origin string
_, err := os.Stat(filepath.Join(input.Dir, ".git")) // without it git would discover a repository around Dir
if err == nil {
// Verify the cached clone still points to the resolved URL before reusing it.
remote, err := r.Remote("origin")
if err == nil && len(remote.Config().URLs) > 0 && remote.Config().URLs[0] == input.URL {
// Reuse existing clone
return r, true, nil
}
switch {
case err != nil:
logger.Debugf("Removing cached clone at %s because origin cannot be read: %v", input.Dir, err)
case len(remote.Config().URLs) == 0:
logger.Debugf("Removing cached clone at %s because origin has no URL", input.Dir)
default:
logger.Debugf("Removing cached clone at %s because origin URL changed from %s to %s", input.Dir, remote.Config().URLs[0], input.URL)
}
if err := os.RemoveAll(input.Dir); err != nil {
return nil, false, fmt.Errorf("remove cached clone %s: %w", input.Dir, err)
}
origin, err = findGitRemoteURL(ctx, input.Dir)
}
var progressWriter io.Writer
if isatty.IsTerminal(os.Stdout.Fd()) || isatty.IsCygwinTerminal(os.Stdout.Fd()) {
if entry, ok := logger.(*log.Entry); ok {
progressWriter = entry.WriterLevel(log.DebugLevel)
} else if lgr, ok := logger.(*log.Logger); ok {
progressWriter = lgr.WriterLevel(log.DebugLevel)
} else {
log.Errorf("Unable to get writer from logger (type=%T)", logger)
progressWriter = os.Stdout
}
if err == nil && origin == input.URL {
return true, nil
}
cloneOptions := git.CloneOptions{
URL: input.URL,
Progress: progressWriter,
InsecureSkipTLS: input.InsecureSkipTLS, // For Gitea
if ctxErr := ctx.Err(); ctxErr != nil { // a cancelled origin lookup must not discard a valid clone
return false, ctxErr
}
if input.Token != "" {
cloneOptions.Auth = &http.BasicAuth{
Username: "token",
Password: input.Token,
}
if err == nil {
logger.Debugf("Removing cached clone at %s because origin URL changed from %s to %s", input.Dir, origin, input.URL)
} else if _, statErr := os.Stat(input.Dir); statErr == nil {
logger.Debugf("Removing cached clone at %s because origin cannot be read: %v", input.Dir, err)
}
r, err = cloneAtDepth(ctx, input, cloneOptions, logger)
if err != nil {
logger.Errorf("Unable to clone %v %s: %v", input.URL, refName, err)
return nil, false, err
if err := os.RemoveAll(input.Dir); err != nil {
return false, fmt.Errorf("remove cached clone %s: %w", input.Dir, err)
}
if err = os.Chmod(input.Dir, 0o755); err != nil {
return nil, false, err
if err := clone(ctx, input); err != nil {
return false, err
}
return r, false, nil
if err := os.Chmod(input.Dir, 0o755); err != nil {
return false, err
}
return false, nil
}
// NewGitCloneExecutor creates an executor to clone git repos
func NewGitCloneExecutor(input NewGitCloneExecutorInput) common.Executor {
return func(ctx context.Context) error {
logger := common.Logger(ctx)
if input.Quiet {
logger.Debugf("git clone '%s' # ref=%s", input.URL, input.Ref)
} else {
logger.Infof("git clone '%s' # ref=%s", input.URL, input.Ref)
}
logger.Infof("git clone '%s' # ref=%s", input.URL, input.Ref)
logger.Debugf(" cloning %s to %s", input.URL, input.Dir)
defer AcquireCloneLock(input.Dir)()
refName := plumbing.ReferenceName("refs/heads/" + input.Ref)
r, reused, err := CloneIfRequired(ctx, refName, input, logger)
unlock, err := AcquireCloneLock(ctx, input.Dir)
if err != nil {
return err
}
defer unlock()
isOfflineMode := input.OfflineMode
fetchOptions := git.FetchOptions{
RefSpecs: []config.RefSpec{"refs/*:refs/*", "HEAD:refs/heads/HEAD", "refs/heads/*:refs/remotes/origin/*"},
Force: true,
InsecureSkipTLS: input.InsecureSkipTLS,
}
if input.Token != "" {
fetchOptions.Auth = &http.BasicAuth{
Username: "token",
Password: input.Token,
}
}
// Action clones only ever need the tip commit, so keep a shallow cache cheap on update at depth 1 regardless of its original depth
// Turning action_shallow_clone off does not convert an existing shallow cache; evict it for a full clone.
if isShallow(r) {
fetchOptions.Depth = 1
if spec, ok := shallowFetchRefSpec(r, input.Ref); ok {
fetchOptions.RefSpecs = []config.RefSpec{spec}
}
}
// A just-cloned ref is as current as a fetch would make it, and a commit hash never moves.
// TODO: revalidate a mutable ref with a conditional archive request instead, once every
// supported Gitea sends an ETag for them: https://github.com/go-gitea/gitea/pull/39289
_, present := refRevision(r, input.Ref)
refresh := !isOfflineMode && (!present || (reused && !plumbing.IsHash(input.Ref)))
if refresh {
err = r.FetchContext(ctx, &fetchOptions)
if err != nil && !errors.Is(err, git.NoErrAlreadyUpToDate) {
return err
}
}
var hash *plumbing.Hash
rev := plumbing.Revision(input.Ref)
if hash, err = r.ResolveRevision(rev); err != nil {
// ResolveRevision returns a nil hash on error, and a branch ref legitimately fails
// here (no local refs/heads/<ref>); the duck-typing below resolves it.
logger.Errorf("Unable to resolve %s: %v", input.Ref, err)
} else if hash.String() != input.Ref && strings.HasPrefix(hash.String(), input.Ref) {
return &Error{
err: ErrShortRef,
commit: hash.String(),
}
}
rev, _ = refRevision(r, input.Ref)
if hash, err = r.ResolveRevision(rev); err != nil {
logger.Errorf("Unable to resolve %s: %v", input.Ref, err)
resolved, reused, err := resolveClone(ctx, input)
if err != nil {
return err
}
var w *git.Worktree
if w, err = r.Worktree(); err != nil {
return err
if _, err := runGit(ctx, input.Dir, nil, "checkout", "--force", "--detach", resolved); err != nil {
return fmt.Errorf("checkout %s: %w", resolved, err)
}
reusedMsg := ""
if isOfflineMode && reused {
if input.OfflineMode && reused {
reusedMsg = " (reused in offline mode)"
}
logger.Debugf("Cloned %s to %s%s", input.URL, input.Dir, reusedMsg)
if err = w.Checkout(&git.CheckoutOptions{
Hash: *hash,
Force: true,
}); err != nil {
logger.Errorf("Unable to checkout %s: %v", *hash, err)
return err
}
logger.Debugf("Checked out %s", input.Ref)
return nil
}
}
// cloneAtDepth falls back to a full clone when the shallow attempt fails.
func cloneAtDepth(ctx context.Context, input NewGitCloneExecutorInput, opts git.CloneOptions, logger log.FieldLogger) (*git.Repository, error) {
if input.Depth > 0 {
if plumbing.IsHash(input.Ref) {
r, err := fetchPinnedSHA(ctx, input, opts)
if err == nil {
return r, nil
}
logger.Debugf("Shallow fetch of %s at %s failed: %v", input.URL, input.Ref, err)
if err := removePartialClone(input.Dir); err != nil {
return nil, err
}
} else {
refNames := []plumbing.ReferenceName{plumbing.NewBranchReferenceName(input.Ref), plumbing.NewTagReferenceName(input.Ref)}
if input.Ref == plumbing.HEAD.String() || strings.HasPrefix(input.Ref, "refs/") {
refNames = []plumbing.ReferenceName{plumbing.ReferenceName(input.Ref)} // HEAD is the remote's default branch
}
for _, refName := range refNames {
shallowOpts := opts
shallowOpts.Depth = input.Depth
shallowOpts.SingleBranch = true
shallowOpts.ReferenceName = refName
shallowOpts.Tags = git.NoTags
func resolveClone(ctx context.Context, input NewGitCloneExecutorInput) (string, bool, error) { // the caller holds input.Dir's lock
logger := common.Logger(ctx)
reused, err := CloneIfRequired(ctx, input, logger)
if err != nil {
return "", false, err
}
resolved, err := resolveRef(ctx, input.Dir, input.Ref)
if errors.Is(err, ErrShortRef) || (err != nil && input.OfflineMode) {
return "", false, err
}
if !input.OfflineMode && (err != nil || (reused && !isFullHash(input.Ref, resolved))) {
if err := fetch(ctx, input); err != nil {
return "", false, err
}
resolved, err = resolveRef(ctx, input.Dir, input.Ref)
}
if err != nil {
logger.Errorf("Unable to resolve %s: %v", input.Ref, err)
}
return resolved, reused, err
}
r, err := git.PlainCloneContext(ctx, input.Dir, false, &shallowOpts)
if err == nil {
return r, nil
}
logger.Debugf("Shallow clone of %s as %s failed: %v", input.URL, refName, err)
if err := removePartialClone(input.Dir); err != nil {
return nil, err
}
func clone(ctx context.Context, input NewGitCloneExecutorInput) error {
if input.Depth > 0 && isFullObjectID(input.Ref) {
if err := shallowPinnedClone(ctx, input); err == nil {
return nil
} else if err := removePartialClone(input.Dir); err != nil {
return err
}
}
args := []string{"clone", "--no-checkout"}
if input.Depth > 0 {
args = append(args, "--no-local", "--depth", strconv.Itoa(input.Depth), "--no-tags")
if input.Ref != "HEAD" && !strings.HasPrefix(input.Ref, "refs/") {
args = append(args, "--branch", input.Ref)
}
}
args = append(args, input.URL, input.Dir)
if _, err := runGit(ctx, "", &input, args...); err == nil {
return nil
} else if input.Depth == 0 {
return err
}
if err := removePartialClone(input.Dir); err != nil {
return err
}
_, err := runGit(ctx, "", &input, "clone", "--no-checkout", input.URL, input.Dir)
return err
}
func shallowPinnedClone(ctx context.Context, input NewGitCloneExecutorInput) error {
if _, err := runGit(ctx, "", &input, "init", "--object-format="+ObjectFormat(input.Ref), input.Dir); err != nil {
return err
}
if _, err := runGit(ctx, input.Dir, &input, "remote", "add", "origin", input.URL); err != nil {
return err
}
_, err := runGit(ctx, input.Dir, &input, "fetch", "--depth", strconv.Itoa(input.Depth), "--no-tags", "origin", "+"+input.Ref+":refs/pinned/"+input.Ref)
return err
}
func fetch(ctx context.Context, input NewGitCloneExecutorInput) error {
args := []string{"fetch", "--force", "--no-prune"}
if _, err := os.Stat(filepath.Join(input.Dir, ".git", "shallow")); err == nil {
args = append(args, "--depth", "1", "--no-tags")
}
_, err := runGit(ctx, input.Dir, &input, append(args, "origin", "+"+input.Ref+":"+cachedRef(input.Ref))...) // git picks the remote ref, a tag before a branch
return err
}
func cachedRef(ref string) string {
return fmt.Sprintf("refs/runner/%x", sha256.Sum256([]byte(ref)))
}
func resolveRef(ctx context.Context, dir, ref string) (string, error) {
if !isFullObjectID(ref) {
for _, name := range []string{cachedRef(ref), "refs/tags/" + ref, "refs/remotes/origin/" + ref} {
if sha, err := gitOutput(ctx, dir, "rev-parse", "--verify", name+"^{commit}"); err == nil {
return sha, nil
}
}
logger.Debugf("Falling back to a full clone of %s for ref %q", input.URL, input.Ref)
}
sha, err := gitOutput(ctx, dir, "rev-parse", "--verify", ref+"^{commit}")
if err == nil && hexRefRegex.MatchString(ref) && len(sha) != len(ref) {
return "", &Error{err: ErrShortRef, commit: sha}
}
return sha, err
}
return git.PlainCloneContext(ctx, input.Dir, false, &opts)
func isFullObjectID(ref string) bool {
return ObjectFormat(ref) != ""
}
func isFullHash(ref, resolved string) bool {
return isFullObjectID(ref) && len(ref) == len(resolved)
}
// ObjectFormat returns the object format of a full commit hash, empty for anything else.
func ObjectFormat(sha string) string {
switch {
case !hexRefRegex.MatchString(sha):
return ""
case len(sha) == 40:
return "sha1"
case len(sha) == 64:
return "sha256"
}
return ""
}
func removePartialClone(dir string) error {
@@ -472,71 +361,100 @@ func removePartialClone(dir string) error {
return nil
}
func fetchPinnedSHA(ctx context.Context, input NewGitCloneExecutorInput, opts git.CloneOptions) (*git.Repository, error) {
r, err := git.PlainInit(input.Dir, false)
func gitOutput(ctx context.Context, dir string, args ...string) (string, error) {
output, err := runGit(ctx, dir, nil, args...)
return strings.TrimSpace(string(output)), err
}
func RemoteConfig(token string, insecureSkipTLS bool) []string {
var config []string
if insecureSkipTLS {
config = append(config, "http.sslVerify=false")
}
if token != "" {
config = append(config, "http.extraHeader=", "http.extraHeader=Authorization: Basic "+base64.StdEncoding.EncodeToString([]byte("token:"+token)))
}
return config
}
// ConfigEnv applies `-c` style entries through the environment, keeping secrets out of argv.
func ConfigEnv(config ...string) map[string]string {
env := map[string]string{"GIT_CONFIG_COUNT": strconv.Itoa(len(config))}
for i, entry := range config {
key, value, _ := strings.Cut(entry, "=")
env["GIT_CONFIG_KEY_"+strconv.Itoa(i)] = key
env["GIT_CONFIG_VALUE_"+strconv.Itoa(i)] = value
}
return env
}
func ListFiles(ctx context.Context, dir string) ([]string, error) {
listed, err := runGit(ctx, dir, nil, "ls-files", "-z", "-t", "--stage", "--others", "--exclude-standard")
if err != nil {
return nil, err
}
if _, err := r.CreateRemote(&config.RemoteConfig{Name: "origin", URLs: []string{input.URL}}); err != nil {
return nil, err
var files, repos []string
for entry := range strings.SplitSeq(string(listed), "\x00") {
tag, file, ok := strings.Cut(entry, " ")
if tag != "?" {
var meta string
meta, file, ok = strings.Cut(file, "\t")
if strings.HasPrefix(meta, "160000 ") {
file += "/"
}
}
if repo, isRepo := strings.CutSuffix(file, "/"); isRepo {
repos = append(repos, repo)
} else if ok {
files = append(files, file)
}
}
if err := r.FetchContext(ctx, &git.FetchOptions{
RefSpecs: []config.RefSpec{pinnedRefSpec(input.Ref)},
Depth: input.Depth,
Tags: git.NoTags,
Auth: opts.Auth,
Progress: opts.Progress,
InsecureSkipTLS: opts.InsecureSkipTLS,
}); err != nil {
return nil, err
slices.Sort(repos)
for _, repo := range slices.Compact(repos) {
files = append(files, repo)
if _, err := os.Lstat(filepath.Join(dir, repo, ".git")); err != nil {
continue // git lists an uninitialized submodule as "./" from inside it
}
nested, err := ListFiles(ctx, filepath.Join(dir, repo))
if err != nil {
return nil, err
}
for _, file := range nested {
files = append(files, repo+"/"+file)
}
}
return r, nil
return files, nil
}
// pinnedRefSpec keeps a hash-shaped name out of refs/heads and refs/tags.
func pinnedRefSpec(sha string) config.RefSpec {
return config.RefSpec(fmt.Sprintf("+%s:refs/pinned/%s", sha, sha))
}
// refRevision picks the revision to check out and reports whether it resolves locally.
func refRevision(r *git.Repository, ref string) (plumbing.Revision, bool) {
if plumbing.IsHash(ref) {
// git ignores a ref named as 40 hex digits, so a full hash always denotes the commit itself.
_, err := r.CommitObject(plumbing.NewHash(ref))
return plumbing.Revision(ref), err == nil
}
if _, err := r.Tag(ref); err == nil {
return plumbing.Revision(path.Join("refs", "tags", ref)), true
}
remoteRef := plumbing.ReferenceName(path.Join("refs", "remotes", "origin", ref))
if _, err := r.Reference(remoteRef, false); err == nil {
return plumbing.Revision(remoteRef), true
}
rev := plumbing.Revision(ref)
_, err := r.ResolveRevision(rev)
return rev, err == nil
}
// isShallow reports whether the local repository was cloned with a limited depth.
func isShallow(r *git.Repository) bool {
shallows, err := r.Storer.Shallow()
return err == nil && len(shallows) > 0
}
// shallowFetchRefSpec limits a shallow update to the requested ref, falling back to the broad refspec when it is not present locally.
func shallowFetchRefSpec(r *git.Repository, ref string) (config.RefSpec, bool) {
tagRef := plumbing.NewTagReferenceName(ref)
if _, err := r.Reference(tagRef, false); err == nil {
return config.RefSpec(fmt.Sprintf("+%s:%s", tagRef, tagRef)), true
}
remoteRef := plumbing.NewRemoteReferenceName("origin", ref)
if _, err := r.Reference(remoteRef, false); err == nil {
branchRef := plumbing.NewBranchReferenceName(ref)
return config.RefSpec(fmt.Sprintf("+%s:%s", branchRef, remoteRef)), true
}
if plumbing.IsHash(ref) {
// The broad refspec carries only advertised tips, never a pinned commit.
return pinnedRefSpec(ref), true
}
return "", false
func runGit(ctx context.Context, dir string, input *NewGitCloneExecutorInput, args ...string) ([]byte, error) {
config := []string{"core.autocrlf=false", "core.eol=lf"} // keep action files LF
if input != nil {
config = append(config, RemoteConfig(input.Token, input.InsecureSkipTLS)...)
if input.ClientCertFile != "" {
config = append(config, "http.sslCert="+input.ClientCertFile, "http.sslKey="+input.ClientKeyFile)
}
}
if dir != "" {
args = append([]string{"-C", dir}, args...)
}
cmd := exec.CommandContext(ctx, "git", args...)
cmd.Env = slices.DeleteFunc(os.Environ(), func(entry string) bool {
name, _, _ := strings.Cut(entry, "=")
return slices.Contains(LocalEnvVars, name)
})
for key, value := range ConfigEnv(config...) {
cmd.Env = append(cmd.Env, key+"="+value)
}
cmd.Env = append(cmd.Env, "GIT_TERMINAL_PROMPT=0")
if runtime.GOOS != "windows" {
cmd.Cancel = func() error { return cmd.Process.Signal(os.Interrupt) } // lets git remove its lock files
}
cmd.WaitDelay = time.Second
var stderr strings.Builder
cmd.Stderr = &stderr
output, err := cmd.Output()
if err != nil {
return output, fmt.Errorf("git %s: %w: %s", strings.Join(args, " "), err, strings.TrimSpace(stderr.String()))
}
return output, nil
}

View File

@@ -6,6 +6,7 @@ package git
import (
"context"
"encoding/base64"
"fmt"
"net/http"
"net/http/httptest"
@@ -15,48 +16,47 @@ import (
"strconv"
"strings"
"sync"
"sync/atomic"
"syscall"
"testing"
"testing/synctest"
"time"
"gitea.com/gitea/runner/act/common"
"gitea.com/gitea/runner/act/common/logging"
"gitea.com/gitea/runner/act/common/logging/logtest"
"gitea.com/gitea/runner/internal/pkg/lock"
gogit "github.com/go-git/go-git/v5"
gogitconfig "github.com/go-git/go-git/v5/config"
"github.com/go-git/go-git/v5/plumbing/transport"
gogitclient "github.com/go-git/go-git/v5/plumbing/transport/client"
gogitfile "github.com/go-git/go-git/v5/plumbing/transport/file"
log "github.com/sirupsen/logrus"
logrustest "github.com/sirupsen/logrus/hooks/test"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
func TestFindGitSlug(t *testing.T) {
assert := assert.New(t)
slugTests := []struct {
url string // input
provider string // expected result
slug string // expected result
for _, testcase := range []struct {
remoteURL string
slug string
}{
{"https://git-codecommit.us-east-1.amazonaws.com/v1/repos/my-repo-name", "CodeCommit", "my-repo-name"},
{"ssh://git-codecommit.us-west-2.amazonaws.com/v1/repos/my-repo", "CodeCommit", "my-repo"},
{"git@github.com:nektos/act.git", "GitHub", "nektos/act"},
{"git@github.com:nektos/act", "GitHub", "nektos/act"},
{"https://github.com/nektos/act.git", "GitHub", "nektos/act"},
{"http://github.com/nektos/act.git", "GitHub", "nektos/act"},
{"https://github.com/nektos/act", "GitHub", "nektos/act"},
{"http://github.com/nektos/act", "GitHub", "nektos/act"},
{"git+ssh://git@github.com/owner/repo.git", "GitHub", "owner/repo"},
{"http://myotherrepo.com/act.git", "", "http://myotherrepo.com/act.git"},
}
for _, tt := range slugTests {
provider, slug := findGitSlug(tt.url, "github.com")
assert.Equal(tt.provider, provider)
assert.Equal(tt.slug, slug)
{"https://git-codecommit.us-east-1.amazonaws.com/v1/repos/my-repo-name", "my-repo-name"},
{"ssh://git-codecommit.us-west-2.amazonaws.com/v1/repos/my-repo", "my-repo"},
{"git@github.com:nektos/act.git", "nektos/act"},
{"git@github.com:nektos/act", "nektos/act"},
{"https://github.com/nektos/act.git", "nektos/act"},
{"http://github.com/nektos/act.git", "nektos/act"},
{"https://github.com/nektos/act", "nektos/act"},
{"http://github.com/nektos/act", "nektos/act"},
{"git+ssh://git@github.com/owner/repo.git", "owner/repo"},
{"https://example.com:3000/gitea/owner/repo.git/", "owner/repo"},
{"ssh://git@example.com:2222/gitea/owner/repo.git", "owner/repo"},
{"example.com:owner/repo.git", "owner/repo"},
{"http://myotherrepo.com/act.git", ""},
{"https://user:secret@example.com/act.git", ""},
{"/tmp/owner/repo.git", ""},
{"./owner/repo:branch", ""},
} {
t.Run(testcase.remoteURL, func(t *testing.T) {
slug, err := findGitSlug(testcase.remoteURL)
assert.Equal(t, testcase.slug == "", err != nil)
assert.NotContains(t, fmt.Sprint(err), "secret")
assert.Equal(t, testcase.slug, slug)
})
}
}
@@ -88,23 +88,37 @@ func cleanGitHooks(dir string) error {
return nil
}
func TestFindGithubRepoUsesOrigin(t *testing.T) {
func TestFindGitMetadataOfSHA256RepositoryUsesOrigin(t *testing.T) {
t.Parallel()
basedir := t.TempDir()
const remoteURL = "https://github.com/owner/repo.git"
require.NoError(t, gitCmd("init", basedir))
require.NoError(t, gitCmd("init", "--object-format=sha256", "--initial-branch=main", basedir))
require.NoError(t, cleanGitHooks(basedir))
require.NoError(t, gitCmd("-C", basedir, "commit", "--allow-empty", "-m", "init"))
require.NoError(t, gitCmd("-C", basedir, "remote", "add", "origin", remoteURL))
require.NoError(t, gitCmd("-C", basedir, "config", "--add", "remote.origin.url", "https://github.com/other/repo.git"))
url, err := findGitRemoteURL(context.Background(), basedir, "origin")
shortSHA, sha, err := FindGitRevision(t.Context(), basedir)
require.NoError(t, err)
require.Equal(t, gitRevParse(t, basedir, "HEAD"), sha)
require.Len(t, sha, 64)
require.Equal(t, sha[:7], shortSHA)
ref, err := FindGitRef(t.Context(), basedir)
require.NoError(t, err)
require.Equal(t, "refs/heads/main", ref)
url, err := findGitRemoteURL(t.Context(), basedir)
require.NoError(t, err)
require.Equal(t, remoteURL, url)
slug, err := FindGithubRepo(context.Background(), basedir, "github.com")
slug, err := FindGithubRepo(t.Context(), basedir)
require.NoError(t, err)
require.Equal(t, "owner/repo", slug)
}
func TestGitFindRef(t *testing.T) {
t.Parallel()
basedir := t.TempDir()
for name, tt := range map[string]struct {
@@ -175,7 +189,7 @@ func TestGitFindRef(t *testing.T) {
require.NoError(t, gitCmd("-C", dir, "init", "--initial-branch=master"))
require.NoError(t, cleanGitHooks(dir))
tt.Prepare(t, dir)
ref, err := FindGitRef(context.Background(), dir)
ref, err := FindGitRef(t.Context(), dir)
tt.Assert(t, ref, err)
})
}
@@ -191,7 +205,9 @@ func TestGitCloneExecutor(t *testing.T) {
workDir := t.TempDir()
require.NoError(t, gitCmd("clone", remoteDir, workDir))
require.NoError(t, gitCmd("-C", workDir, "checkout", "-b", "main"))
require.NoError(t, gitCmd("-C", workDir, "commit", "--allow-empty", "-m", "initial"))
require.NoError(t, os.WriteFile(filepath.Join(workDir, "action.sh"), []byte("#!/bin/sh\necho hi\n"), 0o755))
require.NoError(t, gitCmd("-C", workDir, "add", "action.sh"))
require.NoError(t, gitCmd("-C", workDir, "commit", "-m", "initial"))
require.NoError(t, gitCmd("-C", workDir, "tag", "v2"))
require.NoError(t, gitCmd("-C", workDir, "push", "-u", "origin", "main"))
require.NoError(t, gitCmd("-C", workDir, "push", "origin", "v2"))
@@ -205,6 +221,13 @@ func TestGitCloneExecutor(t *testing.T) {
require.NoError(t, err)
fullSha := strings.TrimSpace(string(out))
require.NoError(t, gitCmd("-C", workDir, "push", "origin", "main:refs/heads/"+fullSha[:4]))
hostConfig := filepath.Join(t.TempDir(), "gitconfig")
require.NoError(t, os.WriteFile(hostConfig, []byte("[core]\n\tautocrlf = true\n[url \""+remoteDir+"\"]\n\tinsteadOf = https://example.invalid/action\n"), 0o644))
t.Setenv("GIT_CONFIG_GLOBAL", hostConfig)
t.Setenv("GIT_DIR", t.TempDir())
for name, tt := range map[string]struct {
Err error
Ref string
@@ -225,26 +248,42 @@ func TestGitCloneExecutor(t *testing.T) {
Err: &Error{ErrShortRef, fullSha},
Ref: fullSha[:7],
},
"HEAD": {
Ref: "HEAD",
},
"hex branch named like its own commit": {
Ref: fullSha[:4],
},
} {
t.Run(name, func(t *testing.T) {
dir := t.TempDir()
clone := NewGitCloneExecutor(NewGitCloneExecutorInput{
URL: remoteDir,
URL: "https://example.invalid/action",
Ref: tt.Ref,
Dir: t.TempDir(),
Dir: dir,
})
err := clone(context.Background())
err := clone(t.Context())
if tt.Err != nil {
assert.Error(t, err) //nolint:testifylint // pre-existing issue from nektos/act
assert.Equal(t, tt.Err, err)
} else {
assert.Empty(t, err) //nolint:testifylint // pre-existing issue from nektos/act
assert.Equal(t, tt.Err, clone(t.Context()), "a retry on the cache keeps the error")
return
}
assert.Empty(t, err) //nolint:testifylint // pre-existing issue from nektos/act
marker := filepath.Join(dir, "marker")
require.NoError(t, os.WriteFile(marker, nil, 0o644))
require.NoError(t, clone(t.Context()))
assert.FileExists(t, marker, "an insteadOf rewrite must not evict the cache")
script, err := os.ReadFile(filepath.Join(dir, "action.sh"))
require.NoError(t, err)
assert.Equal(t, "#!/bin/sh\necho hi\n", string(script), "host core.autocrlf must not convert action files")
})
}
}
func TestGitCloneExecutorReclonesWhenOriginURLChanges(t *testing.T) {
t.Parallel()
createRemote := func(message string) string {
remoteDir := t.TempDir()
require.NoError(t, gitCmd("init", "--bare", "--initial-branch=main", remoteDir))
@@ -268,16 +307,27 @@ func TestGitCloneExecutorReclonesWhenOriginURLChanges(t *testing.T) {
Dir: cacheDir,
})(t.Context()))
nested := filepath.Join(cacheDir, "nested")
require.NoError(t, os.Mkdir(nested, 0o755))
require.NoError(t, NewGitCloneExecutor(NewGitCloneExecutorInput{URL: oldRemoteDir, Ref: "main", Dir: nested})(t.Context()))
assert.DirExists(t, filepath.Join(nested, ".git"), "an empty cache dir inside a clone of the same URL gets its own repository")
markerPath := filepath.Join(cacheDir, "stale-marker")
require.NoError(t, os.WriteFile(markerPath, []byte("stale"), 0o644))
cancelled, cancel := context.WithCancel(t.Context())
cancel()
_, err := CloneIfRequired(cancelled, NewGitCloneExecutorInput{URL: oldRemoteDir, Ref: "main", Dir: cacheDir}, logging.New(logtest.New()))
require.ErrorIs(t, err, context.Canceled)
require.FileExists(t, markerPath)
require.NoError(t, NewGitCloneExecutor(NewGitCloneExecutorInput{
URL: newRemoteDir,
Ref: "main",
Dir: cacheDir,
})(t.Context()))
originURL, err := findGitRemoteURL(t.Context(), cacheDir, "origin")
originURL, err := findGitRemoteURL(t.Context(), cacheDir)
require.NoError(t, err)
assert.Equal(t, newRemoteDir, originURL)
@@ -289,61 +339,47 @@ func TestGitCloneExecutorReclonesWhenOriginURLChanges(t *testing.T) {
require.True(t, os.IsNotExist(err), "stale cached directory should be removed before recloning")
}
func TestGitCloneExecutorNonFastForwardRef(t *testing.T) {
// Simulate the scenario where a remote ref (e.g. a GitHub PR head ref) changes
// non-fast-forward between two fetches. Before the fix, the fetch used Force=false,
// causing go-git to return ErrForceNeeded and short-circuit the checkout.
// Create a bare "remote" repo with an initial commit on main and a feature branch.
func TestGitCloneExecutorFollowsRemoteRefChanges(t *testing.T) {
t.Parallel()
remoteDir := t.TempDir()
require.NoError(t, gitCmd("init", "--bare", "--initial-branch=main", remoteDir))
// We need a working clone to push commits from.
workDir := t.TempDir()
require.NoError(t, gitCmd("clone", remoteDir, workDir))
require.NoError(t, gitCmd("-C", workDir, "checkout", "-b", "main"))
require.NoError(t, gitCmd("-C", workDir, "commit", "--allow-empty", "-m", "initial"))
require.NoError(t, gitCmd("-C", workDir, "commit", "--allow-empty", "-m", "feature-1"))
require.NoError(t, gitCmd("-C", workDir, "push", "-u", "origin", "main"))
// Create a feature branch (simulates refs/pull/N/head).
require.NoError(t, gitCmd("-C", workDir, "checkout", "-b", "feature"))
require.NoError(t, gitCmd("-C", workDir, "commit", "--allow-empty", "-m", "feature-1"))
require.NoError(t, gitCmd("-C", workDir, "push", "origin", "feature"))
dir := t.TempDir()
clone := NewGitCloneExecutor(NewGitCloneExecutorInput{URL: remoteDir, Ref: "main", Dir: dir})
require.NoError(t, clone(t.Context()))
require.NoError(t, gitCmd("-C", dir, "config", "fetch.prune", "true"))
require.NoError(t, clone(t.Context()))
// First clone via the executor — should succeed and cache the repo.
cloneDir := t.TempDir()
clone := NewGitCloneExecutor(NewGitCloneExecutorInput{
URL: remoteDir,
Ref: "main",
Dir: cloneDir,
})
require.NoError(t, clone(context.Background()))
// Now force-push the feature branch to a non-fast-forward commit (simulates
// a PR rebase). This makes refs/heads/feature non-fast-forward.
require.NoError(t, gitCmd("-C", workDir, "checkout", "main"))
require.NoError(t, gitCmd("-C", workDir, "branch", "-D", "feature"))
require.NoError(t, gitCmd("-C", workDir, "checkout", "-b", "feature"))
require.NoError(t, gitCmd("-C", workDir, "reset", "--hard", "HEAD~1"))
require.NoError(t, gitCmd("-C", workDir, "commit", "--allow-empty", "-m", "feature-rewritten"))
require.NoError(t, gitCmd("-C", workDir, "push", "--force", "origin", "feature"))
require.NoError(t, gitCmd("-C", workDir, "push", "--force", "origin", "main"))
require.NoError(t, clone(t.Context()), "a non-fast-forward ref must update")
assert.Equal(t, "feature-rewritten", gitHeadSubject(t, dir))
// Also advance main so we can verify the clone picks up the new commit.
require.NoError(t, gitCmd("-C", workDir, "checkout", "main"))
require.NoError(t, gitCmd("-C", workDir, "commit", "--allow-empty", "-m", "second"))
require.NoError(t, gitCmd("-C", workDir, "push", "origin", "main"))
tagDir := t.TempDir()
tagClone := NewGitCloneExecutor(NewGitCloneExecutorInput{URL: remoteDir, Ref: "v9", Dir: tagDir})
require.Error(t, tagClone(t.Context()))
require.NoError(t, gitCmd("-C", workDir, "tag", "v9"))
require.NoError(t, gitCmd("-C", workDir, "push", "origin", "v9"))
require.NoError(t, tagClone(t.Context()), "a tag published after a failed lookup must be fetched")
// Second clone to the same directory — before the fix this returned ErrForceNeeded
// and left the working tree at the old commit.
err := clone(context.Background())
require.NoError(t, err, "fetch with non-fast-forward refs must not fail when Force=true")
// Verify the working tree was actually updated to the latest main commit.
out, err := exec.Command("git", "-C", cloneDir, "log", "--oneline", "-1", "--format=%s").Output()
require.NoError(t, err)
assert.Equal(t, "second", strings.TrimSpace(string(out)), "working tree should be at the latest commit")
require.NoError(t, gitCmd("-C", workDir, "commit", "--allow-empty", "-m", "tag-moved"))
require.NoError(t, gitCmd("-C", workDir, "tag", "--force", "v9"))
require.NoError(t, gitCmd("-C", workDir, "push", "--force", "origin", "v9"))
for range 2 {
require.NoError(t, tagClone(t.Context()), "a moved tag must update and stay current")
assert.Equal(t, "tag-moved", gitHeadSubject(t, tagDir))
}
}
func TestGitCloneExecutorOfflineMode(t *testing.T) {
t.Parallel()
// Build a local "remote" with a single commit on main.
remoteDir := t.TempDir()
require.NoError(t, gitCmd("init", "--bare", "--initial-branch=main", remoteDir))
@@ -359,7 +395,7 @@ func TestGitCloneExecutorOfflineMode(t *testing.T) {
URL: remoteDir,
Ref: "main",
Dir: cacheDir,
})(context.Background()))
})(t.Context()))
t.Run("cached branch resolves without fetching", func(t *testing.T) {
// Offline reuse of a cached branch must succeed even though ResolveRevision(input.Ref)
@@ -369,7 +405,7 @@ func TestGitCloneExecutorOfflineMode(t *testing.T) {
Ref: "main",
Dir: cacheDir,
OfflineMode: true,
})(context.Background())
})(t.Context())
require.NoError(t, err)
out, err := exec.Command("git", "-C", cacheDir, "log", "--oneline", "-1", "--format=%s").Output()
@@ -384,50 +420,13 @@ func TestGitCloneExecutorOfflineMode(t *testing.T) {
Ref: "never-fetched",
Dir: cacheDir,
OfflineMode: true,
})(context.Background())
})(t.Context())
require.Error(t, err)
})
}
func TestGitCloneExecutorQuietDemotesCloneLine(t *testing.T) {
remoteDir := t.TempDir()
require.NoError(t, gitCmd("init", "--bare", "--initial-branch=main", remoteDir))
workDir := t.TempDir()
require.NoError(t, gitCmd("clone", remoteDir, workDir))
require.NoError(t, gitCmd("-C", workDir, "checkout", "-b", "main"))
require.NoError(t, gitCmd("-C", workDir, "commit", "--allow-empty", "-m", "initial"))
require.NoError(t, gitCmd("-C", workDir, "push", "-u", "origin", "main"))
// Quiet callers report the download themselves, so the clone line must not reach the job log.
for name, quiet := range map[string]bool{"quiet": true, "not quiet": false} {
t.Run(name, func(t *testing.T) {
logger, hook := logrustest.NewNullLogger()
logger.SetLevel(log.InfoLevel)
ctx := common.WithLogger(context.Background(), logger.WithField("job", "j1"))
require.NoError(t, NewGitCloneExecutor(NewGitCloneExecutorInput{
URL: remoteDir,
Ref: "main",
Dir: t.TempDir(),
Quiet: quiet,
})(ctx))
var cloneLines int
for _, entry := range hook.AllEntries() {
if strings.HasPrefix(entry.Message, "git clone ") {
cloneLines++
}
}
if quiet {
assert.Zero(t, cloneLines)
} else {
assert.Equal(t, 1, cloneLines)
}
})
}
}
func TestGitCloneExecutorShallow(t *testing.T) {
t.Parallel()
// Build a local "remote" with several commits on main plus a tag, so a full clone would pull noticeably more history than a shallow one.
remoteDir := t.TempDir()
require.NoError(t, gitCmd("init", "--bare", "--initial-branch=main", remoteDir))
@@ -471,12 +470,11 @@ func TestGitCloneExecutorShallow(t *testing.T) {
return remote
}
t.Run("commit hash falls back to a full clone when the remote refuses unadvertised objects", func(t *testing.T) {
t.Run("commit hash resolves when the remote accepts a direct shallow fetch", func(t *testing.T) {
dir := t.TempDir()
require.NoError(t, NewGitCloneExecutor(NewGitCloneExecutorInput{
URL: unadvertisedRemote(t, false), Ref: unadvertisedSHA, Dir: dir, Depth: 1,
})(t.Context()))
assert.NoFileExists(t, shallowMarker(dir))
assert.Equal(t, unadvertisedSHA, gitRevParse(t, dir, "HEAD"))
})
@@ -503,76 +501,46 @@ func TestGitCloneExecutorShallow(t *testing.T) {
assert.Equal(t, olderSHA, gitRevParse(t, dir, "HEAD"))
})
t.Run("moving branch updates while staying shallow", func(t *testing.T) {
t.Run("moving hexadecimal-named branch updates while staying shallow", func(t *testing.T) {
require.NoError(t, gitCmd("-C", workDir, "push", "origin", "main:deadbeef"))
dir := t.TempDir()
require.NoError(t, NewGitCloneExecutor(NewGitCloneExecutorInput{
URL: remoteDir, Ref: "main", Dir: dir, Depth: 1,
})(t.Context()))
clone := NewGitCloneExecutor(NewGitCloneExecutorInput{
URL: remoteDir, Ref: "deadbeef", Dir: dir, Depth: 1,
})
require.NoError(t, clone(t.Context()))
require.Equal(t, "c3", gitHeadSubject(t, dir))
// Advance main on the remote, then reuse the existing shallow clone.
require.NoError(t, gitCmd("-C", workDir, "commit", "--allow-empty", "-m", "c4"))
require.NoError(t, gitCmd("-C", workDir, "push", "origin", "main"))
require.NoError(t, gitCmd("-C", workDir, "push", "origin", "main:deadbeef"))
require.NoError(t, NewGitCloneExecutor(NewGitCloneExecutorInput{
URL: remoteDir, Ref: "main", Dir: dir, Depth: 1,
})(t.Context()))
require.NoError(t, clone(t.Context()))
assert.Equal(t, "c4", gitHeadSubject(t, dir), "reused shallow clone should update to the new tip")
assert.FileExists(t, shallowMarker(dir), "repo should remain shallow after update")
assert.Equal(t, 1, gitRevCount(t, dir))
})
}
func TestGitCloneExecutorTransportSessions(t *testing.T) {
workDir := t.TempDir()
require.NoError(t, gitCmd("init", "--initial-branch=main", workDir))
require.NoError(t, gitCmd("-C", workDir, "commit", "--allow-empty", "-m", "c1"))
require.NoError(t, gitCmd("-C", workDir, "tag", "v1"))
require.NoError(t, gitCmd("-C", workDir, "tag", "-a", "v2", "-m", "v2"))
for name, tt := range map[string]struct {
Ref string
Depth int
}{
"shallow branch": {"main", 1},
"full clone branch": {"main", 0},
"full lightweight tag": {"v1", 0},
"full annotated tag": {"v2", 0},
} {
t.Run(name, func(t *testing.T) {
counter := installCountingTransport(t)
dir := t.TempDir()
clone := NewGitCloneExecutor(NewGitCloneExecutorInput{
URL: workDir, Ref: tt.Ref, Dir: dir, Depth: tt.Depth,
})
require.NoError(t, clone(t.Context()))
assert.Equal(t, int64(1), counter.sessions.Swap(0), "cold clone")
assert.Equal(t, gitRevParse(t, workDir, tt.Ref+"^{commit}"), gitRevParse(t, dir, "HEAD"))
require.NoError(t, clone(t.Context()))
assert.Equal(t, int64(1), counter.sessions.Swap(0), "unchanged warm cache")
require.NoError(t, os.WriteFile(filepath.Join(workDir, "action.yml"), []byte(name), 0o644))
require.NoError(t, gitCmd("-C", workDir, "add", "action.yml"))
require.NoError(t, gitCmd("-C", workDir, "commit", "-m", name))
require.NoError(t, gitCmd("-C", workDir, "tag", "--force", "v1"))
require.NoError(t, gitCmd("-C", workDir, "tag", "--force", "-a", "v2", "-m", "v2"))
require.NoError(t, os.WriteFile(filepath.Join(dir, "action.yml"), []byte("staged"), 0o644))
require.NoError(t, gitCmd("-C", dir, "add", "action.yml"))
require.NoError(t, os.WriteFile(filepath.Join(dir, "action.yml"), []byte("unstaged"), 0o644))
require.NoError(t, clone(t.Context()))
assert.Equal(t, int64(1), counter.sessions.Load(), "updated warm cache")
assert.Equal(t, gitRevParse(t, workDir, "HEAD"), gitRevParse(t, dir, "HEAD"))
status, err := exec.Command("git", "-C", dir, "status", "--porcelain").Output()
require.NoError(t, err)
assert.Empty(t, string(status))
})
}
t.Run("sha256 full hash is cloned shallowly, reused without the remote, and its 40-character prefix is short", func(t *testing.T) {
remote := filepath.Join(t.TempDir(), "remote.git")
require.NoError(t, gitCmd("init", "--bare", "--object-format=sha256", "--initial-branch=main", remote))
work := t.TempDir()
require.NoError(t, gitCmd("init", "--object-format=sha256", "--initial-branch=main", work))
require.NoError(t, gitCmd("-C", work, "remote", "add", "origin", remote))
require.NoError(t, gitCmd("-C", work, "commit", "--allow-empty", "-m", "pinned"))
require.NoError(t, gitCmd("-C", work, "push", "origin", "main"))
sha := gitRevParse(t, work, "HEAD")
dir := t.TempDir()
clone := NewGitCloneExecutor(NewGitCloneExecutorInput{URL: remote, Ref: sha, Dir: dir, Depth: 1})
require.NoError(t, clone(t.Context()))
assert.FileExists(t, shallowMarker(dir))
require.ErrorIs(t, NewGitCloneExecutor(NewGitCloneExecutorInput{URL: remote, Ref: sha[:40], Dir: dir, Depth: 1, OfflineMode: true})(t.Context()), ErrShortRef)
require.NoError(t, os.RemoveAll(remote))
require.NoError(t, clone(t.Context()))
assert.Equal(t, sha, gitRevParse(t, dir, "HEAD"))
})
}
func TestGitCloneExecutorPinnedHashIgnoresShadowingRef(t *testing.T) {
t.Parallel()
remoteDir := t.TempDir()
require.NoError(t, gitCmd("init", "--bare", "--initial-branch=main", remoteDir))
workDir := t.TempDir()
@@ -596,25 +564,6 @@ func TestGitCloneExecutorPinnedHashIgnoresShadowingRef(t *testing.T) {
}
}
type countingTransport struct {
transport.Transport
sessions atomic.Int64
}
func (c *countingTransport) NewUploadPackSession(ep *transport.Endpoint, auth transport.AuthMethod) (transport.UploadPackSession, error) {
c.sessions.Add(1)
return c.Transport.NewUploadPackSession(ep, auth)
}
func installCountingTransport(t *testing.T) *countingTransport {
t.Helper()
counter := &countingTransport{Transport: gogitfile.DefaultClient}
gogitclient.InstallProtocol("file", counter)
t.Cleanup(func() { gogitclient.InstallProtocol("file", gogitfile.DefaultClient) })
return counter
}
func gitRevParse(t *testing.T, dir, rev string) string {
t.Helper()
out, err := exec.Command("git", "-C", dir, "rev-parse", rev).Output()
@@ -665,14 +614,42 @@ func gitCmd(args ...string) error {
}
func TestAcquireCloneLock(t *testing.T) {
t.Run("blocks on a lock held by another process despite a trailing separator", func(t *testing.T) {
dir := filepath.Join(t.TempDir(), "cache")
release, err := lock.TryLock(dir)
require.NoError(t, err)
defer func() { require.NoError(t, release()) }()
synctest.Test(t, func(t *testing.T) {
ctx, cancel := context.WithTimeout(t.Context(), time.Second)
defer cancel()
_, err := AcquireCloneLock(ctx, dir+string(filepath.Separator))
require.ErrorIs(t, err, context.DeadlineExceeded)
})
})
t.Run("read-only parent falls back to an in-process lock", func(t *testing.T) {
parent := t.TempDir()
dir := filepath.Join(parent, "cache")
require.NoError(t, os.Mkdir(dir, 0o755))
require.NoError(t, os.Chmod(parent, 0o555))
defer func() { require.NoError(t, os.Chmod(parent, 0o755)) }()
unlock, err := AcquireCloneLock(t.Context(), dir)
require.NoError(t, err)
unlock()
})
t.Run("same directory serializes", func(t *testing.T) {
dir := t.TempDir()
unlock1 := AcquireCloneLock(dir)
unlock1, err := AcquireCloneLock(t.Context(), dir)
require.NoError(t, err)
secondAcquired := make(chan struct{})
go func() {
unlock := AcquireCloneLock(dir)
unlock, err := AcquireCloneLock(t.Context(), dir)
if !assert.NoError(t, err) {
return
}
close(secondAcquired)
unlock()
}()
@@ -696,12 +673,16 @@ func TestAcquireCloneLock(t *testing.T) {
dirA := t.TempDir()
dirB := t.TempDir()
unlockA := AcquireCloneLock(dirA)
unlockA, err := AcquireCloneLock(t.Context(), dirA)
require.NoError(t, err)
defer unlockA()
done := make(chan struct{})
go func() {
unlock := AcquireCloneLock(dirB)
unlock, err := AcquireCloneLock(t.Context(), dirB)
if !assert.NoError(t, err) {
return
}
unlock()
close(done)
}()
@@ -715,12 +696,17 @@ func TestAcquireCloneLock(t *testing.T) {
}
// An unresponsive remote must not pin a job: the refresh has to be interruptible.
func TestNewGitCloneExecutorFetchHonoursContext(t *testing.T) {
func TestNewGitCloneExecutorFetchSendsTokenAndHonoursContext(t *testing.T) {
t.Parallel()
block := make(chan struct{})
reached := make(chan struct{})
var once sync.Once
server := httptest.NewServer(http.HandlerFunc(func(_ http.ResponseWriter, _ *http.Request) {
once.Do(func() { close(reached) })
var authorization string
server := httptest.NewServer(http.HandlerFunc(func(_ http.ResponseWriter, request *http.Request) {
once.Do(func() {
authorization = request.Header.Get("Authorization")
close(reached)
})
<-block
}))
t.Cleanup(func() {
@@ -729,17 +715,15 @@ func TestNewGitCloneExecutorFetchHonoursContext(t *testing.T) {
})
dir := filepath.Join(t.TempDir(), "cached-action")
repo, err := gogit.PlainInit(dir, false)
require.NoError(t, err)
_, err = repo.CreateRemote(&gogitconfig.RemoteConfig{Name: "origin", URLs: []string{server.URL}})
require.NoError(t, err)
require.NoError(t, gitCmd("init", dir))
require.NoError(t, gitCmd("-C", dir, "remote", "add", "origin", server.URL))
ctx, cancel := context.WithCancel(t.Context())
defer cancel()
done := make(chan error, 1)
go func() {
done <- NewGitCloneExecutor(NewGitCloneExecutorInput{URL: server.URL, Ref: "main", Dir: dir})(ctx)
done <- NewGitCloneExecutor(NewGitCloneExecutorInput{URL: server.URL, Ref: "main", Dir: dir, Token: "secret"})(ctx)
}()
select {
@@ -747,6 +731,7 @@ func TestNewGitCloneExecutorFetchHonoursContext(t *testing.T) {
case <-time.After(10 * time.Second):
t.Fatal("the executor never reached the remote")
}
assert.Equal(t, "Basic "+base64.StdEncoding.EncodeToString([]byte("token:secret")), authorization)
cancel()
select {

View File

@@ -7,46 +7,53 @@ package common
import (
"context"
"github.com/sirupsen/logrus"
"gitea.com/gitea/runner/act/common/logging"
)
type loggerContextKey string
const loggerContextKeyVal = loggerContextKey("logrus.FieldLogger")
const loggerContextKeyVal = loggerContextKey("logging.Logger")
// Logger returns the appropriate logger for current context
func Logger(ctx context.Context) logrus.FieldLogger {
val := ctx.Value(loggerContextKeyVal)
if val != nil {
if logger, ok := val.(logrus.FieldLogger); ok {
return logger
}
func Logger(ctx context.Context) *logging.Logger {
if logger, ok := ctx.Value(loggerContextKeyVal).(*logging.Logger); ok {
return logger
}
return logrus.StandardLogger()
return logging.Default()
}
// WithLogger adds a value to the context for the logger
func WithLogger(ctx context.Context, logger logrus.FieldLogger) context.Context {
func WithLogger(ctx context.Context, logger *logging.Logger) context.Context {
return context.WithValue(ctx, loggerContextKeyVal, logger)
}
type loggerHookKey string
type loggerHandlerKey string
const loggerHookKeyVal = loggerHookKey("logrus.Hook")
const loggerHandlerKeyVal = loggerHandlerKey("logging.Handler")
// LoggerHook returns the appropriate logger hook for current context
// the hook affects job logger, not global logger
func LoggerHook(ctx context.Context) logrus.Hook {
val := ctx.Value(loggerHookKeyVal)
if val != nil {
if hook, ok := val.(logrus.Hook); ok {
return hook
}
// LoggerHandler returns the extra handler the job logger sends its records to, or nil.
// It affects the job logger, not the global logger.
func LoggerHandler(ctx context.Context) logging.Handler {
if h, ok := ctx.Value(loggerHandlerKeyVal).(logging.Handler); ok {
return h
}
return nil
}
// WithLoggerHook adds a value to the context for the logger hook
func WithLoggerHook(ctx context.Context, hook logrus.Hook) context.Context {
return context.WithValue(ctx, loggerHookKeyVal, hook)
// WithLoggerHandler adds the extra handler for the job logger to the context
func WithLoggerHandler(ctx context.Context, h logging.Handler) context.Context {
return context.WithValue(ctx, loggerHandlerKeyVal, h)
}
type stepIDsContextKey struct{}
// StepIDs returns the IDs of the step and the composite steps enclosing the current one, outermost first.
func StepIDs(ctx context.Context) []string {
ids, _ := ctx.Value(stepIDsContextKey{}).([]string)
return ids
}
// WithStepIDs replaces the step IDs in the context
func WithStepIDs(ctx context.Context, ids []string) context.Context {
return context.WithValue(ctx, stepIDsContextKey{}, ids)
}

View File

@@ -0,0 +1,287 @@
// Copyright 2026 The Gitea Authors. All rights reserved.
// SPDX-License-Identifier: MIT
package logging
import (
"bytes"
"context"
"fmt"
"io"
"log/slog"
"reflect"
"runtime"
"slices"
"strconv"
"strings"
"sync"
"time"
)
const (
ansiReset = "\x1b[0m"
ansiRed = "\x1b[31m"
ansiYellow = "\x1b[33m"
ansiCyan = "\x1b[36m"
ansiDimCyan = "\x1b[2;36m"
ansiDimWhite = "\x1b[2;37m"
)
// consoleStart is the zero of the elapsed-seconds stamp used when full timestamps are off.
var consoleStart = time.Now()
// console is the state shared by every consoleHandler derived from one another.
type console struct {
mu sync.Mutex
out io.Writer
terminal bool
opts TextOptions
reportCaller bool
level slog.LevelVar
}
func newConsole(out io.Writer) *console {
c := &console{}
c.setOutput(out)
return c
}
func (c *console) setOutput(w io.Writer) {
terminal := checkIfTerminal(w)
c.mu.Lock()
defer c.mu.Unlock()
c.out, c.terminal = w, terminal
}
// consoleHandler is a slog.Handler writing the text format the runner has always used, so that
// moving off logrus changes nothing a user can see: sorted key=value fields on a plain stream,
// and a colored prefix with elapsed or full time on a terminal.
type consoleHandler struct {
c *console
attrs []slog.Attr
}
func (h *consoleHandler) Enabled(_ context.Context, level slog.Level) bool {
return level >= h.c.level.Level()
}
func (h *consoleHandler) WithAttrs(attrs []slog.Attr) slog.Handler {
return &consoleHandler{c: h.c, attrs: append(slices.Clone(h.attrs), attrs...)}
}
// WithGroup is a no-op, as only NewSlogHandler wraps a consoleHandler and it never groups.
func (h *consoleHandler) WithGroup(string) slog.Handler { return h }
func (h *consoleHandler) Handle(_ context.Context, r slog.Record) error {
c := h.c
c.mu.Lock()
opts, reportCaller, terminal := c.opts, c.reportCaller, c.terminal
c.mu.Unlock()
data := make(map[string]any, len(h.attrs)+r.NumAttrs())
for _, a := range h.attrs {
data[a.Key] = a.Value.Resolve().Any()
}
r.Attrs(func(a slog.Attr) bool {
data[a.Key] = a.Value.Resolve().Any()
return true
})
var caller *runtime.Frame
if reportCaller && r.PC != 0 {
frame, _ := runtime.CallersFrames([]uintptr{r.PC}).Next()
caller = &frame
}
prefixFieldClashes(data, caller != nil)
keys := make([]string, 0, len(data))
for k := range data {
keys = append(keys, k)
}
slices.Sort(keys)
var b bytes.Buffer
if terminal && !opts.DisableColors {
printColored(&b, opts, r, caller, keys, data)
} else {
printPlain(&b, opts, r, caller, keys, data)
}
line := b.Bytes()
if opts.Filter != nil {
line = opts.Filter(line)
}
c.mu.Lock()
defer c.mu.Unlock()
_, err := c.out.Write(line)
return err
}
// prefixFieldClashes moves fields that share a name with a fixed key out of its way.
func prefixFieldClashes(data map[string]any, reportCaller bool) {
for _, key := range []string{"time", "level", "msg"} {
if v, ok := data[key]; ok {
data["fields."+key] = v
delete(data, key)
}
}
if reportCaller {
for _, key := range []string{"func", "file"} {
if v, ok := data[key]; ok {
data["fields."+key] = v
}
}
}
}
func printPlain(b *bytes.Buffer, opts TextOptions, r slog.Record, caller *runtime.Frame, keys []string, data map[string]any) {
if !opts.DisableTimestamp {
appendKeyValue(b, "time", r.Time.Format(time.RFC3339))
}
appendKeyValue(b, "level", plainLevel(Level(r.Level)))
if r.Message != "" {
appendKeyValue(b, "msg", r.Message)
}
if caller != nil {
funcVal, fileVal := callerText(opts, caller)
if funcVal != "" {
appendKeyValue(b, "func", funcVal)
}
if fileVal != "" {
appendKeyValue(b, "file", fileVal)
}
}
for _, k := range keys {
appendKeyValue(b, k, data[k])
}
b.WriteByte('\n')
}
func callerText(opts TextOptions, caller *runtime.Frame) (funcVal, fileVal string) {
if opts.CallerPrettyfier != nil {
return opts.CallerPrettyfier(caller)
}
return caller.Function, caller.File + ":" + strconv.Itoa(caller.Line)
}
func printColored(b *bytes.Buffer, opts TextOptions, r slog.Record, caller *runtime.Frame, keys []string, data map[string]any) {
msg := strings.TrimSuffix(r.Message, "\n")
var callerStr string
if caller != nil {
funcVal, fileVal := callerText(opts, caller)
if opts.CallerPrettyfier == nil && caller.Function != "" {
funcVal += "()"
}
switch {
case fileVal == "":
callerStr = funcVal
case funcVal == "":
callerStr = fileVal
default:
callerStr = fileVal + " " + funcVal
}
}
level := Level(r.Level)
prefix := colorize(level, strings.ToUpper(plainLevel(level))[:4])
switch {
case opts.DisableTimestamp:
fmt.Fprintf(b, "%s%s %-44s ", prefix, callerStr, msg)
case !opts.FullTimestamp:
fmt.Fprintf(b, "%s[%04d]%s %-44s ", prefix, int(r.Time.Sub(consoleStart)/time.Second), callerStr, msg)
default:
fmt.Fprintf(b, "%s[%s]%s %-44s ", prefix, r.Time.Format(time.RFC3339), callerStr, msg)
}
for _, k := range keys {
b.WriteByte(' ')
b.WriteString(colorize(level, k))
b.WriteByte('=')
appendValue(b, data[k])
}
b.WriteByte('\n')
}
// plainLevel spells Warn the way logrus did, as "warning".
func plainLevel(l Level) string {
if s := l.String(); s != "warn" {
return s
}
return "warning"
}
func colorize(l Level, s string) string {
var color string
switch {
case l <= LevelTrace:
color = ansiDimWhite
case l <= LevelDebug:
color = ansiDimCyan
case l <= LevelInfo:
color = ansiCyan
case l <= LevelWarn:
color = ansiYellow
default:
color = ansiRed
}
return color + s + ansiReset
}
func appendKeyValue(b *bytes.Buffer, key string, value any) {
if b.Len() > 0 {
b.WriteByte(' ')
}
b.WriteString(key)
b.WriteByte('=')
appendValue(b, value)
}
func appendValue(b *bytes.Buffer, value any) {
switch v := value.(type) {
case string:
appendString(b, v)
case []byte:
appendString(b, string(v))
case error:
appendFormatted(b, v, v.Error)
case fmt.Stringer:
appendFormatted(b, v, v.String)
default:
appendString(b, fmt.Sprint(value))
}
}
// appendFormatted writes what get returns, and "<nil>" if that panics on a nil receiver.
func appendFormatted(b *bytes.Buffer, v any, get func() string) {
defer func() {
if r := recover(); r != nil {
if rv := reflect.ValueOf(v); rv.Kind() == reflect.Pointer && rv.IsNil() {
appendString(b, "<nil>")
} else {
appendString(b, fmt.Sprintf("%%!v(PANIC=%v)", r))
}
}
}()
appendString(b, get())
}
func appendString(b *bytes.Buffer, s string) {
if needsQuoting(s) {
b.WriteString(strconv.Quote(s))
return
}
b.WriteString(s)
}
func needsQuoting(s string) bool {
for i := range len(s) {
c := s[i]
switch {
case c >= 'a' && c <= 'z', c >= 'A' && c <= 'Z', c >= '0' && c <= '9':
case c == '-', c == '.', c == '_', c == '/', c == '@', c == '^', c == '+':
default:
return true
}
}
return false
}

View File

@@ -0,0 +1,78 @@
// Copyright 2026 The Gitea Authors. All rights reserved.
// SPDX-License-Identifier: MIT
package logging
import (
"bytes"
"errors"
"log/slog"
"runtime"
"strings"
"testing"
"time"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
func consoleLine(t *testing.T, opts TextOptions, terminal, reportCaller bool, build func(*Logger)) string {
t.Helper()
var out bytes.Buffer
c := newConsole(&out)
c.opts, c.terminal, c.reportCaller = opts, terminal, reportCaller
c.level.Set(slog.LevelDebug)
build(New(NewSlogHandler(&consoleHandler{c: c})))
return out.String()
}
func TestConsolePlain(t *testing.T) {
line := consoleLine(t, TextOptions{DisableTimestamp: true}, false, false, func(l *Logger) {
l.With("job", "build x", "n", 3).WithError(errors.New("boom")).Warnf("hello world")
l.With("msg", "clash", "empty", "").Info("")
})
assert.Equal(t, "level=warning msg=\"hello world\" error=boom job=\"build x\" n=3\n"+
"level=info empty= fields.msg=clash\n", line)
}
func TestConsolePlainTimeFilterAndCaller(t *testing.T) {
opts := TextOptions{
CallerPrettyfier: func(f *runtime.Frame) (string, string) {
return "[fn]", "[" + strings.TrimPrefix(f.Function, "x") + "]"
},
Filter: func(b []byte) []byte { return bytes.ReplaceAll(b, []byte("secret"), []byte("***")) },
}
line := consoleLine(t, opts, false, true, func(l *Logger) { l.With("k", "secret").Debugf("a secret") })
assert.Regexp(t, `^time="[^"]+" level=debug msg="a \*\*\*" func="\[fn\]" file="\[.*TestConsolePlainTimeFilterAndCaller[^"]*\]" k=\*\*\*\n$`, line)
}
func TestConsoleColored(t *testing.T) {
line := consoleLine(t, TextOptions{DisableTimestamp: true}, true, false, func(l *Logger) {
l.With("job", "j").Warn("careful")
})
assert.Equal(t, "\x1b[33mWARN\x1b[0m "+"careful"+strings.Repeat(" ", 44-len("careful"))+" \x1b[33mjob\x1b[0m=j\n", line)
full := consoleLine(t, TextOptions{FullTimestamp: true}, true, false, func(l *Logger) { l.Error("bad") })
assert.Regexp(t, `^\x1b\[31mERRO\x1b\[0m\[\d{4}-\d\d-\d\dT[^\]]+\] bad`, full)
elapsed := consoleLine(t, TextOptions{}, true, false, func(l *Logger) { l.Info("x") })
assert.Regexp(t, `^\x1b\[36mINFO\x1b\[0m\[\d{4}\] x`, elapsed)
plain := consoleLine(t, TextOptions{DisableColors: true, DisableTimestamp: true}, true, false, func(l *Logger) { l.Info("x") })
assert.Equal(t, "level=info msg=x\n", plain)
}
func TestConsoleLevelAndOutput(t *testing.T) {
var out bytes.Buffer
c := newConsole(&out)
h := &consoleHandler{c: c}
assert.False(t, h.Enabled(t.Context(), slog.LevelDebug)) // info unless raised
c.level.Set(slog.LevelError)
assert.False(t, h.Enabled(t.Context(), slog.LevelWarn))
assert.True(t, h.Enabled(t.Context(), slog.LevelError))
rec := slog.NewRecord(time.Unix(0, 0), slog.LevelError, "m", 0)
rec.AddAttrs(slog.String("c", "3"))
require.NoError(t, h.WithAttrs([]slog.Attr{slog.Int("a", 1)}).Handle(t.Context(), rec))
assert.Contains(t, out.String(), "a=1 c=3")
}

View File

@@ -0,0 +1,91 @@
// Copyright 2026 The Gitea Authors. All rights reserved.
// SPDX-License-Identifier: MIT
package logging
import (
"io"
"log/slog"
"os"
"runtime"
"sync/atomic"
"github.com/sirupsen/logrus"
)
// TextOptions describes how the process-wide Logger writes a record as text.
type TextOptions struct {
DisableColors bool
DisableTimestamp bool
FullTimestamp bool
// CallerPrettyfier renames the function and file reported with a record when ReportCaller is on.
CallerPrettyfier func(*runtime.Frame) (function, file string)
// Filter rewrites each formatted line, for example to mask secrets.
Filter func(line []byte) []byte
}
var (
std atomic.Pointer[Logger]
stdState = newConsole(os.Stderr)
)
func init() {
std.Store(New(NewSlogHandler(&consoleHandler{c: stdState})))
}
// Default returns the process-wide Logger.
func Default() *Logger { return std.Load() }
// SetDefault replaces the process-wide Logger, which tests use to capture what the package
// functions log. SetLevel, GetLevel, SetOutput and the format functions keep configuring the
// initial console Logger.
func SetDefault(l *Logger) { std.Store(l) }
// SetLevel sets the minimum level of the initial process-wide Logger and of what libraries log through logrus.
func SetLevel(level Level) {
stdState.level.Set(slog.Level(level))
logrus.SetLevel(toLogrusLevel(level))
}
// GetLevel returns the minimum level of the initial process-wide Logger.
func GetLevel() Level { return Level(stdState.level.Level()) }
// SetOutput sets where the initial process-wide Logger writes.
func SetOutput(w io.Writer) { stdState.setOutput(w) }
// Output returns where the initial process-wide Logger writes.
func Output() io.Writer {
stdState.mu.Lock()
defer stdState.mu.Unlock()
return stdState.out
}
// SetTextFormat sets the text format of the initial process-wide Logger.
func SetTextFormat(o TextOptions) {
stdState.mu.Lock()
defer stdState.mu.Unlock()
stdState.opts = o
}
// SetReportCaller makes the initial process-wide Logger record where each call came from, which costs time.
func SetReportCaller(on bool) {
stdState.mu.Lock()
defer stdState.mu.Unlock()
stdState.reportCaller = on
logrus.SetReportCaller(on)
}
func Enabled(level Level) bool { return Default().Enabled(level) }
func With(args ...any) *Logger { return Default().With(args...) }
func WithError(err error) *Logger { return Default().WithError(err) }
func Tracef(format string, args ...any) { Default().logf(LevelTrace, format, args) }
func Debugf(format string, args ...any) { Default().logf(LevelDebug, format, args) }
func Infof(format string, args ...any) { Default().logf(LevelInfo, format, args) }
func Warnf(format string, args ...any) { Default().logf(LevelWarn, format, args) }
func Errorf(format string, args ...any) { Default().logf(LevelError, format, args) }
func Trace(args ...any) { Default().log(LevelTrace, args) }
func Debug(args ...any) { Default().log(LevelDebug, args) }
func Info(args ...any) { Default().log(LevelInfo, args) }
func Warn(args ...any) { Default().log(LevelWarn, args) }
func Error(args ...any) { Default().log(LevelError, args) }

View File

@@ -0,0 +1,114 @@
// Copyright 2026 The Gitea Authors. All rights reserved.
// SPDX-License-Identifier: MIT
package logging
import (
"context"
"errors"
"slices"
"time"
)
// Attr is a structured key/value pair attached to a log record.
type Attr struct {
Key string
Value any
}
// Record is one log event. Attrs holds the call-site attributes only, the ones added with
// Logger.With are accumulated by the Handler.
type Record struct {
Time time.Time
Level Level
Message string
PC uintptr // program counter of the logging call, 0 if unknown
Attrs []Attr
}
// Attr returns the value of the last attribute named key.
func (r Record) Attr(key string) (any, bool) {
for _, a := range slices.Backward(r.Attrs) {
if a.Key == key {
return a.Value, true
}
}
return nil, false
}
// Handler consumes records, and is shaped like slog.Handler so that a slog backend can replace
// the current one without touching callers. Handle must not modify the Attrs slice it is given.
type Handler interface {
Enabled(ctx context.Context, level Level) bool
Handle(ctx context.Context, r Record) error
// WithAttrs returns a Handler that adds attrs to every record it handles.
WithAttrs(attrs []Attr) Handler
}
type multiHandler []Handler
// NewMultiHandler fans every record out to all handlers that have its level enabled.
func NewMultiHandler(handlers ...Handler) Handler {
return multiHandler(handlers)
}
func (m multiHandler) Enabled(ctx context.Context, level Level) bool {
for _, h := range m {
if h.Enabled(ctx, level) {
return true
}
}
return false
}
func (m multiHandler) Handle(ctx context.Context, r Record) error {
var errs []error
for _, h := range m {
if h.Enabled(ctx, r.Level) {
errs = append(errs, h.Handle(ctx, r))
}
}
return errors.Join(errs...)
}
func (m multiHandler) WithAttrs(attrs []Attr) Handler {
out := make(multiHandler, len(m))
for i, h := range m {
out[i] = h.WithAttrs(attrs)
}
return out
}
type discardHandler struct{}
// DiscardHandler drops every record.
var DiscardHandler Handler = discardHandler{}
func (discardHandler) Enabled(context.Context, Level) bool { return false }
func (discardHandler) Handle(context.Context, Record) error { return nil }
func (d discardHandler) WithAttrs([]Attr) Handler { return d }
type levelHandler struct {
h Handler
minLevel Level
}
// NewLevelHandler passes only records at or above minLevel on to h.
func NewLevelHandler(h Handler, minLevel Level) Handler {
return levelHandler{h: h, minLevel: minLevel}
}
func (l levelHandler) Enabled(ctx context.Context, level Level) bool {
return level >= l.minLevel && l.h.Enabled(ctx, level)
}
func (l levelHandler) Handle(ctx context.Context, r Record) error {
if r.Level < l.minLevel {
return nil
}
return l.h.Handle(ctx, r)
}
func (l levelHandler) WithAttrs(attrs []Attr) Handler {
return levelHandler{h: l.h.WithAttrs(attrs), minLevel: l.minLevel}
}

View File

@@ -0,0 +1,53 @@
// Copyright 2026 The Gitea Authors. All rights reserved.
// SPDX-License-Identifier: MIT
package logging
import (
"fmt"
"strings"
)
// Level is a log severity. The values match log/slog, with Trace added below Debug.
type Level int
const (
LevelTrace Level = -8
LevelDebug Level = -4
LevelInfo Level = 0
LevelWarn Level = 4
LevelError Level = 8
)
func (l Level) String() string {
switch {
case l <= LevelTrace:
return "trace"
case l <= LevelDebug:
return "debug"
case l <= LevelInfo:
return "info"
case l <= LevelWarn:
return "warn"
default:
return "error"
}
}
// ParseLevel accepts every name logrus does. "fatal" and "panic" map to LevelError because
// the Logger has no level above it.
func ParseLevel(s string) (Level, error) {
switch strings.ToLower(s) {
case "trace":
return LevelTrace, nil
case "debug":
return LevelDebug, nil
case "info":
return LevelInfo, nil
case "warn", "warning":
return LevelWarn, nil
case "error", "fatal", "panic":
return LevelError, nil
}
return LevelInfo, fmt.Errorf("not a valid log level: %q", s)
}

View File

@@ -0,0 +1,106 @@
// Copyright 2026 The Gitea Authors. All rights reserved.
// SPDX-License-Identifier: MIT
package logging
import (
"context"
"fmt"
"runtime"
"time"
)
const badKey = "!BADKEY"
// Logger writes printf-style messages with structured attributes to a Handler.
type Logger struct {
h Handler
ctx context.Context
}
func New(h Handler) *Logger {
return &Logger{h: h, ctx: context.Background()}
}
// With returns a Logger that adds the key/value pairs to every record, like slog.Logger.With.
func (l *Logger) With(args ...any) *Logger {
if len(args) == 0 {
return l
}
return &Logger{h: l.h.WithAttrs(argsToAttrs(args)), ctx: l.ctx}
}
func (l *Logger) WithError(err error) *Logger {
return l.With("error", err)
}
// WithContext returns a Logger that passes ctx to its Handler.
func (l *Logger) WithContext(ctx context.Context) *Logger {
return &Logger{h: l.h, ctx: ctx}
}
func (l *Logger) Enabled(level Level) bool {
return l.h.Enabled(l.ctx, level)
}
func (l *Logger) Tracef(format string, args ...any) { l.logf(LevelTrace, format, args) }
func (l *Logger) Debugf(format string, args ...any) { l.logf(LevelDebug, format, args) }
func (l *Logger) Infof(format string, args ...any) { l.logf(LevelInfo, format, args) }
func (l *Logger) Warnf(format string, args ...any) { l.logf(LevelWarn, format, args) }
func (l *Logger) Errorf(format string, args ...any) { l.logf(LevelError, format, args) }
// Trace and the methods below format their arguments like fmt.Sprint.
func (l *Logger) Trace(args ...any) { l.log(LevelTrace, args) }
func (l *Logger) Debug(args ...any) { l.log(LevelDebug, args) }
func (l *Logger) Info(args ...any) { l.log(LevelInfo, args) }
func (l *Logger) Warn(args ...any) { l.log(LevelWarn, args) }
func (l *Logger) Error(args ...any) { l.log(LevelError, args) }
func (l *Logger) logf(level Level, format string, args []any) {
if !l.Enabled(level) {
return
}
l.emit(callerPC(4), level, fmt.Sprintf(format, args...))
}
func (l *Logger) log(level Level, args []any) {
if !l.Enabled(level) {
return
}
l.emit(callerPC(4), level, fmt.Sprint(args...))
}
// emit drops the Handler error, as a logger has nowhere to report it.
func (l *Logger) emit(pc uintptr, level Level, msg string) {
_ = l.h.Handle(l.ctx, Record{Time: time.Now(), Level: level, Message: msg, PC: pc})
}
// callerPC returns the program counter skip frames up, 4 being the caller of an exported method.
func callerPC(skip int) uintptr {
var pcs [1]uintptr
runtime.Callers(skip, pcs[:])
return pcs[0]
}
func argsToAttrs(args []any) []Attr {
attrs := make([]Attr, 0, len(args)/2+1)
for len(args) > 0 {
switch k := args[0].(type) {
case Attr:
attrs = append(attrs, k)
args = args[1:]
case string:
if len(args) == 1 {
attrs = append(attrs, Attr{Key: badKey, Value: k})
args = nil
} else {
attrs = append(attrs, Attr{Key: k, Value: args[1]})
args = args[2:]
}
default:
attrs = append(attrs, Attr{Key: badKey, Value: k})
args = args[1:]
}
}
return attrs
}

View File

@@ -0,0 +1,203 @@
// Copyright 2026 The Gitea Authors. All rights reserved.
// SPDX-License-Identifier: MIT
package logging_test
import (
"errors"
"log/slog"
"runtime"
"strings"
"testing"
"gitea.com/gitea/runner/act/common/logging"
"gitea.com/gitea/runner/act/common/logging/logtest"
"github.com/sirupsen/logrus"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
func TestParseLevel(t *testing.T) {
for in, want := range map[string]logging.Level{
"trace": logging.LevelTrace, "DEBUG": logging.LevelDebug, "info": logging.LevelInfo,
"warn": logging.LevelWarn, "warning": logging.LevelWarn,
"error": logging.LevelError, "fatal": logging.LevelError, "panic": logging.LevelError,
} {
got, err := logging.ParseLevel(in)
require.NoError(t, err, in)
assert.Equal(t, want, got, in)
}
_, err := logging.ParseLevel("loud")
require.Error(t, err)
assert.Equal(t, "warn", logging.LevelWarn.String())
}
type countingStringer struct{ n *int }
func (c countingStringer) String() string { *c.n++; return "x" }
func TestLogger(t *testing.T) {
h := logtest.NewWithLevel(logging.LevelTrace)
l := logging.New(h)
l.With("job", "j1").WithError(errors.New("boom")).Warnf("step %d", 2)
l.Info("a", 1, 2, "b")
l.With("k").With(7).Debug("x")
recs := h.AllRecords()
require.Len(t, recs, 3)
assert.Equal(t, "step 2", recs[0].Message)
assert.Equal(t, logging.LevelWarn, recs[0].Level)
assert.False(t, recs[0].Time.IsZero())
job, _ := recs[0].Attr("job")
assert.Equal(t, "j1", job)
errv, _ := recs[0].Attr("error")
require.EqualError(t, errv.(error), "boom")
assert.Equal(t, "a1 2b", recs[1].Message) // fmt.Sprint spacing
bad, _ := recs[2].Attr("!BADKEY")
assert.Equal(t, 7, bad)
h.Reset()
assert.Nil(t, h.LastRecord())
}
func TestLoggerSkipsFormattingWhenDisabled(t *testing.T) {
n := 0
l := logging.New(logging.NewLevelHandler(logtest.NewWithLevel(logging.LevelTrace), logging.LevelWarn))
l.Debugf("%v", countingStringer{&n})
assert.Zero(t, n)
assert.False(t, l.Enabled(logging.LevelInfo))
}
func TestMultiHandler(t *testing.T) {
assert.False(t, logging.DiscardHandler.Enabled(t.Context(), logging.LevelError))
all, warnOnly := logtest.New(), logtest.NewWithLevel(logging.LevelTrace)
l := logging.New(logging.NewMultiHandler(all, logging.NewLevelHandler(warnOnly, logging.LevelWarn), logging.DiscardHandler)).With("a", 1)
l.Info("i")
l.Error("e")
assert.Len(t, all.AllRecords(), 2)
require.Len(t, warnOnly.AllRecords(), 1)
v, _ := warnOnly.LastRecord().Attr("a")
assert.Equal(t, 1, v)
}
func frameOf(t *testing.T, pc uintptr) runtime.Frame {
t.Helper()
require.NotZero(t, pc)
f, _ := runtime.CallersFrames([]uintptr{pc}).Next()
return f
}
func TestCallerPC(t *testing.T) {
l, h := logtest.NewLogger()
l.Infof("method")
l.Info("method")
prev := logging.Default()
t.Cleanup(func() { logging.SetDefault(prev) })
logging.SetDefault(l)
logging.Infof("global")
logging.Error("global")
require.Len(t, h.AllRecords(), 4)
for _, r := range h.AllRecords() {
assert.True(t, strings.HasSuffix(frameOf(t, r.PC).Function, "TestCallerPC"), r.Message)
}
}
func TestDefaultLevel(t *testing.T) {
prev := logging.GetLevel()
t.Cleanup(func() { logging.SetLevel(prev) })
logging.SetLevel(logging.LevelWarn)
assert.Equal(t, logging.LevelWarn, logging.GetLevel())
assert.True(t, logging.Enabled(logging.LevelError))
assert.False(t, logging.Enabled(logging.LevelInfo))
}
func TestSlogHandler(t *testing.T) {
var buf strings.Builder
slogger := slog.New(slog.NewTextHandler(&buf, &slog.HandlerOptions{Level: slog.LevelDebug}))
l := logging.New(logging.NewSlogHandler(slogger.Handler())).With("job", "j1")
l.Tracef("dropped")
assert.False(t, l.Enabled(logging.LevelTrace))
l.WithError(errors.New("boom")).Errorf("fail %s", "x")
out := buf.String()
assert.Contains(t, out, "level=ERROR")
assert.Contains(t, out, `msg="fail x"`)
assert.Contains(t, out, "job=j1")
assert.Contains(t, out, "error=boom")
}
func TestLogrusBridge(t *testing.T) {
prevLevel, prevDefault := logging.GetLevel(), logging.Default()
t.Cleanup(func() {
logging.SetLevel(prevLevel)
logging.SetReportCaller(false)
logging.SetDefault(prevDefault)
})
logging.SetLevel(logging.LevelInfo)
assert.False(t, logrus.IsLevelEnabled(logrus.DebugLevel))
logging.SetLevel(logging.LevelTrace)
logging.SetReportCaller(true)
h := logtest.NewWithLevel(logging.LevelTrace)
logging.SetDefault(logging.New(h))
logrus.WithField("lib", "docker").Warnf("from a library %d", 1)
_, _, line, _ := runtime.Caller(0)
logrus.Debug("below")
logrus.Trace("lowest")
recs := h.AllRecords()
require.Len(t, recs, 3)
assert.Equal(t, "from a library 1", recs[0].Message)
assert.Equal(t, logging.LevelWarn, recs[0].Level)
v, _ := recs[0].Attr("lib")
assert.Equal(t, "docker", v)
assert.Equal(t, logging.LevelDebug, recs[1].Level)
f := frameOf(t, recs[1].PC)
assert.True(t, strings.HasSuffix(f.Function, "TestLogrusBridge"), f.Function)
assert.Equal(t, line+1, f.Line)
assert.Equal(t, logging.LevelTrace, recs[2].Level)
h = logtest.New() // logrus passes Debug, the Logger does not
logging.SetDefault(logging.New(h))
logrus.Debug("dropped")
assert.Empty(t, h.AllRecords())
}
func TestSlogBridge(t *testing.T) {
h := logtest.SetDefault(t)
slog.With("lib", "actionslib").Warn("from a library", "n", 1)
_, _, line, _ := runtime.Caller(0)
slog.Debug("dropped")
recs := h.AllRecords()
require.Len(t, recs, 1)
assert.Equal(t, "from a library", recs[0].Message)
assert.Equal(t, logging.LevelWarn, recs[0].Level)
lib, _ := recs[0].Attr("lib")
assert.Equal(t, "actionslib", lib)
n, _ := recs[0].Attr("n")
assert.EqualValues(t, 1, n)
assert.Equal(t, line-1, frameOf(t, recs[0].PC).Line)
}
func TestLogrusBridgeIntact(t *testing.T) {
assert.True(t, logging.LogrusBridgeIntact())
prev := logrus.StandardLogger().Out
t.Cleanup(func() { logrus.SetOutput(prev) })
logrus.SetOutput(&strings.Builder{}) // what a dependency doing this would do
assert.False(t, logging.LogrusBridgeIntact())
logrus.SetOutput(prev)
assert.True(t, logging.LogrusBridgeIntact())
prevLevel := logrus.GetLevel()
t.Cleanup(func() { logrus.SetLevel(prevLevel) })
logrus.SetLevel(logrus.PanicLevel)
assert.False(t, logging.LogrusBridgeIntact())
}

View File

@@ -0,0 +1,102 @@
// Copyright 2026 The Gitea Authors. All rights reserved.
// SPDX-License-Identifier: MIT
package logging
import (
"context"
"io"
"github.com/sirupsen/logrus"
)
// Libraries the runner depends on, such as docker/cli and moby/go-archive, log through
// logrus's global logger. Hand what they log to the process-wide Logger, so it follows the same
// level, format and secret masking as the rest of the output.
func init() {
logrus.SetOutput(io.Discard)
logrus.SetFormatter(discardFormatter{})
logrus.SetLevel(toLogrusLevel(GetLevel()))
logrus.AddHook(bridgeHook{})
}
type discardFormatter struct{}
func (discardFormatter) Format(*logrus.Entry) ([]byte, error) { return nil, nil }
type bridgeHook struct{}
func (bridgeHook) Levels() []logrus.Level { return logrus.AllLevels }
func (bridgeHook) Fire(e *logrus.Entry) error {
l := Default()
level := fromLogrusLevel(e.Level)
ctx := e.Context
if ctx == nil {
ctx = context.Background()
}
if !l.h.Enabled(ctx, level) {
return nil
}
attrs := make([]Attr, 0, len(e.Data))
for k, v := range e.Data {
attrs = append(attrs, Attr{Key: k, Value: v})
}
r := Record{Time: e.Time, Level: level, Message: e.Message, Attrs: attrs}
if e.Caller != nil {
r.PC = e.Caller.PC + 1 // Frame.PC is one below the return address Record.PC holds
}
return l.h.Handle(ctx, r)
}
func toLogrusLevel(l Level) logrus.Level {
switch {
case l <= LevelTrace:
return logrus.TraceLevel
case l <= LevelDebug:
return logrus.DebugLevel
case l <= LevelInfo:
return logrus.InfoLevel
case l <= LevelWarn:
return logrus.WarnLevel
default:
return logrus.ErrorLevel
}
}
func fromLogrusLevel(l logrus.Level) Level {
switch l {
case logrus.TraceLevel:
return LevelTrace
case logrus.DebugLevel:
return LevelDebug
case logrus.InfoLevel:
return LevelInfo
case logrus.WarnLevel:
return LevelWarn
default:
return LevelError
}
}
// LogrusBridgeIntact reports whether logrus's global logger is still set up as the bridge needs,
// discarding its own output and following the process-wide level. A dependency that calls
// logrus.SetOutput or SetLevel breaks that, and its lines then show up twice, or at the wrong level.
func LogrusBridgeIntact() bool {
std := logrus.StandardLogger()
if std.Out != io.Discard || std.GetLevel() != toLogrusLevel(GetLevel()) {
return false
}
for _, level := range logrus.AllLevels {
found := false
for _, h := range std.Hooks[level] {
if _, ok := h.(bridgeHook); ok {
found = true
}
}
if !found {
return false
}
}
return true
}

View File

@@ -0,0 +1,89 @@
// Copyright 2026 The Gitea Authors. All rights reserved.
// SPDX-License-Identifier: MIT
// Package logtest provides a Handler that keeps records in memory for assertions.
package logtest
import (
"context"
"slices"
"sync"
"testing"
"gitea.com/gitea/runner/act/common/logging"
)
type store struct {
mu sync.Mutex
records []logging.Record
}
// Handler records what it receives at or above its level. Loggers derived with WithAttrs share one store.
type Handler struct {
store *store
attrs []logging.Attr
minLevel logging.Level
}
// New records from LevelInfo up, like the logger of logrus's hooks/test.
func New() *Handler {
return NewWithLevel(logging.LevelInfo)
}
// NewLogger returns a Logger writing to a new Handler from New, and that Handler.
func NewLogger() (*logging.Logger, *Handler) {
h := New()
return logging.New(h), h
}
func NewWithLevel(minLevel logging.Level) *Handler {
return &Handler{store: &store{}, minLevel: minLevel}
}
func (h *Handler) Enabled(_ context.Context, level logging.Level) bool { return level >= h.minLevel }
// Handle stores the record with the attributes accumulated by WithAttrs placed first.
func (h *Handler) Handle(_ context.Context, r logging.Record) error {
r.Attrs = append(slices.Clone(h.attrs), r.Attrs...)
h.store.mu.Lock()
defer h.store.mu.Unlock()
h.store.records = append(h.store.records, r)
return nil
}
func (h *Handler) WithAttrs(attrs []logging.Attr) logging.Handler {
return &Handler{store: h.store, attrs: append(slices.Clone(h.attrs), attrs...), minLevel: h.minLevel}
}
func (h *Handler) AllRecords() []logging.Record {
h.store.mu.Lock()
defer h.store.mu.Unlock()
return slices.Clone(h.store.records)
}
// LastRecord returns nil when nothing was logged.
func (h *Handler) LastRecord() *logging.Record {
h.store.mu.Lock()
defer h.store.mu.Unlock()
if len(h.store.records) == 0 {
return nil
}
r := h.store.records[len(h.store.records)-1]
return &r
}
func (h *Handler) Reset() {
h.store.mu.Lock()
defer h.store.mu.Unlock()
h.store.records = nil
}
// SetDefault makes the process-wide Logger record into a new Handler until the test ends.
func SetDefault(t testing.TB) *Handler {
t.Helper()
prev := logging.Default()
logger, h := NewLogger()
logging.SetDefault(logger)
t.Cleanup(func() { logging.SetDefault(prev) })
return h
}

View File

@@ -0,0 +1,81 @@
// Copyright 2026 The Gitea Authors. All rights reserved.
// SPDX-License-Identifier: MIT
package logging
import (
"context"
"log/slog"
"slices"
)
func init() {
slog.SetDefault(slog.New(slogBridge{}))
}
// slogBridge hands what libraries log through slog's global logger to the process-wide Logger.
type slogBridge struct {
attrs []Attr
}
func (b slogBridge) Enabled(ctx context.Context, level slog.Level) bool {
return Default().h.Enabled(ctx, Level(level))
}
func (b slogBridge) Handle(ctx context.Context, r slog.Record) error {
h := Default().h
if len(b.attrs) > 0 {
h = h.WithAttrs(b.attrs)
}
attrs := make([]Attr, 0, r.NumAttrs())
r.Attrs(func(a slog.Attr) bool {
attrs = append(attrs, fromSlogAttr(a))
return true
})
return h.Handle(ctx, Record{Time: r.Time, Level: Level(r.Level), Message: r.Message, PC: r.PC, Attrs: attrs})
}
func (b slogBridge) WithAttrs(attrs []slog.Attr) slog.Handler {
out := slices.Clone(b.attrs)
for _, a := range attrs {
out = append(out, fromSlogAttr(a))
}
return slogBridge{attrs: out}
}
func (b slogBridge) WithGroup(string) slog.Handler { return b }
func fromSlogAttr(a slog.Attr) Attr {
return Attr{Key: a.Key, Value: a.Value.Resolve().Any()}
}
type slogHandler struct {
h slog.Handler
}
// NewSlogHandler writes to h. Levels map one to one, as Level uses slog's values.
func NewSlogHandler(h slog.Handler) Handler {
return slogHandler{h: h}
}
func (s slogHandler) Enabled(ctx context.Context, level Level) bool {
return s.h.Enabled(ctx, slog.Level(level))
}
func (s slogHandler) Handle(ctx context.Context, r Record) error {
rec := slog.NewRecord(r.Time, slog.Level(r.Level), r.Message, r.PC)
rec.AddAttrs(toSlogAttrs(r.Attrs)...)
return s.h.Handle(ctx, rec)
}
func (s slogHandler) WithAttrs(attrs []Attr) Handler {
return slogHandler{h: s.h.WithAttrs(toSlogAttrs(attrs))}
}
func toSlogAttrs(attrs []Attr) []slog.Attr {
out := make([]slog.Attr, len(attrs))
for i, a := range attrs {
out[i] = slog.Any(a.Key, a.Value)
}
return out
}

View File

@@ -0,0 +1,20 @@
// Copyright 2026 The Gitea Authors. All rights reserved.
// SPDX-License-Identifier: MIT
//go:build !windows
package logging
import (
"io"
"os"
"golang.org/x/term"
)
func checkIfTerminal(w io.Writer) bool {
if f, ok := w.(*os.File); ok {
return term.IsTerminal(int(f.Fd()))
}
return false
}

View File

@@ -0,0 +1,27 @@
// Copyright 2026 The Gitea Authors. All rights reserved.
// SPDX-License-Identifier: MIT
//go:build windows
package logging
import (
"io"
"os"
"golang.org/x/sys/windows"
)
// checkIfTerminal also turns on virtual terminal processing, which the console needs to show colors.
func checkIfTerminal(w io.Writer) bool {
f, ok := w.(*os.File)
if !ok {
return false
}
handle := windows.Handle(f.Fd())
var mode uint32
if err := windows.GetConsoleMode(handle, &mode); err != nil {
return false
}
return windows.SetConsoleMode(handle, mode|windows.ENABLE_VIRTUAL_TERMINAL_PROCESSING) == nil
}

View File

@@ -10,6 +10,11 @@ import (
"strings"
)
type interfaceIP struct {
net.IP
net.Interface
}
// GetOutboundIP returns an outbound IP address of this machine.
// It tries to access the internet and returns the local IP address of the connection.
// If the machine cannot access the internet, it returns a preferred IP address from network interfaces.
@@ -26,11 +31,7 @@ func GetOutboundIP() net.IP {
// So the machine cannot access the internet. Pick an IP address from network interfaces.
if ifs, err := net.Interfaces(); err == nil {
type IP struct {
net.IP
net.Interface
}
var ips []IP
var ips []interfaceIP
for _, i := range ifs {
if addrs, err := i.Addrs(); err == nil {
for _, addr := range addrs {
@@ -42,40 +43,45 @@ func GetOutboundIP() net.IP {
ip = v.IP
}
if ip.IsGlobalUnicast() {
ips = append(ips, IP{ip, i})
ips = append(ips, interfaceIP{ip, i})
}
}
}
}
if len(ips) > 1 {
sort.Slice(ips, func(i, j int) bool {
ifi := ips[i].Interface
ifj := ips[j].Interface
// ethernet is preferred
if vi, vj := strings.HasPrefix(ifi.Name, "e"), strings.HasPrefix(ifj.Name, "e"); vi != vj {
return vi
}
ipi := ips[i].IP
ipj := ips[j].IP
// IPv4 is preferred
if vi, vj := ipi.To4() != nil, ipj.To4() != nil; vi != vj {
return vi
}
// en0 is preferred to en1
if ifi.Name != ifj.Name {
return ifi.Name < ifj.Name
}
// fallback
return ipi.String() < ipj.String()
})
return ips[0].IP
}
return preferredIP(ips)
}
return nil
}
func preferredIP(ips []interfaceIP) net.IP {
if len(ips) == 0 {
return nil
}
sort.Slice(ips, func(i, j int) bool {
ifi := ips[i].Interface
ifj := ips[j].Interface
// ethernet is preferred
if vi, vj := strings.HasPrefix(ifi.Name, "e"), strings.HasPrefix(ifj.Name, "e"); vi != vj {
return vi
}
ipi := ips[i].IP
ipj := ips[j].IP
// IPv4 is preferred
if vi, vj := ipi.To4() != nil, ipj.To4() != nil; vi != vj {
return vi
}
// en0 is preferred to en1
if ifi.Name != ifj.Name {
return ifi.Name < ifj.Name
}
// fallback
return ipi.String() < ipj.String()
})
return ips[0].IP
}

View File

@@ -0,0 +1,20 @@
// Copyright 2026 The Gitea Authors. All rights reserved.
// SPDX-License-Identifier: MIT
package common
import (
"net"
"testing"
"github.com/stretchr/testify/assert"
)
func TestPreferredIP(t *testing.T) {
assert.Nil(t, preferredIP(nil))
assert.Equal(t, net.ParseIP("2001:db8::1"), preferredIP([]interfaceIP{{IP: net.ParseIP("2001:db8::1")}}))
assert.Equal(t, net.ParseIP("2001:db8::1"), preferredIP([]interfaceIP{
{IP: net.ParseIP("192.0.2.1"), Name: "wlan0"},
{IP: net.ParseIP("2001:db8::1"), Name: "eth0"},
}))
}

View File

@@ -75,6 +75,9 @@ const (
HealthUnhealthy = string(container.Unhealthy)
)
// marks what a runner creates for its jobs, telling its leftovers from another runner's
const runnerUUIDLabel = "com.gitea.runner.uuid"
// ErrContainerNotFound reports a container the daemon no longer knows. Its text is a
// fragment, missingContainerError composes it into the message every operation shares.
var ErrContainerNotFound = errors.New("does not exist")

View File

@@ -8,10 +8,10 @@ package container
import (
"context"
"fmt"
cerrdefs "github.com/containerd/errdefs"
"github.com/moby/moby/client"
specs "github.com/opencontainers/image-spec/specs-go/v1"
)
// ImageExistsLocally returns a boolean indicating if an image with the
@@ -30,11 +30,23 @@ func ImageExistsLocally(ctx context.Context, imageName, platform string) (bool,
return false, err
}
if platform == "" || platform == "any" || fmt.Sprintf("%s/%s", inspectImage.Os, inspectImage.Architecture) == platform {
if platform == "" || platform == "any" {
return true, nil
}
return false, nil
requested, err := parsePlatform(platform)
if err != nil {
return false, err
}
return platformMatches(requested, &specs.Platform{OS: inspectImage.Os, Architecture: inspectImage.Architecture, Variant: inspectImage.Variant}), nil
}
func platformMatches(requested, image *specs.Platform) bool {
variant := image.Variant
if variant == "" {
variant = map[string]string{"amd64": "v1", "arm": "v7", "arm64": "v8"}[image.Architecture]
}
return image.OS == requested.OS && image.Architecture == requested.Architecture && (requested.Variant == "" || requested.Variant == variant)
}
// RemoveImage removes image from local store, the function is used to run different
@@ -46,14 +58,13 @@ func RemoveImage(ctx context.Context, imageName string, force, pruneChildren boo
}
defer cli.Close()
inspectImage, err := cli.ImageInspect(ctx, imageName)
if cerrdefs.IsNotFound(err) {
if _, err := cli.ImageInspect(ctx, imageName); cerrdefs.IsNotFound(err) {
return false, nil
} else if err != nil {
return false, err
}
if _, err = cli.ImageRemove(ctx, inspectImage.ID, client.ImageRemoveOptions{
if _, err = cli.ImageRemove(ctx, imageName, client.ImageRemoveOptions{
Force: force,
PruneChildren: pruneChildren,
}); err != nil {

View File

@@ -5,20 +5,20 @@
package container
import (
"context"
"fmt"
"os"
"os/exec"
"strings"
"testing"
log "github.com/sirupsen/logrus"
"gitea.com/gitea/runner/act/common/logging"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
func init() {
log.SetLevel(log.DebugLevel)
logging.SetLevel(logging.LevelDebug)
}
// buildScratchImage builds a tiny empty image for the given platform locally (FROM scratch, no
@@ -41,7 +41,7 @@ func buildScratchImage(t *testing.T, platform string) string {
func TestImageExistsLocally(t *testing.T) {
requireDocker(t)
ctx := context.Background()
ctx := t.Context()
// a non-existent image is reported absent
missing, err := ImageExistsLocally(ctx, "library/alpine:this-random-tag-will-never-exist", "linux/amd64")

View File

@@ -12,7 +12,7 @@ import (
"errors"
"io"
"github.com/sirupsen/logrus"
"gitea.com/gitea/runner/act/common/logging"
)
type dockerMessage struct {
@@ -26,7 +26,7 @@ type dockerMessage struct {
Progress string `json:"progress"`
}
func logDockerResponse(logger logrus.FieldLogger, dockerResponse io.ReadCloser, isError bool) error {
func logDockerResponse(logger *logging.Logger, dockerResponse io.ReadCloser, isError bool) error {
if dockerResponse == nil {
return nil
}
@@ -77,7 +77,7 @@ func logDockerResponse(logger logrus.FieldLogger, dockerResponse io.ReadCloser,
return nil
}
func writeLog(logger logrus.FieldLogger, isError bool, format string, args ...any) {
func writeLog(logger *logging.Logger, isError bool, format string, args ...any) {
if isError {
logger.Errorf(format, args...)
} else {

View File

@@ -24,10 +24,6 @@ import (
const (
networkCreateAttempts = 3
networkCreateRetryDelay = time.Second
// marks the networks a runner creates for its jobs, so it can tell its own leftovers from
// those of another runner sharing the daemon
runnerUUIDLabel = "com.gitea.runner.uuid"
)
// RemoveOrphanNetworks removes the networks this runner created for jobs whose teardown did
@@ -57,7 +53,9 @@ func removeOrphanNetworks(ctx context.Context, cli client.APIClient, runnerUUID,
networks:
for _, n := range networks.Items {
result, err := cli.NetworkInspect(ctx, n.ID, client.NetworkInspectOptions{})
if err != nil {
if cerrdefs.IsNotFound(err) {
continue
} else if err != nil {
errs = append(errs, fmt.Errorf("failed to inspect network %s: %w", n.Name, err))
continue
}
@@ -75,7 +73,9 @@ networks:
continue networks
}
}
if _, err := cli.NetworkRemove(ctx, n.ID, client.NetworkRemoveOptions{}); err != nil {
if _, err := cli.NetworkRemove(ctx, n.ID, client.NetworkRemoveOptions{}); cerrdefs.IsNotFound(err) {
continue
} else if err != nil {
errs = append(errs, fmt.Errorf("failed to remove network %s: %w", n.Name, err))
continue
}
@@ -91,45 +91,54 @@ func NewDockerNetworkCreateExecutor(name string, opts NewDockerNetworkCreateExec
return err
}
defer cli.Close()
return createDockerNetwork(ctx, cli, name, opts)
}
}
// Only create the network if it doesn't exist
networks, err := cli.NetworkList(ctx, client.NetworkListOptions{})
if err != nil {
func createDockerNetwork(ctx context.Context, cli client.APIClient, name string, opts NewDockerNetworkCreateExecutorInput) error {
// Only create the network if it doesn't exist
networks, err := cli.NetworkList(ctx, client.NetworkListOptions{})
if err != nil {
return err
}
var options map[string]string
// For Gitea, reduce log noise
// common.Logger(ctx).Debugf("%v", networks)
for _, n := range networks.Items {
if n.Name == name {
common.Logger(ctx).Debugf("Network %v exists", name)
return nil
}
// the daemon's mtu only reaches its default bridge, Docker's default is left out to keep default-network-opts
if mtu := n.Options["com.docker.network.driver.mtu"]; n.Name == network.NetworkBridge && mtu != "" && mtu != "1500" {
options = map[string]string{"com.docker.network.driver.mtu": mtu}
}
}
for i := range networkCreateAttempts {
if i > 0 {
common.Logger(ctx).Infof("Waiting for a free docker address pool to create network %s", name)
select {
case <-ctx.Done():
return ctx.Err()
case <-time.After(time.Duration(i) * networkCreateRetryDelay):
}
}
if _, err = cli.NetworkCreate(ctx, name, client.NetworkCreateOptions{
Driver: "bridge",
Scope: "local",
EnableIPv4: opts.EnableIPv4,
EnableIPv6: opts.EnableIPv6,
Options: options,
Labels: runnerLabels(opts.RunnerUUID),
}); err == nil {
return nil
}
if !isAddressPoolExhausted(err) {
return err
}
// For Gitea, reduce log noise
// common.Logger(ctx).Debugf("%v", networks)
for _, n := range networks.Items {
if n.Name == name {
common.Logger(ctx).Debugf("Network %v exists", name)
return nil
}
}
for i := range networkCreateAttempts {
if i > 0 {
common.Logger(ctx).Infof("Waiting for a free docker address pool to create network %s", name)
select {
case <-ctx.Done():
return ctx.Err()
case <-time.After(time.Duration(i) * networkCreateRetryDelay):
}
}
if _, err = cli.NetworkCreate(ctx, name, client.NetworkCreateOptions{
Driver: "bridge",
Scope: "local",
EnableIPv4: opts.EnableIPv4,
EnableIPv6: opts.EnableIPv6,
Labels: runnerLabels(opts.RunnerUUID),
}); err == nil {
return nil
}
if !isAddressPoolExhausted(err) {
return err
}
}
return fmt.Errorf("docker has no address pool left for this job's network, lower runner.capacity or widen default-address-pools in the docker daemon config: %w", err)
}
return fmt.Errorf("docker has no address pool left for this job's network, lower runner.capacity or widen default-address-pools in the docker daemon config: %w", err)
}
func runnerLabels(runnerUUID string) map[string]string {
@@ -198,36 +207,35 @@ func NewDockerNetworkRemoveExecutor(name string) common.Executor {
return err
}
defer cli.Close()
return removeNetwork(ctx, cli, name)
}
}
// Make sure that all network of the specified name are removed
// cli.NetworkRemove refuses to remove a network if there are duplicates
networks, err := cli.NetworkList(ctx, client.NetworkListOptions{})
if err != nil {
func removeNetwork(ctx context.Context, cli client.APIClient, name string) error {
networks, err := cli.NetworkList(ctx, client.NetworkListOptions{}) // NetworkRemove by name refuses duplicates, so remove each by ID
if err != nil {
return err
}
var errs []error
for _, n := range networks.Items {
if n.Name != name {
continue
}
result, err := cli.NetworkInspect(ctx, n.ID, client.NetworkInspectOptions{})
if cerrdefs.IsNotFound(err) {
continue
} else if err != nil {
return err
}
// For Gitea, reduce log noise
// common.Logger(ctx).Debugf("%v", networks)
var errs []error
for _, n := range networks.Items {
if n.Name == name {
result, err := cli.NetworkInspect(ctx, n.ID, client.NetworkInspectOptions{})
if err != nil {
return err
}
// it holds a subnet out of the daemon's pool until something reclaims it
if len(result.Network.Containers) != 0 {
common.Logger(ctx).Warnf("Refusing to remove network %s because it still has active endpoints, the idle cleanup reclaims it once they are gone", name)
continue
}
if _, err = cli.NetworkRemove(ctx, n.ID, client.NetworkRemoveOptions{}); err != nil {
errs = append(errs, fmt.Errorf("failed to remove network %s: %w", name, err))
}
}
if len(result.Network.Containers) != 0 {
common.Logger(ctx).Warnf("Refusing to remove network %s because it still has active endpoints, the orphan cleanup reclaims it once they are gone", name)
continue
}
if _, err = cli.NetworkRemove(ctx, n.ID, client.NetworkRemoveOptions{}); err != nil && !cerrdefs.IsNotFound(err) {
errs = append(errs, fmt.Errorf("failed to remove network %s: %w", name, err))
}
return errors.Join(errs...)
}
return errors.Join(errs...)
}
// IsolatedCacheContainer returns the container holding addr, when jobs on jobNetwork cannot reach it there.

View File

@@ -19,7 +19,7 @@ import (
)
func TestIsolatedCacheContainer(t *testing.T) {
ctx := context.Background()
ctx := t.Context()
cache, lan := netip.MustParseAddr("172.18.0.3"), netip.MustParseAddr("192.168.1.20")
client := &mockDockerClient{}
client.On("ContainerList", ctx, mobyclient.ContainerListOptions{}).Return(mobyclient.ContainerListResult{Items: []container.Summary{{}, {
@@ -57,6 +57,27 @@ func TestIsAddressPoolExhausted(t *testing.T) {
assert.False(t, isAddressPoolExhausted(cerrdefs.ErrInvalidArgument.WithMessage("invalid subnet 10.0.0.0/8: it overlaps with an existing network")))
}
func TestCreateDockerNetworkInheritsDaemonMTU(t *testing.T) {
ctx := t.Context()
for _, tc := range []struct {
bridgeOptions, want map[string]string
}{
{
map[string]string{"com.docker.network.bridge.host_binding_ipv4": "127.0.0.1", "com.docker.network.driver.mtu": "1450"},
map[string]string{"com.docker.network.driver.mtu": "1450"},
},
{map[string]string{"com.docker.network.driver.mtu": "1500"}, nil},
} {
client := &mockDockerClient{}
client.On("NetworkList", ctx, mobyclient.NetworkListOptions{}).
Return(mobyclient.NetworkListResult{Items: []network.Summary{{Name: network.NetworkBridge, Options: tc.bridgeOptions}}}, nil)
client.On("NetworkCreate", ctx, "job-network", mobyclient.NetworkCreateOptions{Driver: "bridge", Scope: "local", Options: tc.want}).
Return(mobyclient.NetworkCreateResult{}, nil)
require.NoError(t, createDockerNetwork(ctx, client, "job-network", NewDockerNetworkCreateExecutorInput{}))
client.AssertExpectations(t)
}
}
func (m *mockDockerClient) NetworkConnect(ctx context.Context, id string, opts mobyclient.NetworkConnectOptions) (mobyclient.NetworkConnectResult, error) {
return mobyclient.NetworkConnectResult{}, m.Called(ctx, id, opts).Error(0)
}
@@ -66,7 +87,7 @@ func (m *mockDockerClient) NetworkDisconnect(ctx context.Context, id string, opt
}
func TestConnectAndDisconnectNetwork(t *testing.T) {
ctx := context.Background()
ctx := t.Context()
connect := func(answer error) error {
client := &mockDockerClient{}
client.On("NetworkConnect", ctx, "jobs", mobyclient.NetworkConnectOptions{
@@ -95,8 +116,8 @@ func TestConnectAndDisconnectNetwork(t *testing.T) {
// Of this runner's networks, only the ones nothing but its own cache container is attached to
// and old enough to predate any job now starting are the runner's to reclaim. An unexpected
// NetworkRemove fails the test on its own, since testify has no expectation to match it against.
func TestRemoveOrphanNetworks(t *testing.T) {
ctx := context.Background()
func TestRemoveNetworks(t *testing.T) {
ctx := t.Context()
cutoff := time.Date(2026, time.April, 29, 20, 0, 0, 0, time.UTC)
client := &mockDockerClient{}
client.On("NetworkList", ctx, mobyclient.NetworkListOptions{
@@ -107,11 +128,13 @@ func TestRemoveOrphanNetworks(t *testing.T) {
{ID: "starting"},
{ID: "cached"},
{ID: "shared"},
{ID: "gone"},
{ID: "raced"},
}}, nil)
client.On("NetworkInspect", ctx, "orphan", mobyclient.NetworkInspectOptions{}).
Return(mobyclient.NetworkInspectResult{}, nil)
Return(mobyclient.NetworkInspectResult{}, nil).Twice()
client.On("NetworkInspect", ctx, "busy", mobyclient.NetworkInspectOptions{}).
Return(mobyclient.NetworkInspectResult{Network: network.Inspect{Containers: map[string]network.EndpointResource{"c": {}}}}, nil)
Return(mobyclient.NetworkInspectResult{Network: network.Inspect{Containers: map[string]network.EndpointResource{"c": {}}}}, nil).Twice()
client.On("NetworkInspect", ctx, "starting", mobyclient.NetworkInspectOptions{}).
Return(mobyclient.NetworkInspectResult{Network: network.Inspect{
Network: network.Network{Created: cutoff.Add(time.Second)},
@@ -121,13 +144,28 @@ func TestRemoveOrphanNetworks(t *testing.T) {
Return(mobyclient.NetworkInspectResult{Network: network.Inspect{Containers: map[string]network.EndpointResource{"0123456789abcdef": {}}}}, nil)
client.On("NetworkInspect", ctx, "shared", mobyclient.NetworkInspectOptions{}).
Return(mobyclient.NetworkInspectResult{Network: network.Inspect{Containers: map[string]network.EndpointResource{"0123456789abcdef": {}, "job": {}}}}, nil)
client.On("NetworkInspect", ctx, "gone", mobyclient.NetworkInspectOptions{}).
Return(mobyclient.NetworkInspectResult{}, cerrdefs.ErrNotFound).Twice()
client.On("NetworkInspect", ctx, "raced", mobyclient.NetworkInspectOptions{}).
Return(mobyclient.NetworkInspectResult{}, nil).Twice()
client.On("NetworkRemove", ctx, "raced", mobyclient.NetworkRemoveOptions{}).
Return(mobyclient.NetworkRemoveResult{}, cerrdefs.ErrNotFound).Twice()
client.On("NetworkRemove", ctx, "orphan", mobyclient.NetworkRemoveOptions{}).
Return(mobyclient.NetworkRemoveResult{}, nil)
Return(mobyclient.NetworkRemoveResult{}, nil).Twice()
client.On("NetworkDisconnect", ctx, "cached", mobyclient.NetworkDisconnectOptions{Container: "0123456789abcdef", Force: true}).
Return(nil).Once()
client.On("NetworkRemove", ctx, "cached", mobyclient.NetworkRemoveOptions{}).
Return(mobyclient.NetworkRemoveResult{}, nil).Once()
require.NoError(t, removeOrphanNetworks(ctx, client, "runner-1", "0123456789ab", cutoff))
client.On("NetworkList", ctx, mobyclient.NetworkListOptions{}).Return(mobyclient.NetworkListResult{Items: []network.Summary{
{ID: "orphan", Name: "job"},
{ID: "busy", Name: "job"},
{ID: "gone", Name: "job"},
{ID: "raced", Name: "job"},
{ID: "shared", Name: "other"},
}}, nil)
require.NoError(t, removeNetwork(ctx, client, "job"))
client.AssertExpectations(t)
}

View File

@@ -61,3 +61,26 @@ func TestParsePlatform(t *testing.T) {
})
}
}
func TestPlatformMatches(t *testing.T) {
for _, testcase := range []struct {
requested, image string
want bool
}{
{"linux/amd64", "linux/amd64", true},
{"linux/amd64/v1", "linux/amd64", true},
{"linux/arm64", "linux/amd64", false},
{"linux/arm64/v8", "linux/arm64", true},
{"linux/arm/v7", "linux/arm/v7", true},
{"linux/arm", "linux/arm/v7", true},
{"linux/arm/v6", "linux/arm/v7", false},
{"linux/arm", "linux/arm/v6", true},
{"windows/arm/v7", "linux/arm/v7", false},
} {
requested, err := parsePlatform(testcase.requested)
require.NoError(t, err)
image, err := parsePlatform(testcase.image)
require.NoError(t, err)
assert.Equal(t, testcase.want, platformMatches(requested, image), "%s on %s", testcase.requested, testcase.image)
}
}

View File

@@ -37,7 +37,7 @@ import (
)
const (
jobLabel = "com.gitea.runner.job"
jobLabel = "com.gitea.runner.job."
maxCreateBody = 8 << 20
dockerProxyProbeTimeout = 5 * time.Second
@@ -302,7 +302,7 @@ func rewriteCreate(r *http.Request, job string, mounts map[string]string) error
if config.Labels == nil {
config.Labels = make(map[string]string)
}
config.Labels[jobLabel] = job
config.Labels[jobLabel+job] = "" // per-job key, so nested proxies keep each other's
if fields["Labels"], err = json.Marshal(config.Labels); err != nil {
return err
}
@@ -560,7 +560,7 @@ func RemoveDockerJobResources(ctx context.Context, job string) error {
func removeLabelled(ctx context.Context, cli client.APIClient, job string) error {
logger := common.Logger(ctx)
filters := make(client.Filters).Add("label", jobLabel+"="+job)
filters := make(client.Filters).Add("label", jobLabel+job)
containers, err := cli.ContainerList(ctx, client.ContainerListOptions{All: true, Filters: filters})
errs := []error{err}
for _, c := range containers.Items {

View File

@@ -146,52 +146,52 @@ func TestDockerProxy(t *testing.T) {
t.Run("labels creates", func(t *testing.T) {
for _, testCase := range []struct{ path, body, want string }{
{
"/v1.47/containers/create", `{"Image":"alpine","Unknown":{"enabled":true},"Labels":{"own":"1","com.gitea.runner.job":"other"}}`,
`{"Image":"alpine","Unknown":{"enabled":true},"Labels":{"own":"1","com.gitea.runner.job":"job-1"}}`,
"/v1.47/containers/create", `{"Image":"alpine","Unknown":{"enabled":true},"Labels":{"own":"1","com.gitea.runner.job.job-0":""}}`,
`{"Image":"alpine","Unknown":{"enabled":true},"Labels":{"own":"1","com.gitea.runner.job.job-0":"","com.gitea.runner.job.job-1":""}}`,
},
{
"/containers/create", `{"HostConfig":{"Binds":["/workspace/o/r:/src:ro","/workspace/o/r/../r/sub/:/sub","workspace/o/r:/relative","/workspace/o/rest:/rest","/workspace/o/r/tmp/x:/tmp","named:/named","/anonymous","/workspace/o/r:ro","/volumes/job/_data/x:/daemon-path"]},"DriverOpts":{"o":"bind","device":"/workspace/o/r"},"Labels":{"Foo":"1","foo":"2"}}`,
`{"HostConfig":{"Binds":["/volumes/job/_data:/src:ro","/volumes/job/_data/sub:/sub","workspace/o/r:/relative","/workspace/o/rest:/rest","/workspace/o/r/tmp/x:/tmp","named:/named","/anonymous","/workspace/o/r:ro","/volumes/job/_data/x:/daemon-path"]},"DriverOpts":{"o":"bind","device":"/workspace/o/r"},"Labels":{"Foo":"1","foo":"2","com.gitea.runner.job":"job-1"}}`,
`{"HostConfig":{"Binds":["/volumes/job/_data:/src:ro","/volumes/job/_data/sub:/sub","workspace/o/r:/relative","/workspace/o/rest:/rest","/workspace/o/r/tmp/x:/tmp","named:/named","/anonymous","/workspace/o/r:ro","/volumes/job/_data/x:/daemon-path"]},"DriverOpts":{"o":"bind","device":"/workspace/o/r"},"Labels":{"Foo":"1","foo":"2","com.gitea.runner.job.job-1":""}}`,
},
{
"/containers/create", `{"HostConfig":{"Mounts":[{"Type":"bind","Source":"/var/run/docker.sock","Target":"/var/run/docker.sock"},{"type":"bind","source":"/workspace/o/r/sub/","target":"/m"},{"Type":"bind","Source":"/workspace/o/r/sub/../data","Target":"/dotted"},{"Type":"bind","Source":"/workspace/o/r/../r","Target":"/escaping"},{"Type":"BIND","Source":"/workspace/o/r"},{"Type":"volume","Source":"/workspace/o/r"}]}}`,
`{"HostConfig":{"Mounts":[{"Type":"bind","Source":"/tmp/p/docker.sock","Target":"/var/run/docker.sock"},{"Type":"bind","Source":"/volumes/job/_data/sub/","target":"/m"},{"Type":"bind","Source":"/volumes/job/_data/sub/../data","Target":"/dotted"},{"Type":"bind","Source":"/volumes/job/_data","Target":"/escaping"},{"Type":"BIND","Source":"/workspace/o/r"},{"Type":"volume","Source":"/workspace/o/r"}]},"Labels":{"com.gitea.runner.job":"job-1"}}`,
`{"HostConfig":{"Mounts":[{"Type":"bind","Source":"/tmp/p/docker.sock","Target":"/var/run/docker.sock"},{"Type":"bind","Source":"/volumes/job/_data/sub/","target":"/m"},{"Type":"bind","Source":"/volumes/job/_data/sub/../data","Target":"/dotted"},{"Type":"bind","Source":"/volumes/job/_data","Target":"/escaping"},{"Type":"BIND","Source":"/workspace/o/r"},{"Type":"volume","Source":"/workspace/o/r"}]},"Labels":{"com.gitea.runner.job.job-1":""}}`,
},
{
"/containers/create", `{"HostConfig":{"Mounts":[{"Type":"volume","VolumeOptions":{"DriverConfig":{"Options":{"o":"bind","device":"/workspace/o/r/data"}}}},{"Type":"volume","VolumeOptions":{"DriverConfig":{"Options":{"o":"unbindable","device":"/workspace/o/r"}}}},{"Type":"volume","VolumeOptions":{"DriverConfig":{"Options":{"o":"bind,remount","device":"/workspace/o/r"}}}},{"Type":"volume","VolumeOptions":{"DriverConfig":{"Name":"plugin","Options":{"o":"bind","device":"/workspace/o/r"}}}}]}}`,
`{"HostConfig":{"Mounts":[{"Type":"volume","VolumeOptions":{"DriverConfig":{"Options":{"o":"bind","device":"/volumes/job/_data/data"}}}},{"Type":"volume","VolumeOptions":{"DriverConfig":{"Options":{"o":"unbindable","device":"/workspace/o/r"}}}},{"Type":"volume","VolumeOptions":{"DriverConfig":{"Options":{"o":"bind,remount","device":"/workspace/o/r"}}}},{"Type":"volume","VolumeOptions":{"DriverConfig":{"Name":"plugin","Options":{"o":"bind","device":"/workspace/o/r"}}}}]},"Labels":{"com.gitea.runner.job":"job-1"}}`,
`{"HostConfig":{"Mounts":[{"Type":"volume","VolumeOptions":{"DriverConfig":{"Options":{"o":"bind","device":"/volumes/job/_data/data"}}}},{"Type":"volume","VolumeOptions":{"DriverConfig":{"Options":{"o":"unbindable","device":"/workspace/o/r"}}}},{"Type":"volume","VolumeOptions":{"DriverConfig":{"Options":{"o":"bind,remount","device":"/workspace/o/r"}}}},{"Type":"volume","VolumeOptions":{"DriverConfig":{"Name":"plugin","Options":{"o":"bind","device":"/workspace/o/r"}}}}]},"Labels":{"com.gitea.runner.job.job-1":""}}`,
},
{
"/containers/create", `{"HostConfig":{"privileged":true,"Privileged":false,"Binds":["/workspace/o/r:/src"]},"hoſtconfig":null}`,
`{"HostConfig":{"privileged":true,"Privileged":false,"Binds":["/workspace/o/r:/src"]},"hoſtconfig":null,"Labels":{"com.gitea.runner.job":"job-1"}}`,
`{"HostConfig":{"privileged":true,"Privileged":false,"Binds":["/workspace/o/r:/src"]},"hoſtconfig":null,"Labels":{"com.gitea.runner.job.job-1":""}}`,
},
{
"/networks/create", `{"Name":"n"}`,
`{"Name":"n","Labels":{"com.gitea.runner.job":"job-1"}}`,
`{"Name":"n","Labels":{"com.gitea.runner.job.job-1":""}}`,
},
{
"/volumes/create", `{"Name":"v","labels":null,"DriverOpts":{"type":"none","o":"rbind,ro","device":"/workspace/o/r/"},"HostConfig":{"Binds":["/workspace/o/r:/src"]}}`,
`{"Name":"v","Labels":{"com.gitea.runner.job":"job-1"},"DriverOpts":{"type":"none","o":"rbind,ro","device":"/volumes/job/_data/"},"HostConfig":{"Binds":["/workspace/o/r:/src"]}}`,
`{"Name":"v","Labels":{"com.gitea.runner.job.job-1":""},"DriverOpts":{"type":"none","o":"rbind,ro","device":"/volumes/job/_data/"},"HostConfig":{"Binds":["/workspace/o/r:/src"]}}`,
},
{
"/volumes/create", `{"Name":"d","DriverOpts":{"o":"bind","device":1,"device":"/workspace/o/r"}}`,
`{"Name":"d","DriverOpts":{"o":"bind","device":1,"device":"/workspace/o/r"},"Labels":{"com.gitea.runner.job":"job-1"}}`,
`{"Name":"d","DriverOpts":{"o":"bind","device":1,"device":"/workspace/o/r"},"Labels":{"com.gitea.runner.job.job-1":""}}`,
},
{
"/volumes/create", `{"Name":"p","Driver":"plugin","DriverOpts":{"o":"bind","device":"/workspace/o/r"}}`,
`{"Name":"p","Driver":"plugin","DriverOpts":{"o":"bind","device":"/workspace/o/r"},"Labels":{"com.gitea.runner.job":"job-1"}}`,
`{"Name":"p","Driver":"plugin","DriverOpts":{"o":"bind","device":"/workspace/o/r"},"Labels":{"com.gitea.runner.job.job-1":""}}`,
},
{
"/volumes/create", "",
`{"Labels":{"com.gitea.runner.job":"job-1"}}`,
`{"Labels":{"com.gitea.runner.job.job-1":""}}`,
},
{
"/volumes/create", "null",
`{"Labels":{"com.gitea.runner.job":"job-1"}}`,
`{"Labels":{"com.gitea.runner.job.job-1":""}}`,
},
{
"/volumes/create", `{"Labels":{"discard":"1"},"labels":null,"LABELS":{"own":"1"},"LABELS":{"extra":"2"}}`,
`{"Labels":{"own":"1","extra":"2","com.gitea.runner.job":"job-1"}}`,
`{"Labels":{"own":"1","extra":"2","com.gitea.runner.job.job-1":""}}`,
},
} {
resp, err := client.Post("http://docker"+testCase.path, "application/json", strings.NewReader(testCase.body))
@@ -244,7 +244,7 @@ func TestDockerProxy(t *testing.T) {
require.NoError(t, err)
require.NoError(t, resp.Body.Close())
require.Equal(t, http.StatusCreated, resp.StatusCode)
assert.JSONEq(t, `{"Labels":{"com.gitea.runner.job":"job-1"}}`, string(<-bodies))
assert.JSONEq(t, `{"Labels":{"com.gitea.runner.job.job-1":""}}`, string(<-bodies))
}
})
@@ -276,7 +276,7 @@ func TestDockerProxy(t *testing.T) {
require.NoError(t, err)
require.NoError(t, resp.Body.Close())
require.Equal(t, http.StatusCreated, resp.StatusCode)
assert.JSONEq(t, `{"Labels":{"com.gitea.runner.job":"job-1"}}`, string(<-bodies))
assert.JSONEq(t, `{"Labels":{"com.gitea.runner.job.job-1":""}}`, string(<-bodies))
require.NoError(t, <-closed)
})
}
@@ -285,8 +285,8 @@ func TestRemoveLabelledRemovesContainersNetworksAndVolumes(t *testing.T) {
containerFailure := errors.New("container removal failed")
listFailure := errors.New("listing failed")
volumeFailure := errors.New("volume removal failed")
ctx := context.Background()
filters := make(mobyclient.Filters).Add("label", jobLabel+"=job-1")
ctx := t.Context()
filters := make(mobyclient.Filters).Add("label", jobLabel+"job-1")
cli := &mockDockerClient{}
cli.On("ContainerList", ctx, mobyclient.ContainerListOptions{All: true, Filters: filters}).
Return(mobyclient.ContainerListResult{Items: []container.Summary{{ID: "c1", Names: []string{"/app"}}}}, nil).Once()
@@ -312,7 +312,8 @@ func TestRemoveLabelledRemovesContainersNetworksAndVolumes(t *testing.T) {
func TestDockerProxyWithDaemon(t *testing.T) {
requireDocker(t)
ctx := context.Background()
ctx := t.Context()
cleanupCtx := context.WithoutCancel(ctx)
require.NoError(t, NewDockerPullExecutor(NewDockerPullExecutorInput{Image: "alpine"})(ctx))
direct, err := GetDockerClient(ctx)
require.NoError(t, err)
@@ -337,22 +338,24 @@ func TestDockerProxyWithDaemon(t *testing.T) {
Config: &container.Config{Image: "alpine", Cmd: []string{"sleep", "300"}, Labels: map[string]string{"own": "1"}},
})
require.NoError(t, err)
t.Cleanup(func() { _, _ = direct.ContainerRemove(ctx, created.ID, mobyclient.ContainerRemoveOptions{Force: true}) })
t.Cleanup(func() {
_, _ = direct.ContainerRemove(cleanupCtx, created.ID, mobyclient.ContainerRemoveOptions{Force: true})
})
_, err = viaProxy.ContainerStart(ctx, created.ID, mobyclient.ContainerStartOptions{})
require.NoError(t, err)
net, err := viaProxy.NetworkCreate(ctx, job, mobyclient.NetworkCreateOptions{})
require.NoError(t, err)
t.Cleanup(func() { _, _ = direct.NetworkRemove(ctx, net.ID, mobyclient.NetworkRemoveOptions{}) })
t.Cleanup(func() { _, _ = direct.NetworkRemove(cleanupCtx, net.ID, mobyclient.NetworkRemoveOptions{}) })
_, err = viaProxy.VolumeCreate(ctx, mobyclient.VolumeCreateOptions{Name: job})
require.NoError(t, err)
t.Cleanup(func() { _, _ = direct.VolumeRemove(ctx, job, mobyclient.VolumeRemoveOptions{Force: true}) })
t.Cleanup(func() { _, _ = direct.VolumeRemove(cleanupCtx, job, mobyclient.VolumeRemoveOptions{Force: true}) })
inspected, err := direct.ContainerInspect(ctx, created.ID, mobyclient.ContainerInspectOptions{})
require.NoError(t, err)
assert.Equal(t, map[string]string{"own": "1", jobLabel: job}, inspected.Container.Config.Labels)
assert.Subset(t, inspected.Container.Config.Labels, map[string]string{"own": "1", jobLabel + job: ""})
vol, err := direct.VolumeInspect(ctx, job, mobyclient.VolumeInspectOptions{})
require.NoError(t, err)
assert.Equal(t, job, vol.Volume.Labels[jobLabel])
assert.Contains(t, vol.Volume.Labels, jobLabel+job)
exec, err := viaProxy.ExecCreate(ctx, created.ID, mobyclient.ExecCreateOptions{Cmd: []string{"cat"}, AttachStdin: true, AttachStdout: true, TTY: true})
require.NoError(t, err)

View File

@@ -5,20 +5,20 @@
package container
import (
"context"
"io"
"strings"
"testing"
"gitea.com/gitea/runner/act/common/logging"
"gitea.com/gitea/runner/act/common/logging/logtest"
"github.com/docker/cli/cli/config"
log "github.com/sirupsen/logrus"
"github.com/sirupsen/logrus/hooks/test"
assert "github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
func init() {
log.SetLevel(log.DebugLevel)
logging.SetLevel(logging.LevelDebug)
}
func TestCleanImage(t *testing.T) {
@@ -36,13 +36,13 @@ func TestCleanImage(t *testing.T) {
}
for _, table := range tables {
imageOut := cleanImage(context.Background(), table.imageIn)
imageOut := cleanImage(t.Context(), table.imageIn)
assert.Equal(t, table.imageOut, imageOut)
}
}
func TestGetImagePullOptions(t *testing.T) {
ctx := context.Background()
ctx := t.Context()
orig := config.Dir()
t.Cleanup(func() { config.SetDir(orig) })
@@ -79,7 +79,7 @@ func TestIsPinnedImage(t *testing.T) {
// The pull path reports a failure the daemon sent mid-stream, so it must carry the reason
// whichever of the two shapes the daemon used.
func TestLogDockerResponseError(t *testing.T) {
logger, _ := test.NewNullLogger()
logger := logging.New(logtest.New())
streamErr := func(line string) error {
return logDockerResponse(logger, io.NopCloser(strings.NewReader(line)), false)
}

View File

@@ -7,8 +7,8 @@
package container
import (
"archive/tar"
"bytes"
"cmp"
"context"
"errors"
"fmt"
@@ -25,17 +25,14 @@ import (
"time"
"gitea.com/gitea/runner/act/common"
"gitea.com/gitea/runner/act/filecollector"
"gitea.com/gitea/runner/act/common/logging"
"dario.cat/mergo"
"github.com/bmatcuk/doublestar/v4"
cerrdefs "github.com/containerd/errdefs"
"github.com/docker/cli/cli/compose/loader"
"github.com/docker/cli/cli/connhelper"
"github.com/go-git/go-billy/v5/helper/polyfill"
"github.com/go-git/go-billy/v5/osfs"
"github.com/go-git/go-git/v5/plumbing/format/gitignore"
"github.com/joho/godotenv"
"github.com/docker/cli/opts"
"github.com/moby/moby/api/pkg/stdcopy"
"github.com/moby/moby/api/types/container"
"github.com/moby/moby/api/types/mount"
@@ -44,7 +41,6 @@ import (
"github.com/moby/moby/client"
"github.com/moby/moby/client/pkg/versions"
specs "github.com/opencontainers/image-spec/specs-go/v1"
"github.com/sirupsen/logrus"
)
// drainGracePeriod bounds how long we wait for an output-copy goroutine to
@@ -99,8 +95,8 @@ func (cr *containerReference) Start(attach bool) common.Executor {
cr.attach().IfBool(attach),
cr.start(),
cr.wait().IfBool(attach),
cr.tryReadUID(),
cr.tryReadGID(),
cr.tryReadID("-u", func(id int) { cr.UID = id }),
cr.tryReadID("-g", func(id int) { cr.GID = id }),
func(ctx context.Context) error {
// If this fails, then folders have wrong permissions on non root container
if cr.UID != 0 || cr.GID != 0 {
@@ -309,7 +305,25 @@ func (cr *containerReference) UpdateFromEnv(srcPath string, env *map[string]stri
}
func (cr *containerReference) UpdateFromImageEnv(env *map[string]string) common.Executor {
return cr.extractFromImageEnv(env).IfNot(common.Dryrun)
envMap := *env
return common.Executor(func(ctx context.Context) error {
inspect, err := cr.cli.ContainerInspect(ctx, cr.id, client.ContainerInspectOptions{})
if err != nil {
common.Logger(ctx).Error(err)
return fmt.Errorf("inspect container: %w", err)
}
if inspect.Container.Config == nil {
return nil
}
for key, value := range opts.ConvertKVStringsToMap(inspect.Container.Config.Env) {
if envMap[key] == "" {
envMap[key] = value
} else if key == "PATH" {
envMap[key] += ":" + value
}
}
return nil
}).IfNot(common.Dryrun)
}
func (cr *containerReference) Exec(command []string, env map[string]string, user, workdir string) common.Executor {
@@ -571,20 +585,6 @@ func (cr *containerReference) mergeContainerConfigs(ctx context.Context, config
return nil, nil, fmt.Errorf("cannot process container options: '%s': '%w'", options, err)
}
// FIXME: If everything is fine after gitea/act v0.260.0, remove the following comment.
// In the old fork version, the code is
// if len(copts.netMode.Value()) == 0 {
// if err = copts.netMode.Set("host"); err != nil {
// return nil, nil, fmt.Errorf("cannot parse networkmode=host. This is an internal error and should not happen: '%w'", err)
// }
// }
// And it has been commented with:
// If a service container's network is set to `host`, the container will not be able to
// connect to the specified network created for the job container and the service containers.
// So comment out the following code.
// Not the if it's necessary to comment it in the new version,
// since it's cr.input.NetworkMode now.
if len(copts.netMode.Value()) == 0 {
if err = copts.netMode.Set(cr.input.NetworkMode); err != nil {
return nil, nil, fmt.Errorf("cannot parse networkmode=%s. This is an internal error and should not happen: '%w'", cr.input.NetworkMode, err)
@@ -593,7 +593,8 @@ func (cr *containerReference) mergeContainerConfigs(ctx context.Context, config
// If the `privileged` config has been disabled, `copts.privileged` need to be forced to false,
// even if the user specifies `--privileged` in the options string.
if !hostConfig.Privileged {
if !hostConfig.Privileged && copts.privileged {
logger.Warnf("container option %q is not allowed when privileged mode is disabled and will be ignored", "--privileged")
copts.privileged = false
}
@@ -629,6 +630,7 @@ func (cr *containerReference) mergeContainerConfigs(ctx context.Context, config
logger.Debugf("Custom container.HostConfig from options ==> %+v", containerConfig.HostConfig)
cr.sanitizeConfig(ctx, nil, containerConfig.HostConfig) // a dropped volume must not displace the runner's own
overlayVolumes(hostConfig, containerConfig.HostConfig)
binds := hostConfig.Binds
mounts := hostConfig.Mounts
@@ -646,6 +648,9 @@ func (cr *containerReference) mergeContainerConfigs(ctx context.Context, config
if flags.Changed("network") || flags.Changed("net") {
logger.Warn("--network and --net in the options will be ignored.")
}
if len(cr.input.Entrypoint) != 0 && flags.Changed("entrypoint") {
logger.Warn("--entrypoint in the options will be ignored.")
}
hostConfig.NetworkMode = networkMode
logger.Debugf("Merged container.HostConfig ==> %+v", hostConfig)
@@ -675,17 +680,11 @@ func (cr *containerReference) create(capAdd, capDrop []string) common.Executor {
ExposedPorts: exposedPorts,
Tty: input.AllocatePTY,
}
// For Gitea, reduce log noise
// logger.Debugf("Common container.Config ==> %+v", config)
if len(input.Cmd) != 0 {
config.Cmd = input.Cmd
}
if len(input.Entrypoint) != 0 {
config.Entrypoint = input.Entrypoint
}
mounts := make([]mount.Mount, 0)
for mountSource, mountTarget := range input.Mounts {
mounts = append(mounts, mount.Mount{
@@ -720,20 +719,20 @@ func (cr *containerReference) create(capAdd, capDrop []string) common.Executor {
PortBindings: portBindings,
AutoRemove: input.AutoRemove,
}
// For Gitea, reduce log noise
// logger.Debugf("Common container.HostConfig ==> %+v", hostConfig)
config, hostConfig, err = cr.mergeContainerConfigs(ctx, config, hostConfig)
if err != nil {
return err
}
if len(input.Entrypoint) != 0 {
config.Entrypoint = input.Entrypoint
}
// For Gitea
config, hostConfig = cr.sanitizeConfig(ctx, config, hostConfig)
var networkingConfig *network.NetworkingConfig
// For Gitea, reduce log noise
// logger.Debugf("input.NetworkAliases ==> %v", input.NetworkAliases)
n := hostConfig.NetworkMode
// IsUserDefined and IsHost are broken on windows
if n.IsUserDefined() && n != "host" && len(input.NetworkAliases) > 0 {
@@ -766,44 +765,6 @@ func (cr *containerReference) create(capAdd, capDrop []string) common.Executor {
}
}
func (cr *containerReference) extractFromImageEnv(env *map[string]string) common.Executor {
envMap := *env
return func(ctx context.Context) error {
logger := common.Logger(ctx)
inspect, err := cr.cli.ImageInspect(ctx, cr.input.Image)
if err != nil {
logger.Error(err)
return fmt.Errorf("inspect image: %w", err)
}
if inspect.Config == nil {
return nil
}
imageEnv, err := godotenv.Unmarshal(strings.Join(inspect.Config.Env, "\n"))
if err != nil {
logger.Error(err)
return fmt.Errorf("unmarshal image env: %w", err)
}
for k, v := range imageEnv {
if k == "PATH" {
if envMap[k] == "" {
envMap[k] = v
} else {
envMap[k] += `:` + v
}
} else if envMap[k] == "" {
envMap[k] = v
}
}
env = &envMap
return nil
}
}
func (cr *containerReference) exec(cmd []string, env map[string]string, user, workdir string) common.Executor {
return func(ctx context.Context) error {
if cr.id == "" {
@@ -821,11 +782,6 @@ func (cr *containerReference) exec(cmd []string, env map[string]string, user, wo
logger.Debugf("Exec command '%s'", cmd)
isTerminal := cr.input.AllocatePTY
envList := make([]string, 0)
for k, v := range env {
envList = append(envList, fmt.Sprintf("%s=%s", k, v))
}
var wd string
if workdir != "" {
if strings.HasPrefix(workdir, "/") {
@@ -842,7 +798,7 @@ func (cr *containerReference) exec(cmd []string, env map[string]string, user, wo
User: user,
Cmd: cmd,
WorkingDir: wd,
Env: envList,
Env: getEnvListFromMap(env),
TTY: isTerminal,
AttachStderr: true,
AttachStdout: true,
@@ -859,7 +815,7 @@ func (cr *containerReference) exec(cmd []string, env map[string]string, user, wo
}
defer resp.Close()
err = cr.waitForCommand(ctx, resp.HijackedResponse, idResp, user, workdir)
err = cr.waitForCommand(ctx, resp.HijackedResponse)
if err != nil {
return err
}
@@ -909,15 +865,7 @@ func (cr *containerReference) tryReadID(opt string, cbk func(id int)) common.Exe
}
}
func (cr *containerReference) tryReadUID() common.Executor {
return cr.tryReadID("-u", func(id int) { cr.UID = id })
}
func (cr *containerReference) tryReadGID() common.Executor {
return cr.tryReadID("-g", func(id int) { cr.GID = id })
}
func (cr *containerReference) waitForCommand(ctx context.Context, resp client.HijackedResponse, _ client.ExecCreateResult, _, _ string) error {
func (cr *containerReference) waitForCommand(ctx context.Context, resp client.HijackedResponse) error {
logger := common.Logger(ctx)
// Buffered so the copy goroutine never blocks on send if the grace-period
@@ -959,6 +907,7 @@ func (cr *containerReference) waitForCommand(ctx context.Context, resp client.Hi
}
func (cr *containerReference) copyDir(dstPath, srcPath string, useGitIgnore, skipGitDir bool) common.Executor {
dstPath = path.Clean(dstPath)
return func(ctx context.Context) error {
if cr.id == "" {
return cr.missingContainerError("copy directory to %s", dstPath)
@@ -980,42 +929,11 @@ func (cr *containerReference) copyDir(dstPath, srcPath string, useGitIgnore, ski
logger.Error(err)
}
}(tarFile)
tw := tar.NewWriter(tarFile)
srcPrefix := filepath.Dir(srcPath)
if !strings.HasSuffix(srcPrefix, string(filepath.Separator)) {
srcPrefix += string(filepath.Separator)
extractionRoot := "/"
if actPath := cr.GetActPath(); strings.HasPrefix(dstPath+"/", actPath+"/") {
extractionRoot = actPath // keeps tar entries from crossing the image's /var/run symlink
}
logger.Debugf("Stripping prefix:%s src:%s", srcPrefix, srcPath)
var ignorer gitignore.Matcher
if useGitIgnore {
ps, err := gitignore.ReadPatterns(polyfill.New(osfs.New(srcPath)), nil)
if err != nil {
logger.Debugf("Error loading .gitignore: %v", err)
}
ignorer = gitignore.NewMatcher(ps)
}
fc := &filecollector.FileCollector{
Ignorer: ignorer,
SrcPath: srcPath,
SrcPrefix: srcPrefix,
SkipGitDir: skipGitDir,
Handler: &filecollector.TarCollector{
TarWriter: tw,
UID: cr.UID,
GID: cr.GID,
DstDir: dstPath[1:],
},
}
err = filepath.Walk(srcPath, fc.CollectFiles(ctx, []string{}))
if err != nil {
return err
}
if err := tw.Close(); err != nil {
if err := writeDirTar(ctx, tarFile, path.Join("/", strings.TrimPrefix(dstPath, extractionRoot)), srcPath, useGitIgnore, skipGitDir, cr.UID, cr.GID); err != nil {
return err
}
@@ -1024,14 +942,7 @@ func (cr *containerReference) copyDir(dstPath, srcPath string, useGitIgnore, ski
if err != nil {
return fmt.Errorf("failed to seek tar archive: %w", err)
}
_, err = cr.cli.CopyToContainer(ctx, cr.id, client.CopyToContainerOptions{
DestinationPath: "/",
Content: tarFile,
})
if err != nil {
return fmt.Errorf("failed to copy content to container: %w", err)
}
return nil
return cr.upload(ctx, extractionRoot, tarFile)
}
}
@@ -1040,41 +951,23 @@ func (cr *containerReference) copyContent(dstPath string, files ...*FileEntry) c
if cr.id == "" {
return cr.missingContainerError("copy to %s", dstPath)
}
logger := common.Logger(ctx)
var buf bytes.Buffer
tw := tar.NewWriter(&buf)
for _, file := range files {
logger.Debugf("Writing entry to tarball %s len:%d", file.Name, len(file.Body))
hdr := &tar.Header{
Name: file.Name,
Mode: file.Mode,
Size: int64(len(file.Body)),
Uid: cr.UID,
Gid: cr.GID,
}
if err := tw.WriteHeader(hdr); err != nil {
return err
}
if _, err := tw.Write([]byte(file.Body)); err != nil {
return err
}
}
if err := tw.Close(); err != nil {
if err := writeFilesTar(ctx, &buf, cr.UID, cr.GID, files...); err != nil {
return err
}
logger.Debugf("Extracting content to '%s'", dstPath)
_, err := cr.cli.CopyToContainer(ctx, cr.id, client.CopyToContainerOptions{
DestinationPath: dstPath,
Content: &buf,
})
if err != nil {
return fmt.Errorf("failed to copy content to container: %w", err)
}
return nil
common.Logger(ctx).Debugf("Extracting content to '%s'", dstPath)
return cr.upload(ctx, dstPath, &buf)
}
}
func (cr *containerReference) upload(ctx context.Context, dstPath string, content io.Reader) error {
if _, err := cr.cli.CopyToContainer(ctx, cr.id, client.CopyToContainerOptions{DestinationPath: dstPath, Content: content}); err != nil {
return fmt.Errorf("failed to copy content to container: %w", err)
}
return nil
}
func (cr *containerReference) attach() common.Executor {
return func(ctx context.Context) error {
out, err := cr.cli.ContainerAttach(ctx, cr.id, client.ContainerAttachOptions{
@@ -1153,7 +1046,7 @@ func (cr *containerReference) wait() common.Executor {
// sanitizeOptionsHostConfig takes back everything a workflow could escape the container with,
// setting each field to trusted, which is what the runner's own options parse to on their own.
// Only for unprivileged mode, since privileged mode grants host access anyway.
func sanitizeOptionsHostConfig(logger logrus.FieldLogger, hostConfig, trusted *container.HostConfig) {
func sanitizeOptionsHostConfig(logger *logging.Logger, hostConfig, trusted *container.HostConfig) {
resetOption(logger, "--pid", &hostConfig.PidMode, trusted.PidMode)
resetOption(logger, "--ipc", &hostConfig.IpcMode, trusted.IpcMode)
resetOption(logger, "--uts", &hostConfig.UTSMode, trusted.UTSMode)
@@ -1173,6 +1066,10 @@ func sanitizeOptionsHostConfig(logger logrus.FieldLogger, hostConfig, trusted *c
// systempaths=unconfined lands in these two rather than in SecurityOpt
resetOption(logger, "--security-opt", &hostConfig.MaskedPaths, trusted.MaskedPaths)
resetOption(logger, "--security-opt", &hostConfig.ReadonlyPaths, trusted.ReadonlyPaths)
resetOption(logger, "--log-driver", &hostConfig.LogConfig.Type, trusted.LogConfig.Type) // drivers run in the daemon, egressing from its network
resetOption(logger, "--log-opt", &hostConfig.LogConfig.Config, trusted.LogConfig.Config)
resetOption(logger, "--oom-score-adj", &hostConfig.OomScoreAdj, trusted.OomScoreAdj) // -1000 exempts it from the host OOM killer
resetOption(logger, "--oom-kill-disable", &hostConfig.OomKillDisable, trusted.OomKillDisable)
// a driver mounts what it likes, e.g. local with device= binds any host path, which
// valid_volumes never gets to see
@@ -1188,7 +1085,7 @@ func sanitizeOptionsHostConfig(logger logrus.FieldLogger, hostConfig, trusted *c
// resetOption puts a field back to the runner's own value. It compares the values rather than
// the flags, so a field that more than one option feeds cannot slip through.
func resetOption[T any](logger logrus.FieldLogger, option string, field *T, trusted T) {
func resetOption[T any](logger *logging.Logger, option string, field *T, trusted T) {
if reflect.DeepEqual(*field, trusted) {
return
}
@@ -1214,9 +1111,9 @@ func parseOptionsHostConfig(options string) (*container.HostConfig, error) {
// sanitizeConfig remove the invalid configurations from `config` and `hostConfig`
func (cr *containerReference) sanitizeConfig(ctx context.Context, config *container.Config, hostConfig *container.HostConfig) (*container.Config, *container.HostConfig) {
logger := common.Logger(ctx)
matcher := newValidVolumeMatcher(ctx, cr.input.ValidVolumes)
if len(cr.input.ValidVolumes) > 0 {
matcher := newValidVolumeMatcher(ctx, cr.input.ValidVolumes)
// sanitize binds
sanitizedBinds := make([]string, 0, len(hostConfig.Binds))
for _, bind := range hostConfig.Binds {
@@ -1230,37 +1127,47 @@ func (cr *containerReference) sanitizeConfig(ctx context.Context, config *contai
sanitizedBinds = append(sanitizedBinds, bind)
continue
}
if matcher.isValid(parsed.Source, mount.Type(parsed.Type)) {
if matcher.isValid(parsed.Source, mount.Type(parsed.Type), parsed.ReadOnly) {
if matcher.needsNoCopy(parsed.Source, mount.Type(parsed.Type)) && (parsed.Volume == nil || !parsed.Volume.NoCopy) {
bind += ",nocopy"
}
sanitizedBinds = append(sanitizedBinds, bind)
} else {
logger.Warnf("[%s] is not a valid volume, will be ignored", parsed.Source)
}
}
hostConfig.Binds = sanitizedBinds
// sanitize mounts
sanitizedMounts := make([]mount.Mount, 0, len(hostConfig.Mounts))
for _, mt := range hostConfig.Mounts {
if matcher.isValid(mt.Source, mt.Type) {
sanitizedMounts = append(sanitizedMounts, mt)
} else {
logger.Warnf("[%s] is not a valid volume, will be ignored", mt.Source)
}
}
hostConfig.Mounts = sanitizedMounts
} else {
for _, bind := range hostConfig.Binds {
logger.Warnf("[%s] is not a valid volume, will be ignored", bind)
}
for _, mt := range hostConfig.Mounts {
hostConfig.Binds = []string{}
}
// sanitize mounts
sanitizedMounts := make([]mount.Mount, 0, len(hostConfig.Mounts))
for _, mt := range hostConfig.Mounts {
readOnly := mt.ReadOnly && (mt.BindOptions == nil || !mt.BindOptions.ReadOnlyNonRecursive) // bind-recursive=writable leaves submounts writable
if matcher.isValid(mt.Source, mt.Type, readOnly) {
if matcher.needsNoCopy(mt.Source, mt.Type) {
mt.VolumeOptions = cmp.Or(mt.VolumeOptions, &mount.VolumeOptions{})
mt.VolumeOptions.NoCopy = true
}
sanitizedMounts = append(sanitizedMounts, mt)
} else {
logger.Warnf("[%s] is not a valid volume, will be ignored", mt.Source)
}
hostConfig.Binds = []string{}
hostConfig.Mounts = []mount.Mount{}
}
hostConfig.Mounts = sanitizedMounts
return config, hostConfig
}
func SanitizeBinds(ctx context.Context, validVolumes, binds []string) []string {
cr := &containerReference{input: &NewContainerInput{ValidVolumes: validVolumes}}
_, hostConfig := cr.sanitizeConfig(ctx, nil, &container.HostConfig{Binds: binds})
return hostConfig.Binds
}
// bindTarget returns the container path a bind mounts onto, empty if it cannot be parsed.
func bindTarget(bind string) string {
parsed, err := loader.ParseVolume(bind)
@@ -1290,26 +1197,37 @@ func overlayVolumes(dst, src *container.HostConfig) {
}
type validVolumeMatcher struct {
allowAll bool
named []string
host []string
allowAll bool
allowAllReadOnly bool
named []validVolumeRule
host []validVolumeRule
}
type validVolumeRule struct {
pattern string
readOnly bool
}
func newValidVolumeMatcher(ctx context.Context, validVolumes []string) validVolumeMatcher {
logger := common.Logger(ctx)
ret := validVolumeMatcher{
named: make([]string, 0, len(validVolumes)),
host: make([]string, 0, len(validVolumes)),
named: make([]validVolumeRule, 0, len(validVolumes)),
host: make([]validVolumeRule, 0, len(validVolumes)),
}
for _, v := range validVolumes {
v, readOnly := strings.CutSuffix(v, ":ro")
if v == "**" {
ret.allowAll = true
if readOnly {
ret.allowAllReadOnly = true
} else {
ret.allowAll = true
}
continue
}
if !isHostVolumePattern(v) {
if doublestar.ValidatePattern(v) {
ret.named = append(ret.named, v)
ret.named = append(ret.named, validVolumeRule{pattern: v, readOnly: readOnly})
} else {
logger.Errorf("invalid volume pattern %s", v)
}
@@ -1321,7 +1239,7 @@ func newValidVolumeMatcher(ctx context.Context, validVolumes []string) validVolu
continue
}
if doublestar.ValidatePathPattern(normalized) {
ret.host = append(ret.host, normalized)
ret.host = append(ret.host, validVolumeRule{pattern: normalized, readOnly: readOnly})
} else {
logger.Errorf("invalid volume pattern %s", normalized)
}
@@ -1330,30 +1248,34 @@ func newValidVolumeMatcher(ctx context.Context, validVolumes []string) validVolu
return ret
}
func (m validVolumeMatcher) isValid(source string, sourceType mount.Type) bool {
if m.allowAll {
func (m validVolumeMatcher) isValid(source string, sourceType mount.Type, readOnly bool) bool {
if m.allowAll || (m.allowAllReadOnly && readOnly) {
return true
}
if sourceType == mount.TypeTmpfs && source == "" { // backed by memory, exposes nothing from the host, same as --tmpfs
return true
}
rules, match := m.named, doublestar.MatchUnvalidated
if isHostVolumeSource(source, sourceType) {
normalized, err := normalizeHostVolumePath(source)
if err != nil {
return false
}
for _, pattern := range m.host {
if doublestar.PathMatchUnvalidated(pattern, normalized) {
return true
}
}
return false
rules, match, source = m.host, doublestar.PathMatchUnvalidated, normalized
}
for _, pattern := range m.named {
if doublestar.MatchUnvalidated(pattern, source) {
for _, rule := range rules {
if (readOnly || !rule.readOnly) && match(rule.pattern, source) {
return true
}
}
return false
}
// needsNoCopy reports a volume only `:ro` rules allow, as dockerd fills an empty volume from the image even when mounted read-only.
func (m validVolumeMatcher) needsNoCopy(source string, sourceType mount.Type) bool {
return sourceType == mount.TypeVolume && !m.isValid(source, sourceType, false)
}
func isHostVolumePattern(pattern string) bool {
return filepath.IsAbs(pattern) ||
strings.HasPrefix(pattern, "."+string(filepath.Separator)) ||

View File

@@ -5,6 +5,7 @@
package container
import (
"archive/tar"
"bufio"
"bytes"
"context"
@@ -18,6 +19,8 @@ import (
"testing"
"gitea.com/gitea/runner/act/common"
"gitea.com/gitea/runner/act/common/logging"
"gitea.com/gitea/runner/act/common/logging/logtest"
cerrdefs "github.com/containerd/errdefs"
"github.com/moby/moby/api/pkg/stdcopy"
@@ -25,50 +28,33 @@ import (
"github.com/moby/moby/api/types/mount"
"github.com/moby/moby/api/types/network"
mobyclient "github.com/moby/moby/client"
"github.com/sirupsen/logrus/hooks/test"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/mock"
"github.com/stretchr/testify/require"
)
func TestDocker(t *testing.T) {
requireDocker(t)
ctx := context.Background()
client, err := GetDockerClient(ctx)
require.NoError(t, err)
defer client.Close()
dockerBuild := NewDockerBuildExecutor(NewDockerBuildExecutorInput{
ContextDir: "testdata",
ImageTag: "envmergetest",
})
err = dockerBuild(ctx)
assert.NoError(t, err) //nolint:testifylint // pre-existing issue from nektos/act
cr := &containerReference{
cli: client,
input: &NewContainerInput{
Image: "envmergetest",
},
}
func TestDockerEnvironmentUsesContainerPath(t *testing.T) {
client := &mockDockerClient{}
client.On("ContainerInspect", mock.Anything, "amd64-job", mobyclient.ContainerInspectOptions{}).
Return(mobyclient.ContainerInspectResult{Container: container.InspectResponse{Config: &container.Config{Env: []string{
"PATH=/opt/node/x64/bin:/usr/bin:/bin", "CONFLICT_VAR=container-value", "SOME_RANDOM_VAR=",
"ANOTHER_ONE=old-value", "ANOTHER_ONE=BUT_I_HAVE_VALUE", "LITERAL='quoted'=$HOME\nsecond line",
}}}}, nil)
env := map[string]string{
"PATH": "/usr/local/bin:/usr/bin:/usr/sbin:/bin:/sbin",
"PATH": "/custom/bin",
"RANDOM_VAR": "WITH_VALUE",
"ANOTHER_VAR": "",
"CONFLICT_VAR": "I_EXIST_IN_MULTIPLE_PLACES",
"CONFLICT_VAR": "step-value",
}
envExecutor := cr.extractFromImageEnv(&env)
err = envExecutor(ctx)
assert.NoError(t, err) //nolint:testifylint // pre-existing issue from nektos/act
require.NoError(t, (&containerReference{cli: client, id: "amd64-job"}).UpdateFromImageEnv(&env)(t.Context()))
assert.Equal(t, map[string]string{
"PATH": "/usr/local/bin:/usr/bin:/usr/sbin:/bin:/sbin:/this/path/does/not/exists/anywhere:/this/either",
"PATH": "/custom/bin:/opt/node/x64/bin:/usr/bin:/bin",
"RANDOM_VAR": "WITH_VALUE",
"ANOTHER_VAR": "",
"SOME_RANDOM_VAR": "",
"ANOTHER_ONE": "BUT_I_HAVE_VALUE",
"CONFLICT_VAR": "I_EXIST_IN_MULTIPLE_PLACES",
"CONFLICT_VAR": "step-value",
"LITERAL": "'quoted'=$HOME\nsecond line",
}, env)
}
@@ -137,6 +123,11 @@ func (m *mockDockerClient) NetworkList(ctx context.Context, opts mobyclient.Netw
return args.Get(0).(mobyclient.NetworkListResult), args.Error(1)
}
func (m *mockDockerClient) NetworkCreate(ctx context.Context, name string, opts mobyclient.NetworkCreateOptions) (mobyclient.NetworkCreateResult, error) {
args := m.Called(ctx, name, opts)
return args.Get(0).(mobyclient.NetworkCreateResult), args.Error(1)
}
func (m *mockDockerClient) NetworkInspect(ctx context.Context, id string, opts mobyclient.NetworkInspectOptions) (mobyclient.NetworkInspectResult, error) {
args := m.Called(ctx, id, opts)
return args.Get(0).(mobyclient.NetworkInspectResult), args.Error(1)
@@ -175,7 +166,7 @@ func (m *mockConn) Close() (err error) {
}
func TestDockerExecAbort(t *testing.T) {
ctx, cancel := context.WithCancel(context.Background())
ctx, cancel := context.WithCancel(t.Context())
reader := &interruptReader{started: make(chan struct{}), interrupted: make(chan struct{}), stopped: make(chan struct{})}
conn := &mockConn{}
@@ -216,7 +207,7 @@ func TestDockerExecAbort(t *testing.T) {
}
func TestDockerExecFailure(t *testing.T) {
ctx := context.Background()
ctx := t.Context()
conn := &mockConn{}
@@ -263,7 +254,7 @@ func stdcopyFrame(stream stdcopy.StdType, payload string) []byte {
// attach() streaming goroutine has drained and flushed the container's output,
// so a final line without a trailing newline is not lost.
func TestDockerAttachFlushesTrailingLine(t *testing.T) {
ctx := context.Background()
ctx := t.Context()
framed := bytes.NewBuffer(stdcopyFrame(stdcopy.Stdout, "line one\nlast line without newline"))
@@ -312,7 +303,7 @@ func TestDockerAttachFlushesTrailingLine(t *testing.T) {
}
func TestDockerWaitFailure(t *testing.T) {
ctx := context.Background()
ctx := t.Context()
statusCh := make(chan container.WaitResponse, 1)
statusCh <- container.WaitResponse{StatusCode: 2}
@@ -363,8 +354,8 @@ func TestRemoveIgnoresAutoRemoveRace(t *testing.T) {
{name: "real failure", err: removeFailure, wantErr: removeFailure},
} {
t.Run(tc.name, func(t *testing.T) {
logger, hook := test.NewNullLogger()
ctx := common.WithLogger(context.Background(), logger)
logger, hook := logtest.NewLogger()
ctx := common.WithLogger(t.Context(), logger)
client := &mockDockerClient{}
client.On("ContainerKill", ctx, "abc", killOpts).Return(mobyclient.ContainerKillResult{}, nil)
client.On("ContainerRemove", ctx, "abc", removeOpts).Return(mobyclient.ContainerRemoveResult{}, tc.err)
@@ -387,7 +378,7 @@ func TestRemoveIgnoresAutoRemoveRace(t *testing.T) {
} else {
assert.Empty(t, cr.id)
}
assert.Empty(t, hook.AllEntries())
assert.Empty(t, hook.AllRecords())
client.AssertExpectations(t)
})
}
@@ -397,7 +388,7 @@ func TestRemoveIgnoresAutoRemoveRace(t *testing.T) {
// create() lost its reply, must still be removed rather than leaking with its network. It
// was never started here, so it is not worth a kill of its own.
func TestRemoveWithoutIDUsesName(t *testing.T) {
ctx := context.Background()
ctx := t.Context()
client := &mockDockerClient{}
client.On("ContainerRemove", ctx, "job-1", mobyclient.ContainerRemoveOptions{RemoveVolumes: true, Force: true}).
Return(mobyclient.ContainerRemoveResult{}, nil)
@@ -410,7 +401,7 @@ func TestRemoveWithoutIDUsesName(t *testing.T) {
// find() must drop a stale cached id so later Copy/Exec don't hit the
// daemon with a torn-down container.
func TestFindRevalidatesStaleID(t *testing.T) {
ctx := context.Background()
ctx := t.Context()
notFound := cerrdefs.ErrNotFound.WithMessage("No such container")
boom := errors.New("daemon unreachable")
newCR := func(id string) (*containerReference, *mockDockerClient) {
@@ -468,7 +459,7 @@ func TestFindRevalidatesStaleID(t *testing.T) {
// Every daemon entry point fails fast with a clear, container-named
// error when no live cr.id is known.
func TestRejectsMissingContainer(t *testing.T) {
ctx := context.Background()
ctx := t.Context()
client := &mockDockerClient{}
client.On("ContainerList", ctx, mobyclient.ContainerListOptions{All: true}).Return(mobyclient.ContainerListResult{}, nil)
cr := &containerReference{cli: client, input: &NewContainerInput{Name: "job-1"}}
@@ -493,25 +484,45 @@ func TestRejectsMissingContainer(t *testing.T) {
check("Inspect after removal", err)
}
// End-to-end: a stale cr.id is cleared, repopulated from name lookup,
// and the Copy completes against the fresh id.
func TestPublicCopyPipelineHandlesStaleID(t *testing.T) {
ctx := context.Background()
client := &mockDockerClient{}
client.On("ContainerInspect", ctx, "stale", mobyclient.ContainerInspectOptions{}).
Return(mobyclient.ContainerInspectResult{}, cerrdefs.ErrNotFound.WithMessage("gone"))
client.On("ContainerList", ctx, mobyclient.ContainerListOptions{All: true}).
Return(mobyclient.ContainerListResult{Items: []container.Summary{
{ID: "fresh", Names: []string{"/job-1"}},
}}, nil)
client.On("CopyToContainer", ctx, "fresh", mock.MatchedBy(func(opts mobyclient.CopyToContainerOptions) bool {
return opts.DestinationPath == "/var/run/act"
})).Return(mobyclient.CopyToContainerResult{}, nil)
for _, testcase := range []struct {
name, destination, extractionRoot, entry string
directory bool
}{
{name: "files", destination: "/var/run/act", extractionRoot: "/var/run/act", entry: "x"},
{name: "action below act volume", directory: true, destination: "/var/run/act/actions/checkout/", extractionRoot: "/var/run/act", entry: "actions/checkout/x"},
{name: "sibling prefix", directory: true, destination: "/var/run/actor/action", extractionRoot: "/", entry: "var/run/actor/action/x"},
} {
t.Run(testcase.name, func(t *testing.T) {
ctx := t.Context()
client := &mockDockerClient{}
client.On("ContainerInspect", ctx, "stale", mobyclient.ContainerInspectOptions{}).
Return(mobyclient.ContainerInspectResult{}, cerrdefs.ErrNotFound.WithMessage("gone"))
client.On("ContainerList", ctx, mobyclient.ContainerListOptions{All: true}).
Return(mobyclient.ContainerListResult{Items: []container.Summary{
{ID: "fresh", Names: []string{"/job-1"}},
}}, nil)
client.On("CopyToContainer", ctx, "fresh", mock.Anything).
Run(func(args mock.Arguments) {
opts := args.Get(2).(mobyclient.CopyToContainerOptions)
assert.Equal(t, testcase.extractionRoot, opts.DestinationPath)
header, err := tar.NewReader(opts.Content).Next()
require.NoError(t, err)
assert.Equal(t, testcase.entry, header.Name)
}).Return(mobyclient.CopyToContainerResult{}, nil).Once()
cr := &containerReference{id: "stale", cli: client, input: &NewContainerInput{Name: "job-1"}}
require.NoError(t, cr.Copy("/var/run/act", &FileEntry{Name: "x", Mode: 0o644})(ctx))
assert.Equal(t, "fresh", cr.id)
client.AssertExpectations(t)
cr := &containerReference{id: "stale", cli: client, input: &NewContainerInput{Name: "job-1"}}
if testcase.directory {
source := t.TempDir()
require.NoError(t, os.WriteFile(filepath.Join(source, "x"), nil, 0o644))
require.NoError(t, cr.CopyDir(testcase.destination, source+string(filepath.Separator), false, true)(ctx))
} else {
require.NoError(t, cr.Copy(testcase.destination, &FileEntry{Name: "x", Mode: 0o644})(ctx))
}
assert.Equal(t, "fresh", cr.id)
client.AssertExpectations(t)
})
}
}
// Type assert containerReference implements ExecutionsEnvironment
@@ -519,10 +530,12 @@ var _ ExecutionsEnvironment = &containerReference{}
func TestCheckVolumes(t *testing.T) {
testCases := []struct {
desc string
validVolumes []string
binds []string
expectedBinds []string
desc string
validVolumes []string
binds []string
expectedBinds []string
mounts []mount.Mount
expectedMounts []mount.Mount
}{
{
desc: "match all volumes",
@@ -578,19 +591,41 @@ func TestCheckVolumes(t *testing.T) {
"/opt/app/cache:/cache",
},
},
{
desc: "read-only rules allow only read-only mounts and copy nothing into volumes",
validVolumes: []string{"/etc/ssl:ro", "data:ro", "shared"},
binds: []string{"/etc/ssl:/ssl:ro", "/etc/ssl:/ssl", "data:/data:ro", "data:/data", "shared:/shared:ro"},
expectedBinds: []string{"/etc/ssl:/ssl:ro", "data:/data:ro,nocopy", "shared:/shared:ro"},
mounts: []mount.Mount{
{Type: mount.TypeBind, Source: "/etc/ssl", Target: "/ssl", ReadOnly: true},
{Type: mount.TypeBind, Source: "/etc/ssl", Target: "/ssl", ReadOnly: true, BindOptions: &mount.BindOptions{ReadOnlyNonRecursive: true}},
{Type: mount.TypeVolume, Source: "data", Target: "/data", ReadOnly: true},
},
expectedMounts: []mount.Mount{
{Type: mount.TypeBind, Source: "/etc/ssl", Target: "/ssl", ReadOnly: true},
{Type: mount.TypeVolume, Source: "data", Target: "/data", ReadOnly: true, VolumeOptions: &mount.VolumeOptions{NoCopy: true}},
},
},
{
desc: "read-only wildcard keeps writable rules writable",
validVolumes: []string{"**:ro", "shared"},
binds: []string{"/etc/ssl:/ssl:ro", "data:/data", "shared:/shared"},
expectedBinds: []string{"/etc/ssl:/ssl:ro", "shared:/shared"},
},
}
for _, tc := range testCases {
t.Run(tc.desc, func(t *testing.T) {
logger, hook := test.NewNullLogger()
ctx := common.WithLogger(context.Background(), logger)
logger, hook := logtest.NewLogger()
ctx := common.WithLogger(t.Context(), logger)
cr := &containerReference{
input: &NewContainerInput{
ValidVolumes: tc.validVolumes,
},
}
_, hostConf := cr.sanitizeConfig(ctx, &container.Config{}, &container.HostConfig{Binds: tc.binds})
_, hostConf := cr.sanitizeConfig(ctx, &container.Config{}, &container.HostConfig{Binds: tc.binds, Mounts: tc.mounts})
assert.Equal(t, tc.expectedBinds, hostConf.Binds)
assert.Len(t, hook.AllEntries(), len(tc.binds)-len(tc.expectedBinds)) // every drop is warned about
assert.ElementsMatch(t, tc.expectedMounts, hostConf.Mounts)
assert.Len(t, hook.AllRecords(), len(tc.binds)+len(tc.mounts)-len(tc.expectedBinds)-len(tc.expectedMounts)) // every drop is warned about
})
}
}
@@ -618,33 +653,33 @@ func TestMergeContainerConfigsKeepsTheRunnersFilesAndEnvToItself(t *testing.T) {
t.Setenv("RUNNER_SECRET", "s3cr3t")
for _, option := range []string{"--env-file " + hostFile, "--label-file " + hostFile} {
logger, _ := test.NewNullLogger()
logger := logging.New(logtest.New())
cr := &containerReference{input: &NewContainerInput{NetworkMode: "bridge", WorkflowOptions: option}}
_, _, err := cr.mergeContainerConfigs(common.WithLogger(context.Background(), logger), &container.Config{}, &container.HostConfig{})
_, _, err := cr.mergeContainerConfigs(common.WithLogger(t.Context(), logger), &container.Config{}, &container.HostConfig{})
require.ErrorContains(t, err, "not allowed in a workflow")
// the runner reading its own files is what those options are for
cr = &containerReference{input: &NewContainerInput{NetworkMode: "bridge", RunnerOptions: option}}
_, _, err = cr.mergeContainerConfigs(common.WithLogger(context.Background(), logger), &container.Config{}, &container.HostConfig{})
_, _, err = cr.mergeContainerConfigs(common.WithLogger(t.Context(), logger), &container.Config{}, &container.HostConfig{})
require.NoError(t, err)
}
// a bare name is no longer resolved from the runner's environment, for either source
logger, _ := test.NewNullLogger()
logger := logging.New(logtest.New())
cr := &containerReference{input: &NewContainerInput{
NetworkMode: "bridge",
RunnerOptions: "--env RUNNER_SECRET",
WorkflowOptions: "--env RUNNER_SECRET --env GIVEN=value",
}}
config, _, err := cr.mergeContainerConfigs(common.WithLogger(context.Background(), logger), &container.Config{}, &container.HostConfig{})
config, _, err := cr.mergeContainerConfigs(common.WithLogger(t.Context(), logger), &container.Config{}, &container.HostConfig{})
require.NoError(t, err)
assert.Equal(t, []string{"RUNNER_SECRET", "RUNNER_SECRET", "GIVEN=value"}, config.Env)
}
func TestSanitizeOptionsHostConfig(t *testing.T) {
logger, _ := test.NewNullLogger()
logger := logging.New(logtest.New())
// every field the sanitizer resets, so a reset dropped in a refactor fails here
hostConfig := &container.HostConfig{
@@ -666,6 +701,9 @@ func TestSanitizeOptionsHostConfig(t *testing.T) {
DeviceCgroupRules: []string{"a *:* rwm"},
DeviceRequests: []container.DeviceRequest{{Count: -1, Capabilities: [][]string{{"gpu"}}}},
Sysctls: map[string]string{"net.ipv4.ip_forward": "1"},
LogConfig: container.LogConfig{Type: "syslog", Config: map[string]string{"syslog-address": "tcp://169.254.169.254:514"}},
OomScoreAdj: -1000,
OomKillDisable: new(true),
}
sanitizeOptionsHostConfig(logger, hostConfig, &container.HostConfig{})
@@ -674,17 +712,18 @@ func TestSanitizeOptionsHostConfig(t *testing.T) {
}
// mergeOptions merges both option sources into a bare container, returning the result and its log.
func mergeOptions(t *testing.T, runnerOptions, workflowOptions string, privileged bool) (*container.HostConfig, *test.Hook) {
func mergeOptions(t *testing.T, runnerOptions, workflowOptions string, privileged bool) (*container.HostConfig, *logtest.Handler) {
t.Helper()
logger, hook := test.NewNullLogger()
logger, hook := logtest.NewLogger()
cr := &containerReference{input: &NewContainerInput{
RunnerOptions: runnerOptions,
WorkflowOptions: workflowOptions,
NetworkMode: "bridge",
UsernsMode: "private",
ValidVolumes: []string{"**"},
}}
_, hostConfig, err := cr.mergeContainerConfigs(common.WithLogger(context.Background(), logger), &container.Config{}, &container.HostConfig{
_, hostConfig, err := cr.mergeContainerConfigs(common.WithLogger(t.Context(), logger), &container.Config{}, &container.HostConfig{
Privileged: privileged,
UsernsMode: container.UsernsMode("private"),
NetworkMode: container.NetworkMode("bridge"),
@@ -698,7 +737,9 @@ func TestMergeContainerConfigsStripsDangerousOptionsWhenUnprivileged(t *testing.
const dangerousOptions = "--pid=host --ipc=host --uts=host --cgroupns=host " +
"--userns=host --cap-add=ALL --security-opt seccomp=unconfined " +
"--security-opt apparmor=unconfined --volumes-from other --isolation process " +
"--runtime runc --cgroup-parent /custom --sysctl net.ipv4.ip_forward=1"
"--runtime runc --cgroup-parent /custom --sysctl net.ipv4.ip_forward=1 " +
"--log-driver=syslog --log-opt syslog-address=tcp://169.254.169.254:514 " +
"--oom-score-adj=-1000 --oom-kill-disable"
// whatever the workflow adds, an unprivileged container comes out exactly as the runner's
// own options alone describe it, field for field
@@ -725,8 +766,8 @@ func TestMergeContainerConfigsStripsDangerousOptionsWhenUnprivileged(t *testing.
}
func TestCheckVolumesRejectsEscapingHostPaths(t *testing.T) {
logger, _ := test.NewNullLogger()
ctx := common.WithLogger(context.Background(), logger)
logger := logging.New(logtest.New())
ctx := common.WithLogger(t.Context(), logger)
base := t.TempDir()
allowed := filepath.Join(base, "allowed")
@@ -761,6 +802,22 @@ func TestCheckVolumesRejectsEscapingHostPaths(t *testing.T) {
assert.Empty(t, hostConf.Binds)
}
func TestSanitizeConfigKeepsTmpfsMounts(t *testing.T) {
hostConfig, _ := mergeOptions(t, "", "--mount type=tmpfs,dst=/var/lib/mysql", false)
require.Len(t, hostConfig.Mounts, 1)
for _, validVolumes := range [][]string{nil, {"data"}, {"**:ro"}} {
logger, hook := logtest.NewLogger()
_, sanitized := (&containerReference{input: &NewContainerInput{ValidVolumes: validVolumes}}).sanitizeConfig(common.WithLogger(t.Context(), logger), &container.Config{}, &container.HostConfig{Mounts: []mount.Mount{
hostConfig.Mounts[0],
{Type: mount.TypeVolume, Target: "/anonymous"},
{Type: mount.TypeTmpfs, Source: "/etc", Target: "/etc-copy"},
}})
assert.Equal(t, hostConfig.Mounts, sanitized.Mounts)
assert.Len(t, hook.AllRecords(), 2)
}
}
func TestContainerInfoFromInspect(t *testing.T) {
t.Run("reports no healthcheck when the image declares none", func(t *testing.T) {
info := containerInfoFromInspect(container.InspectResponse{
@@ -840,12 +897,14 @@ func TestContainerInfoFromInspect(t *testing.T) {
}
func TestMergeContainerConfigsVolumesReplaceRunnerMounts(t *testing.T) {
logger, _ := test.NewNullLogger()
ctx := common.WithLogger(context.Background(), logger)
logger := logging.New(logtest.New())
ctx := common.WithLogger(t.Context(), logger)
cr := &containerReference{
input: &NewContainerInput{
NetworkMode: "bridge",
RunnerOptions: "--volume /host/tools:/opt/hostedtoolcache",
NetworkMode: "bridge",
RunnerOptions: "--volume /host/tools:/opt/hostedtoolcache",
WorkflowOptions: "--volume /unapproved:/var/run/docker.sock",
ValidVolumes: []string{"/host/tools"},
},
}
@@ -861,16 +920,46 @@ func TestMergeContainerConfigsVolumesReplaceRunnerMounts(t *testing.T) {
func TestMergeContainerConfigsWarnsOnlyAboutOptionsThatWereGiven(t *testing.T) {
warnings := func(runnerOptions, workflowOptions string) int {
_, hook := mergeOptions(t, runnerOptions, workflowOptions, false)
return len(hook.AllEntries())
return len(hook.AllRecords())
}
assert.Zero(t, warnings("--volume /host/tools:/opt/hostedtoolcache", ""))
assert.Zero(t, warnings("", "--shm-size 1g"))
assert.Equal(t, 1, warnings("--network host", ""))
assert.Equal(t, 1, warnings("", "--privileged"))
}
func TestDockerCreateKeepsInputEntrypoint(t *testing.T) {
for _, entrypoint := range [][]string{nil, {"sleep", "3600"}} {
for _, optionEntrypoint := range []string{"/sbin/dumb-init", ""} {
logger, hook := logtest.NewLogger()
cr := &containerReference{
input: &NewContainerInput{
NetworkMode: "bridge",
Entrypoint: entrypoint,
RunnerOptions: "--entrypoint=runner-entrypoint",
WorkflowOptions: "--entrypoint=" + optionEntrypoint + " --init",
},
cli: &probeClient{create: func(opts mobyclient.ContainerCreateOptions) (mobyclient.ContainerCreateResult, error) {
if len(entrypoint) > 0 {
assert.Equal(t, entrypoint, opts.Config.Entrypoint)
} else {
assert.Equal(t, []string{optionEntrypoint}, opts.Config.Entrypoint)
}
require.NotNil(t, opts.HostConfig.Init)
assert.True(t, *opts.HostConfig.Init)
return mobyclient.ContainerCreateResult{ID: "created"}, nil
}},
}
require.NoError(t, cr.create(nil, nil)(common.WithLogger(t.Context(), logger)))
assert.Equal(t, "created", cr.id)
assert.Equal(t, len(entrypoint) > 0, len(hook.AllRecords()) == 1)
}
}
}
func TestMergeContainerConfigsKeepsNetworkAliasesFromOptions(t *testing.T) {
logger, _ := test.NewNullLogger()
logger := logging.New(logtest.New())
cr := &containerReference{input: &NewContainerInput{
NetworkMode: "job-network", NetworkAliases: []string{"redis"}, WorkflowOptions: "--network-alias redis-primary",
}}

View File

@@ -107,6 +107,10 @@ func RemoveOrphanNetworks(_ context.Context, _, _ string, _ time.Time) error {
return nil
}
func SanitizeBinds(_ context.Context, _, binds []string) []string {
return binds
}
func CreateJobVolumes(ctx context.Context, runnerUUID string, volumeNames []string) error {
return nil
}

View File

@@ -4,7 +4,6 @@
package container
import (
"context"
"testing"
mobyclient "github.com/moby/moby/client"
@@ -15,7 +14,7 @@ import (
// so the daemon has to be pinged explicitly here to decide whether to skip.
func requireDocker(t *testing.T) {
t.Helper()
ctx := context.Background()
ctx := t.Context()
cli, err := GetDockerClient(ctx)
if err != nil {
t.Skipf("skipping: docker client unavailable: %v", err)

View File

@@ -27,9 +27,6 @@ import (
"gitea.com/gitea/runner/internal/pkg/process"
"github.com/creack/pty"
"github.com/go-git/go-billy/v5/helper/polyfill"
"github.com/go-git/go-billy/v5/osfs"
"github.com/go-git/go-git/v5/plumbing/format/gitignore"
"golang.org/x/term"
)
@@ -100,25 +97,16 @@ func (e *HostEnvironment) CopyDir(destPath, srcPath string, useGitIgnore, skipGi
srcPrefix += string(filepath.Separator)
}
logger.Debugf("Stripping prefix:%s src:%s", srcPrefix, srcPath)
var ignorer gitignore.Matcher
if useGitIgnore {
ps, err := gitignore.ReadPatterns(polyfill.New(osfs.New(srcPath)), nil)
if err != nil {
logger.Debugf("Error loading .gitignore: %v", err)
}
ignorer = gitignore.NewMatcher(ps)
}
fc := &filecollector.FileCollector{
Ignorer: ignorer,
SrcPath: srcPath,
SrcPrefix: srcPrefix,
SkipGitDir: skipGitDir,
SrcPath: srcPath,
SrcPrefix: srcPrefix,
SkipGitDir: skipGitDir,
UseGitIgnore: useGitIgnore,
Handler: &filecollector.CopyCollector{
DstDir: destPath,
},
}
return filepath.Walk(srcPath, fc.CollectFiles(ctx, []string{}))
return filepath.Walk(srcPath, fc.CollectFiles(ctx))
}
}
@@ -152,7 +140,7 @@ func (e *HostEnvironment) GetContainerArchive(ctx context.Context, srcPath strin
SrcPrefix: srcPrefix,
Handler: tc,
}
err = filepath.Walk(srcPath, fc.CollectFiles(ctx, []string{}))
err = filepath.Walk(srcPath, fc.CollectFiles(ctx))
if err != nil {
return nil, err
}

View File

@@ -18,8 +18,8 @@ import (
"time"
"gitea.com/gitea/runner/act/common"
"gitea.com/gitea/runner/act/common/logging"
"github.com/sirupsen/logrus"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
@@ -33,38 +33,25 @@ func TestActionPlatformNames(t *testing.T) {
}
func TestCopyDir(t *testing.T) {
dir := t.TempDir()
ctx := context.Background()
e := &HostEnvironment{
Path: filepath.Join(dir, "path"),
TmpDir: filepath.Join(dir, "tmp"),
ToolCache: filepath.Join(dir, "tool_cache"),
ActPath: filepath.Join(dir, "act_path"),
StdOut: os.Stdout,
Workdir: path.Join("testdata", "scratch"),
}
_ = os.MkdirAll(e.Path, 0o700)
_ = os.MkdirAll(e.TmpDir, 0o700)
_ = os.MkdirAll(e.ToolCache, 0o700)
_ = os.MkdirAll(e.ActPath, 0o700)
err := e.CopyDir(e.Workdir, e.Path, true, false)(ctx)
assert.NoError(t, err)
}
func TestCopyDirSkipGitDir(t *testing.T) {
for name, skipGitDir := range map[string]bool{"kept for a workspace copy": false, "skipped for an action copy": true} {
t.Run(name, func(t *testing.T) {
for _, test := range []struct {
name string
useGitIgnore, skipGitDir bool
}{
{name: "git dir kept for a workspace copy"},
{name: "git dir skipped for an action copy", skipGitDir: true},
{name: "unfiltered outside a git repository", useGitIgnore: true},
} {
t.Run(test.name, func(t *testing.T) {
src := filepath.Join(t.TempDir(), "action")
require.NoError(t, os.MkdirAll(filepath.Join(src, ".git", "objects"), 0o700))
require.NoError(t, os.WriteFile(filepath.Join(src, ".git", "objects", "pack"), []byte("x"), 0o600))
require.NoError(t, os.WriteFile(filepath.Join(src, "action.yml"), []byte("name: x\n"), 0o600))
dest := filepath.Join(t.TempDir(), "dest")
e := &HostEnvironment{StdOut: os.Stdout}
require.NoError(t, e.CopyDir(dest, src+string(filepath.Separator), false, skipGitDir)(context.Background()))
require.NoError(t, (&HostEnvironment{}).CopyDir(dest, src+string(filepath.Separator), test.useGitIgnore, test.skipGitDir)(t.Context()))
assert.FileExists(t, filepath.Join(dest, "action.yml"))
if skipGitDir {
if test.skipGitDir {
assert.NoDirExists(t, filepath.Join(dest, ".git"))
} else {
assert.FileExists(t, filepath.Join(dest, ".git", "objects", "pack"))
@@ -75,7 +62,7 @@ func TestCopyDirSkipGitDir(t *testing.T) {
func TestGetContainerArchive(t *testing.T) {
dir := t.TempDir()
ctx := context.Background()
ctx := t.Context()
e := &HostEnvironment{
Path: filepath.Join(dir, "path"),
TmpDir: filepath.Join(dir, "tmp"),
@@ -110,7 +97,7 @@ func TestHostEnvironmentExecExitCode(t *testing.T) {
t.Skip("uses POSIX shell")
}
dir := t.TempDir()
ctx := context.Background()
ctx := t.Context()
e := &HostEnvironment{
Path: filepath.Join(dir, "path"),
TmpDir: filepath.Join(dir, "tmp"),
@@ -161,7 +148,7 @@ func TestHostEnvironmentAllocatePTY(t *testing.T) {
err := e.Exec(
[]string{"sh", "-c", "[ -t 1 ] && printf TTY || printf NOTTY"},
map[string]string{"PATH": os.Getenv("PATH")}, "", "",
)(context.Background())
)(t.Context())
require.NoError(t, err)
got := strings.TrimSpace(strings.ReplaceAll(buf.String(), "\r", ""))
assert.Equal(t, tc.expect, got)
@@ -170,8 +157,7 @@ func TestHostEnvironmentAllocatePTY(t *testing.T) {
}
func TestHostEnvironmentRemovePreservesWorkdirByDefault(t *testing.T) {
logger := logrus.New()
ctx := common.WithLogger(context.Background(), logrus.NewEntry(logger))
ctx := common.WithLogger(t.Context(), logging.Default())
base := t.TempDir()
miscRoot := filepath.Join(base, "misc")
path := filepath.Join(miscRoot, "hostexecutor")
@@ -193,8 +179,7 @@ func TestHostEnvironmentRemovePreservesWorkdirByDefault(t *testing.T) {
}
func TestHostEnvironmentRemoveCleansWorkdirWhenOwned(t *testing.T) {
logger := logrus.New()
ctx := common.WithLogger(context.Background(), logrus.NewEntry(logger))
ctx := common.WithLogger(t.Context(), logging.Default())
base := t.TempDir()
miscRoot := filepath.Join(base, "misc")
path := filepath.Join(miscRoot, "hostexecutor")
@@ -235,7 +220,7 @@ func TestRemoveAllWithContextDoesNotHangOnStuckDelete(t *testing.T) {
removeAll = orig
})
ctx, cancel := context.WithTimeout(context.Background(), 100*time.Millisecond)
ctx, cancel := context.WithTimeout(t.Context(), 100*time.Millisecond)
defer cancel()
err := removeAllWithContext(ctx, t.TempDir())
@@ -252,8 +237,7 @@ func TestHostEnvironmentRemoveDoesNotHangOnStuckCleanUp(t *testing.T) {
hostCleanupTimeout = 100 * time.Millisecond
t.Cleanup(func() { hostCleanupTimeout = orig })
logger := logrus.New()
ctx := common.WithLogger(context.Background(), logrus.NewEntry(logger))
ctx := common.WithLogger(t.Context(), logging.Default())
base := t.TempDir()
path := filepath.Join(base, "misc", "hostexecutor")
require.NoError(t, os.MkdirAll(path, 0o700))
@@ -306,8 +290,7 @@ func TestHostEnvironmentRemoveDoesNotHangOnStuckPathRemoval(t *testing.T) {
removeAll = origRemoveAll
})
logger := logrus.New()
ctx := common.WithLogger(context.Background(), logrus.NewEntry(logger))
ctx := common.WithLogger(t.Context(), logging.Default())
base := t.TempDir()
path := filepath.Join(base, "misc", "hostexecutor")
require.NoError(t, os.MkdirAll(path, 0o700))

576
act/container/kubernetes.go Normal file
View File

@@ -0,0 +1,576 @@
// Copyright 2026 The Gitea Authors. All rights reserved.
// SPDX-License-Identifier: MIT
package container
import (
"bufio"
"bytes"
"cmp"
"context"
"crypto/rand"
"errors"
"fmt"
"io"
"maps"
"net/http"
"net/url"
"os"
"path"
"regexp"
"slices"
"strings"
"time"
"gitea.com/gitea/runner/act/common"
"github.com/kballard/go-shellquote"
"github.com/spf13/pflag"
)
// KubernetesOptions configures the pod a job runs in.
type KubernetesOptions struct {
Kubeconfig string
Namespace string
PodTemplates []map[string]any // merged in order, a later one winning
MaxLifetime time.Duration
ForcePull bool
RunnerUUID string
}
// KubernetesPod runs a job container and its service containers as one pod.
type KubernetesPod struct {
options KubernetesOptions
client *kubeClient
name, secretName string
job *kubernetesContainer
services []*kubernetesContainer
arch, path string
}
type kubernetesContainer struct {
LinuxContainerEnvironmentExtensions
pod *KubernetesPod
id string // the workflow's service id, name is its DNS label form
name string
input *NewContainerInput
probe map[string]any
}
type containerStatus struct {
Name string
Ready bool
State struct {
Waiting *struct{ Reason, Message string }
Running *struct{}
Terminated *struct {
ExitCode int
Message string
}
}
}
type podStatus struct {
Spec struct{ Containers []struct{} }
Status struct {
Phase, Reason, Message string
Conditions []struct{ Type, Status, Message string }
InitContainerStatuses, ContainerStatuses []containerStatus
}
}
func NewKubernetesPod(options KubernetesOptions) (*KubernetesPod, error) {
client, err := newKubeClient(options.Kubeconfig, options.Namespace)
if err != nil {
return nil, fmt.Errorf("connect to Kubernetes: %w", err)
}
return &KubernetesPod{options: options, client: client}, nil
}
// JobContainer returns the container of the pod the job's steps run in.
func (p *KubernetesPod) JobContainer(input *NewContainerInput) ExecutionsEnvironment {
p.job = &kubernetesContainer{pod: p, name: "job", input: input}
return p.job
}
var invalidLabelChars = regexp.MustCompile(`[^a-z0-9-]+`)
// KubernetesServiceName returns the container name and hostname of service id, made a DNS label as both must be.
func KubernetesServiceName(id string) string {
name := strings.Trim(invalidLabelChars.ReplaceAllString(strings.ToLower(id), "-"), "-")
return strings.TrimRight(name[:min(63, len(name))], "-")
}
// ServiceContainer returns a container of the pod the job reaches at hostname KubernetesServiceName(id).
func (p *KubernetesPod) ServiceContainer(id string, input *NewContainerInput) ExecutionsEnvironment {
service := &kubernetesContainer{pod: p, id: id, name: KubernetesServiceName(id), input: input}
p.services = append(p.services, service)
return service
}
func (p *KubernetesPod) create(ctx context.Context) error {
prefix := strings.Trim(strings.ToLower(p.job.input.Name), "-")
prefix = cmp.Or(strings.TrimRight(prefix[:min(40, len(prefix))], "-"), "gitea-runner") + "-"
name := prefix + strings.ToLower(rand.Text()[:8]) // chosen here, so cleanup knows it even when a create response is lost
labels := map[string]any{"app.kubernetes.io/managed-by": "gitea-runner"}
if p.options.RunnerUUID != "" {
labels[runnerUUIDLabel] = p.options.RunnerUUID
}
metadata := map[string]any{"name": name, "labels": labels}
if hostname, err := os.Hostname(); p.client.inCluster && err == nil {
var runner struct{ Metadata struct{ Name, UID string } }
if err := p.client.do(ctx, http.MethodGet, p.client.path("pods/"+hostname), nil, &runner); err == nil {
metadata["ownerReferences"] = []any{map[string]any{"apiVersion": "v1", "kind": "Pod", "name": runner.Metadata.Name, "uid": runner.Metadata.UID}}
} else {
common.Logger(ctx).Debugf("The runner is not pod %s, job pods get no owner: %v", hostname, err)
}
}
secretData := map[string]string{}
manifest, err := p.manifest(name, secretData, metadata)
if err != nil {
return err
}
for _, service := range p.services {
if !strings.EqualFold(service.id, service.name) {
common.Logger(ctx).Warnf("Service %s is reachable as %s or localhost, as a Kubernetes hostname cannot be %q", service.id, service.name, service.id)
}
}
if len(secretData) > 0 {
p.secretName = name // named like the pod, RemoveOrphanKubernetesResources finds it by that
if err := p.client.do(ctx, http.MethodPost, p.client.path("secrets"), map[string]any{"metadata": metadata, "stringData": secretData}, nil); err != nil {
return fmt.Errorf("create secret: %w", err)
}
}
p.name = name
if err := p.client.do(ctx, http.MethodPost, p.client.path("pods"), manifest, nil); err != nil {
return fmt.Errorf("create pod: %w", err)
}
return nil
}
func (p *KubernetesPod) manifest(secret string, secretData map[string]string, metadata map[string]any) (any, error) {
template := map[string]any{"spec": map[string]any{
"automountServiceAccountToken": false,
"enableServiceLinks": false,
"terminationGracePeriodSeconds": 0, // the job container's tail ignores SIGTERM
}}
for _, layer := range p.options.PodTemplates {
template, _ = mergeManifest(template, layer, "").(map[string]any)
}
templateSpec, _ := template["spec"].(map[string]any)
templateContainers, _ := templateSpec["containers"].([]any)
templateVolumes, _ := templateSpec["volumes"].([]any)
var serviceTemplate any
if index := slices.IndexFunc(templateContainers, func(container any) bool { return manifestField(container, "name") == "$services" }); index >= 0 {
serviceTemplate = templateContainers[index]
templateSpec["containers"] = slices.Delete(slices.Clone(templateContainers), index, index+1)
}
var containers, hostnames, volumes []any
names := map[string]bool{}
for _, container := range templateContainers {
names[manifestField(container, "name")] = true
}
for _, c := range append([]*kubernetesContainer{p.job}, p.services...) {
if c != p.job && (c.name == "" || names[c.name]) {
return nil, fmt.Errorf("service %s maps to container %q, which is empty or named like another container of the pod", c.id, c.name)
}
names[c.name] = true
spec, err := c.spec(secret, secretData)
if err != nil {
return nil, err
}
if c == p.job {
containers = append(containers, spec)
continue
}
containers = append(containers, mergeManifest(serviceTemplate, spec, ""))
hostnames = append(hostnames, c.name)
}
for _, name := range []string{"workspace", "act"} {
if !slices.ContainsFunc(templateVolumes, func(volume any) bool { return manifestField(volume, "name") == name }) {
volumes = append(volumes, map[string]any{"name": name, "emptyDir": map[string]any{}})
}
}
spec := map[string]any{"restartPolicy": "Never", "containers": containers, "volumes": volumes}
if len(hostnames) > 0 {
spec["hostAliases"] = []any{map[string]any{"ip": "127.0.0.1", "hostnames": hostnames}}
}
if p.options.MaxLifetime > 0 {
spec["activeDeadlineSeconds"] = max(1, int64(p.options.MaxLifetime.Seconds()))
}
return mergeManifest(template, map[string]any{"apiVersion": "v1", "kind": "Pod", "metadata": metadata, "spec": spec}, ""), nil
}
// mergeManifest merges over onto base like a strategic merge patch: maps key by key, map list items by name or volumeMounts by mountPath, env entries, volumes and other values replaced.
func mergeManifest(base, over any, field string) any {
switch over := over.(type) {
case map[string]any:
merged := map[string]any{}
if base, ok := base.(map[string]any); ok {
maps.Copy(merged, base)
}
for key, value := range over {
merged[key] = mergeManifest(merged[key], value, key)
}
return merged
case []any:
key := cmp.Or(map[string]string{"volumeMounts": "mountPath"}[field], "name")
baseList, _ := base.([]any)
merged := slices.Clone(baseList)
for _, item := range over {
if _, ok := item.(map[string]any); !ok {
return over
}
name := manifestField(item, key)
index := slices.IndexFunc(merged, func(existing any) bool { return name != "" && manifestField(existing, key) == name })
switch {
case index < 0:
merged = append(merged, item)
case field == "env" || field == "volumes": // each holds one source, merging two makes a pod the API rejects
merged[index] = item
default:
merged[index] = mergeManifest(merged[index], item, "")
}
}
return merged
}
return over
}
func manifestField(item any, key string) string {
fields, _ := item.(map[string]any)
value, _ := fields[key].(string)
return value
}
func (c *kubernetesContainer) spec(secret string, secretData map[string]string) (map[string]any, error) {
env := []any{}
for index, entry := range c.input.Env {
key, value, _ := strings.Cut(entry, "=")
if c == c.pod.job {
env = append(env, map[string]any{"name": key, "value": value})
continue
}
secretKey := fmt.Sprintf("%s.%d", c.name, index)
secretData[secretKey] = value
env = append(env, map[string]any{"name": key, "valueFrom": map[string]any{"secretKeyRef": map[string]any{"name": secret, "key": secretKey}}})
}
spec := map[string]any{
"name": c.name, "image": c.input.Image, "command": escapeExpansion(c.input.Entrypoint), "args": escapeExpansion(c.input.Cmd),
"workingDir": c.input.WorkingDir, "env": env,
}
if c.pod.options.ForcePull {
spec["imagePullPolicy"] = "Always"
}
if c == c.pod.job {
spec["volumeMounts"] = []any{
map[string]any{"name": "workspace", "mountPath": c.input.WorkingDir},
map[string]any{"name": "act", "mountPath": c.GetActPath()},
}
}
if err := c.parseOptions(); err != nil {
return nil, fmt.Errorf("service %s: %w", c.name, err)
}
if c.probe != nil {
spec["startupProbe"] = c.probe
}
return spec, nil
}
// escapeExpansion keeps kubelet from expanding $(VAR) and $$ in command and args, which docker passes as is.
func escapeExpansion(args []string) []string {
args = slices.Clone(args)
for index := range args {
args[index] = strings.ReplaceAll(args[index], "$", "$$")
}
return args
}
// parseOptions turns docker health options, the only ones a pod supports, into a startup probe, which like docker gives up after the retries.
func (c *kubernetesContainer) parseOptions() error {
args, err := shellquote.Split(c.input.WorkflowOptions)
if err != nil {
return err
}
flags := pflag.NewFlagSet("options", pflag.ContinueOnError)
flags.SetOutput(io.Discard)
command := flags.String("health-cmd", "", "")
interval := flags.Duration("health-interval", 0, "")
timeout := flags.Duration("health-timeout", 0, "")
retries := flags.Int("health-retries", 0, "")
startPeriod := flags.Duration("health-start-period", 0, "")
if err := flags.Parse(args); err != nil {
return fmt.Errorf("options %q: %w, only --health-* options are supported on Kubernetes", c.input.WorkflowOptions, err)
}
if *command == "" {
return nil
}
period := max(1, int(cmp.Or(*interval, 30*time.Second).Seconds()))
c.probe = map[string]any{
"exec": map[string]any{"command": []string{"/bin/sh", "-c", *command}},
"periodSeconds": period,
"timeoutSeconds": max(1, int(cmp.Or(*timeout, 30*time.Second).Seconds())),
"failureThreshold": max(1, cmp.Or(*retries, 3)) + int(startPeriod.Seconds())/period, // docker does not count failures in the start period
}
return nil
}
func (p *KubernetesPod) get(ctx context.Context) (*podStatus, error) {
var pod podStatus
return &pod, p.client.do(ctx, http.MethodGet, p.client.path("pods/"+p.name), nil, &pod)
}
func (p *KubernetesPod) waitStarted(ctx context.Context) error {
var unscheduled string
for delay := 100 * time.Millisecond; ; delay = min(2*delay, time.Second) {
pod, err := p.get(ctx)
if err != nil {
return err
}
if pod.Status.Phase == "Failed" {
return fmt.Errorf("pod %s failed: %s", p.name, cmp.Or(pod.Status.Message, pod.Status.Reason))
}
started := len(pod.Status.ContainerStatuses) == len(pod.Spec.Containers)
for _, status := range append(pod.Status.InitContainerStatuses, pod.Status.ContainerStatuses...) {
if waiting := status.State.Waiting; waiting != nil {
switch waiting.Reason {
case "ErrImagePull", "ImagePullBackOff", "InvalidImageName", "ErrImageNeverPull", "CreateContainerConfigError", "CreateContainerError", "RunContainerError":
return fmt.Errorf("container %s of pod %s: %s: %s", status.Name, p.name, waiting.Reason, waiting.Message)
}
started = false
}
}
if started {
return nil
}
for _, condition := range pod.Status.Conditions {
if condition.Type == "PodScheduled" && condition.Status == "False" && condition.Message != unscheduled {
unscheduled = condition.Message
common.Logger(ctx).Infof("Waiting for pod %s to be scheduled: %s", p.name, unscheduled)
}
}
select {
case <-ctx.Done():
return ctx.Err()
case <-time.After(delay):
}
}
}
func (c *kubernetesContainer) Create(_, _ []string) common.Executor {
return func(ctx context.Context) error {
if c != c.pod.job {
return nil
}
return c.pod.create(ctx)
}
}
func (c *kubernetesContainer) Start(_ bool) common.Executor {
return func(ctx context.Context) error {
if c != c.pod.job {
return nil
}
if err := c.pod.waitStarted(ctx); err != nil {
return err
}
var stdout bytes.Buffer
if err := c.pod.client.exec(ctx, c.pod.name, c.name, []string{"sh", "-c", `printf '%s\n%s' "$(uname -m)" "$PATH"`}, nil, &stdout, io.Discard); err != nil {
return err
}
machine, path, _ := strings.Cut(stdout.String(), "\n")
c.pod.arch, c.pod.path = goArchToActionArch(machine), path
return nil
}
}
func (*kubernetesContainer) Pull(bool) common.Executor {
return func(context.Context) error { return nil }
}
var (
shellName = regexp.MustCompile(`^[A-Za-z_][A-Za-z0-9_]*$`)
bashReadonly = []string{"BASHOPTS", "BASH_VERSINFO", "EUID", "PPID", "SHELLOPTS", "UID"} // bash as sh cannot export these
)
// Exec feeds the command to sh on stdin, which exports the env so it stays out of the request URL and, where sh can export a name, out of process arguments.
func (c *kubernetesContainer) Exec(command []string, env map[string]string, _, workdir string) common.Executor {
return func(ctx context.Context) error {
script, args := "", []string{"env", "--"}
for name, value := range env {
if shellName.MatchString(name) && !slices.Contains(bashReadonly, name) {
script += "command export " + shellquote.Join(name+"="+value) + "\n"
} else {
args = append(args, name+"="+value)
}
}
script += "exec " + shellquote.Join(append(args, command...)...)
if workdir != "" {
script = "cd " + shellquote.Join(workdir) + " || exit\n" + script
}
defer common.FlushWriter(c.input.Stdout)
defer common.FlushWriter(c.input.Stderr)
return c.pod.client.exec(ctx, c.pod.name, c.name, []string{"sh"}, strings.NewReader(script), c.input.Stdout, c.input.Stderr)
}
}
func (c *kubernetesContainer) extract(ctx context.Context, destPath string, write func(io.Writer) error) error {
reader, writer := io.Pipe()
defer reader.Close()
go func() { writer.CloseWithError(write(writer)) }()
var stderr bytes.Buffer
if err := c.pod.client.exec(ctx, c.pod.name, c.name, []string{"sh", "-c", `mkdir -p "$1" && tar -xf - -C "$1"`, "sh", destPath}, reader, io.Discard, &stderr); err != nil {
return fmt.Errorf("extract to %s: %w: %s", destPath, err, strings.TrimSpace(stderr.String()))
}
return nil
}
func (c *kubernetesContainer) Copy(destPath string, files ...*FileEntry) common.Executor {
return func(ctx context.Context) error {
return c.extract(ctx, destPath, func(writer io.Writer) error { return writeFilesTar(ctx, writer, 0, 0, files...) })
}
}
func (c *kubernetesContainer) CopyDir(destPath, srcPath string, useGitIgnore, skipGitDir bool) common.Executor {
return func(ctx context.Context) error {
return c.extract(ctx, destPath, func(writer io.Writer) error {
return writeDirTar(ctx, writer, "/", srcPath, useGitIgnore, skipGitDir, 0, 0)
})
}
}
// GetContainerArchive streams the archive, checking the path first as tar writes an empty archive for a missing one.
func (c *kubernetesContainer) GetContainerArchive(ctx context.Context, srcPath string) (io.ReadCloser, error) {
reader, writer := io.Pipe()
go func() {
var stderr bytes.Buffer
err := c.pod.client.exec(ctx, c.pod.name, c.name, []string{"sh", "-c", `test -e "$1" -o -L "$1" && exec tar -cf - -C "$2" "$3"`, "sh", srcPath, path.Dir(srcPath), path.Base(srcPath)}, nil, writer, &stderr)
if err != nil {
err = fmt.Errorf("archive %s: %w: %s", srcPath, err, strings.TrimSpace(stderr.String()))
}
writer.CloseWithError(err)
}()
buffered := bufio.NewReader(reader)
if _, err := buffered.Peek(1); err != nil {
reader.Close()
return nil, err
}
return struct {
io.Reader
io.Closer
}{buffered, reader}, nil
}
func (c *kubernetesContainer) UpdateFromEnv(srcPath string, env *map[string]string) common.Executor {
return parseEnvFile(c, srcPath, env)
}
func (c *kubernetesContainer) UpdateFromImageEnv(env *map[string]string) common.Executor {
return func(context.Context) error {
(*env)["PATH"] = cmp.Or((*env)["PATH"], c.pod.path)
return nil
}
}
func (c *kubernetesContainer) Inspect(ctx context.Context) (*Info, error) {
if c.pod.name == "" {
return nil, fmt.Errorf("container %s %w", c.name, ErrContainerNotFound)
}
pod, err := c.pod.get(ctx)
if isNotFound(err) {
return nil, fmt.Errorf("pod %s %w", c.pod.name, ErrContainerNotFound)
} else if err != nil {
return nil, err
}
info := &Info{ID: c.pod.name + "/" + c.name, State: "created", Health: HealthNone, Ports: map[string]string{}}
for port := range c.input.ExposedPorts {
info.Ports[port.Port()] = port.Port() // services share the pod's network
}
for _, status := range pod.Status.ContainerStatuses {
if status.Name != c.name {
continue
}
if status.State.Running != nil {
info.State = StateRunning
} else if terminated := status.State.Terminated; terminated != nil {
info.State, info.ExitCode, info.HealthOutput = "exited", terminated.ExitCode, terminated.Message
}
if c.probe != nil {
switch {
case status.Ready:
info.Health = HealthHealthy
case info.State == "exited":
info.Health = HealthUnhealthy
default:
info.Health = HealthStarting
}
}
}
return info, nil
}
func (c *kubernetesContainer) DumpLogs(ctx context.Context) error {
if c.pod.name == "" {
return fmt.Errorf("container %s %w", c.name, ErrContainerNotFound)
}
defer common.FlushWriter(c.input.Stdout)
return c.pod.client.do(ctx, http.MethodGet, c.pod.client.path("pods/"+c.pod.name+"/log?container="+c.name), nil, c.input.Stdout)
}
func (c *kubernetesContainer) Remove() common.Executor {
return func(ctx context.Context) error {
if c != c.pod.job {
return nil
}
return errors.Join(c.pod.client.delete(ctx, "pods", c.pod.name), c.pod.client.delete(ctx, "secrets", c.pod.secretName))
}
}
// RemoveOrphanKubernetesResources removes this runner's job pods created before createdBefore and the Secrets named like them, as listing Secrets would reveal all of the namespace's.
func RemoveOrphanKubernetesResources(ctx context.Context, options KubernetesOptions, createdBefore time.Time) error {
client, err := newKubeClient(options.Kubeconfig, options.Namespace)
if err != nil {
return fmt.Errorf("connect to Kubernetes: %w", err)
}
defer client.http.CloseIdleConnections()
var pods struct {
Items []struct {
Metadata struct {
Name string
CreationTimestamp time.Time
}
}
}
if err := client.do(ctx, http.MethodGet, client.path("pods?labelSelector="+url.QueryEscape(runnerUUIDLabel+"="+options.RunnerUUID)), nil, &pods); err != nil {
return fmt.Errorf("list pods: %w", err)
}
var errs []error
for _, item := range pods.Items {
if !item.Metadata.CreationTimestamp.Before(createdBefore) {
continue
}
if err := client.delete(ctx, "secrets", item.Metadata.Name); err != nil {
errs = append(errs, err) // leave the pod for the next cleanup, it is the only way back to its Secret
continue
}
errs = append(errs, client.delete(ctx, "pods", item.Metadata.Name))
}
return errors.Join(errs...)
}
func (c *kubernetesContainer) Close() common.Executor {
return func(context.Context) error {
c.pod.client.http.CloseIdleConnections()
return nil
}
}
func (c *kubernetesContainer) ReplaceLogWriter(stdout, stderr io.Writer) (io.Writer, io.Writer) {
oldStdout, oldStderr := c.input.Stdout, c.input.Stderr
c.input.Stdout, c.input.Stderr = stdout, stderr
return oldStdout, oldStderr
}
func (c *kubernetesContainer) GetRunnerContext(_ context.Context) map[string]any {
return map[string]any{"os": "Linux", "arch": c.pod.arch, "temp": "/tmp", "tool_cache": DefaultToolCache}
}

View File

@@ -0,0 +1,400 @@
// Copyright 2026 The Gitea Authors. All rights reserved.
// SPDX-License-Identifier: MIT
package container
import (
"bytes"
"cmp"
"context"
"crypto/tls"
"crypto/x509"
"encoding/base64"
"encoding/json"
"errors"
"fmt"
"io"
"net"
"net/http"
"net/url"
"os"
"path/filepath"
"strconv"
"strings"
"time"
"github.com/coder/websocket"
"go.yaml.in/yaml/v4"
)
const (
serviceAccountDir = "/var/run/secrets/kubernetes.io/serviceaccount"
execProtocol = "v5.channel.k8s.io"
)
type kubeClient struct {
server, namespace, token string
tokenFile string // re-read on every request, service account tokens rotate
inCluster bool
http *http.Client
}
type apiError struct {
code int
message string
}
func (e *apiError) Error() string { return e.message }
func isNotFound(err error) bool {
var apiErr *apiError
return errors.As(err, &apiErr) && apiErr.code == http.StatusNotFound
}
func newKubeClient(kubeconfig, namespace string) (*kubeClient, error) {
client := &kubeClient{namespace: namespace}
transport := &http.Transport{Proxy: http.ProxyFromEnvironment, TLSClientConfig: &tls.Config{MinVersion: tls.VersionTLS12}}
if host := os.Getenv("KUBERNETES_SERVICE_HOST"); host != "" && kubeconfig == "" {
if err := client.loadInCluster(host, transport.TLSClientConfig); err != nil {
return nil, err
}
} else if err := client.loadKubeconfig(kubeconfig, transport); err != nil {
return nil, err
}
client.http = &http.Client{Transport: transport}
return client, nil
}
func (c *kubeClient) loadInCluster(host string, tlsConfig *tls.Config) error {
c.server = "https://" + net.JoinHostPort(host, os.Getenv("KUBERNETES_SERVICE_PORT"))
c.tokenFile = serviceAccountDir + "/token"
ca, err := os.ReadFile(serviceAccountDir + "/ca.crt")
if err != nil {
return err
}
tlsConfig.RootCAs = x509.NewCertPool()
tlsConfig.RootCAs.AppendCertsFromPEM(ca)
if c.namespace == "" {
namespace, err := os.ReadFile(serviceAccountDir + "/namespace")
if err != nil {
return err
}
c.namespace = strings.TrimSpace(string(namespace))
}
c.inCluster = true
return nil
}
func (c *kubeClient) loadKubeconfig(path string, transport *http.Transport) error {
if paths := filepath.SplitList(os.Getenv("KUBECONFIG")); path == "" && len(paths) > 0 {
path = paths[0]
}
if path == "" {
home, err := os.UserHomeDir()
if err != nil {
return err
}
path = filepath.Join(home, ".kube", "config")
}
content, err := os.ReadFile(path)
if err != nil {
return fmt.Errorf("no in-cluster service account and no kubeconfig: %w", err)
}
var config struct {
CurrentContext string `yaml:"current-context"`
Contexts []struct {
Name string
Context struct{ Cluster, User, Namespace string }
}
Clusters []struct {
Name string
Cluster struct {
Server string
TLSServerName string `yaml:"tls-server-name"`
InsecureSkipTLSVerify bool `yaml:"insecure-skip-tls-verify"`
ProxyURL string `yaml:"proxy-url"`
CertificateAuthority string `yaml:"certificate-authority"`
CertificateAuthorityData string `yaml:"certificate-authority-data"`
}
}
Users []struct {
Name string
User struct {
Token string
TokenFile string `yaml:"tokenFile"`
ClientCertificate string `yaml:"client-certificate"`
ClientCertificateData string `yaml:"client-certificate-data"`
ClientKey string `yaml:"client-key"`
ClientKeyData string `yaml:"client-key-data"`
Exec any
AuthProvider any `yaml:"auth-provider"`
}
}
}
if err := yaml.Unmarshal(content, &config); err != nil {
return fmt.Errorf("parse kubeconfig %s: %w", path, err)
}
var current struct{ Cluster, User, Namespace string }
for _, entry := range config.Contexts {
if entry.Name == config.CurrentContext {
current = entry.Context
}
}
if current.Cluster == "" {
return fmt.Errorf("kubeconfig %s has no current context", path)
}
c.namespace = cmp.Or(c.namespace, current.Namespace, "default")
resolve := func(file string) string {
if file == "" || filepath.IsAbs(file) {
return file
}
return filepath.Join(filepath.Dir(path), file)
}
read := func(data, file string) ([]byte, error) {
if data != "" {
return base64.StdEncoding.DecodeString(data)
}
if file == "" {
return nil, nil
}
return os.ReadFile(resolve(file))
}
tlsConfig := transport.TLSClientConfig
for _, cluster := range config.Clusters {
if cluster.Name != current.Cluster {
continue
}
c.server = strings.TrimSuffix(cluster.Cluster.Server, "/")
tlsConfig.ServerName, tlsConfig.InsecureSkipVerify = cluster.Cluster.TLSServerName, cluster.Cluster.InsecureSkipTLSVerify
if cluster.Cluster.ProxyURL != "" {
proxy, err := url.Parse(cluster.Cluster.ProxyURL)
if err != nil {
return fmt.Errorf("kubeconfig %s: %w", path, err)
}
transport.Proxy = http.ProxyURL(proxy)
}
ca, err := read(cluster.Cluster.CertificateAuthorityData, cluster.Cluster.CertificateAuthority)
if err != nil {
return err
}
if ca != nil {
tlsConfig.RootCAs = x509.NewCertPool()
tlsConfig.RootCAs.AppendCertsFromPEM(ca)
}
}
for _, user := range config.Users {
if user.Name != current.User {
continue
}
if user.User.Exec != nil || user.User.AuthProvider != nil {
return fmt.Errorf("kubeconfig %s: user %q authenticates through a credential plugin, which the runner does not support, use a token or client certificate", path, user.Name)
}
c.token, c.tokenFile = user.User.Token, resolve(user.User.TokenFile)
cert, err := read(user.User.ClientCertificateData, user.User.ClientCertificate)
if err != nil {
return err
}
key, err := read(user.User.ClientKeyData, user.User.ClientKey)
if err != nil {
return err
}
if cert != nil {
pair, err := tls.X509KeyPair(cert, key)
if err != nil {
return err
}
tlsConfig.Certificates = []tls.Certificate{pair}
}
}
if c.server == "" {
return fmt.Errorf("kubeconfig %s names no server for context %q", path, config.CurrentContext)
}
return nil
}
func (c *kubeClient) authorize(header http.Header) error {
token := c.token
if c.tokenFile != "" {
content, err := os.ReadFile(c.tokenFile)
if err != nil {
return err
}
token = strings.TrimSpace(string(content))
}
if token != "" {
header.Set("Authorization", "Bearer "+token)
}
return nil
}
func (c *kubeClient) path(resource string) string {
return "/api/v1/namespaces/" + c.namespace + "/" + resource
}
func (c *kubeClient) do(ctx context.Context, method, path string, in, out any) error {
var body io.Reader
if in != nil {
content, err := json.Marshal(in)
if err != nil {
return err
}
body = bytes.NewReader(content)
}
req, err := http.NewRequestWithContext(ctx, method, c.server+path, body)
if err != nil {
return err
}
if err := c.authorize(req.Header); err != nil {
return err
}
req.Header.Set("Content-Type", "application/json")
resp, err := c.http.Do(req)
if err != nil {
return err
}
defer resp.Body.Close()
if resp.StatusCode >= http.StatusMultipleChoices {
return statusError(resp)
}
if out == nil {
out = io.Discard // draining lets the connection be reused
}
if writer, ok := out.(io.Writer); ok {
_, err = io.Copy(writer, resp.Body)
return err
}
return json.NewDecoder(resp.Body).Decode(out)
}
func (c *kubeClient) delete(ctx context.Context, resource, name string) error {
if name == "" {
return nil
}
if err := c.do(ctx, http.MethodDelete, c.path(resource+"/"+name), nil, nil); err != nil && !isNotFound(err) {
return fmt.Errorf("delete %s %s: %w", resource, name, err)
}
return nil
}
func statusError(resp *http.Response) error {
content, _ := io.ReadAll(resp.Body)
var status struct{ Message string }
if json.Unmarshal(content, &status) != nil || status.Message == "" {
status.Message = strings.TrimSpace(string(content))
}
return &apiError{code: resp.StatusCode, message: status.Message}
}
func (c *kubeClient) exec(ctx context.Context, pod, containerName string, command []string, stdin io.Reader, stdout, stderr io.Writer) error {
query := url.Values{"container": {containerName}, "command": command, "stdout": {"true"}, "stderr": {"true"}}
if stdin != nil {
query.Set("stdin", "true")
}
header := http.Header{}
if err := c.authorize(header); err != nil {
return err
}
ctx, cancel := context.WithCancelCause(ctx)
defer cancel(nil)
conn, resp, err := websocket.Dial(ctx, c.server+c.path("pods/"+pod+"/exec?")+query.Encode(), &websocket.DialOptions{
HTTPClient: c.http,
HTTPHeader: header,
Subprotocols: []string{execProtocol},
})
if err != nil {
if resp != nil && resp.StatusCode >= http.StatusMultipleChoices {
defer resp.Body.Close()
err = statusError(resp)
}
return fmt.Errorf("exec in pod %s: %w", pod, err)
}
defer func() { _ = conn.CloseNow() }()
if conn.Subprotocol() != execProtocol {
return fmt.Errorf("the Kubernetes API server does not speak %s, which needs Kubernetes 1.30 or later", execProtocol)
}
conn.SetReadLimit(-1)
if stdin != nil {
go func() {
if err := sendStdin(ctx, conn, stdin); err != nil {
cancel(err)
}
}()
}
go func() {
ticker := time.NewTicker(5 * time.Second)
defer ticker.Stop()
for {
select {
case <-ctx.Done():
return
case <-ticker.C:
if conn.Ping(ctx) != nil {
return
}
}
}
}()
for {
_, frame, err := conn.Read(ctx)
if err != nil {
return cmp.Or(context.Cause(ctx), fmt.Errorf("exec in pod %s ended without an exit status: %w", pod, err))
}
if len(frame) < 2 {
continue
}
switch frame[0] {
case 1:
_, err = stdout.Write(frame[1:])
case 2:
_, err = stderr.Write(frame[1:])
case 3:
return cmp.Or(context.Cause(ctx), execStatus(frame[1:]))
}
if err != nil {
return err
}
}
}
func sendStdin(ctx context.Context, conn *websocket.Conn, stdin io.Reader) error {
frame := make([]byte, 1+32*1024) // frame[0] is the stdin channel, 0
for {
n, err := stdin.Read(frame[1:])
if n > 0 {
if err := conn.Write(ctx, websocket.MessageBinary, frame[:1+n]); err != nil {
return err
}
}
if errors.Is(err, io.EOF) {
return conn.Write(ctx, websocket.MessageBinary, []byte{255, 0}) // v5 close signal for stdin
}
if err != nil {
return err
}
}
}
func execStatus(content []byte) error {
var status struct {
Status, Message string
Details struct {
Causes []struct{ Reason, Message string }
}
}
if err := json.Unmarshal(content, &status); err != nil {
return fmt.Errorf("decode exec status: %w", err)
}
if status.Status == "Success" {
return nil
}
for _, cause := range status.Details.Causes {
if code, err := strconv.Atoi(cause.Message); cause.Reason == "ExitCode" && err == nil {
return ExitCodeError(code)
}
}
return fmt.Errorf("exec failed: %s", status.Message)
}

View File

@@ -0,0 +1,300 @@
// Copyright 2026 The Gitea Authors. All rights reserved.
// SPDX-License-Identifier: MIT
package container
import (
"archive/tar"
"bytes"
"encoding/json"
"errors"
"fmt"
"io"
"net/http"
"net/http/httptest"
"net/url"
"os"
"os/exec"
"path/filepath"
"runtime"
"strings"
"sync"
"testing"
"time"
"github.com/coder/websocket"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
type podManifest struct {
Metadata struct{ Labels map[string]string }
Spec struct {
AutomountServiceAccountToken bool
EnableServiceLinks bool
HostAliases []struct{ Hostnames []string }
Volumes []map[string]any
Containers []struct {
Name, ImagePullPolicy string
Resources map[string]any
StartupProbe *struct{ PeriodSeconds, TimeoutSeconds, FailureThreshold int }
}
}
}
type fakeCluster struct {
mu sync.Mutex
requests map[string][]string
}
func (f *fakeCluster) record(request, content string) {
f.mu.Lock()
defer f.mu.Unlock()
f.requests[request] = append(f.requests[request], content)
}
func fakeKubernetes(t *testing.T, podStatus string) (string, *fakeCluster) {
cluster := &fakeCluster{requests: map[string][]string{}}
server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
if r.Header.Get("Authorization") != "Bearer test-token" {
w.WriteHeader(http.StatusUnauthorized)
return
}
body, err := io.ReadAll(r.Body)
assert.NoError(t, err)
cluster.record(r.Method+" "+r.URL.Path, string(body))
pods, secrets := "/api/v1/namespaces/ci/pods", "/api/v1/namespaces/ci/secrets"
switch {
case r.Method == http.MethodGet && r.URL.Path == pods:
cluster.record("list", r.URL.Query().Get("labelSelector"))
_, _ = io.WriteString(w, `{"items":[{"metadata":{"name":"old","creationTimestamp":"2020-01-01T00:00:00Z"}},{"metadata":{"name":"new","creationTimestamp":"2999-01-01T00:00:00Z"}}]}`)
case strings.HasSuffix(r.URL.Path, "/exec"):
cluster.record("exec", r.URL.RawQuery)
serveLocalExec(t, w, r)
case strings.HasSuffix(r.URL.Path, "/log"):
_, _ = io.WriteString(w, "log of "+r.URL.Query().Get("container"))
case r.Method == http.MethodGet && strings.HasPrefix(r.URL.Path, pods+"/"):
_, _ = io.WriteString(w, podStatus)
case r.Method == http.MethodPost && (r.URL.Path == pods || r.URL.Path == secrets), r.Method == http.MethodDelete:
default:
w.WriteHeader(http.StatusNotFound)
}
}))
t.Cleanup(server.Close)
kubeconfig := filepath.Join(t.TempDir(), "kubeconfig")
require.NoError(t, os.WriteFile(filepath.Join(filepath.Dir(kubeconfig), "token"), []byte("test-token\n"), 0o600))
require.NoError(t, os.WriteFile(kubeconfig, fmt.Appendf(nil, `
current-context: test
contexts: [{name: test, context: {cluster: test, user: test, namespace: ci}}]
clusters: [{name: test, cluster: {server: "http://kubernetes.invalid", proxy-url: %q}}]
users: [{name: test, user: {tokenFile: token}}]
`, server.URL), 0o600))
return kubeconfig, cluster
}
func serveLocalExec(t *testing.T, w http.ResponseWriter, r *http.Request) {
conn, err := websocket.Accept(w, r, &websocket.AcceptOptions{Subprotocols: []string{execProtocol}})
if err != nil {
t.Error(err)
return
}
defer func() { _ = conn.CloseNow() }()
conn.SetReadLimit(-1)
command := r.URL.Query()["command"]
var stdin bytes.Buffer
for r.URL.Query().Get("stdin") == "true" {
_, frame, err := conn.Read(r.Context())
if err != nil {
t.Error(err)
return
}
if frame[0] == 255 {
break
}
stdin.Write(frame[1:])
}
cmd := exec.Command(command[0], command[1:]...)
cmd.Env = append(os.Environ(), "COPYFILE_DISABLE=1")
var stdout, stderr bytes.Buffer
cmd.Stdin, cmd.Stdout, cmd.Stderr = &stdin, &stdout, &stderr
status := `{"status":"Success"}`
var exitErr *exec.ExitError
if err := cmd.Run(); errors.As(err, &exitErr) {
status = fmt.Sprintf(`{"status":"Failure","reason":"NonZeroExitCode","details":{"causes":[{"reason":"ExitCode","message":"%d"}]}}`, exitErr.ExitCode())
} else {
assert.NoError(t, err)
}
for _, frame := range [][]byte{append([]byte{1}, stdout.Bytes()...), append([]byte{2}, stderr.Bytes()...), append([]byte{3}, status...)} {
_ = conn.Write(r.Context(), websocket.MessageBinary, frame)
}
}
func TestKubernetesPodRunsJobThroughAPI(t *testing.T) {
if runtime.GOOS == "windows" {
t.Skip("uses POSIX shell")
}
ctx := t.Context()
kubeconfig, cluster := fakeKubernetes(t, `{"spec":{"containers":[{},{}]},"status":{"phase":"Running","containerStatuses":[
{"name":"job","ready":true,"state":{"running":{}}},
{"name":"my-db","ready":true,"state":{"running":{}}}]}}`)
workdir, err := filepath.EvalSymlinks(t.TempDir())
require.NoError(t, err)
options := KubernetesOptions{Kubeconfig: kubeconfig, RunnerUUID: "runner-uuid", PodTemplates: []map[string]any{
{"spec": map[string]any{
"automountServiceAccountToken": true,
"containers": []any{
map[string]any{"name": "job", "resources": map[string]any{"limits": map[string]any{"cpu": "2"}}},
map[string]any{
"name": "$services", "imagePullPolicy": "Always", "resources": map[string]any{"limits": map[string]any{"memory": "256Mi"}},
"env": []any{map[string]any{"name": "POSTGRES_PASSWORD", "value": "template"}},
},
},
}},
{"spec": map[string]any{
"volumes": []any{map[string]any{"name": "workspace", "emptyDir": map[string]any{"sizeLimit": "1Gi"}}},
"containers": []any{map[string]any{"name": "job", "resources": map[string]any{"limits": map[string]any{"memory": "1Gi"}}}},
}},
}}
pod, err := NewKubernetesPod(options)
require.NoError(t, err)
var jobLog, serviceLog bytes.Buffer
job := pod.JobContainer(&NewContainerInput{Name: "Job-Name", Image: "node", WorkingDir: workdir, Stdout: &jobLog, Stderr: &jobLog})
service := pod.ServiceContainer("My_DB", &NewContainerInput{
Image: "postgres",
Cmd: []string{"postgres", "-D", "$(PGDATA)"},
Env: []string{"POSTGRES_PASSWORD=service-secret"},
WorkflowOptions: "--health-cmd pg_isready --health-interval 2s --health-start-period 4s",
Stdout: &serviceLog,
})
require.NoError(t, job.Create(nil, nil)(ctx))
var manifest podManifest
require.NoError(t, json.Unmarshal([]byte(cluster.requests["POST /api/v1/namespaces/ci/pods"][0]), &manifest))
assert.Equal(t, "gitea-runner", manifest.Metadata.Labels["app.kubernetes.io/managed-by"])
assert.Equal(t, "runner-uuid", manifest.Metadata.Labels[runnerUUIDLabel])
assert.True(t, manifest.Spec.AutomountServiceAccountToken)
assert.False(t, manifest.Spec.EnableServiceLinks)
assert.Equal(t, []string{"my-db"}, manifest.Spec.HostAliases[0].Hostnames)
require.Len(t, manifest.Spec.Containers, 2)
assert.Equal(t, map[string]any{"limits": map[string]any{"cpu": "2", "memory": "1Gi"}}, manifest.Spec.Containers[0].Resources)
assert.Empty(t, manifest.Spec.Containers[0].ImagePullPolicy)
assert.Equal(t, "my-db", manifest.Spec.Containers[1].Name)
assert.Equal(t, "Always", manifest.Spec.Containers[1].ImagePullPolicy)
assert.Equal(t, map[string]any{"limits": map[string]any{"memory": "256Mi"}}, manifest.Spec.Containers[1].Resources)
assert.Equal(t, []map[string]any{
{"name": "workspace", "emptyDir": map[string]any{"sizeLimit": "1Gi"}},
{"name": "act", "emptyDir": map[string]any{}},
}, manifest.Spec.Volumes)
assert.Equal(t, struct{ PeriodSeconds, TimeoutSeconds, FailureThreshold int }{2, 30, 5}, *manifest.Spec.Containers[1].StartupProbe)
assert.NotContains(t, cluster.requests["POST /api/v1/namespaces/ci/pods"][0], "service-secret")
assert.Contains(t, cluster.requests["POST /api/v1/namespaces/ci/secrets"][0], "service-secret")
assert.Contains(t, cluster.requests["POST /api/v1/namespaces/ci/secrets"][0], `"com.gitea.runner.uuid":"runner-uuid"`)
assert.Contains(t, cluster.requests["POST /api/v1/namespaces/ci/pods"][0], `{"name":"POSTGRES_PASSWORD","valueFrom":{"secretKeyRef":{"key":"my-db.0","name":"`+pod.secretName+`"}}}`)
assert.Equal(t, []any{"new"}, mergeManifest([]any{"old"}, []any{"new"}, ""))
mount := func(path string) any { return map[string]any{"name": "config", "mountPath": path} }
assert.Equal(t, []any{mount("/a"), mount("/b")}, mergeManifest([]any{mount("/a")}, []any{mount("/b")}, "volumeMounts"))
tmpfs := map[string]any{"name": "workspace", "emptyDir": map[string]any{"medium": "Memory"}}
ephemeral := map[string]any{"name": "workspace", "ephemeral": map[string]any{}}
assert.Equal(t, []any{ephemeral}, mergeManifest([]any{tmpfs}, []any{ephemeral}, "volumes"))
assert.Contains(t, cluster.requests["POST /api/v1/namespaces/ci/pods"][0], `"args":["postgres","-D","$$(PGDATA)"]`)
require.NoError(t, os.WriteFile(filepath.Join(workdir, "env"), []byte("#!/bin/sh\necho \"$@\" > \"$0-args\"\nexec /usr/bin/env \"$@\"\n"), 0o755))
t.Setenv("PATH", workdir+":"+os.Getenv("PATH"))
require.NoError(t, job.Start(false)(ctx))
stepEnv := map[string]string{"INPUT_WHO-TO-GREET": "it's\nme", "SECRET": "hunter2", "UID": "5"}
require.NoError(t, job.Exec([]string{"printenv", "SECRET"}, stepEnv, "", "")(ctx))
envArgs, err := os.ReadFile(filepath.Join(workdir, "env-args"))
require.NoError(t, err)
assert.NotContains(t, string(envArgs), "hunter2")
require.NoError(t, job.Exec([]string{"printenv", "INPUT_WHO-TO-GREET"}, stepEnv, "", "")(ctx))
require.NoError(t, job.Exec([]string{"printenv", "UID"}, stepEnv, "", "")(ctx))
require.NoError(t, job.Exec([]string{"pwd", "-P"}, nil, "", workdir)(ctx))
assert.Equal(t, "hunter2\nit's\nme\n5\n"+workdir+"\n", jobLog.String())
assert.Equal(t, ExitCodeError(3), job.Exec([]string{"sh", "-c", "echo failing >&2; exit 3"}, nil, "", "")(ctx))
assert.Contains(t, jobLog.String(), "failing\n")
for _, query := range cluster.requests["exec"] {
assert.NotContains(t, query, "hunter2")
}
require.NoError(t, job.Copy(workdir+"/act/", &FileEntry{Name: "workflow/event.json", Mode: 0o644, Body: `{"ref":"main"}`})(ctx))
archive, err := job.GetContainerArchive(ctx, workdir+"/act/workflow/event.json")
require.NoError(t, err)
reader := tar.NewReader(archive)
header, err := reader.Next()
require.NoError(t, err)
content, err := io.ReadAll(reader)
require.NoError(t, err)
assert.Equal(t, "event.json", header.Name)
assert.JSONEq(t, `{"ref":"main"}`, string(content))
require.NoError(t, archive.Close())
_, err = job.GetContainerArchive(ctx, workdir+"/missing")
require.Error(t, err)
actionDir := filepath.Join(t.TempDir(), "action")
require.NoError(t, os.MkdirAll(actionDir, 0o755))
require.NoError(t, os.WriteFile(filepath.Join(actionDir, "index.js"), []byte("main()"), 0o644))
require.NoError(t, job.CopyDir(workdir+"/act/actions/sha/", actionDir+"/", false, false)(ctx))
script, err := os.ReadFile(workdir + "/act/actions/sha/index.js")
require.NoError(t, err)
assert.Equal(t, "main()", string(script))
assert.Contains(t, cluster.requests["exec"][len(cluster.requests["exec"])-1], "command="+url.QueryEscape(workdir+"/act/actions/sha/"))
pathEnv := map[string]string{}
require.NoError(t, job.UpdateFromImageEnv(&pathEnv)(ctx))
assert.Equal(t, os.Getenv("PATH"), pathEnv["PATH"])
info, err := service.Inspect(ctx)
require.NoError(t, err)
assert.Equal(t, HealthHealthy, info.Health)
require.NoError(t, service.DumpLogs(ctx))
assert.Equal(t, "log of my-db", serviceLog.String())
require.NoError(t, job.Remove()(ctx))
assert.Len(t, cluster.requests["DELETE /api/v1/namespaces/ci/pods/"+pod.name], 1)
assert.Len(t, cluster.requests["DELETE /api/v1/namespaces/ci/secrets/"+pod.secretName], 1)
require.NoError(t, RemoveOrphanKubernetesResources(ctx, options, time.Now()))
assert.Equal(t, []string{runnerUUIDLabel + "=runner-uuid"}, cluster.requests["list"])
assert.Len(t, cluster.requests["DELETE /api/v1/namespaces/ci/pods/old"], 1)
assert.Len(t, cluster.requests["DELETE /api/v1/namespaces/ci/secrets/old"], 1)
assert.Empty(t, cluster.requests["DELETE /api/v1/namespaces/ci/pods/new"])
assert.Empty(t, cluster.requests["DELETE /api/v1/namespaces/ci/secrets/new"])
}
func TestKubernetesPodFailsBeforeSteps(t *testing.T) {
kubeconfig, _ := fakeKubernetes(t, `{"spec":{"containers":[{}]},"status":{"containerStatuses":[
{"name":"job","state":{"waiting":{"reason":"ErrImagePull","message":"no such image"}}}]}}`)
template := map[string]any{"spec": map[string]any{"containers": []any{map[string]any{"name": "sidecar"}}}}
for _, test := range []struct {
services []string
options, err string
}{
{[]string{"db"}, "--hostname db", "only --health-* options are supported"},
{[]string{"sidecar"}, "", "named like another container"},
{[]string{"my_db", "My-DB"}, "", `service My-DB maps to container "my-db"`},
} {
pod, err := NewKubernetesPod(KubernetesOptions{Kubeconfig: kubeconfig, Namespace: "ci", PodTemplates: []map[string]any{template}})
require.NoError(t, err)
job := pod.JobContainer(&NewContainerInput{Image: "node"})
for _, service := range test.services {
pod.ServiceContainer(service, &NewContainerInput{Image: "postgres", WorkflowOptions: test.options})
}
require.ErrorContains(t, job.Create(nil, nil)(t.Context()), test.err)
}
pod, err := NewKubernetesPod(KubernetesOptions{Kubeconfig: kubeconfig, Namespace: "ci"})
require.NoError(t, err)
job := pod.JobContainer(&NewContainerInput{Image: "missing"})
require.NoError(t, job.Create(nil, nil)(t.Context()))
require.ErrorContains(t, job.Start(false)(t.Context()), "ErrImagePull: no such image")
plugin := filepath.Join(t.TempDir(), "kubeconfig")
require.NoError(t, os.WriteFile(plugin, []byte(`
current-context: eks
contexts: [{name: eks, context: {cluster: eks, user: eks}}]
clusters: [{name: eks, cluster: {server: "https://eks.example"}}]
users: [{name: eks, user: {exec: {command: aws}}}]
`), 0o600))
_, err = NewKubernetesPod(KubernetesOptions{Kubeconfig: plugin})
require.ErrorContains(t, err, "credential plugin")
}

View File

@@ -11,7 +11,7 @@ import (
"runtime"
"strings"
log "github.com/sirupsen/logrus"
"gitea.com/gitea/runner/act/common/logging"
)
type LinuxContainerEnvironmentExtensions struct{}
@@ -21,13 +21,13 @@ type LinuxContainerEnvironmentExtensions struct{}
// For use in docker volumes and binds
func (*LinuxContainerEnvironmentExtensions) ToContainerPath(path string) string {
if runtime.GOOS == "windows" && strings.Contains(path, "/") {
log.Error("You cannot specify linux style local paths (/mnt/etc) on Windows as it does not understand them.")
logging.Error("You cannot specify linux style local paths (/mnt/etc) on Windows as it does not understand them.")
return ""
}
abspath, err := filepath.Abs(path)
if err != nil {
log.Error(err)
logging.Error(err)
return ""
}

View File

@@ -11,7 +11,8 @@ import (
"strings"
"testing"
log "github.com/sirupsen/logrus"
"gitea.com/gitea/runner/act/common/logging"
"github.com/stretchr/testify/assert"
)
@@ -27,7 +28,7 @@ func TestContainerPath(t *testing.T) {
if runtime.GOOS == "windows" {
cwd, err := os.Getwd()
if err != nil {
log.Error(err)
logging.Error(err)
}
rootDrive := os.Getenv("SystemDrive")
@@ -49,7 +50,7 @@ func TestContainerPath(t *testing.T) {
} else {
cwd, err := os.Getwd()
if err != nil {
log.Error(err)
logging.Error(err)
}
for _, v := range []containerPathJob{
{"/home/act/go/src/github.com/nektos/act", "/home/act/go/src/github.com/nektos/act", ""},

View File

@@ -5,7 +5,6 @@ package container
import (
"bufio"
"context"
"os"
"path/filepath"
"strings"
@@ -28,7 +27,7 @@ func TestParseEnvFileSingleLine(t *testing.T) {
require.NoError(t, os.WriteFile(envPath, []byte("FOO=bar\nBAZ=qux\n"), 0o600))
env := map[string]string{}
require.NoError(t, parseEnvFile(e, envPath, &env)(context.Background()))
require.NoError(t, parseEnvFile(e, envPath, &env)(t.Context()))
assert.Equal(t, "bar", env["FOO"])
assert.Equal(t, "qux", env["BAZ"])
}
@@ -39,7 +38,7 @@ func TestParseEnvFileMultiLine(t *testing.T) {
require.NoError(t, os.WriteFile(envPath, []byte(content), 0o600))
env := map[string]string{}
require.NoError(t, parseEnvFile(e, envPath, &env)(context.Background()))
require.NoError(t, parseEnvFile(e, envPath, &env)(t.Context()))
assert.Equal(t, "line1\nline2", env["FOO"])
}
@@ -50,7 +49,7 @@ func TestParseEnvFileLargeValueWithinLimit(t *testing.T) {
require.NoError(t, os.WriteFile(envPath, []byte(content), 0o600))
env := map[string]string{}
require.NoError(t, parseEnvFile(e, envPath, &env)(context.Background()))
require.NoError(t, parseEnvFile(e, envPath, &env)(t.Context()))
assert.Equal(t, big, env["FOO"])
}
@@ -61,7 +60,7 @@ func TestParseEnvFileLineExceedsBufferReportsScannerError(t *testing.T) {
require.NoError(t, os.WriteFile(envPath, []byte(content), 0o600))
env := map[string]string{}
err := parseEnvFile(e, envPath, &env)(context.Background())
err := parseEnvFile(e, envPath, &env)(t.Context())
require.ErrorIs(t, err, bufio.ErrTooLong)
assert.Contains(t, err.Error(), "reading env file")
}
@@ -73,7 +72,7 @@ func TestParseEnvFileBlankLines(t *testing.T) {
require.NoError(t, os.WriteFile(envPath, []byte("\nFOO=bar\n\n \nBAZ=qux\n\n"), 0o600))
env := map[string]string{}
require.NoError(t, parseEnvFile(e, envPath, &env)(context.Background()))
require.NoError(t, parseEnvFile(e, envPath, &env)(t.Context()))
assert.Equal(t, "bar", env["FOO"])
assert.Equal(t, "qux", env["BAZ"])
}
@@ -84,12 +83,12 @@ func TestParseEnvFileMultiLineKeepsBlankLines(t *testing.T) {
require.NoError(t, os.WriteFile(envPath, []byte("FOO<<EOF\nline1\n\nline2\nEOF\n"), 0o600))
env := map[string]string{}
require.NoError(t, parseEnvFile(e, envPath, &env)(context.Background()))
require.NoError(t, parseEnvFile(e, envPath, &env)(t.Context()))
assert.Equal(t, "line1\n\nline2", env["FOO"])
require.NoError(t, os.WriteFile(envPath, []byte("FOO<<EOF\n\nline2\nEOF\n"), 0o600))
env = map[string]string{}
require.NoError(t, parseEnvFile(e, envPath, &env)(context.Background()))
require.NoError(t, parseEnvFile(e, envPath, &env)(t.Context()))
assert.Equal(t, "\nline2", env["FOO"])
}
@@ -99,7 +98,7 @@ func TestParseEnvFileUTF8BOM(t *testing.T) {
require.NoError(t, os.WriteFile(envPath, content, 0o600))
env := map[string]string{}
require.NoError(t, parseEnvFile(e, envPath, &env)(context.Background()))
require.NoError(t, parseEnvFile(e, envPath, &env)(t.Context()))
assert.Equal(t, "bar", env["FOO"])
}
@@ -121,7 +120,7 @@ func TestParseEnvFileUTF16(t *testing.T) {
require.NoError(t, os.WriteFile(envPath, content, 0o600))
env := map[string]string{}
require.NoError(t, parseEnvFile(e, envPath, &env)(context.Background()))
require.NoError(t, parseEnvFile(e, envPath, &env)(t.Context()))
assert.Equal(t, "bar", env["FOO"])
assert.Equal(t, "line1", env["MULTI"])
})
@@ -133,7 +132,7 @@ func TestParseEnvFileMissingDelimiter(t *testing.T) {
require.NoError(t, os.WriteFile(envPath, []byte("FOO<<EOF\nline1\nline2\n"), 0o600))
env := map[string]string{}
err := parseEnvFile(e, envPath, &env)(context.Background())
err := parseEnvFile(e, envPath, &env)(t.Context())
require.Error(t, err)
assert.Contains(t, err.Error(), "delimiter")
}

55
act/container/tar.go Normal file
View File

@@ -0,0 +1,55 @@
// Copyright 2026 The Gitea Authors. All rights reserved.
// Copyright 2020 The nektos/act Authors. All rights reserved.
// SPDX-License-Identifier: MIT
package container
import (
"archive/tar"
"context"
"io"
"path/filepath"
"strings"
"gitea.com/gitea/runner/act/common"
"gitea.com/gitea/runner/act/filecollector"
)
func writeFilesTar(ctx context.Context, w io.Writer, uid, gid int, files ...*FileEntry) error {
tw := tar.NewWriter(w)
for _, file := range files {
common.Logger(ctx).Debugf("Writing entry to tarball %s len:%d", file.Name, len(file.Body))
if err := tw.WriteHeader(&tar.Header{
Name: file.Name,
Mode: file.Mode,
Size: int64(len(file.Body)),
Uid: uid,
Gid: gid,
}); err != nil {
return err
}
if _, err := tw.Write([]byte(file.Body)); err != nil {
return err
}
}
return tw.Close()
}
// writeDirTar prefixes entries with dstPath for extraction at root.
func writeDirTar(ctx context.Context, w io.Writer, dstPath, srcPath string, useGitIgnore, skipGitDir bool, uid, gid int) error {
logger := common.Logger(ctx)
tw := tar.NewWriter(w)
srcPrefix := strings.TrimSuffix(filepath.Dir(srcPath), string(filepath.Separator)) + string(filepath.Separator)
logger.Debugf("Stripping prefix:%s src:%s", srcPrefix, srcPath)
if err := filepath.Walk(srcPath, (&filecollector.FileCollector{
SrcPath: srcPath,
SrcPrefix: srcPrefix,
SkipGitDir: skipGitDir,
UseGitIgnore: useGitIgnore,
Handler: &filecollector.TarCollector{TarWriter: tw, UID: uid, GID: gid, DstDir: dstPath[1:]},
}).CollectFiles(ctx)); err != nil {
return err
}
return tw.Close()
}

View File

@@ -1,5 +0,0 @@
FROM scratch
ENV PATH="/this/path/does/not/exists/anywhere:/this/either"
ENV SOME_RANDOM_VAR=""
ENV ANOTHER_ONE="BUT_I_HAVE_VALUE"
ENV CONFLICT_VAR="I_EXIST_ONLY_HERE"

View File

@@ -16,10 +16,10 @@ import (
"path/filepath"
"strings"
git "github.com/go-git/go-git/v5"
"github.com/go-git/go-git/v5/plumbing/filemode"
"github.com/go-git/go-git/v5/plumbing/format/gitignore"
"github.com/go-git/go-git/v5/plumbing/format/index"
"gitea.com/gitea/runner/act/common"
"gitea.com/gitea/runner/act/common/git"
"golang.org/x/text/unicode/norm"
)
type Handler interface {
@@ -94,73 +94,62 @@ func (cc *CopyCollector) WriteFile(fpath string, fi fs.FileInfo, linkName string
}
type FileCollector struct {
Ignorer gitignore.Matcher
SrcPath string
SrcPrefix string
SkipGitDir bool
Handler Handler
SrcPath string
SrcPrefix string
SkipGitDir bool
UseGitIgnore bool
Handler Handler
}
func openGitIndex(path string) (*index.Index, error) {
repo, err := git.PlainOpen(path)
if err != nil {
return nil, err
// foldPath compares git's spelling of a path with the filesystem's, which may differ in case or Unicode normalization.
func foldPath(name string) string {
return strings.ToLower(norm.NFC.String(name))
}
func (fc *FileCollector) gitFiles(ctx context.Context) map[string]bool {
if !fc.UseGitIgnore {
return nil
}
return repo.Storer.Index()
files, err := git.ListFiles(ctx, fc.SrcPath)
if err != nil {
common.Logger(ctx).Debugf("Copying %s without applying .gitignore: %v", fc.SrcPath, err)
return nil
}
keep := map[string]bool{".": true}
for _, file := range files {
for file = foldPath(file); !keep[file]; file = path.Dir(file) {
keep[file] = true
}
}
return keep
}
func (fc *FileCollector) CollectFiles(ctx context.Context, submodulePath []string) filepath.WalkFunc {
i, _ := openGitIndex(path.Join(fc.SrcPath, path.Join(submodulePath...)))
func (fc *FileCollector) CollectFiles(ctx context.Context) filepath.WalkFunc {
keep := fc.gitFiles(ctx)
return func(file string, fi os.FileInfo, err error) error {
if err != nil {
return err
}
if ctx != nil && ctx.Err() != nil {
if ctx.Err() != nil {
return errors.New("copy cancelled")
}
sansPrefix := strings.TrimPrefix(file, fc.SrcPrefix)
split := strings.Split(sansPrefix, string(filepath.Separator))
// The root folders should be skipped, submodules only have the last path component set to "." by filepath.Walk
if fi.IsDir() && len(split) > 0 && split[len(split)-1] == "." {
return nil
}
if fc.SkipGitDir && split[len(split)-1] == ".git" {
if fc.SkipGitDir && fi.Name() == ".git" {
if fi.IsDir() {
return filepath.SkipDir
}
return nil
}
var entry *index.Entry
if i != nil {
entry, err = i.Entry(strings.Join(split[len(submodulePath):], "/"))
} else {
err = index.ErrEntryNotFound
}
if err != nil && fc.Ignorer != nil && fc.Ignorer.Match(split, fi.IsDir()) {
if fi.IsDir() {
if i != nil {
ms, err := i.Glob(strings.Join(append(split[len(submodulePath):], "**"), "/"))
if err != nil || len(ms) == 0 {
return filepath.SkipDir
}
} else {
if keep != nil {
rel, _ := filepath.Rel(fc.SrcPath, file)
if rel = filepath.ToSlash(rel); !keep[foldPath(rel)] && !strings.Contains("/"+rel+"/", "/.git/") {
if fi.IsDir() {
return filepath.SkipDir
}
} else {
return nil
}
}
if err == nil && entry.Mode == filemode.Submodule {
err = filepath.Walk(file, fc.CollectFiles(ctx, split))
if err != nil {
return err
}
return filepath.SkipDir
}
path := filepath.ToSlash(sansPrefix)
path := filepath.ToSlash(strings.TrimPrefix(file, fc.SrcPrefix))
// return on non-regular files (thanks to [kumo](https://medium.com/@komuw/just-like-you-did-fbdd7df829d3) for this suggested update)
if fi.Mode()&os.ModeSymlink == os.ModeSymlink {
linkName, err := os.Readlink(file)
if err != nil {
@@ -171,17 +160,12 @@ func (fc *FileCollector) CollectFiles(ctx context.Context, submodulePath []strin
return nil
}
// open file
f, err := os.Open(file)
if err != nil {
return err
}
defer f.Close()
if ctx != nil {
stop := context.AfterFunc(ctx, func() { _ = f.Close() })
defer stop()
}
defer context.AfterFunc(ctx, func() { _ = f.Close() })()
return fc.Handler.WriteFile(path, fi, "", f)
}

View File

@@ -9,106 +9,78 @@ import (
"bytes"
"context"
"io"
"maps"
"os"
"os/exec"
"path/filepath"
"runtime"
"strings"
"testing"
git "github.com/go-git/go-git/v5"
"github.com/go-git/go-git/v5/plumbing/format/gitignore"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
func TestIgnoredTrackedfile(t *testing.T) {
func gitCmd(t *testing.T, args ...string) {
t.Helper()
out, err := exec.Command("git", args...).CombinedOutput()
require.NoError(t, err, string(out))
}
func TestCollectFilesKeepsWhatGitTracksOrDoesNotIgnore(t *testing.T) {
repoDir := filepath.Join(t.TempDir(), "mygitrepo")
repo, err := git.PlainInit(repoDir, false)
require.NoError(t, err)
require.NoError(t, os.WriteFile(filepath.Join(repoDir, ".gitignore"), []byte(".*\n"), 0o644))
require.NoError(t, os.WriteFile(filepath.Join(repoDir, ".env"), []byte("test=val1\n"), 0o644))
worktree, err := repo.Worktree()
require.NoError(t, err)
_, err = worktree.Add(".gitignore")
require.NoError(t, err)
for _, dir := range []string{repoDir, filepath.Join(repoDir, "sub"), filepath.Join(repoDir, "nested")} {
gitCmd(t, "init", "-q", dir)
}
for name, body := range map[string]string{
".gitignore": ".*\nign/\n",
".env": "",
"Renamed": "",
"ign/tracked": "",
"ign/untracked": "",
"sub/.gitignore": "dropped\n",
"sub/dropped": "",
"sub/kept": "",
"nested/kept": "",
} {
require.NoError(t, os.MkdirAll(filepath.Dir(filepath.Join(repoDir, name)), 0o755))
require.NoError(t, os.WriteFile(filepath.Join(repoDir, name), []byte(body), 0o644))
}
require.NoError(t, os.Mkdir(filepath.Join(repoDir, "uninitialized"), 0o755))
gitCmd(t, "-C", repoDir, "add", "-f", ".gitignore", "Renamed", "ign/tracked")
require.NoError(t, os.Rename(filepath.Join(repoDir, "Renamed"), filepath.Join(repoDir, "renamed")))
for _, gitlink := range []string{"sub", "uninitialized"} {
gitCmd(t, "-C", repoDir, "update-index", "--add", "--cacheinfo", "160000,4b825dc642cb6eb9a060e54bf8d69288fbee4904,"+gitlink)
}
expected := map[string]string{".gitignore": "", "ign/tracked": "", "nested/kept": "", "renamed": "", "sub/.gitignore": "", "sub/kept": ""}
if runtime.GOOS != "windows" {
require.NoError(t, os.Symlink(".env", filepath.Join(repoDir, "test.env")))
expected["test.env"] = ".env"
}
var archive bytes.Buffer
tw := tar.NewWriter(&archive)
patterns, err := gitignore.ReadPatterns(worktree.Filesystem, nil)
require.NoError(t, err)
ignorer := gitignore.NewMatcher(patterns)
fc := &FileCollector{
Ignorer: ignorer,
SrcPath: repoDir,
SrcPrefix: repoDir + string(filepath.Separator),
Handler: &TarCollector{
TarWriter: tw,
},
}
err = filepath.Walk(repoDir, fc.CollectFiles(context.Background(), nil))
assert.NoError(t, err, "successfully collect files")
fc := &FileCollector{SrcPath: repoDir, SrcPrefix: repoDir + string(filepath.Separator), UseGitIgnore: true, Handler: &TarCollector{TarWriter: tw}}
require.NoError(t, filepath.Walk(repoDir, fc.CollectFiles(t.Context())))
require.NoError(t, tw.Close())
files := map[string]string{}
tr := tar.NewReader(&archive)
h, err := tr.Next()
assert.NoError(t, err, "tar must not be empty") //nolint:testifylint // pre-existing issue from nektos/act
assert.Equal(t, ".gitignore", h.Name)
_, err = tr.Next()
assert.ErrorIs(t, err, io.EOF, "tar must only contain one element")
for header, err := tr.Next(); err != io.EOF; header, err = tr.Next() {
require.NoError(t, err)
files[header.Name] = header.Linkname
}
assert.Contains(t, files, ".git/HEAD")
maps.DeleteFunc(files, func(name, _ string) bool { return strings.Contains("/"+name, "/.git/") })
assert.Equal(t, expected, files)
}
func TestSymlinks(t *testing.T) {
if runtime.GOOS == "windows" {
t.Skip("creating symlinks requires elevated privileges on Windows")
}
repoDir := filepath.Join(t.TempDir(), "mygitrepo")
repo, err := git.PlainInit(repoDir, false)
require.NoError(t, err)
require.NoError(t, os.WriteFile(filepath.Join(repoDir, ".env"), []byte("test=val1\n"), 0o644))
require.NoError(t, os.Symlink(".env", filepath.Join(repoDir, "test.env")))
worktree, err := repo.Worktree()
require.NoError(t, err)
_, err = worktree.Add("test.env")
require.NoError(t, err)
var archive bytes.Buffer
tw := tar.NewWriter(&archive)
fc := &FileCollector{
SrcPath: repoDir,
SrcPrefix: repoDir + string(filepath.Separator),
Handler: &TarCollector{
TarWriter: tw,
},
}
err = filepath.Walk(repoDir, fc.CollectFiles(context.Background(), nil))
assert.NoError(t, err, "successfully collect files")
require.NoError(t, tw.Close())
tr := tar.NewReader(&archive)
h, err := tr.Next()
files := map[string]tar.Header{}
for err == nil {
files[h.Name] = *h
h, err = tr.Next()
}
assert.Equal(t, ".env", files[".env"].Name)
assert.Equal(t, "test.env", files["test.env"].Name)
assert.Equal(t, ".env", files["test.env"].Linkname)
assert.ErrorIs(t, err, io.EOF, "tar must be read cleanly to EOF")
}
// Regression for https://gitea.com/gitea/runner/issues/876 and /941:
// re-copying an action directory must overwrite a pre-existing read-only
// file (e.g. a git pack .idx at mode 0444) instead of failing with EACCES
// on macOS or "Access is denied" on Windows.
func TestCopyCollectorWriteFileOverwritesReadOnlyFile(t *testing.T) {
dst := t.TempDir()
target := filepath.Join(dst, "sub", "pack.idx")
require.NoError(t, os.MkdirAll(filepath.Dir(target), 0o755))
require.NoError(t, os.WriteFile(target, []byte("old"), 0o444))
src := filepath.Join(t.TempDir(), "pack.idx")
require.NoError(t, os.WriteFile(src, []byte("new"), 0o444))
fi, err := os.Stat(src)
fi, err := os.Stat(target)
require.NoError(t, err)
cc := &CopyCollector{DstDir: dst}
@@ -119,8 +91,6 @@ func TestCopyCollectorWriteFileOverwritesReadOnlyFile(t *testing.T) {
assert.Equal(t, "new", string(got))
}
// Without the destination removal, os.Symlink fails with EEXIST when the
// path already holds a regular file from an earlier copy of the action.
func TestCopyCollectorWriteFileOverwritesFileWithSymlink(t *testing.T) {
if runtime.GOOS == "windows" {
t.Skip("creating symlinks requires elevated privileges on Windows")
@@ -129,11 +99,8 @@ func TestCopyCollectorWriteFileOverwritesFileWithSymlink(t *testing.T) {
target := filepath.Join(dst, "link")
require.NoError(t, os.WriteFile(target, []byte("stale"), 0o644))
fi, err := os.Lstat(target)
require.NoError(t, err)
cc := &CopyCollector{DstDir: dst}
require.NoError(t, cc.WriteFile("link", fi, "target", nil))
require.NoError(t, cc.WriteFile("link", nil, "target", nil))
resolved, err := os.Readlink(target)
require.NoError(t, err)
@@ -143,7 +110,7 @@ func TestCopyCollectorWriteFileOverwritesFileWithSymlink(t *testing.T) {
func TestFileCollectorCancellationAndWalkError(t *testing.T) {
ctx, cancel := context.WithCancel(t.Context())
cancel()
walk := (&FileCollector{}).CollectFiles(ctx, nil)
walk := (&FileCollector{}).CollectFiles(ctx)
err := walk("file", nil, nil)
require.EqualError(t, err, "copy cancelled")

View File

@@ -74,7 +74,7 @@ func SetRef(ctx context.Context, ghc *model.GithubContext, repoPath string) {
if ghc.Ref == "" {
ref, err := pinned(ctx, "ref:"+repoPath, func() (string, error) { return findGitRef(ctx, repoPath) })
if err != nil {
logger.Warningf("unable to get git ref: %v", err)
logger.Warnf("unable to get git ref: %v", err)
} else {
logger.Debugf("using github ref: %s", ref)
ghc.Ref = ref
@@ -121,7 +121,7 @@ func SetSha(ctx context.Context, ghc *model.GithubContext, repoPath string) {
return revision, err
})
if err != nil {
logger.Warningf("unable to get git revision: %v", err)
logger.Warnf("unable to get git revision: %v", err)
} else {
ghc.Sha = sha
}
@@ -130,11 +130,11 @@ func SetSha(ctx context.Context, ghc *model.GithubContext, repoPath string) {
// SetRepositoryAndOwner resolves the repository of the context from the git
// remote in repoPath when it is not set yet, and derives its owner.
func SetRepositoryAndOwner(ctx context.Context, ghc *model.GithubContext, githubInstance, repoPath string) {
func SetRepositoryAndOwner(ctx context.Context, ghc *model.GithubContext, repoPath string) {
if ghc.Repository == "" {
repo, err := findGithubRepo(ctx, repoPath, githubInstance)
repo, err := findGithubRepo(ctx, repoPath)
if err != nil {
common.Logger(ctx).Warningf("unable to get git repo (githubInstance: %v, repoPath: %v): %v", githubInstance, repoPath, err)
common.Logger(ctx).Warnf("unable to get git repo (repoPath: %v): %v", repoPath, err)
return
}
ghc.Repository = repo

View File

@@ -9,13 +9,14 @@ import (
"errors"
"testing"
"gitea.com/gitea/runner/act/common/logging"
"gitea.dev/actionslib/pkg/model"
log "github.com/sirupsen/logrus"
"github.com/stretchr/testify/assert"
)
func TestSetRef(t *testing.T) {
log.SetLevel(log.DebugLevel)
logging.SetLevel(logging.LevelDebug)
oldFindGitRef := findGitRef
oldFindGitRevision := findGitRevision
@@ -104,7 +105,7 @@ func TestSetRef(t *testing.T) {
Event: table.event,
}
SetRef(context.Background(), ghc, "/some/dir")
SetRef(t.Context(), ghc, "/some/dir")
ghc.SetRefTypeAndName()
assert.Equal(t, table.ref, ghc.Ref)
@@ -122,14 +123,14 @@ func TestSetRef(t *testing.T) {
Event: map[string]any{},
}
SetRef(context.Background(), ghc, "/some/dir")
SetRef(t.Context(), ghc, "/some/dir")
assert.Equal(t, "refs/heads/master", ghc.Ref)
})
}
func TestSetSha(t *testing.T) {
log.SetLevel(log.DebugLevel)
logging.SetLevel(logging.LevelDebug)
oldFindGitRef := findGitRef
oldFindGitRevision := findGitRevision
@@ -209,14 +210,14 @@ func TestSetSha(t *testing.T) {
Event: table.event,
}
SetSha(context.Background(), ghc, "/some/dir")
SetSha(t.Context(), ghc, "/some/dir")
assert.Equal(t, table.sha, ghc.Sha)
})
}
t.Run("pinned checkout keeps the first sha and ref", func(t *testing.T) {
ctx := WithPinnedCheckout(context.Background())
ctx := WithPinnedCheckout(t.Context())
for _, head := range []string{"first", "moved"} {
findGitRevision = func(context.Context, string) (string, string, error) { return "", head, nil }
findGitRef = func(context.Context, string) (string, error) { return "refs/heads/" + head, nil }

View File

@@ -6,6 +6,7 @@ package runner
import (
"context"
"crypto/rand"
"crypto/sha256"
"embed"
"encoding/hex"
@@ -90,7 +91,7 @@ func readActionImpl(ctx context.Context, step *model.Step, actionDir, actionPath
if b, err = trampoline.ReadFile("res/trampoline.js"); err != nil {
return nil, err
}
err2 := writeFile(filepath.Join(actionDir, actionPath, "trampoline.js"), b, 0o400)
err2 := writeFile(filepath.Join(actionDir, actionPath, "trampoline.js"), b, 0o644)
if err2 != nil {
return nil, err2
}
@@ -144,18 +145,18 @@ func maybeCopyToActionDir(ctx context.Context, step actionStep, actionDir, actio
containerActionDirCopy += `/`
}
defer git.AcquireCloneLock(actionDir)()
unlock, err := git.AcquireCloneLock(ctx, actionDir)
if err != nil {
return err
}
defer unlock()
if !rc.Config.NoActionPatch {
// A concurrent job's prepare resets this directory, so patch under the copy's lock.
// Other jobs share the directory, so patch under the copy's lock.
defer patchActions(ctx, actionScriptPaths(filepath.Join(actionDir, actionPath), step.getActionModel()))()
}
if err := removeGitIgnore(ctx, actionDir); err != nil {
return err
}
return rc.JobContainer.CopyDir(containerActionDirCopy, actionDir+"/", rc.Config.UseGitIgnore, true)(ctx)
return rc.JobContainer.CopyDir(containerActionDirCopy, actionDir+"/", false, true)(ctx) // an exported commit has no .git to apply gitignore with
}
func runActionImpl(step actionStep, actionDir string, remoteAction *remoteAction) common.Executor {
@@ -178,7 +179,7 @@ func runActionImpl(step actionStep, actionDir string, remoteAction *remoteAction
}
actionLocation := path.Join(actionDir, actionPath)
actionName, containerActionDir := getContainerActionPaths(stepModel, actionLocation, rc)
actionName, containerActionDir := getContainerActionPaths(step)
logger.Debugf("type=%v actionDir=%s actionPath=%s workdir=%s actionCacheDir=%s actionName=%s containerActionDir=%s", stepModel.Type(), actionDir, actionPath, rc.Config.Workdir, rc.ActionCacheDir(), actionName, containerActionDir)
@@ -240,23 +241,6 @@ func nodeActionCommand(script string) []string {
return []string{"node", "--preserve-symlinks-main", script}
}
// https://github.com/nektos/act/issues/228#issuecomment-629709055
// files in .gitignore are not copied in a Docker container
// this causes issues with actions that ignore other important resources
// such as `node_modules` for example
func removeGitIgnore(ctx context.Context, directory string) error {
gitIgnorePath := path.Join(directory, ".gitignore")
if _, err := os.Stat(gitIgnorePath); err == nil {
// .gitignore exists
common.Logger(ctx).Debugf("Removing %s before docker cp", gitIgnorePath)
err := os.Remove(gitIgnorePath)
if err != nil {
return err
}
}
return nil
}
// dockerActionImageTag derives the local docker image tag used when an action
// is built from a Dockerfile.
//
@@ -287,11 +271,30 @@ func dockerActionImageTag(repository, actionName string, localAction bool) strin
return strings.ToLower(image)
}
// One image per step and job: its stages share a build, and concurrent jobs never run each other's.
func (rc *RunContext) dockerActionImage(ctx context.Context, step actionStep, actionName string, localAction bool) string {
job := rc.topLevelRunContext()
if job.dockerActionImages == nil {
job.dockerActionImages = map[*model.Step]string{}
}
image, ok := job.dockerActionImages[step.getStepModel()]
if !ok {
image = dockerActionImageTag(step.getGithubContext(ctx).Repository, actionName, localAction) + "-" + strings.ToLower(rand.Text())
if !common.Dryrun(ctx) { // a dry run builds nothing to remove
job.dockerActionImages[step.getStepModel()] = image
}
}
return image
}
// TODO: break out parts of function to reduce complexicity
func execAsDocker(ctx context.Context, step actionStep, actionName, actionDir, basedir string, localAction bool, stage stepStage) error {
logger := common.Logger(ctx)
rc := step.getRunContext()
action := step.getActionModel()
if rc.kubernetes {
return errKubernetesDocker
}
var prepImage common.Executor
var image string
@@ -301,7 +304,7 @@ func execAsDocker(ctx context.Context, step actionStep, actionName, actionDir, b
// Apply forcePull only for prebuild docker images
forcePull = rc.Config.ForcePull
} else {
image = dockerActionImageTag(step.getGithubContext(ctx).Repository, actionName, localAction)
image = rc.dockerActionImage(ctx, step, actionName, localAction)
contextDir, fileName := filepath.Split(filepath.Join(basedir, action.Runs.Image))
anyArchExists, err := ContainerImageExistsLocally(ctx, image, "any")
@@ -346,7 +349,11 @@ func execAsDocker(ctx context.Context, step actionStep, actionName, actionDir, b
// Held across the whole build: the daemon drains contextDir lazily.
inner := prepImage
prepImage = func(ctx context.Context) error {
defer git.AcquireCloneLock(actionDir)()
unlock, err := git.AcquireCloneLock(ctx, actionDir)
if err != nil {
return err
}
defer unlock()
return inner(ctx)
}
}
@@ -442,12 +449,15 @@ func newStepContainer(ctx context.Context, step step, image string, cmd, entrypo
logWriter := rc.commandLogWriter(ctx)
envList := make([]string, 0)
for k, v := range *step.getEnv() {
if rc.IsHostEnv() && strings.EqualFold(k, rc.JobContainer.GetPathVariableName()) && v == rc.JobContainer.DefaultPathVariable() { // the runner's own PATH stays out, as on GitHub
continue
}
envList = append(envList, fmt.Sprintf("%s=%s", k, v))
}
envList = append(envList, rc.runnerEnv(ctx)...)
binds, mounts := rc.GetBindsAndMounts()
binds, mounts := rc.GetBindsAndMounts(ctx)
networkMode := "container:" + rc.jobContainerName()
if rc.IsHostEnv() {
networkMode = "default"
@@ -499,25 +509,14 @@ func populateEnvsFromInput(ctx context.Context, env *map[string]string, action *
return nil
}
func getContainerActionPaths(step *model.Step, actionDir string, rc *RunContext) (string, string) {
actionName := ""
containerActionDir := "."
if step.Type() != model.StepTypeUsesActionRemote {
actionName = getOsSafeRelativePath(actionDir, rc.Config.Workdir)
containerActionDir = rc.JobContainer.ToContainerPath(rc.Config.Workdir) + "/" + actionName
actionName = "./" + actionName
} else if step.Type() == model.StepTypeUsesActionRemote {
actionName = getOsSafeRelativePath(actionDir, rc.ActionCacheDir())
containerActionDir = rc.JobContainer.GetActPath() + "/actions/" + actionName
func getContainerActionPaths(step actionStep) (string, string) {
rc := step.getRunContext()
if sar, ok := step.(*stepActionRemote); ok { // a copy per action, as its repository's directory is shared
actionName := path.Join(sar.containerKey(), sar.remoteAction.Path)
return actionName, rc.JobContainer.GetActPath() + "/actions/" + actionName
}
if actionName == "" {
actionName = filepath.Base(actionDir)
if runtime.GOOS == "windows" {
actionName = strings.ReplaceAll(actionName, "\\", "/")
}
}
return actionName, containerActionDir
actionName := getOsSafeRelativePath(filepath.Join(rc.Config.Workdir, step.getStepModel().Uses), rc.Config.Workdir)
return "./" + actionName, rc.JobContainer.ToContainerPath(rc.Config.Workdir) + "/" + actionName
}
func getOsSafeRelativePath(s, prefix string) string {
@@ -563,13 +562,13 @@ func actionStagePaths(step actionStep) (actionDir, actionPath, actionName, conta
stepModel := step.getStepModel()
if sar, ok := step.(*stepActionRemote); ok {
actionDir = sar.actionDir()
actionDir = sar.actionDir
actionPath = sar.remoteAction.Path
} else {
actionDir = filepath.Join(rc.Config.Workdir, stepModel.Uses)
}
actionName, containerActionDir = getContainerActionPaths(stepModel, path.Join(actionDir, actionPath), rc)
actionName, containerActionDir = getContainerActionPaths(step)
return actionDir, actionPath, actionName, containerActionDir
}
@@ -658,17 +657,17 @@ func shouldRunPostStep(step actionStep) common.Conditional {
stepResult := stepResults[step.getStepModel().ID]
if stepResult == nil {
log.WithField("stepResult", model.StepStatusSkipped).Debugf("skipping post step for '%s'; step was not executed", step.getStepModel())
log.With("stepResult", model.StepStatusSkipped).Debugf("skipping post step for '%s'; step was not executed", step.getStepModel())
return false
}
if stepResult.Conclusion == model.StepStatusSkipped {
log.WithField("stepResult", model.StepStatusSkipped).Debugf("skipping post step for '%s'; main step was skipped", step.getStepModel())
log.With("stepResult", model.StepStatusSkipped).Debugf("skipping post step for '%s'; main step was skipped", step.getStepModel())
return false
}
if step.getActionModel() == nil {
log.WithField("stepResult", model.StepStatusSkipped).Debugf("skipping post step for '%s': no action model available", step.getStepModel())
log.With("stepResult", model.StepStatusSkipped).Debugf("skipping post step for '%s': no action model available", step.getStepModel())
return false
}

View File

@@ -29,7 +29,7 @@ func evaluateCompositeInputAndEnv(ctx context.Context, parent *RunContext, step
}
}
ee := parent.NewActionInputsExpressionEvaluator(ctx, step)
ee := parent.NewStepExpressionEvaluator(ctx, step)
for inputID, input := range step.getActionModel().Inputs {
envKey := regexp.MustCompile("[^A-Z0-9-]").ReplaceAllString(strings.ToUpper(inputID), "_")
@@ -104,6 +104,7 @@ func newCompositeRunContext(ctx context.Context, parent *RunContext, step action
Parent: parent,
EventJSON: parent.EventJSON,
platformImage: parent.platformImage,
kubernetes: parent.kubernetes,
jobIndex: parent.jobIndex,
jobTotal: parent.jobTotal,
}

View File

@@ -4,9 +4,9 @@
package runner
import (
"context"
"testing"
"gitea.com/gitea/runner/act/common"
"gitea.com/gitea/runner/act/common/git"
"gitea.dev/actionslib/pkg/model"
@@ -50,7 +50,9 @@ func TestCompositeActionParity(t *testing.T) {
})
t.Run("propagates pre failures", func(t *testing.T) {
setCloneExecutor(t, func(git.NewGitCloneExecutorInput) common.Executor { return common.NewErrorExecutor(assert.AnError) })
setFetch(t, func(_ctx context.Context, _input git.NewGitCloneExecutorInput) (string, string, error) {
return "", "", assert.AnError
})
rc := &RunContext{
Config: &Config{GitHubInstance: "github.com", ActionCacheDir: t.TempDir()},
Run: &model.Run{JobID: "job", Workflow: &model.Workflow{Jobs: map[string]*model.Job{"job": {}}}},

View File

@@ -128,11 +128,11 @@ runs:
writeFile := func(filename string, data []byte, perm fs.FileMode) error {
assert.Equal(t, "actionDir/actionPath/trampoline.js", filename)
assert.Equal(t, fs.FileMode(0o400), perm)
assert.Equal(t, fs.FileMode(0o644), perm)
return nil
}
action, err := readActionImpl(context.Background(), tt.step, "actionDir", "actionPath", readFile, writeFile)
action, err := readActionImpl(t.Context(), tt.step, "actionDir", "actionPath", readFile, writeFile)
assert.NoError(t, err) //nolint:testifylint // pre-existing issue from nektos/act
assert.Equal(t, tt.expected, action)
@@ -143,6 +143,7 @@ runs:
}
func TestActionRunner(t *testing.T) {
t.Chdir(t.TempDir())
table := []struct {
name string
step actionStep
@@ -177,7 +178,8 @@ func TestActionRunner(t *testing.T) {
Using: "node16",
},
},
env: map[string]string{},
env: map[string]string{},
remoteAction: mustNewRemoteAction(t, "org/repo/path@ref"),
},
expectedEnv: map[string]string{"INPUT_KEY": "default value"},
},
@@ -216,7 +218,8 @@ func TestActionRunner(t *testing.T) {
Using: "node16",
},
},
env: map[string]string{},
env: map[string]string{},
remoteAction: mustNewRemoteAction(t, "org/repo/path@ref"),
},
expectedEnv: map[string]string{"STATE_name": "state value"},
},
@@ -224,10 +227,11 @@ func TestActionRunner(t *testing.T) {
for _, tt := range table {
t.Run(tt.name, func(t *testing.T) {
ctx := context.Background()
ctx := t.Context()
cm := &containerMock{}
cm.On("CopyDir", "/var/run/act/actions/dir/", "dir/", false, true).Return(func(ctx context.Context) error { return nil })
containerActionDir := "/var/run/act/actions/" + model.UsesHash("/org/repo/path@ref") + "/"
cm.On("CopyDir", containerActionDir, "dir/", false, true).Return(func(ctx context.Context) error { return nil })
envMatcher := mock.MatchedBy(func(env map[string]string) bool {
for k, v := range tt.expectedEnv {
@@ -238,7 +242,7 @@ func TestActionRunner(t *testing.T) {
return true
})
cm.On("Exec", []string{"node", "--preserve-symlinks-main", "/var/run/act/actions/dir/path"}, envMatcher, "", "").Return(func(ctx context.Context) error { return nil })
cm.On("Exec", []string{"node", "--preserve-symlinks-main", containerActionDir + "path"}, envMatcher, "", "").Return(func(ctx context.Context) error { return nil })
tt.step.getRunContext().JobContainer = cm
@@ -289,7 +293,7 @@ func TestNewStepContainerDoesNotUseDockerSecrets(t *testing.T) {
ContainerNewContainer = origContainerNewContainer
}()
ctx := context.Background()
ctx := t.Context()
rc := &RunContext{
Name: "job",
Config: &Config{
@@ -326,7 +330,7 @@ func TestNewStepContainerDoesNotUseDockerSecrets(t *testing.T) {
}
func TestMaybeCopyToActionDirHoldsCloneLock(t *testing.T) {
ctx := context.Background()
ctx := t.Context()
actionDir := t.TempDir()
@@ -366,7 +370,10 @@ func TestMaybeCopyToActionDirHoldsCloneLock(t *testing.T) {
peerAcquired := make(chan struct{})
go func() {
unlock := git.AcquireCloneLock(actionDir)
unlock, err := git.AcquireCloneLock(t.Context(), actionDir)
if !assert.NoError(t, err) {
return
}
close(peerAcquired)
unlock()
}()
@@ -400,7 +407,9 @@ func TestMaybeCopyToActionDirHoldsCloneLock(t *testing.T) {
func TestExecAsDockerHoldsCloneLockForRemoteUncached(t *testing.T) {
actionDir := t.TempDir()
unlockOnce := sync.OnceFunc(git.AcquireCloneLock(actionDir))
unlock, err := git.AcquireCloneLock(t.Context(), actionDir)
require.NoError(t, err)
unlockOnce := sync.OnceFunc(unlock)
defer unlockOnce()
innerEntered := make(chan struct{})
@@ -441,7 +450,7 @@ func TestExecAsDockerHoldsCloneLockForRemoteUncached(t *testing.T) {
env: map[string]string{},
}
ctx, cancel := context.WithCancel(context.Background())
ctx, cancel := context.WithCancel(t.Context())
defer cancel()
done := make(chan error, 1)
@@ -513,6 +522,17 @@ func TestDockerActionImageTag(t *testing.T) {
dockerActionImageTag("owner/repo", "./", true),
dockerActionImageTag("owner/repo", "./sub", true),
)
workflow := &model.Workflow{Jobs: map[string]*model.Job{"job1": {}}}
job := newTestRC(workflow, nil)
step := &stepActionRemote{Step: &model.Step{}, RunContext: job}
image := job.dockerActionImage(t.Context(), step, "abc123", false)
assert.True(t, strings.HasPrefix(image, "act-abc123-dockeraction:latest-"))
assert.Equal(t, image, (&RunContext{Parent: job}).dockerActionImage(t.Context(), step, "abc123", false))
assert.NotEqual(t, image, job.dockerActionImage(t.Context(), &stepActionRemote{Step: &model.Step{}, RunContext: job}, "abc123", false))
assert.NotEqual(t, image, newTestRC(workflow, nil).dockerActionImage(t.Context(), step, "abc123", false))
job.dockerActionImage(common.WithDryrun(t.Context(), true), &stepActionRemote{Step: &model.Step{}, RunContext: job}, "abc123", false)
assert.Len(t, job.dockerActionImages, 2)
}
func TestExecAsDockerStageEntrypoint(t *testing.T) {
@@ -602,7 +622,7 @@ func TestExecAsDockerStageEntrypoint(t *testing.T) {
cm.On("Start", true).Return(func(context.Context) error { return nil })
cm.On("Close").Return(func(context.Context) error { return nil })
require.NoError(t, execAsDocker(context.Background(), step, "action", t.TempDir(), t.TempDir(), false, tc.stage))
require.NoError(t, execAsDocker(t.Context(), step, "action", t.TempDir(), t.TempDir(), false, tc.stage))
require.NotNil(t, input)
assert.Equal(t, tc.wantCmd, input.Cmd)
assert.Equal(t, tc.wantEntrypoint, input.Entrypoint)
@@ -617,7 +637,7 @@ func TestDockerActionHasPreAndPostStep(t *testing.T) {
newStep := func(runs model.ActionRuns) actionStep {
return &stepActionRemote{action: &model.Action{Runs: runs}}
}
ctx := context.Background()
ctx := t.Context()
assert.False(t, hasPreStep(newStep(model.ActionRuns{Using: "docker", Image: "Dockerfile"}))(ctx))
assert.False(t, hasPostStep(newStep(model.ActionRuns{Using: "docker", Image: "Dockerfile"}))(ctx))

View File

@@ -62,7 +62,7 @@ func TestCancelledJobStatusEnablesAlwaysAndCancelledSteps(t *testing.T) {
enabled, err := interp.Evaluate("always()", exprparser.DefaultStatusCheckSuccess)
require.NoError(t, err)
assert.Equal(t, true, enabled, "`if: always()` step must run on a cancelled job")
setJobResult(context.Background(), rc, rc, true)
setJobResult(t.Context(), rc, rc, true)
assert.Equal(t, "cancelled", rc.Run.Job().Result)
}
@@ -76,7 +76,7 @@ func TestMainStepsExecutorRunsAlwaysStepsAfterCancel(t *testing.T) {
"job1": createJob(t, `runs-on: ubuntu-latest`, ""),
})
ctx, cancel := context.WithCancel(context.Background())
ctx, cancel := context.WithCancel(t.Context())
defer cancel()
var ran []string
@@ -110,7 +110,7 @@ func TestMainStepsExecutorMarksFailedOnTimeoutBetweenSteps(t *testing.T) {
"job1": createJob(t, `runs-on: ubuntu-latest`, ""),
})
ctx := newControllableDeadlineContext(context.Background())
ctx := newControllableDeadlineContext(t.Context())
var ran []string
var laterStepCtxErr error
@@ -149,7 +149,7 @@ func TestStepsExecutorRunsMainStepsAfterPreCancel(t *testing.T) {
"job1": createJob(t, `runs-on: ubuntu-latest`, ""),
})
ctx, cancel := context.WithCancel(context.Background())
ctx, cancel := context.WithCancel(t.Context())
defer cancel()
var ran []string
@@ -200,7 +200,7 @@ func TestStepsExecutorRunsMainStepsAfterPreFailure(t *testing.T) {
},
}
err := newStepsExecutor(rc, preSteps, steps)(context.Background())
err := newStepsExecutor(rc, preSteps, steps)(t.Context())
require.ErrorIs(t, err, assert.AnError, "the pre-step error is propagated")
assert.Equal(t, []string{"pre1", "always-step"}, ran, "the main always() step runs after a pre-step failure")
@@ -215,7 +215,7 @@ func TestPreStepFailureAffectsMainStepIfStatus(t *testing.T) {
rc := createIfTestRunContext(map[string]*model.Job{
"job1": createJob(t, `runs-on: ubuntu-latest`, ""),
})
ctx := common.WithJobErrorContainer(context.Background())
ctx := common.WithJobErrorContainer(t.Context())
reportStepError(ctx, rc, assert.AnError)
@@ -251,7 +251,7 @@ func TestPreStepFailureAffectsMainStepIfStatus(t *testing.T) {
// from cancellation (so the steps run) yet still carry a usable error container.
func TestPostStepsContextCancelledIsUsableForFailingStep(t *testing.T) {
jobSpan := trace.NewSpanContext(trace.SpanContextConfig{TraceID: trace.TraceID{1}, SpanID: trace.SpanID{1}})
cancelled, cancel := context.WithCancel(trace.ContextWithSpanContext(common.WithJobErrorContainer(context.Background()), jobSpan))
cancelled, cancel := context.WithCancel(trace.ContextWithSpanContext(common.WithJobErrorContainer(t.Context()), jobSpan))
cancel()
require.ErrorIs(t, cancelled.Err(), context.Canceled)
@@ -273,7 +273,7 @@ func TestPostStepsContextCancelledIsUsableForFailingStep(t *testing.T) {
// job-error container (via context.WithoutCancel), so the timeout failure and any post-step error
// survive into the post phase and the job is still reported as failed.
func TestPostStepsContextDeadlinePreservesJobError(t *testing.T) {
base := common.WithJobErrorContainer(context.Background())
base := common.WithJobErrorContainer(t.Context())
common.SetJobError(base, assert.AnError)
expired, cancel := context.WithDeadline(base, time.Now().Add(-time.Hour))
defer cancel()
@@ -292,14 +292,14 @@ func TestReportStepErrorTreatsCancelAsInterruption(t *testing.T) {
rc := &RunContext{}
// stray read cancellation while the job context is live: ignored, not a failure
live := common.WithJobErrorContainer(context.Background())
live := common.WithJobErrorContainer(t.Context())
reportStepError(live, rc, context.Canceled)
require.NoError(t, common.JobError(live))
assert.False(t, rc.jobFailed)
assert.False(t, rc.jobCancelled)
// genuine job cancellation: recorded as cancelled, still not a failure
cancelled, cancel := context.WithCancel(common.WithJobErrorContainer(context.Background()))
cancelled, cancel := context.WithCancel(common.WithJobErrorContainer(t.Context()))
cancel()
reportStepError(cancelled, rc, context.Canceled)
require.NoError(t, common.JobError(cancelled))
@@ -307,7 +307,7 @@ func TestReportStepErrorTreatsCancelAsInterruption(t *testing.T) {
assert.True(t, rc.jobCancelled)
// a real error still fails the job
failed := common.WithJobErrorContainer(context.Background())
failed := common.WithJobErrorContainer(t.Context())
reportStepError(failed, rc, assert.AnError)
require.ErrorIs(t, common.JobError(failed), assert.AnError)
assert.True(t, rc.jobFailed)

View File

@@ -117,7 +117,7 @@ func (rc *RunContext) refuseUnsecureCommand(ctx context.Context, command string)
}
// The step executor logs the failure itself, so keep this line's wording distinct.
common.Logger(ctx).WithField(rawOutputField, true).Errorf("##[error]%s", EscapeCommandData(fmt.Sprintf(
common.Logger(ctx).With(rawOutputField, true).Errorf("##[error]%s", EscapeCommandData(fmt.Sprintf(
"The `%s` command is disabled: it can set the environment of every later step from untrusted output. "+
"Write to $GITHUB_ENV or $GITHUB_PATH instead, or set ACTIONS_ALLOW_UNSECURE_COMMANDS to allow it",
command)))
@@ -159,7 +159,7 @@ func (rc *RunContext) setEnv(ctx context.Context, kvPairs map[string]string, arg
if fromCommand {
message = "Can't update NODE_OPTIONS environment variable using ::set-env:: command."
}
common.Logger(ctx).WithField(rawOutputField, true).Errorf("##[error]%s", EscapeCommandData(message))
common.Logger(ctx).With(rawOutputField, true).Errorf("##[error]%s", EscapeCommandData(message))
return
}
common.Logger(ctx).Infof("::set-env:: %s=%s", name, arg)

View File

@@ -6,15 +6,14 @@ package runner
import (
"bytes"
"context"
"io"
"os"
"testing"
"gitea.com/gitea/runner/act/common"
"gitea.com/gitea/runner/act/common/logging/logtest"
"gitea.dev/actionslib/pkg/model"
"github.com/sirupsen/logrus/hooks/test"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
@@ -26,8 +25,8 @@ func unsecureRC() *RunContext {
func TestSetEnv(t *testing.T) {
a := assert.New(t)
logger, hook := test.NewNullLogger()
ctx := common.WithLogger(context.Background(), logger)
logger, hook := logtest.NewLogger()
ctx := common.WithLogger(t.Context(), logger)
rc := unsecureRC()
handler := rc.commandHandler(ctx)
@@ -37,7 +36,7 @@ func TestSetEnv(t *testing.T) {
rc.setEnvFile(ctx, map[string]string{"name": "node_options"}, "--require env.js")
a.NotContains(rc.Env, "NODE_OPTIONS")
a.NotContains(rc.Env, "node_options")
entries := hook.AllEntries()
entries := hook.AllRecords()
require.Len(t, entries, 3)
a.Equal("##[error]Can't update NODE_OPTIONS environment variable using ::set-env:: command.", entries[1].Message)
a.Equal("##[error]Can't store NODE_OPTIONS output parameter using '$GITHUB_ENV' command.", entries[2].Message)
@@ -45,7 +44,7 @@ func TestSetEnv(t *testing.T) {
func TestStopCommandsKeepsSuppressedLinesInLog(t *testing.T) {
a := assert.New(t)
ctx := context.Background()
ctx := t.Context()
rc := unsecureRC()
handler := rc.commandHandler(ctx)
@@ -68,7 +67,7 @@ func TestStopCommandsKeepsSuppressedLinesInLog(t *testing.T) {
func TestSetOutput(t *testing.T) {
a := assert.New(t)
ctx := context.Background()
ctx := t.Context()
rc := new(RunContext)
rc.StepResults = make(map[string]*model.StepResult)
handler := rc.commandHandler(ctx)
@@ -108,7 +107,7 @@ func TestSetOutput(t *testing.T) {
func TestAddpath(t *testing.T) {
a := assert.New(t)
ctx := context.Background()
ctx := t.Context()
rc := unsecureRC()
handler := rc.commandHandler(ctx)
@@ -120,10 +119,10 @@ func TestAddpath(t *testing.T) {
}
func TestStopCommands(t *testing.T) {
logger, hook := test.NewNullLogger()
logger, hook := logtest.NewLogger()
a := assert.New(t)
ctx := common.WithLogger(context.Background(), logger)
ctx := common.WithLogger(t.Context(), logger)
rc := unsecureRC()
handler := rc.commandHandler(ctx)
@@ -137,7 +136,7 @@ func TestStopCommands(t *testing.T) {
a.Equal("abcd", rc.Env["x"])
messages := make([]string, 0)
for _, entry := range hook.AllEntries() {
for _, entry := range hook.AllRecords() {
messages = append(messages, entry.Message)
}
@@ -149,7 +148,7 @@ func TestStopCommands(t *testing.T) {
func TestStopCommandsResumesOnCommandNamedToken(t *testing.T) {
a := assert.New(t)
rc := unsecureRC()
handler := rc.commandHandler(context.Background())
handler := rc.commandHandler(t.Context())
handler("::stop-commands::add-mask\n")
handler("::set-env name=x::suppressed\n")
@@ -162,7 +161,7 @@ func TestStopCommandsResumesOnCommandNamedToken(t *testing.T) {
func TestAddpathADO(t *testing.T) {
a := assert.New(t)
ctx := context.Background()
ctx := t.Context()
rc := unsecureRC()
handler := rc.commandHandler(ctx)
@@ -174,17 +173,17 @@ func TestAddpathADO(t *testing.T) {
}
func TestAddmask(t *testing.T) {
logger, hook := test.NewNullLogger()
logger, hook := logtest.NewLogger()
a := assert.New(t)
ctx := context.Background()
ctx := t.Context()
loggerCtx := common.WithLogger(ctx, logger)
rc := new(RunContext)
handler := rc.commandHandler(loggerCtx)
handler("::ADD-MASK::my::secret")
a.Equal("***", hook.LastEntry().Message)
a.Equal("***", hook.LastRecord().Message)
a.Equal([]string{"my::secret"}, rc.Masks)
}
@@ -231,7 +230,7 @@ func TestAddmaskUsemask(t *testing.T) {
}
re := captureOutput(t, func() {
ctx := context.Background()
ctx := t.Context()
ctx = WithJobLogger(ctx, "0", "testjob", config, &rc.Masks, map[string]any{})
handler := rc.commandHandler(ctx)
@@ -248,7 +247,7 @@ func TestSaveState(t *testing.T) {
StepResults: map[string]*model.StepResult{},
}
ctx := context.Background()
ctx := t.Context()
handler := rc.commandHandler(ctx)
handler("::save-state name=state-name::state-value\n")
@@ -281,7 +280,7 @@ func TestUnsecureCommands(t *testing.T) {
a := assert.New(t)
rc := &RunContext{Env: tt.jobEnv}
rc.setCurrentStepEnv(tt.stepEnv)
handler := rc.commandHandler(context.Background())
handler := rc.commandHandler(t.Context())
handler("::set-env name=x::valz\n")
handler("::add-path::/opt/bin\n")

View File

@@ -15,12 +15,12 @@ import (
"time"
"gitea.com/gitea/runner/act/common"
"gitea.com/gitea/runner/act/common/logging/logtest"
"gitea.com/gitea/runner/act/container"
"gitea.dev/actionslib/pkg/model"
cerrdefs "github.com/containerd/errdefs"
"github.com/moby/moby/client"
"github.com/sirupsen/logrus/hooks/test"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
@@ -83,10 +83,10 @@ func TestDockerProxyMountedJob(t *testing.T) {
}
})
hook := &test.Hook{}
require.NoError(t, runner.NewPlanExecutor(plan)(common.WithLoggerHook(ctx, hook)))
hook := logtest.New()
require.NoError(t, runner.NewPlanExecutor(plan)(common.WithLoggerHandler(ctx, hook)))
var messages []string
for _, entry := range hook.AllEntries() {
for _, entry := range hook.AllRecords() {
messages = append(messages, strings.TrimSpace(entry.Message))
}
require.Contains(t, messages, "docker proxy post verified")

View File

@@ -246,10 +246,19 @@ func inputsFromEnv(env map[string]string) map[string]any {
}
func getEvaluatorInputs(rc *RunContext, stepInputs map[string]any, ghc *model.GithubContext) map[string]any {
if stepInputs != nil { // an action's own inputs, which workflow inputs must not shadow
return maps.Clone(stepInputs)
}
inputs := map[string]any{}
// only a called workflow's job itself takes the inputs Gitea resolved,
// composite actions copy its Config but have their own `inputs`
if rc.Config.WorkflowCallInputs != nil && rc.Parent == nil {
maps.Copy(inputs, rc.Config.WorkflowCallInputs)
return inputs
}
maps.Copy(inputs, rc.workflowCallInputs)
maps.Copy(inputs, stepInputs)
switch ghc.EventName {
case "workflow_dispatch":

View File

@@ -5,7 +5,6 @@
package runner
import (
"context"
"strings"
"testing"
@@ -82,7 +81,7 @@ func createRunContext(t *testing.T) *RunContext {
func TestEvaluateRunContext(t *testing.T) {
rc := createRunContext(t)
ee := rc.NewExpressionEvaluator(context.Background())
ee := rc.NewExpressionEvaluator(t.Context())
tables := []struct {
in string
@@ -142,7 +141,7 @@ func TestEvaluateRunContext(t *testing.T) {
for _, table := range tables {
t.Run(table.in, func(t *testing.T) {
assertObject := assert.New(t)
out, err := ee.evaluate(context.Background(), table.in, exprparser.DefaultStatusCheckNone)
out, err := ee.evaluate(t.Context(), table.in, exprparser.DefaultStatusCheckNone)
if table.errMesg == "" {
assertObject.NoError(err, table.in) //nolint:testifylint // pre-existing issue from nektos/act
assertObject.Equal(table.out, out, table.in)
@@ -162,7 +161,7 @@ func TestEvaluateStep(t *testing.T) {
env: map[string]string{"INPUT_FORGED": "leaked"},
}
ee := rc.NewStepExpressionEvaluator(context.Background(), step)
ee := rc.NewStepExpressionEvaluator(t.Context(), step)
tables := []struct {
in string
@@ -184,7 +183,7 @@ func TestEvaluateStep(t *testing.T) {
for _, table := range tables {
t.Run(table.in, func(t *testing.T) {
assertObject := assert.New(t)
out, err := ee.evaluate(context.Background(), table.in, exprparser.DefaultStatusCheckNone)
out, err := ee.evaluate(t.Context(), table.in, exprparser.DefaultStatusCheckNone)
if table.errMesg == "" {
assertObject.NoError(err, table.in) //nolint:testifylint // pre-existing issue from nektos/act
assertObject.Equal(table.out, out, table.in)
@@ -224,7 +223,7 @@ func TestInterpolate(t *testing.T) {
},
},
}
ee := rc.NewExpressionEvaluator(context.Background())
ee := rc.NewExpressionEvaluator(t.Context())
tables := []struct {
in string
out string
@@ -270,14 +269,14 @@ func TestInterpolate(t *testing.T) {
for _, table := range tables {
t.Run("interpolate", func(t *testing.T) {
out, err := ee.Interpolate(context.Background(), table.in)
out, err := ee.Interpolate(t.Context(), table.in)
require.NoError(t, err, table.in)
assert.Equal(t, table.out, out, table.in)
})
}
for _, in := range []string{"${{ 1) && (2 }}", "run ${{ 1) && (2 }} now", "${{ 1"} {
_, err := ee.Interpolate(context.Background(), in)
_, err := ee.Interpolate(t.Context(), in)
assert.Error(t, err, in)
}
}
@@ -392,6 +391,53 @@ jobs:
})
}
func TestGetEvaluatorInputsResolvedByGitea(t *testing.T) {
wf, err := model.ReadWorkflow(strings.NewReader(`
on:
workflow_call:
inputs:
name: {type: string, default: callee}
workflow_dispatch:
inputs:
name: {type: string}
`))
require.NoError(t, err)
config := &Config{Workdir: ".", WorkflowCallInputs: map[string]any{"name": "caller", "target": "dispatch"}}
ghc := &model.GithubContext{EventName: "workflow_dispatch", Event: map[string]any{"inputs": map[string]any{"name": "dispatch"}}}
// the event's own inputs must not override those Gitea resolved
job := &RunContext{Config: config, Run: &model.Run{JobID: "job1", Workflow: wf}}
assert.Equal(t, map[string]any{"name": "caller", "target": "dispatch"}, getEvaluatorInputs(job, nil, ghc))
composite := &RunContext{Config: config, Run: &model.Run{JobID: "job1", Workflow: &model.Workflow{}}, Parent: job}
assert.Equal(t, map[string]any{"arg": "x"}, getEvaluatorInputs(composite, map[string]any{"arg": "x"}, ghc))
}
func TestActionInputsShadowDispatchInputs(t *testing.T) {
wf, err := model.ReadWorkflow(strings.NewReader(`
on:
workflow_dispatch:
inputs:
message: {type: string}
jobs:
job1: {}
`))
require.NoError(t, err)
rc := newTestRC(wf, nil)
rc.Config.EventName = "workflow_dispatch"
rc.EventJSON = `{"inputs":{"message":"workflow"}}`
step := &stepActionRemote{RunContext: rc, env: map[string]string{"INPUT_MESSAGE": "step"}}
assert.Equal(t, "step", rc.NewActionInputsExpressionEvaluator(t.Context(), step).InterpolateName(t.Context(), "${{ inputs.message }}"))
assert.Equal(t, "workflow", rc.NewStepExpressionEvaluator(t.Context(), step).InterpolateName(t.Context(), "${{ inputs.message }}"))
enabled, err := isStepEnabled(t.Context(), "inputs.message == 'workflow'", step, stepStageMain)
require.NoError(t, err)
assert.True(t, enabled)
step.Step, step.action = &model.Step{}, &model.Action{Inputs: map[string]model.Input{"fallback": {Default: "${{ inputs.message }}"}}}
env, err := evaluateCompositeInputAndEnv(t.Context(), rc, step)
require.NoError(t, err)
assert.Equal(t, "workflow", env["INPUT_FALLBACK"])
}
func TestResolveWorkflowCallToleratesMismatchesAndPassesInheritedSecretsVerbatim(t *testing.T) {
callerWorkflow, err := model.ReadWorkflow(strings.NewReader(`
jobs:

View File

@@ -4,7 +4,7 @@
package runner
import (
"context"
"os"
"os/exec"
"runtime"
"testing"
@@ -29,7 +29,7 @@ func requireLinuxDocker(t *testing.T) {
// so the daemon has to be pinged explicitly here to decide whether to skip.
func requireDocker(t *testing.T) {
t.Helper()
ctx := context.Background()
ctx := t.Context()
cli, err := container.GetDockerClient(ctx)
if err != nil {
t.Skipf("skipping: docker client unavailable: %v", err)
@@ -40,6 +40,16 @@ func requireDocker(t *testing.T) {
}
}
func requireDockerSharingFiles(t *testing.T) {
t.Helper()
requireDocker(t)
for _, marker := range []string{"/.dockerenv", "/run/.containerenv"} {
if _, err := os.Stat(marker); err == nil {
t.Skip("skipping: inside a container, the docker daemon cannot bind-mount this process's files")
}
}
}
// requireHostTools skips the test unless every named executable is on PATH. Used by the
// self-hosted (host environment) suite, which runs steps directly on the host.
func requireHostTools(t *testing.T, tools ...string) {

View File

@@ -13,6 +13,7 @@ import (
"errors"
"fmt"
"io"
"maps"
"net/http"
"path"
"slices"
@@ -28,7 +29,10 @@ import (
"gitea.dev/actionslib/pkg/model"
)
const maxJobSummaryBytes = 1024 * 1024
const (
jobCleanupTimeout = time.Minute
maxJobSummaryBytes = 1024 * 1024
)
// jobSummaryTruncationMarker is appended to a summary that exceeded the size limit
// so the rendered output makes the truncation visible instead of silently cutting off.
@@ -76,7 +80,7 @@ func reportStepError(ctx context.Context, rc *RunContext, err error) {
// executor can fetch all of them up front.
type actionPreparer interface {
prepareActionExecutor() common.Executor
actionDownloadInfo() (reference, sha string, ok bool)
actionDownloadInfo() (reference, sha string)
}
// printPrepareActions downloads every action the job uses before its first step runs and reports
@@ -88,29 +92,30 @@ func printPrepareActions(rc *RunContext, preparers []actionPreparer) common.Exec
return nil
}
rawLogger := common.Logger(ctx).WithField(rawOutputField, true)
rawLogger := common.Logger(ctx).With(rawOutputField, true)
rawLogger.Infof("Prepare all required actions")
for _, preparer := range preparers {
if err := preparer.prepareActionExecutor()(ctx); err != nil {
// No step has run yet, so the failure belongs to the job.
reportStepError(ctx, rc, err)
return err
return setupError{err}
}
reference, sha, ok := preparer.actionDownloadInfo()
if !ok {
reference, sha := preparer.actionDownloadInfo()
if sha == "" {
continue
}
if sha == "" {
rawLogger.Infof("Download action repository '%s'", reference)
} else {
rawLogger.Infof("Download action repository '%s' (SHA:%s)", reference, sha)
}
rawLogger.Infof("Download action repository '%s' (SHA:%s)", reference, sha)
}
return nil
}
}
// setupError is a failure of the job setup, such as an action that cannot be downloaded.
type setupError struct{ error }
func (e setupError) Unwrap() error { return e.error }
// printCompleteJobName closes the setup section the way actions/runner ends its "Set up job" step.
func printCompleteJobName(rc *RunContext) common.Executor {
return func(ctx context.Context) error {
@@ -122,7 +127,7 @@ func printCompleteJobName(rc *RunContext) common.Executor {
if name == "" && rc.Run != nil {
name = rc.Run.JobID
}
common.Logger(ctx).WithField(rawOutputField, true).Infof("Complete job name: %s", name)
common.Logger(ctx).With(rawOutputField, true).Infof("Complete job name: %s", name)
return nil
}
}
@@ -135,20 +140,18 @@ func newJobExecutor(info jobInfo, sf stepFactory, rc *RunContext) common.Executo
preparers := make([]actionPreparer, 0)
var postExecutor common.Executor
steps = append(steps, func(ctx context.Context) error {
logger := common.Logger(ctx)
if len(info.matrix()) > 0 {
logger.Infof("Matrix: %v", info.matrix())
}
return nil
})
infoSteps := info.steps()
if len(infoSteps) == 0 {
return common.NewDebugExecutor("No steps found")
}
preSteps = append(preSteps, func(ctx context.Context) error {
if len(info.matrix()) > 0 {
common.Logger(ctx).Infof("Matrix: %v", info.matrix())
}
return nil
})
preSteps = append(preSteps, func(ctx context.Context) error {
// Have to be skipped for some Tests
if rc.Run == nil {
@@ -227,14 +230,15 @@ func newJobExecutor(info jobInfo, sf stepFactory, rc *RunContext) common.Executo
// The setup section of the job log. The started hook goes first, so what it sets up is
// in place for the first action download and the first step.
// The pre steps come after it, as steps of their own.
preSteps = append(preSteps, rc.runJobStartedHook)
preSteps = append(preSteps, printPrepareActions(rc, preparers))
preSteps = append(preSteps, stepPreSteps...)
preSteps = append(preSteps, printCompleteJobName(rc))
preSteps = append(preSteps, stepPreSteps...)
// Ahead of the teardown below, while the job environment is still up.
postExecutor = postExecutor.Finally(rc.runJobCompletedHook)
postExecutor = postExecutor.Finally(func(ctx context.Context) error {
postExecutor = postExecutor.Finally(useJobStageLogger(StageComplete, rc.runJobCompletedHook))
postExecutor = postExecutor.Finally(useJobStageLogger(StageComplete, func(ctx context.Context) error {
// swallowed: a bad output fails this job, it must not abandon the rest of the plan
if err := info.interpolateOutputs()(ctx); err != nil {
reportStepError(ctx, rc, err)
@@ -242,13 +246,13 @@ func newJobExecutor(info jobInfo, sf stepFactory, rc *RunContext) common.Executo
reportStepError(ctx, rc, err)
}
return nil
})
}))
postExecutor = postExecutor.Finally(func(ctx context.Context) error {
postExecutor = postExecutor.Finally(useJobStageLogger(StageComplete, func(ctx context.Context) error {
jobError := common.JobError(ctx)
var err error
// always allow 1 min for stopping and removing the runner, even if we were cancelled
ctx, cancel := context.WithTimeout(common.WithLogger(context.Background(), common.Logger(ctx)), time.Minute)
ctx, cancel := context.WithTimeout(common.WithLogger(context.Background(), common.Logger(ctx)), jobCleanupTimeout)
defer cancel()
logger := common.Logger(ctx)
@@ -258,9 +262,10 @@ func newJobExecutor(info jobInfo, sf stepFactory, rc *RunContext) common.Executo
logger.Errorf("##[error]%s", EscapeCommandData("Error while stop job container: "+err.Error()))
}
setJobResult(ctx, info, rc, jobError == nil)
rc.removeDockerActionImages(ctx)
return err
})
}))
return common.Executor(func(ctx context.Context) error {
if err := info.startContainer()(ctx); err != nil {
@@ -314,6 +319,9 @@ func newStepsExecutor(rc *RunContext, preSteps, steps []common.Executor) common.
mainExecutor := newMainStepsExecutor(rc, steps)
return func(ctx context.Context) error {
preErr := preExecutor(ctx)
if errors.As(preErr, new(setupError)) {
return preErr // as on GitHub, no step runs after the job setup failed
}
mainErr := mainExecutor(ctx)
if preErr != nil {
return preErr
@@ -407,39 +415,47 @@ func setJobResult(ctx context.Context, info jobInfo, rc *RunContext, success boo
jobResultMessage = "cancelled"
}
logger.WithField("jobResult", jobResult).Infof("Job %s", jobResultMessage)
logger.With("jobResult", jobResult).Infof("Job %s", jobResultMessage)
}
// evaluateJobEnv returns the workflow and job env evaluated, as far as it got on error, leaving rc.Env raw.
func evaluateJobEnv(ctx context.Context, rc *RunContext) (map[string]string, error) {
env := rc.GetEnv()
evaluated := maps.Clone(env)
evaluator := rc.NewExpressionEvaluatorWithEnv(ctx, evaluated)
var workflowEnv map[string]string
if rc.Run.Workflow.Env == nil {
if err := model.DecodeEvaluated("workflow env", rc.Run.Workflow.RawEnv, evaluator.shared(ctx).EvaluateYamlNode, &workflowEnv); err != nil {
return evaluated, err
}
}
if workflowEnv != nil {
evaluated = mergeMaps(workflowEnv, env)
evaluator = rc.NewExpressionEvaluatorWithEnv(ctx, evaluated)
}
var err error
for k, v := range env {
if evaluated[k], err = evaluator.Interpolate(ctx, v); err != nil {
return evaluated, fmt.Errorf("unable to interpolate env %s: %w", k, err)
}
}
return evaluated, nil
}
// evaluateJobEnvAndDefaults resolves the job's env, defaults.run and container env once, as GitHub does at job setup.
func evaluateJobEnvAndDefaults(ctx context.Context, rc *RunContext) error {
env, err := evaluateJobEnv(ctx, rc)
rc.Env = env
rc.ExprEval = rc.NewExpressionEvaluator(ctx)
env := rc.GetEnv()
var workflowEnv map[string]string
if rc.Run.Workflow.Env == nil {
if err := model.DecodeEvaluated("workflow env", rc.Run.Workflow.RawEnv, rc.ExprEval.shared(ctx).EvaluateYamlNode, &workflowEnv); err != nil {
return err
}
}
if workflowEnv != nil {
rc.Env = mergeMaps(workflowEnv, env)
rc.ExprEval = rc.NewExpressionEvaluator(ctx)
}
var err error
for k, v := range env {
if rc.Env[k], err = rc.ExprEval.Interpolate(ctx, v); err != nil {
return fmt.Errorf("unable to interpolate env %s: %w", k, err)
}
if err != nil {
return err
}
var defaults model.Defaults
if err := model.DecodeEvaluated("defaults", rc.Run.Job().RawDefaults, rc.ExprEval.shared(ctx).EvaluateYamlNode, &defaults); err != nil {
return err
}
rc.jobRunDefaults = defaults.Run
if rc.containerSpec.Image == "" {
return nil
}
_, containerEnv := splitContainerEnv(rc.Run.Job().RawContainer)
return model.DecodeEvaluated("container env", containerEnv, rc.ExprEval.shared(ctx).EvaluateYamlNode, &rc.containerSpec.Env)
return model.DecodeEvaluated("container env", rc.containerEnv, rc.ExprEval.shared(ctx).EvaluateYamlNode, &rc.containerSpec.Env)
}
func setJobOutputs(ctx context.Context, rc *RunContext) error {
@@ -522,7 +538,7 @@ func tryUploadJobSummary(ctx context.Context, rc *RunContext) {
"/jobs/" + strconv.FormatInt(jobID, 10) + "/steps/"
actPath := rc.JobContainer.GetActPath()
// Reuse a single client across all step uploads so connections can be pooled.
client := &http.Client{Timeout: jobSummaryUploadRequestTimeout}
client := &http.Client{Timeout: jobSummaryUploadRequestTimeout, Transport: rc.Config.instanceTransportFor(base)}
for i := range rc.Run.Job().Steps {
summaryPath := path.Join(actPath, "workflow", "step-summary-"+strconv.Itoa(i)+".md")
body, ok := readSingleFileFromContainerArchive(ctx, rc.JobContainer, summaryPath, maxJobSummaryBytes)
@@ -534,6 +550,15 @@ func tryUploadJobSummary(ctx context.Context, rc *RunContext) {
}
}
// For Gitea
// instanceTransportFor returns InstanceTransport only for URLs on the registered address, as steps can rewrite the summary URL through GITHUB_ENV.
func (c Config) instanceTransportFor(rawURL string) http.RoundTripper {
if !strings.HasPrefix(rawURL, c.GitHubInstance+"/") {
return nil
}
return c.InstanceTransport
}
// extractJobIDFromRuntimeToken returns the JobID claim from an ACTIONS_RUNTIME_TOKEN JWT
// without verifying its signature. Returns 0 if the token is unparseable or has no JobID.
func extractJobIDFromRuntimeToken(token string) int64 {
@@ -662,6 +687,13 @@ func archiveEntryMatchesPath(entryName, requestedPath string) bool {
return entryName == requestedPath || entryName == path.Base(requestedPath)
}
// useJobStageLogger tags the log entries of a job level executor with the stage they belong to.
func useJobStageLogger(stage string, executor common.Executor) common.Executor {
return func(ctx context.Context) error {
return executor(common.WithLogger(ctx, common.Logger(ctx).With("stage", stage)))
}
}
func useStepLogger(rc *RunContext, stepModel *model.Step, stage stepStage, executor common.Executor) common.Executor {
return func(ctx context.Context) error {
ctx = withStepLogger(ctx, stepModel.Number, stepModel.ID, rc.ExprEval.InterpolateName(ctx, stepModel.String()), stage.String())

View File

@@ -21,11 +21,11 @@ import (
"time"
"gitea.com/gitea/runner/act/common"
"gitea.com/gitea/runner/act/common/logging"
"gitea.com/gitea/runner/act/common/logging/logtest"
"gitea.com/gitea/runner/act/container"
"gitea.dev/actionslib/pkg/model"
log "github.com/sirupsen/logrus"
logrustest "github.com/sirupsen/logrus/hooks/test"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/mock"
"github.com/stretchr/testify/require"
@@ -44,7 +44,7 @@ func TestJobExecutor(t *testing.T) {
{workdir, "job-nil-step", "push", "invalid Step 0: missing run or uses key", platforms, secrets},
}
// These tests are sufficient to only check syntax.
ctx := common.WithDryrun(context.Background(), true)
ctx := common.WithDryrun(t.Context(), true)
for _, table := range tables {
t.Run(table.workflowPath, func(t *testing.T) {
t.Parallel()
@@ -118,7 +118,6 @@ func (sfm *stepFactoryMock) newStep(model *model.Step, rc *RunContext) (step, er
type actionPreparerMock struct {
reference string
sha string
ok bool
err error
prepared int
}
@@ -130,31 +129,25 @@ func (apm *actionPreparerMock) prepareActionExecutor() common.Executor {
}
}
func (apm *actionPreparerMock) actionDownloadInfo() (string, string, bool) {
return apm.reference, apm.sha, apm.ok
func (apm *actionPreparerMock) actionDownloadInfo() (string, string) {
return apm.reference, apm.sha
}
func TestPrintPrepareActionsGolden(t *testing.T) {
buf := &bytes.Buffer{}
logger := log.New()
logger.SetOutput(buf)
logger.SetLevel(log.InfoLevel)
logger.SetFormatter(&jobLogFormatter{color: cyan})
ctx := common.WithLogger(context.Background(), logger.WithFields(log.Fields{"job": "j1"}))
logger := logging.New(logging.NewLevelHandler(newJobLogHandler(buf, cyan), logging.LevelInfo))
ctx := common.WithLogger(t.Context(), logger.With("job", "j1"))
preparers := []actionPreparer{
&actionPreparerMock{reference: "actions/checkout@v7", sha: "9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0", ok: true},
// A resolved commit is best effort; the ref alone is reported when it is unknown.
&actionPreparerMock{reference: "actions/setup-go@v6", ok: true},
&actionPreparerMock{reference: "actions/checkout@v7", sha: "9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0"},
// A step that downloads nothing, such as the checkout of the workflow's own repository.
&actionPreparerMock{ok: false},
&actionPreparerMock{},
}
require.NoError(t, printPrepareActions(&RunContext{}, preparers)(ctx))
want := strings.Join([]string{
"[j1] | Prepare all required actions",
"[j1] | Download action repository 'actions/checkout@v7' (SHA:9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0)",
"[j1] | Download action repository 'actions/setup-go@v6'",
"",
}, "\n")
assert.Equal(t, want, buf.String())
@@ -162,10 +155,8 @@ func TestPrintPrepareActionsGolden(t *testing.T) {
func TestPrintPrepareActionsSkipsWithoutActions(t *testing.T) {
buf := &bytes.Buffer{}
logger := log.New()
logger.SetOutput(buf)
logger.SetFormatter(&jobLogFormatter{color: cyan})
ctx := common.WithLogger(context.Background(), logger.WithFields(log.Fields{"job": "j1"}))
logger := logging.New(logging.NewLevelHandler(newJobLogHandler(buf, cyan), logging.LevelInfo))
ctx := common.WithLogger(t.Context(), logger.With("job", "j1"))
require.NoError(t, printPrepareActions(&RunContext{}, nil)(ctx))
@@ -173,12 +164,12 @@ func TestPrintPrepareActionsSkipsWithoutActions(t *testing.T) {
}
func TestPrintPrepareActionsFailsJobOnDownloadError(t *testing.T) {
logger, _ := logrustest.NewNullLogger()
ctx := common.WithJobErrorContainer(common.WithLogger(context.Background(), logger.WithField("job", "j1")))
logger := logging.New(logtest.New())
ctx := common.WithJobErrorContainer(common.WithLogger(t.Context(), logger.With("job", "j1")))
downloadErr := errors.New("failed to fetch \"actions/checkout\"")
rc := &RunContext{}
remaining := &actionPreparerMock{reference: "actions/setup-go@v6", ok: true}
remaining := &actionPreparerMock{reference: "actions/setup-go@v6", sha: "sha"}
err := printPrepareActions(rc, []actionPreparer{
&actionPreparerMock{err: downloadErr},
@@ -192,6 +183,27 @@ func TestPrintPrepareActionsFailsJobOnDownloadError(t *testing.T) {
assert.Zero(t, remaining.prepared)
}
func TestNewStepsExecutorSkipsStepsAfterSetupFailure(t *testing.T) {
for name, tt := range map[string]struct {
preErr error
mainRan bool
}{
"setup failure": {preErr: setupError{assert.AnError}},
"pre step failure": {preErr: assert.AnError, mainRan: true}, // always() steps still run, as on GitHub
} {
t.Run(name, func(t *testing.T) {
ctx := common.WithJobErrorContainer(common.WithLogger(context.Background(), logging.New(logtest.New())))
mainRan := false
err := newStepsExecutor(&RunContext{},
[]common.Executor{func(context.Context) error { return tt.preErr }},
[]common.Executor{func(context.Context) error { mainRan = true; return nil }},
)(ctx)
require.ErrorIs(t, err, assert.AnError)
assert.Equal(t, tt.mainRan, mainRan)
})
}
}
func TestPrintCompleteJobName(t *testing.T) {
for name, tt := range map[string]struct {
rc *RunContext
@@ -203,10 +215,8 @@ func TestPrintCompleteJobName(t *testing.T) {
} {
t.Run(name, func(t *testing.T) {
buf := &bytes.Buffer{}
logger := log.New()
logger.SetOutput(buf)
logger.SetFormatter(&jobLogFormatter{color: cyan})
ctx := common.WithLogger(context.Background(), logger.WithFields(log.Fields{"job": "j1"}))
logger := logging.New(logging.NewLevelHandler(newJobLogHandler(buf, cyan), logging.LevelInfo))
ctx := common.WithLogger(t.Context(), logger.With("job", "j1"))
require.NoError(t, printCompleteJobName(tt.rc)(ctx))
@@ -225,7 +235,7 @@ type actionStepMock struct {
// every action is downloaded before any step runs, and the job name closes the section. A pre
// step that downloaded its own action would leave the log interleaved with the downloads.
func TestNewJobExecutorDownloadsAllActionsBeforeTheFirstStep(t *testing.T) {
ctx := common.WithJobErrorContainer(context.Background())
ctx := common.WithJobErrorContainer(t.Context())
jim := &jobInfoMock{}
sfm := &stepFactoryMock{}
rc := &RunContext{
@@ -253,11 +263,12 @@ func TestNewJobExecutorDownloadsAllActionsBeforeTheFirstStep(t *testing.T) {
for _, stepModel := range steps {
sm := &stepMock{}
apm := &actionPreparerMock{reference: "actions/checkout@v" + stepModel.ID, ok: true}
apm := &actionPreparerMock{reference: "actions/checkout@v" + stepModel.ID, sha: "sha" + stepModel.ID}
sfm.On("newStep", stepModel, rc).Return(&actionStepMock{stepMock: sm, actionPreparerMock: apm}, nil)
sm.On("pre").Return(func(context.Context) error {
sm.On("pre").Return(func(ctx context.Context) error {
executorOrder = append(executorOrder, "pre"+stepModel.ID)
common.Logger(ctx).Infof("Run Pre %s", stepModel.ID)
return nil
})
sm.On("main").Return(func(context.Context) error {
@@ -269,24 +280,31 @@ func TestNewJobExecutorDownloadsAllActionsBeforeTheFirstStep(t *testing.T) {
defer sm.AssertExpectations(t)
}
logger, hook := logrustest.NewNullLogger()
err := newJobExecutor(jim, sfm, rc)(common.WithLogger(ctx, logger.WithField("job", "test")))
logger, hook := logtest.NewLogger()
err := newJobExecutor(jim, sfm, rc)(common.WithLogger(ctx, logger.With("job", "test")))
require.NoError(t, err)
assert.Equal(t, []string{"pre1", "pre2", "step1", "step2"}, executorOrder)
// the setup is logged ahead of the pre steps and the cleanup is tagged, so the reporter can show both as steps of their own
setup := make([]string, 0)
for _, entry := range hook.AllEntries() {
for _, entry := range hook.AllRecords() {
if strings.HasPrefix(entry.Message, "Prepare all required actions") || strings.HasPrefix(entry.Message, "Download action") ||
strings.HasPrefix(entry.Message, "Complete job name") {
strings.HasPrefix(entry.Message, "Complete job name") || strings.HasPrefix(entry.Message, "Run Pre") {
setup = append(setup, entry.Message)
}
if _, ok := entry.Attr("jobResult"); ok || strings.HasPrefix(entry.Message, "Cleaning up container") {
stage, _ := entry.Attr("stage")
assert.Equal(t, StageComplete, stage, entry.Message)
}
}
assert.Equal(t, []string{
"Prepare all required actions",
"Download action repository 'actions/checkout@v1'",
"Download action repository 'actions/checkout@v2'",
"Download action repository 'actions/checkout@v1' (SHA:sha1)",
"Download action repository 'actions/checkout@v2' (SHA:sha2)",
"Complete job name: test",
"Run Pre 1",
"Run Pre 2",
}, setup)
}
@@ -462,7 +480,7 @@ func TestNewJobExecutor(t *testing.T) {
t.Run(tt.name, func(t *testing.T) {
fmt.Printf("::group::%s\n", tt.name) //nolint:forbidigo // pre-existing issue from nektos/act
ctx, cancel := context.WithCancel(common.WithJobErrorContainer(context.Background()))
ctx, cancel := context.WithCancel(common.WithJobErrorContainer(t.Context()))
defer cancel()
jim := &jobInfoMock{}
sfm := &stepFactoryMock{}
@@ -610,7 +628,7 @@ func (ctx *controllableDeadlineContext) expire() {
// still run against a fresh, non-expired context, and the job must still be
// reported as failed.
func TestNewJobExecutorRunsPostStepsAfterTimeout(t *testing.T) {
ctx := newControllableDeadlineContext(common.WithJobErrorContainer(context.Background()))
ctx := newControllableDeadlineContext(common.WithJobErrorContainer(t.Context()))
jim := &jobInfoMock{}
sfm := &stepFactoryMock{}
@@ -691,7 +709,7 @@ func TestSetJobResultMatrixContinueOnError(t *testing.T) {
return &model.Plan{Stages: []*model.Stage{{Runs: []*model.Run{{Workflow: wf, JobID: "job1"}}}}}
}
ctx := context.Background()
ctx := t.Context()
// fail drives a single matrix combination through the failure path; each
// RunContext is its own jobInfo (rc implements jobInfo) and shares the job.
@@ -786,7 +804,7 @@ func TestTryUploadJobSummaryRetriesTransientFailure(t *testing.T) {
}))
defer server.Close()
ctx := context.Background()
ctx := t.Context()
cm := &containerMock{}
cm.On("GetContainerArchive", mock.Anything, "/var/run/act/workflow/step-summary-0.md").Return(
io.NopCloser(bytes.NewReader(tarArchive(t, tarEntry{name: "step-summary-0.md", body: "# summary"}))),
@@ -830,7 +848,7 @@ func TestTryUploadJobSummaryStopsAtPhaseTimeout(t *testing.T) {
defer server.Close()
defer close(release)
ctx := context.Background()
ctx := t.Context()
cm := &containerMock{}
cm.On("GetContainerArchive", mock.Anything, "/var/run/act/workflow/step-summary-0.md").Return(
io.NopCloser(bytes.NewReader(tarArchive(t, tarEntry{name: "step-summary-0.md", body: "# summary"}))),
@@ -858,7 +876,7 @@ func TestTryUploadJobSummaryStopsAtPhaseTimeout(t *testing.T) {
cm.AssertExpectations(t)
}
func TestTryUploadJobSummaryUploadsEachStepIndependently(t *testing.T) {
func TestTryUploadJobSummaryUploadsEachStepIndependentlyWithInstanceTransport(t *testing.T) {
runtimeToken := fakeRuntimeToken(34)
type upload struct {
@@ -866,7 +884,7 @@ func TestTryUploadJobSummaryUploadsEachStepIndependently(t *testing.T) {
body string
}
var got []upload
server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
server := httptest.NewTLSServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
body, err := io.ReadAll(r.Body)
assert.NoError(t, err)
got = append(got, upload{r.URL.Path, string(body)})
@@ -874,7 +892,7 @@ func TestTryUploadJobSummaryUploadsEachStepIndependently(t *testing.T) {
}))
defer server.Close()
ctx := context.Background()
ctx := t.Context()
cm := &containerMock{}
// Three steps: 0 has content, 1 has empty content (skipped), 2 has content.
cm.On("GetContainerArchive", mock.Anything, "/var/run/act/workflow/step-summary-0.md").Return(
@@ -896,6 +914,7 @@ func TestTryUploadJobSummaryUploadsEachStepIndependently(t *testing.T) {
"ACTIONS_RUNTIME_TOKEN": runtimeToken,
"GITEA_RUN_ID": "12",
}, cm, 3)
rc.Config.GitHubInstance, rc.Config.InstanceTransport = server.URL, server.Client().Transport
tryUploadJobSummary(ctx, rc)
@@ -921,7 +940,7 @@ func TestTryUploadJobSummaryRequiresExactCapability(t *testing.T) {
"GITEA_RUN_ID": "12",
}, &containerMock{}, 1)
tryUploadJobSummary(context.Background(), rc)
tryUploadJobSummary(t.Context(), rc)
assert.Equal(t, 0, requests)
}
@@ -941,7 +960,7 @@ func TestTryUploadJobSummarySkipsWhenJobIDMissingFromToken(t *testing.T) {
"GITEA_RUN_ID": "12",
}, &containerMock{}, 1)
tryUploadJobSummary(context.Background(), rc)
tryUploadJobSummary(t.Context(), rc)
assert.Equal(t, 0, requests)
}
@@ -954,7 +973,7 @@ func TestExtractJobIDFromRuntimeToken(t *testing.T) {
}
func TestReadSingleFileFromContainerArchiveFindsMatchingRegularFile(t *testing.T) {
ctx := context.Background()
ctx := t.Context()
cm := &containerMock{}
cm.On("GetContainerArchive", ctx, "/var/run/act/workflow/SUMMARY.md").Return(
io.NopCloser(bytes.NewReader(tarArchive(t,
@@ -973,8 +992,8 @@ func TestReadSingleFileFromContainerArchiveFindsMatchingRegularFile(t *testing.T
}
func TestReadSingleFileFromContainerArchiveTruncatesWhenTooLarge(t *testing.T) {
logger, hook := logrustest.NewNullLogger()
ctx := common.WithLogger(context.Background(), logger)
logger, hook := logtest.NewLogger()
ctx := common.WithLogger(t.Context(), logger)
cm := &containerMock{}
content := strings.Repeat("a", 300)
cm.On("GetContainerArchive", ctx, "/var/run/act/workflow/SUMMARY.md").Return(
@@ -991,15 +1010,15 @@ func TestReadSingleFileFromContainerArchiveTruncatesWhenTooLarge(t *testing.T) {
assert.LessOrEqual(t, len(body), maxBytes)
keep := maxBytes - len(jobSummaryTruncationMarker)
assert.Equal(t, []byte(content[:keep]+jobSummaryTruncationMarker), body)
if assert.Len(t, hook.Entries, 1) {
assert.Contains(t, hook.Entries[0].Message, "job summary truncated")
if assert.Len(t, hook.AllRecords(), 1) {
assert.Contains(t, hook.AllRecords()[0].Message, "job summary truncated")
}
cm.AssertExpectations(t)
}
func TestReadSingleFileFromContainerArchiveKeepsExactLimitWithoutWarning(t *testing.T) {
logger, hook := logrustest.NewNullLogger()
ctx := common.WithLogger(context.Background(), logger)
logger, hook := logtest.NewLogger()
ctx := common.WithLogger(t.Context(), logger)
cm := &containerMock{}
cm.On("GetContainerArchive", ctx, "/var/run/act/workflow/SUMMARY.md").Return(
io.NopCloser(bytes.NewReader(tarArchive(t, tarEntry{name: "SUMMARY.md", body: "abc"}))),
@@ -1011,7 +1030,7 @@ func TestReadSingleFileFromContainerArchiveKeepsExactLimitWithoutWarning(t *test
// A summary that is exactly at the limit is kept whole and not flagged as truncated.
assert.True(t, ok)
assert.Equal(t, []byte("abc"), body)
assert.Empty(t, hook.Entries)
assert.Empty(t, hook.AllRecords())
cm.AssertExpectations(t)
}
@@ -1069,7 +1088,7 @@ func makeTestRC(t *testing.T, jobYAML string) *RunContext {
Name: "workflow1",
Jobs: map[string]*model.Job{"job1": job},
}, nil)
rc.ExprEval = rc.NewExpressionEvaluator(context.Background())
rc.ExprEval = rc.NewExpressionEvaluator(t.Context())
return rc
}
@@ -1087,7 +1106,7 @@ func TestApplyJobTimeout(t *testing.T) {
for _, tc := range cases {
t.Run(tc.name, func(t *testing.T) {
rc := makeTestRC(t, tc.yaml)
ctx := context.Background()
ctx := t.Context()
newCtx, cancel := applyJobTimeout(ctx, rc, rc.Run.Job())
defer cancel()
_, hasDeadline := newCtx.Deadline()
@@ -1111,7 +1130,7 @@ func TestEvaluateJobContinueOnError(t *testing.T) {
for _, tc := range cases {
t.Run(tc.name, func(t *testing.T) {
rc := makeTestRC(t, tc.yaml)
got := evaluateJobContinueOnError(context.Background(), rc, rc.Run.Job())
got := evaluateJobContinueOnError(t.Context(), rc, rc.Run.Job())
assert.Equal(t, tc.want, got)
})
}
@@ -1176,7 +1195,7 @@ func TestTryUploadJobSummaryMasksSecrets(t *testing.T) {
rc.Config.ExtraMasks = []string{"pr0xypw"}
rc.Masks = []string{"runtime-added"}
tryUploadJobSummary(context.Background(), rc)
tryUploadJobSummary(t.Context(), rc)
assert.Equal(t, "deployed true with *** and *** via ***", got)
cm.AssertExpectations(t)

View File

@@ -42,7 +42,7 @@ func (rc *RunContext) runJobHook(ctx context.Context, hookPath, name string) err
}
cmd, shell := hookCommand(hookPath)
rawLogger := common.Logger(ctx).WithField(rawOutputField, true)
rawLogger := common.Logger(ctx).With(rawOutputField, true)
defer rawLogger.Infof("::endgroup::")
rawLogger.Infof("::group::Run '%s'", EscapeCommandData(hookPath))
rawLogger.Infof("A %s hook has been configured by the runner administrator", name)

View File

@@ -12,9 +12,10 @@ import (
"testing"
"gitea.com/gitea/runner/act/common"
"gitea.com/gitea/runner/act/common/logging"
"gitea.com/gitea/runner/act/common/logging/logtest"
"gitea.dev/actionslib/pkg/model"
"github.com/sirupsen/logrus/hooks/test"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
@@ -64,8 +65,8 @@ func newHookRunContext(jobContainer *hookContainer, config *Config) (*RunContext
Run: &model.Run{JobID: "job", Workflow: &model.Workflow{Jobs: map[string]*model.Job{"job": {}}}},
JobContainer: jobContainer,
}
logger, _ := test.NewNullLogger()
ctx := common.WithJobErrorContainer(common.WithLogger(context.Background(), logger.WithField("test", true)))
logger := logging.New(logtest.New())
ctx := common.WithJobErrorContainer(common.WithLogger(context.Background(), logger.With("test", true)))
rc.ExprEval = rc.NewExpressionEvaluator(ctx)
return rc, ctx
}

View File

@@ -19,8 +19,8 @@ import (
"sync"
"gitea.com/gitea/runner/act/common"
"gitea.com/gitea/runner/act/common/logging"
"github.com/sirupsen/logrus"
"golang.org/x/term"
)
@@ -35,6 +35,16 @@ const (
gray = 37
)
const (
// The "stage" of the log entries of a step, and of the job level entries that finalize the job after its steps.
StagePre = "Pre"
StageMain = "Main"
StagePost = "Post"
StageComplete = "Complete"
// StepStartedField marks the log entry of a step that is about to run, unlike one that is skipped.
StepStartedField = "stepStarted"
)
const (
rawOutputField = "raw_output"
scriptLineCyanField = "script_line_cyan"
@@ -73,8 +83,9 @@ func WithMasks(ctx context.Context, masks *[]string) context.Context {
return context.WithValue(ctx, masksContextKeyVal, masks)
}
// JobLoggerFactory supplies the handler a job's own log output goes to, in place of stdout.
type JobLoggerFactory interface {
WithJobLogger() *logrus.Logger
WithJobLogger() logging.Handler
}
type jobLoggerFactoryContextKey string
@@ -89,84 +100,70 @@ func WithJobLoggerFactory(ctx context.Context, factory JobLoggerFactory) context
func WithJobLogger(ctx context.Context, jobID, jobName string, config *Config, masks *[]string, matrix map[string]any) context.Context {
ctx = WithMasks(ctx, masks)
var logger *logrus.Logger
var output logging.Handler
if jobLoggerFactory, ok := ctx.Value(jobLoggerFactoryContextKeyVal).(JobLoggerFactory); ok && jobLoggerFactory != nil {
logger = jobLoggerFactory.WithJobLogger()
output = jobLoggerFactory.WithJobLogger()
} else if config.JSONLogger {
output = newJobLogJSONHandler(os.Stdout)
} else {
var formatter logrus.Formatter
if config.JSONLogger {
formatter = &logrus.JSONFormatter{}
} else {
mux.Lock()
defer mux.Unlock()
nextColor++
formatter = &jobLogFormatter{color: colors[nextColor%len(colors)]}
}
logger = logrus.New()
logger.SetOutput(os.Stdout)
logger.SetLevel(logrus.GetLevel())
logger.SetFormatter(formatter)
mux.Lock()
nextColor++
color := colors[nextColor%len(colors)]
mux.Unlock()
output = newJobLogHandler(os.Stdout, color)
}
{ // Adapt to Gitea
if hook := common.LoggerHook(ctx); hook != nil {
logger.AddHook(hook)
}
if config.JobLoggerLevel != nil {
logger.SetLevel(*config.JobLoggerLevel)
} else {
logger.SetLevel(logrus.TraceLevel)
}
var handlers []logging.Handler
if h := common.LoggerHandler(ctx); h != nil { // Adapt to Gitea
handlers = append(handlers, h)
}
logger.SetFormatter(&maskedFormatter{
Formatter: logger.Formatter,
masker: valueMasker(config.InsecureSecrets, config.maskers()),
handlers = append(handlers, &maskedHandler{
Handler: output,
masker: valueMasker(config.InsecureSecrets, config.maskers()),
})
rtn := logger.WithFields(logrus.Fields{
"job": jobName,
"jobID": jobID,
"dryrun": common.Dryrun(ctx),
"matrix": matrix,
}).WithContext(ctx)
level := logging.LevelTrace
if config.JobLoggerLevel != nil {
level = *config.JobLoggerLevel
}
rtn := logging.New(logging.NewLevelHandler(logging.NewMultiHandler(handlers...), level)).With(
"job", jobName,
"jobID", jobID,
"dryrun", common.Dryrun(ctx),
"matrix", matrix,
).WithContext(ctx)
return common.WithLogger(ctx, rtn)
}
func WithCompositeLogger(ctx context.Context, masks *[]string) context.Context {
ctx = WithMasks(ctx, masks)
return common.WithLogger(ctx, common.Logger(ctx).WithFields(logrus.Fields{}).WithContext(ctx))
return common.WithLogger(ctx, common.Logger(ctx).WithContext(ctx))
}
func WithCompositeStepLogger(ctx context.Context, stepID string) context.Context {
val := common.Logger(ctx)
stepIDs := make([]string, 0)
stepIDs := append(slices.Clone(common.StepIDs(ctx)), stepID)
ctx = common.WithStepIDs(ctx, stepIDs)
if logger, ok := val.(*logrus.Entry); ok {
if oldStepIDs, ok := logger.Data["stepID"].([]string); ok {
stepIDs = append(stepIDs, oldStepIDs...)
}
}
stepIDs = append(stepIDs, stepID)
return common.WithLogger(ctx, common.Logger(ctx).WithFields(logrus.Fields{
"stepID": stepIDs,
}).WithContext(ctx))
return common.WithLogger(ctx, common.Logger(ctx).With("stepID", stepIDs).WithContext(ctx))
}
func withStepLogger(ctx context.Context, stepNumber int, stepID, stepName, stageName string) context.Context {
rtn := common.Logger(ctx).WithFields(logrus.Fields{
"stepNumber": stepNumber,
"step": stepName,
"stepID": []string{stepID},
"stage": stageName,
})
stepIDs := []string{stepID}
ctx = common.WithStepIDs(ctx, stepIDs)
rtn := common.Logger(ctx).With(
"stepNumber", stepNumber,
"step", stepName,
"stepID", stepIDs,
"stage", stageName,
)
return common.WithLogger(ctx, rtn)
}
type entryProcessor func(entry *logrus.Entry) *logrus.Entry
// entryProcessor rewrites a record before the output handler sees it.
type entryProcessor func(ctx context.Context, r logging.Record) logging.Record
var secretValueEncoders = []func(string) string{
func(v string) string { return base64.StdEncoding.EncodeToString([]byte(v)) },
@@ -321,16 +318,16 @@ func valueMasker(insecureSecrets bool, oldnew []string) entryProcessor {
replacer *strings.Replacer
)
return func(entry *logrus.Entry) *logrus.Entry {
return func(ctx context.Context, r logging.Record) logging.Record {
if insecureSecrets {
return entry
return r
}
masks := Masks(entry.Context)
masks := Masks(ctx)
if len(*masks) == 0 {
entry.Message = defReplacer.Replace(entry.Message)
return entry
r.Message = defReplacer.Replace(r.Message)
return r
}
mu.Lock()
@@ -349,89 +346,129 @@ func valueMasker(insecureSecrets bool, oldnew []string) entryProcessor {
cmasker := replacer
mu.Unlock()
entry.Message = cmasker.Replace(entry.Message)
r.Message = cmasker.Replace(r.Message)
return entry
return r
}
}
type maskedFormatter struct {
logrus.Formatter
type maskedHandler struct {
logging.Handler
masker entryProcessor
}
func (f *maskedFormatter) Format(entry *logrus.Entry) ([]byte, error) {
return f.Formatter.Format(f.masker(entry))
func (m *maskedHandler) Handle(ctx context.Context, r logging.Record) error {
return m.Handler.Handle(ctx, m.masker(ctx, r))
}
type jobLogFormatter struct {
func (m *maskedHandler) WithAttrs(attrs []logging.Attr) logging.Handler {
return &maskedHandler{Handler: m.Handler.WithAttrs(attrs), masker: m.masker}
}
// jobLogHandler writes a job's log to w, one record per line in the shape format gives it.
type jobLogHandler struct {
w io.Writer
mu *sync.Mutex // shared by every handler derived with WithAttrs, as they share w
attrs []logging.Attr
format func(logging.Record) ([]byte, error)
}
// newJobLogHandler writes readable text.
func newJobLogHandler(w io.Writer, color int) *jobLogHandler {
return &jobLogHandler{w: w, mu: &sync.Mutex{}, format: textFormatter{w: w, color: color}.format}
}
func (h *jobLogHandler) Enabled(context.Context, logging.Level) bool { return true }
func (h *jobLogHandler) WithAttrs(attrs []logging.Attr) logging.Handler {
c := *h
c.attrs = slices.Concat(h.attrs, attrs)
return &c
}
func (h *jobLogHandler) Handle(_ context.Context, r logging.Record) error {
r.Attrs = slices.Concat(h.attrs, r.Attrs)
line, err := h.format(r)
if err != nil {
return err
}
h.mu.Lock()
defer h.mu.Unlock()
_, err = h.w.Write(line)
return err
}
type textFormatter struct {
w io.Writer
color int
}
func (f *jobLogFormatter) Format(entry *logrus.Entry) ([]byte, error) {
func (f textFormatter) format(r logging.Record) ([]byte, error) {
b := &bytes.Buffer{}
// the web renderer decodes command data, so this local view has to as well
if _, _, _, ok := tryParseRawActionCommand(entry.Message + "\n"); ok {
entry.Message = UnescapeCommandData(entry.Message)
if _, _, _, ok := tryParseRawActionCommand(r.Message + "\n"); ok {
r.Message = UnescapeCommandData(r.Message)
}
r.Message = strings.TrimSuffix(r.Message, "\n")
if f.isColored(entry) {
f.printColored(b, entry)
if f.isColored() {
f.printColored(b, r)
} else {
f.print(b, entry)
f.print(b, r)
}
b.WriteByte('\n')
return b.Bytes(), nil
}
func (f *jobLogFormatter) printColored(b *bytes.Buffer, entry *logrus.Entry) {
entry.Message = strings.TrimSuffix(entry.Message, "\n")
func attrIsTrue(r logging.Record, key string) bool {
v, _ := r.Attr(key)
return v == true
}
job := entry.Data["job"]
func (f textFormatter) printColored(b *bytes.Buffer, r logging.Record) {
job, _ := r.Attr("job")
debugFlag := ""
if entry.Level == logrus.DebugLevel {
if r.Level == logging.LevelDebug {
debugFlag = "[DEBUG] "
}
switch {
case entry.Data[rawOutputField] == true:
if entry.Data[scriptLineCyanField] == true {
fmt.Fprintf(b, "\x1b[%dm|\x1b[0m \x1b[36;1m%s\x1b[0m", f.color, entry.Message)
case attrIsTrue(r, rawOutputField):
if attrIsTrue(r, scriptLineCyanField) {
fmt.Fprintf(b, "\x1b[%dm|\x1b[0m \x1b[36;1m%s\x1b[0m", f.color, r.Message)
} else {
fmt.Fprintf(b, "\x1b[%dm|\x1b[0m %s", f.color, entry.Message)
fmt.Fprintf(b, "\x1b[%dm|\x1b[0m %s", f.color, r.Message)
}
case entry.Data["dryrun"] == true:
fmt.Fprintf(b, "\x1b[1m\x1b[%dm\x1b[7m*DRYRUN*\x1b[0m \x1b[%dm[%s] \x1b[0m%s%s", gray, f.color, job, debugFlag, entry.Message)
case attrIsTrue(r, "dryrun"):
fmt.Fprintf(b, "\x1b[1m\x1b[%dm\x1b[7m*DRYRUN*\x1b[0m \x1b[%dm[%s] \x1b[0m%s%s", gray, f.color, job, debugFlag, r.Message)
default:
fmt.Fprintf(b, "\x1b[%dm[%s] \x1b[0m%s%s", f.color, job, debugFlag, entry.Message)
fmt.Fprintf(b, "\x1b[%dm[%s] \x1b[0m%s%s", f.color, job, debugFlag, r.Message)
}
}
func (f *jobLogFormatter) print(b *bytes.Buffer, entry *logrus.Entry) {
entry.Message = strings.TrimSuffix(entry.Message, "\n")
job := entry.Data["job"]
func (f textFormatter) print(b *bytes.Buffer, r logging.Record) {
job, _ := r.Attr("job")
debugFlag := ""
if entry.Level == logrus.DebugLevel {
if r.Level == logging.LevelDebug {
debugFlag = "[DEBUG] "
}
switch {
case entry.Data[rawOutputField] == true:
fmt.Fprintf(b, "[%s] | %s", job, entry.Message)
case entry.Data["dryrun"] == true:
fmt.Fprintf(b, "*DRYRUN* [%s] %s%s", job, debugFlag, entry.Message)
case attrIsTrue(r, rawOutputField):
fmt.Fprintf(b, "[%s] | %s", job, r.Message)
case attrIsTrue(r, "dryrun"):
fmt.Fprintf(b, "*DRYRUN* [%s] %s%s", job, debugFlag, r.Message)
default:
fmt.Fprintf(b, "[%s] %s%s", job, debugFlag, entry.Message)
fmt.Fprintf(b, "[%s] %s%s", job, debugFlag, r.Message)
}
}
func (f *jobLogFormatter) isColored(entry *logrus.Entry) bool {
isColored := checkIfTerminal(entry.Logger.Out)
func (f textFormatter) isColored() bool {
isColored := checkIfTerminal(f.w)
if force, ok := os.LookupEnv("CLICOLOR_FORCE"); ok && force != "0" {
isColored = true

52
act/runner/logger_json.go Normal file
View File

@@ -0,0 +1,52 @@
// Copyright 2026 The Gitea Authors. All rights reserved.
// SPDX-License-Identifier: MIT
package runner
import (
"encoding/json"
"fmt"
"io"
"sync"
"time"
"gitea.com/gitea/runner/act/common/logging"
)
// newJobLogJSONHandler writes one JSON object per record, in the shape of logrus's JSONFormatter.
func newJobLogJSONHandler(w io.Writer) *jobLogHandler {
return &jobLogHandler{w: w, mu: &sync.Mutex{}, format: formatJSON}
}
func formatJSON(r logging.Record) ([]byte, error) {
data := make(map[string]any, len(r.Attrs)+3)
for _, a := range r.Attrs {
switch v := a.Value.(type) {
case error:
data[a.Key] = v.Error() // an error marshals to {}
default:
data[a.Key] = v
}
}
for _, key := range []string{"time", "msg", "level"} {
if v, ok := data[key]; ok { // keep the fixed keys from being overwritten
data["fields."+key] = v
}
}
data["time"] = r.Time.Format(time.RFC3339)
data["msg"] = r.Message
data["level"] = jsonLevel(r.Level)
line, err := json.Marshal(data)
if err != nil {
return nil, fmt.Errorf("failed to marshal fields to JSON, %w", err)
}
return append(line, '\n'), nil
}
func jsonLevel(l logging.Level) string {
if l.String() == "warn" {
return "warning" // what logrus calls it
}
return l.String()
}

View File

@@ -4,13 +4,19 @@
package runner
import (
"bytes"
"encoding/base64"
"encoding/json"
"errors"
"io"
"net/url"
"strings"
"testing"
"github.com/sirupsen/logrus"
"gitea.com/gitea/runner/act/common"
"gitea.com/gitea/runner/act/common/logging"
"gitea.com/gitea/runner/act/common/logging/logtest"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
@@ -49,10 +55,7 @@ func TestValueMasker(t *testing.T) {
ctx := WithMasks(t.Context(), &entry.masks)
masker := valueMasker(false, AppendSecretMaskers(nil, entry.secrets))
for line := range strings.SplitSeq(entry.lines, "\n") {
lentry := masker(&logrus.Entry{
Context: ctx,
Message: line,
})
lentry := masker(ctx, logging.Record{Message: line})
for _, line := range entry.disallowed {
assert.NotContains(t, lentry.Message, line)
}
@@ -79,7 +82,7 @@ func TestValueMaskerEncodedSecrets(t *testing.T) {
} {
t.Run(tc.name, func(t *testing.T) {
masker := valueMasker(false, AppendSecretMaskers(nil, map[string]string{"TOKEN": tc.secret}))
entry := masker(&logrus.Entry{Context: t.Context(), Message: strings.Join(tc.encoded, " ")})
entry := masker(t.Context(), logging.Record{Message: strings.Join(tc.encoded, " ")})
assert.Contains(t, entry.Message, "***")
for _, disallowed := range tc.encoded {
@@ -105,7 +108,7 @@ func TestValueMaskerJSONEscapesBothWays(t *testing.T) {
{"literal (JS JSON.stringify / .NET)", jsonStringEscapeNoHTML(secret)},
} {
t.Run(tc.name, func(t *testing.T) {
entry := masker(&logrus.Entry{Context: t.Context(), Message: `{"t":"` + tc.form + `"}`})
entry := masker(t.Context(), logging.Record{Message: `{"t":"` + tc.form + `"}`})
assert.Contains(t, entry.Message, "***")
assert.NotContains(t, entry.Message, tc.form)
@@ -118,7 +121,7 @@ func TestValueMaskerJSONEscapesBothWays(t *testing.T) {
func TestValueMaskerHidesExtraMasks(t *testing.T) {
masker := valueMasker(false, (&Config{ExtraMasks: []string{"pr0xypw"}}).maskers())
entry := masker(&logrus.Entry{Context: t.Context(), Message: "proxy is http://user:pr0xypw@proxy:3128"})
entry := masker(t.Context(), logging.Record{Message: "proxy is http://user:pr0xypw@proxy:3128"})
assert.Equal(t, "proxy is http://user:***@proxy:3128", entry.Message)
}
@@ -137,7 +140,7 @@ func TestValueMaskerEncodedMasks(t *testing.T) {
{"encoded: " + base64.StdEncoding.EncodeToString([]byte("s3cr3t")), "encoded: ***"},
{"encoded: " + base64.StdEncoding.EncodeToString([]byte("first")), "encoded: ***"},
} {
entry := masker(&logrus.Entry{Context: WithMasks(t.Context(), &masks), Message: tc.line})
entry := masker(WithMasks(t.Context(), &masks), logging.Record{Message: tc.line})
assert.Equal(t, tc.want, entry.Message, tc.line)
}
}
@@ -154,7 +157,7 @@ func TestValueMaskerBase64Alignments(t *testing.T) {
for _, prefix := range []string{"x-access-token:", "user:", "ab:"} {
t.Run(prefix, func(t *testing.T) {
encoded := base64.StdEncoding.EncodeToString([]byte(prefix + secret))
entry := masker(&logrus.Entry{Context: t.Context(), Message: "Authorization: Basic " + encoded})
entry := masker(t.Context(), logging.Record{Message: "Authorization: Basic " + encoded})
assert.Contains(t, entry.Message, "***")
assert.NotEqual(t, "Authorization: Basic "+encoded, entry.Message)
@@ -172,7 +175,7 @@ func TestValueMaskerBase64Alignments(t *testing.T) {
func TestValueMaskerCachedReplacerSeesNewMasks(t *testing.T) {
masker := valueMasker(false, AppendSecretMaskers(nil, map[string]string{"TOKEN": "secret-token"}))
mask := func(masks *[]string, message string) string {
return masker(&logrus.Entry{Context: WithMasks(t.Context(), masks), Message: message}).Message
return masker(WithMasks(t.Context(), masks), logging.Record{Message: message}).Message
}
job := []string{"first mask"}
@@ -206,15 +209,61 @@ func TestAppendSecretMaskerSkipsUselessEncodings(t *testing.T) {
}
func TestJobLogFormatterDecodesCommandData(t *testing.T) {
logger := logrus.New()
logger.Out = io.Discard
format := func(message string) string {
out, err := (&jobLogFormatter{}).Format(&logrus.Entry{Logger: logger, Message: message, Data: logrus.Fields{rawOutputField: true}})
line, err := textFormatter{w: io.Discard}.format(logging.Record{Message: message, Attrs: []logging.Attr{{Key: rawOutputField, Value: true}}})
require.NoError(t, err)
return string(out)
return string(line)
}
assert.Contains(t, format("##[error]deploy 50%25 traffic"), "##[error]deploy 50% traffic")
// a plain line is not command data and keeps its literal escapes
assert.Contains(t, format("progress 50%25 done"), "progress 50%25 done")
}
type factoryFunc func() logging.Handler
func (f factoryFunc) WithJobLogger() logging.Handler { return f() }
func TestWithJobLoggerFansOutAndMasksOnlyOutput(t *testing.T) {
output, hook := logtest.NewWithLevel(logging.LevelTrace), logtest.NewWithLevel(logging.LevelTrace)
ctx := WithJobLoggerFactory(t.Context(), factoryFunc(func() logging.Handler { return output }))
ctx = common.WithLoggerHandler(ctx, hook)
level := logging.LevelInfo
cfg := &Config{Secrets: map[string]string{"TOKEN": "s3cr3t-value"}, JobLoggerLevel: &level}
ctx = WithJobLogger(ctx, "j1", "build", cfg, &[]string{}, nil)
ctx = withStepLogger(ctx, 2, "s2", "step two", "Main")
ctx = WithCompositeStepLogger(ctx, "inner")
common.Logger(ctx).Infof("token is %s", "s3cr3t-value")
common.Logger(ctx).Debugf("below the job logger level")
// the reporter masks for itself, so its record keeps the original
require.Len(t, hook.AllRecords(), 1)
assert.Equal(t, "token is s3cr3t-value", hook.LastRecord().Message)
require.Len(t, output.AllRecords(), 1)
assert.Equal(t, "token is ***", output.LastRecord().Message)
for _, want := range []struct {
key string
val any
}{{"job", "build"}, {"jobID", "j1"}, {"stepNumber", 2}, {"stage", "Main"}, {"stepID", []string{"s2", "inner"}}} {
got, ok := hook.LastRecord().Attr(want.key)
assert.True(t, ok, want.key)
assert.Equal(t, want.val, got, want.key)
}
}
func TestJobLogJSONHandler(t *testing.T) {
buf := &bytes.Buffer{}
logger := logging.New(newJobLogJSONHandler(buf)).With("job", "build", "msg", "clash").WithError(errors.New("boom"))
logger.Warnf("careful %d", 1)
var got map[string]any
require.NoError(t, json.Unmarshal(buf.Bytes(), &got))
assert.Equal(t, "careful 1", got["msg"])
assert.Equal(t, "warning", got["level"])
assert.Equal(t, "build", got["job"])
assert.Equal(t, "boom", got["error"])
assert.Equal(t, "clash", got["fields.msg"])
assert.NotEmpty(t, got["time"])
}

View File

@@ -223,8 +223,6 @@ func TestActionScriptPaths(t *testing.T) {
assert.Nil(t, actionScriptPaths("/a", escaping))
}
// The bundle has to be patched whatever state the shared action directory is in, because a
// concurrent job's prepare checks the action out again and resets it.
func TestPatchActionsAtTheContainerCopy(t *testing.T) {
copiedBundle := func(t *testing.T, noPatch bool) string {
t.Helper()
@@ -234,12 +232,13 @@ func TestPatchActionsAtTheContainerCopy(t *testing.T) {
Step: &model.Step{Uses: "owner/repo/sub@v1"},
remoteAction: &remoteAction{Org: "owner", Repo: "repo", Path: "sub", Ref: "v1"},
action: &model.Action{Runs: model.ActionRuns{Using: "node20", Main: "index.js"}},
actionDir: t.TempDir(),
RunContext: &RunContext{
Config: &Config{ActionCacheDir: t.TempDir(), NoActionPatch: noPatch},
JobContainer: cm,
},
}
script := filepath.Join(sar.actionDir(), "sub", "index.js")
script := filepath.Join(sar.actionDir, "sub", "index.js")
require.NoError(t, os.MkdirAll(filepath.Dir(script), 0o755))
require.NoError(t, os.WriteFile(script, []byte(gateTSC), 0o600))
@@ -250,7 +249,7 @@ func TestPatchActionsAtTheContainerCopy(t *testing.T) {
copied = string(body)
return nil
})
require.NoError(t, maybeCopyToActionDir(t.Context(), sar, sar.actionDir(), "sub", "/var/run/act/actions/repo/sub"))
require.NoError(t, maybeCopyToActionDir(t.Context(), sar, sar.actionDir, "sub", "/var/run/act/actions/repo/sub"))
source, err := os.ReadFile(script)
require.NoError(t, err)
assert.Equal(t, gateTSC, string(source), "the shared copy is left as it was found")

View File

@@ -10,7 +10,6 @@ import (
"net/url"
"path"
"regexp"
"strings"
"gitea.com/gitea/runner/act/common"
"gitea.com/gitea/runner/act/common/git"
@@ -27,7 +26,7 @@ func newLocalReusableWorkflowExecutor(rc *RunContext) common.Executor {
if !rc.Config.NoSkipCheckout {
// resolve the local workflow against the workspace root, not the process
// working directory, so it is found regardless of where the runner is invoked
return newReusableWorkflowExecutor(rc, rc.Config.Workdir, localWorkflow.Path)
return newReusableWorkflowExecutor(rc, rc.Config.Workdir, localWorkflow.Path, false)
}
uses := fmt.Sprintf("%s/%s@%s", rc.Config.PresetGitHubContext.Repository, localWorkflow.Path, rc.Config.PresetGitHubContext.Sha)
@@ -39,12 +38,9 @@ func newLocalReusableWorkflowExecutor(rc *RunContext) common.Executor {
workflowDir := fmt.Sprintf("%s/%s", rc.ActionCacheDir(), safeFilename(uses))
// If the repository is private, we need a token to clone it
token := rc.Config.GetToken()
return common.NewPipelineExecutor(
cloneRemoteReusableWorkflow(rc, remoteReusableWorkflow.CloneURL(), remoteReusableWorkflow.Ref, workflowDir, token),
newReusableWorkflowExecutor(rc, workflowDir, remoteReusableWorkflow.Path),
cloneRemoteReusableWorkflow(rc, remoteReusableWorkflow.CloneURL(), remoteReusableWorkflow.Ref, workflowDir),
newReusableWorkflowExecutor(rc, workflowDir, remoteReusableWorkflow.Path, true),
)
}
@@ -60,11 +56,9 @@ func newRemoteReusableWorkflowExecutor(rc *RunContext) common.Executor {
filename := fmt.Sprintf("%s/%s@%s", remoteReusableWorkflow.Owner, remoteReusableWorkflow.Repo, remoteReusableWorkflow.Ref)
workflowDir := fmt.Sprintf("%s/%s", rc.ActionCacheDir(), safeFilename(filename))
token := getGitCloneToken(rc.Config, remoteReusableWorkflow.CloneURL())
return common.NewPipelineExecutor(
cloneRemoteReusableWorkflow(rc, remoteReusableWorkflow.CloneURL(), remoteReusableWorkflow.Ref, workflowDir, token),
newReusableWorkflowExecutor(rc, workflowDir, remoteReusableWorkflow.Path),
cloneRemoteReusableWorkflow(rc, remoteReusableWorkflow.CloneURL(), remoteReusableWorkflow.Ref, workflowDir),
newReusableWorkflowExecutor(rc, workflowDir, remoteReusableWorkflow.Path, true),
)
}
@@ -76,28 +70,32 @@ func newRemoteReusableWorkflowExecutor(rc *RunContext) common.Executor {
// 2. Gitea has already full URL with rc.Config.GitHubInstance when calling newRemoteReusableWorkflow
//
// remoteReusableWorkflow.URL = rc.getGithubContext(ctx).ServerURL
func cloneRemoteReusableWorkflow(rc *RunContext, cloneURL, ref, targetDirectory, token string) common.Executor {
func cloneRemoteReusableWorkflow(rc *RunContext, cloneURL, ref, targetDirectory string) common.Executor {
return func(ctx context.Context) error {
interpolatedURL, err := rc.NewExpressionEvaluator(ctx).Interpolate(ctx, cloneURL)
if err != nil {
return fmt.Errorf("unable to interpolate the workflow clone URL: %w", err)
}
return git.NewGitCloneExecutor(git.NewGitCloneExecutorInput{
return git.NewGitCloneExecutor(rc.Config.withInstanceAuth(git.NewGitCloneExecutorInput{
URL: interpolatedURL,
Ref: ref,
Dir: targetDirectory,
Token: token,
OfflineMode: rc.Config.ActionOfflineMode,
Depth: rc.Config.ActionCloneDepth,
})(ctx)
}, rc.Config.GetToken()))(ctx)
}
}
func newReusableWorkflowExecutor(rc *RunContext, directory, workflow string) common.Executor {
func newReusableWorkflowExecutor(rc *RunContext, directory, workflow string, cached bool) common.Executor {
return func(ctx context.Context) error {
// Serialize workflow reads with cache updates.
planner, err := func() (model.WorkflowPlanner, error) {
defer git.AcquireCloneLock(directory)()
if cached {
unlock, err := git.AcquireCloneLock(ctx, directory)
if err != nil {
return nil, err
}
defer unlock()
}
return model.NewWorkflowPlanner(path.Join(directory, workflow), true)
}()
if err != nil {
@@ -141,11 +139,7 @@ type remoteReusableWorkflow struct {
}
func (r *remoteReusableWorkflow) CloneURL() string {
// In Gitea, r.URL always has the protocol prefix, we don't need to add extra prefix in this case.
if strings.HasPrefix(r.URL, "http://") || strings.HasPrefix(r.URL, "https://") {
return fmt.Sprintf("%s/%s/%s", r.URL, r.Owner, r.Repo)
}
return fmt.Sprintf("https://%s/%s/%s", r.URL, r.Owner, r.Repo)
return fmt.Sprintf("%s/%s/%s", withHTTPS(r.URL), r.Owner, r.Repo)
}
var absoluteReusableWorkflowURLRegex = regexp.MustCompile(`^(https?://.*)/([^/]+/[^/]+/\.[^/]+/workflows/[^@]+@.*)$`)
@@ -199,7 +193,7 @@ func setReusedWorkflowCallerResult(rc *RunContext, runner *runnerImpl) common.Ex
unlock := lockJob(rc.Run.Job())
rc.result(reusedWorkflowJobResult)
unlock()
common.Logger(ctx).WithField("jobResult", reusedWorkflowJobResult).Infof("Job %s", reusedWorkflowJobResultMessage)
common.Logger(ctx).With("jobResult", reusedWorkflowJobResult).Infof("Job %s", reusedWorkflowJobResultMessage)
}
}
@@ -208,13 +202,20 @@ func setReusedWorkflowCallerResult(rc *RunContext, runner *runnerImpl) common.Ex
}
// For Gitea
// getGitCloneToken returns GITEA_TOKEN when shouldCloneURLUseToken returns true,
// otherwise returns an empty string
func getGitCloneToken(conf *Config, cloneURL string) string {
if !shouldCloneURLUseToken(conf.GitHubInstance, conf.trustedActionInstance(), cloneURL) {
return ""
// withInstanceAuth passes instance credentials only to clones from this Gitea instance, and runner.insecure only to its registered address.
func (c Config) withInstanceAuth(input git.NewGitCloneExecutorInput, token string) git.NewGitCloneExecutorInput {
u, err := url.Parse(input.URL)
if err != nil || !onInstanceHost(u.Host, c.GitHubInstance, c.trustedActionInstance()) {
return input
}
return conf.GetToken()
input.ClientCertFile, input.ClientKeyFile = c.ClientCertFile, c.ClientKeyFile
if onInstanceHost(u.Host, c.GitHubInstance) {
input.InsecureSkipTLS, input.Transport = c.InsecureSkipTLS, c.InstanceTransport
}
if u.User == nil {
input.Token = token
}
return input
}
// For Gitea
@@ -228,26 +229,13 @@ func (c Config) trustedActionInstance() string {
}
// For Gitea
// shouldCloneURLUseToken returns true when the following conditions are met:
// 1. cloneURL's host matches this Gitea instance: either the registered instance
// (instanceURL) or, for DEFAULT_ACTIONS_URL=self on a different hostname, the
// self-hosted action instance (trustedActionInstance, "" when not trusted)
// 2. the cloneURL does not have basic auth embedded
func shouldCloneURLUseToken(instanceURL, trustedActionInstance, cloneURL string) bool {
u2, err := url.Parse(cloneURL)
if err != nil || u2.User != nil {
return false
}
for _, candidate := range []string{instanceURL, trustedActionInstance} {
// onInstanceHost reports whether host is that of one of the instance URLs, which may lack a scheme.
func onInstanceHost(host string, instanceURLs ...string) bool {
for _, candidate := range instanceURLs {
if candidate == "" {
continue
}
if !strings.HasPrefix(candidate, "http://") &&
!strings.HasPrefix(candidate, "https://") {
candidate = "https://" + candidate
}
if u1, err := url.Parse(candidate); err == nil && u1.Host == u2.Host {
if u, err := url.Parse(withHTTPS(candidate)); err == nil && u.Host == host {
return true
}
}

View File

@@ -5,7 +5,11 @@ package runner
import (
"context"
"io/fs"
"crypto/tls"
"crypto/x509"
"encoding/pem"
"net/http"
"net/http/httptest"
"os"
"os/exec"
"path/filepath"
@@ -59,7 +63,7 @@ func TestReusableWorkflowCachedBranchRefRefreshes(t *testing.T) {
}
cacheDir := t.TempDir()
require.NoError(t, cloneRemoteReusableWorkflow(rc, remoteDir, "master", cacheDir, "")(context.Background()))
require.NoError(t, cloneRemoteReusableWorkflow(rc, remoteDir, "master", cacheDir)(t.Context()))
got, err := os.ReadFile(filepath.Join(cacheDir, workflowPath))
require.NoError(t, err)
require.Equal(t, tmpl("v1"), string(got))
@@ -69,7 +73,7 @@ func TestReusableWorkflowCachedBranchRefRefreshes(t *testing.T) {
gitMust(t, workDir, "commit", "-am", "v2")
gitMust(t, workDir, "push", "origin", "master")
require.NoError(t, cloneRemoteReusableWorkflow(rc, remoteDir, "master", cacheDir, "")(context.Background()))
require.NoError(t, cloneRemoteReusableWorkflow(rc, remoteDir, "master", cacheDir)(t.Context()))
got, err = os.ReadFile(filepath.Join(cacheDir, workflowPath))
require.NoError(t, err)
require.Equal(t, tmpl("v2"), string(got), "cached workflow file must reflect the updated branch tip")
@@ -79,14 +83,16 @@ func TestNewReusableWorkflowExecutorHoldsCloneLock(t *testing.T) {
workflowDir := t.TempDir()
require.NoError(t, os.WriteFile(filepath.Join(workflowDir, "reusable.yml"), []byte(":"), 0o644))
unlockOnce := sync.OnceFunc(git.AcquireCloneLock(workflowDir))
unlock, err := git.AcquireCloneLock(t.Context(), workflowDir)
require.NoError(t, err)
unlockOnce := sync.OnceFunc(unlock)
defer unlockOnce()
rc := &RunContext{
Config: &Config{},
Run: &model.Run{Workflow: &model.Workflow{Jobs: map[string]*model.Job{}}},
}
exec := newReusableWorkflowExecutor(rc, workflowDir, "reusable.yml")
exec := newReusableWorkflowExecutor(rc, workflowDir, "reusable.yml", true)
done := make(chan error, 1)
go func() { done <- exec(context.Background()) }()
@@ -108,8 +114,9 @@ func TestNewReusableWorkflowExecutorHoldsCloneLock(t *testing.T) {
}
func TestNewLocalReusableWorkflowExecutorFindsSameRepositoryPaths(t *testing.T) {
workdir := t.TempDir()
workdir := filepath.Join(t.TempDir(), "workspace")
require.NoError(t, os.MkdirAll(filepath.Join(workdir, ".gitea", "workflows"), 0o755))
require.NoError(t, os.Mkdir(workdir+".lock", 0o755))
require.NoError(t, os.WriteFile(filepath.Join(workdir, ".gitea", "workflows", "reusable.yml"), []byte(":"), 0o644))
for _, uses := range []string{"./.gitea/workflows/reusable.yml", "$/.gitea/workflows/reusable.yml"} {
@@ -118,112 +125,87 @@ func TestNewLocalReusableWorkflowExecutorFindsSameRepositoryPaths(t *testing.T)
Run: &model.Run{JobID: "job", Workflow: &model.Workflow{Jobs: map[string]*model.Job{"job": {Uses: uses}}}},
}
err := newLocalReusableWorkflowExecutor(rc)(t.Context())
require.Error(t, err)
require.NotErrorIs(t, err, fs.ErrNotExist)
require.ErrorContains(t, err, "workflow is not valid")
}
}
func TestGetGitCloneTokenWithSchemalessGiteaInstance(t *testing.T) {
conf := &Config{
GitHubInstance: "gitea.example.net",
Secrets: map[string]string{
"GITEA_TOKEN": "token-value",
func TestCloneRemoteReusableWorkflowSendsInstanceTokenAndClientCert(t *testing.T) {
var gotCert bool
var gotToken string
server := httptest.NewUnstartedServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
gotCert = len(r.TLS.PeerCertificates) > 0
_, gotToken, _ = r.BasicAuth()
w.WriteHeader(http.StatusNotFound)
}))
server.TLS = &tls.Config{ClientAuth: tls.RequireAnyClientCert}
server.StartTLS()
defer server.Close()
pair := server.TLS.Certificates[0]
keyDER, err := x509.MarshalPKCS8PrivateKey(pair.PrivateKey)
require.NoError(t, err)
dir := t.TempDir()
certFile, keyFile := filepath.Join(dir, "client.crt"), filepath.Join(dir, "client.key")
require.NoError(t, os.WriteFile(certFile, pem.EncodeToMemory(&pem.Block{Type: "CERTIFICATE", Bytes: pair.Certificate[0]}), 0o600))
require.NoError(t, os.WriteFile(keyFile, pem.EncodeToMemory(&pem.Block{Type: "PRIVATE KEY", Bytes: keyDER}), 0o600))
rc := &RunContext{
Config: &Config{
GitHubInstance: server.URL,
Secrets: map[string]string{"GITEA_TOKEN": "token"},
InsecureSkipTLS: true,
ClientCertFile: certFile,
ClientKeyFile: keyFile,
},
Run: &model.Run{JobID: "j1", Workflow: &model.Workflow{Jobs: map[string]*model.Job{"j1": {}}}},
}
token := getGitCloneToken(conf, "https://gitea.example.net/actions/tools")
require.Equal(t, "token-value", token)
require.Error(t, cloneRemoteReusableWorkflow(rc, server.URL+"/owner/repo", "main", t.TempDir())(t.Context()))
require.True(t, gotCert)
require.Equal(t, "token", gotToken)
}
func TestGetGitCloneTokenSelfHostedActionsDifferentHost(t *testing.T) {
// The runner registered with one hostname while DEFAULT_ACTIONS_URL=self resolves
// actions against AppURL on a different hostname for the same instance.
conf := &Config{
GitHubInstance: "gitea.local",
DefaultActionInstance: "https://gitea.my-nas.lan",
DefaultActionInstanceIsSelfHosted: true,
Secrets: map[string]string{
"GITEA_TOKEN": "token-value",
},
}
token := getGitCloneToken(conf, "https://gitea.my-nas.lan/owner/action")
require.Equal(t, "token-value", token)
}
func TestShouldCloneURLUseToken(t *testing.T) {
tests := []struct {
name string
instanceURL string
trustedActionInstance string
cloneURL string
want bool
func TestInstanceAuthAndTransportOnlyApplyToTheInstance(t *testing.T) {
for _, tt := range []struct {
name, instanceURL, trustedActionInstance, cloneURL string
token, cert, insecure bool
}{
{
name: "same host with schemaless instance",
instanceURL: "gitea.example.net",
cloneURL: "https://gitea.example.net/actions/tools",
want: true,
},
{
name: "same host with schemaless instance and port",
instanceURL: "gitea.example.net:3000",
cloneURL: "https://gitea.example.net:3000/actions/tools",
want: true,
},
{
name: "different host",
instanceURL: "gitea.example.net",
cloneURL: "https://github.com/actions/tools",
want: false,
},
{
name: "embedded basic auth",
instanceURL: "gitea.example.net",
cloneURL: "https://user:pass@gitea.example.net/actions/tools",
want: false,
},
{
name: "invalid clone URL",
instanceURL: "gitea.example.net",
cloneURL: "://gitea.example.net/actions/tools",
want: false,
},
{
// self-hosted DEFAULT_ACTIONS_URL on a different hostname than the
// registered instance: the token must still be attached.
name: "self-hosted action instance on different host",
instanceURL: "gitea.local",
trustedActionInstance: "https://gitea.my-nas.lan",
cloneURL: "https://gitea.my-nas.lan/owner/action",
want: true,
},
{
// embedded basic auth must still be rejected even when the host matches
// the trusted action instance.
name: "self-hosted action instance with embedded basic auth",
instanceURL: "gitea.local",
trustedActionInstance: "https://gitea.my-nas.lan",
cloneURL: "https://user:pass@gitea.my-nas.lan/owner/action",
want: false,
},
{
// github.com / mirror hosts are never trusted: trustedActionInstance is
// empty in github mode, so an off-instance clone URL gets no token.
name: "github mode does not trust mirror host",
instanceURL: "gitea.local",
cloneURL: "https://mirror.example.com/owner/action",
want: false,
},
}
for _, tt := range tests {
{"same host with schemaless instance", "gitea.example.net", "", "https://gitea.example.net/actions/tools", true, true, true},
{"same host with schemaless instance and port", "gitea.example.net:3000", "", "https://gitea.example.net:3000/actions/tools", true, true, true},
{"different host", "gitea.example.net", "", "https://github.com/actions/tools", false, false, false},
{"embedded basic auth keeps its own credentials", "gitea.example.net", "", "https://user:pass@gitea.example.net/actions/tools", false, true, true},
{"invalid clone URL", "gitea.example.net", "", "://gitea.example.net/actions/tools", false, false, false},
{"self-hosted action instance on different host skips no verification", "gitea.local", "https://gitea.my-nas.lan", "https://gitea.my-nas.lan/owner/action", true, true, false},
{"self-hosted action instance with embedded basic auth", "gitea.local", "https://gitea.my-nas.lan", "https://user:pass@gitea.my-nas.lan/owner/action", false, true, false},
{"github mode does not trust mirror host", "gitea.local", "", "https://mirror.example.com/owner/action", false, false, false},
} {
t.Run(tt.name, func(t *testing.T) {
require.Equal(t, tt.want, shouldCloneURLUseToken(tt.instanceURL, tt.trustedActionInstance, tt.cloneURL))
conf := Config{
GitHubInstance: tt.instanceURL,
DefaultActionInstance: tt.trustedActionInstance,
DefaultActionInstanceIsSelfHosted: tt.trustedActionInstance != "",
InsecureSkipTLS: true,
ClientCertFile: "/cert",
ClientKeyFile: "/key",
InstanceTransport: &http.Transport{},
}
want := git.NewGitCloneExecutorInput{URL: tt.cloneURL, InsecureSkipTLS: tt.insecure}
if tt.insecure {
want.Transport = conf.InstanceTransport
}
if tt.token {
want.Token = "token"
}
if tt.cert {
want.ClientCertFile, want.ClientKeyFile = conf.ClientCertFile, conf.ClientKeyFile
}
require.Equal(t, want, conf.withInstanceAuth(git.NewGitCloneExecutorInput{URL: tt.cloneURL}, "token"))
})
}
transport := &http.Transport{}
conf := Config{GitHubInstance: "https://gitea.local", InstanceTransport: transport}
require.Same(t, transport, conf.instanceTransportFor("https://gitea.local/api/actions_pipeline/"))
require.Nil(t, conf.instanceTransportFor("https://gitea.local.example/api/actions_pipeline/"))
require.Nil(t, conf.instanceTransportFor("https://other.example/x#/api/actions_pipeline/"))
}
func gitMust(t *testing.T, dir string, args ...string) {

View File

@@ -7,6 +7,7 @@ package runner
import (
"archive/tar"
"bufio"
"cmp"
"context"
"crypto/rand"
"crypto/sha256"
@@ -21,7 +22,9 @@ import (
"path/filepath"
"regexp"
"runtime"
"runtime/debug"
"slices"
"strconv"
"strings"
"sync"
"time"
@@ -64,7 +67,9 @@ type RunContext struct {
ExprEval *expressionEvaluator
JobContainer container.ExecutionsEnvironment
serviceContainers []*serviceContainer
containerSpec model.ContainerSpec // container:, resolved once with platformImage, zero without a job container
kubernetes bool
containerSpec model.ContainerSpec // container:, resolved once with platformImage
containerEnv yaml.Node // container.env of the kept containerSpec, decoded once the job env resolves
JobName string
ActionPath string
Parent *RunContext
@@ -78,6 +83,11 @@ type RunContext struct {
jobRunDefaults model.RunDefaults // defaults.run, resolved once at job setup as GitHub does
platformImage string // container.image or the runs-on pick, resolved once by isEnabled
jobIndex, jobTotal int // strategy.job-index and job-total, total 0 when unknown
jobEnv map[string]string // workflow and job env as evaluated for the container and service specs
dockerActionImages map[*model.Step]string // Dockerfile action images built for this job, removed when it ends
actionDownloads map[string]actionDownload // remote action repositories by clone URL and ref, fetched once per job
// summaryFileInitialized tracks which per-step summary files (workflow/step-summary-N.md)
// have already been created on the JobContainer. The runner sets up file-command files
// via JobContainer.Copy at the start of every phase, which truncates them — fine for
@@ -113,6 +123,7 @@ type serviceContainer struct {
name string
image string
container container.ExecutionsEnvironment
created bool
logsDumped bool
info *container.Info // last poll, the source of the `job.services` entry
}
@@ -251,6 +262,9 @@ const sharedToolCacheVolume = "act-toolcache" // mounted only when the tool cach
func (rc *RunContext) validVolumes() []string {
name := rc.jobContainerName()
volumes := slices.Clone(rc.Config.ValidVolumes)
if hostEnv, ok := rc.JobContainer.(*container.HostEnvironment); ok {
volumes = append(volumes, hostEnv.Path, hostEnv.GetActPath())
}
if rc.Config.SharedToolCache {
volumes = append(volumes, sharedToolCacheVolume)
}
@@ -334,12 +348,12 @@ func splitVolumes(specs []string) ([]string, map[string]string, map[string]bool)
}
// Returns the binds and mounts for the container, resolving paths as appopriate
func (rc *RunContext) GetBindsAndMounts() ([]string, map[string]string) {
func (rc *RunContext) GetBindsAndMounts(ctx context.Context) ([]string, map[string]string) {
name := rc.jobContainerName()
ext := container.LinuxContainerEnvironmentExtensions{}
// the runner's own mounts below yield to the targets the job claims
binds, mounts, claimed := splitVolumes(rc.containerSpec.Volumes)
binds, mounts, claimed := splitVolumes(container.SanitizeBinds(ctx, rc.validVolumes(), rc.containerSpec.Volumes))
if rc.containerDaemonSocket() != "-" && !claimed["/var/run/docker.sock"] {
binds = append(binds, rc.jobDockerSocket()+":/var/run/docker.sock")
@@ -349,6 +363,9 @@ func (rc *RunContext) GetBindsAndMounts() ([]string, map[string]string) {
mounts[sharedToolCacheVolume] = toolCache
}
}
if hostEnv, ok := rc.JobContainer.(*container.HostEnvironment); ok {
binds = append(binds, hostEnv.Path+":"+hostEnv.Path, hostEnv.GetActPath()+":"+hostEnv.GetActPath())
}
mounts[name+"-env"] = ext.GetActPath() // runner-internal, never overridable
if workdir := ext.ToContainerPath(rc.Config.Workdir); !claimed[workdir] {
@@ -415,11 +432,7 @@ func (rc *RunContext) startHostEnvironment() common.Executor {
AllocatePTY: rc.Config.AllocatePTY,
}
rc.cleanUpJobContainer = rc.JobContainer.Remove()
for k, v := range rc.getRunnerContext(ctx) {
if v, ok := v.(string); ok {
rc.Env["RUNNER_"+strings.ToUpper(k)] = v
}
}
rc.setRunnerContextEnv(ctx)
for _, env := range os.Environ() {
if k, v, ok := strings.Cut(env, "="); ok {
// don't override
@@ -429,16 +442,70 @@ func (rc *RunContext) startHostEnvironment() common.Executor {
}
}
return rc.copyWorkflowFiles()(ctx)
}
}
func (rc *RunContext) setRunnerContextEnv(ctx context.Context) {
for k, v := range rc.getRunnerContext(ctx) {
if v, ok := v.(string); ok {
rc.Env["RUNNER_"+strings.ToUpper(k)] = v
}
}
}
var errKubernetesDocker = errors.New("docker actions and docker:// steps are not supported on Kubernetes")
func (rc *RunContext) startKubernetesPod() common.Executor {
return func(ctx context.Context) error {
if rc.containerSpec.Options != "" || len(rc.containerSpec.Volumes) > 0 || rc.containerSpec.Credentials != nil {
return errors.New("container options, volumes and credentials are not supported on Kubernetes")
}
var options container.KubernetesOptions
if rc.Config.KubernetesPicker != nil {
options = rc.Config.KubernetesPicker(rc.runsOnPlatformNames(ctx))
}
options.MaxLifetime, options.ForcePull = rc.Config.ContainerMaxLifetime, rc.Config.ForcePull
pod, err := container.NewKubernetesPod(options)
if err != nil {
return err
}
services, err := rc.jobServices(ctx, "")
if err != nil {
return err
}
for _, serviceID := range slices.Sorted(maps0.Keys(services)) {
input := services[serviceID]
if input.Username != "" || len(input.Binds) > 0 || len(input.Mounts) > 0 {
return fmt.Errorf("service %s: volumes and credentials are not supported on Kubernetes", serviceID)
}
rc.serviceContainers = append(rc.serviceContainers, &serviceContainer{name: serviceID, image: input.Image, container: pod.ServiceContainer(serviceID, input)})
}
logWriter := rc.commandLogWriter(ctx)
rc.JobContainer = pod.JobContainer(&container.NewContainerInput{
Entrypoint: []string{"tail", "-f", "/dev/null"},
WorkingDir: (&container.LinuxContainerEnvironmentExtensions{}).ToContainerPath(rc.Config.Workdir),
Image: rc.platformImage,
Name: rc.jobContainerName(),
Env: []string{"LANG=C.UTF-8"},
Stdout: logWriter,
Stderr: logWriter,
})
rc.cleanUpJobContainer = func(ctx context.Context) error {
rc.printServiceLogs(ctx)
return rc.JobContainer.Remove()(ctx)
}
defer printStartJobContainerGroup(ctx, rc.platformImage, rc.jobContainerName(), "")()
return common.NewPipelineExecutor(
rc.JobContainer.Copy(rc.JobContainer.GetActPath()+"/", &container.FileEntry{
Name: "workflow/event.json",
Mode: 0o644,
Body: rc.EventJSON,
}, &container.FileEntry{
Name: "workflow/envs.txt",
Mode: 0o666,
Body: "",
}),
rc.JobContainer.Create(nil, nil),
rc.startServiceContainers(),
rc.JobContainer.Start(false),
func(ctx context.Context) error { rc.setRunnerContextEnv(ctx); return nil },
rc.captureJobContainerInfo(),
rc.copyWorkflowFiles(),
rc.reportUnstartedServices(),
rc.waitForServiceContainers(),
)(ctx)
}
}
@@ -446,7 +513,7 @@ func (rc *RunContext) startHostEnvironment() common.Executor {
// printStartJobContainerGroup mirrors actions/runner's "Starting job container"
// section: emit the group header and summary, return a closer for ::endgroup::.
func printStartJobContainerGroup(ctx context.Context, image, name, network string) func() {
rawLogger := common.Logger(ctx).WithField(rawOutputField, true)
rawLogger := common.Logger(ctx).With(rawOutputField, true)
rawLogger.Infof("::group::Starting job container")
rawLogger.Infof("image: %s", image)
rawLogger.Infof("name: %s", name)
@@ -468,7 +535,6 @@ func WithJobVolumeCleanup(ctx context.Context, deferCleanup func(common.Executor
func (rc *RunContext) startJobContainer() common.Executor {
return func(ctx context.Context) error {
rc.deferVolumeCleanup, _ = ctx.Value(jobVolumeCleanupKey{}).(func(common.Executor))
logger := common.Logger(ctx)
image := rc.platformImage
logWriter := rc.commandLogWriter(ctx)
@@ -481,6 +547,7 @@ func (rc *RunContext) startJobContainer() common.Executor {
// For gitea, to support --volumes-from <container_name_or_id> in options.
// We need to set the container name to the environment variable.
rc.Env["JOB_CONTAINER_NAME"] = name
delete(rc.jobEnv, "JOB_CONTAINER_NAME") // the runtime name wins over a workflow env of the same name
envList := make([]string, 0)
@@ -493,75 +560,13 @@ func (rc *RunContext) startJobContainer() common.Executor {
networkName, createAndDeleteNetwork := rc.networkNameForGitea()
rc.cleanUpJobContainer = rc.cleanupJobResources(networkName, createAndDeleteNetwork, false)
var services map[string]*model.ContainerSpec
if err := model.DecodeEvaluated("job services", rc.Run.Job().RawServices, rc.ExprEval.shared(ctx).EvaluateYamlNode, &services); err != nil {
services, err := rc.jobServices(ctx, networkName)
if err != nil {
return err
}
for serviceID, spec := range services {
if spec == nil {
return fmt.Errorf("service %s has no container definition", serviceID)
}
// GitHub compatibility: skip services whose image evaluates to an
// empty string, enabling conditional services via expressions
if spec.Image == "" {
logger.Infof("The service '%s' will not be started because the container definition has an empty image.", serviceID)
continue
}
envs := make([]string, 0, len(spec.Env)+len(rc.Config.ProxyEnv))
// a service reaches the internet the way the job does; its own env still wins
for k, v := range mergeMaps(rc.Config.ProxyEnv, spec.Env) {
envs = append(envs, fmt.Sprintf("%s=%s", k, v))
}
// keep these local: reusing username/password would overwrite the
// credentials the job container is pulled with further down
serviceUsername, servicePassword, err := registryCredentials(spec.Credentials, "")
if err != nil {
return fmt.Errorf("failed to handle service %s credentials: %w", serviceID, err)
}
cmd := spec.Cmd
if len(cmd) == 0 && spec.Command != "" {
if cmd, err = shellquote.Split(spec.Command); err != nil {
return fmt.Errorf("failed to parse service %s command: %w", serviceID, err)
}
}
var entrypoint []string
if spec.Entrypoint != "" {
entrypoint = []string{spec.Entrypoint}
}
serviceBinds, serviceMounts, _ := splitVolumes(spec.Volumes)
exposedPorts, portBindings, err := nat.ParsePortSpecs(spec.Ports)
if err != nil {
return fmt.Errorf("failed to parse service %s ports: %w", serviceID, err)
}
serviceContainerName := createContainerName(rc.jobContainerName(), serviceID)
c := newContainer(&container.NewContainerInput{
Name: serviceContainerName,
Image: spec.Image,
Username: serviceUsername,
Password: servicePassword,
Entrypoint: entrypoint,
Cmd: cmd,
Env: envs,
Mounts: serviceMounts,
Binds: serviceBinds,
Stdout: logWriter,
Stderr: logWriter,
Privileged: rc.Config.Privileged,
UsernsMode: rc.Config.UsernsMode,
Platform: rc.Config.ContainerArchitecture,
AutoRemove: false, // so a dead service's log survives, cleanupJobResources removes it
WorkflowOptions: spec.Options,
NetworkMode: networkName,
NetworkAliases: []string{serviceID},
ExposedPorts: exposedPorts,
PortBindings: portBindings,
ValidVolumes: rc.Config.ValidVolumes, // not validVolumes(), a service gets no docker socket
AllocatePTY: rc.Config.AllocatePTY,
})
rc.serviceContainers = append(rc.serviceContainers, &serviceContainer{name: serviceID, image: spec.Image, container: c})
for _, serviceID := range slices.Sorted(maps0.Keys(services)) {
input := services[serviceID]
rc.serviceContainers = append(rc.serviceContainers, &serviceContainer{name: serviceID, image: input.Image, container: newContainer(input)})
}
// For Gitea, `jobContainerNetwork` should be the same as `networkName`
@@ -570,7 +575,7 @@ func (rc *RunContext) startJobContainer() common.Executor {
ext := container.LinuxContainerEnvironmentExtensions{}
containerInput := &container.NewContainerInput{
Cmd: nil,
Entrypoint: []string{"/bin/sleep", fmt.Sprint(rc.Config.ContainerMaxLifetime.Round(time.Second).Seconds())},
Entrypoint: []string{"sleep", strconv.FormatInt(int64(rc.Config.ContainerMaxLifetime.Round(time.Second).Seconds()), 10)},
WorkingDir: ext.ToContainerPath(rc.Config.Workdir),
Image: image,
Username: username,
@@ -602,7 +607,7 @@ func (rc *RunContext) startJobContainer() common.Executor {
return err
}
rc.startDockerProxy(ctx)
containerInput.Binds, containerInput.Mounts = rc.GetBindsAndMounts()
containerInput.Binds, containerInput.Mounts = rc.GetBindsAndMounts(ctx)
containerInput.ValidVolumes = rc.validVolumes()
rc.jobNetworkName = networkName
@@ -618,57 +623,152 @@ func (rc *RunContext) startJobContainer() common.Executor {
rc.JobContainer.Create(rc.Config.ContainerCapAdd, rc.Config.ContainerCapDrop),
rc.JobContainer.Start(false),
rc.captureJobContainerInfo(),
rc.JobContainer.Copy(rc.JobContainer.GetActPath()+"/", &container.FileEntry{
Name: "workflow/event.json",
Mode: 0o644,
Body: rc.EventJSON,
}, &container.FileEntry{
Name: "workflow/envs.txt",
Mode: 0o666,
Body: "",
}),
rc.copyWorkflowFiles(),
rc.waitForServiceContainers(),
)(ctx)
}
}
func (rc *RunContext) jobServices(ctx context.Context, networkName string) (map[string]*container.NewContainerInput, error) {
var services map[string]*model.ContainerSpec
if err := model.DecodeEvaluated("job services", rc.Run.Job().RawServices, rc.NewExpressionEvaluatorWithEnv(ctx, mergeMaps(rc.GetEnv(), rc.jobEnv)).shared(ctx).EvaluateYamlNode, &services); err != nil {
return nil, err
}
inputs := map[string]*container.NewContainerInput{}
for _, serviceID := range slices.Sorted(maps0.Keys(services)) {
spec := services[serviceID]
if spec == nil {
return nil, fmt.Errorf("service %s has no container definition", serviceID)
}
spec.Image = strings.TrimPrefix(spec.Image, "docker://")
// GitHub compatibility: skip services whose image evaluates to an
// empty string, enabling conditional services via expressions
if spec.Image == "" {
common.Logger(ctx).Infof("The service '%s' will not be started because the container definition has an empty image.", serviceID)
continue
}
envs := make([]string, 0, len(spec.Env)+len(rc.Config.ProxyEnv))
// a service reaches the internet the way the job does; its own env still wins
for k, v := range mergeMaps(rc.Config.ProxyEnv, spec.Env) {
envs = append(envs, fmt.Sprintf("%s=%s", k, v))
}
serviceUsername, servicePassword, err := registryCredentials(spec.Credentials, "")
if err != nil {
return nil, fmt.Errorf("failed to handle service %s credentials: %w", serviceID, err)
}
cmd := spec.Cmd
if len(cmd) == 0 && spec.Command != "" {
if cmd, err = shellquote.Split(spec.Command); err != nil {
return nil, fmt.Errorf("failed to parse service %s command: %w", serviceID, err)
}
}
var entrypoint []string
if spec.Entrypoint != "" {
entrypoint = []string{spec.Entrypoint}
}
serviceBinds, serviceMounts, _ := splitVolumes(spec.Volumes)
exposedPorts, portBindings, err := nat.ParsePortSpecs(spec.Ports)
if err != nil {
return nil, fmt.Errorf("failed to parse service %s ports: %w", serviceID, err)
}
serviceLog := common.NewLineWriter(rawLogLine(ctx)) // raw, so a service's output cannot set the job's env, outputs or state
inputs[serviceID] = &container.NewContainerInput{
Name: createContainerName(rc.jobContainerName(), serviceID),
Image: spec.Image,
Username: serviceUsername,
Password: servicePassword,
Entrypoint: entrypoint,
Cmd: cmd,
Env: envs,
Mounts: serviceMounts,
Binds: serviceBinds,
Stdout: serviceLog,
Stderr: serviceLog,
Privileged: rc.Config.Privileged,
UsernsMode: rc.Config.UsernsMode,
Platform: rc.Config.ContainerArchitecture,
AutoRemove: false, // so a dead service's log survives, cleanupJobResources removes it
WorkflowOptions: spec.Options,
NetworkMode: networkName,
NetworkAliases: []string{serviceID},
ExposedPorts: exposedPorts,
PortBindings: portBindings,
ValidVolumes: rc.Config.ValidVolumes, // not validVolumes(), a service gets no docker socket
AllocatePTY: rc.Config.AllocatePTY,
}
}
return inputs, nil
}
func (rc *RunContext) commandLogWriter(ctx context.Context) io.Writer {
rawLogger := common.Logger(ctx).WithField(rawOutputField, true)
return common.NewLineWriter(rc.commandHandler(ctx), func(line string) bool {
return common.NewLineWriter(rc.commandHandler(ctx), rawLogLine(ctx))
}
func rawLogLine(ctx context.Context) common.LineHandler {
rawLogger := common.Logger(ctx).With(rawOutputField, true)
return func(line string) bool {
rawLogger.Infof("%s", line)
return true
})
}
}
func (rc *RunContext) cleanupJobResources(networkName string, createAndDeleteNetwork, preclean bool) common.Executor {
return func(ctx context.Context) error {
logger := common.Logger(ctx)
rc.printServiceLogs(ctx)
errs := []error{rc.closeDockerProxy(ctx)}
var containers []common.Executor
if rc.JobContainer != nil {
errs = append(errs, rc.JobContainer.Remove()(ctx))
containers = append(containers, rc.JobContainer.Remove())
}
if len(rc.serviceContainers) > 0 {
logger.Infof("Cleaning up services for job %s", rc.JobName)
errs = append(errs, rc.stopServiceContainers()(ctx))
containers = append(containers, rc.stopServiceContainers())
}
if !common.Dryrun(ctx) && (preclean || rc.hadDockerProxy) {
errs = append(errs, container.RemoveDockerJobResources(ctx, rc.jobContainerName()))
errs = append(errs, runAll(ctx, containers))
ctx, cancel := context.WithTimeout(context.WithoutCancel(ctx), jobCleanupTimeout)
defer cancel()
if !common.Dryrun(ctx) && !rc.Config.NoSweep && (preclean || rc.hadDockerProxy) {
errs = append(errs, container.RemoveDockerJobResources(ctx, rc.jobContainerName())) // after the containers, whose endpoints would block removing the job networks
}
var rest []common.Executor
if preclean || rc.deferVolumeCleanup == nil {
errs = append(errs, rc.cleanupJobVolumes(ctx))
rest = append(rest, rc.cleanupJobVolumes)
} else {
rc.deferVolumeCleanup(rc.cleanupJobVolumes)
}
if createAndDeleteNetwork {
logger.Infof("Cleaning up network for job %s, and network name is: %s", rc.JobName, networkName)
errs = append(errs, container.NewDockerNetworkDisconnectExecutor(networkName, rc.Config.CacheContainer).
IfBool(rc.Config.CacheContainer != "")(ctx))
errs = append(errs, container.NewDockerNetworkRemoveExecutor(networkName)(ctx))
rest = append(rest, func(ctx context.Context) error {
return errors.Join(
container.NewDockerNetworkDisconnectExecutor(networkName, rc.Config.CacheContainer).IfBool(rc.Config.CacheContainer != "")(ctx),
container.NewDockerNetworkRemoveExecutor(networkName)(ctx))
})
}
return errors.Join(errs...)
return errors.Join(append(errs, runAll(ctx, rest))...)
}
}
func runAll(ctx context.Context, executors []common.Executor) error {
errs := make([]error, len(executors))
var wg sync.WaitGroup
for i, executor := range executors {
wg.Go(func() {
defer func() {
if recovered := recover(); recovered != nil {
errs[i] = fmt.Errorf("panic: %v\n%s", recovered, debug.Stack())
}
}()
errs[i] = executor(ctx)
})
}
wg.Wait()
return errors.Join(errs...)
}
func (rc *RunContext) createJobVolumes(ctx context.Context, mounts map[string]string) error {
runnerUUID := rc.Config.ContainerNetworkCreateOptions.RunnerUUID
if runnerUUID == "" || common.Dryrun(ctx) {
@@ -773,6 +873,14 @@ func (rc *RunContext) stopJobContainer() common.Executor {
}
}
func (rc *RunContext) removeDockerActionImages(ctx context.Context) {
for _, image := range rc.dockerActionImages {
if _, err := container.RemoveImage(ctx, image, true, false); err != nil {
common.Logger(ctx).Warnf("failed to remove action image %s: %v", image, err)
}
}
}
func (rc *RunContext) pullServicesImages(forcePull bool) common.Executor {
return func(ctx context.Context) error {
execs := []common.Executor{}
@@ -789,6 +897,7 @@ func (rc *RunContext) startServiceContainers() common.Executor {
for _, svc := range rc.serviceContainers {
execs = append(execs, common.NewPipelineExecutor(
svc.container.Create(rc.Config.ContainerCapAdd, rc.Config.ContainerCapDrop),
func(context.Context) error { svc.created = true; return nil },
svc.container.Start(false),
))
}
@@ -874,7 +983,7 @@ func (rc *RunContext) waitForServiceContainers() common.Executor {
// waitUntilHealthy waits on the healthcheck alone, so a container that declares none is
// ready at once and one that exited is left to the steps that need it.
func (svc *serviceContainer) waitUntilHealthy(ctx context.Context, timeout time.Duration) error {
rawLogger := common.Logger(ctx).WithField(rawOutputField, true)
rawLogger := common.Logger(ctx).With(rawOutputField, true)
loggedStarting := false
ctx, cancel := context.WithTimeout(ctx, timeout)
@@ -927,6 +1036,26 @@ func (svc *serviceContainer) dumpLogs(ctx context.Context) {
}
}
const serviceLogsTimeout = 15 * time.Second // cleanup gets one minute in total
func (rc *RunContext) printServiceLogs(ctx context.Context) {
ctx, cancel := context.WithTimeout(ctx, serviceLogsTimeout)
defer cancel()
rawLogger := common.Logger(ctx).With(rawOutputField, true)
for _, svc := range rc.serviceContainers {
if !svc.created || svc.logsDumped {
continue
}
rawLogger.Infof("::group::Print service container logs: %s", EscapeCommandData(svc.name))
svc.dumpLogs(ctx)
rawLogger.Infof("::endgroup::")
if ctx.Err() != nil {
common.Logger(ctx).Warnf("Timed out printing the service logs after %s", serviceLogsTimeout)
return
}
}
}
// inspect also records the state for the `job.services` context.
func (svc *serviceContainer) inspect(ctx context.Context) (*container.Info, error) {
info, err := svc.container.Inspect(ctx)
@@ -944,6 +1073,13 @@ func (svc *serviceContainer) healthOutputSuffix() string {
return ": " + svc.info.HealthOutput
}
func (rc *RunContext) copyWorkflowFiles() common.Executor {
return rc.JobContainer.Copy(rc.JobContainer.GetActPath()+"/",
&container.FileEntry{Name: "workflow/event.json", Mode: 0o644, Body: rc.EventJSON},
&container.FileEntry{Name: "workflow/envs.txt", Mode: 0o666},
)
}
// captureJobContainerInfo is a convenience: failing to describe the container must not
// fail the job.
func (rc *RunContext) captureJobContainerInfo() common.Executor {
@@ -1012,9 +1148,12 @@ func (rc *RunContext) interpolateOutputs() common.Executor {
func (rc *RunContext) startContainer() common.Executor {
return func(ctx context.Context) error {
var err error
if rc.IsHostEnv() {
switch {
case rc.kubernetes:
err = rc.startKubernetesPod()(ctx)
case rc.IsHostEnv():
err = rc.startHostEnvironment()(ctx)
} else {
default:
err = rc.startJobContainer()(ctx)
}
if err != nil {
@@ -1039,7 +1178,7 @@ func (rc *RunContext) cleanupFailedStart(ctx context.Context) {
}
func (rc *RunContext) IsHostEnv() bool {
return strings.EqualFold(rc.platformImage, "-self-hosted")
return rc.containerSpec.Image == "" && strings.EqualFold(rc.platformImage, "-self-hosted")
}
func (rc *RunContext) stopContainer() common.Executor {
@@ -1123,10 +1262,6 @@ func (rc *RunContext) Executor() (common.Executor, error) {
}
func (rc *RunContext) runsOnImage(ctx context.Context) (string, error) {
if rc.Run.Job().RunsOn() == nil {
common.Logger(ctx).Errorf("'runs-on' key not defined in %s", rc.String())
}
runsOn := rc.Run.Job().RunsOn()
for i, v := range runsOn {
var err error
@@ -1159,20 +1294,38 @@ func (rc *RunContext) runsOnPlatformNames(ctx context.Context) []string {
return model.RunsOnFromNode(rawRunsOn)
}
// resolvePlatformImage evaluates the job's container once for every consumer, ignoring one without an image as GitHub does.
// resolvePlatformImage evaluates the job's container once for every consumer, skipping one whose image is empty as GitHub does.
func (rc *RunContext) resolvePlatformImage(ctx context.Context) error {
withoutEnv, _ := splitContainerEnv(rc.Run.Job().RawContainer)
var spec *model.ContainerSpec
if err := model.DecodeEvaluated("container", withoutEnv, rc.ExprEval.shared(ctx).EvaluateYamlNode, &spec); err != nil {
rc.jobEnv, _ = evaluateJobEnv(ctx, rc) // a broken env fails the job at setup instead
withoutEnv, env := splitContainerEnv(rc.Run.Job().RawContainer)
var spec model.ContainerSpec
if err := model.DecodeEvaluated("container", withoutEnv, rc.NewExpressionEvaluatorWithEnv(ctx, rc.jobEnv).shared(ctx).EvaluateYamlNode, &spec); err != nil {
return err
}
if spec != nil && spec.Image != "" {
rc.containerSpec, rc.platformImage = *spec, spec.Image
return nil
spec.Image = strings.TrimPrefix(spec.Image, "docker://")
if spec.Image != "" || imagelessContainer(withoutEnv) {
rc.containerSpec, rc.containerEnv = spec, env
}
image, err := rc.runsOnImage(ctx)
rc.platformImage = image
return err
if err != nil {
return err
}
image, rc.kubernetes = strings.CutPrefix(image, "kubernetes://")
rc.platformImage = cmp.Or(rc.containerSpec.Image, image)
return nil
}
// imagelessContainer reports a container: mapping without an image key, which configures the runs-on image.
func imagelessContainer(container yaml.Node) bool {
if container.Kind != yaml.MappingNode {
return false
}
for i := 0; i < len(container.Content); i += 2 {
if container.Content[i].Value == "image" {
return false
}
}
return true
}
// splitContainerEnv takes `env` out of a container mapping, since env reads the job env that resolves after the job environment starts.
@@ -1211,7 +1364,7 @@ func (rc *RunContext) isEnabled(ctx context.Context) (bool, error) {
rc.caller.setReusedWorkflowJobResult(rc.Run.JobID, "skipped")
return false, nil
}
l.WithField("jobResult", "skipped").Debugf("Skipping job '%s' due to '%s'", job.Name, job.If.Value)
l.With("jobResult", "skipped").Debugf("Skipping job '%s' due to '%s'", job.Name, job.If.Value)
return false, nil
}
@@ -1414,14 +1567,8 @@ func (rc *RunContext) getGithubContext(ctx context.Context) *model.GithubContext
merged.RunnerPerflog, merged.RunnerTrackingID = ghc.RunnerPerflog, ghc.RunnerTrackingID
*ghc = merged
instance := rc.Config.GitHubInstance
if !strings.HasPrefix(instance, "http://") &&
!strings.HasPrefix(instance, "https://") {
instance = "https://" + instance
}
ghc.ServerURL = instance
ghc.APIURL = instance + "/api/v1" // the version of Gitea is v1
ghc.GraphQLURL = "" // Gitea doesn't support graphql
rc.setInstanceURLs(ghc)
ghc.GraphQLURL = "" // Gitea doesn't support graphql
return ghc
}
}
@@ -1435,7 +1582,7 @@ func (rc *RunContext) getGithubContext(ctx context.Context) *model.GithubContext
ghc.SetBaseAndHeadRef()
repoPath := rc.Config.Workdir
ghcontext.SetRepositoryAndOwner(ctx, ghc, rc.Config.GitHubInstance, repoPath)
ghcontext.SetRepositoryAndOwner(ctx, ghc, repoPath)
if ghc.Ref == "" {
ghcontext.SetRef(ctx, ghc, repoPath)
}
@@ -1445,27 +1592,7 @@ func (rc *RunContext) getGithubContext(ctx context.Context) *model.GithubContext
ghc.SetRefTypeAndName()
// defaults
ghc.ServerURL = "https://github.com"
ghc.APIURL = "https://api.github.com"
ghc.GraphQLURL = "https://api.github.com/graphql"
// per GHES
if rc.Config.GitHubInstance != "github.com" {
ghc.ServerURL = "https://" + rc.Config.GitHubInstance
ghc.APIURL = fmt.Sprintf("https://%s/api/v3", rc.Config.GitHubInstance)
ghc.GraphQLURL = fmt.Sprintf("https://%s/api/graphql", rc.Config.GitHubInstance)
}
{ // Adapt to Gitea
instance := rc.Config.GitHubInstance
if !strings.HasPrefix(instance, "http://") &&
!strings.HasPrefix(instance, "https://") {
instance = "https://" + instance
}
ghc.ServerURL = instance
ghc.APIURL = instance + "/api/v1" // the version of Gitea is v1
ghc.GraphQLURL = "" // Gitea doesn't support graphql
}
rc.setInstanceURLs(ghc)
// allow to be overridden by user
if rc.Config.Env["GITHUB_SERVER_URL"] != "" {
@@ -1481,6 +1608,20 @@ func (rc *RunContext) getGithubContext(ctx context.Context) *model.GithubContext
return ghc
}
func (rc *RunContext) setInstanceURLs(ghc *model.GithubContext) {
if instance := strings.TrimRight(rc.Config.GitHubInstance, "/"); instance != "" {
ghc.ServerURL = withHTTPS(instance)
ghc.APIURL = ghc.ServerURL + "/api/v1"
}
}
func withHTTPS(u string) string {
if strings.HasPrefix(u, "http://") || strings.HasPrefix(u, "https://") {
return u
}
return "https://" + u
}
func isLocalCheckout(ghc *model.GithubContext, step *model.Step) bool {
if step.Type() == model.StepTypeInvalid {
// This will be errored out by the executor later, we need this here to avoid a null panic though
@@ -1530,20 +1671,16 @@ func (rc *RunContext) withGithubEnv(ctx context.Context, github *model.GithubCon
env["GITHUB_HEAD_REF"] = github.HeadRef
env["GITHUB_SERVER_URL"] = github.ServerURL
env["GITHUB_API_URL"] = github.APIURL
env["GITHUB_GRAPHQL_URL"] = github.GraphQLURL
{ // Adapt to Gitea
instance := rc.Config.GitHubInstance
if !strings.HasPrefix(instance, "http://") &&
!strings.HasPrefix(instance, "https://") {
instance = "https://" + instance
for _, name := range []string{"GITHUB_SERVER_URL", "GITHUB_API_URL"} {
if env[name] == "" {
delete(env, name)
}
env["GITHUB_SERVER_URL"] = instance
env["GITHUB_API_URL"] = instance + "/api/v1" // the version of Gitea is v1
env["GITHUB_GRAPHQL_URL"] = "" // Gitea doesn't support graphql
env["GITHUB_RUN_ATTEMPT"] = github.RunAttempt
}
env["GITHUB_GRAPHQL_URL"] = ""
env["GITHUB_RUN_ATTEMPT"] = github.RunAttempt
env["RUNNER_NAME"] = rc.Config.RunnerName
env["RUNNER_ENVIRONMENT"] = "self-hosted"
if workspace := parentDir(github.Workspace); workspace != "" {
@@ -1577,9 +1714,7 @@ func parentDir(p string) string {
// still names a release when the label is a rolling one such as ubuntu-latest.
func (rc *RunContext) imageOS(ctx context.Context) string {
if rc.Run.Job().RunsOn() == nil {
// A composite action runs on a synthetic job, and resolving its image would only
// log that runs-on is missing.
return ""
return "" // a composite action runs on a synthetic job without runs-on
}
if imageOS := imageOSFromImage(rc.platformImage); imageOS != "" {
return imageOS

View File

@@ -20,18 +20,20 @@ import (
"runtime"
"slices"
"strings"
"sync"
"testing"
"testing/synctest"
"time"
"gitea.com/gitea/runner/act/common"
"gitea.com/gitea/runner/act/common/logging"
"gitea.com/gitea/runner/act/common/logging/logtest"
"gitea.com/gitea/runner/act/container"
"gitea.dev/actionslib/pkg/exprparser"
"gitea.dev/actionslib/pkg/model"
"github.com/docker/cli/cli/compose/loader"
"github.com/moby/moby/api/types/volume"
log "github.com/sirupsen/logrus"
assert "github.com/stretchr/testify/assert"
"github.com/stretchr/testify/mock"
require "github.com/stretchr/testify/require"
@@ -82,7 +84,7 @@ func TestRunContext_EvalBool(t *testing.T) {
},
},
}
rc.ExprEval = rc.NewExpressionEvaluator(context.Background())
rc.ExprEval = rc.NewExpressionEvaluator(t.Context())
tables := []struct {
in string
@@ -175,7 +177,7 @@ func TestRunContext_EvalBool(t *testing.T) {
for _, table := range tables {
t.Run(table.in, func(t *testing.T) {
assertObject := assert.New(t)
b, err := EvalBool(context.Background(), rc.ExprEval, table.in, exprparser.DefaultStatusCheckSuccess)
b, err := EvalBool(t.Context(), rc.ExprEval, table.in, exprparser.DefaultStatusCheckSuccess)
if table.wantErr {
assertObject.Error(err) //nolint:testifylint // pre-existing issue from nektos/act
}
@@ -250,7 +252,6 @@ func startJobContainerInputs(t *testing.T, workflowYAML string, cfg *Config) []*
rc := &RunContext{
Name: "test",
Config: cfg,
Env: map[string]string{},
Run: &model.Run{
JobID: "job",
Workflow: workflow,
@@ -342,12 +343,12 @@ jobs:
require.Nil(t, redis.Cmd)
}
func TestStartJobContainerEvaluatesContainersOnce(t *testing.T) {
func TestStartJobContainerBuildsEvaluatedJobAndServiceInputs(t *testing.T) {
inputs := startJobContainerInputs(t, `
jobs:
job:
container: ${{ fromJSON('{"image":"node:20","options":"--label ${{ github.job }}","credentials":{"username":"user","password":"${{ vars.TITLE }}"}}') }}
services: ${{ fromJSON('{"redis":{"image":"redis:latest","volumes":["data:/data"],"env":{"TITLE":"${{ github.job }}"},"credentials":{"username":"user","password":"${{ github.job }}"},"entrypoint":"/entry.sh","command":"redis-server --port 6380"}}') }}
container: ${{ fromJSON('{"image":"docker://node:20","options":"--label ${{ github.job }}","credentials":{"username":"user","password":"${{ vars.TITLE }}"}}') }}
services: ${{ fromJSON('{"redis":{"image":"docker://redis:latest","volumes":["data:/data"],"env":{"TITLE":"${{ github.job }}"},"credentials":{"username":"user","password":"${{ github.job }}"},"entrypoint":"/entry.sh","command":"redis-server --port 6380"}}') }}
`, &Config{})
redis, job := inputs[0], inputs[1]
@@ -358,8 +359,32 @@ jobs:
require.Equal(t, []string{"/entry.sh"}, redis.Entrypoint)
require.Equal(t, []string{"redis-server", "--port", "6380"}, redis.Cmd)
require.Equal(t, "node:20", job.Image)
require.Equal(t, "sleep", job.Entrypoint[0])
require.Equal(t, "--label ${{ github.job }}", job.WorkflowOptions)
require.Equal(t, "${{ vars.TITLE }}", job.Password)
inputs = startJobContainerInputs(t, `
env:
REG_USER: ${{ vars.REG_USER }}
TEXT: ${{ vars.TEXT }}
JOB_CONTAINER_NAME: from-workflow
jobs:
job:
container:
image: node:20
credentials: {username: "${{ env.REG_USER }}", password: pass}
services:
redis:
image: redis:latest
credentials: {username: "${{ env.REG_USER }}", password: pass}
env: {TEXT: "${{ env.TEXT }}"}
options: --volumes-from ${{ env.JOB_CONTAINER_NAME }}
`, &Config{Vars: map[string]string{"REG_USER": "user", "TEXT": "${{ 'literal' }}"}})
redis, job = inputs[0], inputs[1]
require.Equal(t, "user", job.Username)
require.Equal(t, "user", redis.Username)
require.Equal(t, []string{"TEXT=${{ 'literal' }}"}, redis.Env)
require.Equal(t, "--volumes-from "+job.Name, redis.WorkflowOptions)
}
// Only the workflow's options may be stripped later, so the two sources have to reach the
@@ -412,17 +437,14 @@ jobs:
steps: []
`, &Config{ProxyEnv: map[string]string{"http_proxy": "http://proxy:3128", "no_proxy": "internal.example"}})
env := map[string][]string{}
for _, in := range inputs {
env[in.Image] = in.Env
}
db, redis := inputs[0], inputs[1]
require.Contains(t, env["redis:latest"], "http_proxy=http://proxy:3128")
require.Contains(t, env["redis:latest"], "no_proxy=internal.example")
require.Contains(t, redis.Env, "http_proxy=http://proxy:3128")
require.Contains(t, redis.Env, "no_proxy=internal.example")
// the service's own env wins over what the runner injected, without dropping the rest
require.Contains(t, env["postgres:latest"], "no_proxy=db-only.example")
require.NotContains(t, env["postgres:latest"], "no_proxy=internal.example")
require.Contains(t, env["postgres:latest"], "http_proxy=http://proxy:3128")
require.Contains(t, db.Env, "no_proxy=db-only.example")
require.NotContains(t, db.Env, "no_proxy=internal.example")
require.Contains(t, db.Env, "http_proxy=http://proxy:3128")
}
// act builds Dockerfile actions through the API, which does not pre-populate the proxy
@@ -483,7 +505,7 @@ func TestRunContext_GetBindsAndMounts(t *testing.T) {
config := testcase.rc.Config
config.Workdir = testcase.name
config.BindWorkdir = bindWorkDir
gotbind, gotmount := rctemplate.GetBindsAndMounts()
gotbind, gotmount := rctemplate.GetBindsAndMounts(t.Context())
// Name binds/mounts are either/or
if config.BindWorkdir {
@@ -533,7 +555,8 @@ func TestRunContext_GetBindsAndMounts(t *testing.T) {
},
},
Config: &Config{
BindWorkdir: false,
BindWorkdir: false,
ValidVolumes: []string{"**"},
Secrets: map[string]string{
"MAME": "/host/mame/roms",
},
@@ -541,10 +564,10 @@ func TestRunContext_GetBindsAndMounts(t *testing.T) {
}
rc.Run.JobID = "job1"
rc.Run.Workflow.Jobs = map[string]*model.Job{"job1": job}
rc.ExprEval = rc.NewExpressionEvaluator(context.Background())
require.NoError(t, rc.resolvePlatformImage(context.Background()))
rc.ExprEval = rc.NewExpressionEvaluator(t.Context())
require.NoError(t, rc.resolvePlatformImage(t.Context()))
gotbind, gotmount := rc.GetBindsAndMounts()
gotbind, gotmount := rc.GetBindsAndMounts(t.Context())
assert.Contains(t, gotbind, "/host/mame/roms:/root/.mame/roms:ro")
assert.NotContains(t, gotbind, "${{ secrets.MAME }}")
assert.NotContains(t, gotmount, "${{ secrets.MAME }}")
@@ -562,12 +585,13 @@ func TestRunContext_GetBindsAndMounts(t *testing.T) {
Config: &Config{
BindWorkdir: false,
SharedToolCache: true, // so OverridesToolCache has a mount to displace
ValidVolumes: []string{"**"},
},
containerSpec: model.ContainerSpec{Volumes: testcase.volumes},
}
rc.Run.JobID = "job1"
gotbind, gotmount := rc.GetBindsAndMounts()
gotbind, gotmount := rc.GetBindsAndMounts(t.Context())
if len(testcase.wantbind) > 0 {
assert.Contains(t, gotbind, testcase.wantbind)
@@ -602,11 +626,12 @@ func TestRunContext_GetBindsAndMounts(t *testing.T) {
Config: &Config{},
}
_, gotmount := rc.GetBindsAndMounts()
_, gotmount := rc.GetBindsAndMounts(t.Context())
assert.NotContains(t, gotmount, sharedToolCacheVolume)
rc.Config.SharedToolCache = true
_, gotmount = rc.GetBindsAndMounts()
rc.containerSpec.Volumes = []string{"/unapproved:" + container.DefaultToolCache}
_, gotmount = rc.GetBindsAndMounts(t.Context())
assert.Equal(t, container.DefaultToolCache, gotmount[sharedToolCacheVolume])
})
@@ -616,18 +641,18 @@ func TestRunContext_GetBindsAndMounts(t *testing.T) {
}
rc := &RunContext{
Run: &model.Run{JobID: "job1", Workflow: &model.Workflow{Jobs: map[string]*model.Job{"job1": {}}}},
Config: &Config{BindWorkdir: true, Workdir: "/workspace/1/owner/repo", PresetGitHubContext: &model.GithubContext{}},
Config: &Config{BindWorkdir: true, Workdir: "/workspace/1/owner/repo", PresetGitHubContext: &model.GithubContext{}, ValidVolumes: []string{"claimed"}},
}
gotbind, _ := rc.GetBindsAndMounts()
gotbind, _ := rc.GetBindsAndMounts(t.Context())
assert.True(t, slices.ContainsFunc(gotbind, func(bind string) bool { return strings.HasPrefix(bind, "/workspace/1:/workspace/1") }), gotbind)
rc.Config.BindWorkdir = false
_, gotmount := rc.GetBindsAndMounts()
_, gotmount := rc.GetBindsAndMounts(t.Context())
assert.Equal(t, "/workspace/1", gotmount[rc.jobContainerName()])
rc.containerSpec.Volumes = []string{"claimed:/workspace/1"}
_, gotmount = rc.GetBindsAndMounts()
_, gotmount = rc.GetBindsAndMounts(t.Context())
assert.Equal(t, "/workspace/1/owner/repo", gotmount[rc.jobContainerName()])
})
}
@@ -660,6 +685,15 @@ func TestRunContextValidVolumes(t *testing.T) {
assert.Contains(t, rc.validVolumes(), rc.Config.Workdir)
rc.Config.PresetGitHubContext = &model.GithubContext{}
assert.Contains(t, rc.validVolumes(), filepath.FromSlash("/workspace/1"))
hostEnv := &container.HostEnvironment{Path: t.TempDir(), ActPath: t.TempDir()}
rc.JobContainer = hostEnv
rc.Config.BindWorkdir = false
rc.Config.SharedToolCache = true
binds, mounts := rc.GetBindsAndMounts(t.Context())
assert.Subset(t, binds, []string{hostEnv.Path + ":" + hostEnv.Path, hostEnv.GetActPath() + ":" + hostEnv.GetActPath()})
assert.Contains(t, mounts, sharedToolCacheVolume)
assert.Equal(t, binds, container.SanitizeBinds(t.Context(), rc.validVolumes(), binds))
}
func TestCleanupJobResourcesCleansServicesWithoutJobContainer(t *testing.T) {
@@ -728,22 +762,49 @@ func TestCleanupJobVolumesReapsAbandonedDeferredCleanup(t *testing.T) {
assert.ElementsMatch(t, []string{"became-active", "remove-failed", "foreign", "fresh", "unknown-age"}, slices.Collect(maps.Keys(volumes)))
}
func TestCleanupJobResourcesContinuesAfterFailure(t *testing.T) {
func TestRemoveDockerActionImagesByTag(t *testing.T) {
var removed []string
fakeDockerDaemon(t, func(writer http.ResponseWriter, request *http.Request) {
if request.Method == http.MethodDelete {
removed = append(removed, strings.TrimPrefix(request.URL.Path, "/v1.47/images/"))
_, _ = io.WriteString(writer, "[]")
return
}
_, _ = io.WriteString(writer, `{"Id":"sha256:shared"}`)
})
(&RunContext{dockerActionImages: map[*model.Step]string{{}: "act-action-dockeraction:latest-job"}}).removeDockerActionImages(t.Context())
assert.Equal(t, []string{"act-action-dockeraction:latest-job"}, removed)
}
func TestCleanupJobResources(t *testing.T) {
t.Setenv("TMPDIR", "/tmp")
proxyDir := t.TempDir()
for _, name := range []string{"synchronous", "proxy", "closed proxy", "deferred", "preclean deferred", "canceled"} {
for _, name := range []string{"synchronous", "proxy", "closed proxy", "kept proxy", "deferred", "preclean deferred", "canceled proxy"} {
t.Run(name, func(t *testing.T) {
proxy, preclean, deferred := strings.HasSuffix(name, "proxy"), strings.HasPrefix(name, "preclean"), strings.HasSuffix(name, "deferred")
removesLabelled := (proxy || preclean) && name != "kept proxy"
if proxy && runtime.GOOS == "windows" {
t.Skip("Unix socket ownership is unavailable on Windows")
}
jobError, removeError, closeError := errors.New("remove job"), errors.New("remove service"), errors.New("close service")
job, service := &containerMock{}, &containerMock{}
job.On("Remove").Return(func(context.Context) error { return jobError }).Once()
service.On("Remove").Return(func(context.Context) error { return removeError }).Once()
removeAlongside := func(started, other chan struct{}, failure error) func(context.Context) error {
return func(context.Context) error {
close(started)
select {
case <-other:
case <-time.After(10 * time.Second):
t.Error("the containers are removed one after the other instead of alongside each other")
}
return failure
}
}
jobRemoving, serviceRemoving := make(chan struct{}), make(chan struct{})
job.On("Remove").Return(removeAlongside(jobRemoving, serviceRemoving, jobError)).Once()
service.On("Remove").Return(removeAlongside(serviceRemoving, jobRemoving, removeError)).Once()
service.On("Close").Return(func(context.Context) error { return closeError }).Once()
rc := &RunContext{
Config: &Config{CacheContainer: "cache-container"},
Config: &Config{CacheContainer: "cache-container", NoSweep: name == "kept proxy"},
Run: &model.Run{Workflow: &model.Workflow{Name: "wf"}, JobID: "job"},
JobContainer: job,
serviceContainers: []*serviceContainer{{name: "svc", container: service}},
@@ -777,7 +838,7 @@ func TestCleanupJobResourcesContinuesAfterFailure(t *testing.T) {
}
ctx, cancel := context.WithCancel(t.Context())
defer cancel()
if name == "canceled" {
if name == "canceled proxy" {
cancel()
}
err := rc.cleanupJobResources("job-network", true, preclean)(ctx)
@@ -791,16 +852,12 @@ func TestCleanupJobResourcesContinuesAfterFailure(t *testing.T) {
for _, failure := range []error{jobError, removeError, closeError} {
require.ErrorIs(t, err, failure)
}
if name == "canceled" {
require.ErrorIs(t, err, context.Canceled)
} else {
for _, operation := range []string{"DELETE /volumes/" + rc.jobContainerName(), "DELETE /volumes/" + rc.jobContainerName() + "-env", "POST /networks/job-network/disconnect", "GET /networks"} {
require.ErrorContains(t, err, operation)
}
assert.Equal(t, 2, volumeRemovals)
for _, operation := range []string{"DELETE /volumes/" + rc.jobContainerName(), "DELETE /volumes/" + rc.jobContainerName() + "-env", "POST /networks/job-network/disconnect", "GET /networks"} {
require.ErrorContains(t, err, operation)
}
assert.Equal(t, proxy || preclean, strings.Contains(err.Error(), "labelled GET /containers/json"))
if proxy || preclean {
assert.Equal(t, 2, volumeRemovals)
assert.Equal(t, removesLabelled, strings.Contains(err.Error(), "labelled GET /containers/json"))
if removesLabelled {
require.ErrorContains(t, err, "labelled GET /networks")
require.ErrorContains(t, err, "labelled GET /volumes")
}
@@ -816,7 +873,7 @@ func TestInterpolateOutputsIsPerMatrixComboKeepingNeedsOutputs(t *testing.T) {
needed := &model.Job{Outputs: map[string]string{"o": "from-gitea"}}
run := &model.Run{JobID: "j", Workflow: &model.Workflow{Name: "w", Jobs: map[string]*model.Job{"j": job, "needed": needed}}}
r := &runnerImpl{config: &Config{}}
ctx := context.Background()
ctx := t.Context()
_, err := r.newRunContext(ctx, &model.Run{JobID: "needed", Workflow: run.Workflow}, nil)
require.NoError(t, err)
@@ -851,7 +908,7 @@ func TestInterpolateOutputsFromExpression(t *testing.T) {
}
func TestGetGitHubContext(t *testing.T) {
log.SetLevel(log.DebugLevel)
logging.SetLevel(logging.LevelDebug)
cwd, err := os.Getwd()
assert.NoError(t, err) //nolint:testifylint // pre-existing issue from nektos/act
@@ -867,6 +924,7 @@ func TestGetGitHubContext(t *testing.T) {
Run: &model.Run{
Workflow: &model.Workflow{
Name: "GitHubContextTest",
Jobs: map[string]*model.Job{"job1": {}},
},
},
Name: "GitHubContextTest",
@@ -878,9 +936,9 @@ func TestGetGitHubContext(t *testing.T) {
}
rc.Run.JobID = "job1"
ghc := rc.getGithubContext(context.Background())
ghc := rc.getGithubContext(t.Context())
log.Debugf("%v", ghc)
logging.Debugf("%v", ghc)
actor := "nektos/act"
if a := os.Getenv("ACT_ACTOR"); a != "" {
@@ -908,10 +966,40 @@ func TestGetGitHubContext(t *testing.T) {
assert.Equal(t, ghc.Job, "job1") //nolint:testifylint // pre-existing issue from nektos/act
rc.Config.PresetGitHubContext = &model.GithubContext{Actor: "preset-actor", TriggeringActor: "triggerer", Job: "preset-job"}
ghc = rc.getGithubContext(context.Background())
ghc = rc.getGithubContext(t.Context())
assert.Equal(t, "preset-actor", ghc.Actor)
assert.Equal(t, "triggerer", ghc.TriggeringActor)
assert.Equal(t, "job1", ghc.Job)
preset := &model.GithubContext{ServerURL: "https://preset.test", APIURL: "https://preset.test/api/v1"}
for _, testcase := range []struct {
name, instance, serverURL, apiURL string
preset *model.GithubContext
env map[string]string
}{
{name: "unknown instance"},
{name: "bare host with trailing slash", instance: "gitea.test/", serverURL: "https://gitea.test", apiURL: "https://gitea.test/api/v1"},
{name: "scheme port and base path", instance: "http://gitea.test:3000/gitea/", serverURL: "http://gitea.test:3000/gitea", apiURL: "http://gitea.test:3000/gitea/api/v1"},
{name: "daemon instance replaces preset endpoints", instance: "gitea.test", preset: preset, serverURL: "https://gitea.test", apiURL: "https://gitea.test/api/v1"},
{name: "daemon keeps preset endpoints without instance", preset: preset, serverURL: "https://preset.test", apiURL: "https://preset.test/api/v1"},
{name: "environment overrides", env: map[string]string{"GITHUB_SERVER_URL": "https://override.test", "GITHUB_API_URL": "https://override.test/api"}, serverURL: "https://override.test", apiURL: "https://override.test/api"},
} {
t.Run(testcase.name, func(t *testing.T) {
rc.Config.GitHubInstance = testcase.instance
rc.Config.PresetGitHubContext = testcase.preset
rc.Config.Env = map[string]string{"GITHUB_REPOSITORY": "owner/repo", "GITHUB_REF": "refs/heads/main", "SHA_REF": "1234567890"}
maps.Copy(rc.Config.Env, testcase.env)
github := rc.getGithubContext(t.Context())
assert.Equal(t, testcase.serverURL, github.ServerURL)
assert.Equal(t, testcase.apiURL, github.APIURL)
env := map[string]string{}
rc.withGithubEnv(t.Context(), github, env)
_, exported := env["GITHUB_SERVER_URL"]
assert.Equal(t, testcase.serverURL != "", exported)
assert.Equal(t, testcase.serverURL, env["GITHUB_SERVER_URL"])
assert.Equal(t, testcase.apiURL, env["GITHUB_API_URL"])
})
}
}
func TestGetGithubContextRef(t *testing.T) {
@@ -948,7 +1036,7 @@ func TestGetGithubContextRef(t *testing.T) {
},
}
ghc := rc.getGithubContext(context.Background())
ghc := rc.getGithubContext(t.Context())
assert.Equal(t, data.ref, ghc.Ref)
})
@@ -958,7 +1046,6 @@ func TestGetGithubContextRef(t *testing.T) {
func createIfTestRunContext(jobs map[string]*model.Job) *RunContext {
rc := &RunContext{
Config: &Config{Workdir: ".", PlatformPicker: func([]string) string { return "ubuntu-latest" }},
Env: map[string]string{},
Run: &model.Run{
JobID: "job1",
Workflow: &model.Workflow{
@@ -982,62 +1069,65 @@ func createJob(t *testing.T, input, result string) *model.Job {
}
func TestRunContextRunsOnPlatformNames(t *testing.T) {
log.SetLevel(log.DebugLevel)
logging.SetLevel(logging.LevelDebug)
assertObject := assert.New(t)
rc := createIfTestRunContext(map[string]*model.Job{
"job1": createJob(t, `runs-on: ubuntu-latest`, ""),
})
assertObject.Equal([]string{"ubuntu-latest"}, rc.runsOnPlatformNames(context.Background()))
assertObject.Equal([]string{"ubuntu-latest"}, rc.runsOnPlatformNames(t.Context()))
rc = createIfTestRunContext(map[string]*model.Job{
"job1": createJob(t, `runs-on: ${{ 'ubuntu-latest' }}`, ""),
})
assertObject.Equal([]string{"ubuntu-latest"}, rc.runsOnPlatformNames(context.Background()))
assertObject.Equal([]string{"ubuntu-latest"}, rc.runsOnPlatformNames(t.Context()))
rc = createIfTestRunContext(map[string]*model.Job{
"job1": createJob(t, `runs-on: [self-hosted, my-runner]`, ""),
})
assertObject.Equal([]string{"self-hosted", "my-runner"}, rc.runsOnPlatformNames(context.Background()))
assertObject.Equal([]string{"self-hosted", "my-runner"}, rc.runsOnPlatformNames(t.Context()))
rc = createIfTestRunContext(map[string]*model.Job{
"job1": createJob(t, `runs-on: [self-hosted, "${{ 'my-runner' }}"]`, ""),
})
assertObject.Equal([]string{"self-hosted", "my-runner"}, rc.runsOnPlatformNames(context.Background()))
assertObject.Equal([]string{"self-hosted", "my-runner"}, rc.runsOnPlatformNames(t.Context()))
rc = createIfTestRunContext(map[string]*model.Job{
"job1": createJob(t, `runs-on: ${{ fromJSON('["ubuntu-latest"]') }}`, ""),
})
assertObject.Equal([]string{"ubuntu-latest"}, rc.runsOnPlatformNames(context.Background()))
assertObject.Equal([]string{"ubuntu-latest"}, rc.runsOnPlatformNames(t.Context()))
// test missing / invalid runs-on
rc = createIfTestRunContext(map[string]*model.Job{
"job1": createJob(t, `name: something`, ""),
})
assertObject.Equal([]string{}, rc.runsOnPlatformNames(context.Background()))
assertObject.Equal([]string{}, rc.runsOnPlatformNames(t.Context()))
rc = createIfTestRunContext(map[string]*model.Job{
"job1": createJob(t, `runs-on:
mapping: value`, ""),
})
assertObject.Equal([]string{}, rc.runsOnPlatformNames(context.Background()))
assertObject.Equal([]string{}, rc.runsOnPlatformNames(t.Context()))
rc = createIfTestRunContext(map[string]*model.Job{
"job1": createJob(t, `runs-on: ${{ invalid expression }}`, ""),
})
assertObject.Equal([]string{}, rc.runsOnPlatformNames(context.Background()))
assertObject.Equal([]string{}, rc.runsOnPlatformNames(t.Context()))
}
func TestRunContextIsEnabled(t *testing.T) {
log.SetLevel(log.DebugLevel)
logging.SetLevel(logging.LevelDebug)
assertObject := assert.New(t)
// success()
rc := createIfTestRunContext(map[string]*model.Job{
"job1": createJob(t, `runs-on: ubuntu-latest
env:
BROKEN: ${{ fromJSON('invalid') }}
if: success()`, ""),
})
assertObject.True(rc.isEnabled(context.Background()))
assertObject.True(rc.isEnabled(t.Context()))
require.Error(t, evaluateJobEnvAndDefaults(t.Context(), rc))
rc = createIfTestRunContext(map[string]*model.Job{
"job1": createJob(t, `runs-on: ubuntu-latest`, "failure"),
@@ -1046,7 +1136,7 @@ needs: [job1]
if: success()`, ""),
})
rc.Run.JobID = "job2"
assertObject.False(rc.isEnabled(context.Background()))
assertObject.False(rc.isEnabled(t.Context()))
rc = createIfTestRunContext(map[string]*model.Job{
"job1": createJob(t, `runs-on: ubuntu-latest`, "success"),
@@ -1055,7 +1145,7 @@ needs: [job1]
if: success()`, ""),
})
rc.Run.JobID = "job2"
assertObject.True(rc.isEnabled(context.Background()))
assertObject.True(rc.isEnabled(t.Context()))
rc = createIfTestRunContext(map[string]*model.Job{
"job1": createJob(t, `runs-on: ubuntu-latest`, "failure"),
@@ -1063,14 +1153,14 @@ if: success()`, ""),
if: success()`, ""),
})
rc.Run.JobID = "job2"
assertObject.True(rc.isEnabled(context.Background()))
assertObject.True(rc.isEnabled(t.Context()))
// failure()
rc = createIfTestRunContext(map[string]*model.Job{
"job1": createJob(t, `runs-on: ubuntu-latest
if: failure()`, ""),
})
assertObject.False(rc.isEnabled(context.Background()))
assertObject.False(rc.isEnabled(t.Context()))
rc = createIfTestRunContext(map[string]*model.Job{
"job1": createJob(t, `runs-on: ubuntu-latest`, "failure"),
@@ -1079,7 +1169,7 @@ needs: [job1]
if: failure()`, ""),
})
rc.Run.JobID = "job2"
assertObject.True(rc.isEnabled(context.Background()))
assertObject.True(rc.isEnabled(t.Context()))
rc = createIfTestRunContext(map[string]*model.Job{
"job1": createJob(t, `runs-on: ubuntu-latest`, "success"),
@@ -1088,7 +1178,7 @@ needs: [job1]
if: failure()`, ""),
})
rc.Run.JobID = "job2"
assertObject.False(rc.isEnabled(context.Background()))
assertObject.False(rc.isEnabled(t.Context()))
rc = createIfTestRunContext(map[string]*model.Job{
"job1": createJob(t, `runs-on: ubuntu-latest`, "failure"),
@@ -1096,14 +1186,14 @@ if: failure()`, ""),
if: failure()`, ""),
})
rc.Run.JobID = "job2"
assertObject.False(rc.isEnabled(context.Background()))
assertObject.False(rc.isEnabled(t.Context()))
// always()
rc = createIfTestRunContext(map[string]*model.Job{
"job1": createJob(t, `runs-on: ubuntu-latest
if: always()`, ""),
})
assertObject.True(rc.isEnabled(context.Background()))
assertObject.True(rc.isEnabled(t.Context()))
rc = createIfTestRunContext(map[string]*model.Job{
"job1": createJob(t, `runs-on: ubuntu-latest`, "failure"),
@@ -1112,7 +1202,7 @@ needs: [job1]
if: always()`, ""),
})
rc.Run.JobID = "job2"
assertObject.True(rc.isEnabled(context.Background()))
assertObject.True(rc.isEnabled(t.Context()))
rc = createIfTestRunContext(map[string]*model.Job{
"job1": createJob(t, `runs-on: ubuntu-latest`, "success"),
@@ -1121,7 +1211,7 @@ needs: [job1]
if: always()`, ""),
})
rc.Run.JobID = "job2"
assertObject.True(rc.isEnabled(context.Background()))
assertObject.True(rc.isEnabled(t.Context()))
rc = createIfTestRunContext(map[string]*model.Job{
"job1": createJob(t, `runs-on: ubuntu-latest`, "success"),
@@ -1129,23 +1219,23 @@ if: always()`, ""),
if: always()`, ""),
})
rc.Run.JobID = "job2"
assertObject.True(rc.isEnabled(context.Background()))
assertObject.True(rc.isEnabled(t.Context()))
rc = createIfTestRunContext(map[string]*model.Job{
"job1": createJob(t, `uses: ./.github/workflows/reusable.yml`, ""),
})
assertObject.True(rc.isEnabled(context.Background()))
assertObject.True(rc.isEnabled(t.Context()))
rc = createIfTestRunContext(map[string]*model.Job{
"job1": createJob(t, `uses: ./.github/workflows/reusable.yml
if: false`, ""),
})
assertObject.False(rc.isEnabled(context.Background()))
assertObject.False(rc.isEnabled(t.Context()))
rc = createIfTestRunContext(map[string]*model.Job{
"job1": createJob(t, `if: ${{ fromJSON('not-json') }}`, ""),
})
enabled, err := rc.isEnabled(context.Background())
enabled, err := rc.isEnabled(t.Context())
require.NoError(t, err)
assertObject.False(enabled)
}
@@ -1225,12 +1315,9 @@ func TestCreateContainerNameBoundedForLongMatrixInput(t *testing.T) {
func TestPrintStartJobContainerGroupGolden(t *testing.T) {
buf := &bytes.Buffer{}
logger := log.New()
logger.SetOutput(buf)
logger.SetLevel(log.InfoLevel)
logger.SetFormatter(&jobLogFormatter{color: cyan})
entry := logger.WithFields(log.Fields{"job": "j1"})
ctx := common.WithLogger(context.Background(), entry)
logger := logging.New(logging.NewLevelHandler(newJobLogHandler(buf, cyan), logging.LevelInfo))
entry := logger.With("job", "j1")
ctx := common.WithLogger(t.Context(), entry)
printStartJobContainerGroup(ctx, "node:20", "GITEA-WORKFLOW-build-JOB-test", "gitea-runner-network")()
@@ -1272,17 +1359,21 @@ func TestRunContext_cleanupFailedStart(t *testing.T) {
}
for _, testcase := range []struct {
name, health string
pullError error
name, health string
pullError, startError error
}{
{name: "healthy", health: container.HealthHealthy},
{name: "unhealthy", health: container.HealthUnhealthy},
{name: "pull failure", pullError: errors.New("pull failed")},
{name: "start failure", startError: errors.New("start failed")},
} {
t.Run(testcase.name, func(t *testing.T) {
var operations []string
var operationsMu sync.Mutex
record := func(operation string, failure error) func(context.Context) error {
return func(context.Context) error {
operationsMu.Lock()
defer operationsMu.Unlock()
operations = append(operations, operation)
return failure
}
@@ -1291,7 +1382,7 @@ func TestRunContext_cleanupFailedStart(t *testing.T) {
for name, instance := range map[string]*containerMock{"job": job, "service": service} {
instance.On("Pull", true).Return(record(name+".Pull", map[string]error{"job": testcase.pullError}[name]))
instance.On("Create", mock.Anything, mock.Anything).Return(record(name+".Create", nil))
instance.On("Start", false).Return(record(name+".Start", nil))
instance.On("Start", false).Return(record(name+".Start", map[string]error{"service": testcase.startError}[name]))
instance.On("Remove").Return(record(name+".Remove", nil))
instance.On("Close").Return(record(name+".Close", nil))
}
@@ -1302,10 +1393,14 @@ func TestRunContext_cleanupFailedStart(t *testing.T) {
service.On("Inspect", mock.Anything).Return(&container.Info{State: "running", Health: health}, nil).
Run(func(mock.Arguments) { operations = append(operations, "service.Inspect") }).Once()
}
service.On("DumpLogs", mock.Anything).Return(nil).
Run(func(mock.Arguments) { operations = append(operations, "service.DumpLogs") })
stdouts := map[string]io.Writer{}
service.On("DumpLogs", mock.Anything).Return(nil).Run(func(mock.Arguments) {
operations = append(operations, "service.DumpLogs")
fmt.Fprintln(stdouts["postgres:latest"], "::add-mask::leak")
})
origNewContainer := newContainer
newContainer = func(input *container.NewContainerInput) container.ExecutionsEnvironment {
stdouts[input.Image] = input.Stdout
return map[string]*containerMock{"postgres:latest": service, "node:20": job}[input.Image]
}
t.Cleanup(func() { newContainer = origNewContainer })
@@ -1316,22 +1411,32 @@ func TestRunContext_cleanupFailedStart(t *testing.T) {
Run: &model.Run{JobID: "job", Workflow: workflow},
platformImage: "node:20",
}
ctx := common.WithDryrun(t.Context(), true)
logger, hook := logtest.NewLogger()
ctx := common.WithLogger(common.WithDryrun(t.Context(), true), logger)
rc.ExprEval = rc.NewExpressionEvaluator(ctx)
err = rc.startContainer().Then(rc.stopContainer()).Finally(rc.closeContainer())(ctx)
want := "job.Remove service.Remove service.Close service.Pull job.Pull"
if testcase.pullError != nil {
want := "job.Remove service.Close service.Remove service.Pull job.Pull"
switch {
case testcase.pullError != nil:
require.ErrorIs(t, err, testcase.pullError)
} else {
want += " service.Create service.Start service.Inspect job.Create job.Start job.Inspect job.Copy service.Inspect"
case testcase.startError != nil:
require.ErrorIs(t, err, testcase.startError)
want += " service.Create service.Start service.DumpLogs"
default:
want += " service.Create service.Start service.Inspect job.Create job.Start job.Inspect job.Copy service.Inspect service.DumpLogs"
if testcase.health == container.HealthUnhealthy {
require.ErrorContains(t, err, "the service 'postgres' is unhealthy")
want += " service.DumpLogs"
} else {
require.NoError(t, err)
}
}
assert.Equal(t, strings.Fields(want+" job.Remove service.Remove service.Close job.Close"), operations)
slices.Sort(operations[:3])
slices.Sort(operations[len(operations)-4 : len(operations)-1])
assert.Equal(t, strings.Fields(want+" job.Remove service.Close service.Remove job.Close"), operations)
assert.Empty(t, rc.Masks)
assert.Equal(t, testcase.pullError == nil && testcase.health != container.HealthUnhealthy, slices.ContainsFunc(hook.AllRecords(), func(entry logging.Record) bool {
return strings.HasPrefix(entry.Message, "::group::Print service container logs")
}))
})
}
@@ -1352,7 +1457,7 @@ func TestWaitForServiceContainers(t *testing.T) {
Return(&container.Info{ID: "id", State: "running", Health: container.HealthNone}, nil).Once()
rc := newRunContext(0, &serviceContainer{name: "redis", container: service})
require.NoError(t, rc.waitForServiceContainers()(context.Background()))
require.NoError(t, rc.waitForServiceContainers()(t.Context()))
service.AssertExpectations(t)
})
@@ -1369,9 +1474,8 @@ func TestWaitForServiceContainers(t *testing.T) {
Return(&container.Info{ID: "id", State: "running", Health: container.HealthHealthy}, nil).Once()
var output bytes.Buffer
logger := log.New()
logger.SetOutput(&output)
require.NoError(t, newRunContext(0, &serviceContainer{name: "postgres", container: service}).waitForServiceContainers()(common.WithLogger(t.Context(), logger.WithFields(nil))))
logger := logging.New(logging.NewLevelHandler(newJobLogHandler(&output, cyan), logging.LevelInfo))
require.NoError(t, newRunContext(0, &serviceContainer{name: "postgres", container: service}).waitForServiceContainers()(common.WithLogger(t.Context(), logger)))
assert.Equal(t, []time.Duration{0, time.Second, 2 * time.Second}, pollTimes)
assert.Equal(t, 1, strings.Count(output.String(), "postgres service is starting."))
assert.Contains(t, output.String(), "postgres service is healthy.")
@@ -1388,7 +1492,7 @@ func TestWaitForServiceContainers(t *testing.T) {
service.On("DumpLogs", mock.Anything).Return(nil).Once()
rc := newRunContext(0, &serviceContainer{name: "postgres", container: service})
err := rc.waitForServiceContainers()(context.Background())
err := rc.waitForServiceContainers()(t.Context())
require.Error(t, err)
assert.Contains(t, err.Error(), "the service 'postgres' is unhealthy: connection refused")
service.AssertExpectations(t)
@@ -1400,7 +1504,7 @@ func TestWaitForServiceContainers(t *testing.T) {
Return(&container.Info{State: "exited", ExitCode: 2, Health: container.HealthNone}, nil).Once()
rc := newRunContext(0, &serviceContainer{name: "postgres", container: service})
require.NoError(t, rc.waitForServiceContainers()(context.Background()))
require.NoError(t, rc.waitForServiceContainers()(t.Context()))
})
t.Run("proceeds when the container is gone", func(t *testing.T) {
@@ -1409,7 +1513,7 @@ func TestWaitForServiceContainers(t *testing.T) {
Return((*container.Info)(nil), container.ErrContainerNotFound).Once()
rc := newRunContext(0, &serviceContainer{name: "postgres", container: service})
require.NoError(t, rc.waitForServiceContainers()(context.Background()))
require.NoError(t, rc.waitForServiceContainers()(t.Context()))
})
t.Run("fails right away when one service fails while another is still starting", func(t *testing.T) {
@@ -1443,7 +1547,7 @@ func TestWaitForServiceContainers(t *testing.T) {
Return(&container.Info{State: "running", Health: container.HealthStarting}, nil)
rc := newRunContext(20*time.Millisecond, &serviceContainer{name: "postgres", container: service})
err := rc.waitForServiceContainers()(context.Background())
err := rc.waitForServiceContainers()(t.Context())
require.Error(t, err)
assert.Contains(t, err.Error(), "did not become healthy within")
})
@@ -1455,7 +1559,7 @@ func TestWaitForServiceContainers(t *testing.T) {
Return((*container.Info)(nil), errors.New("inspect aborted"))
rc := newRunContext(20*time.Millisecond, &serviceContainer{name: "postgres", container: service})
err := rc.waitForServiceContainers()(context.Background())
err := rc.waitForServiceContainers()(t.Context())
require.Error(t, err)
assert.Contains(t, err.Error(), "did not become healthy within")
})
@@ -1468,7 +1572,7 @@ func TestWaitForServiceContainers(t *testing.T) {
svc := &serviceContainer{name: "postgres", container: service}
rc := newRunContext(-1, svc)
require.NoError(t, rc.waitForServiceContainers()(context.Background()))
require.NoError(t, rc.waitForServiceContainers()(t.Context()))
service.AssertExpectations(t)
assert.Equal(t, "id", svc.info.ID)
})
@@ -1478,13 +1582,13 @@ func TestWaitForServiceContainers(t *testing.T) {
service.On("Inspect", mock.Anything).Return((*container.Info)(nil), errors.New("daemon is gone")).Once()
rc := newRunContext(-1, &serviceContainer{name: "postgres", container: service})
err := rc.waitForServiceContainers()(context.Background())
err := rc.waitForServiceContainers()(t.Context())
require.Error(t, err)
assert.Contains(t, err.Error(), "failed to inspect service 'postgres'")
})
t.Run("is a no-op without services", func(t *testing.T) {
require.NoError(t, newRunContext(0).waitForServiceContainers()(context.Background()))
require.NoError(t, newRunContext(0).waitForServiceContainers()(t.Context()))
})
}
@@ -1500,7 +1604,7 @@ func TestReportUnstartedServices(t *testing.T) {
{name: "redis", container: running},
}}
require.NoError(t, rc.reportUnstartedServices()(context.Background()))
require.NoError(t, rc.reportUnstartedServices()(t.Context()))
dead.AssertExpectations(t)
running.AssertExpectations(t)
}
@@ -1540,14 +1644,14 @@ func TestCaptureJobContainerInfoExportsDockerWorkspace(t *testing.T) {
JobContainer: job,
}
require.NoError(t, rc.captureJobContainerInfo()(context.Background()))
require.NoError(t, rc.captureJobContainerInfo()(t.Context()))
assert.Equal(t, "job-container-id", rc.jobContainerID)
assert.Equal(t, "/var/lib/docker/volumes/job/_data/owner/repo", rc.Env["GITEA_DOCKER_WORKSPACE"])
rc.Config.Workdir = "/elsewhere"
rc.Env = map[string]string{}
require.NoError(t, rc.captureJobContainerInfo()(context.Background()))
require.NoError(t, rc.captureJobContainerInfo()(t.Context()))
assert.NotContains(t, rc.Env, "GITEA_DOCKER_WORKSPACE")
}
@@ -1587,14 +1691,44 @@ func TestImageOSFromImage(t *testing.T) {
}
}
func TestResolvePlatformImageAppliesImagelessContainerToRunsOnImage(t *testing.T) {
for _, tc := range []struct {
name, image string
want model.ContainerSpec
}{
{"without image key", "", model.ContainerSpec{Volumes: []string{"/host/data:/data"}, Env: map[string]string{"KEY": "value"}}},
{"with empty image", "image: '', ", model.ContainerSpec{}},
} {
t.Run(tc.name, func(t *testing.T) {
rc := createIfTestRunContext(map[string]*model.Job{
"job1": createJob(t, "runs-on: toolkit\ncontainer: {"+tc.image+"volumes: [/host/data:/data], env: {KEY: value}}", ""),
})
require.NoError(t, rc.resolvePlatformImage(t.Context()))
require.NoError(t, evaluateJobEnvAndDefaults(t.Context(), rc))
assert.Equal(t, "ubuntu-latest", rc.platformImage)
assert.Equal(t, tc.want, rc.containerSpec)
})
}
}
func createRunsOnRunContext(t *testing.T, runsOn string) *RunContext {
return createIfTestRunContext(map[string]*model.Job{
"job1": createJob(t, "runs-on: "+runsOn, ""),
})
}
func TestRunContextContainerImageCannotSelectBackend(t *testing.T) {
rc := createIfTestRunContext(map[string]*model.Job{"job1": createJob(t, "container: -self-hosted", "")})
require.NoError(t, rc.resolvePlatformImage(t.Context()))
assert.False(t, rc.IsHostEnv())
rc = createIfTestRunContext(map[string]*model.Job{"job1": createJob(t, "runs-on: ${{ fromJSON('x') }}\ncontainer: node", "")})
require.ErrorContains(t, rc.resolvePlatformImage(t.Context()), "runs-on")
}
func TestRunContextImageOS(t *testing.T) {
ctx := context.Background()
ctx := t.Context()
t.Run("prefers the release in the resolved image tag", func(t *testing.T) {
rc := createRunsOnRunContext(t, "ubuntu-latest")
@@ -1621,7 +1755,7 @@ func TestRunContextImageOS(t *testing.T) {
}
func TestRunContextGetRunnerContext(t *testing.T) {
ctx := context.Background()
ctx := t.Context()
t.Run("adds the runner values the container cannot know", func(t *testing.T) {
rc := createRunsOnRunContext(t, "ubuntu-latest")
@@ -1673,8 +1807,7 @@ func TestRunContextUpdateExtraPath(t *testing.T) {
{name: "over 16 MiB", content: []byte(strings.Repeat("x", 16*1024*1024+1)), wantErr: true},
} {
t.Run(testcase.name, func(t *testing.T) {
logger := log.New()
logger.SetOutput(io.Discard)
logger := logging.New(logging.DiscardHandler)
ctx := common.WithLogger(t.Context(), logger)
jobContainer := &containerMock{}
jobContainer.On("GetContainerArchive", mock.Anything, "/github/path").
@@ -1702,7 +1835,7 @@ func TestParentDir(t *testing.T) {
}
func TestRunContextWithGithubEnvRunnerValues(t *testing.T) {
ctx := context.Background()
ctx := t.Context()
rc := createRunsOnRunContext(t, "ubuntu-latest")
rc.Config.RunnerName = "runner-1"
rc.Config.Secrets = map[string]string{"ACTIONS_STEP_DEBUG": "true"}

View File

@@ -5,20 +5,23 @@
package runner
import (
"cmp"
"context"
"fmt"
"net/http"
"os"
"runtime"
"slices"
"sync"
"time"
"gitea.com/gitea/runner/act/common"
"gitea.com/gitea/runner/act/common/logging"
"gitea.com/gitea/runner/act/container"
"gitea.com/gitea/runner/act/ghcontext"
"gitea.dev/actionslib/pkg/model"
docker_container "github.com/moby/moby/api/types/container"
log "github.com/sirupsen/logrus"
"go.yaml.in/yaml/v4"
)
@@ -47,7 +50,7 @@ type Config struct {
ContainerDaemonSocket string // Path to Docker daemon socket
ContainerOptions string // Options for the job container
UseGitIgnore bool // controls if paths in .gitignore should not be copied into container, default true
GitHubInstance string // GitHub instance to use, default "github.com"
GitHubInstance string // instance hostname or URL, empty when unknown
ContainerCapAdd []string // list of kernel capabilities to add to the containers
ContainerCapDrop []string // list of kernel capabilities to remove from the containers
ArtifactServerPath string // the path where the artifact server stores uploads
@@ -62,6 +65,7 @@ type Config struct {
NoActionPatch bool // run actions exactly as published, applying no compatibility patches, see patch_actions.go
PresetGitHubContext *model.GithubContext // overrides actor, ref, repository, token and related context fields
WorkflowCallInputs map[string]any // the `inputs` Gitea resolved for a called workflow's job, nil for other jobs
EventJSON string // the content of JSON file to use for event.json in containers, overrides EventPath
ContainerNamePrefix string // the prefix of container name
ContainerMaxLifetime time.Duration // the max lifetime of job containers
@@ -74,16 +78,20 @@ type Config struct {
// AppURL. It is never set for github.com or a GithubMirror, so the token stays on-instance.
DefaultActionInstanceIsSelfHosted bool
PlatformPicker func(labels []string) string
JobLoggerLevel *log.Level // the level of job logger
ValidVolumes []string // only volumes (and bind mounts) in this slice can be mounted on the job container or service containers
SharedToolCache bool // one tool cache for all jobs instead of one per job
InsecureSkipTLS bool // whether to skip verifying TLS certificate of the Gitea instance
MaxParallel int // max parallel jobs to run across all workflows (0 = no limit, uses CPU count)
AllocatePTY bool // allocate a pseudo-TTY for each step's process
ServiceReadyTimeout time.Duration // how long a job waits for its service containers to report healthy (0 uses the default)
RunnerName string // name this runner registered with, reported as `runner.name`, defaults to the hostname
JobStartedHook string // script run inside the job environment before the job's first step; ACTIONS_RUNNER_HOOK_JOB_STARTED is read from Env when empty
JobCompletedHook string // script run inside the job environment after the job's last step; ACTIONS_RUNNER_HOOK_JOB_COMPLETED is read from Env when empty
KubernetesPicker func(labels []string) container.KubernetesOptions
InstanceTransport http.RoundTripper
JobLoggerLevel *logging.Level // the level of job logger
ValidVolumes []string // only volumes (and bind mounts) in this slice can be mounted on the job container or service containers
SharedToolCache bool // one tool cache for all jobs instead of one per job
InsecureSkipTLS bool // whether to skip verifying TLS certificate of the Gitea instance
ClientCertFile, ClientKeyFile string // PEM client certificate for the Gitea instance
MaxParallel int // max parallel jobs to run across all workflows (0 = no limit, uses CPU count)
AllocatePTY bool // allocate a pseudo-TTY for each step's process
ServiceReadyTimeout time.Duration // how long a job waits for its service containers to report healthy (0 uses the default)
NoSweep bool // keep the containers, networks and volumes a job creates through its docker proxy
RunnerName string // name this runner registered with, reported as `runner.name`, defaults to the hostname
JobStartedHook string // script run inside the job environment before the job's first step; ACTIONS_RUNNER_HOOK_JOB_STARTED is read from Env when empty
JobCompletedHook string // script run inside the job environment after the job's last step; ACTIONS_RUNNER_HOOK_JOB_COMPLETED is read from Env when empty
}
// RunnerDebug reports whether debug logging is on, exposed as `runner.debug` and
@@ -104,13 +112,7 @@ func (c Config) GetToken() string {
// DefaultActionURL returns the host used for implicit remote actions.
func (c Config) DefaultActionURL() string {
if c.DefaultActionInstance != "" {
return c.DefaultActionInstance
}
if c.GitHubInstance != "" {
return c.GitHubInstance
}
return "github.com"
return cmp.Or(c.DefaultActionInstance, c.GitHubInstance, "github.com")
}
type caller struct {
@@ -147,7 +149,7 @@ func (runner *runnerImpl) configure() (*runnerImpl, error) {
if runner.config.EventJSON != "" {
runner.eventJSON = runner.config.EventJSON
} else if runner.config.EventPath != "" {
log.Debugf("Reading event.json from %s", runner.config.EventPath)
logging.Debugf("Reading event.json from %s", runner.config.EventPath)
eventJSONBytes, err := os.ReadFile(runner.config.EventPath)
if err != nil {
return nil, err
@@ -161,7 +163,7 @@ func maxParallelFor(strategy *model.Strategy, combinations int) int {
maxParallel := 4 // actionslib has no default for an undeclared max-parallel
if strategy != nil {
if limit, err := strategy.MaxParallel(); err != nil {
log.Errorf("Ignoring invalid max-parallel: %v", err)
logging.Errorf("Ignoring invalid max-parallel: %v", err)
} else if limit > 0 {
maxParallel = limit
}
@@ -171,39 +173,48 @@ func maxParallelFor(strategy *model.Strategy, combinations int) int {
// NewPlanExecutor ...
func (runner *runnerImpl) NewPlanExecutor(plan *model.Plan) common.Executor {
for _, stage := range plan.Stages {
for _, run := range stage.Runs {
job := run.Job()
if runsOn := job.RunsOn(); job.Uses == "" && len(job.Steps) > 0 && (len(runsOn) == 0 || slices.Contains(runsOn, "")) { // Gitea's needs placeholders carry no steps
return common.NewErrorExecutor(fmt.Errorf("job %q requires a non-empty 'runs-on'", run.JobID))
}
}
}
maxJobNameLen := 0
stagePipeline := make([]common.Executor, 0)
log.Debugf("Plan Stages: %v", plan.Stages)
logging.Debugf("Plan Stages: %v", plan.Stages)
for i := range plan.Stages {
stage := plan.Stages[i]
stagePipeline = append(stagePipeline, func(ctx context.Context) error {
pipeline := make([]common.Executor, 0)
for _, run := range stage.Runs {
log.Debugf("Stages Runs: %v", stage.Runs)
logging.Debugf("Stages Runs: %v", stage.Runs)
stageExecutor := make([]common.Executor, 0)
job := run.Job()
log.Debugf("Job.Name: %v", job.Name)
log.Debugf("Job.RawNeeds: %v", job.RawNeeds)
log.Debugf("Job.RawRunsOn: %v", job.RawRunsOn)
log.Debugf("Job.Env: %v", job.Env)
log.Debugf("Job.If: %v", job.If)
logging.Debugf("Job.Name: %v", job.Name)
logging.Debugf("Job.RawNeeds: %v", job.RawNeeds)
logging.Debugf("Job.RawRunsOn: %v", job.RawRunsOn)
logging.Debugf("Job.Env: %v", job.Env)
logging.Debugf("Job.If: %v", job.If)
for step := range job.Steps {
if nil != job.Steps[step] {
log.Debugf("Job.Steps: %v", job.Steps[step].String())
logging.Debugf("Job.Steps: %v", job.Steps[step].String())
}
}
log.Debugf("Job.TimeoutMinutes: %v", job.TimeoutMinutes)
log.Debugf("Job.Services: %v", job.Services)
log.Debugf("Job.Strategy: %v", job.Strategy)
log.Debugf("Job.RawContainer: %v", job.RawContainer)
log.Debugf("Job.Defaults.Run.Shell: %v", job.Defaults.Run.Shell)
log.Debugf("Job.Defaults.Run.WorkingDirectory: %v", job.Defaults.Run.WorkingDirectory)
log.Debugf("Job.Outputs: %v", job.Outputs)
log.Debugf("Job.Uses: %v", job.Uses)
log.Debugf("Job.With: %v", job.With)
log.Debugf("Job.Result: %v", job.Result)
logging.Debugf("Job.TimeoutMinutes: %v", job.TimeoutMinutes)
logging.Debugf("Job.Services: %v", job.Services)
logging.Debugf("Job.Strategy: %v", job.Strategy)
logging.Debugf("Job.RawContainer: %v", job.RawContainer)
logging.Debugf("Job.Defaults.Run.Shell: %v", job.Defaults.Run.Shell)
logging.Debugf("Job.Defaults.Run.WorkingDirectory: %v", job.Defaults.Run.WorkingDirectory)
logging.Debugf("Job.Outputs: %v", job.Outputs)
logging.Debugf("Job.Uses: %v", job.Uses)
logging.Debugf("Job.With: %v", job.With)
logging.Debugf("Job.Result: %v", job.Result)
if job.Strategy != nil || job.RawStrategy.Kind == yaml.ScalarNode {
strategyRc, err := runner.newRunContext(ctx, run, nil)
@@ -215,13 +226,13 @@ func (runner *runnerImpl) NewPlanExecutor(plan *model.Plan) common.Executor {
return err
}
} else {
log.Debugf("Job.Strategy.FailFast: %v", job.Strategy.GetFailFast())
log.Debugf("Job.Strategy.FailFastString: %v", job.Strategy.FailFastString)
log.Debugf("Job.Strategy.MaxParallelString: %v", job.Strategy.MaxParallelString)
log.Debugf("Job.Strategy.RawMatrix: %v", job.Strategy.RawMatrix)
logging.Debugf("Job.Strategy.FailFast: %v", job.Strategy.GetFailFast())
logging.Debugf("Job.Strategy.FailFastString: %v", job.Strategy.FailFastString)
logging.Debugf("Job.Strategy.MaxParallelString: %v", job.Strategy.MaxParallelString)
logging.Debugf("Job.Strategy.RawMatrix: %v", job.Strategy.RawMatrix)
// An unevaluated expression is left in place, which GetMatrixes below rejects.
if err := strategyRc.NewExpressionEvaluator(ctx).EvaluateYamlNode(ctx, &job.Strategy.RawMatrix); err != nil {
log.Errorf("Error while evaluating matrix: %v", err)
logging.Errorf("Error while evaluating matrix: %v", err)
}
for _, value := range []*string{&job.Strategy.FailFastString, &job.Strategy.MaxParallelString} {
if evaluated, err := strategyRc.ExprEval.Interpolate(ctx, *value); err == nil {
@@ -235,11 +246,11 @@ func (runner *runnerImpl) NewPlanExecutor(plan *model.Plan) common.Executor {
if err != nil {
return fmt.Errorf("could not get job matrix: %w", err)
}
log.Debugf("Job Matrices: %v", matrixes)
logging.Debugf("Job Matrices: %v", matrixes)
maxParallel := maxParallelFor(job.Strategy, len(matrixes))
log.Infof("Running job with maxParallel=%d for %d matrix combinations", maxParallel, len(matrixes))
logging.Infof("Running job with maxParallel=%d for %d matrix combinations", maxParallel, len(matrixes))
for i, matrix := range matrixes {
index, total := i, len(matrixes)
@@ -298,7 +309,7 @@ func (runner *runnerImpl) NewPlanExecutor(plan *model.Plan) common.Executor {
if len(pipeline) == 1 {
// Single run/job: execute directly without additional parallelization wrapper
// This ensures max-parallel is the only limiting factor
log.Debugf("Single pipeline element, executing directly")
logging.Debugf("Single pipeline element, executing directly")
return pipeline[0](ctx)
}
@@ -308,9 +319,9 @@ func (runner *runnerImpl) NewPlanExecutor(plan *model.Plan) common.Executor {
// If MaxParallel is set in config, use it
if runner.config.MaxParallel > 0 {
parallelism = runner.config.MaxParallel
log.Debugf("Using configured max-parallel: %d", parallelism)
logging.Debugf("Using configured max-parallel: %d", parallelism)
} else {
log.Debugf("Using CPU count for parallelism: %d", parallelism)
logging.Debugf("Using CPU count for parallelism: %d", parallelism)
}
// Don't exceed the number of pipeline elements
@@ -318,7 +329,7 @@ func (runner *runnerImpl) NewPlanExecutor(plan *model.Plan) common.Executor {
parallelism = len(pipeline)
}
log.Infof("Executing %d pipeline elements with parallelism %d", len(pipeline), parallelism)
logging.Infof("Executing %d pipeline elements with parallelism %d", len(pipeline), parallelism)
return common.NewParallelExecutor(parallelism, pipeline...)(ctx)
})
}

View File

@@ -17,17 +17,18 @@ import (
"time"
"gitea.com/gitea/runner/act/common"
"gitea.com/gitea/runner/act/common/logging"
"gitea.dev/actionslib/pkg/model"
"github.com/joho/godotenv"
log "github.com/sirupsen/logrus"
assert "github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
var (
baseImage = "node:24-bookworm-slim"
platforms map[string]string
logLevel = log.DebugLevel
logLevel = logging.LevelDebug
workdir = "testdata"
secrets map[string]string
testRunID = strings.ToLower(rand.Text()[:8]) // keeps docker names apart from concurrent test processes on one daemon
@@ -54,7 +55,7 @@ func init() {
}
if l := os.Getenv("ACT_TEST_LOG_LEVEL"); l != "" {
if lvl, err := log.ParseLevel(l); err == nil {
if lvl, err := logging.ParseLevel(l); err == nil {
logLevel = lvl
}
}
@@ -70,10 +71,11 @@ func TestNoWorkflowsFoundByPlanner(t *testing.T) {
planner, err := model.NewWorkflowPlanner("res", true)
assert.NoError(t, err) //nolint:testifylint // pre-existing issue from nektos/act
out := log.StandardLogger().Out
out := logging.Output()
t.Cleanup(func() { logging.SetOutput(out) })
var buf bytes.Buffer
log.SetOutput(&buf)
log.SetLevel(log.DebugLevel)
logging.SetOutput(&buf)
logging.SetLevel(logging.LevelDebug)
plan, err := planner.PlanEvent("pull_request")
assert.NotNil(t, plan)
assert.NoError(t, err) //nolint:testifylint // pre-existing issue from nektos/act
@@ -83,25 +85,24 @@ func TestNoWorkflowsFoundByPlanner(t *testing.T) {
assert.NotNil(t, plan)
assert.NoError(t, err) //nolint:testifylint // pre-existing issue from nektos/act
assert.Contains(t, buf.String(), "no workflows found by planner")
log.SetOutput(out)
}
func TestGraphMissingEvent(t *testing.T) {
planner, err := model.NewWorkflowPlanner("testdata/issue-1595/no-event.yml", true)
assert.NoError(t, err) //nolint:testifylint // pre-existing issue from nektos/act
out := log.StandardLogger().Out
out := logging.Output()
t.Cleanup(func() { logging.SetOutput(out) })
var buf bytes.Buffer
log.SetOutput(&buf)
log.SetLevel(log.DebugLevel)
logging.SetOutput(&buf)
logging.SetLevel(logging.LevelDebug)
plan, err := planner.PlanEvent("push")
assert.NoError(t, err) //nolint:testifylint // pre-existing issue from nektos/act
assert.NotNil(t, plan)
assert.Empty(t, plan.Stages)
assert.Contains(t, buf.String(), "no events found for workflow: no-event.yml")
log.SetOutput(out)
assert.Contains(t, buf.String(), `msg="no events found for workflow" file=no-event.yml`)
}
func TestGraphMissingFirst(t *testing.T) {
@@ -118,29 +119,30 @@ func TestGraphWithMissing(t *testing.T) {
planner, err := model.NewWorkflowPlanner("testdata/issue-1595/missing.yml", true)
assert.NoError(t, err) //nolint:testifylint // pre-existing issue from nektos/act
out := log.StandardLogger().Out
out := logging.Output()
t.Cleanup(func() { logging.SetOutput(out) })
var buf bytes.Buffer
log.SetOutput(&buf)
log.SetLevel(log.DebugLevel)
logging.SetOutput(&buf)
logging.SetLevel(logging.LevelDebug)
plan, err := planner.PlanEvent("push")
assert.NotNil(t, plan)
assert.Empty(t, plan.Stages)
assert.EqualError(t, err, "unable to build dependency graph for missing (missing.yml)") //nolint:testifylint // pre-existing issue from nektos/act
assert.Contains(t, buf.String(), "unable to build dependency graph for missing (missing.yml)")
log.SetOutput(out)
}
func TestGraphWithSomeMissing(t *testing.T) {
log.SetLevel(log.DebugLevel)
logging.SetLevel(logging.LevelDebug)
planner, err := model.NewWorkflowPlanner("testdata/issue-1595/", true)
assert.NoError(t, err) //nolint:testifylint // pre-existing issue from nektos/act
out := log.StandardLogger().Out
out := logging.Output()
t.Cleanup(func() { logging.SetOutput(out) })
var buf bytes.Buffer
log.SetOutput(&buf)
log.SetLevel(log.DebugLevel)
logging.SetOutput(&buf)
logging.SetLevel(logging.LevelDebug)
plan, err := planner.PlanAll()
assert.Error(t, err, "unable to build dependency graph for no first (no-first.yml)") //nolint:testifylint // pre-existing issue from nektos/act
@@ -148,7 +150,21 @@ func TestGraphWithSomeMissing(t *testing.T) {
assert.Len(t, plan.Stages, 1)
assert.Contains(t, buf.String(), "unable to build dependency graph for missing (missing.yml)")
assert.Contains(t, buf.String(), "unable to build dependency graph for no first (no-first.yml)")
log.SetOutput(out)
}
func TestNewPlanExecutorRequiresRunsOnBeforeRunningAnyJob(t *testing.T) {
for _, runsOn := range []string{"", "if: false", "runs-on: [ubuntu-latest, '']"} {
workflow, err := model.ReadWorkflow(strings.NewReader("jobs:\n first:\n runs-on: ubuntu-latest\n steps: [run: echo]\n test:\n needs: first\n " + runsOn + "\n steps: [run: echo]\n"))
require.NoError(t, err)
plan, err := model.CombineWorkflowPlanner(workflow).PlanJob("test")
require.NoError(t, err)
require.ErrorContains(t, (&runnerImpl{config: &Config{}}).NewPlanExecutor(plan)(t.Context()), `job "test" requires a non-empty 'runs-on'`, runsOn)
assert.Empty(t, workflow.Jobs["first"].Result, runsOn)
}
workflow := &model.Workflow{Jobs: map[string]*model.Job{"build": {Result: "success"}}}
require.NoError(t, (&runnerImpl{config: &Config{}}).NewPlanExecutor(&model.Plan{Stages: []*model.Stage{{Runs: []*model.Run{{Workflow: workflow, JobID: "build"}}}}})(t.Context()))
assert.Equal(t, "success", workflow.Jobs["build"].Result)
}
func TestGraphEvent(t *testing.T) {
@@ -205,7 +221,7 @@ type TestJobFileInfo struct {
func (j *TestJobFileInfo) runTest(ctx context.Context, t *testing.T, cfg *Config) {
fmt.Printf("::group::%s\n", j.workflowPath) //nolint:forbidigo // pre-existing issue from nektos/act
log.SetLevel(logLevel)
logging.SetLevel(logLevel)
workdir, err := filepath.Abs(j.workdir)
assert.NoError(t, err, workdir) //nolint:testifylint // pre-existing issue from nektos/act
@@ -221,7 +237,6 @@ func (j *TestJobFileInfo) runTest(ctx context.Context, t *testing.T, cfg *Config
ContainerNamePrefix: strings.ReplaceAll(t.Name(), "/", "-") + "-" + testRunID,
// 0 would run jobs runtime.NumCPU()-wide, making the network peak machine-dependent
MaxParallel: 2,
ForceRebuild: true,
Env: cfg.Env,
Secrets: cfg.Secrets,
GitHubInstance: "github.com",
@@ -266,7 +281,7 @@ func TestRunEvent(t *testing.T) {
requireDocker(t)
t.Parallel()
ctx := context.Background()
ctx := t.Context()
tables := []TestJobFileInfo{
// Shells
@@ -369,7 +384,7 @@ func TestRunEventHostEnvironment(t *testing.T) {
// nix-specific cases gate on their own tool below so a missing pwsh/nix skips just those.
requireHostTools(t, "bash", "node")
ctx := context.Background()
ctx := t.Context()
tables := []TestJobFileInfo{}
@@ -434,6 +449,8 @@ func TestRunEventHostEnvironment(t *testing.T) {
tables = append(tables, []TestJobFileInfo{
{workdir, "nix-prepend-path", "push", "", platforms, secrets},
{workdir, "uses-docker-url", "push", "", platforms, secrets},
{workdir, "local-action-dockerfile", "push", "", platforms, secrets},
{workdir, "builtin-checkout", "push", "", platforms, secrets},
{workdir, "inputs-via-env-context", "push", "", platforms, secrets},
{workdir, "do-not-leak-step-env-in-composite", "push", "", platforms, secrets},
@@ -451,6 +468,8 @@ func TestRunEventHostEnvironment(t *testing.T) {
requireHostTools(t, "pwsh")
case "nix-prepend-path":
requireHostTools(t, "nix")
case "uses-docker-url", "local-action-dockerfile":
requireDockerSharingFiles(t)
}
t.Parallel()
hostPlanSlots <- struct{}{}
@@ -463,7 +482,7 @@ func TestRunEventHostEnvironment(t *testing.T) {
func TestDryrunEvent(t *testing.T) {
t.Parallel()
// Dryrun plans without containers or network (shells and local actions only).
ctx := common.WithDryrun(context.Background(), true)
ctx := common.WithDryrun(t.Context(), true)
tables := []TestJobFileInfo{
// Shells
@@ -494,7 +513,7 @@ func TestReusableWorkflowCaller(t *testing.T) {
requireDocker(t)
t.Parallel()
table := TestJobFileInfo{workdir, "uses-workflow", "push", "", platforms, map[string]string{"secret": "keep_it_private"}}
table.runTest(context.Background(), t, &Config{Secrets: table.secrets})
table.runTest(t.Context(), t, &Config{Secrets: table.secrets})
}
func TestRunEventSecrets(t *testing.T) {
@@ -515,7 +534,7 @@ func TestRunEventSecrets(t *testing.T) {
secrets, _ := godotenv.Read(filepath.Join(workdir, workflowPath, ".secrets"))
assert.NoError(t, err, "Failed to read .secrets") //nolint:testifylint // pre-existing issue from nektos/act
tjfi.runTest(context.Background(), t, &Config{Secrets: secrets, Env: env})
tjfi.runTest(t.Context(), t, &Config{Secrets: secrets, Env: env})
}
func TestRunEventPullRequest(t *testing.T) {
@@ -532,5 +551,5 @@ func TestRunEventPullRequest(t *testing.T) {
platforms: platforms,
}
tjfi.runTest(context.Background(), t, &Config{EventPath: filepath.Join(workdir, workflowPath, "event.json")})
tjfi.runTest(t.Context(), t, &Config{EventPath: filepath.Join(workdir, workflowPath, "event.json")})
}

View File

@@ -47,11 +47,11 @@ const maxSymlinkDepth = 10
func (s stepStage) String() string {
switch s {
case stepStagePre:
return "Pre"
return StagePre
case stepStageMain:
return "Main"
return StageMain
case stepStagePost:
return "Post"
return StagePost
}
return "Unknown"
}
@@ -97,14 +97,14 @@ func runStepExecutor(step step, stage stepStage, executor common.Executor) commo
if err != nil {
stepResult.Conclusion = model.StepStatusFailure
stepResult.Outcome = model.StepStatusFailure
logger.WithField("stepResult", stepResult.Conclusion).Infof("Failure - %s %s", stage, stepModel)
logger.With("stepResult", stepResult.Conclusion).Infof("Failure - %s %s", stage, stepModel)
return err
}
if !runStep {
stepResult.Conclusion = model.StepStatusSkipped
stepResult.Outcome = model.StepStatusSkipped
logger.WithField("stepResult", stepResult.Conclusion).Debugf("Skipping step '%s' due to '%s'", stepModel, ifExpression)
logger.With("stepResult", stepResult.Conclusion).Debugf("Skipping step '%s' due to '%s'", stepModel, ifExpression)
return nil
}
@@ -112,12 +112,13 @@ func runStepExecutor(step step, stage stepStage, executor common.Executor) commo
if strings.Contains(stepString, "::add-mask::") {
stepString = "add-mask command"
}
startLogger := logger.With(StepStartedField, true)
if stage == stepStageMain {
// Main steps print their own raw "Run <title>" header, so this line is redundant and
// only leaks into the "Set up job" section for the first step; keep it as a debug trace.
logger.Debugf("Run %s %s", stage, stepString)
startLogger.Debugf("Run %s %s", stage, stepString)
} else {
logger.Infof("Run %s %s", stage, stepString)
startLogger.Infof("Run %s %s", stage, stepString)
}
// Prepare and clean Runner File Commands
@@ -186,29 +187,35 @@ func runStepExecutor(step step, stage stepStage, executor common.Executor) commo
if err == nil {
err = insecureErr
}
if fileErr := processRunnerEnvFileCommand(ctx, envFileCommand, rc, rc.setEnvFile); fileErr != nil && err == nil {
fileCommandCtx := ctx
if ctx.Err() != nil { // an interrupted step still records its file commands, a live deadline stays in force
var cancelFileCommands context.CancelFunc
fileCommandCtx, cancelFileCommands = context.WithTimeout(context.WithoutCancel(ctx), jobCleanupTimeout)
defer cancelFileCommands()
}
if fileErr := processRunnerEnvFileCommand(fileCommandCtx, envFileCommand, rc, rc.setEnvFile); fileErr != nil && err == nil {
err = fileErr
}
if fileErr := processRunnerEnvFileCommand(ctx, stateFileCommand, rc, rc.saveState); fileErr != nil && err == nil {
if fileErr := processRunnerEnvFileCommand(fileCommandCtx, stateFileCommand, rc, rc.saveState); fileErr != nil && err == nil {
err = fileErr
}
if fileErr := processRunnerEnvFileCommand(ctx, outputFileCommand, rc, rc.setOutput); fileErr != nil && err == nil {
if fileErr := processRunnerEnvFileCommand(fileCommandCtx, outputFileCommand, rc, rc.setOutput); fileErr != nil && err == nil {
err = fileErr
}
if fileErr := rc.UpdateExtraPath(ctx, path.Join(actPath, pathFileCommand)); fileErr != nil && err == nil {
if fileErr := rc.UpdateExtraPath(fileCommandCtx, path.Join(actPath, pathFileCommand)); fileErr != nil && err == nil {
err = fileErr
}
_ = rc.JobContainer.Copy(actPath, files...)(ctx)
_ = rc.JobContainer.Copy(actPath, files...)(fileCommandCtx)
if err == nil {
logger.WithField("stepResult", stepResult.Conclusion).Infof("Success - %s %s", stage, stepString)
logger.With("stepResult", stepResult.Conclusion).Infof("Success - %s %s", stage, stepString)
} else {
stepResult.Outcome = model.StepStatusFailure
continueOnError, parseErr := isContinueOnError(ctx, stepModel.RawContinueOnError, step, stage)
if parseErr != nil {
stepResult.Conclusion = model.StepStatusFailure
logger.WithField("stepResult", stepResult.Conclusion).Infof("Failure - %s %s", stage, stepString)
logger.With("stepResult", stepResult.Conclusion).Infof("Failure - %s %s", stage, stepString)
return parseErr
}
@@ -223,7 +230,7 @@ func runStepExecutor(step step, stage stepStage, executor common.Executor) commo
// Infof: Errorf entries are promoted to the user log by the reporter,
// which would duplicate the ##[error] annotation emitted elsewhere.
logger.WithField("stepResult", stepResult.Conclusion).Infof("Failure - %s %s", stage, stepString)
logger.With("stepResult", stepResult.Conclusion).Infof("Failure - %s %s", stage, stepString)
}
return err
}

View File

@@ -34,10 +34,11 @@ func (sab *stepActionBuiltin) main() common.Executor {
}
printRunActionHeader(ctx, sab.Step, sab.env, sab.RunContext)
rawLogger := common.Logger(ctx).WithField(rawOutputField, true)
rawLogger := common.Logger(ctx).With(rawOutputField, true)
defer rawLogger.Infof("::endgroup::")
rc := sab.RunContext
rc.ApplyExtraPath(ctx, &sab.env)
return sab.run(ctx, &action.Context{
Container: rc.JobContainer,
Github: rc.getGithubContext(ctx),
@@ -48,6 +49,9 @@ func (sab *stepActionBuiltin) main() common.Executor {
NoSkipCheckout: rc.Config.NoSkipCheckout,
UseGitIgnore: rc.Config.UseGitIgnore,
InsecureSkipTLS: rc.Config.InsecureSkipTLS,
ClientCertFile: rc.Config.ClientCertFile,
ClientKeyFile: rc.Config.ClientKeyFile,
Env: sab.env,
})
})
}

View File

@@ -43,11 +43,11 @@ func (sal *stepActionLocal) main() common.Executor {
}
printRunActionHeader(ctx, sal.Step, sal.env, sal.getRunContext())
rawLogger := common.Logger(ctx).WithField(rawOutputField, true)
rawLogger := common.Logger(ctx).With(rawOutputField, true)
defer rawLogger.Infof("::endgroup::")
actionDir := filepath.Join(sal.getRunContext().Config.Workdir, sal.Step.Uses)
_, containerActionPath := getContainerActionPaths(sal.Step, path.Join(actionDir, ""), sal.RunContext)
_, containerActionPath := getContainerActionPaths(sal)
localReader := func(filename string) (io.Reader, io.Closer, error) {
spath := path.Join(containerActionPath, filename)
@@ -61,11 +61,14 @@ func (sal *stepActionLocal) main() common.Executor {
treader := tar.NewReader(tars)
header, err := treader.Next()
if errors.Is(err, io.EOF) {
tars.Close()
return nil, nil, os.ErrNotExist
} else if err != nil {
tars.Close()
return nil, nil, err
}
if header.FileInfo().Mode()&os.ModeSymlink == os.ModeSymlink {
tars.Close()
spath, err = symlinkJoin(spath, header.Linkname, containerActionPath)
if err != nil {
return nil, nil, err
@@ -124,8 +127,7 @@ func (sal *stepActionLocal) getActionModel() *model.Action {
func (sal *stepActionLocal) getCompositeRunContext(ctx context.Context) (*RunContext, error) {
if sal.compositeRunContext == nil {
actionDir := filepath.Join(sal.RunContext.Config.Workdir, sal.Step.Uses)
_, containerActionDir := getContainerActionPaths(sal.getStepModel(), actionDir, sal.RunContext)
_, containerActionDir := getContainerActionPaths(sal)
compositeRunContext, err := newCompositeRunContext(ctx, sal.RunContext, sal, containerActionDir)
if err != nil {

View File

@@ -35,7 +35,7 @@ func (salm *stepActionLocalMocks) readAction(_ context.Context, step *model.Step
}
func TestStepActionLocalTest(t *testing.T) {
ctx := context.Background()
ctx := t.Context()
cm := &containerMock{}
salm := &stepActionLocalMocks{}
@@ -225,7 +225,7 @@ func TestStepActionLocalPost(t *testing.T) {
for _, tt := range table {
t.Run(tt.name, func(t *testing.T) {
ctx := context.Background()
ctx := t.Context()
cm := &containerMock{}

View File

@@ -18,7 +18,6 @@ import (
"gitea.com/gitea/runner/act/common/git"
"gitea.dev/actionslib/pkg/model"
gogit "github.com/go-git/go-git/v5"
)
type stepActionRemote struct {
@@ -31,13 +30,16 @@ type stepActionRemote struct {
action *model.Action
env map[string]string
remoteAction *remoteAction
resolvedSha string
actionDir string // the repository at the resolved commit, shared by the job's actions in it
resolvedSha string // set on the step that made the job's download of the repository
}
var stepActionRemoteNewCloneExecutor = git.NewGitCloneExecutor
type actionDownload struct {
dir string
err error
}
// selfRepoPrefix introduces a self-repository reference: the action lives in the repo holding the file that wrote the `uses:`.
const selfRepoPrefix = "$/"
var stepActionRemoteFetch = git.FetchAction
func (sar *stepActionRemote) prepareActionExecutor() common.Executor {
return func(ctx context.Context) error {
@@ -65,75 +67,57 @@ func (sar *stepActionRemote) prepareActionExecutor() common.Executor {
common.Logger(ctx).Debugf("Skipping local actions/checkout because workdir was already copied")
return nil
}
actionDir := sar.actionDir()
defaultActionURL := sar.RunContext.Config.DefaultActionURL()
// For Gitea
// A composite RunContext nils Config.Secrets, so getGitCloneToken would yield an
// empty token and clone the action anonymously (401 against the authenticated
// instance). github.Token survives the composite config copy and matches the
// top-level token; keep the shouldCloneURLUseToken host gate to avoid leaking it.
cloneURL := sar.remoteAction.CloneURL(defaultActionURL)
token := ""
if shouldCloneURLUseToken(sar.RunContext.Config.GitHubInstance, sar.RunContext.Config.trustedActionInstance(), cloneURL) {
token = github.Token
}
gitClone := stepActionRemoteNewCloneExecutor(git.NewGitCloneExecutorInput{
URL: cloneURL,
Ref: sar.remoteAction.Ref,
Dir: actionDir,
Token: token,
OfflineMode: sar.RunContext.Config.ActionOfflineMode,
Depth: sar.RunContext.Config.ActionCloneDepth,
// printPrepareActions reports the download with its resolved commit.
Quiet: true,
InsecureSkipTLS: sar.cloneSkipTLS(), // For Gitea
})
var ntErr common.Executor
if err := gitClone(ctx); err != nil {
var refErr *git.Error
switch {
case errors.As(err, &refErr) && errors.Is(err, git.ErrShortRef):
return fmt.Errorf("unable to resolve action `%s`, the provided ref `%s` is the shortened version of a commit SHA, which is not supported. Please use the full commit SHA `%s` instead",
sar.Step.Uses, sar.remoteAction.Ref, refErr.Commit())
case errors.Is(err, gogit.ErrForceNeeded): // TODO: figure out if it will be easy to shadow/alias go-git err's
ntErr = common.NewInfoExecutor("Non-terminating error while running 'git clone': %v", err)
default:
return err
cloneURL := sar.remoteAction.CloneURL(sar.RunContext.Config.DefaultActionURL())
job := sar.RunContext.topLevelRunContext()
key := cloneURL + "@" + sar.remoteAction.Ref
download, ok := job.actionDownloads[key]
if !ok {
// For Gitea
// github.Token, unlike Config.GetToken, survives the composite config copy, which nils Config.Secrets.
download.dir, sar.resolvedSha, download.err = stepActionRemoteFetch(ctx, sar.RunContext.Config.withInstanceAuth(git.NewGitCloneExecutorInput{
URL: cloneURL,
Ref: sar.remoteAction.Ref,
Dir: filepath.Join(sar.RunContext.ActionCacheDir(), model.UsesHash(key)),
OfflineMode: sar.RunContext.Config.ActionOfflineMode,
Depth: sar.RunContext.Config.ActionCloneDepth,
}, github.Token))
if job.actionDownloads == nil {
job.actionDownloads = map[string]actionDownload{}
}
job.actionDownloads[key] = download
}
var refErr *git.Error
switch {
case errors.As(download.err, &refErr) && errors.Is(download.err, git.ErrShortRef):
return fmt.Errorf("unable to resolve action `%s`, the provided ref `%s` is the shortened version of a commit SHA, which is not supported. Please use the full commit SHA `%s` instead",
sar.Step.Uses, sar.remoteAction.Ref, refErr.Commit())
case download.err != nil:
return download.err
}
sar.actionDir = download.dir
// Best effort: the download report falls back to the ref alone when the commit is unknown.
if _, sha, err := git.FindGitRevision(ctx, actionDir); err != nil {
common.Logger(ctx).Debugf("unable to resolve the commit of %s: %v", sar.remoteAction.Reference(), err)
} else {
sar.resolvedSha = sha
unlock, err := git.AcquireCloneLock(ctx, sar.actionDir)
if err != nil {
return err
}
defer unlock()
remoteReader := func(filename string) (io.Reader, io.Closer, error) {
f, err := os.Open(filepath.Join(actionDir, sar.remoteAction.Path, filename))
f, err := os.Open(filepath.Join(sar.actionDir, sar.remoteAction.Path, filename))
return f, f, err
}
return common.NewPipelineExecutor(
ntErr,
func(ctx context.Context) error {
defer git.AcquireCloneLock(actionDir)()
actionModel, err := sar.readAction(ctx, sar.Step, actionDir, sar.remoteAction.Path, remoteReader, os.WriteFile)
sar.action = actionModel
return err
},
)(ctx)
actionModel, err := sar.readAction(ctx, sar.Step, sar.actionDir, sar.remoteAction.Path, remoteReader, os.WriteFile)
sar.action = actionModel
return err
}
}
// actionDownloadInfo reports the action this step downloaded and the commit it resolved to. ok is
// false when nothing was fetched, as for the local checkout of the workflow's own repository.
func (sar *stepActionRemote) actionDownloadInfo() (reference, sha string, ok bool) {
if sar.remoteAction == nil || sar.action == nil {
return "", "", false
func (sar *stepActionRemote) actionDownloadInfo() (reference, sha string) { // once per job, as actions/runner reports
if sar.resolvedSha == "" {
return "", ""
}
return sar.remoteAction.Reference(), sar.resolvedSha, true
return fmt.Sprintf("%s/%s@%s", sar.remoteAction.Org, sar.remoteAction.Repo, sar.remoteAction.Ref), sar.resolvedSha
}
func (sar *stepActionRemote) pre() common.Executor {
@@ -149,7 +133,7 @@ func (sar *stepActionRemote) main() common.Executor {
sar.prepareActionExecutor(),
runStepExecutor(sar, stepStageMain, func(ctx context.Context) error {
printRunActionHeader(ctx, sar.Step, sar.env, sar.RunContext)
rawLogger := common.Logger(ctx).WithField(rawOutputField, true)
rawLogger := common.Logger(ctx).With(rawOutputField, true)
defer rawLogger.Infof("::endgroup::")
github := sar.getGithubContext(ctx)
@@ -162,7 +146,7 @@ func (sar *stepActionRemote) main() common.Executor {
return sar.RunContext.JobContainer.CopyDir(copyToPath, sar.RunContext.Config.Workdir+string(filepath.Separator)+".", sar.RunContext.Config.UseGitIgnore, false)(ctx)
}
return sar.runAction(sar, sar.actionDir(), sar.remoteAction)(ctx)
return sar.runAction(sar, sar.actionDir, sar.remoteAction)(ctx)
}),
)
}
@@ -171,13 +155,8 @@ func (sar *stepActionRemote) post() common.Executor {
return runStepExecutor(sar, stepStagePost, runPostStep(sar)).If(hasPostStep(sar)).If(shouldRunPostStep(sar))
}
func (sar *stepActionRemote) actionDir() string {
uses := sar.Step.Uses
if strings.HasPrefix(uses, selfRepoPrefix) {
// The same `$/x` names a different action per enclosing repo, so key the cache on what it resolved to.
uses = sar.remoteAction.URL + "/" + sar.remoteAction.Reference()
}
return fmt.Sprintf("%s/%s", sar.RunContext.ActionCacheDir(), model.UsesHash(uses))
func (sar *stepActionRemote) containerKey() string {
return model.UsesHash(sar.remoteAction.URL + "/" + sar.remoteAction.Reference())
}
func (sar *stepActionRemote) getRunContext() *RunContext {
@@ -228,9 +207,7 @@ func (sar *stepActionRemote) getActionModel() *model.Action {
func (sar *stepActionRemote) getCompositeRunContext(ctx context.Context) (*RunContext, error) {
if sar.compositeRunContext == nil {
actionDir := sar.actionDir()
actionLocation := path.Join(actionDir, sar.remoteAction.Path)
_, containerActionDir := getContainerActionPaths(sar.getStepModel(), actionLocation, sar.RunContext)
_, containerActionDir := getContainerActionPaths(sar)
compositeRunContext, err := newCompositeRunContext(ctx, sar.RunContext, sar, containerActionDir)
if err != nil {
@@ -259,22 +236,6 @@ func (sar *stepActionRemote) getCompositeSteps() *compositeSteps {
return sar.compositeSteps
}
// For Gitea
// cloneSkipTLS returns true if the runner can clone an action from the Gitea instance
func (sar *stepActionRemote) cloneSkipTLS() bool {
if !sar.RunContext.Config.InsecureSkipTLS {
// Return false if the Gitea instance is not an insecure instance
return false
}
if sar.remoteAction.URL == "" {
// Empty URL means the default action instance should be used
// Return true if the URL of the Gitea instance is the same as the URL of the default action instance
return sar.RunContext.Config.DefaultActionURL() == sar.RunContext.Config.GitHubInstance
}
// Return true if the URL of the remote action is the same as the URL of the Gitea instance
return sar.remoteAction.URL == sar.RunContext.Config.GitHubInstance
}
type remoteAction struct {
URL string
Org string
@@ -287,8 +248,8 @@ func (ra *remoteAction) CloneURL(u string) string {
if ra.URL == "" {
// keep an absolute local path as-is (used by tests to resolve actions from a local
// repo); only bare host names get the https:// scheme prepended
if !strings.HasPrefix(u, "http://") && !strings.HasPrefix(u, "https://") && !filepath.IsAbs(u) {
u = "https://" + u
if !filepath.IsAbs(u) {
u = withHTTPS(u)
}
} else {
u = ra.URL
@@ -337,6 +298,9 @@ func newRemoteAction(action string, github *model.GithubContext, instanceURL str
}
ra.Org, ra.Repo, _ = strings.Cut(repo, "/")
ra.URL, ra.Ref = github.ServerURL, ref
if ra.URL == "" {
return nil, fmt.Errorf("unable to resolve %q without a Gitea instance", action)
}
if ra.Org == "" || ra.Repo == "" || ra.Ref == "" {
return nil, fmt.Errorf("unable to resolve %q without a repository and commit", action)
}

View File

@@ -12,6 +12,7 @@ import (
"io"
"os"
"os/exec"
"path"
"path/filepath"
"strings"
"testing"
@@ -31,14 +32,10 @@ type stepActionRemoteMocks struct {
mock.Mock
}
func actionDirSuffix(suffix string) any {
return mock.MatchedBy(func(actionDir string) bool { return strings.HasSuffix(actionDir, suffix) })
}
func setCloneExecutor(t *testing.T, executor func(git.NewGitCloneExecutorInput) common.Executor) {
original := stepActionRemoteNewCloneExecutor
stepActionRemoteNewCloneExecutor = executor
t.Cleanup(func() { stepActionRemoteNewCloneExecutor = original })
func setFetch(t *testing.T, fetch func(context.Context, git.NewGitCloneExecutorInput) (string, string, error)) {
original := stepActionRemoteFetch
stepActionRemoteFetch = fetch
t.Cleanup(func() { stepActionRemoteFetch = original })
}
func TestShortSHAActionRejected(t *testing.T) {
@@ -175,25 +172,23 @@ func TestStepActionRemote(t *testing.T) {
for _, tt := range table {
t.Run(tt.name, func(t *testing.T) {
ctx := context.Background()
ctx := t.Context()
cm := &containerMock{}
sarm := &stepActionRemoteMocks{}
clonedAction := false
setCloneExecutor(t, func(input git.NewGitCloneExecutorInput) common.Executor {
return func(ctx context.Context) error {
clonedAction = true
return nil
}
actionDir := t.TempDir()
setFetch(t, func(_ctx context.Context, _input git.NewGitCloneExecutorInput) (string, string, error) {
clonedAction = true
return actionDir, "", nil
})
sar := &stepActionRemote{
RunContext: &RunContext{
Config: &Config{
GitHubInstance: "github.com",
ActionCacheDir: "/tmp/test-cache",
ActionCacheDir: t.TempDir(),
},
Run: &model.Run{
JobID: "1",
@@ -213,10 +208,10 @@ func TestStepActionRemote(t *testing.T) {
sar.RunContext.ExprEval = sar.RunContext.NewExpressionEvaluator(ctx)
if tt.mocks.read {
sarm.On("readAction", sar.Step, actionDirSuffix(sar.Step.UsesHash()), "", mock.Anything, mock.Anything).Return(&model.Action{}, nil)
sarm.On("readAction", sar.Step, actionDir, "", mock.Anything, mock.Anything).Return(&model.Action{}, nil)
}
if tt.mocks.run {
sarm.On("runAction", sar, actionDirSuffix(sar.Step.UsesHash()), mustNewRemoteAction(t, sar.Step.Uses)).Return(func(ctx context.Context) error { return tt.runError })
sarm.On("runAction", sar, actionDir, mustNewRemoteAction(t, sar.Step.Uses)).Return(func(ctx context.Context) error { return tt.runError })
cm.On("Copy", "/var/run/act", mock.AnythingOfType("[]*container.FileEntry")).Return(noopExecutor)
@@ -263,40 +258,60 @@ func TestStepActionRemote(t *testing.T) {
}
func TestStepActionRemotePrepare(t *testing.T) {
for _, test := range []struct {
name, uses, instance, actionPath, wantURL string
}{
{name: "nested action", uses: "org/repo/path@ref", instance: "https://github.com", actionPath: "path", wantURL: "https://github.com/org/repo"},
{name: "instance fallback", uses: "actions/setup-go@v4", instance: "gitea.example", wantURL: "https://gitea.example/actions/setup-go"},
} {
t.Run(test.name, func(t *testing.T) {
var actualURL string
setCloneExecutor(t, func(input git.NewGitCloneExecutorInput) common.Executor {
return func(context.Context) error {
actualURL = input.URL
return nil
}
})
actionMocks := &stepActionRemoteMocks{}
action := &stepActionRemote{
Step: &model.Step{Uses: test.uses},
RunContext: &RunContext{
Config: &Config{GitHubInstance: test.instance, ActionCacheDir: t.TempDir()},
Run: &model.Run{JobID: "1", Workflow: &model.Workflow{
Jobs: map[string]*model.Job{"1": {}},
}},
},
readAction: actionMocks.readAction,
}
actionMocks.On("readAction", action.Step, actionDirSuffix(action.Step.UsesHash()), test.actionPath,
mock.Anything, mock.Anything).Return(&model.Action{}, nil)
require.NoError(t, action.prepareActionExecutor()(t.Context()))
assert.Equal(t, test.wantURL, actualURL)
actionMocks.AssertExpectations(t)
})
var fetched, read []string
setFetch(t, func(_ctx context.Context, input git.NewGitCloneExecutorInput) (string, string, error) {
fetched = append(fetched, input.URL+"@"+input.Ref)
if input.Ref == "missing" {
return "", "", assert.AnError
}
return filepath.Join(input.Dir, "commit"), "sha-" + input.Ref, nil
})
newJob := func(instance string) *RunContext {
return &RunContext{
Config: &Config{GitHubInstance: instance, ActionCacheDir: t.TempDir()},
Run: &model.Run{JobID: "1", Workflow: &model.Workflow{Jobs: map[string]*model.Job{"1": {}}}},
JobContainer: &jobContainerMock{},
}
}
prepare := func(rc *RunContext, uses string) (*stepActionRemote, error) {
sar := &stepActionRemote{Step: &model.Step{Uses: uses}, RunContext: rc, readAction: func(_ctx context.Context, _step *model.Step, _actionDir, actionPath string, _readFile actionYamlReader, _writeFile fileWriter) (*model.Action, error) {
read = append(read, actionPath)
return &model.Action{}, nil
}}
return sar, sar.prepareActionExecutor()(t.Context())
}
job := newJob("https://github.com")
restore, err := prepare(job, "org/repo/restore@v1")
require.NoError(t, err)
root, err := prepare(job, "org/repo@v1")
require.NoError(t, err)
nested, err := prepare(&RunContext{Config: job.Config, Run: job.Run, JobContainer: job.JobContainer, Parent: job}, "org/repo/save@v1")
require.NoError(t, err)
other, err := prepare(job, "org/repo@v2")
require.NoError(t, err)
_, err = prepare(newJob("gitea.example"), "org/repo@v1")
require.NoError(t, err)
for range 2 {
_, err = prepare(job, "org/repo@missing")
require.ErrorIs(t, err, assert.AnError)
}
assert.Equal(t, []string{"https://github.com/org/repo@v1", "https://github.com/org/repo@v2", "https://gitea.example/org/repo@v1", "https://github.com/org/repo@missing"}, fetched)
assert.Equal(t, []string{"restore", "", "save", "", ""}, read)
assert.Equal(t, root.actionDir, restore.actionDir)
assert.Equal(t, root.actionDir, nested.actionDir)
assert.NotEqual(t, root.actionDir, other.actionDir)
reference, sha := restore.actionDownloadInfo()
assert.Equal(t, []string{"org/repo@v1", "sha-v1"}, []string{reference, sha})
_, sha = root.actionDownloadInfo()
assert.Empty(t, sha)
_, rootContainerDir := getContainerActionPaths(root)
_, restoreContainerDir := getContainerActionPaths(restore)
assert.NotEqual(t, path.Dir(rootContainerDir), path.Dir(restoreContainerDir))
assert.Equal(t, "restore", path.Base(restoreContainerDir))
}
func TestStepActionRemotePost(t *testing.T) {
@@ -438,7 +453,7 @@ func TestStepActionRemotePost(t *testing.T) {
for _, tt := range table {
t.Run(tt.name, func(t *testing.T) {
ctx := context.Background()
ctx := t.Context()
cm := &containerMock{}
@@ -447,7 +462,7 @@ func TestStepActionRemotePost(t *testing.T) {
RunContext: &RunContext{
Config: &Config{
GitHubInstance: "https://github.com",
ActionCacheDir: "/tmp/test-cache",
ActionCacheDir: t.TempDir(),
},
JobContainer: cm,
Run: &model.Run{
@@ -688,6 +703,7 @@ func Test_newRemoteActionSelfRepo(t *testing.T) {
Ref: "v1",
},
},
{name: "unknown instance", action: "$/.gitea/actions/build", github: &model.GithubContext{Repository: "owner/workflow-repo", Sha: "abc123"}},
{name: "empty path", action: "$/", github: workflow},
{name: "ref suffix is not allowed", action: "$/.gitea/actions/build@v1", github: workflow},
{name: "path traversal", action: "$/../escape", github: workflow},
@@ -700,17 +716,15 @@ func Test_newRemoteActionSelfRepo(t *testing.T) {
}
}
func Test_stepActionRemoteSelfRepoActionDir(t *testing.T) {
dirFor := func(repo string) string {
func Test_stepActionRemoteSelfRepoContainerKeyDiffersPerRepo(t *testing.T) {
keyFor := func(repo string) string {
sar := &stepActionRemote{
Step: &model.Step{Uses: "$/.gitea/actions/build"},
RunContext: &RunContext{Config: &Config{ActionCacheDir: "/cache"}},
remoteAction: &remoteAction{Org: "owner", Repo: repo, Ref: "v1"},
}
return sar.actionDir()
return sar.containerKey()
}
// The same `$/x` in two repos must not share a cache directory.
assert.NotEqual(t, dirFor("one"), dirFor("two"))
assert.NotEqual(t, keyFor("one"), keyFor("two"))
}
func Test_remoteActionReference(t *testing.T) {
@@ -771,39 +785,13 @@ func TestStepActionRemotePreResolvesDownloadedCommit(t *testing.T) {
readAction: readActionImpl,
}
require.NoError(t, sar.prepareActionExecutor()(context.Background()))
require.NoError(t, sar.prepareActionExecutor()(t.Context()))
reference, sha, ok := sar.actionDownloadInfo()
assert.True(t, ok)
reference, sha := sar.actionDownloadInfo()
assert.Equal(t, "actions/setup-go@main", reference)
assert.Equal(t, wantSha, sha)
}
func TestStepActionRemoteActionDownloadInfo(t *testing.T) {
t.Run("reports the action and its resolved commit", func(t *testing.T) {
sar := &stepActionRemote{
remoteAction: mustNewRemoteAction(t, "actions/checkout@v7"),
action: &model.Action{},
resolvedSha: "9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0",
}
reference, sha, ok := sar.actionDownloadInfo()
assert.True(t, ok)
assert.Equal(t, "actions/checkout@v7", reference)
assert.Equal(t, "9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0", sha)
})
t.Run("reports nothing when no action was downloaded", func(t *testing.T) {
// The local checkout of the workflow's own repository resolves no action.
sar := &stepActionRemote{remoteAction: mustNewRemoteAction(t, "actions/checkout@v7")}
_, _, ok := sar.actionDownloadInfo()
assert.False(t, ok)
})
}
func Test_safeFilename(t *testing.T) {
tests := []struct {
s string
@@ -852,12 +840,13 @@ func TestStepActionRemoteCloneTokenSurvivesNilSecrets(t *testing.T) {
for _, tt := range table {
t.Run(tt.name, func(t *testing.T) {
ctx := context.Background()
ctx := t.Context()
var capturedToken string
setCloneExecutor(t, func(input git.NewGitCloneExecutorInput) common.Executor {
actionDir := t.TempDir()
setFetch(t, func(_ctx context.Context, input git.NewGitCloneExecutorInput) (string, string, error) {
capturedToken = input.Token
return func(ctx context.Context) error { return nil }
return actionDir, "", nil
})
sarm := &stepActionRemoteMocks{}
@@ -867,7 +856,7 @@ func TestStepActionRemoteCloneTokenSurvivesNilSecrets(t *testing.T) {
Config: &Config{
GitHubInstance: tt.gitHubInstance,
DefaultActionInstance: tt.defaultActionInstance,
ActionCacheDir: "/tmp/test-cache",
ActionCacheDir: t.TempDir(),
// Mirrors the state of a composite RunContext: job secrets are
// stripped, but the job token is still reachable via Config.Token.
Secrets: nil,
@@ -885,7 +874,7 @@ func TestStepActionRemoteCloneTokenSurvivesNilSecrets(t *testing.T) {
}
sar.RunContext.ExprEval = sar.RunContext.NewExpressionEvaluator(ctx)
sarm.On("readAction", sar.Step, actionDirSuffix(sar.Step.UsesHash()), "", mock.Anything, mock.Anything).Return(&model.Action{}, nil)
sarm.On("readAction", sar.Step, actionDir, "", mock.Anything, mock.Anything).Return(&model.Action{}, nil)
err := sar.prepareActionExecutor()(ctx)
require.NoError(t, err)

View File

@@ -56,6 +56,9 @@ func (sd *stepDocker) runUsesContainer() common.Executor {
step := sd.Step
return func(ctx context.Context) error {
if rc.kubernetes {
return errKubernetesDocker
}
image := strings.TrimPrefix(step.Uses, "docker://")
cmd, err := shellquote.Split(step.With["args"])
if err != nil {

View File

@@ -6,8 +6,8 @@ package runner
import (
"bytes"
"context"
"io"
"slices"
"strings"
"testing"
@@ -34,7 +34,7 @@ func TestStepDockerMain(t *testing.T) {
ContainerNewContainer = origContainerNewContainer
}()
ctx := context.Background()
ctx := t.Context()
sd := &stepDocker{
RunContext: &RunContext{
@@ -123,7 +123,7 @@ func TestStepDockerNewStepContainerAllocatePTY(t *testing.T) {
ContainerNewContainer = origContainerNewContainer
}()
ctx := context.Background()
ctx := t.Context()
sd := &stepDocker{
RunContext: &RunContext{
StepResults: map[string]*model.StepResult{},
@@ -151,22 +151,35 @@ func TestStepDockerNewStepContainerAllocatePTY(t *testing.T) {
}
}
func TestStepDockerNewStepContainerNetworkMode(t *testing.T) {
func TestStepDockerNewStepContainerNetworkModeAndPath(t *testing.T) {
inheritedPath := (&container.LinuxContainerEnvironmentExtensions{}).DefaultPathVariable()
cases := []struct {
name string
platform string
path string
expectDefault bool
keepsPath bool
}{
{
name: "docker mode attaches to job container network",
platform: "node:14",
path: inheritedPath,
expectDefault: false,
keepsPath: true,
},
{
name: "host mode uses default network",
name: "host mode uses default network and the image's PATH",
platform: "-self-hosted",
path: inheritedPath,
expectDefault: true,
},
{
name: "host mode keeps a PATH that is not the runner's",
platform: "-self-hosted",
path: "/custom/bin",
expectDefault: true,
keepsPath: true,
},
}
for _, tc := range cases {
@@ -183,7 +196,7 @@ func TestStepDockerNewStepContainerNetworkMode(t *testing.T) {
ContainerNewContainer = origContainerNewContainer
}()
ctx := context.Background()
ctx := t.Context()
platform := tc.platform
sd := &stepDocker{
@@ -208,6 +221,7 @@ func TestStepDockerNewStepContainerNetworkMode(t *testing.T) {
ID: "1",
Uses: "docker://alpine:3.20",
},
env: map[string]string{"PATH": tc.path},
}
sd.RunContext.ExprEval = sd.RunContext.NewExpressionEvaluator(ctx)
@@ -215,6 +229,7 @@ func TestStepDockerNewStepContainerNetworkMode(t *testing.T) {
assert.Equal(t, tc.expectDefault, sd.RunContext.IsHostEnv(), "IsHostEnv mismatch for platform %q", tc.platform)
newStepContainer(ctx, sd, "alpine:3.20", []string{"echo", "hello"}, nil, "")
assert.Equal(t, tc.keepsPath, slices.Contains(captured.Env, "PATH="+tc.path))
if tc.expectDefault {
assert.Equal(t, "default", captured.NetworkMode,

View File

@@ -14,12 +14,12 @@ import (
"strings"
"gitea.com/gitea/runner/act/common"
"gitea.com/gitea/runner/act/common/logging"
"gitea.com/gitea/runner/act/container"
"gitea.com/gitea/runner/act/lookpath"
"gitea.dev/actionslib/pkg/model"
"github.com/kballard/go-shellquote"
"github.com/sirupsen/logrus"
yaml "go.yaml.in/yaml/v4"
)
@@ -58,8 +58,8 @@ func (sr *stepRun) main() common.Executor {
// printRunScriptActionDetails mirrors actions/runner ScriptHandler.PrintActionDetails
// for script steps.
func (sr *stepRun) printRunScriptActionDetails(ctx context.Context) {
rawLogger := common.Logger(ctx).WithField(rawOutputField, true)
scriptLineLogger := rawLogger.WithField(scriptLineCyanField, true)
rawLogger := common.Logger(ctx).With(rawOutputField, true)
scriptLineLogger := rawLogger.With(scriptLineCyanField, true)
normalized := strings.TrimRight(strings.ReplaceAll(sr.interpolatedScript, "\r\n", "\n"), "\n")
@@ -84,7 +84,7 @@ func printRunActionHeader(ctx context.Context, step *model.Step, env map[string]
if step == nil {
return
}
rawLogger := common.Logger(ctx).WithField(rawOutputField, true)
rawLogger := common.Logger(ctx).With(rawOutputField, true)
title := step.Uses
if step.Name != "" {
@@ -105,7 +105,7 @@ func printRunActionHeader(ctx context.Context, step *model.Step, env map[string]
// printStepEnvBlock emits the declared-env block (YAML order, internal vars filtered)
// shared by the run: and uses: "Run" headers.
func printStepEnvBlock(ctx context.Context, step *model.Step, env map[string]string, rc *RunContext) {
rawLogger := common.Logger(ctx).WithField(rawOutputField, true)
rawLogger := common.Logger(ctx).With(rawOutputField, true)
caseInsensitive := rc != nil && rc.JobContainer != nil && rc.JobContainer.IsEnvironmentCaseInsensitive()
var visible []string
for _, k := range stepDeclaredEnvKeysInOrder(step) {
@@ -135,7 +135,7 @@ func printStepEnvBlock(ctx context.Context, step *model.Step, env map[string]str
// logKeyedValue prints one row per line of value: Gitea stores one log row per line, so an
// embedded newline would reach the user as a literal "\n".
func logKeyedValue(rawLogger *logrus.Entry, key, value string) {
func logKeyedValue(rawLogger *logging.Logger, key, value string) {
lines := strings.Split(strings.ReplaceAll(value, "\r\n", "\n"), "\n")
rawLogger.Infof(" %s: %s", key, lines[0])
for _, line := range lines[1:] {

View File

@@ -6,14 +6,14 @@ package runner
import (
"bytes"
"context"
"io"
"strings"
"testing"
"gitea.com/gitea/runner/act/common"
"gitea.com/gitea/runner/act/common/logging"
"gitea.dev/actionslib/pkg/model"
"github.com/sirupsen/logrus"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
yaml "go.yaml.in/yaml/v4"
@@ -84,12 +84,9 @@ env:
require.NoError(t, yaml.Unmarshal([]byte(raw), &step))
buf := &bytes.Buffer{}
logger := logrus.New()
logger.SetOutput(buf)
logger.SetLevel(logrus.InfoLevel)
logger.SetFormatter(&jobLogFormatter{color: cyan})
entry := logger.WithFields(logrus.Fields{"job": "j1"})
ctx := common.WithLogger(context.Background(), entry)
logger := logging.New(logging.NewLevelHandler(newJobLogHandler(buf, cyan), logging.LevelInfo))
entry := logger.With("job", "j1")
ctx := common.WithLogger(t.Context(), entry)
sr := &stepRun{
Step: &step,
@@ -133,12 +130,9 @@ env:
require.NoError(t, yaml.Unmarshal([]byte(raw), &step))
buf := &bytes.Buffer{}
logger := logrus.New()
logger.SetOutput(buf)
logger.SetLevel(logrus.InfoLevel)
logger.SetFormatter(&jobLogFormatter{color: cyan})
entry := logger.WithFields(logrus.Fields{"job": "j1"})
ctx := common.WithLogger(context.Background(), entry)
logger := logging.New(logging.NewLevelHandler(newJobLogHandler(buf, cyan), logging.LevelInfo))
entry := logger.With("job", "j1")
ctx := common.WithLogger(t.Context(), entry)
printRunActionHeader(ctx, &step, map[string]string{"CUSTOM": "value", "GITHUB_TOKEN": "tok"}, &RunContext{})
@@ -166,17 +160,17 @@ func TestIsInternalEnvKey(t *testing.T) {
}
func TestPrintColoredScriptLineCyan(t *testing.T) {
f := &jobLogFormatter{color: cyan}
entry := &logrus.Entry{
Level: logrus.InfoLevel,
f := textFormatter{w: io.Discard, color: cyan}
record := logging.Record{
Level: logging.LevelInfo,
Message: "echo one",
Data: logrus.Fields{
"job": "j1",
rawOutputField: true,
scriptLineCyanField: true,
Attrs: []logging.Attr{
{Key: "job", Value: "j1"},
{Key: rawOutputField, Value: true},
{Key: scriptLineCyanField, Value: true},
},
}
buf := &bytes.Buffer{}
f.printColored(buf, entry)
f.printColored(buf, record)
assert.Equal(t, "\x1b[36m|\x1b[0m \x1b[36;1mecho one\x1b[0m", buf.String())
}

View File

@@ -68,7 +68,7 @@ func TestStepRun(t *testing.T) {
cm.On("UpdateFromEnv", "/var/run/act/workflow/outputcmd.txt", mock.AnythingOfType("*map[string]string")).Return(noopExecutor)
ctx := context.Background()
ctx := t.Context()
cm.On("GetContainerArchive", ctx, "/var/run/act/workflow/pathcmd.txt").Return(io.NopCloser(&bytes.Buffer{}), nil)

View File

@@ -7,18 +7,20 @@ package runner
import (
"context"
"errors"
"io"
"os"
"strings"
"testing"
"testing/synctest"
"gitea.com/gitea/runner/act/common"
"gitea.com/gitea/runner/act/common/logging"
"gitea.com/gitea/runner/act/common/logging/logtest"
"gitea.com/gitea/runner/act/container"
"gitea.com/gitea/runner/internal/pkg/telemetry"
"gitea.dev/actionslib/pkg/model"
runnerv1 "gitea.dev/actionslib/runner/v1"
log "github.com/sirupsen/logrus"
logrustest "github.com/sirupsen/logrus/hooks/test"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/mock"
"github.com/stretchr/testify/require"
@@ -164,8 +166,8 @@ func TestSetupEnv(t *testing.T) {
sm.On("getStepModel").Return(step)
sm.On("getEnv").Return(&env)
require.NoError(t, setupEnv(context.Background(), sm))
require.NoError(t, setupInputs(context.Background(), sm))
require.NoError(t, setupEnv(t.Context(), sm))
require.NoError(t, setupInputs(t.Context(), sm))
// These are commit or system specific
delete(env, "GITHUB_REF")
@@ -189,7 +191,6 @@ func TestSetupEnv(t *testing.T) {
"GITHUB_ACTION_PATH": "",
"GITHUB_ACTION_REF": "",
"GITHUB_ACTION_REPOSITORY": "",
"GITHUB_API_URL": "https:///api/v1", // Gitea uses api/v1 (upstream GitHub: api/v3)
"GITHUB_BASE_REF": "",
"GITHUB_EVENT_NAME": "",
"GITHUB_EVENT_PATH": "/var/run/act/workflow/event.json",
@@ -200,7 +201,6 @@ func TestSetupEnv(t *testing.T) {
"GITHUB_RUN_ATTEMPT": "",
"GITHUB_RUN_ID": "runId",
"GITHUB_RUN_NUMBER": "1",
"GITHUB_SERVER_URL": "https://",
"GITHUB_WORKFLOW": "",
"INPUT_ID": "1234567",
"INPUT_STEP_WITH": "with-value",
@@ -299,63 +299,63 @@ func TestIsStepEnabled(t *testing.T) {
}
}
log.SetLevel(log.DebugLevel)
logging.SetLevel(logging.LevelDebug)
assertObject := assert.New(t)
// success()
step := createTestStep(t, "if: success()")
assertObject.True(isStepEnabled(context.Background(), step.getIfExpression(context.Background(), stepStageMain), step, stepStageMain))
assertObject.True(isStepEnabled(t.Context(), step.getIfExpression(t.Context(), stepStageMain), step, stepStageMain))
step = createTestStep(t, "if: success()")
step.getRunContext().StepResults["a"] = &model.StepResult{
Conclusion: model.StepStatusSuccess,
}
assertObject.True(isStepEnabled(context.Background(), step.getStepModel().If.Value, step, stepStageMain))
assertObject.True(isStepEnabled(t.Context(), step.getStepModel().If.Value, step, stepStageMain))
step = createTestStep(t, "if: success()")
step.getRunContext().StepResults["a"] = &model.StepResult{
Conclusion: model.StepStatusFailure,
}
assertObject.False(isStepEnabled(context.Background(), step.getStepModel().If.Value, step, stepStageMain))
assertObject.False(isStepEnabled(t.Context(), step.getStepModel().If.Value, step, stepStageMain))
// failure()
step = createTestStep(t, "if: failure()")
assertObject.False(isStepEnabled(context.Background(), step.getStepModel().If.Value, step, stepStageMain))
assertObject.False(isStepEnabled(t.Context(), step.getStepModel().If.Value, step, stepStageMain))
step = createTestStep(t, "if: failure()")
step.getRunContext().StepResults["a"] = &model.StepResult{
Conclusion: model.StepStatusSuccess,
}
assertObject.False(isStepEnabled(context.Background(), step.getStepModel().If.Value, step, stepStageMain))
assertObject.False(isStepEnabled(t.Context(), step.getStepModel().If.Value, step, stepStageMain))
step = createTestStep(t, "if: failure()")
step.getRunContext().StepResults["a"] = &model.StepResult{
Conclusion: model.StepStatusFailure,
}
assertObject.True(isStepEnabled(context.Background(), step.getStepModel().If.Value, step, stepStageMain))
assertObject.True(isStepEnabled(t.Context(), step.getStepModel().If.Value, step, stepStageMain))
// always()
step = createTestStep(t, "if: always()")
assertObject.True(isStepEnabled(context.Background(), step.getStepModel().If.Value, step, stepStageMain))
assertObject.True(isStepEnabled(t.Context(), step.getStepModel().If.Value, step, stepStageMain))
step = createTestStep(t, "if: always()")
step.getRunContext().StepResults["a"] = &model.StepResult{
Conclusion: model.StepStatusSuccess,
}
assertObject.True(isStepEnabled(context.Background(), step.getStepModel().If.Value, step, stepStageMain))
assertObject.True(isStepEnabled(t.Context(), step.getStepModel().If.Value, step, stepStageMain))
step = createTestStep(t, "if: always()")
step.getRunContext().StepResults["a"] = &model.StepResult{
Conclusion: model.StepStatusFailure,
}
assertObject.True(isStepEnabled(context.Background(), step.getStepModel().If.Value, step, stepStageMain))
assertObject.True(isStepEnabled(t.Context(), step.getStepModel().If.Value, step, stepStageMain))
// neither env nor the step's own with: values are inputs, at any stage
step = createTestStep(t, "if: inputs.forged")
step.getRunContext().Env["INPUT_FORGED"] = "leaked"
*step.getEnv() = map[string]string{"INPUT_FORGED": "leaked"}
assertObject.False(isStepEnabled(context.Background(), step.getStepModel().If.Value, step, stepStageMain))
assertObject.False(isStepEnabled(context.Background(), step.getStepModel().If.Value, step, stepStagePost))
assertObject.False(isStepEnabled(t.Context(), step.getStepModel().If.Value, step, stepStageMain))
assertObject.False(isStepEnabled(t.Context(), step.getStepModel().If.Value, step, stepStagePost))
}
func TestIsContinueOnError(t *testing.T) {
@@ -383,49 +383,49 @@ func TestIsContinueOnError(t *testing.T) {
}
}
log.SetLevel(log.DebugLevel)
logging.SetLevel(logging.LevelDebug)
assertObject := assert.New(t)
// absent
step := createTestStep(t, "name: test")
continueOnError, err := isContinueOnError(context.Background(), step.getStepModel().RawContinueOnError, step, stepStageMain)
continueOnError, err := isContinueOnError(t.Context(), step.getStepModel().RawContinueOnError, step, stepStageMain)
assertObject.False(continueOnError)
assertObject.NoError(err) //nolint:testifylint // pre-existing issue from nektos/act
// explcit true
step = createTestStep(t, "continue-on-error: true")
continueOnError, err = isContinueOnError(context.Background(), step.getStepModel().RawContinueOnError, step, stepStageMain)
continueOnError, err = isContinueOnError(t.Context(), step.getStepModel().RawContinueOnError, step, stepStageMain)
assertObject.True(continueOnError)
assertObject.NoError(err) //nolint:testifylint // pre-existing issue from nektos/act
// explicit false
step = createTestStep(t, "continue-on-error: false")
continueOnError, err = isContinueOnError(context.Background(), step.getStepModel().RawContinueOnError, step, stepStageMain)
continueOnError, err = isContinueOnError(t.Context(), step.getStepModel().RawContinueOnError, step, stepStageMain)
assertObject.False(continueOnError)
assertObject.NoError(err) //nolint:testifylint // pre-existing issue from nektos/act
// expression true
step = createTestStep(t, "continue-on-error: ${{ 'test' == 'test' }}")
continueOnError, err = isContinueOnError(context.Background(), step.getStepModel().RawContinueOnError, step, stepStageMain)
continueOnError, err = isContinueOnError(t.Context(), step.getStepModel().RawContinueOnError, step, stepStageMain)
assertObject.True(continueOnError)
assertObject.NoError(err) //nolint:testifylint // pre-existing issue from nektos/act
// expression false
step = createTestStep(t, "continue-on-error: ${{ 'test' != 'test' }}")
continueOnError, err = isContinueOnError(context.Background(), step.getStepModel().RawContinueOnError, step, stepStageMain)
continueOnError, err = isContinueOnError(t.Context(), step.getStepModel().RawContinueOnError, step, stepStageMain)
assertObject.False(continueOnError)
assertObject.NoError(err) //nolint:testifylint // pre-existing issue from nektos/act
// the step's own with: values are not inputs
step = createTestStep(t, "continue-on-error: ${{ inputs.forged }}")
*step.getEnv() = map[string]string{"INPUT_FORGED": "true"}
continueOnError, err = isContinueOnError(context.Background(), step.getStepModel().RawContinueOnError, step, stepStageMain)
continueOnError, err = isContinueOnError(t.Context(), step.getStepModel().RawContinueOnError, step, stepStageMain)
assertObject.False(continueOnError)
require.NoError(t, err)
// expression parse error
step = createTestStep(t, "continue-on-error: ${{ 'test' != test }}")
continueOnError, err = isContinueOnError(context.Background(), step.getStepModel().RawContinueOnError, step, stepStageMain)
continueOnError, err = isContinueOnError(t.Context(), step.getStepModel().RawContinueOnError, step, stepStageMain)
assertObject.False(continueOnError)
assertObject.Error(err)
}
@@ -449,9 +449,9 @@ func TestRunStepExecutorDoesNotLeakRefusalToNextStep(t *testing.T) {
StepResults: map[string]*model.StepResult{},
JobContainer: cm,
}
rc.ExprEval = rc.NewExpressionEvaluator(context.Background())
rc.ExprEval = rc.NewExpressionEvaluator(t.Context())
// Dryrun skips reading the path file back from the (mocked) container.
ctx := common.WithDryrun(context.Background(), true)
ctx := common.WithDryrun(t.Context(), true)
// A refusal parsed out of the job container's own output belongs to no step, so the
// first step must not be failed by it.
@@ -475,11 +475,31 @@ func TestRunStepExecutorDoesNotLeakRefusalToNextStep(t *testing.T) {
require.NoError(t, errB)
}
type fileCommandTestContainer struct {
*container.HostEnvironment
}
func (c *fileCommandTestContainer) UpdateFromEnv(srcPath string, env *map[string]string) common.Executor {
return func(ctx context.Context) error {
if err := ctx.Err(); err != nil {
return err
}
return c.HostEnvironment.UpdateFromEnv(srcPath, env)(ctx)
}
}
func (c *fileCommandTestContainer) GetContainerArchive(ctx context.Context, srcPath string) (io.ReadCloser, error) {
if err := ctx.Err(); err != nil {
return nil, err
}
return c.HostEnvironment.GetContainerArchive(ctx, srcPath)
}
func TestRunStepExecutorParity(t *testing.T) {
newStep := func(t *testing.T, stepModel *model.Step) *stepRun {
rc := createRunContext(t)
rc.JobContainer = &container.HostEnvironment{ActPath: t.TempDir()}
rc.ExprEval = rc.NewExpressionEvaluator(context.Background())
rc.ExprEval = rc.NewExpressionEvaluator(t.Context())
return &stepRun{RunContext: rc, Step: stepModel, env: map[string]string{}}
}
badExpression := "${{ 'test' != test }}"
@@ -493,30 +513,48 @@ func TestRunStepExecutorParity(t *testing.T) {
} {
t.Run(test.name, func(t *testing.T) {
step := newStep(t, test.step)
logger, hook := logrustest.NewNullLogger()
err := runStepExecutor(step, stepStageMain, test.executor)(common.WithLogger(context.Background(), logger))
logger, hook := logtest.NewLogger()
err := runStepExecutor(step, stepStageMain, test.executor)(common.WithLogger(t.Context(), logger))
require.ErrorContains(t, err, test.wantError)
assert.Equal(t, model.StepStatusFailure, step.RunContext.StepResults[test.step.ID].Conclusion)
assert.Equal(t, model.StepStatusFailure, hook.LastEntry().Data["stepResult"])
assert.Equal(t, model.StepStatusFailure, lastRecordAttr(hook, "stepResult"))
})
}
t.Run("only a step that runs is marked started", func(t *testing.T) {
for _, test := range []struct {
condition string
started int
}{{"true", 1}, {"false", 0}} {
logger, hook := logtest.NewLogger()
step := newStep(t, &model.Step{ID: "pre", If: yaml.Node{Value: test.condition}})
require.NoError(t, runStepExecutor(step, stepStagePre, noopExecutor)(common.WithLogger(context.Background(), logger)))
started := 0
for _, entry := range hook.AllRecords() {
if v, _ := entry.Attr(StepStartedField); v == true {
started++
}
}
assert.Equal(t, test.started, started, test.condition)
}
})
t.Run("file command error honors continue-on-error", func(t *testing.T) {
step := newStep(t, &model.Step{ID: "commands", RawContinueOnError: "true"})
logger, hook := logrustest.NewNullLogger()
logger, hook := logtest.NewLogger()
err := runStepExecutor(step, stepStageMain, func(context.Context) error {
require.NoError(t, os.WriteFile(step.env["GITHUB_ENV"], []byte("GOOD=1\nmalformed\n"), 0o600))
require.NoError(t, os.WriteFile(step.env["GITHUB_OUTPUT"], []byte("kept=value\n"), 0o600))
require.NoError(t, os.WriteFile(step.env["GITHUB_STATE"], []byte("saved=value\n"), 0o600))
return nil
})(common.WithLogger(context.Background(), logger))
})(common.WithLogger(t.Context(), logger))
require.NoError(t, err)
result := step.RunContext.StepResults[step.Step.ID]
assert.Equal(t, model.StepStatusFailure, result.Outcome)
assert.Equal(t, model.StepStatusSuccess, result.Conclusion)
assert.Equal(t, model.StepStatusSuccess, hook.LastEntry().Data["stepResult"])
assert.Equal(t, model.StepStatusSuccess, lastRecordAttr(hook, "stepResult"))
assert.Equal(t, "1", step.RunContext.Env["GOOD"])
assert.Equal(t, "value", result.Outputs["kept"])
assert.Equal(t, "value", step.RunContext.IntraActionState[step.Step.ID]["saved"])
@@ -527,13 +565,37 @@ func TestRunStepExecutorParity(t *testing.T) {
}
})
t.Run("file commands survive a job timeout", func(t *testing.T) {
synctest.Test(t, func(t *testing.T) {
step := newStep(t, &model.Step{ID: "interrupted"})
ctx, cancel := applyJobTimeout(t.Context(), step.RunContext, &model.Job{TimeoutMinutes: "1"})
defer cancel()
step.RunContext.JobContainer = &fileCommandTestContainer{step.RunContext.JobContainer.(*container.HostEnvironment)}
err := runStepExecutor(step, stepStageMain, func(execCtx context.Context) error {
for name, content := range map[string]string{
"GITHUB_ENV": "KEPT_ENV=value\n", "GITHUB_STATE": "saved=value\n",
"GITHUB_OUTPUT": "kept=value\n", "GITHUB_PATH": "/kept/bin\n",
} {
require.NoError(t, os.WriteFile(step.env[name], []byte(content), 0o600))
}
<-execCtx.Done()
return execCtx.Err()
})(ctx)
require.ErrorIs(t, err, context.DeadlineExceeded)
assert.Equal(t, "value", step.RunContext.Env["KEPT_ENV"])
assert.Equal(t, "value", step.RunContext.IntraActionState[step.Step.ID]["saved"])
assert.Equal(t, "value", step.RunContext.StepResults[step.Step.ID].Outputs["kept"])
assert.Contains(t, step.RunContext.ExtraPath, "/kept/bin")
})
})
t.Run("composite child files do not leak", func(t *testing.T) {
outer := newStep(t, &model.Step{ID: "outer"})
childRC := &RunContext{
Config: outer.RunContext.Config, Run: outer.RunContext.Run, Env: map[string]string{}, StepResults: map[string]*model.StepResult{},
JobContainer: outer.RunContext.JobContainer, Parent: outer.RunContext,
}
childRC.ExprEval = childRC.NewExpressionEvaluator(context.Background())
childRC.ExprEval = childRC.NewExpressionEvaluator(t.Context())
child := &stepRun{RunContext: childRC, Step: &model.Step{ID: "child"}, env: map[string]string{}}
err := runStepExecutor(outer, stepStageMain, func(ctx context.Context) error {
@@ -544,7 +606,7 @@ func TestRunStepExecutorParity(t *testing.T) {
})(ctx))
outer.RunContext.setOutput(ctx, map[string]string{"name": "declared"}, "outer")
return nil
})(context.Background())
})(t.Context())
require.NoError(t, err)
assert.Equal(t, map[string]string{"declared": "outer"}, outer.RunContext.StepResults[outer.Step.ID].Outputs)
@@ -577,12 +639,17 @@ func TestRunStepExecutorParity(t *testing.T) {
})
t.Run("timeout must be positive", func(t *testing.T) {
exprEval := createRunContext(t).NewExpressionEvaluator(context.Background())
exprEval := createRunContext(t).NewExpressionEvaluator(t.Context())
for timeout, wantDeadline := range map[string]bool{"-1": false, "0": false, "1": true} {
ctx, cancel := evaluateStepTimeout(context.Background(), exprEval, &model.Step{TimeoutMinutes: timeout})
ctx, cancel := evaluateStepTimeout(t.Context(), exprEval, &model.Step{TimeoutMinutes: timeout})
_, hasDeadline := ctx.Deadline()
cancel()
assert.Equal(t, wantDeadline, hasDeadline, timeout)
}
})
}
func lastRecordAttr(h *logtest.Handler, key string) any {
v, _ := h.LastRecord().Attr(key)
return v
}

View File

@@ -37,10 +37,10 @@ async function main() {
await request('POST', `${api}/networks/${name}/connect`, {Container: job});
fs.appendFileSync(process.env.GITHUB_STATE, 'post=true\n');
}
const label = process.env.PROXY_TEST_MODE === 'proxy' ? job : undefined;
assert.equal(JSON.parse(await request('GET', `${api}/volumes/${name}`)).Labels?.['com.gitea.runner.job'], label);
const label = process.env.PROXY_TEST_MODE === 'proxy' ? '' : undefined;
assert.equal(JSON.parse(await request('GET', `${api}/volumes/${name}`)).Labels?.[`com.gitea.runner.job.${job}`], label);
const network = JSON.parse(await request('GET', `${api}/networks/${name}`));
assert.equal(network.Labels?.['com.gitea.runner.job'], label);
assert.equal(network.Labels?.[`com.gitea.runner.job.${job}`], label);
assert(Object.values(network.Containers).some((container) => container.Name === job));
if (post) {
console.log('docker proxy post verified');

View File

@@ -52,5 +52,5 @@ jobs:
- run: |
docker run --rm --user 1001 --group-add 2375 -v /var/run/docker.sock:/var/run/docker.sock -v /usr/local/bin/docker:/usr/local/bin/docker:ro -v "${GITEA_DOCKER_WORKSPACE:?}/data:/data:ro" \
"$PROXY_TEST_IMAGE" sh -c 'grep -qx from-job /data/marker && docker volume create "$0"' "$PROXY_TEST_RESOURCE-nested"
test "$(docker volume inspect -f '{{ index .Labels "com.gitea.runner.job" }}' "$PROXY_TEST_RESOURCE-nested")" = "$JOB_CONTAINER_NAME"
test "$(docker volume ls -q -f "label=com.gitea.runner.job.$JOB_CONTAINER_NAME" -f "name=$PROXY_TEST_RESOURCE-nested")" = "$PROXY_TEST_RESOURCE-nested"
echo "docker binds verified"

View File

@@ -5,6 +5,7 @@ jobs:
test:
runs-on: ubuntu-latest
steps:
- run: printf 'workspace=from-host-step\n' > workspace-marker
- uses: docker://node:24-bookworm-slim
with:
somekey: somevalue
@@ -16,8 +17,10 @@ jobs:
with:
entrypoint: /bin/sh
args: -c -- uname -a
- uses: docker://node:24-bookworm-slim
- id: workspace
uses: docker://node:24-bookworm-slim
with:
entrypoint: /bin/sh
args: -c 'uname -a'
args: -c 'uname -a && cat "$GITHUB_WORKSPACE/workspace-marker" >> "$GITHUB_OUTPUT"'
- run: test '${{ steps.workspace.outputs.workspace }}' = 'from-host-step'

View File

@@ -8,14 +8,32 @@ Built-in actions ship with the runner, so a job needs no action download and no
## checkout
Works like `actions/checkout` with these inputs. Any other input fails the step.
Works like `actions/checkout` with these inputs. Any other input fails the step. It runs `git` where the job runs, see the [prerequisites](../README.md#prerequisites).
| Input | Default |
| --- | --- |
| `repository` | `github.repository` |
| `ref` | the triggering commit, or the default branch of another repository |
| `ref` | the triggering ref and commit, or the default branch of another repository |
| `token` | `github.token` |
| `ssh-key` | empty, a private key fetches over SSH instead |
| `ssh-known-hosts` | empty, host keys trusted besides `~/.ssh/known_hosts` |
| `ssh-strict` | `true`, rejects unknown host keys |
| `ssh-user` | the user of the repository's SSH URL, else `git` |
| `persist-credentials` | `true`, later steps of the job can fetch and push with `token` or `ssh-key` |
| `path` | workspace root |
| `fetch-depth` | `1`, `0` for full history |
| `clean` | `true`, removes untracked files, also in submodules |
| `filter` | empty, a partial clone filter such as `blob:none` |
| `sparse-checkout` | empty, the paths to check out, one per line |
| `sparse-checkout-cone-mode` | `true`, `false` reads `sparse-checkout` as patterns |
| `fetch-depth` | `1`, `0` for the full history of every branch and tag |
| `fetch-tags` | `false`, `true` fetches every tag |
| `show-progress` | `true`, shows the fetch progress |
| `lfs` | `false`, `true` downloads Git LFS files |
| `submodules` | `false`, `true` checks out submodules, `recursive` also theirs |
| `set-safe-directory` | `true`, lets checkout's `git` work in a directory another user owns |
With `gitea-runner exec`, checking out the workflow's own repository copies your local working directory, unless `--no-skip-checkout` is set.
A branch is checked out as a local branch tracking `origin`, a tag or commit as a detached `HEAD`. A triggering tag that moved since the event fails the step. The step outputs `ref` and `commit` like `actions/checkout`.
Persisted credentials end with the job. `token` is only sent to `github.server_url`, while `ssh-key` is offered to any SSH host, as with `actions/checkout`. The runner's `client_cert_file` is only offered to `github.server_url` and only during the step, so later steps do not get it. Without `ssh-key`, submodule URLs of the form `git@<server host>:` are fetched from `github.server_url` with `token`. With `ssh-key`, the repository is fetched from the SSH URL in the event's `repository.ssh_url`, or from the server's host on port 22.
With `gitea-runner exec`, checking out the workflow's own repository copies your local working directory, unless `--no-skip-checkout` or `ssh-key` is set.

58
docs/kubernetes.md Normal file
View File

@@ -0,0 +1,58 @@
# Jobs as Kubernetes pods
A `kubernetes` label runs each job as a pod, without a Docker daemon:
```text
ubuntu-latest:kubernetes://docker.gitea.com/runner-images:ubuntu-latest
```
The pod has a `job` container from the label's image or the job's `container.image`, plus one container per service.
## Setup
In a cluster, the runner uses its pod's service account and namespace. [`examples/kubernetes/kubernetes-pods.yaml`](../examples/kubernetes/kubernetes-pods.yaml) deploys a runner with these permissions:
| Resource | Verbs |
| --- | --- |
| `pods` | `create`, `get`, `list`, `delete` |
| `pods/exec` | `create`, `get` |
| `pods/log` | `get` |
| `secrets` | `create`, `delete` |
Outside a cluster, it uses the current context of the first file in `$KUBECONFIG`, or of `~/.kube/config`, which must authenticate with a token, token file or client certificate. The cluster must run Kubernetes 1.30 or later, and job images need `sh`, `tar`, `tail` and `env`.
## Configuration
```yaml
kubernetes:
kubeconfig: "" # instead of the service account
namespace: "" # defaults to the service account's, the kubeconfig context's, or default
pod_template: # merged under each job pod, a container named job merges into the job container, one named $services into each service container
spec:
containers:
- name: job
resources:
limits:
memory: 4Gi
- name: $services
resources:
limits:
memory: 1Gi
pod_templates: # merged over pod_template for jobs whose runs-on has the label they are keyed by
gpu:
spec:
nodeSelector:
nvidia.com/gpu.present: "true"
```
A job with `runs-on: [linux, gpu]` gets the `linux` and then the `gpu` entry of `pod_templates` merged over `pod_template`, while its image comes from the first of its labels the runner has. A template volume named `workspace` or `act` replaces the `emptyDir` the runner mounts at the workspace or at `/var/run/act`, for example to limit its size. Images are pulled by the cluster's `imagePullPolicy` default, a template's, or always with `container.force_pull`. Like every config value, templates expand `${NAME}` from the runner's environment, so a shell command in a template writes `$${NAME}` for the pod's own variable.
Job pods mount no service account token unless the template sets `automountServiceAccountToken: true`. Job pods and their Secrets are labeled `app.kubernetes.io/managed-by: gitea-runner` and `com.gitea.runner.uuid: <runner UUID>`. The idle cleanup removes those the runner did not get to remove, once they are older than `runner.workdir_cleanup_age`.
## Services
Services share the pod's network, so a step reaches one by its id or `localhost`, and two services cannot listen on the same port. An id is lowercased and has characters other than letters, digits and `-` replaced by `-`, so the service `my_db` is reached as `my-db`. Pass secrets to services through `env`, as `command` is visible in the pod spec. Of `options`, only `--health-*` is supported. They become a startup probe with Docker's defaults, which stops a service that fails its retries, so the job fails as on Docker.
## Unsupported
Docker actions, `docker://` steps, and the `options`, `volumes` and `credentials` of the job container fail the job, as do `volumes` and `credentials` of services. Run such workflows on a [Docker in Docker](../examples/kubernetes/README.md) runner. Pull private images with `imagePullSecrets` in `pod_template`.

Some files were not shown because too many files have changed in this diff Show More