Without a docker daemon six `act/runner` tests failed instead of passing. A dry run still created and removed job networks, the job teardown dropped the dry-run flag along with every other context value, and the container mocks asked the daemon for the runner arch.
This also speeds up the tests and CI without dropping assertions. Most of the time went to git and docker round trips, which are now pinned, batched or coalesced, and the dind and e2e steps cache or overlap more of their setup. On CI, `act/runner` went from 77s to 55s and `test-dind` from 108s to 55s. Locally, `act/runner` without docker went from 53s to 7s.
Split out of https://gitea.com/gitea/runner/pulls/1348, which adds Windows CI on top. The go cache changes moved to https://gitea.com/gitea/runner/pulls/1350.
Written by Claude Code.
---------
Co-authored-by: bircni <bircni@icloud.com>
Reviewed-on: https://gitea.com/gitea/runner/pulls/1349
Reviewed-by: bircni <bircni@icloud.com>
Co-authored-by: silverwind <me@silverwind.io>
The toolkit patch rewrote bundles in place in the shared action cache. Since https://gitea.com/gitea/runner/pulls/1316 trees are never refreshed, so a bundle left truncated failed every later job using it.
Patched bundles now only overwrite the job container's copy, so the runner no longer writes to an extracted tree.
Extraction records each file's size beside the tree, and a tree that no longer matches is fetched again, offline mode included, as a missing one always was. This also catches damage from outside the runner, such as storage losing a write, for a stat per file, a fraction of the copy each stage already makes. Trees from older runners have no record and download once more, except in offline mode, which keeps using them as before.
Generated with Claude Code.
Reviewed-on: https://gitea.com/gitea/runner/pulls/1352
Reviewed-by: Lunny Xiao <xiaolunwen@gmail.com>
Reviewed-by: bircni <bircni@icloud.com>
Co-authored-by: Zettat123 <zettat123@gmail.com>
Contains a number of small bug fixes where jobs behaved differently from GitHub or failed silently.
1. Hooks fail on a refused `::set-env::` or `::add-path::` and no longer inherit the last step's identity
1. Docker build, exec and wait errors are returned instead of swallowed, token-only registry credentials work, logs of `--tty` services decode
1. Env files split lines like GitHub on each OS, so a carriage return is kept on Linux and CRLF ends a line on Windows, empty heredoc names or delimiters are rejected
1. Post steps run after their pre step even when main was skipped, composites see undeclared inputs, `runs.args` wins without `with.args` being parsed
1. Docker actions skip evaluating defaults of inputs given in `with` and read `entryPoint` and `args` in any case, a lowercase `dockerfile` is found, undeclared inputs get GitHub's warning
1. A user-set `CI` is kept, `GITHUB_REPOSITORY_ID` and the other missing `GITHUB_*` variables are exported
1. `timeout-minutes` can read the step's own `env`, a step-level `ACTIONS_ALLOW_UNSECURE_COMMANDS: false` beats a job-level `true` while the runner's own opt-in still wins, `pwsh` quotes the script path
1. A step `env:` given as one expression such as `${{ fromJSON(matrix.env) }}` is applied, `hashFiles` fails with GitHub's messages on script errors and after 120 seconds, a job image without node still gets an empty hash
1. A step whose command files cannot be reset fails instead of importing stale ones, a timeout between steps fails the job instead of reporting success
1. One named volume can be mounted at several paths, host-mode PTY steps keep their last output when they fail, host scratch directories are removed when job setup fails
1. Cancelling a step right after it starts kills its whole process tree
1. Cache prefix restores pick the newest entry created within the same second, also on S3, cache commits with a wrong size are rejected
1. Artifact chunks are written at their offset, download URLs are escaped, listing a run without artifacts no longer crashes
1. Poll backoff reaches `fetch_interval_max`, proxy passwords are masked in more forms, secrets are also masked in step names and in their quoted form in the runner log, a failed final report is logged
1. On Windows hosts a differently cased unsecure-commands opt-in or proxy variable counts, and an early cancel no longer leaks a job object handle
1. Interactive `register` prompts for a missing token and stops on SIGTERM, `exec --list --detect-event` no longer panics without events, `config set` works on `cache.s3` keys
Reviewed-on: https://gitea.com/gitea/runner/pulls/1338
Reviewed-by: bircni <bircni@icloud.com>
Co-authored-by: silverwind <me@silverwind.io>
Removes dead code and redundant logic and consolidates tests, so there is less to read and maintain. No behavior change.
1. **act/runner**: inline single-use wrappers, compile regexes once instead of per call, embed the trampoline as a string, drop the unused legacy result of `parseActionCommand`
2. **act/container**: `sanitizeConfig` and `mergeContainerConfigs` stop returning the configs they already mutate in place, drop never-set option fields and the unused `user` parameter of `ExecWithCmdLine`
3. **act/common and cache server**: drop the unused `Trace`/`Tracef` loggers and the short SHA of `FindGitRevision`, `RegisterJob` releases through `RevokeJob`
4. **daemon, poller and reporter**: one shutdown wait for both poll modes, drop an unreachable fetch timeout branch, guard the reporter's last report time with its existing lock instead of an atomic
5. **build**: drop unused Makefile and release workflow variables, collapse the goreleaser ignore list to the same build targets
6. **tests**: fold near-duplicate tests into tables with shared setup
Throughout, hand-rolled logic gives way to `cmp.Or`, `strings.Cut`, `slices` and `min`/`max`.
Reviewed-on: https://gitea.com/gitea/runner/pulls/1335
Reviewed-by: bircni <bircni@icloud.com>
Co-authored-by: silverwind <me@silverwind.io>
Migrates the runner to `connectrpc.com/connect/v2` by pinning actionslib v1.5.0, which carries https://gitea.com/gitea/actionslib/pulls/28. The wire behaviour towards Gitea is unchanged and works against Gitea servers on connect v1 and v2. The client keeps v1's unlimited response size, and the error codes and messages the runner checks stay the same.
Reviewed-on: https://gitea.com/gitea/runner/pulls/1325
Reviewed-by: bircni <bircni@icloud.com>
Co-authored-by: silverwind <me@silverwind.io>
Commands the runner applies itself, like the `::set-output::` older actions still print, showed up raw in the job log. The reporter also re-parsed every line with one job-wide stop-commands and echo state, while act keeps one command handler per step, stage, composite child and hook, as GitHub does. A `::stop-commands::` left active in one of them kept the reporter suppressed for the others, so their `::add-mask::` values were never registered, and post steps logged command lines twice.
- act's command handler decides what reaches the log, as GitHub's runner does: applied commands only under `ACTIONS_STEP_DEBUG` or `::echo::on`, `::stop-commands::` and its resume line shown with long tokens masked
- Masks go straight to the reporter, which no longer parses command lines
- One parser follows GitHub's rules: leading whitespace, `##[...]` anywhere in a line, registered commands only
- Service logs honour only `::add-mask::`
---------
Co-authored-by: bircni <bircni@icloud.com>
Reviewed-on: https://gitea.com/gitea/runner/pulls/1333
Reviewed-by: bircni <bircni@icloud.com>
Co-authored-by: silverwind <me@silverwind.io>
After https://gitea.com/gitea/runner/pulls/1304, logrus stayed only to forward logs from docker/cli and moby/go-archive. The runner used a small part of those: volume parsing, the build context tar, the `ssh://` connection and registry credentials. Own code that matches them replaces them, so logrus leaves the module graph. docker/cli stays for its logrus-free `opts` and `connhelper/ssh` packages.
Other dependencies that the standard library or existing ones cover go too: mergo, httprouter (replaced by `http.ServeMux`), retry-go, go-isatty, go-selinux, container-device-interface, gotest.tools and go-cmp (replaced by testify). The binary shrinks by 5%.
Behaviour differences:
1. An unreadable file in a build context is skipped with a warning, where go-archive left a truncated tar.
2. Build contexts no longer carry `security.capability` xattrs.
3. The cache and artifact servers answer HEAD on GET routes, and no longer redirect trailing slashes or answer OPTIONS.
*Written by Claude.*
---------
Co-authored-by: bircni <bircni@icloud.com>
Reviewed-on: https://gitea.com/gitea/runner/pulls/1319
Reviewed-by: bircni <bircni@icloud.com>
Reviewed-by: Lunny Xiao <xiaolunwen@gmail.com>
Co-authored-by: silverwind <2021+silverwind@noreply.gitea.com>
Job teardown ran every step on one shared minute. When removing the job container took longer, the network removal ran on an expired context and the network stayed behind until the orphan reclaim, which only takes networks older than `workdir_cleanup_age`, 24h by default. All that time it held a subnet of the daemon's address pool.
The job and service containers are now removed in parallel, and the docker proxy sweep, the network and the volumes get a fresh budget after them. The idle reclaim takes unused networks and volumes once they are 10 minutes old, and network removal treats a network that is already gone as removed.
Reviewed-on: https://gitea.com/gitea/runner/pulls/1318
Reviewed-by: bircni <bircni@icloud.com>
Co-authored-by: silverwind <2021+silverwind@noreply.gitea.com>
Each step refreshed its own clone of an action, so a composite using one action repository ten times fetched it ten times per job, and a single network error failed the job.
Like actions/runner, a job now downloads each action repository and ref once, as an archive extracted into a directory per commit. Each action still gets its own copy in the job container. This restores the per-job dedupe of https://gitea.com/gitea/runner/pulls/1178, which only reached a since-deleted branch.
How a download works:
1. The action URL is taken after `insteadOf` rewriting, as git would use it.
2. The archive comes from `<url>/archive/<ref>.tar.gz`, or from codeload.github.com for github.com, as for actions/runner.
3. A cached tag or branch is revalidated with its ETag, which GitHub and Gitea 28+ answer with a 304. A cached commit pin needs no request.
4. Transient failures (5xx, 408, 429 honouring `Retry-After`, dropped transfers) are retried three times with a 10 to 30 second backoff.
5. Git takes over, cloning and exporting the commit into the same directory layout, when the URL is not HTTP(S), the ref is a short commit, the host serves no usable archive, the host cannot be connected to, or the retries run out.
Median of 3 runs, git (shallow clone, then fetch) against archives, on a cold and a warm cache:
| Action | git cold | git warm | archive cold | archive warm |
|---|---|---|---|---|
| github.com actions/checkout@v4 | 937ms | 726ms | 286ms | 171ms |
| github.com actions/cache@v4 | 1006ms | 801ms | 516ms | 161ms |
| github.com actions/setup-node@v4 | 1046ms | 786ms | 463ms | 156ms |
| gitea.com actions/checkout@v4 | 2007ms | 1165ms | 667ms | 222ms |
The action cache layout changes, so existing caches download once more.
*Written by Claude.*
Fixes: https://gitea.com/gitea/runner/issues/1220
Reviewed-on: https://gitea.com/gitea/runner/pulls/1316
Reviewed-by: bircni <bircni@icloud.com>
Co-authored-by: silverwind <me@silverwind.io>
The runner's own messages between steps (not step output) were uploaded to the job log at `log.level`, so raising `log.level` for local debugging also changed what workflow authors see in Gitea, and lowering it hid messages from them.
Add `runner.log_report_level` to set that independently. Unset, it still follows `log.level`, so existing setups behave as before. An invalid value is rejected when the config loads.
Assisted-by: Codet
Assisted-by: Claude Code:claude-opus-5-5
Co-authored-by: silverwind <me@silverwind.io>
Co-authored-by: silverwind <2021+silverwind@noreply.gitea.com>
Reviewed-on: https://gitea.com/gitea/runner/pulls/1310
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
Gitea derives the steps of a job from the workflow file, so the `pre` and `post` steps of actions only show up inside "Set up job" and "Complete job", see https://github.com/go-gitea/gitea/issues/24604.
The runner now also reports the steps as it runs them (https://gitea.com/gitea/actions-proto-def/pulls/19): "Set up job", a "Pre" and "Post" step for each action that has one, the workflow steps with their evaluated names, and "Complete job", as actions/runner shows them. The per-step state Gitea reads today is unchanged, and the job log only moves "Complete job name" and "Matrix:" ahead of the pre steps, so older Gitea versions keep showing everything as before. Job hooks and containers stay part of "Set up job" and "Complete job".
The `replace` of `gitea.dev/actionslib` is swapped for the release once the proto change is merged.
Reviewed-on: https://gitea.com/gitea/runner/pulls/1293
Reviewed-by: Zettat123 <39446+zettat123@noreply.gitea.com>
Co-authored-by: bircni <bircni@icloud.com>
Move the log calls in `internal` onto `logging`, and add what the process-wide logger setup and its tests need: `SetTextFormat` with a line filter for secret masking (replacing `report.MaskingFormatter`), `SetReportCaller`, `SetDefault` and `logtest.SetDefault`. After this only the `logging` package imports logrus.
Stacked on https://gitea.com/gitea/runner/pulls/1301.
Assisted-by: Codet
Co-authored-by: bircni <bircni@icloud.com>
Reviewed-on: https://gitea.com/gitea/runner/pulls/1302
Reviewed-by: bircni <bircni@icloud.com>
Add `act/common/logging`: a printf-style `Logger` on a slog-shaped `Handler`, with a logrus-backed handler as the only backend, plus a `logtest` handler for tests. Move the job logger onto it: `common.Logger` returns `*logging.Logger`, the job log formatters and the `Reporter` hook become handlers, and composite step IDs travel in the context.
Logging is tightly coupled to logrus types (hooks, formatters, `Entry.Data`). Going through our own interface first lets the remaining call sites move over with unchanged behavior, and lets the backend be swapped for `log/slog` in one place afterwards. The global process logger and its masking formatter stay on logrus for now.
Assisted-by: Codet
Co-authored-by: bircni <bircni@icloud.com>
Reviewed-on: https://gitea.com/gitea/runner/pulls/1295
Reviewed-by: bircni <bircni@icloud.com>
Sends the OTLP metrics under [OpenTelemetry semantic convention](https://opentelemetry.io/docs/specs/semconv/) names where the runner's data matches the definition, like `cicd.worker.count`, `cicd.system.errors` and the `go.*` and `process.*` runtime metrics, and under `gitea.runner.*` otherwise. OTel dashboards then work without remapping, as they already do for the traces from https://gitea.com/gitea/runner/pulls/1206.
Adds a `gitea_runner_state` gauge (`busy`, `idle`, `unavailable`), and `/readyz` now fails once the runner shuts down.
**Breaking**
1. OTLP metric names, units, types and attribute values change, and the info and start time metrics move to resource attributes.
1. Prometheus `gitea_runner_client_errors_total` gains a `code` label, and `gitea_runner_job_total` counts jobs stopped by `runner.timeout` as `timeout` instead of `failure`.
---------
Co-authored-by: silverwind <2021+silverwind@noreply.gitea.com>
Co-authored-by: silverwind <me@silverwind.io>
Co-authored-by: bircni <bircni@icloud.com>
Reviewed-on: https://gitea.com/gitea/runner/pulls/1282
Reviewed-by: silverwind <2021+silverwind@noreply.gitea.com>
Co-authored-by: wmTJc9IK0Q <231276+wmtjc9ik0q@noreply.gitea.com>
The Docker proxy labels every container, network and volume a job creates and removes them when the job ends, which breaks jobs that deploy long-running services to the runner's own Docker daemon. Document that behavior and add `container.sweep` (default `true`) so operators who deploy to the same daemon can opt out. Kept resources stay labeled with `com.gitea.runner.job`.
Fixes: https://gitea.com/gitea/runner/issues/1243
Co-authored-by: silverwind <me@silverwind.io>
Reviewed-on: https://gitea.com/gitea/runner/pulls/1275
Reviewed-by: silverwind <2021+silverwind@noreply.gitea.com>
Co-authored-by: bircni <bircni@icloud.com>
Add `runner.client_cert_file` and `runner.client_key_file` for Gitea instances that require mutual TLS. The runner presents the certificate on its RPCs, job summary uploads, and the action, reusable workflow and `builtin:checkout` clones from the instance. The files are read on use, so renewed certificates apply without a restart. Job steps do not get them. An incomplete or mismatched pair fails startup.
Clones now decide the token, client certificate and `runner.insecure` with one host check, which also gives reusable workflow clones the TLS settings they lacked. `runner.insecure` still only covers the registered address.
Fixes: https://gitea.com/gitea/runner/issues/321
Co-authored-by: silverwind <2021+silverwind@noreply.gitea.com>
Co-authored-by: silverwind <me@silverwind.io>
Reviewed-on: https://gitea.com/gitea/runner/pulls/1273
Reviewed-by: silverwind <2021+silverwind@noreply.gitea.com>
Co-authored-by: bircni <bircni@icloud.com>
Config values can reference the runner's environment as `${NAME}`, for example `capacity: ${RUNNER_CAPACITY}`, so container deployments can configure the runner without templating the file. An unset `NAME` fails the load instead of silently becoming empty, and `$${NAME}` writes a literal `${NAME}`. Keys are never expanded.
Breaking: an existing value containing `${NAME}`, such as a shell command in a Kubernetes `pod_template`, now needs `$${NAME}`.
Closes https://gitea.com/gitea/runner/issues/551
Co-authored-by: silverwind <2021+silverwind@noreply.gitea.com>
Co-authored-by: silverwind <me@silverwind.io>
Reviewed-on: https://gitea.com/gitea/runner/pulls/1274
Reviewed-by: silverwind <2021+silverwind@noreply.gitea.com>
Co-authored-by: bircni <bircni@icloud.com>
Add `runner.extra_headers` for reverse proxies that require request headers on the runner's API requests to Gitea, like registration, polling and log and status reports.
Headers the runner sets itself (authentication, User-Agent, protocol headers) stay authoritative. Extra headers are dropped when a redirect changes the hostname or leaves HTTPS, while the runner's own authentication headers follow redirects as before. Invalid or case-insensitively duplicated header names are rejected at config load.
Fixes: https://gitea.com/gitea/runner/issues/171
Co-authored-by: silverwind <me@silverwind.io>
Reviewed-on: https://gitea.com/gitea/runner/pulls/1269
Co-authored-by: bircni <bircni@icloud.com>
Allow `valid_volumes` entries ending in `:ro` to permit only read-only mounts, so a workflow can no longer mount writable a host path the runner only shares read-only, like a certificate store. `**:ro` allows any volume read-only. An entry without `:ro` that matches too still allows writable mounts.
A named volume allowed only through `:ro` is mounted with `nocopy`, because Docker otherwise fills an empty volume from the image even when it is mounted read-only.
Fixes: https://gitea.com/gitea/runner/issues/407
Co-authored-by: silverwind <me@silverwind.io>
Reviewed-on: https://gitea.com/gitea/runner/pulls/1272
Reviewed-by: silverwind <2021+silverwind@noreply.gitea.com>
Co-authored-by: bircni <bircni@icloud.com>
The runner now uses the `git` command line instead of go-git to download actions and reusable workflows. go-git had recurring bugs and does not support SHA-256 repositories, and full clones of actions are several times faster.
`builtin:checkout` now runs `git` where the job runs and behaves like `actions/checkout`:
1. A branch is checked out as a local branch tracking `origin`, tag and pull request runs keep their ref, and a triggering tag that moved fails the step.
1. Later steps can fetch and push with the persisted token or SSH key.
1. It supports the `ssh-*`, `lfs`, `submodules`, `sparse-checkout`, `filter`, `fetch-tags`, `clean`, `persist-credentials`, `show-progress` and `set-safe-directory` inputs and the `ref` and `commit` outputs.
1. SHA-256 repositories work for actions, reusable workflows and `builtin:checkout`.
Git keeps the token off its command line, ignores `GIT_DIR` and similar variables inherited by the runner, and keeps action files LF regardless of `core.autocrlf`. A cancelled job lets git remove its lock files.
This is a breaking change: the runner needs Git 2.34.1 or newer on its `PATH`, and so do job images that use `builtin:checkout`. The official runner images include it.
Fixes: https://gitea.com/gitea/runner/issues/1077
Fixes: https://gitea.com/gitea/runner/issues/1267
Fixes: https://gitea.com/gitea/runner/issues/1279
Fixes: https://gitea.com/gitea/runner/issues/1284
Fixes: https://gitea.com/gitea/runner/issues/1285
Co-authored-by: silverwind <me@silverwind.io>
Co-authored-by: silverwind <2021+silverwind@noreply.gitea.com>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
Reviewed-on: https://gitea.com/gitea/runner/pulls/1277
Reviewed-by: Lunny Xiao <xiaolunwen@gmail.com>
Reviewed-by: silverwind <2021+silverwind@noreply.gitea.com>
Co-authored-by: bircni <bircni@icloud.com>
Support tmpfs in container `options` via `--mount type=tmpfs,...`, like GitHub does. The `valid_volumes` check silently dropped these mounts because they have no source. A tmpfs is backed by memory and exposes nothing from the host, same as `--tmpfs`, which already passes. Every other mount still goes through the allowlist.
Both forms now work:
```yaml
services:
mysql:
image: mysql:8.4
options: --tmpfs /var/lib/mysql # already worked
```
```yaml
services:
mysql:
image: mysql:8.4
options: --mount type=tmpfs,destination=/var/lib/mysql # was dropped, now works
```
Reviewed-on: https://gitea.com/gitea/runner/pulls/1289
Reviewed-by: bircni <bircni@icloud.com>
Co-authored-by: silverwind <me@silverwind.io>
Since https://github.com/go-gitea/gitea/pull/39578 (backported to v28), Gitea answers an unregistered runner with a Connect `Unauthenticated` error instead of a gRPC status wrapped as `Unknown`. `IsRegistrationRejected` missed it, so an ephemeral runner kept its spent registration file and failed on every restart. This also fails the `gitea/gitea:main-nightly` e2e job.
Match on the `unregistered runner` message suffix, which both shapes share. The code can't distinguish Gitea from a proxy, because a proxy's 401 with a JSON body also decodes as a wire `Unauthenticated` error.
Written by Claude Code.
Reviewed-on: https://gitea.com/gitea/runner/pulls/1287
Reviewed-by: bircni <bircni@icloud.com>
Co-authored-by: silverwind <me@silverwind.io>
Push the runner's Prometheus metrics to the OTLP endpoint next to the job traces, so both land in the same backend.
The `/metrics` registry is converted as the [Prometheus compatibility spec](https://opentelemetry.io/docs/specs/otel/compatibility/prometheus_and_openmetrics/) describes and sent through the existing gRPC-free client, so names stay unchanged and no second metrics stack is linked. Metrics are on by default and pushed every interval and once on shutdown, so ephemeral runners report too.
New standard variables, each `OTEL_EXPORTER_OTLP_*` one with `TRACES` and `METRICS` variants:
- `OTEL_METRICS_EXPORTER`, `none` turns metrics off
- `OTEL_METRIC_EXPORT_INTERVAL`, `OTEL_METRIC_EXPORT_TIMEOUT`
- `OTEL_EXPORTER_OTLP_TIMEOUT`, `OTEL_EXPORTER_OTLP_COMPRESSION`
- `OTEL_EXPORTER_OTLP_CERTIFICATE`, `OTEL_EXPORTER_OTLP_CLIENT_CERTIFICATE`, `OTEL_EXPORTER_OTLP_CLIENT_KEY`
---------
Co-authored-by: silverwind <me@silverwind.io>
Co-authored-by: silverwind <2021+silverwind@noreply.gitea.com>
Co-authored-by: bircni <bircni@icloud.com>
Reviewed-on: https://gitea.com/gitea/runner/pulls/1248
Reviewed-by: silverwind <2021+silverwind@noreply.gitea.com>
Reviewed-by: bircni <bircni@icloud.com>
Co-authored-by: wmTJc9IK0Q <231276+wmtjc9ik0q@noreply.gitea.com>
Ephemeral registrations are single-use, but the daemon kept `.runner` after the task, so a restarted container failed `Declare` with `unregistered runner` forever.
An ephemeral runner now removes its registration file as soon as it fetches a task, so a container killed mid-job registers afresh on restart. It also removes the file when Gitea answers `unregistered runner` at startup or while polling. A proxy's 401 does not count. Persistent runners keep their file, because Gitea sends the same reply for a token mismatch.
Closes https://gitea.com/gitea/runner/issues/818
---------
Co-authored-by: silverwind <me@silverwind.io>
Reviewed-on: https://gitea.com/gitea/runner/pulls/1256
Reviewed-by: silverwind <2021+silverwind@noreply.gitea.com>
Adds a Kubernetes backend. A `name:kubernetes://image` label runs the job as one pod through the Kubernetes API, with services as containers of the same pod. Step env is exported by a script sent over exec stdin, keeping it out of the pod spec and process arguments, and service env goes into a per-job Secret. Docker actions and container options are rejected. See `docs/kubernetes.md`.
Operators shape job pods with `kubernetes.pod_template`, with per-label `pod_templates` merged in `runs-on` order and a `$services` entry for service containers. The idle cleanup removes pods and Secrets a runner left behind.
The client calls the Kubernetes API directly and runs step commands over `coder/websocket` (same dependency used in gitea) instead of pulling in the heavyweight `kubernetes/client-go`. Includes an e2e test against k3s.
Replaces: https://gitea.com/gitea/runner/pulls/1000
Fixes: https://gitea.com/gitea/runner/issues/31
Assisted-by: Claude Code:claude-opus-5-5
Co-authored-by: Zettat123 <zettat123@gmail.com>
Co-authored-by: bircni <bircni@icloud.com>
Reviewed-on: https://gitea.com/gitea/runner/pulls/1260
Reviewed-by: Lunny Xiao <xiaolunwen@gmail.com>
Reviewed-by: Zettat123 <39446+zettat123@noreply.gitea.com>
Co-authored-by: silverwind <me@silverwind.io>
Store the cache used by `actions/cache` in an S3-compatible bucket such as AWS S3 or RustFS, with no `cache-server`. Runners with the same settings use the same cache.
A cache saved on one runner can be restored on the others within 30 seconds. The runner talks to the bucket over the S3 REST API with SigV4, without an SDK. Lifecycle rules on the bucket expire old caches.
*Description rewritten by Claude.*
---------
Co-authored-by: silverwind <me@silverwind.io>
Reviewed-on: https://gitea.com/gitea/runner/pulls/1244
Reviewed-by: silverwind <2021+silverwind@noreply.gitea.com>
Co-authored-by: bircni <bircni@icloud.com>
Make cache requests go through the runner and remove outbound traffic from the cache server. The runner forwards them to `cache.external_server` when one is set, and attaches itself to each job network so jobs can reach it.
Benefits:
1. Jobs need no http proxy rules for the cache server, which `@actions/http-client` mishandles for CIDR ranges in `no_proxy` (https://github.com/actions/toolkit/pull/2495).
2. Only runners reach a shared cache server, which needs no route to Gitea.
3. A runner container sharing the host Docker socket now caches, where jobs previously could not reach it.
Breaks:
1. Jobs that reach `external_server` but not their runner, such as a remote `DOCKER_HOST`. The built-in cache has always needed that reachability, so both modes now agree. Set `cache.host` and `cache.port` to an address those jobs reach.
2. Jobs on `container.network: bridge`, where Docker rejects the alias, so the runner cannot attach and their cache fails as it does today. Leave `container.network` empty, or name a user-defined network.
*Written by Claude.*
---------
Co-authored-by: bircni <bircni@icloud.com>
Reviewed-on: https://gitea.com/gitea/runner/pulls/1229
Reviewed-by: bircni <bircni@icloud.com>
Co-authored-by: silverwind <2021+silverwind@noreply.gitea.com>
`valid_volumes` patterns now match with doublestar instead of gobwas/glob, which was compiled without separators so `*` crossed slashes. doublestar is already in the module graph, keeps `*` within one path segment and matches host paths with the OS separator, which fixes Windows patterns where a backslash was read as an escape.
Breaking: a host pattern that relied on `*` matching nested paths needs `**`, e.g. `/data/**` instead of `/data/*`. Named volumes and the bare `**` are unaffected.
Co-authored-by: bircni <bircni@icloud.com>
Reviewed-on: https://gitea.com/gitea/runner/pulls/1214
Reviewed-by: bircni <bircni@icloud.com>
Co-authored-by: silverwind <2021+silverwind@noreply.gitea.com>
Adds built-in actions, native Go actions shipped with the runner and selected with `uses: builtin:<name>`. Jobs need no action download and no Node in their image.
The first one is `builtin:checkout`, a subset of `actions/checkout` with `repository`, `ref`, `token`, `path` and `fetch-depth`. Other inputs fail the step. The runner clones shallowly on its host and copies the tree into the job, while `gitea-runner exec` copies the local working directory for the workflow's own ref.
`actions/checkout` and other remote actions are unchanged. A clone ref of `HEAD` now shallow-clones the remote's default branch, which checkout uses for other repositories.
---------
Co-authored-by: silverwind <me@silverwind.io>
Co-authored-by: silverwind <2021+silverwind@noreply.gitea.com>
Reviewed-on: https://gitea.com/gitea/runner/pulls/871
Reviewed-by: silverwind <2021+silverwind@noreply.gitea.com>
Co-authored-by: bircni <bircni@icloud.com>
Containers a job starts through its Docker socket, for example `docker run -v "$PWD:/src"`, `./data:/data` in docker compose, or actions like dockerfile-roast, can now bind-mount the workspace and other paths the job sees, as on a host, without `bind_workdir`.
The per-job Docker proxy rewrites container and volume create requests. A bind source, or the device of a `local` volume with `o: bind`, that lies under one of the job container's mounts is pointed at that mount's path on the daemon, read from inspecting the job container. Paths that already name a daemon path, like `GITEA_DOCKER_WORKSPACE`, and paths outside the job's mounts pass through unchanged.
The proxy now also starts when the runner runs in a container given the host's Docker socket, by placing its socket in the runner's working directory, and in rootless dind, by granting the daemon socket's group through an ACL.
Fixes https://gitea.com/gitea/runner/issues/1219
Fixes https://gitea.com/gitea/runner/issues/1193
Reviewed-on: https://gitea.com/gitea/runner/pulls/1226
Reviewed-by: bircni <bircni@icloud.com>
Co-authored-by: silverwind <2021+silverwind@noreply.gitea.com>
Every cache hit rewrote the entry's access time, costing a bolt write and fsync per restore. Completed entries accessed within the last minute are no longer rewritten, and the in-use and retention windows grow by that minute so nothing is evicted earlier. Same-second rewrites are skipped, reservations are inserted in one transaction, and prefix lookups use the repository index instead of scanning every entry.
The store is still opened per request. Keeping it open, as this commit first did, is faster under concurrency but stops a second process sharing the cache directory from starting its cache server.
The README caching section is condensed and corrected.
| Benchmark, v2 | main | store kept open | this PR |
| --- | ---: | ---: | ---: |
| Exact hit, 1 client | 16.42 | 0.15 | 0.24 |
| Exact hit, 8 clients | 13.70 | 0.08 | 1.28 |
| Exact hit, 32 clients | 16.01 | 0.07 | 4.26 |
| Prefix hit, 8 clients | 18.34 | 0.12 | 1.28 |
| Miss, 8 clients | 3.21 | 0.11 | 1.28 |
| Save and restore 1 MiB, 8 clients | 74.4 | 22.9 | 34.9 |
| Geomean, all 22 benchmarks | 14.03 | 0.31 | 2.11 |
Mean ms per request, medians of 4 rounds of 300 requests over 1,000 entries on an Apple M3 Max.
---------
Co-authored-by: silverwind <2021+silverwind@noreply.gitea.com>
Co-authored-by: silverwind <me@silverwind.io>
Reviewed-on: https://gitea.com/gitea/runner/pulls/1222
Reviewed-by: silverwind <2021+silverwind@noreply.gitea.com>
Implement speedups to job start and shutdown.
- Create the job container while services are still becoming healthy, and poll their health at a flat one second instead of a 2s to 32s doubling backoff
- Pull each service image once instead of twice, and fetch a warm action cache once instead of twice
- Report the job result before reclaiming its volumes, and reap volumes stranded by a runner that died mid-job
| Step | Scenario | Before | After |
| --- | --- | --- | --- |
| Complete job | Large workspace volume | 4.2s | 0.4s |
| Set up job | One service, 2s health interval | 7.08s | 3.26s |
| Set up job | Two cached actions from github.com | 1.81s | 1.34s |
| Set up job | Two cached actions from gitea.com | 2.42s | 1.95s |
| Set up job | Two actions, cold action cache | 6.62s | unchanged |
| Set up job | Minimal job, no services or actions | 0.62s | unchanged |
Assisted-by: Claude Code:Opus 5
Reviewed-on: https://gitea.com/gitea/runner/pulls/1218
Reviewed-by: bircni <bircni@icloud.com>
Co-authored-by: silverwind <me@silverwind.io>
Cache v2 makes the cache server the `ACTIONS_RESULTS_URL` origin, so artifact calls arrived there and were proxied on to Gitea, failing whenever it could not reach the instance.
- Artifact calls are answered with a redirect, so the cache server opens no connection to Gitea. A scheme change or an untrusted instance is still proxied, but there the cache server is the runner itself, which already reaches Gitea.
- Failures answer in twirp, not an empty `502` that clients report as `Unexpected end of JSON input`.
- `cache.v2: false` really points artifacts at Gitea now.
- Cache reservations are bound to the job that made them, so two jobs saving one key cannot commit against each other's upload, and a retry after a lost answer no longer fails a saved entry.
- The toolkit patch, which edits the GitHub-host check out of an action's bundle, was left in the shared checkout where a job running with `runner.patch_actions: false` could inherit it. It is put back after the job's copy.
- `exec` names an origin for the cache v2 it advertises, and masks its runtime token.
Behaviour changes: `no_proxy` no longer exempts `cache.external_server`, and `cache.enabled: false` also stops external registration.
Fixes https://gitea.com/gitea/runner/issues/1208
Fixes https://gitea.com/gitea/runner/issues/1211
Assisted by Claude (Opus 5).
Reviewed-on: https://gitea.com/gitea/runner/pulls/1216
Reviewed-by: bircni <bircni@icloud.com>
Co-authored-by: silverwind <me@silverwind.io>
1. Add `GITEA_DOCKER_WORKSPACE` which holds the workspace path as the daemon sees it, enabling `${GITEA_DOCKER_WORKSPACE:-.}/data:/app/data` in a compose file without having to resort to `bind_workdir` (which causes much more problems like breaking `actions/cache` because of unstable workspace paths).
2. Add container/network/volume cleanup for containers started within jobs, for example via `docker compose` inside a job. It works by running a lightweight docker socket proxy and injecting a `com.gitea.runner.job` label into every container creation and that label is used to remove containers started by that job at the end. Perf impact of this is near-zero.
Docs: https://gitea.com/gitea/docs/pulls/535
Assisted by Claude (Fable 5.1).
Co-authored-by: bircni <bircni@icloud.com>
Reviewed-on: https://gitea.com/gitea/runner/pulls/1204
Reviewed-by: bircni <bircni@icloud.com>
Co-authored-by: silverwind <me@silverwind.io>
Automatically allow workspace mounts in `valid_volumes` for the mounts done via `bind_workdir`, this obsoletes the need for `/workspace/**` or other insecure configurations which would expose workspaces between tasks.
Reviewed-on: https://gitea.com/gitea/runner/pulls/1203
Reviewed-by: bircni <bircni@icloud.com>
Co-authored-by: silverwind <me@silverwind.io>
Fixes 51 bugs discovered via comparison with `actions/runner`. Every fix has test coverage.
### Secrets
- A short secret registered no shifted-base64 form, so `base64("user:$TOKEN")` printed in the clear
- Encoded forms came only from the whole trimmed value, missing padded and per-line spellings
- Masks split only on `\n`, so `::add-mask::a%0Db` registered neither half
- Adds XML, expression-string and quote-trimming encoders
### Workflow commands
- Split at the last `::` or `]` rather than the first, so `::add-mask::a::b` registered no mask
- A command on the last line without a newline was ignored, and `::ADD-MASK::` did nothing
- `##[...]` did not decode `%3B`/`%5D`, properties lost anything after a second `=`
- `$GITHUB_ENV` and `::set-env::` now refuse `NODE_OPTIONS`
### Status
- `continue-on-error` reported failed, a cancelled job reported success, an `if:` error reported cancelled
- File commands ran after `continue-on-error`, failing the job while the step stayed green
- A bad job output aborted the whole run instead of that job
### Steps and actions
- `${{ matrix.* }}` and `${{ strategy.* }}` were empty inside composite actions
- Composite inputs leaked into nested actions as `INPUT_*`, `with:` matched case-sensitively, `pre` failures were dropped
- Docker actions dropped `runs.env` when the caller passed `with: args:`, and caller `args`/`entrypoint` beat the manifest
- An implicit shell ran with `pipefail`, a `shell:` without `{0}` passed without running
- `container.env` overrode job env and every `$GITHUB_ENV` write, heredocs lost leading blank lines, `$GITHUB_PATH` was not BOM-decoded
Written by Claude Opus 5.
Reviewed-on: https://gitea.com/gitea/runner/pulls/1194
Reviewed-by: bircni <bircni@icloud.com>
Co-authored-by: silverwind <me@silverwind.io>
Service containers were built without a volume policy, so every bind and mount they declared was dropped, whatever `valid_volumes` allowed. GitHub passes a service's declared volumes straight to `docker create`, so a workflow that mounts into a service silently did nothing here. Services now get the configured policy, but not `validVolumes()`, which would also hand them the docker daemon socket that GitHub mounts only into the job container.
### What changes for users
On the default `valid_volumes: []` nothing changes: a service's volumes are still dropped, now with a warning rather than in silence. Once `valid_volumes` is configured, a service's declared volumes are honored under it instead of discarded, which is what that setting already documents. No workflow that worked before stops working, and a service can reach no volume the policy does not already allow the job container, so this is not a breaking change.
---------
Co-authored-by: bircni <bircni@icloud.com>
Reviewed-on: https://gitea.com/gitea/runner/pulls/1186
Reviewed-by: bircni <bircni@icloud.com>
Co-authored-by: silverwind <me@silverwind.io>
A secret in a matrix value reached the log in the clear:
```yaml
strategy:
matrix:
include: "${{ github.token }}"
```
Chasing that one route is pointless, so this masks every sink a secret leaves a job by: the uploaded log rows and the on-disk `job.log`, both through one choke point in `appendLogRow`; the runner's own log, which is where planning errors like that one land with no job logger in reach; the job logger's stdout under debug logging; job summaries; job outputs; and the job name that becomes a container name.
Values the runner knows but the job never declared, the proxy password and the task token, are hidden the same way. Masks apply longest first, since `strings.Replacer` matches in argument order and one secret prefixing another would otherwise mask the prefix and print the rest.
### What changes for users
An output whose value carries a secret is skipped with a warning instead of sent, matching GitHub. Output that showed a secret now shows `***`. `ACTIONS_STEP_DEBUG` and `ACTIONS_RUNNER_DEBUG` are never masked, also matching GitHub, so an output of `true` still reaches the jobs that need it.
Each fix has a test that fails without it.
Reviewed-on: https://gitea.com/gitea/runner/pulls/1188
Reviewed-by: bircni <bircni@icloud.com>
Co-authored-by: silverwind <me@silverwind.io>
Gitea slices a task's single log stream into steps by the LogIndex/LogLength that UpdateTask carries. The reporter's daemon flushed log rows without those counters, which only left on the separate state ticker, so rows the server acked between two state reports belonged to no step. The web UI attributes them to no step while the job runs, and a runner that stops reporting in that window leaves them under "Complete job" for good once Gitea finalizes the task.
The three log-flush paths now report the state describing the rows the server just took, so the gap is at most one RPC. Live output also stops waiting up to `state_report_interval` to become visible. An idle job adds no requests.
This does not explain why the runner in that job stopped reporting for 13m40s until Gitea reaped the task as a zombie, that needs the runner host's own log.
Fixes https://gitea.com/gitea/runner/issues/1184
---------
Co-authored-by: silverwind <2021+silverwind@noreply.gitea.com>
Co-authored-by: silverwind <me@silverwind.io>
Reviewed-on: https://gitea.com/gitea/runner/pulls/1189
Reviewed-by: silverwind <2021+silverwind@noreply.gitea.com>
Co-authored-by: bircni <bircni@icloud.com>
The runner's own `container.options` and the workflow's were joined into one string before parsing, so the host-escape filter added in https://gitea.com/gitea/runner/pulls/1058 dropped the administrator's options along with the workflow's. Setups that need `--device` or `--security-opt` from the config file had no way left to get them short of enabling privileged mode.
`NewContainerInput` now carries the two sources apart, as `RunnerOptions` and `WorkflowOptions`, down to the point where the filter runs. With privileged mode off, the host-escape fields are reset to what the runner's own options parse to on their own, so only the workflow's contribution is dropped.
Three further ways a workflow's options reached past its container, all resolved on the runner before anything reaches the daemon:
1. `--env-file` and `--label-file` name files that are read on the runner, so any file it could read became container environment or labels. Both are refused from a workflow now, and still serve the runner's own options.
2. A bare `--env NAME` was resolved from the runner's own environment by docker's validator. That lookup is gone, for every source. Use `runner.envs` or `runner.env_file` to pass a variable on.
3. A volume driver decides for itself what it mounts, and the local driver's `device=` option turns a name `valid_volumes` allows into a bind of any host path. A workflow's mounts may no longer carry one.
`--isolation`, `--volume-driver` and the two paths `--security-opt systempaths=unconfined` lands in were also missing from the fields a workflow may not set.
Last, the `--network and --net in the options will be ignored.` warning fired for every container, because the runner's own network mode is fed into the parsed options before the check runs.
Fixes https://gitea.com/gitea/runner/issues/1142
Reviewed-on: https://gitea.com/gitea/runner/pulls/1151
Reviewed-by: bircni <bircni@icloud.com>
Co-authored-by: silverwind <me@silverwind.io>
