docs: explain labels, the docker image cache volume, and the dind-rootless UID (#1086)

Fixes #106
Fixes #570
Fixes #627

Reviewed-on: https://gitea.com/gitea/runner/pulls/1086
Reviewed-by: Lunny Xiao <xiaolunwen@gmail.com>

README.md

@@ -85,6 +85,8 @@ docker run -e GITEA_INSTANCE_URL=https://your_gitea.com -e GITEA_RUNNER_REGISTRA
 
 Mount a volume on `/data` if you want the registration file and optional config to survive container recreation (see [scripts/run.sh](scripts/run.sh)).
 
+> **`/data` does not hold the image cache.** It is the runner's working directory and contains only the `.runner` registration file and, optionally, your config file. Images pulled for jobs live in the *Docker daemon's* data root, which for the `dind` flavours is inside the container (`/var/lib/docker`, or `/home/rootless/.local/share/docker` for `dind-rootless`). To keep the image cache across restarts, give that path its own volume as well — otherwise every new container re-pulls the job images. With the `basic` flavour the images live on whichever daemon you point the runner at, so there is nothing extra to persist.
+
 ### Image flavours
 
 The image is published in three flavours, all built from the single multi-stage [Dockerfile](Dockerfile) in this repository. They differ only in how a Docker daemon is made available to the jobs the runner executes; the `gitea-runner` binary inside them is identical.
@@ -121,6 +123,8 @@ Two processes have to run side by side here (the Docker daemon and the runner),
 
 Same idea as `dind`, but built on `docker:dind-rootless` so the bundled daemon and the runner run as an unprivileged user (`rootless`, UID 1000) rather than `root`. `DOCKER_HOST` is preset to `unix:///run/user/1000/docker.sock` so the runner talks to the rootless daemon. This reduces the blast radius compared to the privileged `dind` flavour, but rootless Docker carries the usual rootless limitations (networking, cgroups, storage drivers, and some operations that need additional host configuration such as `/etc/subuid` / `/etc/subgid` mappings and unprivileged user-namespace support).
 
+> **The UID is fixed at 1000.** It comes from the `rootless` user baked into the upstream `docker:dind-rootless` base image, and the bundled daemon always listens on `/run/user/1000/docker.sock` inside the container, so running this flavour as a different user (`--user 1001`) does not work. If you need the runner to talk to a *host* rootless daemon that runs under some other UID, use the `basic` flavour instead and bind-mount that daemon's socket (see [examples/vm/rootless-docker.md](examples/vm/rootless-docker.md)); pointing `DOCKER_HOST` at a host socket from inside `dind-rootless` will not work. Changing the UID otherwise means rebuilding the image from a base with a different `rootless` user.
+
 > **Note on Podman:** these images target the Docker daemon. The bundled `dind`/`dind-rootless` daemons are `dockerd`, not Podman, and the `basic` flavour expects a Docker-compatible socket. Running them under rootless Podman is not a supported configuration, though pointing the `basic` flavour at a Podman socket that emulates the Docker API may work for some workloads.
 
 ### Configuration
@@ -147,6 +151,50 @@ If you omit `-c`, built-in defaults apply (same as an empty YAML document).
 
 Earlier releases let a small set of environment variables (`GITEA_DEBUG`, `GITEA_TRACE`, `GITEA_RUNNER_CAPACITY`, `GITEA_RUNNER_FILE`, `GITEA_RUNNER_ENVIRON`, `GITEA_RUNNER_ENV_FILE`) override parts of the default config. Those overrides have been removed — use a YAML config file for all settings instead. For the Docker images, the entrypoint still understands a separate set of variables (such as `RUNNER_STATE_FILE`); see [scripts/run.sh](scripts/run.sh) and the container documentation below.
 
+### Labels
+
+Labels decide **which jobs a runner accepts** and **how it runs them**. A job's `runs-on` is matched against the runner's label names; the first match wins and selects the execution environment for that job.
+
+A label is written as:
+
+```text
+<name>[:<schema>[:<args>]]
+```
+
+| Part | Meaning |
+| --- | --- |
+| `name` | The name a workflow refers to in `runs-on`, e.g. `ubuntu-latest`. |
+| `schema` | Either `docker` or `host`. Defaults to `host` when omitted. |
+| `args` | Only used by the `docker` schema: the image to run the job in. |
+
+Two schemas are supported:
+
+- **`docker://<image>`** — the job runs inside a container created from `<image>`:
+
+  ```text
+  ubuntu-latest:docker://docker.gitea.com/runner-images:ubuntu-latest
+  ```
+
+- **`host`** — the job's steps run directly on the machine the runner is on, using the tools installed there:
+
+  ```text
+  macos:host
+  ```
+
+So with the labels
+
+```text
+ubuntu-latest:docker://docker.gitea.com/runner-images:ubuntu-latest,macos:host
+```
+
+a workflow with `runs-on: ubuntu-latest` is executed in the `runner-images:ubuntu-latest` container, and one with `runs-on: macos` is executed directly on the host.
+
+Names may themselves contain a colon (for example `pool:e57e18d4-10d4-406f-93bf-60f127221bdd`); only `host` and `docker` are treated as schemas.
+
+If a job's `runs-on` matches none of the runner's labels, the job still runs, in the default `docker.gitea.com/runner-images:ubuntu-latest` image. Images maintained for this purpose are listed at [gitea/runner-images](https://gitea.com/gitea/runner-images).
+
+Labels are chosen at registration time (`--labels`, or the interactive prompt) and can be changed afterwards by editing `runner.labels` in the config file, or in the Gitea UI under the runner's settings.
+
 #### Registration vs config labels
 
 If `runner.labels` is set in the YAML file, those labels are used during `register` and the `--labels` CLI flag is ignored.

examples/kubernetes/README.md

@@ -6,6 +6,11 @@ NOTE: `dind-docker.yaml` uses the native sidecar pattern (init container with `r
 
 NOTE: A helm chart for `gitea-runner` also exists for easier deployments https://gitea.com/gitea/helm-actions
 
+Each example persists **two** things, and it is worth knowing which is which:
+
+- `/data` is the runner's working directory. It holds the `.runner` registration file and, optionally, the config file — so the runner re-attaches to the server instead of registering again.
+- The Docker daemon's data root holds the images pulled for jobs (`/var/lib/docker` for the dind sidecar, `/home/rootless/.local/share/docker` for `dind-rootless`). It is *not* under `/data`. If you drop this volume, the examples still work, but the image cache is discarded whenever the pod is recreated and every job re-pulls its images.
+
 Files in this directory:
 
 - [`dind-docker.yaml`](dind-docker.yaml)

examples/kubernetes/dind-docker.yaml

@@ -1,3 +1,5 @@
+# Holds the runner's working directory (/data): the .runner registration file
+# and, optionally, the config file.
 kind: PersistentVolumeClaim
 apiVersion: v1
 metadata:
@@ -10,6 +12,21 @@ spec:
       storage: 1Gi
   storageClassName: standard
 ---
+# Holds the Docker daemon's data root (/var/lib/docker), i.e. the images pulled
+# for jobs. Without it, the image cache is lost whenever the pod is recreated
+# and every job re-pulls its images. Size it for the images you expect to cache.
+kind: PersistentVolumeClaim
+apiVersion: v1
+metadata:
+  name: docker-vol
+spec:
+  accessModes:
+  - ReadWriteOnce
+  resources:
+    requests:
+      storage: 20Gi
+  storageClassName: standard
+---
 apiVersion: v1
 data:
   # The registration token can be obtained from the web UI, API or command-line.
@@ -45,6 +62,9 @@ spec:
       - name: runner-data
         persistentVolumeClaim:
           claimName: runner-vol
+      - name: docker-data
+        persistentVolumeClaim:
+          claimName: docker-vol
       initContainers:
       - name: docker
         image: docker:28.2.2-dind
@@ -53,6 +73,8 @@ spec:
         volumeMounts:
         - name: docker-socket
           mountPath: /var/run
+        - name: docker-data
+          mountPath: /var/lib/docker
         startupProbe:
           exec:
             command: ["/usr/bin/test", "-S", "/var/run/docker.sock"]

examples/kubernetes/rootless-docker.yaml

@@ -1,3 +1,5 @@
+# Holds the runner's working directory (/data): the .runner registration file
+# and, optionally, the config file.
 kind: PersistentVolumeClaim
 apiVersion: v1
 metadata:
@@ -10,6 +12,21 @@ spec:
       storage: 1Gi
   storageClassName: standard
 ---
+# Holds the rootless Docker daemon's data root, i.e. the images pulled for jobs.
+# Without it, the image cache is lost whenever the pod is recreated and every job
+# re-pulls its images. Size it for the images you expect to cache.
+kind: PersistentVolumeClaim
+apiVersion: v1
+metadata:
+  name: docker-vol
+spec:
+  accessModes:
+  - ReadWriteOnce
+  resources:
+    requests:
+      storage: 20Gi
+  storageClassName: standard
+---
 apiVersion: v1
 data:
   # The registration token can be obtained from the web UI, API or command-line.
@@ -43,7 +60,12 @@ spec:
       - name: runner-data
         persistentVolumeClaim:
           claimName: runner-vol
+      - name: docker-data
+        persistentVolumeClaim:
+          claimName: docker-vol
       securityContext:
+        # The dind-rootless image runs as the `rootless` user (UID/GID 1000);
+        # fsGroup makes both volumes writable for it.
         fsGroup: 1000
       containers:
       - name: runner
@@ -68,4 +90,7 @@ spec:
         volumeMounts:
         - name: runner-data
           mountPath: /data
+        # The rootless daemon keeps its images here, not under /data.
+        - name: docker-data
+          mountPath: /home/rootless/.local/share/docker
 

examples/kubernetes/statefulset-dind.yaml

@@ -44,6 +44,10 @@ spec:
         volumeMounts:
         - name: docker-socket
           mountPath: /var/run
+        # Keeps the images pulled for jobs across restarts. Without this, the
+        # daemon's data root is ephemeral and every job re-pulls its images.
+        - name: docker-data
+          mountPath: /var/lib/docker
         startupProbe:
           exec:
             command: ["/usr/bin/test", "-S", "/var/run/docker.sock"]
@@ -68,6 +72,8 @@ spec:
         - name: docker-socket
           mountPath: /var/run
   volumeClaimTemplates:
+  # The runner's working directory: the .runner registration file and, optionally,
+  # the config file.
   - metadata:
       name: runner-data
     spec:
@@ -77,3 +83,14 @@ spec:
         requests:
           storage: 1Gi
       storageClassName: standard
+  # The Docker daemon's data root: the images pulled for jobs. Size it for the
+  # images you expect to cache.
+  - metadata:
+      name: docker-data
+    spec:
+      accessModes:
+      - ReadWriteOnce
+      resources:
+        requests:
+          storage: 20Gi
+      storageClassName: standard