Auto_Bangumi/e2e

Hermetic E2E tests

The E2E suites run a production-built AutoBangumi image against local, deterministic upstream services. Docker Compose publishes random loopback ports and uses an internal network for service-to-service traffic. Tests must not use public RSS, TMDB, image, player, or downloader endpoints.

Prerequisites

• Docker Engine with Compose v2
• Python 3.13 and uv
• Node.js 20 and pnpm 9.11
• enough disk space for the AutoBangumi, Python, Playwright, and qBittorrent images

Install dependencies and browser binaries before starting a stack:

uv sync --directory backend --locked --group dev
pnpm --dir webui install --frozen-lockfile
pnpm --dir webui exec playwright install --with-deps chromium webkit firefox

Build the production test image

The build helper places the compiled WebUI and generated version module in a temporary Docker context. It does not write dist or module/__version__.py into the source tree.

pnpm --dir webui run build
python3 e2e/scripts/build_test_image.py \
  --version 3.3.999-e2e.1 \
  --image auto-bangumi:e2e \
  --dist webui/dist

Run a lane locally

Create a fresh project and work directory for every invocation. Reusing a work directory also reuses setup state, which defeats first-run coverage.

export LANE=runtime
export AB_E2E_PROJECT="ab-e2e-local-$LANE-$$"
export AB_E2E_WORK_DIR="$(mktemp -d "${TMPDIR:-/tmp}/ab-e2e-$LANE.XXXXXX")"
export AB_E2E_APP_IMAGE=auto-bangumi:e2e

Runtime API and process coverage:

python3 e2e/scripts/stack.py run \
  --profile browser \
  --project-name "$AB_E2E_PROJECT" \
  --work-dir "$AB_E2E_WORK_DIR" \
  -- uv run --directory backend pytest src/test/e2e/runtime -m e2e -q

Chromium desktop:

python3 e2e/scripts/stack.py run \
  --profile browser \
  --project-name "$AB_E2E_PROJECT" \
  --work-dir "$AB_E2E_WORK_DIR" \
  -- pnpm --dir webui run test:e2e:chromium --retries=0

WebKit at the 390×844 mobile viewport:

python3 e2e/scripts/stack.py run \
  --profile browser \
  --project-name "$AB_E2E_PROJECT" \
  --work-dir "$AB_E2E_WORK_DIR" \
  -- pnpm --dir webui run test:e2e:webkit --retries=0

Real qBittorrent plus the packaged-image workflow:

python3 e2e/scripts/stack.py run \
  --profile downloader \
  --project-name "$AB_E2E_PROJECT" \
  --work-dir "$AB_E2E_WORK_DIR" \
  -- python3 e2e/scripts/run_downloader_lane.py

The downloader runner executes the focused Chromium download/rename spec and backend/src/test/e2e/downloader inside that same fresh stack.

Firefox is a nightly, non-required lane:

python3 e2e/scripts/stack.py run \
  --profile browser \
  --project-name "$AB_E2E_PROJECT" \
  --work-dir "$AB_E2E_WORK_DIR" \
  -- pnpm --dir webui run test:e2e --project=firefox-nightly --retries=0

The nightly workflow runs browser and downloader twice as separate matrix passes. Each pass receives a fresh project and work directory; locally, repeat the corresponding command after recreating both variables above.

Useful lifecycle checks:

python3 e2e/scripts/stack.py validate --profile browser
python3 e2e/scripts/stack.py validate --profile downloader
python3 e2e/scripts/stack.py smoke --profile browser
python3 e2e/scripts/stack.py ports --profile browser \
  --project-name "$AB_E2E_PROJECT" --work-dir "$AB_E2E_WORK_DIR"
python3 e2e/scripts/stack.py env --profile browser \
  --project-name "$AB_E2E_PROJECT" --work-dir "$AB_E2E_WORK_DIR"
python3 e2e/scripts/stack.py restart --profile browser \
  --project-name "$AB_E2E_PROJECT" --work-dir "$AB_E2E_WORK_DIR" app
python3 e2e/scripts/stack.py stop --profile browser \
  --project-name "$AB_E2E_PROJECT" --work-dir "$AB_E2E_WORK_DIR"

Duration and artifacts

The pull-request checks have a hard 15-minute limit. Typical targets are 5–8 minutes for runtime, 8–12 minutes for each browser, and 10–15 minutes for the real-downloader/image lane.

stack.py run collects diagnostics before Compose teardown, including Compose state and logs, mock-service request journals, qBittorrent state, backend logs, and SQLite files. Local stack artifacts are written to $AB_E2E_WORK_DIR/artifacts. Playwright writes screenshots and videos to webui/test-results, with its HTML report in webui/playwright-report. Raw Playwright traces are disabled because they serialize authenticated storage state and response bodies; token-reveal tests also disable visual recordings. Browser console errors and failed responses remain available in the redacted browser-diagnostics.json attachment. CI uploads these paths with if: always() and retains them for 14 days. Never print credentials or session material from a test.

Legacy workflow migration

The former numbered, session-scoped workflow and its fixed ports were removed. Setup/auth/config/restart persistence now live in isolated runtime tests; parser, offset-review, and Windows-path regressions have focused runtime and browser scenarios; the real qB lane owns download, rename, idempotency, and file deletion. Endpoint shape, validation, CRUD, empty-list, log, search, and notification contracts remain in the faster backend/src/test/test_api_*.py unit/integration suites. No test depends on state created by an earlier test.

Add or change a mock scenario

1. Put static RSS, TMDB, player, torrent, image, or file data under e2e/fixtures. Fixture URLs must use only Compose service names or loopback addresses.
2. Add named TMDB behavior to e2e/fixtures/tmdb/scenarios.json.
3. Select or reset scenarios through /__admin/scenario/<name> and /__admin/reset; assert requests through /__admin/requests or /__admin/state.
4. Keep responses deterministic and add a contract test for any new endpoint, error mode, or request field.
5. Run python3 e2e/scripts/audit_sources.py before opening a PR.

Regenerate and verify the deterministic media and torrent assets with:

python3 e2e/scripts/generate_fixtures.py
python3 e2e/scripts/generate_fixtures.py --check

Commit the generated media, torrent, and checksum changes together.

Update the qBittorrent image digest

The downloader overlay must retain both an exact patch tag and an immutable manifest digest. To update it:

1. Choose the exact linuxserver/qbittorrent patch release and inspect its published manifest:

   docker buildx imagetools inspect linuxserver/qbittorrent:5.2.3
   

2. Replace only the tag and sha256 digest in e2e/compose/downloader.yml. Never use latest or a moving major/minor tag.

3. Pull that exact reference, then run the downloader validation and smoke checks. Startup verifies the fixed admin/adminadmin credentials and rejects enabled DHT, PeX, LSD, UPnP, RSS processing, or a changed save path.

4. Run the downloader Playwright lane and inspect its artifacts before merging.