CI

CI logic lives in a dagger module (.dagger/, Go), not inline in YAML: the GitHub Actions workflows are thin wrappers that call dagger call <function>, so the same commands run locally and in CI. dagger functions lists what the module exposes on your checkout.

Dagger functions

The converter is a pure Python package as well as an image. A campaigns repository's own CI and a workstation install it with uvx --from "git+https://github.com/AI-Riksarkivet/htrflow-batch#subdirectory=packages/converter" htrflow-campaigns; the image (build-campaigns) is only for the Argo CD hook, which runs in the cluster where nothing can be installed.

What the containers see: exclude, not include

lint, typecheck and test build their container with buildWithUv in .dagger/main.go, which mounts the whole repository minus repoExclude: VCS and worktree plumbing, .gitignore's entries, and dependency and build caches. check-frontend does the same for the frontend/ subtree with its own short exclude list.

A new path is therefore visible by default, and a missing exclude is a harmless over-inclusion. packages/converter/tests/test_repo_visible_in_ci.py pins this by reading docs/reference/campaign-yaml.md inside the test container. Keep repoExclude in step with .gitignore by hand; nothing parses one into the other.

The --ca-bundle flag

Every network-touching function accepts an optional --ca-bundle <file> (withCaBundle in .dagger/main.go) for networks behind a TLS-inspecting corporate proxy. It mounts the bundle and points SSL_CERT_FILE (Python, uv) and NODE_EXTRA_CA_CERTS (node, npm, bun) at it. The Makefile passes it whenever the bundle file exists, which is harmless on an open network:

CA_BUNDLE ?= /etc/ssl/certs/ca-certificates.crt
DAGGER_CA := $(shell test -f $(CA_BUNDLE) && echo --ca-bundle $(CA_BUNDLE))

The same file reaches make build-web (the ca build secret), make test-driver-real and the frontend targets. Without it, behind such a proxy, the viewer's npm install fails with an npm-internal crash rather than a certificate error, since node does not read the system store.

Makefile targets

• Workspace: install, format, lint, check (format + lint), test, typecheck, test-driver-real, ci (typecheck, then dagger checks and test), clean.
• Images: build (dagger call build-wrapper), build-wrapper and build-web and build-campaigns (plain docker build, tagged $(HTR_REGISTRY)/…:$(IMAGE_TAG)), poc-push (the wrapper and web builds, pushed to the dev registry, digests printed), the htrflow base built from a checkout and the refresh of its committed lock (lock-htrflow-base, Dev cluster), scan (dagger scan-json), scan-web (Trivy, HIGH/CRITICAL with a fix fails), scan-image (Trivy over an image in the local docker daemon, CRITICAL with a fix fails; how CI gates the wrapper it builds with plain docker build), publish (manual, needs DOCKERHUB_USERNAME / DOCKERHUB_TOKEN; refuses an existing tag).
• Compose: compose-up, compose-test, compose-smoke (both images built from the checkout, then the smoke — see Testing), compose-smoke-run (the smoke on any WRAPPER_IMAGE / WEB_IMAGE), compose-down.
• Charts: helm-lint, helm-template (lint, render both charts on defaults and ci/full-values.yaml, kubeconform).
• Cluster: install-kueue (Kueue's Helm chart at KUEUE_VERSION), install-kyverno (the admission controller security.policies.enabled needs, at KYVERNO_CHART_VERSION), install-devstack, psa-labels, campaigns-apply (render a campaigns repo, server-side apply its pipelines then its campaigns, sync each campaign's pause; PRUNE=1 adds --prune), e2e (validate, apply, then block until every campaign Job reaches a terminal condition; a Failed Job fails the target).
• Documentation: docs-serve, docs-build (through scripts/docs-site.sh), config-reference (regenerates docs/reference/configuration.md; a test asserts the committed page equals it). Slides: scripts/slides.sh builds the Marp decks at the top of docs/slides into site/slides after the site build (HTML and PDF, linked from Presentations). CI renders no diagrams, so render and commit a changed diagram locally; superseded decks go in docs/slides/archive.
• Contracts: api-contract (regenerates frontend/src/lib/fixtures/api-contract.json from real read-API output; a test asserts the committed fixture equals it — see Testing).
• Frontend: frontend-install, frontend-test, frontend-check, frontend-build, frontend-dev.

The cluster constants these targets use come from .env (Dev cluster).

Workflows

• ci.yml ("Tests") — on push to main, on pull requests and by hand. The ci job runs dagger call checks, dagger call test and the scripts/loc-budget.sh line budgets. A scan-campaigns job runs the converter image's CRITICAL gate on every trigger, pull requests included: it is a small distroless build with nothing to clone. Two scan jobs, scan-web and scan-wrapper, run scan-web and scan with --severity CRITICAL --ignore-unfixed on pushes to main and manual runs only (scan-wrapper then runs test-driver on the same build): each has to build its image first, and both builds are expensive (the wrapper's CUDA base; the web image's viewer clone and npm and bun builds). A pull request that changes a dockerfile gets its scan when it lands on main, before any image is published from it. A further job, on every trigger, builds the htrflow base and the wrapper on a native runner of the second architecture, runs the level-0 pin test on it and (on main) make scan-image, and pushes nothing.
• publish.yml — manual, one explicit tag per run; tests, builds, pushes, signs and attests all three images for both of the CPU architectures they ship for — each on a runner of its own architecture, joined into one manifest list per image (Releasing).
• ci.yml also runs, on every trigger: a docs job, the lint (the site's pages, the root and package READMEs and SECURITY.md: no project ids, dates, versions, hardware or one site's hosts) and the strict site build below without the deploy, so a pull request that breaks the site fails before it lands; verify-published; and a dagger-go job, go vet and go test over .dagger/publishcheck (Testing).
• docs.yml ("Documentation") — on push to main and by hand: uv sync --locked --only-group docs (zensical pinned and hash-checked in uv.lock), scripts/docs-site.sh build --clean --strict with that zensical, then deploy to GitHub Pages.
• security.yml ("Security") — weekly, by hand, and on pushes to main that change an image's inputs. One job per image (wrapper, web and campaigns): scan-sarif uploads the Trivy report to the Security tab, then the same CRITICAL gate as ci.yml runs, so an advisory published between changes fails a scheduled run. A further job does the same for the wrapper on the second architecture, on a native runner of it, through make scan-image. On a push the gate is skipped, since ci.yml has just run it.
• published.yml ("Published images") — weekly, by hand, and on pushes to main that move a pin (the chart's web image, the demo pipeline's wrapper, the Argo CD hook's converter image). It builds nothing: scan-published gates each pinned digest on both architectures, and verify-published checks their signatures, so a signature that stops verifying fails a scheduled run too. entrypoints-published runs here and not in ci.yml: a change that starts a new program fails it until the release commit re-pins the images, and that commit's push is the gate (Releasing).
• codeql.yml ("CodeQL") — on push and pull request to main and weekly: static analysis of the Python, frontend, dagger and workflow code.
• trufflehog.yml ("Secret Leaks") — on every push and pull request: TruffleHog over the git history; .github/trufflehog-exclude-paths.txt names the paths whose history holds dummy test credentials.
• scorecard.yml — OpenSSF Scorecard weekly, on push to main and on branch-protection changes, publishing the score the README badge shows.

No workflow uses the dagger-for-github action: it installs the CLI with a piped install script. .github/actions/setup-dagger downloads the CLI's GitHub release asset, checks it against the SHA-256 committed there, and starts the engine from its image digest; steps then run dagger call directly, with their arguments passed through env:. The action's version must equal engineVersion in dagger.json (a test asserts it).

Dependency pins

Every input is pinned. Dependabot (.github/dependabot.yml) proposes version updates weekly — one grouped pull request per ecosystem, a new major on its own, and nothing younger than a week — for the pins a registry can answer for: the actions, uv.lock, frontend/bun.lock, the dockerfiles' FROM lines and the dagger module's go.mod. Its security updates, enabled in the repository's settings, come as soon as an advisory is published. Every such pull request goes through the full CI. Patch and minor updates of the actions, uv.lock and frontend/bun.lock merge themselves once every required check passes (.github/workflows/dependabot-automerge.yml); a major version, a base image and the dagger module are merged by hand, because a new major is a migration and pull-request CI does not build every image it would ship. Everything else moves by hand, in a pull request of its own.

Inside the builds, dagger containers sync with uv sync --frozen --all-packages, and the wrapper image installs its dependencies from uv export --locked … --require-hashes, so a stale uv.lock fails the build instead of resolving freshly.