← Decisions operate mode →

Decisions

ADR-094: docker-compose.yml bundled inside the project's GHCR image

Today's first end-to-end exercise of the new ADR-092 deploy pipeline hit "needs source on disk" pain twice in one evening. The `deploy-project` ansible role's `compose.yml` task copies a `docker-compose.yml` from the…

#094

ADR-094: docker-compose.yml bundled inside the project's GHCR image

Status: Accepted Date: 2026-04-25 Related: ADR-092 (host/project separation — defines image-as-deployment-unit), ADR-093 (manifest-driven project tools — companion generic-shape decision)

Context

Today's first end-to-end exercise of the new ADR-092 deploy pipeline hit "needs source on disk" pain twice in one evening. The deploy-project ansible role's compose.yml task copies a docker-compose.yml from the control host's filesystem to the target host (project_compose_src), which means something upstream has to put that file on the control host. The current upstream is gh repo clone gyrum-labs/<project> into a /tmp/<deploy>/ workspace — once for warp (ai-research #658, the clone-fresh fix) and again when a missed rsync produced the protracted project_compose_src debate that consumed an hour of operator attention. Two unrelated failure modes, one shared root cause: the deploy needs a fresh project source tree on disk to find one file.

ADR-092 already named the cut: a project's deployment unit IS its image (ghcr.io/gyrum-labs/<project>:<tag>), and the host pulls images from GHCR rather than building them. The docker-compose.yml was the one piece of project shape left dangling outside that contract — declared in the project repo, copied through git/rsync to the deploy host, parsed by docker compose up. Operators correctly ask "where does compose live?" because the answer today is "the project repo on github.com, then a /tmp directory on the control host, then /srv/<project>/ on the target host." Three locations, three fetch mechanisms, three drift surfaces.

The cost per deploy is small individually — a gh repo clone is two to five seconds, a /tmp directory is free, github.com reads are free under the rate limit. But the cost compounds with two compounding factors: (a) every new project the fleet adds re-implements the clone-on-control-host step, and the implementations drift (cache vs. fresh, shallow vs. full, branch pin vs. tag pin); (b) every operator-facing failure ("compose file missing", "wrong version of compose", "clone hung on a private repo without GH_TOKEN") is a debug session against a moving target because the source on disk is ephemeral.

Two days into life under ADR-092 is the right time to close the dangling shape — before distill, buzzy, and the rest of the queued projects each implement their own clone-the-source step.

Decision

The project's docker-compose.yml ships inside the project's GHCR image as a baked-in file at /deploy/compose.yml. The deploy-project role extracts it from the image at deploy time using docker create + docker cp + docker rm, never gh repo clone and never a control-host workspace.

The contract every project's Dockerfile gains:

# Final stage, after the binary copy:
COPY docker-compose.yml /deploy/compose.yml

The contract the deploy-project role gains (in a follow-up PR, not this one):

- name: "compose — extract docker-compose.yml from project image"
  ansible.builtin.shell: |
    cid=$(docker create {{ project_image_backend }}:{{ project_image_tag }})
    docker cp "$cid:/deploy/compose.yml" "{{ project_install_dir }}/docker-compose.yml"
    docker rm "$cid" >/dev/null

The project's image is now a self-contained deployment artefact. docker pull it, extract the compose, run docker compose up. The host needs zero source-on-disk and zero github.com fetches to deploy.

Three rules follow:

  1. Every project's backend Dockerfile MUST COPY docker-compose.yml /deploy/compose.yml in its final stage. No exemptions — even backend-only projects with the placeholder nginx:alpine UI per ADR-092 still ship a compose file (a one-service stack is still a stack).

  2. The deploy-project role's compose.yml task extracts compose from the image, not from the control host's filesystem. Once the role flips (follow-up PR), project_compose_src becomes a derived value (extracted to a tmp path during the role run) rather than an operator-supplied input.

  3. Compose changes are deployment-shape changes and ship through the same release pipeline as code changes. A docker-compose.yml edit triggers a fresh GHCR image build the same way an api/ edit does. The image tag is the version pin for both the binary and its compose file simultaneously — they cannot drift from each other because they ship as one artefact.

Consequences

What becomes easier:

  • Deploy needs zero source-on-disk. The control host stops cloning repos into /tmp, stops carrying GH_TOKEN for private-repo access during deploy, stops paying github.com latency on every deploy. Image + manifest = enough.
  • The "where does compose live" question collapses to one answer. Compose lives inside the image. There is no second copy on disk anywhere except the deploy-time extraction (which is transient and not authoritative). Operator mental model gets simpler.
  • Compose and binary cannot version-skew. Today, a project could have docker-compose.yml on main referencing an environment variable the binary at the deployed image tag doesn't recognise — because compose is fetched from main while the image is pinned to a tag. After this ADR, the compose file at the deployed image tag is the compose file that matches the deployed binary. Skew becomes structurally impossible.
  • Rollbacks roll back compose too. docker tag rollback (pull an older tag, compose up from it) automatically reverts the compose-shape changes that shipped with that older binary. Today rollback is binary-only and operators have to manually git checkout compose to the matching commit.
  • GHCR becomes the single source of truth for "what is deployable." The image is the deployment unit, and "deployment unit" is no longer a category that requires an asterisk for "plus the compose file from the repo."
  • Tonight's two failures are structurally extinct. No clone means no clone-fresh bug class (ai-research #658). No control-host workspace means no missed-rsync project_compose_src debate.

What becomes harder:

  • Every project's Dockerfile gains one line. COPY docker-compose.yml /deploy/compose.yml near the top of the final stage. Cheap individually; the cost is the unconditional rule rather than any single line. We pay it because exempting projects re-introduces the conditional shape that ADR-092 exists to kill.
  • Compose changes can't be tested without rebuilding the image. Today an operator can edit docker-compose.yml on a deployed host and docker compose up -d to test the change in seconds. After this ADR, compose changes go through the GHCR build pipeline (~30s for a layer-cache hit, ~2-3 min for a full rebuild) before they can be deployed. Acceptable — compose changes ARE deployment-shape changes per ADR-092, and treating them as ad-hoc edits is exactly the drift the ADR is closing. For local iteration on compose itself, operators run docker compose up against a checkout in the project repo — the development loop is unchanged; only the deploy loop tightens.
  • The image gains a tiny non-binary file in its final layer. The compose file is a few KB; the layer overhead is negligible (distroless images are dominated by the binary itself). No measurable size impact on docker pull.
  • Two-stage Dockerfiles need the COPY in the right stage. It MUST live in the final runtime stage, not the build stage — otherwise the file never makes it into the published image. Easy to get wrong on first authoring; mitigated by the structural check that follows-up will add (docker create <image> && docker cp <cid>:/deploy/compose.yml - must succeed in CI before publish).

What we have signed up to operate:

  • A one-line COPY addition in every project's Dockerfile (warp first, this PR's pair; distill, buzzy, hello-world, and the queued projects in follow-ups as their first deploy under the new shape).
  • A change to dark-factory's project-deploy ansible role (deferred to a follow-up PR per the SWARM-P scope) that flips compose.yml from ansible.builtin.copy of project_compose_src to a docker create + docker cp + docker rm extraction from the project's image.
  • A structural check in gyrum-create-repo's scaffold (deferred follow-up) so any newly-bootstrapped project's Dockerfile ships with the COPY line by default and the absence of /deploy/compose.yml in a published image fails CI.

What we revisit:

  • If a project ever needs multiple compose files (e.g. one for db_mode: local, one for db_mode: external), the shape generalises to /deploy/compose.local.yml + /deploy/compose.external.yml and the deploy-project role picks the right one based on the manifest's db.mode. Not built today — same compose file with profiles: covers the current modes (per ADR-092's compose pattern).
  • If the cost of "compose changes require an image rebuild" becomes painful enough that operators start hand-editing /srv/<project>/docker-compose.yml on hosts to bypass it, that's a signal the development-loop ergonomics need a separate solution (e.g. a gyrum-deploy-compose-only shortcut that builds-and-pushes a fresh image with just the compose layer changed). Not pre-built; watch for the pain.

Alternatives considered

  • Keep cloning the project source on the control host (status quo). What today does. Loses on every dimension above: drift surface, version skew, github.com dependency at deploy time, "where does compose live" cognitive load. Rejected — the cost compounds with every new project the fleet adds, and tonight's two failures are not the last.
  • Publish docker-compose.yml as a separate GHCR artefact alongside the image. Could ship as ghcr.io/gyrum-labs/<project>-compose:<tag> (an OCI artefact) or as a release asset on the GHCR image's manifest. Rejected — adds a second artefact to publish (twice the GHCR push pipeline complexity), a second tag to keep in sync (skew returns), and a second registry interaction at deploy time (docker pull plus an oras pull or equivalent). The image is already a multi-file container; baking the compose into it costs a dozen lines in the Dockerfile and zero new infra. The OCI-artefact path is right when the artefact is genuinely independent of the image; compose isn't, it's the operating manual for the image.
  • Move docker-compose.yml to gyrum-catalog (the deploy manifest repo) instead of either repo. Ship gyrum-catalog/projects/<project>/docker-compose.yml and have deploy-project read it from there. Rejected — gyrum-catalog is the manifest-and-policy surface (per-host bindings, image tags, hostname triples), not a code-shipping surface. Putting compose there separates the project's compose from the rest of the project's source, complicating local development (operators would need both repos checked out to iterate). The image-bundled shape co-locates compose with the binary it composes — they ship and version together because they ARE the same release.
  • Keep compose in the project repo but have GHCR's build step push it as an OCI annotation on the image manifest. Technically clever (OCI annotations support arbitrary metadata, and docker manifest inspect can read them out without pulling the image). Rejected — annotations are size-limited (typically a few KB hard limit, project-dependent in practice), tooling support is uneven (docker manifest inspect parses them but docker compose does not), and the operator-facing extraction (docker manifest inspect | jq -r '.annotations["gyrum.compose"]' | base64 -d) is more brittle than docker create + docker cp. The bundled-as-file path uses primitive Docker mechanics that every operator already understands.
  • Wait until a third project hits the same pain before generalising. The "you ain't gonna need it" framing. Rejected — distill is queued as the second project, and waiting until it ships means the second project's first deploy re-implements the source-clone step (more work than adding one COPY line to its Dockerfile up front), and tonight's two failures become three. The COPY-line cost is low enough that the third-repeat heuristic doesn't apply; the pattern is cheap to make universal now.

Supersedes: none. Tightens ADR-092 (host/project separation) by closing the one shape — docker-compose.yml location — that the two-images-per-project rule left ambiguous. The image is now the complete deployment unit; nothing about a project's deploy lives outside its GHCR tag. Superseded by: {{leave blank until a later ADR reverses this one}}