ADR-019: Deploy order in multi-repo products

Status: Proposed Date: 2026-04-21

Context

Multi-repo products (frontend + backend + worker, typically also migrations) introduce a coordination problem single-repo products do not have: in what order do the components deploy when "deploy the product" is one human click?

Today's factory is single-repo products only. infrastructure/deploy.sh runs against one repo and leaves. Distill, Buzzy, and every static game are all role: all-in-one (per ADR-018); the question has never surfaced.

The product-concept doc proposes a "Deploy all" action on the product page that runs each component's deploy script in sequence. For that to be safe, it needs a default sequence. Four facts shape that sequence:

Schema before code. Code that reads a column before the migration that adds it gets either a nil value or a panic depending on language and driver. Any deploy ordering that runs code before its schema risks exactly this class of bug.
Backend before frontend. The frontend calls the backend. If the frontend ships a new call that only exists on a newer backend, and the frontend deploys first, there is a window (seconds to minutes per infrastructure/deploy.sh timings) where user requests hit a 404 on the backend. Backend-first makes the window "zero or new-frontend calls succeed"; frontend-first makes the window "new-frontend calls fail for every user."
Workers last. Workers consume the same schema and APIs but are not on the request path. A broken worker deploy does not block a user — the queue grows; once the worker deploys correctly, it catches up. Worker-first means the worker may hit a schema older than its code expects or may process a job against a backend that is still being restarted.
Factory is single-VPS. Deploy sequencing is not a distributed transaction. If step 2 fails after step 1 succeeded, we are not rolling back step 1 — the operator fixes step 2 and retries. This ADR is about ordering the happy path, not rollback-on-partial-failure.

Three products we can foresee wanting an override (per §5.1 of the product-concept doc):

A product whose worker is the schema author (not the backend). The owner can set deploy_order: 2 on the worker repo so it precedes the backend, expressing the actual dependency.
A product whose migrations must run concurrently across N shards. Not something we have today; flagged as an out-of-scope case the override syntax accommodates.
A blue/green frontend that needs to co-exist briefly with two backend versions. Out of scope; flagged in case a product surfaces it.

Decision

The default deploy order for a multi-repo product is:

migration → backend → frontend → worker

Keyed on the role field from .gyrum/product.yml (ADR-018).

Per-repo override via deploy_order: <int> in that repo's .gyrum/product.yml. Lower numbers deploy first. Ties are deployed in role alphabetical order for deterministic sequencing. Absent an override, each role has an implicit order: migration=10, backend=20, frontend=30, worker=40. A product whose worker must precede backend sets deploy_order: 15 on the worker.

"Deploy all" on the product page (see product-concept §4.7) sorts the product's repos by this rule and invokes the existing per-repo deploy.sh sequentially. It is not a new deploy tool; it is a new orchestration step on top of the existing tool.

On failure of any step, the sequence aborts and reports which step failed. The operator is expected to fix the failing step's repo and re-run the sequence; subsequent runs of successful steps are safe because deploy.sh is idempotent on an unchanged commit.

Single-repo (role: all-in-one) products are not affected — "Deploy all" on a monolithic product degenerates to "Deploy."

Consequences

What becomes easier. A new multi-repo product works out of the box. The operator does not think about order; the tooling does. Deploy scripts stay simple — no per-repo lockstep logic, no cross-repo health gates inside the deploy script itself.

What becomes harder. Overrides require editing .gyrum/product.yml on the affected repo and pushing — a small friction that keeps the order rule visible in the file rather than hidden in tooling. Teams hunting a flaky deploy of a multi-repo product have one more lever (the order) that could be wrong.

What we sign up to operate. The orchestration step on "Deploy all." The tie-breaking convention (alphabetical by role). The default-order numbers (10/20/30/40) — these are now a number-named constant in devtools; changing them is this ADR again or a successor.

What we accept as risk. The default order is wrong for some product we have not seen yet. The override mechanism (deploy_order: <int>) exists precisely for this case, but a team may not notice the default is wrong until a bad deploy. Mitigation: the first multi-repo product (whichever it is) treats the default as a prediction to test, and we amend if needed.

Non-consequence: this ADR does not promise rollback semantics. A partial failure is fixed forward. Rollback across multiple repos is harder and is a separate decision — flagged in product-concept §4.7 as worth revisiting if incidents show it is load-bearing.

Alternatives considered

No default order; operator picks at deploy time. Rejected. That moves a runbook into the operator's head every deploy — the exact opposite of the factory benefit.
Frontend first to fail fast on frontend bugs. Considered. The window of "user hits a route the old backend doesn't serve" is the thing it optimises against, but that window's cost (user-visible 404s) is strictly worse than the inverse (a brief "new frontend calls not yet wired" when the backend has already shipped). Rejected.
Parallel deploys (migrations then everything-else in parallel). Considered. In-principle faster; in-practice on one VPS the deploys compete for disk, CPU, and build cache. No measured win. Rejected for now; revisit if deploy throughput becomes a factory-level bottleneck.
DAG-based ordering from depends. Rejected. The depends list is a UI hint (see product-concept §2.5); promoting it to load-bearing for deploy sequencing would require cycle detection, transitive resolution, and a cross-product scheduler that we have no present-day demand for. If this ever becomes warranted, a successor ADR does that lift.
Deploy everything at once, rely on health checks to retry stale reads. Rejected. The failure mode is partially-correct user data written against the wrong schema — not retryable at the deploy layer.

Supersedes: none Superseded by: