ADR-161: Auto-deploy fires on `workflow_run` completion of `release.yml`, not on push to `main`

Status: Proposed Date: 2026-05-06

Related: ADR-092 (host/project separation), ADR-094 (compose bundled in image), warp#186 (deploy-drift / auto-deploy epic), warp#191 (r5 polling watcher), warp#192 (r6 push receiver), warp#1528 (Phase 1 — release-status visibility), warp#1530 (this Phase 2 ticket)

Context

The fleet's existing auto-deploy surface (warp#191 r5 + warp#192 r6) listens for push events on each opted-in repo's default branch. When a push lands on main, the receiver in ai-research/cmd/server/webhooks/github.go consults gyrum-catalog/manifests/<slug>.deploy.yaml for auto_deploy.on_merge: true and fires the deploy-project playbook. The playbook resolves :latest (or :main) on GHCR, pulls, and runs the host-side compose-up.

The problem this ADR closes is upstream of the receiver, not in it. Push lands on main before release.yml has run; :latest does not yet point at the new SHA. The deploy fires anyway and either (a) re-pulls the previously-built image (a wasted no-op deploy that operator-side metrics show as "deploy succeeded but nothing changed") or (b) races with the in-flight release.yml run and pulls a half-published image.

Today's session (2026-05-06) gave the empirical evidence:

Operator clicked deploy-project ten times in thirty minutes against warp.gyrum.ai.
Each click was a no-op because :latest had not actually been updated by release.yml (an upstream queue-jam delayed the build).
Even when :latest finally landed, the operator had to remember to click again — there was no signal that the deploy was now meaningful.
Operator's question "how do I know when it's deploying?" reflects the structural gap: the visible step (push to main) is decoupled from the meaningful step (image landed at :latest).

The structural fix is to invert the trigger source. The deploy is meaningful only when an image actually lands at :latest; the push that started release.yml is not. GitHub already publishes a workflow_run event with conclusion=success when the build job finishes. Listening for that event instead of push collapses the queue-jam class of failure: a deploy fires if and only if the image it needs is genuinely there.

Phase 1 of this epic (warp#1528, in flight) ships the visibility half — a /release-status page surfacing live-vs-:latest SHA per project. This ADR is Phase 2: closing the loop so the operator's role narrows from "watch + click" to "watch (or don't); receive notification on completion".

Decision

Auto-deploy fires on workflow_run completion of release.yml with conclusion=success, not on push to main. The push-driven path (auto_deploy.on_merge) is preserved through a deprecation window and superseded by auto_deploy.on_release once ADR-161's Phase 2 implementation lands across all opted-in projects. A project opts into the new semantics by setting auto_deploy.on_release: true in gyrum-catalog/manifests/<slug>.deploy.yaml. The webhook receiver in ai-research is extended to handle workflow_run events; the existing push handler remains as the legacy path during the migration window.

The shape has six parts: schema extension, receiver extension, loop guard, notification surface, failure-mode contract, and the migration strategy.

1. Schema extension (in `gyrum-catalog/pkg/deploy/manifest.go`)

AutoDeploySpec gains a sibling field. The struct stays a single block (not a per-trigger sub-struct) so projects opting into both during the migration window declare the intent in one place.

type AutoDeploySpec struct {
    // OnMerge enables the legacy push-driven trigger (warp#191 r5 +
    // warp#192 r6). Preserved for the ADR-161 deprecation window.
    // Default false.
    OnMerge bool `yaml:"on_merge" json:"on_merge"`

    // OnRelease enables the workflow_run-driven trigger (ADR-161).
    // The receiver fires deploy-project iff the workflow named
    // `release.yml` completes with conclusion=success on the default
    // branch. Default false.
    OnRelease bool `yaml:"on_release" json:"on_release"`
}

Schema rules:

Both flags default false. A project with neither flag set continues to deploy via operator-fired runs only.
Both flags true is allowed during the migration window — the receiver de-duplicates fires within a 10-minute window keyed on (slug, target SHA) so the on_merge fire and the on_release fire of the same SHA do not double-deploy. After the deprecation window closes, the validator begins rejecting on_merge: true (Phase 3 follow-up).

The schema file (pkg/deploy/manifest.schema.json) gains the new property; additionalProperties: false already on the spec means a typo at the YAML layer fails validation rather than silently defaulting.
gyrum-catalog's test corpus gains: schema-accepts-on-release-true, schema-accepts-both-flags, schema-rejects-typo-on_releace.

2. Receiver extension (in `ai-research/cmd/server/webhooks/github.go`)

The handler is extended to recognise X-GitHub-Event: workflow_run in addition to push. A new helper handleWorkflowRun is the dispatch body. The push and workflow_run paths share the verifyHMAC, manifest-lookup, opt-in-check, and firer call — only the trigger discrimination and the per-event payload-decoding shape differ.

// (sketch — implementation rides on warp follow-up filed against ai-research)
type workflowRunPayload struct {
    Action     string `json:"action"`           // "completed"
    Workflow   struct{ Name string `json:"name"`; Path string `json:"path"` } `json:"workflow"`
    WorkflowRun struct {
        Conclusion string `json:"conclusion"`   // "success" | "failure" | ...
        HeadBranch string `json:"head_branch"`
        HeadSHA    string `json:"head_sha"`
        Actor      struct{ Login string `json:"login"` } `json:"actor"`
    } `json:"workflow_run"`
    Repository struct {
        Name string `json:"name"`; FullName string `json:"full_name"`; DefaultBranch string `json:"default_branch"`
    } `json:"repository"`
}

Dispatch gates (in order; every short-circuit returns 200 OK):

action == "completed" — GitHub sends requested/in_progress/completed; only completed is decision-relevant.
workflow.path == ".github/workflows/release.yml" — the receiver dispatches on the canonical release workflow only. Other workflows (CI, lint, e2e) MUST NOT fire deploys; multi-workflow per repo is the common case.
head_branch == repository.default_branch — non-default-branch runs (PR builds, release branches) do not fire.
conclusion == "success" — failed/cancelled runs do not fire.
Manifest opt-in: auto_deploy.on_release: true for this slug.
Master switch: AUTODEPLOY_ENABLED=true.
Loop guard (see §3).

On all gates passing, the firer dispatches deploy-project with slug=<repository.name> and (new) an optional target_sha=<head_sha> hint. The hint is informational — deploy-project still resolves :latest from GHCR, but the SHA is logged so the operator-facing notification can quote it.

3. Loop guard

A release.yml run that fires because deploy-project ran (e.g. a deploy-tagging side effect that re-fires CI) MUST NOT trigger another auto-deploy. The receiver inspects workflow_run.actor.login:

If actor.login == "github-actions[bot]" AND the run was triggered by a tag push from the bot, drop without firing. The audit log records dropped: bot-actor loop guard.
If actor.login is a real user (the typical case after a merge), proceed.
Sliding-window dedupe: same (slug, head_sha) within ten minutes is dropped. Catches multi-workflow-completion echoes (e.g. release.yml triggers a release-notification.yml that GitHub also delivers via workflow_run).

The dedupe TTL is operator-tunable via env (AUTODEPLOY_DEDUPE_WINDOW_SECONDS, default 600). Implementation is an in-process LRU keyed on the tuple — durability across receiver restarts is not required because GitHub's redelivery semantics already include duplicate-protection at the delivery layer.

4. Notification surface

On a fired deploy completing (success or failure), the operator receives a notification. v1 ships the simplest legible surface:

Success. A bell on factory.gyrum.ai's nav surfaces <slug> now live at <version> (sha <short>). Link points at the deploy-project run output.
Failure. A louder notification — the bell goes red, the badge count increments, a one-line diagnosis (compose render error, image pull timeout, healthz timeout) is rendered, and the link points at the failed run with the relevant log section anchored.
No notification on dry-run / preflight reject. A receiver-side opt-out (e.g. :latest already matches live SHA, no actual deploy needed) is logged but does not surface in the operator UI — those are ambient, not events.

The notification source is the deploy-project playbook's terminal step writing a structured event to a factory-events topic that ai-frontend's nav-bell subscriber reads. The event schema:

event: deploy-completed
slug: <project-slug>
result: success | failure
version: <vX.Y.Z>
sha: <full-sha>
duration_seconds: <int>
diagnosis: <one-line>     # populated only when result=failure
run_url: <playbook-run-link>

The factory-events topic and the ai-frontend nav-bell subscriber are sister deliverables filed as follow-ups — see §Cross-references. v1 nav-bell-only is enough; session-log auto-population is a separate concern.

5. Failure-mode contract

The auto-deploy is a convenience, not the operator's only path. Every failure mode preserves the manual-deploy escape hatch:

Webhook never arrives. The polling watcher (warp#191 r5) keeps running; drift closes within the 60s poll cycle. The push-driven path also persists during the migration window.
Webhook arrives but receiver returns non-2xx. GitHub retries per its delivery contract; if all retries fail, polling backstops within 60s. Operator-side: Recent Deliveries on the GitHub webhook page shows the failure.
Receiver fires but deploy-project fails. Operator gets the louder notification (§4); the manual-fire button still works to re-run after the underlying issue is fixed. The auto-fire does not lock the project — a manual click on the failed slug re-enters the queue.
Loop-guard false positive. A real user push ten minutes after a bot push to the same SHA is dropped by the dedupe window. Cost: one missed deploy that the next push (or the operator's manual click) recovers within minutes. Acceptable trade for closing the bot-loop class of failure.

6. Migration strategy

Three phases over a defined window:

Phase 2 (this ADR; warp#1530). Schema extension + receiver extension + notification surface ship. Each opted-in project flips auto_deploy.on_release: true in addition to on_merge: true. The de-dupe window absorbs the double-fire. Validation: a real merge to warp's main produces exactly one deploy run within ~10 min, no operator click, with a notification on completion.
Phase 2.5 — soak. Run with both flags enabled for two weeks across warp + distill + ai-frontend. Dashboard-track: count of (on_merge fire dropped by dedupe) vs (on_release fire that became the real trigger). Soak-pass criterion: every successful deploy in the window was driven by on_release; the on_merge fire was always the dedupe-loser. (Confirms on_release is genuinely later than on_merge in every observed case, which it should be — the build takes time.)
Phase 3 — deprecation. Schema validator begins warning on on_merge: true; operator PRs flip every project to on_release: true only. After all projects are migrated, the manifest validator begins rejecting on_merge: true and the receiver's push-handler is removed in a follow-up PR. Rolled out in a single sweep so the schema and the receiver track. Phase 3 is filed as its own warp ticket; this ADR does not pre-commit the date — it depends on Phase 2.5 soak outcome.

Consequences

What becomes easier:

An operator merge to main produces a deploy without further clicks, and a notification on completion. The "is it deploying yet" question stops being asked.
The ten-no-op-clicks failure mode is structurally extinct: a deploy cannot fire before the image exists at :latest.
Operator visibility (Phase 1, warp#1528) and operator action (this ADR, Phase 2) compose: the /release-status page shows the gap, the auto-deploy closes it, and the notification surfaces the close.
The contract is uniform: every deployable project opts in via the same one-line manifest field. There is no per-project special-case.
Failure modes are louder than before — a deploy that fails on a real merge surfaces in the operator's UI immediately rather than being noticed on the next manual fire.
Multi-workflow repos are correctly disambiguated (only release.yml fires deploy) by construction; the path-match gate is a structural check, not operator memory.

What becomes harder:

The receiver now handles two GitHub event shapes (push + workflow_run). The test surface roughly doubles — the workflow_run path needs its own table-driven tests covering each gate, plus loop-guard cases.
The schema migration window requires both flags on each opted-in project's manifest temporarily. PR review must catch the case where a new project sets on_merge: true only (legacy default) — a guideline-version bump (per ADR-117) names on_release as the canonical opt-in going forward.

A misconfigured release.yml (e.g. workflow renamed to release-prod.yml) silently disables auto-deploy for that project — the path match is exact. Mitigation: the validator in gyrum-catalog/cmd/validate-deploy-manifest checks that the named repo has a .github/workflows/release.yml file at the manifest's declared default branch. Drift detection is structural at PR time.
The notification surface is a new dependency: ai-frontend's nav-bell subscriber must be live, the factory-events topic must be operational. Without them the auto-deploy still fires correctly, but the operator gets no completion signal — degrading to the warp#1528 /release-status page as the visibility-of-last-resort.
The loop-guard heuristics (bot-actor + sliding-window dedupe) are pragmatic, not formal. A pathological CI graph (a bot-pushed SHA that should genuinely deploy, e.g. dependabot security patches gated through workflow_run) is the false-negative class. Currently rare in the gyrum fleet; if it surfaces, Phase 4 introduces an explicit per-project actor_allowlist: knob.

What we sign up to operate:

A new auto_deploy.on_release field in every opted-in project's deploy manifest. Operator PRs flip it on; the catalog validator gates schema correctness.
Two webhook event handlers in ai-research's receiver. Both share verifyHMAC + opt-in checks; divergence between them is its own bug class. The test corpus pins parity.
A factory-events-driven nav-bell on factory.gyrum.ai. Outage of the bell is operator-visible (the page works without it; deploy completion is then noticed via /release-status polling).
The ten-minute (slug, sha) dedupe window's TTL choice. If migration evidence shows on_merge fires arriving more than ten minutes before on_release fires, the window widens; the env knob exists for this.
A migration deadline (Phase 3) that retires the push handler. Holding both shapes indefinitely is the failure mode where the legacy path silently keeps deploying while operators believe they're on the new path.

Alternatives considered

Keep push-driven (on_merge) and accept the queue-jam class. The ten-no-op-clicks failure mode is the cost. Today's session showed it is not theoretical — it happened. Status quo loses on the explicit operator-evidence axis.
Polling for :latest SHA change, no webhook (today's r5 watcher only). Drift-to-deploy latency is bounded by the poll cycle (60s). Webhooks drop typical drift-close to <10s and remove the polling cost (rate-limit pressure on the GitHub API at fleet scale). Polling stays as the backstop, not the primary surface.
Use the package event (GHCR publishes a webhook on image push). GHCR does emit package webhooks but the event lacks the head_branch / actor discriminators the loop guard needs, and the GHCR-event-to-image-name mapping is messier than the workflow-name match. workflow_run carries the structured metadata the dispatch decision needs in a single payload.
A new dedicated webhook endpoint (/api/v1/webhooks/github/release). Splits the receiver across two paths. Loses on operational simplicity — one path with internal dispatch is one rotation target for the GitHub-webhook secret; two paths means double-rotation. The path stays single; only the in-receiver dispatch shape changes.
A dark-factory-side trigger (project manifest at dark-factory/projects/<repo>.yaml) instead of catalog-side. This was the ticket's literal phrasing, but it would create a second per-project allowlist substrate competing with gyrum-catalog/manifests/<slug>.deploy.yaml's existing auto_deploy block. ADR-092 puts every per-project deploy-time fact in the catalog manifest by design; this ADR keeps that contract intact.
Defer Phase 2 until Phase 1 (/release-status) ships and operators decide if visibility-only is enough. Visibility makes the queue-jam legible; it does not close it. The operator still has to click after seeing the gap close. The ten-clicks evidence is exactly the case where visibility plus action are both required. Phase 2 ships in parallel with Phase 1's soak so the loop closes before the visibility data motivates a redesign.
Auto-rollback on deploy failure. Out of scope for v1 (per ticket). Operator manually fixes a failed deploy; the auto-deploy does not lock them out. Rollback automation is a separate ADR with its own tradeoffs (which SHA is "good"; how :previous is maintained; what the rollback's own gating looks like).

Cross-references

This ADR locks the design; the implementation rides on follow-up tickets across the three repos that carry the substrate:

gyrum-catalog — schema extension (OnRelease field on AutoDeploySpec, manifest.schema.json update, validator-tests). Filed as a sub-ticket of warp#1530 against gyrum-labs/gyrum-catalog.
ai-research — webhook receiver extension (handleWorkflowRun plus the gate cascade in §2, loop guard in §3). Filed as a sub-ticket of warp#1530 against gyrum-labs/ai-research.
ai-frontend — nav-bell subscriber + factory-events topic consumer. Filed as a sub-ticket of warp#1530 against gyrum-labs/ai-frontend.
dark-factory — opt-in flip in each project manifest (projects/<repo>.yaml is the project registry; the deploy manifest lives in gyrum-catalog/manifests/<repo>.deploy.yaml). Project-by-project flips are filed as one ticket per project after ai-research and gyrum-catalog land.

The Phase 2 acceptance demo (per ticket): a fresh merge to warp main → release.yml completes → auto-deploy fires → warp.gyrum.ai updates within 10 min, no operator click; failed-case demo with a deliberate compose error producing the louder notification.

Supersedes: none Superseded by:

ADR-161: Auto-deploy fires on workflow_run completion of release.yml, not on push to main