ADR-080: ADR-078 Phase 2c amendment — warp as the durable trigger queue

Status: Proposed Date: 2026-04-24 Extends: ADR-078 (trigger-driven playbook orchestration) — reshapes §14 Phase 2c; extends §4 dispatcher architecture; neighbours §6 migration 003. Related: ADR-068 (playbook runtime — claim state on playbook_runs), ADR-077 (agent coordination layer — approval gates on claimed runs), ADR-067 (playbooks unified primitive), warp (gyrum-labs/warp — the durable queue)

Decision (one paragraph)

ADR-078 ships trigger-driven orchestration in four phases; the Phase 2b MVP (cron-only, direct in-process invocation) is single-replica and best-effort-on-miss by design. This amendment names warp as the durable queue for Phase 2c onward: once warp's v2 contract stabilises (scaffolding / lifecycle / admin — gyrum-labs/warp#23, #24, #25, landed 2026-04-24), triggers enqueue a PlaybookRunSpec into warp instead of invoking the runtime directly, and ai-research becomes both producer (the trigger dispatcher) and consumer (a warp worker) of the same queue. The state machine grows a claimed position between enqueued and running — backed either by a new status value or a claimed_by / claim_expires_at pair on playbook_runs — so a worker crash releases the claim via warp's lease expiry and another replica resumes. Step-level idempotency (already shape-compatible with Phase 1b's playbook_run_steps) becomes a documented playbook-author requirement, not a runtime guarantee. Nothing in this amendment contradicts ADR-078; §14's Phase 2c row is reshaped from "webhook" to "webhook + warp enqueue", and §8 concurrency semantics move from replica-local to queue-mediated. Phase 2b stays direct-invocation and ships first; this amendment is the door 2c walks through.

Context

ADR-078 §4 dispatches cron ticks and webhook / alert fires by calling the existing runtime's enqueue path — the same code the PlaybookRunner UI button hits. That is correct for Phase 2b (one replica, MVP cron, health checks), and §7.4 is explicit that a missed tick stays missed.

But §14's Phase 2c onward is where the vision's operational loop lives (Hetzner → onboarding, alert → triage with ADR-077 approval gates, webhook-driven deploys). That loop needs four properties the direct- invocation path does not provide:

warp is the gyrum-labs agent work-coordination queue (gyrum-labs/warp). Its v2 contract split landed today — scaffolding (#23), lifecycle (#24), admin (#25) — which means the claim/lease/heartbeat shape and the admin endpoints we would build against are no longer moving targets. The v2 contract stabilising is the precondition this amendment names.

ai-research is already the runtime that executes playbooks. Giving it a second role — a warp worker — reuses the same binary, the same executors, the same SSE / event stream, the same audit trail. The only new integration is the warp client: enqueue on the producer side, claim / heartbeat / ack on the consumer side.

Decision

1. Phase shape for 2c onward

ADR-078 §14 reads (today):

2c — webhook on: webhook + secret/HMAC + payload_map:. Hetzner host-joined → host-onboarding.yaml. Vault wiring.

This amendment reshapes it to:

2c — webhook + warp on: webhook + secret/HMAC + payload_map:. Trigger dispatcher enqueues a PlaybookRunSpec onto warp; ai-research runs a warp worker that claims and executes. Direct-invocation path from 2b is removed once 2c lands; no dual-mode production state. Hetzner host-joined → warp → host-onboarding.yaml. Vault wiring. Exit criterion unchanged (first Hetzner host joins end-to-end with no human click).

Phase 2d (alert) and 2e (polled_change) inherit the warp path automatically — an alert fire and a polled-diff fire each enqueue a PlaybookRunSpec the same way a webhook does.

2. Mapping — trigger, queue, worker

The enqueue shape, one line:

warp.Enqueue(ctx, warp.Item{
    Spec:           PlaybookRunSpec{PlaybookID, TriggerKind, TriggerID, Inputs},
    Priority:       triggerPriority(kind),          // alert > webhook > cron > polled
    IdempotencyKey: idempotencyKey(trigger, fire),  // dedup key — see §3
})

The worker side:

1. Claim with a lease (TTL 60s, renewable via heartbeat).
2. Execute through the existing runtime — same playbook_runs insert, same SSE stream, same ADR-077 approval gates.
3. Ack on terminal status (success, failed, cancelled).
4. On crash: heartbeat stops → warp's lease expires → warp re-offers the item → another worker claims and resumes.

Two roles, one binary: the ai-research process enqueues on the trigger path (§4 of ADR-078) and consumes on the worker path (new). Same code, same deploy, no second service.

Warp kanban UI becomes a live, cross-trigger queue view — Pending | Claimed | Running | Done | Failed. The per-playbook Triggers section in ADR-078 §10 stays — warp UI shows the queue, playbook UI shows the runs. The warp export API (spec'd in #25 admin) becomes the cross- queue audit log on top of per-playbook playbook_run_events.

3. Idempotency key — the dedup seam

ADR-078 §4.3 already dedupes alerts inside a 5-minute window using (trigger_id, fingerprint). With warp in the loop, that same tuple becomes the warp idempotency key: warp rejects a duplicate enqueue inside its own window, so two replicas seeing the same alert POST produce one queued item, not two.

Per trigger kind:

4. State-machine implication — the claimed state

playbook_runs.status today (ADR-068 §5) runs enqueued → running → success | failed | cancelled. The queue-mediated path needs a step between "we put it on warp" and "a worker has it and is running it":

Two options. Pick one in the Phase 2c schema migration (migration 004, the companion to ADR-078's migration 003):

• Option A — new status value claimed. Cleanest read model: one column, one enum, three clear phases. Costs: every consumer of the status enum (UI, audit, metrics) learns the new value.
• Option B — claimed_by + claim_expires_at columns alongside existing running. The run jumps straight to running when a worker takes it; claimed_by tells you which worker, claim_expires_at tells you when warp will re-offer. Costs: two places to look instead of one.

Recommendation: Option A. The read-model cost is paid once; the two-place cost (B) is paid on every status check. Option A also lines up with warp's own lifecycle vocabulary (#24) — pending, claimed, running, done — so warp rows and playbook_runs rows tell the same story.

Overlap with ADR-078 §13 Q2. Q2 resolved that trigger history lives on playbook_runs + a trigger_fired event, not in a side table. claimed is a new state on the same row — it lands in the same migration family (004, follow-on to 003), not in a separate trigger-history migration. No contradiction; if anything the shape Q2 chose (trigger identity on the run) makes queue claim identity a natural neighbour column. Spelled out here so the 2c migration reviewer knows both concerns are expected.

5. What Phase 2b artefacts retire

Once Phase 2c lands, two Phase-2b structures stop earning their keep:

• playbook_cron_leases (ADR-078 §4.1) — warp's idempotency key on <playbook_id>:<trigger_id>:<unix_minute> owns the "first replica wins the tick" semantic. Keep the table for migration compatibility; stop writing to it. Drop in a later migration once no reader remains.
• In-memory per-replica rate-limit state (ADR-078 §7.3) — warp's admin API (#25) exposes queue depth and claim rate per priority, so backpressure becomes visible globally. Per-trigger token buckets stay at the door (they protect against auth-failure floods before warp sees the fire) but cross-replica drift (Q6) resolves itself: warp is single-source.

6. Prerequisites

• warp v2 contract stable. Scaffolding (#23), lifecycle (#24), admin (#25) merged 2026-04-24. ✓
• warp client library. A thin Go client (pkg/warp/client) in ai-research that speaks the v2 contract. New work, blocks Phase 2c.
• ai-research as a warp worker. Integration PR: add the claim / heartbeat / ack loop in a goroutine next to the trigger dispatcher. Dependency: warp client library.
• Step-level idempotency — playbook-author requirement. Phase 1b's playbook_run_steps already tracks per-step state (pending, running, success, failed), so a replayed run resumes from the first non-terminal step. This is shape-compatible today but semantically depends on step idempotency: a type: shell that runs apt install twice is fine; one that curls a payment API is not. Doc this in the playbook-author guide (follow-on) before 2c lands; surface in the gyrum-fire-trigger dry-run output as a warning when a playbook has no declared idempotency posture.

7. Phasing — reprise

2b direct-invocation path is removed when 2c ships; there is no feature-flag period where both paths run. Feature-flagged dual modes are a known source of silent divergence — the single-replica cutover is cheap because Phase 2b only wires one playbook (fleet-health-scan every 15 min), and cutting it to warp is a one-playbook migration.

8. Non-goals

• This amendment does not touch the warp repo. The warp v2 contract (#23, #24, #25) is the given; this ADR consumes it. Any warp-side change needed to support ai-research (e.g. bulk-claim, priority fairness tuning) is a warp ADR, not this one.
• No change to Phase 2b. Phase 2b ships as ADR-078 specifies. The amendment is future-facing.
• No new trigger kinds. The four in ADR-078 §3 are the set. warp is the transport, not a source.

Consequences

Easier.

• Server crashes stop dropping work. The runtime can restart, be redeployed, or be replaced; in-flight runs resume on claim expiry.
• Multi-replica trivially becomes correct. Two replicas both dispatch and both consume; warp's idempotency makes duplicate enqueues idempotent and its claim semantics make single-consumer execution the default.
• Backpressure is visible. Queue depth, claim rate, priority starvation — all in warp's admin API (#25), surfaceable in one dashboard pack rather than per-trigger metrics (ADR-078 §9).
• The cron-lease table goes away (slowly — see §5). One fewer thing to clean up nightly.

Harder.

• Two roles in one binary. ai-research is both producer and consumer. A deadlock (producer blocked waiting on a consumer slot that the same process owns) is newly possible; the worker pool must be sized greater than the producer burst or run out-of-process conceptually. Integration test owns this.
• Warp becomes a critical dependency. When warp is down, triggers cannot fire. Mitigation: warp runs on the same ops VPS with the same uptime budget as the runtime it serves; a short-term outage is the same outage as an ai-research outage. Long-term, warp HA is a warp ADR.
• Playbook authors learn idempotency discipline. A replayed step must be safe. Step-level idempotency is the author requirement 2c forces. Surface it in the YAML schema and in gyrum-fire-trigger before the first non-fleet-health playbook gets a warp trigger.
• Two lifecycle vocabularies to keep aligned. Warp rows and playbook_runs rows tell the same story and must stay in sync. Option A (§4) picks warp's vocabulary to minimise translation.

What we sign up to maintain.

• The ai-research ↔ warp integration surface (enqueue shape, idempotency-key derivation, claim-loop heartbeat).
• Migration 004 (the claimed state + claim_* columns).
• The playbook-author idempotency guide. Without it, Phase 2c retries become incident-generating features.

Alternatives considered

• Stay on direct in-process invocation forever. Rejected. Phase 2b's best-effort-on-miss is correct for health checks; it is wrong for Hetzner onboarding, deploys, or triage. The four properties in the Context table are non-negotiable for the operational loop.
• Build a bespoke queue inside ai-research. Rejected. A durable queue with lease semantics, idempotency, and an admin UI is warp's job by charter. Reinventing it inside ai-research duplicates the artefact we already committed to and splits operational attention.
• Temporal / SQS / Redis Streams. Rejected. Temporal was already rejected for runs (ADR-068 §16); importing it for the dispatcher fails the same tests. SQS / Redis Streams are external dependencies on a self-hosted stack (ADR-020) and give us less than warp does (no kanban UI, no cross-queue audit).
• Dual-mode (warp + direct, per-trigger). Rejected. Silent divergence between the two paths would be the dominant failure class. The single-playbook 2b footprint makes the cutover cheap.
• Put the claimed state in a separate playbook_run_claims table. Rejected. Q2 already chose the on-run shape for trigger identity; claim identity is the same kind of fact about the same row. A side table adds a join to every read.
• Make step-level idempotency a runtime guarantee, not an author requirement. Rejected for Phase 2c scope. The runtime cannot inspect arbitrary type: shell steps for idempotency. The right shape is "authors declare, runtime checks declaration, operator reviews declaration in PR" — future ADR if demand.

References

• ADR-078 — trigger-driven playbook orchestration; this amendment reshapes §14 Phase 2c, extends §6 (migration family), and resolves §13 Q6 (cross-replica rate-limit state) via warp being single-source.
• ADR-068 — playbook runtime; the state machine that grows a claimed phase in migration 004.
• ADR-077 — agent coordination layer; approval gates on a claimed run behave identically to today — the queue is upstream of the parking seam.
• ADR-067 — playbooks unified primitive; PlaybookRunSpec is the enqueue payload shape.
• warp v2 contract — gyrum-labs/warp#23 (scaffolding), gyrum-labs/warp#24 (lifecycle), gyrum-labs/warp#25 (admin) — merged 2026-04-24, the "warp stable" milestone this amendment cites as its precondition.
• Operator framing, 2026-04-24: "trigger→action dispatch should flow through warp rather than direct in-process invocation once warp stabilises."

Supersedes: none Superseded by: leave blank until a later ADR reverses this one