← Decisions operate mode →

Decisions

ADR-111: gyrum-ai-broker as the single model-routing layer

The recent fan-out of `type: claude` steps ([warp #334](https://warp.gyrum.ai/items/334)), structural checks for those steps ([warp #335](https://warp.gyrum.ai/items/335)) and the observe-and-file pattern ([warp…

#111

ADR-111: gyrum-ai-broker as the single model-routing layer

Status: Proposed Date: 2026-04-26 Related: ADR-067 (playbooks-unified-primitive), ADR-090 (ai-as-scaffold-pipelines-as-runtime), ADR-107 (gates-as-playbook-primitives), ADR-109 (type-claude-step-and-observer-field), ADR-110 (observe-and-file-dominant-pattern)

Context

The recent fan-out of type: claude steps (warp #334), structural checks for those steps (warp #335) and the observe-and-file pattern (warp #336) all assume that playbook authors pin a model directly in the step body — model: claude-haiku-4-5@2026-04 or similar. That works as a first cut but accumulates N integration points for what should be a single concern: every call site has to log audit independently, track its own budget, handle its own fallback when a provider hiccups, and be edited by hand when we want to migrate to a different model. The lock-in is structural — switching provider means walking every playbook.

The operator named the same problem from the cost angle on 2026-04-26: observers (the dominant pattern after ADR-110) are routine "read this log, classify as anomaly or not" calls that do not need Sonnet or Opus. A small open-source model — Llama or Mistral via Ollama on existing factory infrastructure — is sufficient and effectively free per call. Today's projected steady-state of roughly $3/month for observers can drop to ~$0/month, but only if the runtime knows how to pick a cheaper backend per call site without each playbook author having to think about it.

The repo gyrum-labs/gyrum-ai-broker already exists for this purpose — a routing layer that maps named routes to underlying provider models, with policy (budget, fallback, drift detection, audit) living in one place. The decision here is whether to make broker the only path for model calls in the factory, or leave it optional.

Decision

Every model call originating from the playbook runtime, sub-agents, and observers MUST route through gyrum-ai-broker. Direct provider calls (anthropic.Messages.create(...), OpenAI SDK, etc.) are deprecated outside the broker itself.

The schema for model selection in playbooks becomes:

- type: claude
  model: broker:dryrun-verdict@v3
  inputs: { ... }

broker:<route-name>@<route-version> replaces the today-form model: claude-<variant>@<date>. Routes are versioned (immutable once published); bumping a route requires a new version and a regression-corpus run before swap. Broker config is the single source mapping a named route to its primary model, fallback chain, max_tokens, max_latency, prompt_constraints, and audit-log path.

The broker grows three new backends beyond today's Anthropic/OpenAI: an Ollama adapter (local Llama/Mistral on existing factory infrastructure), a Together adapter (hosted open-source), and a generic custom-endpoint adapter for whatever turns up later. Observer routes default to Ollama-via-local-Llama with Anthropic Haiku as fallback when Ollama is unreachable or returns malformed output.

Existing direct-Anthropic call sites — likely in ai-research, devtools, ai-frontend — are migrated to broker calls. The Anthropic SDK becomes one possible broker backend, not a parallel call path. The runtime, when it sees model: broker:<route>, calls POST /api/v1/route/<name>?version=<v> on the broker rather than a provider directly.

Consequences

What becomes easier.

  • A single audit surface. Every model call is logged once, in the broker, with route + backend + cost + latency. The "where did this Claude call go?" question has one answer.
  • A single budget envelope. Cumulative token spend per route per day lives in broker state; emitting 429 when exceeded is one enforcement point, not N. ADR-090's "cost is structural" line lands here.
  • A single drift-detection path. The nightly drift corpus runs against broker routes, not against every step's prompt. Refactoring a route's underlying model is one config edit.
  • Provider switching is a config change, not a code change. Moving from Anthropic Haiku to OpenAI mini for a route = one broker config commit. No playbook edits.
  • Open-source fallback unlocks the cost win. The dominant observer pattern becomes effectively free; the projected ~$3/month becomes ~$0/month with Anthropic as warm-spare.

What becomes harder.

  • The broker becomes a critical path. Broker outage = playbook failures across the fleet. Mitigation: broker runs the same provision-host playbook as everything else (re-creatable infra), and the runtime's broker-unreachable behaviour is configurable per route — fail-closed for release_flow steps, fall-back-to-direct-provider with audit warning for routine observers.
  • Broker config becomes load-bearing. A bad route definition affects every call site using that route. Mitigation: route config goes through gyrum-validate-playbook-style validation on PR review, and route versions are immutable so a bad version cannot retroactively poison historical fires.
  • Two layers of indirection between author and provider. Debugging "why did this step return malformed output" now means inspecting both the playbook step and the broker route. Mitigation: broker audit log includes the resolved provider call so the call chain is reconstructible from one log file.

What we sign up to maintain.

  • The broker as a first-class service. Provisioned via playbook, monitored as carefully as the runtime, tested in CI like any other surface. It joins the factory-executor side (per ADR-112) — no internet exposure.
  • The route catalogue. Every named route documented with its purpose, primary model, fallback chain, declared output schema, and the failure-mode class it covers. Same shape as roles/gates/tasks/.
  • Backend adapters. Ollama + Together + custom-endpoint adapters added today; future provider additions land here, not in N call sites.
  • The migration of existing call sites. Every direct Anthropic call in ai-research, devtools, ai-frontend gets reshaped to a broker route before this ADR ships its acceptance criterion.

Alternatives considered

  • Leave model selection per-playbook, add a thin audit/budget wrapper. Considered and rejected — a wrapper that does not own the call path can be bypassed by any author who imports the SDK directly. The whole point is that there is one path; an opt-in wrapper is not one path.
  • Use a single hard-coded model everywhere. Rejected on the cost angle (observer work over-pays for Sonnet) and on the resilience angle (provider outage takes everything down). The broker exists to make per-call-site policy possible without per-call-site code.
  • Build broker functionality into the runtime directly. Rejected on substrate-count and on the runtime-stays-minimal line ADR-107 drew. Routing policy is not playbook execution; co-locating them means the runtime grows a second responsibility and the broker cannot be replaced independently.
  • Defer broker integration until a provider migration forces it. Rejected because a forced migration is the worst time to introduce an abstraction; it stacks two risks. Adding the broker now, while the fleet is small, lets the migration be incremental.

Supersedes: none Superseded by: leave blank until a later ADR reverses this one

References

  • ADR-067 — playbooks-unified-primitive (the primitive the broker call lives inside)
  • ADR-090 — ai-as-scaffold-pipelines-as-runtime (the line this ADR continues — runtime stays minimal, policy moves out)
  • ADR-107 — gates-as-playbook-primitives (the parallel "policy as YAML, runtime as minimal executor" pattern)
  • ADR-109 — type-claude-step-and-observer-field (the step shape that grows the broker: model prefix)
  • ADR-110 — observe-and-file-dominant-pattern (the observer routes are the primary cost-saving target)
  • warp #337 — implementation epic (this ADR is r1)
  • warp #321 — postmortem (motivating context for "single source of truth" patterns)
  • warp #334 — type: claude step (consumer of broker routes)
  • warp #335 — structural checks (cost + drift checks live broker-side)
  • warp #336 — observe-and-file (primary beneficiary of open-source fallback)
  • gyrum-labs/gyrum-ai-broker — the existing repo this ADR makes load-bearing
  • Memory: feedback_token_thrift_routine_agents_haiku.md — the principle this ADR generalises ("cheapest sufficient model for routine work")