ADR-007: Layered redaction — keys, value patterns, message scrub (Phase 1 and 1.5)

Status: Accepted Date: 2026-04-21

Context

Every log line leaves the process with the possibility of embedding a secret. The failure mode is catastrophic and one-way: once a credential has reached Loki, it is on disk across backups, mirror replicas, and potentially off-box (Alloy WAL, future Grafana Cloud mirror per ADR-001). A compliance control that relies on "developers remember" is not a control.

Phase 1 shipped a key-only redactor: a list of ~20 well-known attribute key names (password, token, api_key...) whose values are rewritten to [REDACTED] before the slog JSON encoder sees them. We shipped it, ran an internal redaction audit against real logs, and found two classes of miss worth acting on immediately:

Values-that-look-like-secrets under innocent keys. "note": "bearer eyJ…", "ctx": "api_key=xxxxx from openai". The key-only layer can't see these.
Secrets baked into the message itself. log.Info("failed to parse token eyJ…"). The key-only layer can't see these either — Record.Message is a first-class field, not an slog.Attr.

Phase 1.5 (gyrum-go commit 9e47c23) adds two more layers without changing the library's public API.

Decision

Redaction is three layers applied to every log line emitted through observ.Logger() or observ.LoggerFor:

Attribute KEY match. Case-insensitive list of ~35 well-known sensitive key names (SensitiveKeys). Hit rewrites the value to [REDACTED].
Attribute VALUE pattern match. Ordered list of compiled regexes (SensitiveValuePatterns) matching known secret formats: bearer/basic header, JWT, AWS access key, GitHub PAT, Stripe live key, Slack token, PEM private key, URI with inline user:pass@. First-match-wins. Hit rewrites to [REDACTED:<tag>] — the tag is the shape that matched (jwt, bearer-header, ...), never the content. Tagging enables a Phase 5 "Redaction Events" dashboard that alerts on redactions by tag without exposing what was redacted.
Message scrub. Same value-pattern regexes are run as substring replacements across Record.Message. Implemented as a slog.Handler decorator (Pattern: Decorator — see msgRedactHandler in gyrum-go pkg/observ/msg_handler.go).

Both extension points (SensitiveKeys, SensitiveValuePatterns) are package-level vars, mutable from init() only — the hot path reads them without a lock.

The design and its limitations are documented in gyrum-go pkg/observ/README.md#redaction--layered-defence.

Consequences

~75% coverage of common leak patterns. The three layers together catch the accidental cases the internal audit classified as A and B (known-key leaks and well-known-format value leaks). Categories C/D/E from the audit — secrets buried in nested JSON payloads, novel token formats, entropy-based secrets — are explicitly not caught. The What is NOT caught list in the README is the permanent record of this gap.
Tagged redactions enable observability-of-redaction. [REDACTED:jwt] in a log line is a signal Phase 5 can count. A spike in [REDACTED:jwt] events indicates someone is logging tokens that shouldn't be there; the fix is a code review, not a grafana retrofit.
Developer discipline remains the compliance control. The security team's audit of this ADR is the record of that division of labour: the library is defence-in-depth, not contract. The standards codify the "never log" list.
False-positive ceiling is acceptable. The regexes are anchored and prefiltered; the prefilter is deliberately loose so no real match is ever silently dropped (see prefilterAllows in redact.go). Occasional false-positive redactions (an innocent string that happens to look like a token) are preferred over silent misses.
Extension has a narrow API. Adding a new sensitive key or pattern is a one-liner in init(). No framework changes needed. The library tests guard the existing set so regressions surface in CI.
The redactor is NOT a compliance control. Security-team reviewers look at this ADR when auditing. The stance is explicit: "we do not log that data" is the control; "the redactor catches it if we slip" is the safety net.

Alternatives considered

Key-only redaction (what Phase 1 shipped). Simple, low false-positive rate, covers common cases. Rejected as insufficient after the internal audit: values-under-innocent-keys and message-body leaks were both observed in real logs. We kept Phase 1's key layer and added two more.
Strict schema-based logging (structured loggers refuse attrs that aren't in a declared schema). Maximum safety. Rejected on ergonomics: every ad-hoc debug log becomes a schema-amendment PR, and the factory model depends on cheap logging. The cost of type-safe logging across four-plus services was higher than the cost of the residual leak risk after layered redaction.
Entropy-based scanning. Treat any high-entropy string as a probable secret. Rejected: false-positive rate explodes on innocent hashes, UUIDs, base64-encoded IDs. The engineering cost of tuning entropy thresholds per service outweighs the marginal catch rate, and the library's stance is "loud about what we catch, loud about what we don't".
Deep-walk into JSON-valued attributes. Redact inside body: "{\"password\": \"..\"}". Rejected for now: parsing untrusted JSON inside the logger adds an attack surface (DoS by deep recursion), and the standards already prohibit logging request bodies verbatim. A future ADR may revisit this if the audit shows meaningful leakage despite the existing prohibition.
Outbound log scrubber (redact at Alloy, not in-process). Defence-at-the-gate. Rejected as the sole control: a secret on disk inside the docker json-file is already "stored" for our purposes. We accept Alloy-side redaction as a future additional layer, not a replacement.

Supersedes: none Superseded by: