Mosaic one model, many lenses

ADR 0023 — On-prem deploy: TLS issuer, app-level env, workflow-shaped chart

Status: accepted

Context

mosaic has a public on-prem deployment target (AGENTS.md → DevOps infrastructure): a single-node k3s cluster (li7) behind the *.eisler-systems.de wildcard, k3s Traefik on 80/443, cert-manager with a letsencrypt-prod ClusterIssuer, and a self-hosted GitHub Actions runner on the box (deploy-onprem.yml: render → docker build → k3s ctr images import → helm upgrade --install). The tessera deploy lens (ADR 0010, ingress refined in ADR 0020) must meet that workflow where it stands. Three gaps:

  1. No TLS in the tessera front-end. A public ingress on a wildcard domain needs a certificate; the on-prem cluster issues them with cert-manager. The chart's tls values existed (added for the workflow's --set overrides) but the tessera could not declare them — a spec that wants TLS had to rely on out-of-band helm flags.
  2. No app-level env. The generated app is configured by env (MOAIC_CHAT_API_KEY, MOAIC_EMBED_BASE_URL, MOAIC_JWT_SECRET, …). Part config env bindings exist (ADR 0016), but model/API keys are not a part's concern — there was no place for app-wide bindings, so the operator had to hand-write helm --set env.* for every deploy.
  3. Chart/workflow value mismatch. The workflow overrides image.repository / image.tag / image.pullPolicy, ingress.enabled, and db.password; the tessera chart rendered image as a single string, rendered the Ingress unconditionally (a hostless spec would install an empty-host rule — catch-all on a public controller), and left secret env secretKeyRefs non-optional (a missing AI key would wedge the whole Deployment).

Decision

Deploy spec: tls_issuer + tls_secret

deploys:
  - name: li7
    target: k8s
    host: pages.eisler-systems.de
    ingress_class: traefik
    tls_issuer: letsencrypt-prod     # cert-manager ClusterIssuer
    # tls_secret: pages-tls          # optional; defaults to <host, dots→dashes>-tls
  • tls_issuer enables the Ingress TLS block: the cert-manager.io/cluster-issuer annotation + spec.tls (host → secret).
  • tls_secret overrides the secret name; the default is the host with dots replaced by dashes plus -tls (the convention the workflow already used).
  • Fail closed (spec dropped, diagnostic raised):
    • deploy-tls-without-host — a certificate is issued for a host;
    • deploy-tls-secret-without-issuer — a secret name without an issuer is meaningless.

App-level env: app.env

app:
  env:
    MOAIC_CHAT_API_KEY:
      secret: true        # never inlined; the operator provides the k8s Secret
    MOAIC_EMBED_API_KEY:
      secret: true
    MOAIC_CHAT_BASE_URL: "https://openrouter.ai/api/v1"   # scalar = plain value
    MOAIC_JWT_SECRET: "dev-jwt-secret"
  • Each entry is var -> (value, secret, required): a scalar is a non-secret literal; a map is {value?, secret?, required?}.
  • A secret with a literal value is fail-closed (app-env-secret-literal, binding dropped) — a secret in the tessera would be inlined into the chart, which is exactly what the part-config rule already forbids (ADR 0016).
  • Plan order: app env first, then part config env bindings — a part binding for the same var wins (insert semantics, same as today's part env).
  • Render: non-secret entries land in the chart's env values map; secret entries land in the secrets list and render as secretKeyRef (secret name = key = env var name, as before).

Chart: workflow-shaped values

The per-spec tessera chart (ADR 0020's deploy/<spec>/helm) now matches what deploy-onprem.yml sets:

  • image: {repository, tag: latest, pullPolicy: IfNotPresent} — the runner builds the image, imports it into k3s, and overrides all three with --set (pullPolicy Never).
  • ingress.enabled — true when the spec declares a host, false otherwise; the Ingress template is guarded by it. A hostless spec installs no Ingress: on a public controller an empty host would match every request. (k3d-internal specs, e.g. data-sync, were hostless before and gain nothing from an Ingress object.)
  • tls: {enabled, secretName, clusterIssuer} derived from the spec's tls_issuer / tls_secret (the workflow may still override via --set).
  • Secret env secretKeyRefs carry optional: true unless the binding is required — the app boots without its AI key (the AI surfaces fail soft); a required binding keeps the wedge (boot fails loudly instead of serving a degraded app).
  • In-cluster store passwords stay required in the chart templates; the workflow derives a stable per-release password (sha256("$RELEASE-db")) so a redeploy does not rotate the credential under a live StatefulSet volume.

The workflow (deploy-onprem.yml) gains a repo input (default eugeis/mosaic) — a second checkout renders app repos that live outside the mosaic repository (e.g. eugeis/mosaic-pages) — and an AI key secrets step: when /home/ee/.ssh/openrouter exists on the runner it creates the MOAIC_CHAT_API_KEY / MOAIC_EMBED_API_KEY secrets in the namespace (one key feeds both, both are OpenAI-compatible families); without the file it warns and the deploy proceeds AI-less.

Consequences

  • A public on-prem app is declarable end to end from the tessera: host + ingress_class: traefik + tls_issuer: letsencrypt-prod renders a Traefik Ingress with a cert-manager certificate — no out-of-band helm flags.
  • Model providers (chat/embed/rerank) are configured per app via app.env + the model registry (ADR 0017), not per deploy flag; the same tessera deploys AI-less anywhere the secrets are missing.
  • Re-pinned conformance goldens: every tessera chart's values.yaml now carries the image map + ingress.enabled (full-stack, data-sync).
  • The legacy v0 lens (emit_helm) is untouched — it already had the workflow's image.repository/image.tag shape.