Mosaic one model, many lenses

ADR 0013: Knowledge sources can be git repositories

  • Status: accepted
  • Date: 2026-10-01
  • Supersedes: nothing (extends the vector_dbs[].sources construct from the knowledge engine, K1; the git CLI seam from K2)

Context

The knowledge engine (K1–K7) indexes local paths: every vector_dbs[].sources entry is a file or directory walked at boot. The use cases in sync/knowledge.md are about legacy systems — code bases and repositories that agents must study to re-implement or to quote from. A declared source that must be cloned by hand before the app can see it breaks the "one tessera.yaml describes the app" contract: the app's knowledge is no longer a function of its declaration + its working directory.

Two concrete gaps:

  1. No way to declare a remote (or bundled) repository as a knowledge source. sources[].path is a local path; a git URL is not a path.
  2. No hermetic way to prove it. CI runs offline-ish and must not depend on a network clone of an arbitrary repo; a test fixture must be a committed file, not a nested .git directory (which git cannot track).

Git itself is already an accepted external seam: K2 derives git facts (branch / commits / last commit / contributors) through the git CLI, fail-soft, and the engine crate stays environment-pure (Z7). Cloning through the same seam is consistent, not a new dependency class.

Decision

One DSL addition, one engine addition, no new runtime dependency:

  1. sources[].git — a source can be a git repository.

    vector_dbs:
  - name: kb
    sources:
      - path: legacyrepo            # optional label hint when `git` is set
        git: { url: "knowledge/legacyrepo.bundle", ref: "main" }
    • git.url is a git-clone URL: https://…, ssh://…, a local path, or a git bundle file (a single committed file that git clone accepts — this is what makes a repo fixture committable and CI-hermetic).
    • git.ref (optional) = branch/tag/rev passed as --branch.
    • When git is set, path is not required; the stable per-source label (used for entry ids and stats) is the last path segment of the URL with a trailing .git / .bundle stripped, slugified (deterministic, independent of any cache location).
    • Plan validation: git present ⇒ url non-empty; git absent ⇒ path non-empty (the previous rule).

  2. The engine clones through the git CLI seam, into a hidden cache. git::prepare_git_source(url, ref, base):

    • cache dir = <base>/.gitcache/<label> (a dot-dir: every source walk skips dot-dirs, so a cache can never be ingested by another source);
    • if the cache dir is already a git work tree it is reused as-is (deterministic boot, offline-friendly; update = delete the cache dir and reindex — documented);
    • otherwise git clone <url> <cache>, adding --depth 1 for non-local URLs (a local path / bundle is cloned in full — --depth is rejected for local clones by git itself). The clone runs with -c protocol.file.allow=always: the seam must clone local paths and bundle files from a non-interactive server process (no effect on remote URLs).
    • Post-clone verification. Some git versions exit 0 for a bundle clone whose HEAD ref the bundle does not contain ("remote HEAD refers to nonexistent ref" — observed on runners whose init.defaultBranch mismatches the bundled branch), leaving an empty checkout. The clone is therefore verified (rev-parse HEAD must resolve): when no ref was pinned, the first branch (local, else remote-tracking — sorted, so the recovery is deterministic) is checked out; with a pinned ref an empty checkout is an Err. A failed clone or an unrecoverable checkout removes the cache (never a half-clone).
    • any failure (git missing, bad URL, network) is an Err for that one source: the generated app warns and continues with the remaining sources (the existing fail-soft posture of ingest_source callers).
    • strategy.git: false turns the entire git layer off, including git sources (fail-closed for that source with a named error): "with and without git" stays a flag, not a fork.
    • entry ids for a git source are <label>/<rel-in-checkout> slugs (e.g. legacyrepo-billing-c), so a repo ingested from a URL and a local tree of the same repo get distinguishable, stable ids.
    • git facts (K2) are derived from the checkout, so a cloned source reports its branch / commits / last commit like any work-tree source.

  3. The vendored engine gains no dependency and no module. The clone lives in the existing git.rs (already vendored); Source gains git_url / git_ref. The generated knowledge_sources() emits the two fields. No new REST route: git sources are declared sources, ingested at boot and on reindex like every other source — the existing /stats, /search, /profile, MCP tools all see them without change.

Proof

  • Engine unit tests (hermetic, tempfile + the git CLI, skipped when git is absent): clone from a locally created repo, reuse of an existing cache, error on a bad URL; ingest_source on a git source produces entries with the <label>/… ids.
  • Committed fixture examples/full-stack/knowledge/legacyrepo.bundle — a git bundle of a tiny legacy repo (committed as a regular file; CI-hermetic, no network). The full-stack kb declares it as a git source; e2e asserts the boot stats list it, git facts are present, and its content is hybrid-searchable (the re-implementation use case: a repo the app has never seen on disk becomes quotable, citeable and profiled).
  • Conformance goldens re-pinned (the generated knowledge_sources() gains the git fields); workspace tests + clippy -D warnings + fmt clean.

Consequences

  • A generated app can now index repositories it does not hold on disk, from one line of DSL; the re-implementation playbook's first step ("profile the legacy repo") works against a URL or a bundle.
  • Caches persist between runs under .gitcache/; operators delete them to force a fresh clone. Shallow cloning is automatic for remote URLs (--depth 1) — large remote repos stay cheap; a ref pin makes a source reproducible.
  • What is deliberately NOT built here: remote fetching/syncing on reindex (reusing the cache is the contract), credential handling (the git CLI's own credential helpers apply, as with K2's git facts), and non-git VCS.
  • Scale note (follow-up, out of scope): the in-memory BM25 rebuild is the cost ceiling for very large corpora; the opt-in persistent index (tantivy, K4 roadmap) remains the scale path.