Skip to content

Make project config the source of truth for runtime docs loading - #162

Open
KayleeWilliams wants to merge 2 commits into
dx/153-git-source-groupsfrom
dx/154-docs-project-runtime
Open

Make project config the source of truth for runtime docs loading#162
KayleeWilliams wants to merge 2 commits into
dx/153-git-source-groupsfrom
dx/154-docs-project-runtime

Conversation

@KayleeWilliams

@KayleeWilliams KayleeWilliams commented Aug 3, 2026

Copy link
Copy Markdown
Collaborator

Closes #154. Stacked on #161, part of #157.

The artifact pipeline understands the whole project — collections, remote sync, inherited source config, mounts, navigation, route prefixes. The rendering path did not. leadtype init wrote both a config with product identity and navigation and a lib/source.ts restating the content root, nav, and base URL; multi-collection apps restated far more. Two descriptions of one project drift, and when they do the rendered site and the generated agent artifacts disagree about what exists.

Dogfooding found exactly that. This repo's Astro example rendered 51 pages while generate emitted 54 and advertised a "Leadtype REST API" nav group. The three OpenAPI reference pages were in llms.txt and the sitemap but had no route, because lib/source.ts never passed openapi. Nothing was wrong with either half — they were just two descriptions.

The primitive

const source = await createDocsProject({
  config: docsConfig,
  configPath: "docs/docs.config.ts",
  baseUrl: "https://example.com",
});

It returns a superset of DocsSource, so first-party adapters accept it with no parallel code path. Multi-collection projects get one merged, route-aware page API: listPages() tags each page with its collection, and loadPage() accepts a collection-local slug or the full route, so one handler can serve everything or a route can be mounted per collection. collections, sources, and getSource(key) expose the resolved graph for custom integrations.

Remote collections are cache-only. A request handler must never clone, so a cache that is missing, unverifiable, or holds a different revision than the config asks for fails naming leadtype sync — a stale checkout would otherwise render content that looks fine.

Shared inheritance

Source-owned inheritance and config loading move to config/inherit.ts and are now called by both build and runtime. Two implementations of "inherit this repo's navigation" would eventually disagree, which is the failure the shared content graph exists to prevent. That is the bulk of the diff and it is a mechanical move.

configPath fixes the content root and relative-path resolution using the same rules the CLI uses — docs.config.* inside the docs directory, leadtype.config.* at the root above it. createDocsSource() stays fully supported and is what the project is built on.

@coderabbitai

coderabbitai Bot commented Aug 3, 2026

Copy link
Copy Markdown

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: a8318968-87d8-48bc-870d-82d801724609

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review

Comment @coderabbitai help to get the list of available commands.

@pullfrog pullfrog Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Caution

The premise is right, but createDocsProject() is still a partial re-read of the config: five fields the CLI honors are dropped or resolved differently, and in a multi-collection project one of them makes /docs/1-0 silently serve changelog content. The build/runtime drift class this PR exists to close is still open on the multi-collection path.

Reviewed changes

  • packages/leadtype/src/project/index.ts (new, 437 lines)createDocsProject() runs source-owned inheritance, normalizes, then builds one createDocsSource() per resolved collection. Returns a structural superset of DocsSource plus collections, sources, getSource(key); listPages() tags pages with collection, buildSearchIndex() merges into one inverted index.
  • packages/leadtype/src/config/inherit.ts (new, 400 lines) — config loading + source-owned inheritance moved out of cli/generate.ts (filename lists, importConfigModule, the validateDocsNav/Groups/Mounts family, inheritCollectionSourceConfigs). I checked this as a faithful move; no behavior change on the build side.
  • Remote collections are cache-only at runtimeresolveCollectionDir() probes <cacheDir>/.git (the same probe sync/sync.ts:429 uses), reads the sync manifest, and rejects a cache whose repository/ref drifted from the config. Never clones. Four diagnostics, three tests.
  • apps/astro-example switches to createDocsProject({ configPath }); the four leadtype init templates switch to createDocsProject({ config, configDir }).
  • New docs page + minor changeset.

Verified as not problems, so nobody re-derives them: single-source URL parity is exact (normalize.ts:392-423 synthesizes routePrefix: "/docs", and collectionMounts()'s prepended {pathPrefix:"", urlPrefix:"/docs"} equals the hardcoded default at internal/docs-url.ts:69); the configPath content-root rule matches the CLI's configDir = path.dirname(configPath); listPages() caching mirrors the caching createDocsSource already does, so it isn't a new dev-server staleness bug; openapiCwd matches generate.ts:2834.

🚨 The new docs promise adapter compatibility that multi-collection projects don't have

The docs page says a DocsProject "is a DocsSource, so every first-party adapter takes it unchanged" and one route handler can serve everything. That holds for single-source. It does not hold for multi-collection, and no adapter file is touched by this PR.

Every adapter enumerates routes from page.slug — the collection-local, prefix-unqualified slug — never page.urlPath:

  • internal/framework.ts:77-82listJoinedSlugspages.map((page) => joinRouteSlug(page.slug))
  • next/index.ts:318-321createGenerateStaticParamspages.map((page) => ({ slug: page.slug }))
  • Nuxt's createPrerenderRoutes joins every collection's slug onto a single basePath.

Take this PR's own multiFixture, whose urlPaths the test at project.test.ts:189 pins as ["/changelog/1-0", "/docs", "/docs/auth"]. createGenerateStaticParams({ source: project }) on it yields [{slug: []}, {slug: ["auth"]}, {slug: ["1-0"]}]. Under a /docs/[[...slug]] route that means: nothing is generated at /changelog/1-0, a page is generated at /docs/1-0, and because loadPage matches the local slug first (see the inline comment on project/index.ts:339) /docs/1-0 renders changelog content. No adapter accepts a per-collection filter, so there's no way for a caller to fix this from the outside.

Either the multi-collection story needs adapter support in this PR (an adapter that consumes urlPath, or a getSource(key)-per-route pattern the docs spell out), or the docs need to scope the compatibility claim to single-source and say what multi-collection callers should do instead. Worth deciding explicitly rather than leaving readers to discover it at deploy time.

⚠️ Does the sync cache ship with the deployed artifact?

resolveCollectionDir() requires the .leadtype/ sync checkout to exist in the runtime process's filesystem, not just on the build machine. The new docs tell readers to run leadtype sync, which covers local dev and CI-build-then-run, but not the common deploy shapes where it silently won't be there: Next.js output-file-tracing won't trace a directory reached through a runtime path.resolve, and a Dockerfile that copies only .next (or a serverless bundle) drops it. The failure lands at request time as "run leadtype sync" on a machine where nobody can.

If remote collections at runtime are meant to be supported in production, the docs probably need a deployment note (ship the cache, or outputFileTracingIncludes). If they're dev-only for now, saying so plainly is enough.

⚠️ Coverage gaps in project.test.ts

The tests cover multi-collection routing, the merged search index, route-prefix collisions, and all three remote-cache rejections well. Three headline behaviors are untested:

  • No test passes configPath. Every test uses configDir, so the configIsSourceOwned branch (project/index.ts:222-232) — the rule that makes docs.config.* its own content root, which the astro example in this same PR depends on — has no coverage.
  • No test exercises inheritConfig through createDocsProject(). Source-owned inheritance is one of the PR's central claims and the reason config/inherit.ts was extracted, but it's only covered on the build path.
  • None of the dropped fields have a parity test. flatteners, openapi + collections, include/exclude, top-level mounts, and typeTableBasePath (all flagged inline) are exactly the set with no test — which is consistent with how they went unnoticed. A single table-driven "generate's page set == project.listPages()'s page set for config X" test would have caught all five and would keep catching the next one.

ℹ️ Nitpicks

  • config/inherit.ts:70 re-exports SOURCE_CONFIG_INHERIT_FIELDS and DEFAULT_SOURCE_CONFIG_INHERIT after declaring them const, while every sibling in the file uses inline export const. Artifact of the move; harmless.

Pullfrog  | Fix all ➔Fix 👍s ➔View workflow run | Using Claude Opus𝕏

Comment on lines +207 to +215
const withInheritance: DocsConfig = input.config.collections
? {
...input.config,
collections: await inheritCollectionSourceConfigs(
input.config.collections,
configDir
),
}
: input.config;

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Inheritance runs against the authored config; generate.ts:2726 runs it against the normalized one. That ordering difference makes inheritance silently skip entirely for two supported authoring shapes:

  • sources: {...} git-source groups (the feature this branch is based on). Normalization is what flattens sources into collections (normalize.ts:428), so on the authored config input.config.collections is undefined → the ternary takes the input.config branch → inheritCollectionSourceConfigs never runs. The CLI applies it.
  • The deprecated sourceConfig alias, which only normalization folds to inheritConfig. inherit.ts reads inheritConfig, so on the pre-normalized config the alias is invisible here and honored by the CLI.

In both cases a pinned source repo's navigation/groups/frontmatterSchema land in the generated artifacts but not in the rendered site — the precise drift this PR exists to eliminate.

Normalizing first and inheriting into resolved.collections (as generate does) would make the two paths agree by construction.

Comment on lines +278 to +280
...(config.openapi && resolved.mode === "single-source"
? { openapi: config.openapi, openapiCwd: configDir }
: {}),

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The resolved.mode === "single-source" guard drops OpenAPI pages entirely for multi-collection projects. generate.ts has no equivalent guard: writeOpenApiPages runs whenever metadata.openapi is set (generate.ts:2826-2838), and the pages it emits get nav nodes, sitemap entries, llms.txt lines, and search-index documents.

openapi is a top-level DocsConfig field with its own urlPrefix (openapi/index.ts:58-100), independent of collections, and nothing validates the combination — only navigation/groups are mutually exclusive with collections. So this config is legal today:

collections: { docs: { dir: "docs" }, api: { dir: "api", routePrefix: "/api" } },
openapi: { input: "openapi.yaml", urlPrefix: "/api-reference" },

generate emits /api-reference/* and advertises it everywhere; project.listPages() never produces it. That is the "generate emitted 54 pages, the site rendered 51, three routes advertised with no page behind them" incident from the changeset, reproduced exactly.

If multi-collection OpenAPI genuinely can't be attributed to a collection yet, throwing on openapi + collections would at least keep it loud instead of silent.

...(config.openapi && resolved.mode === "single-source"
? { openapi: config.openapi, openapiCwd: configDir }
: {}),
...(authored?.flatteners ? {} : {}),

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This spread is a no-op — both branches evaluate to {}, so it compiles, type-checks, and does nothing. flatteners is never forwarded anywhere in this file, and top-level config.flatteners is never read at all.

CreateDocsSourceConfig (source/index.ts:134-186) has no flatteners field, so there's no accidental pass-through either — reaching parity means converting them into remarkPlugins before the createDocsSource call.

This matters because both other paths treat flatteners as real: generate.ts:1805-1812 merges top-level and every collection's into one list applied to every page, and inherit.ts:370 explicitly inherits them from a source-owned config. A user with a custom component flattener gets flattened markdown in llms.txt/search and the raw component in the rendered site.

If threading them is out of scope for this PR, deleting the dead line and noting the gap in the docs would be clearer than a line that looks like it handles the case.

// A single-source project mounts at its route prefix like any other;
// a multi-collection project gets one mount set per collection, which
// is what makes each collection's URLs correct on its own.
mounts: collectionMounts(collection.routePrefix, collection.mounts),

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Top-level config.mounts is dropped for multi-collection projects. normalize.ts:370-424 folds config.mounts into the resolved collection only in the single-source branch; in multi-source mode it survives only in top-level provenance and never lands on any ResolvedDocsCollection.mounts. So collection.mounts here is per-collection mounts only.

generate.ts combines both regardless of mode: mounts: loaded.config.mounts (1832) and const effectiveMounts = [...mounts, ...(metadata.mounts ?? [])] (2912, where mounts is sourceMounts(docsSources)).

Nothing forbids top-level mounts alongside collections, so a site-wide mounts: [{ pathPrefix: "legal", urlPrefix: "/legal" }] remaps URLs in the generated artifacts and not in the rendered site. Reading config.mounts here (appended after the collection's own, matching generate's order) would close it.

Comment on lines +275 to +277
...(input.typeTableBasePath
? { typeTableBasePath: input.typeTableBasePath }
: {}),

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

typeTableBasePath is read only from input, with no ?? config.typeTableBasePath fallback — while its immediate sibling three lines up does exactly that (typeTableStrict: input.typeTableStrict ?? config.typeTableStrict, line 239). generate.ts:1838-1839 resolves it from the config against srcDir.

Since none of the four init templates pass typeTableBasePath, a user who sets it in docs.config.ts — the documented place — gets correct <AutoTypeTable> resolution at build time and the default base path at render time. The asymmetry with typeTableStrict suggests this is an oversight rather than a decision.

Comment on lines +250 to +256
for (const collection of resolved.collections) {
const contentDir = await resolveCollectionDir(
collection,
config,
configDir,
fallbackContentDir
);

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Per-collection include/exclude are never applied on this path, and structurally can't be: CreateDocsSourceConfig has no such option and source/index.ts:502 globs **/*.{md,mdx} unconditionally under each content root.

They are a first-class field, not vestigial — validated at generate.ts:1100-1107, turned into GenerateFilters at generate.ts:1979-1993, and applied as a genuine page-existence filter at generate.ts:2067,2076.

The consequence runs the opposite direction from the other gaps here, which is why I'd treat it as the most serious: collections: { docs: { dir: "content", exclude: ["drafts/**"] } } correctly keeps drafts out of the build, and listPages()/loadPage() enumerate and serve them anyway. An author using exclude to keep internal or unpublished content off the site does not get that at runtime, with no error or warning.

Threading include/exclude into createDocsSource and filtering in the glob is the real fix; short of that, throwing when a collection declares them would keep it from failing silently.

Comment on lines +339 to +341
const target =
pages.find((page) => page.slug.join("/") === wanted) ??
pages.find((page) => page.urlPath.replace(LEADING_SLASH, "") === wanted);

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Matching the collection-local slug before the prefix-qualified route path makes cross-collection lookups ambiguous, and the collision guard above (303-317) can't catch it because it compares urlPath — which distinct route prefixes make distinct by construction, so it never fires for a slug collision.

Two collections that each contain overview.mdx both have slug: ["overview"]. loadPage(["overview"]) returns whichever collection is declared first, deterministically and silently. Same for the near-universal index.mdx, whose slug is [] in every collection.

This is also what turns the adapter mismatch (see the review body) from "missing pages" into "wrong content": adapters emit static params built from page.slug, so /docs/1-0 in the multi-collection fixture resolves through this line to the changelog page.

Trying the route path first, or extending the collision guard to slug.join("/"), would make it either correct or loud.

nav: docsConfig.navigation,
export const source = await createDocsProject({
config: docsConfig,
configDir: process.cwd(),

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All four templates pass configDir: process.cwd() (here, plus 256 / 335 / 470) while the scaffolded config lands at docs/docs.config.ts (line 65). Everything the CLI resolves relative to the config directory therefore resolves one level up at runtime: configDir = path.dirname(configPath) = <root>/docs for the CLI (generate.ts:1474) vs <root> here.

Latent today because the scaffolded config only contains product — but it activates the moment a user adds openapi: { input: "./openapi/spec.yaml" }, since openapiCwd: configDir (project/index.ts:279) inherits the wrong base. That's the same class of failure the changeset opens with.

configPath avoids the whole question: it derives configDir itself, the astro example in this PR uses it, and the new docs page calls it "preferred over configDir". Since the templates already import the config by relative path, configPath: new URL("../docs/docs.config.ts", import.meta.url).pathname (or the framework's equivalent) would keep the scaffold on the documented path.

Comment on lines +193 to +195
const postings = Object.values(bundle.index.terms).flat();
expect(postings.length).toBeGreaterThan(0);
expect(bundle.index.chunks.length).toBeGreaterThanOrEqual(3);

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These two assertions don't check the invariant the comment above them describes. postings.length > 0 and chunks.length >= 3 hold just as well for two concatenated per-collection indexes with postings pointing at the wrong documents — the failure mode named in the comment. The exact documents assertion above is what actually carries this test.

Resolving one term to its posting and asserting the document it points at (e.g. a changelog-only term resolving to the /changelog/1-0 document id) would pin the merge.

@KayleeWilliams
KayleeWilliams force-pushed the dx/154-docs-project-runtime branch from cf5e310 to 32b4dfc Compare August 4, 2026 07:43

@pullfrog pullfrog Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Important

This push is a rebase, not a revision — every file this PR owns is byte-identical to the previously reviewed cf5e310, and all nine open threads still stand. What the rebase does change is the contract underneath resolveCollectionDir(): the base branch now records sparse in the sync manifest and treats a path-set mismatch as an unusable cache, and the runtime check was not extended to match.

Reviewed changes — the delta between cf5e310 and 32b4dfc is entirely base-branch movement pulled in by a rebase; no commit in this PR's own scope changed.

  • Rebased onto dx/153-git-source-groups with sparse git-source support. The base gained sparse-checkout for pinned sources: sparse on DocsCollection/GitSourceConfig, cascade through expandGitSources/resolveSources, git clone --filter=blob:none --sparse + sparse-checkout set -- in sync.ts, the path set recorded in the sync manifest, and validation in validateCollections.
  • Picked up one generate.ts change. The only PR-touched file the rebase modified is cli/generate.ts, which gained the base's sparse validation in validateCollections. The extraction into config/inherit.ts that this PR performs is untouched by it — no semantic conflict.
  • Left this PR's own diff unchanged. project/index.ts, project/project.test.ts, config/inherit.ts, index.ts, cli/init-templates.ts, apps/astro-example/src/lib/source.ts, the docs page, and the changeset are identical to the reviewed revision.

Re-verified so nobody re-derives it: normalizeDocsConfig returns config: { ...withoutSources, collections: canonicalCollections } (normalize.ts:485-487), so the normalized config.collections is populated for a sources-shaped config, cascaded sparse included. resolveCollectionDir reads that normalized map, so a remote collection declared under sources does resolve to its sync cache. That scopes the open inheritance finding to inheritance alone — it is not also a content-root bug.

ℹ️ The sync manifest is validated in two places with no shared predicate

sync.ts and project/index.ts each decide independently whether a cache matches the config, and this rebase is the first demonstration that the two drift: the sync side learned a new field and the runtime side did not. sameSparse() is private to sync.ts, so there was nothing for resolveCollectionDir to call even if the author had thought to.

Worth deciding whether that seam should exist, because the next manifest field will have the same shape. This is a judgement call, not a defect — the inline finding is the actionable half.

Technical details
# One predicate for "this cache matches the config"

## Affected sites
- `packages/leadtype/src/sync/sync.ts:167-180``sameSparse`, module-private.
- `packages/leadtype/src/sync/sync.ts:521-525``syncOne`'s `manifestMatches`: repository + ref + sparse.
- `packages/leadtype/src/project/index.ts:175-185` — the runtime's version of the same decision: repository + ref only.

## Required outcome
- Adding a field to `SyncManifest` should not be able to leave one of the two consumers silently behind.

## Suggested approach (optional)
- Export a single `syncManifestMatches(manifest, source)` from `sync/sync.ts` and have both `syncOne` and `resolveCollectionDir` call it. The two call sites differ only in what they do on a false result (re-clone vs. throw), not in what "matches" means.

## Open questions for the human
- Is the runtime deliberately more permissive than `sync` on any manifest field, or is repository + ref simply the set that existed when `resolveCollectionDir` was written?

Pullfrog  | Fix all ➔Fix 👍s ➔View workflow run | Using Claude Opus𝕏

`createDocsProject: the cache for collection "${collection.key}" at "${cacheDir}" has no sync manifest, so its revision can't be verified. Run \`leadtype sync --refresh\`.`
);
}
if (manifest.repository !== repository || manifest.ref !== ref) {

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The rebase made this check incomplete. syncOne now folds sameSparse(manifest.sparse, source.sparse) into manifestMatches (sync/sync.ts:521-525), so a cache checked out with a narrower path set than the config asks for is not a usable cache and gets re-cloned — but here only repository and ref are compared, and sparse appears nowhere in this file.

The consequence is the silent half of the split: content the docs reach outside the collection dir still renders as if it were there. sparse: ["docs", "packages"] — the base branch's own documented example, where packages exists solely so <AutoTypeTable path="./packages/…"> can read real TypeScript — degrades to empty type tables against a cache synced before packages was added, while generate (which re-clones) gets the full tree.

Technical details
# `resolveCollectionDir` accepts a cache `leadtype sync` would reject

## Affected sites
- `packages/leadtype/src/project/index.ts:175-185` — reads the manifest, compares `repository` and `ref`, never `sparse`. `resolved.remote.sparse` is already available: `resolveCollection` populates it (`sync/sync.ts:148-150`) and `readSyncManifest` validates `manifest.sparse` as `string[]` (`sync/sync.ts:244-248`).
- `packages/leadtype/src/project/index.ts:186-190` — when the collection `dir` *itself* falls outside the stale path set, the error raised is the generic "expects `<dir>` inside `<repo>@<ref>` … Check the collection's `dir`". That misattributes the cause: the `dir` is correct, the checkout is narrow.

## Reproduction
1. Collection with `repository`, `ref: "main"`, `sparse: ["docs"]`. Run `leadtype sync`.
2. Add `packages` to `sparse` (or add a `<AutoTypeTable>` that needs it) and do not re-run sync — the ordinary shape of a dev-server restart, or a container image built before the config edit.
3. `createDocsProject()` passes every check: `.git` exists, manifest parses, `repository`/`ref` match, `dir` exists.
4. Type tables resolve to nothing. No error, no warning.

## Required outcome
- A cache whose recorded path set differs from the configured one must fail here, naming `leadtype sync --refresh`, in the same voice as the existing revision-mismatch diagnostic.
- The `dir`-missing diagnostic should not blame `dir` when the real cause is a narrow checkout.
- `project.test.ts`'s remote-cache block covers the three existing rejections exactly; a fourth case belongs alongside them.

## Suggested approach (optional)
- Export `sameSparse` (or a whole-manifest predicate — see the review body) from `sync/sync.ts` and add the comparison to the existing mismatch branch, reporting both path sets the way `formatSparse` already does for the collision error at `sync/sync.ts:211-213`.

@KayleeWilliams
KayleeWilliams force-pushed the dx/154-docs-project-runtime branch from 32b4dfc to da7ab52 Compare August 4, 2026 09:22

@pullfrog pullfrog Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ℹ️ The new commit is a clean, minimal change and I verified it doesn't move any route — two suggestions inline. Note that it does not touch any of the ten open threads from the previous reviews: 46dc63a is byte-identical to the previously reviewed 32b4dfc for every file this PR owns.

Reviewed changes — the delta since the prior review is a single new commit, da7ab52; everything else in the range is base-branch movement pulled in by the rebase (apps/next-example/leadtype.config.ts, docs/docs.config.tsdefineDocsConfig, committed apps/sveltekit-example/static/* artifacts).

  • Taught the fumadocs adapter to accept an already-built source. fumadocsSource() now takes FumadocsSourceConfig, a union of the old CreateDocsSourceConfig shape and { source: DocsSource }, discriminated at runtime by "source" in config. fumadocs is the only adapter that calls createDocsSource() internally — every other adapter already receives a source — so nothing else needed the same treatment.
  • Rewrote the fumadocs example onto createDocsProject. apps/fumadocs-example/lib/source.ts drops the restated contentDir / nav / mounts / openapi in favour of { source: project }, keeping typeTableBasePath: repoRoot and includeMetaJson: false.
  • Documented the project path on the fumadocs integration page, plus a docs/paths.lock.json hash refresh.

Verified so nobody re-derives it: the example's page set, slugs and navigation are unchanged. configIsSourceOwned is true for docs.config.ts, so the content root resolves to <repoRoot>/docs exactly as the old resolve(repoRoot, "docs") did; nav passes through normalization verbatim (normalize.ts:410); openapiCwd: <repoRoot>/docs equals the old implicit config.openapiCwd ?? sourceContentDir default (source/index.ts:446); typeTableBasePath is forwarded (project/index.ts:275-277); and the prepended {pathPrefix: "", urlPrefix: "/docs"} mount only duplicates the fallback resolveDocsPathMount already inserted (internal/docs-url.ts:66-70) — and is moot here anyway, since fumadocsSource derives slugs from meta.slug, never meta.urlPath. On types: DocsProject only widens the return types of listPages/loadPage (project/index.ts:77-88), so it is assignable to DocsSource; bun --filter leadtype check-types passes, and tsgo --noEmit in apps/fumadocs-example reports only the two pre-existing missing-public/ artifact errors, nothing in lib/source.ts.

ℹ️ Nothing in CI exercises the new adapter branch

CI runs build, lint, check-types and test against packages/leadtype only (.github/workflows/ci.yml:33-53); no example app is ever built or typechecked. The new { source } branch has no unit test — src/fumadocs/ has no test file at all — and its only consumer is apps/fumadocs-example, which CI never touches. So the one thing that would catch a regression in the branch (or in the example's rewrite) is a human running it locally.

Technical details
# The `{ source }` branch has no automated coverage

## Affected sites
- `packages/leadtype/src/fumadocs/index.ts:104-105` — the new `"source" in config` branch; no test file exists anywhere under `src/fumadocs/`.
- `apps/fumadocs-example/lib/source.ts` — the only consumer of the branch.
- `.github/workflows/ci.yml:33-53` — every step is `bun run --filter leadtype …`; no example app is built or typechecked.

## Required outcome
- A regression in the pre-built-source path should fail CI rather than surfacing when someone next runs the fumadocs example.

## Suggested approach (optional)
- One test in a new `packages/leadtype/src/fumadocs/fumadocs.test.ts`: build a fixture with `createDocsProject`, pass it as `{ source: project }`, and assert the returned `files` entries match the project's `listPages()` output (paths, `slugs`, `data.title`) and that `result.leadtype === project`. That pins the branch without needing the example app.

## Open questions for the human
- Is the fumadocs adapter deliberately untested (thin-mapping-layer judgement call), or is this simply a gap nobody has filled yet?

Pullfrog  | Fix all ➔Fix 👍s ➔View workflow run | Using Claude Opus𝕏

Comment on lines +97 to +99
export type FumadocsSourceConfig =
| (CreateDocsSourceConfig & { includeMetaJson?: boolean })
| { source: DocsSource; includeMetaJson?: boolean };

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Because this is a union, TypeScript relaxes the excess-property check across members — a key is allowed if it appears in any constituent. So fumadocsSource({ source: project, typeTableBasePath: repoRoot, nav: [...] }) compiles cleanly and silently drops everything but source. That is exactly the migration this commit performs, in reverse: a caller who moves to a project but forgets to delete one of the CreateDocsSourceConfig options loses it with no diagnostic — the silent config drift this PR exists to close.

Technical details
# `FumadocsSourceConfig` accepts and discards source-description options

## Affected sites
- `packages/leadtype/src/fumadocs/index.ts:97-99` — the union; `contentDir` is the only required key of `CreateDocsSourceConfig`, so the other fourteen (`nav`, `mounts`, `openapi`, `openapiCwd`, `typeTableBasePath`, `typeTableStrict`, `frontmatterSchema`, `transformers`, `groups`, `baseUrl`, `remarkPlugins`, `searchIndex`, `toc`, `i18n`/`locale`) are all silently accepted alongside `source`.
- `packages/leadtype/src/fumadocs/index.ts:104-105` — the runtime discriminant takes `config.source` and never looks at the rest.
- `apps/fumadocs-example/lib/source.ts` — the migration this commit performs: `nav`, `mounts` and `openapi` deleted from the adapter call, `typeTableBasePath` moved into `createDocsProject`. Leaving any of them behind would have been a no-op with no error.

## Required outcome
- Passing a source-description option together with `source` should be a compile error, not a silent drop.

## Suggested approach (optional)
Make the union exclusive. Verified against this repo's `tsc --strict`: this errors on `{ source, typeTableBasePath }` and on `{ contentDir, source }`, while `{ source }`, `{ source, includeMetaJson }` and `{ contentDir, … }` all still compile.

```ts
export type FumadocsSourceConfig =
  | (CreateDocsSourceConfig & { includeMetaJson?: boolean; source?: never })
  | ({ source: DocsSource; includeMetaJson?: boolean } & {
      [K in keyof CreateDocsSourceConfig]?: never;
    });
```

* const source = await fumadocsSource({ contentDir: "./content/docs" });
* const loader = loader({ baseUrl: "/docs", source });
*/
/**

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Inserting the type here left the pre-existing fumadocsSource docblock — description plus @example — stranded above FumadocsSourceConfig, so two docblocks now stack on the type (only the nearer one applies) and the exported function has no doc comment at all. Worth folding the new paragraph into the existing block and leaving it attached to the function, with a one-liner on the type instead.

@KayleeWilliams
KayleeWilliams force-pushed the dx/154-docs-project-runtime branch from da7ab52 to 0d30b51 Compare August 4, 2026 10:05
The artifact pipeline understands the whole project — collections, remote sync,
inherited source config, mounts, navigation, route prefixes. The rendering path
did not. `leadtype init` wrote both a config with product identity and
navigation *and* a `lib/source.ts` restating the content root, nav, and base
URL; multi-collection apps restated far more. Two descriptions of one project
drift, and when they do the rendered site and the generated agent artifacts
disagree about what exists.

Dogfooding found exactly that: this repo's Astro example rendered 51 pages
while `generate` emitted 54 and advertised a "Leadtype REST API" nav group.
The three OpenAPI reference pages were in llms.txt and the sitemap but had no
route, because `lib/source.ts` never passed `openapi`. Nothing was wrong with
either half — they were just two descriptions.

`createDocsProject()` reads the resolved config and returns a superset of
`DocsSource`, so first-party adapters accept it with no parallel code path.
Multi-collection projects get one merged, route-aware page API: `listPages()`
tags each page with its collection, and `loadPage()` accepts a
collection-local slug or the full route, so one handler can serve everything
or a route can be mounted per collection. `collections`, `sources`, and
`getSource(key)` expose the resolved graph for custom integrations.

Remote collections are cache-only. A request handler must never clone, so a
cache that is missing, unverifiable, or holds a different revision than the
config asks for fails naming `leadtype sync` — a stale checkout would
otherwise render content that looks fine.

Source-owned inheritance and config loading move to `config/inherit.ts` and are
now called by both build and runtime. Two implementations of "inherit this
repo's navigation" would eventually disagree, which is the failure the shared
content graph exists to prevent.

`configPath` fixes the content root and relative-path resolution using the same
rules the CLI uses — `docs.config.*` inside the docs directory,
`leadtype.config.*` at the root above it. `createDocsSource()` stays fully
supported and is what the project is built on.
The PR claimed first-party adapters accept the project primitive. The fumadocs
adapter did not: `fumadocsSource(config)` built its own source internally, so
the one example using it still restated the content root, navigation, mounts,
and the OpenAPI overlay — the exact drift this branch exists to remove, in the
app meant to demonstrate the fix.

It now accepts either a source description to build or an already-resolved
`DocsSource`, which a project satisfies. The example passes a project and drops
four restated fields.
@KayleeWilliams
KayleeWilliams force-pushed the dx/154-docs-project-runtime branch from 0d30b51 to 0d78b7c Compare August 4, 2026 10:48

@pullfrog pullfrog Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Important

This push is a rebase — no file this PR owns changed. But the new base tip taught the build path to skip content-derived navigation for localized projects, and the runtime path did not learn it. A config with i18n and no curated navigation now builds fine and throws at render time. That is the second consecutive rebase to add a build-side behavior createDocsProject() doesn't mirror.

Reviewed changes — the delta since the prior review (0d30b51) is entirely base-branch movement: the PR was rebased onto the new base tip f394e04 ("Address review on the git source groups branch"), and origin/dx/153-git-source-groups is now exactly that commit, so this branch sits directly on the base tip.

  • Rebased onto f394e04; this PR's own diff is unchanged. diff-tree 0d30b51 0d78b7c is 16 files, of which only cli/generate.ts and docs/paths.lock.json are PR-owned — and both deltas are the base's own new code, not revisions to this PR. The PR-vs-base diff for generate.ts is still exactly the same mechanical deletions plus one import block. project/index.ts, config/inherit.ts, project.test.ts, fumadocs/index.ts, index.ts, cli/init-templates.ts, both example source.ts files, both docs pages and the changeset are byte-identical to the reviewed revision, so none of the twelve open threads is touched.
  • Base added a derivation guard for localized projects. generate.ts:2948 now requires metadata.i18n === undefined before deriving navigation from content, because derivation keys sections off the first path segment — which for docs/en/… is the locale, while navigation resolves per locale over locale-stripped paths. This is the change the inline comment is about.
  • Base changed the inheritance call protocol on the build side. executeGenerate now re-normalizes after inheritCollectionSourceConfigs() instead of swapping the collections into the resolved model, and carries sources/deprecations forward from the first pass. Worth noting because it validates the ordering createDocsProject() already uses (inherit, then normalize) — the runtime is correct on that axis; the open thread at project/index.ts:207-215 is about which collections map it feeds in, which is unaffected.
  • Base also rejected acquisition fields (repository/ref/cacheDir/sparse) on collections nested under a gitSource, fixed named-source id resolution in normalize.ts:243-251, and made createDocsSource's resolveNav test the authored nav rather than the openapi-augmented one. None of those leaves a runtime gap — the last one closes one.

Verified at this head so nobody re-derives it: bun --filter leadtype check-types exits 0 and bun run test in packages/leadtype is 749/749 green. A fresh checkout gives 10 cli.test.ts failures reading No "exports" main defined in node_modules/leadtype/package.json — that is an unbuilt workspace, not a defect; run bun run build first.

Pullfrog  | Fix all ➔Fix 👍s ➔View workflow run | Using Claude Opus𝕏

const shared = {
baseUrl: input.baseUrl,
locale: input.locale,
i18n: input.i18n ?? config.i18n,

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Forwarding config.i18n here is what makes a localized project reach createDocsSource's resolveNav (source/index.ts:602-623), which derives navigation from content whenever the authored nav/groups are empty and has no locale guard. The base commit this PR was just rebased onto added exactly that guard to the build path (generate.ts:2948, metadata.i18n === undefined), so on a config with i18n and no curated navigation, generate now succeeds while createDocsProject(...).getNavigation() throws Nav page "index" under "en" did not match a documentation page. I confirmed this by running both against the same fixture; listPages() and buildSearchIndex() are fine, so routes and content resolve and it is the sidebar render that dies.

Technical details
# A localized project with derived navigation builds fine and throws at runtime

## Affected sites
- `packages/leadtype/src/project/index.ts:237` — forwards `config.i18n` into every collection's `createDocsSource()`, while `nav` is passed only when `collection.navigation` is set.
- `packages/leadtype/src/source/index.ts:602-623``resolveNav()` falls through to `inferNavigationFromContent()` whenever the authored `nav`/`groups` are empty. No `i18n` check.
- `packages/leadtype/src/cli/generate.ts:2944-2952` — the build path's derivation guard, which now includes `metadata.i18n === undefined`.
- `packages/leadtype/src/llm/llm.ts:2071` — where the throw originates.

## Reproduction
Fixture: `docs/en/index.mdx`, `docs/en/guides/auth.mdx`, and the same two files under `docs/es/`. Config:

```ts
{
  product: { name: "Acme", tagline: "Acme does one thing." },
  i18n: { defaultLocale: "en", locales: [{ code: "en" }, { code: "es" }] },
}
```

- `createDocsProject({ config, configDir }).listPages()``["/docs/guides/auth", "/docs"]`, `buildSearchIndex()` → ok, `getNavigation()`**throws** `Nav page "index" under "en" did not match a documentation page.`
- `leadtype generate` on the same config resolves navigation with no error and puts both pages in `ungrouped` at `/docs` and `/docs/guides/auth` (checked in the emitted `public/docs/agent-readability.json`).
- Control: remove `i18n` from the config and `getNavigation()` succeeds with derived groups. So `i18n` is the trigger, not missing content.

## Required outcome
- `createDocsProject()` and `generate` agree on navigation for a config that sets `i18n` and no curated `navigation`, and neither throws.

## Suggested approach (optional)
- Mirror the build-side guard inside `resolveNav()`: skip derivation when `config.i18n` is set. That preserves the invariant the `resolveNav` docblock states — "both sides infer or neither does" — since the build side no longer infers for localized projects.

## Open questions for the human
- Is derived navigation meant to work for localized projects eventually (keying sections off the first *post-locale* segment), or is skipping derivation the permanent answer? If the former, the guard belongs in `inferNavigationFromContent` and both call sites keep deriving.
- `sparse` last rebase, the i18n guard this one: is there an intended mechanism that makes a build-path behavior change surface on the runtime path, or does it stay on reviewer memory?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Make project config the source of truth for runtime docs loading

1 participant