docs: refactor with nuxt-agent-discovery module - #6883
Open
benjamincanac wants to merge 11 commits into
Open
Conversation
…covery `nuxt/ui` docs, `nuxt.com`, Docus and `comark-docs` each hand-rolled markdown content negotiation, CDN rewrites and the discovery documents that advertise them, and the four copies had drifted. Move that surface to `nuxt-agent-discovery` and keep only what is genuinely site-specific. Deleted, now owned by the module: the Vercel rewrite module, the markdown middleware, the markdown error handler, the negotiation utility, the raw markdown route, the api-catalog, `sitemap.md`, `sitemap.xml`, the MCP server card handler, the static `robots.txt` and the hand-written skills index. `@nuxtjs/sitemap` and `@nuxtjs/robots` take over their own files, with the module feeding both from the shared user-agent list. Kept, wired through the module's hooks: `transformMDC` runs on `agent-discovery:document`, the MCP server card is enriched with the live tool list on `agent-discovery:mcp-server-card`, and the hand-written homepage markdown renders the discovery registry with `renderAgentResources()`. Behaviour changes worth knowing: - `Accept: text/markdown;q=0, text/html` now returns HTML. The old substring matcher ignored q-values. - Raw pages gain the related links from their `links` frontmatter, which is what `@nuxt/content`'s own raw route does and this site had diverged from. - `sitemap.xml` covers 192 URLs instead of 183: the homepage and every non-docs section were missing, because the old route only queried the `docs` collection. - `robots.txt` allows 18 agents instead of 9, generated from the same list negotiation matches so the two cannot drift. `CCBot` is no longer named explicitly; the wildcard group still allows it. - The skills index is generated from the files on disk rather than hand-maintained. `llms.txt`, `openapi.json`, `/raw/index.md` and all 184 prerendered raw pages were diffed against the pre-migration build; the only differences are the ones listed above.
benjamincanac
changed the title
commit: |
`nuxt-agent-discovery` now removes `@nuxt/content`'s llms plugin and renders `llms-full.txt` through the content adapter, so `content:llms:generate:document` no longer fires. `transformMDC` already runs on `agent-discovery:document`, which is the same path `llms-full.txt` takes now, so the transform is unaffected.
benjamincanac
changed the title
nuxt-agent-discovery module
The negotiated pages, their raw twins, the sitemaps, the llms indexes and the `.well-known` documents are the same on every site running the module, derived from `routes`, `rawPrefix` and the link registry. Restating them here meant a second copy of the route config to keep in step. `agentDiscoveryOpenApi()` contributes them, along with the `Vary` header, the markdown 404 and the `Linkset` / `SkillsIndex` schemas. What stays is what only this site knows: `/api/**`, the MCP endpoint, and the prose. Site values merge last, so any generated path can still be overridden. The 11 site-specific paths and all 11 schemas come out unchanged; the 11 module-owned ones now carry generic descriptions and, for the moment, no `operationId`.
The hook carries the title and description now, so the hand-written route has nothing left that the module cannot build.
The card now reads `listMcpDefinitions()` itself, and `getAgentDocument()` returns the bytes `/raw/**.md` serves without the tools fetching their own route back out of the function.
`listAgentPages()` resolves both URLs from the same route config the negotiation uses, and the remaining MCP definitions take the site URL from `SITE_URL` rather than writing it out.
The static `public/robots.txt` carried eight disallowed prefixes that nothing put back: `nuxt-agent-discovery` only contributes `Allow: /` groups for the agent user agents.
|
Review the following changes in direct dependencies. Learn more about Socket for GitHub.
|
benjamincanac
marked this pull request as ready for review
August 27, 2026 12:42
Contributor
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Plus Run ID: ⛔ Files ignored due to path filters (1)
📒 Files selected for processing (32)
💤 Files with no reviewable changes (14)
Included review availability: Your plan provides up to 8 included reviews per hour; 7 remain after this review. 📝 WalkthroughWalkthroughThe documentation site now uses Estimated code review effort: 4 (Complex) | ~60 minutes Merge Risk: 🔵 Low · up to The migration centralizes documentation discovery behavior but makes the public OpenAPI contract dependent on a new external module that receives request context without local validation. Fixed canonical configuration and prerendering limit the impact, so the PR is mergeable with explicit owner awareness of this integration boundary. 🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
Full details: Docstring CoverageExplanation Docstring coverage is 33.33% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 3 functions across 16 files. (2 skipped: 2 unsupported.)
✨ Finishing Touches 💡 1📝 Generate docstrings 💡
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
🔗 Linked issue
❓ Type of change
📚 Description
nuxt/uidocs,nuxt.com, Docus andcomark-docseach hand-rolled markdown content negotiation, CDN rewrites and the discovery documents that advertise them. The four copies drifted, and the drift causes live bugs (docus.dev serves HTML to ClaudeBot today). This moves that surface intonuxt-agent-discoveryand keeps only what is genuinely site-specific.Deleted, now owned by the module
docs/server/utils/markdownNegotiation.tsdocs/server/routes/raw/[...slug].md.get.tsdocs/server/routes/raw/index.md.get.tsdocs/server/routes/.well-known/api-catalog.get.tsdocs/server/middleware/markdown.tsdocs/server/error.tsdocs/server/routes/sitemap.md.get.tsdocs/server/routes/.well-known/mcp/server-card.json.get.tsdocs/modules/md-rewrite.tsdocs/server/routes/sitemap.xml.get.tsdocs/app/composables/useCanonical.tsdocs/server/utils/extractSections.tsdocs/server/utils/openapi.ts(discovery half)docs/public/robots.txt(agent list; itsDisallowprefixes move torobots.disallow),skills/index.json@nuxtjs/sitemapand@nuxtjs/robotstake over their own files, with the module feeding both from the same user-agent list negotiation matches. The module also removes@nuxt/content's own llms feature and generatesllms.txt/llms-full.txtfrom the content adapter, so a page reads identically whether an agent fetches/raw/**.mdor the single full document.Kept, wired through the module's hooks and helpers
transformMDCruns onagent-discovery:document, which is now the path both/raw/**.mdandllms-full.txttake./raw/index.mdis generated by the module. The homepage is a Vue page with no document behind it, soagent-discovery:indexsupplies its title and description from theindexcollection plus the agent-facing prose; the frontmatter, canonical links, resources block and trailer come from the module and now match what a content page returns.@nuxtjs/mcp-toolkit's own definitions. The site plugin that copied them onto the card is gone.get-documentation-pageandget-componentcallgetAgentDocument()instead of$fetching/raw/**.mdback out of their own serverless function, andsearch-documentationcallslistAgentPages()instead of querying the collection directly. Both URLs in its results come from the module's route config now.openapi.jsonkeeps its/api/**paths, the MCP endpoint and its prose; the discovery half comes fromagentDiscoveryOpenApi(). The 11 site-specific paths and all 11 schemas are unchanged.extractSections.tsis deleted in favour of the module's, which falls back to the whole document when no requested heading matches instead of returning the title alone.server/mcp/is otherwise untouched.Behaviour changes
Verified against the preview deployment.
linksfrontmatter — what@nuxt/content's own raw route does and this site had diverged from. In-body relative links and the## Sitemaptrailer are now absolute, since these files are read detached from the site. Of 184 pages, every diff is one of those three changes.llms-full.txtrenders through the same adapter, so it matches the raw twins page for page: 166 → 164 documents (the two installation pages were previously rendered twice, matching both/docs/getting-started/installation%and/docs/getting-started%).llms.txtgains an Overview section linking/raw/index.md, 169 → 170 links. Same link set otherwise, reordered.sitemap.mdlinks point at the/raw/**twins rather than.mdpage URLs, so it agrees withllms.txton every page. 192 entries.sitemap.xmlcovers 192 URLs instead of 183. The homepage and every non-docs section (/blog,/templates,/showcase,/figma,/team,/community,/releases) were missing, because the old route only queried thedocscollection. Every page collection declaresdefineSitemapSchema(), which is what@nuxtjs/sitemapwalks.robots.txtallows 18 agents instead of 9, generated from the shared list so the two can't drift.CCBotis no longer named explicitly; the wildcardAllow: /still permits it. The eightDisallowprefixes the static file carried move torobots.disallowinnuxt.config.ts, since the module only contributesAllow: /groups./raw/figma.md,/raw/showcase.md,/raw/templates.mdnow 404. Those pages are YAML data with no markdown body, so they have no markdown representation.search-documentationreturns 183 pages instead of 194, and requires every search term to match across the title, path and description where it previously matched one substring against the title or description. The 11 dropped entries are@nuxt/content's.navigationfiles, which the old collection query listed as if they were documentation: every one rendered a 404 body and had no markdown twin. Results gain amarkdown_url; the existing fields are unchanged and identical page for page./.well-known/api-cataloggroupsllms.txtunder the/anchor rather than/docs, and is now cached for an hour rather than carryings-maxage.Linkheader on/carries the same links plus the skills index, in a different order.Known limitation:
Acceptq-values at the edgeThe module parses q-values per RFC 9110, so the Nitro middleware serves HTML to a client that ranks markdown below it. The CDN rewrite is a regex over the raw header, and Vercel runs RE2, so it can only express the outright refusal: a
missingmatcher keepstext/markdown;q=0from being rewritten, which is confirmed working on the preview. A partial preference liketext/markdown;q=0.1, text/html;q=0.9still gets markdown on a prerendered page.Verification
Diffed against a full pre-migration build:
llms.txt,openapi.json,/raw/index.md,sitemap.mdand all 184 prerendered raw pages, with every difference listed above and nothing else moved.On the preview deployment: negotiation by
Accept, by user agent and by.mdURL, including all fourq=0spellings falling through to HTML;Varyand the discoveryLinkheader; the markdown 404 for agents against the HTML one for browsers; and every discovery document (llms.txt170 links,llms-full.txt164 documents,sitemap.xml192 URLs,sitemap.md192 entries,robots.txt19 groups, 15 skill files).The MCP server card lists all 12 tools, 5 resources and 3 prompts with its capabilities, and
get-documentation-pagereturns bytes identical to/raw/docs/components/badge.md, narrows correctly to named##sections, falls back to the whole document for a heading that does not exist, and resolves a section path to its first document.Re-run end to end against the published
nuxt-agent-discovery@0.1.0: every count is unchanged andllms-full.txtis byte-identical at 2244037 bytes, so the module's pre-publish correctness pass moved nothing here.pnpm lint,pnpm typecheckand aNITRO_PRESET=vercelbuild pass.📝 Checklist