Skip to content
Open
7 changes: 7 additions & 0 deletions .changeset/inline-ref-in-tool-schema.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
---
'@modelcontextprotocol/core-internal': patch
'@modelcontextprotocol/client': patch
'@modelcontextprotocol/server': patch
---

Inline local `$ref` pointers in tool `inputSchema` so schemas are self-contained and LLM-consumable. LLMs cannot resolve JSON Schema `$ref` — they serialize referenced parameters as strings instead of objects. Recursive schemas are handled gracefully — cyclic `$ref` pointers are left in place with only their `$defs` entries preserved, while all non-cyclic refs are fully inlined.
116 changes: 116 additions & 0 deletions packages/core-internal/src/util/schema.ts
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,122 @@ export type AnyObjectSchema = z.core.$ZodObject;
*/
export type SchemaOutput<T extends AnySchema> = z.output<T>;

/**
* Resolves all local `$ref` pointers in a JSON Schema by inlining the
* referenced definitions.
*
* - Caches resolved defs to avoid redundant work with diamond references
* (A→B→D, A→C→D — D is resolved once and reused).
* - Gracefully handles cycles — cyclic `$ref` are left in place with their
* `$defs` entries preserved. Non-cyclic refs in the same schema are still
* fully inlined. This avoids breaking existing servers that have recursive
* schemas which work (degraded) today.
* - Preserves sibling keywords alongside `$ref` per JSON Schema 2020-12
* (e.g. `{ "$ref": "...", "description": "override" }`).
*
* @internal Exported for testing only.
*/
export function dereferenceLocalRefs(schema: Record<string, unknown>): Record<string, unknown> {
// "$defs" is the standard keyword since JSON Schema 2019-09.
// See: https://json-schema.org/draft/2020-12/json-schema-core#section-8.2.4
// "definitions" is the legacy equivalent from drafts 04–07.
// See: https://json-schema.org/draft-07/json-schema-validation#section-9
// If both exist (malformed schema), "$defs" takes precedence.
const defsKey = '$defs' in schema ? '$defs' : 'definitions' in schema ? 'definitions' : undefined;
const defs: Record<string, unknown> = defsKey ? (schema[defsKey] as Record<string, unknown>) : {};
Comment on lines +44 to +45

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 (optional) If any non-'mcp' Standard Schema converter returns $defs (or definitions) as null/a non-object, schema.ts:44-45 selects it as the defs container without a type check and the first local $ref dereferences defs[defName] on null at :88, so tools/list (mcp.ts:189, uncached, every request) throws a TypeError and returns a JSON-RPC error for the whole tool list, where base passed the schema through untouched. Fix: only treat $defs/definitions as a container when it is a non-null, non-array object (fall back to returning schema unchanged otherwise), so malformed converter output degrades to the base behaviour instead of a 500.

Extended reasoning...

Path: standardSchemaToJsonSchema :242 -> dereferenceLocalRefs. :44 defsKey = '$defs' in schema ? '$defs' : ... is a key-presence test, so {$defs: null, properties:{a:{$ref:'#/$defs/A'}}} selects '$defs'; :45 casts null to Record without checking; :50 does not return because defsKey is set; walk reaches {$ref} at :78 -> prefix matches -> :88 defs['A'] -> TypeError: Cannot read properties of null. Consumer: mcp.ts:189 runs this inside the tools/list handler on every request (no memo) so the exception becomes a JSON-RPC internal error for tools/list; registerTool's eager conversion at :831 swallows it per its warn-never-throw contract, so registration appears to succeed and the failure only surfaces at list time. Base: { type:'object', ...result } returned as-is, list succeeds. Population (external trigger, conditional): any StandardJSONSchemaV1 vendor other than 'mcp' whose ~standard.jsonSchema.input() emits a null/empty-literal defs slot — the mcp guard at standardSchema.ts:239 does not protect ArkType/Valibot/third-party converters, and five finders independently…

Verification: nit — triggers only when a non-'mcp' StandardJSONSchemaV1 converter returns $defs (or definitions) as null/undefined while also emitting a #/$defs/X local $ref (a self-contradictory schema no known library — Zod v4, ArkType, Valibot — produces; realistically only a custom or buggy third-party ~standard.jsonSchema implementation). Mechanism verified in… | nit. Trigger: a non-'mcp'…


// No definitions container — nothing to inline.
// Note: $ref: "#" (root self-reference) is intentionally not handled — no schema
// library produces it, no other MCP SDK handles it, and it's always cyclic.
if (!defsKey) return schema;

// Cache resolved defs to avoid redundant traversal on diamond references
// (A→B→D, A→C→D — D is resolved once and reused). Cached values are shared
// by reference, which is safe because schemas are immutable after generation.
const resolvedDefs = new Map<string, unknown>();
// Def names where a cycle was detected — these $ref are left in place
// and their $defs entries must be preserved in the output.
const cyclicDefs = new Set<string>();

/**
* Recursively inlines `$ref` pointers in a JSON Schema node by replacing
* them with the referenced definition content.
*
* @param node - The current schema node being traversed.
* @param stack - Def names currently being inlined (ancestor chain). If a
* def is encountered while already on the stack, it's a cycle — the
* `$ref` is left in place and the def name is added to `cyclicDefs`.
*/
function inlineRefs(node: unknown, stack: Set<string>): unknown {
if (node === null || typeof node !== 'object') return node;
if (Array.isArray(node)) return node.map(item => inlineRefs(item, stack));

const obj = node as Record<string, unknown>;

// JSON Schema 2020-12 allows keywords alongside $ref (e.g. description, default).
// Destructure to get the ref target and any sibling keywords to merge later.
const { $ref: ref, ...siblings } = obj;
if (typeof ref === 'string') {
const hasSiblings = Object.keys(siblings).length > 0;

let resolved: unknown;

// Local definition reference: #/$defs/Name or #/definitions/Name
const prefix = `#/${defsKey}/`;
if (!ref.startsWith(prefix)) return obj; // Non-local $ref (external URL, etc.) — leave as-is

const defName = ref.slice(prefix.length);
const def = defs[defName];
if (def === undefined) return obj; // Unknown def — leave as-is
if (stack.has(defName)) {
cyclicDefs.add(defName);
return obj; // Cycle — leave $ref in place
}

if (resolvedDefs.has(defName)) {
resolved = resolvedDefs.get(defName);
} else {
stack.add(defName);
resolved = inlineRefs(def, stack);
stack.delete(defName);
resolvedDefs.set(defName, resolved);
Comment on lines +95 to +101

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 (optional) Servers whose Zod schemas use z.globalRegistry/.meta({id}) types that reference each other (OpenAPI-derived graphs) now ship a tools/list payload that grows with the product of fan-outs (N^D) instead of one $defs copy per type as on the base, with no size cap and no opt-out — a per-session cost paid by every client and every LLM prompt that embeds the tool list. Fix: bound the expansion (e.g. inline only up to a byte/depth budget or a per-def use count and leave $ref+$defs beyond it), or expose an opt-out so large registered-type graphs keep the linear form. The dismissal's 'pre_existing' basis is wrong for this population: Zod's reused:'inline' never applied to registered types, which always emitted $ref on base. [also at: packages/core-internal/src/util/standardSchema.ts:242 - If a library-converted inputSchema has shared definitions referenced from several places at several nesting levels (z.globalRegistry / .meta({id}) types, common in OpenAPI-derived Zod), unbounded inlining turns the DAG into a tree whose serialized size is the product of the fan-in along every path,…]

Extended reasoning...

Trigger: a tool registered with a Zod inputSchema built from registered types that fan out, e.g. Point = z.object({x,y}).meta({id:'Point'}), Range = z.object({a: Point, b: Point}).meta({id:'Range'}), Box = z.object({r1: Range, r2: Range}).meta({id:'Box'}), Doc = z.object({b1: Box, b2: Box, ...}). On the base commit (git show b654261:packages/core-internal/src/util/standardSchema.ts, the final return { type: 'object', ...result }) Zod's z.toJSONSchema emits $defs: {Point, Range, Box} once each and $ref at every use site: serialized size is linear in the number of types. After this change standardSchemaToJsonSchema (standardSchema.ts:242) calls dereferenceLocalRefs, whose cache at schema.ts:95-101 keeps the in-memory graph shared but every $ref site returns the same…

Verification: nit — conflicts with stated purpose: a size bound would leave some $ref+$defs in place, which is exactly what the PR sets out to eliminate. Trigger: a library-converted (non-mcp vendor) tool inputSchema whose registered/reused types reference each other at multiple levels with multiple uses per level (e.g. Point used 4x in Range, Range 4x in Box, Box 4x in the tool). Mechanism…

}

// Merge sibling keywords onto the resolved definition.
// Note: boolean JSON Schemas (true/false) skip this merge — siblings are dropped.
// This is acceptable: the SDK's JsonSchemaType excludes boolean schemas by design,
// and no schema library (Zod v4, ArkType, Valibot) produces boolean $defs entries.
if (hasSiblings && resolved !== null && typeof resolved === 'object' && !Array.isArray(resolved)) {
const resolvedSiblings = Object.fromEntries(Object.entries(siblings).map(([k, v]) => [k, inlineRefs(v, stack)]));
return { ...(resolved as Record<string, unknown>), ...resolvedSiblings };
Comment on lines +108 to +110

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 (optional) The sibling merge at schema.ts:110 spreads the $ref node's siblings OVER the resolved definition, so a node like { $ref:'#/$defs/Base', properties:{extra:...}, required:['extra'] } loses Base's own properties/required entirely — the advertised inputSchema silently drops constraints that base (which left $ref for the validator to conjoin) enforced, and the LLM no longer sees Base's fields. Fix: merge structural keywords conjunctively (emit allOf:[resolved, siblings] when a sibling key collides with a key of the resolved def, or deep-merge properties/required) and only shallow-override annotation keys (description/title/default/examples); the same rule applies to the root-level $ref case where type and other root keys are merged.

Extended reasoning...

JSON Schema 2020-12 defines $ref as an in-place applicator that is conjoined with its siblings (core §8.2.3.1); Ajv and cfworker both validate $ref AND siblings. On base the schema went out untouched, so a validator saw both Base's required:['id'] and the sibling required:['extra']. After merge: :77 splits {$ref, ...siblings}; :99 resolves Base to {type:'object', properties:{id:...}, required:['id']}; :110 returns {...resolved, ...resolvedSiblings} -> properties and required are replaced by the sibling values -> id is gone from both properties and required. Consequence: the client's Ajv-based argument validation and the server's memoized inputSchema (mcp.ts:109) now accept {extra:1} without id, and the model is never told about id, so tools/call reaches the handler and fails in Zod parsing with a confusing error, or worse passes an unvalidated shape. The finder dismissed this because Zod's own flattenRef uses override semantics and 'no supported library emits structural applicator siblings' — but this path runs for ANY Standard Schema vendor except 'mcp'…

Verification: nit — triggers when a non-'mcp' vendor schema (a custom Standard JSON Schema adapter, or a JSON-Schema-native library) yields a $ref node carrying structural siblings such as properties/required; known converters (Zod v4 derives $ref siblings only from metadata like description/default/readOnly because a schema with _zod.parent never generates its own body) do not produce this shape,…

}
return resolved;
}

// Regular object — recurse into values, skipping root-level $defs container
const result: Record<string, unknown> = {};
for (const [key, value] of Object.entries(obj)) {
if (obj === schema && (key === '$defs' || key === 'definitions')) continue;
Comment on lines +89 to +118

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 (optional) A local $ref that the walker cannot resolve (a JSON-pointer into a def such as #/$defs/Foo/properties/bar, or a #/definitions/X ref when $defs is also present and wins) is left in place, yet the root $defs/definitions container is still stripped at line 118, so tools/list now advertises a dangling $ref where the base emitted a resolvable schema and clients that compile inputSchema (Ajv MissingRefError) fail. Fix: only strip a container (or an entry) once every ref into it has been inlined — e.g. track unresolved local refs the way cyclicDefs is tracked and keep both containers' referenced entries, or skip stripping entirely when any #/$defs//#/definitions/ ref survives. [also at: packages/core-internal/src/util/schema.ts:121 - When a library-converted schema carries both $defs and definitions, only #/$defs/ refs are resolved but the root loop strips BOTH containers, so #/definitions/X refs (left as-is by the prefix check) are advertised dangling and the server's own Ajv validation of that tool's arguments throws…; packages/core-internal/src/util/schema.ts:89 - A local $ref the resolver declines to inline (JSON-pointer-escaped or percent-encoded def name such as…; +1 more]

Extended reasoning...

Condition: a non-mcp StandardJSONSchemaV1 converter whose output contains a local ref the helper cannot look up by bare def name — I could not enumerate every library's emitted pointer forms, so this is conditional. Path: defsKey (line 44) picks one container; defs[defName] at line 89 is a flat lookup, so #/$defs/Foo/properties/bar or #/definitions/X (when $defs exists) returns obj untouched. Later the regular-object loop at line 118 unconditionally drops both $defs and definitions from the root, and the re-attach block only restores names in cyclicDefs. Result on the wire: { ..., properties: { x: { $ref: '#/$defs/Foo/properties/bar' } } } with no…

Verification: nit — triggers only when a non-'mcp' StandardJSONSchemaV1 converter emits a local $ref that is not a bare top-level def name (a deep JSON pointer such as #/$defs/Foo/properties/bar, a JSON-pointer-escaped/percent-encoded name, or #/definitions/X while a root $defs also exists); the mechanism is real in the code but I could not name any library the repo targets that produces such refs.…

result[key] = inlineRefs(value, stack);
}
return result;
Comment thread
claude[bot] marked this conversation as resolved.
}

const resolved = inlineRefs(schema, new Set()) as Record<string, unknown>;

// Re-attach $defs only for cyclic definitions, using their resolved/cached
// versions so that any non-cyclic refs inside them are already inlined.
if (defsKey && cyclicDefs.size > 0) {
const prunedDefs: Record<string, unknown> = {};
for (const name of cyclicDefs) {
prunedDefs[name] = resolvedDefs.get(name) ?? defs[name];
}
resolved[defsKey] = prunedDefs;
}

return resolved;
}

/**
* Parses data against a Zod schema (synchronous).
* Returns a discriminated union with success/error.
Expand Down
10 changes: 9 additions & 1 deletion packages/core-internal/src/util/standardSchema.ts
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@
import * as z from 'zod/v4';

import type { StringSchema } from '../types/types';
import { dereferenceLocalRefs } from './schema';

// Standard Schema interfaces — vendored from https://standardschema.dev (spec v1, Jan 2025)

Expand Down Expand Up @@ -231,7 +232,14 @@ export function standardSchemaToJsonSchema(schema: StandardJSONSchemaV1, io: 'in
`Wrap your schema in z.object({...}) or equivalent.`
);
}
return { type: 'object', ...result };
// Hand-authored JSON Schema (wrapped via fromJsonSchema, vendor 'mcp') is advertised
// verbatim — SEP-1613 requires $schema/$defs/$ref to survive tools/list unchanged.
// Library-converted schemas (Zod, ArkType, Valibot) get local $ref inlined: their
// $ref is a conversion artifact (z.globalRegistry, z.lazy) that LLMs cannot resolve.
if (std.vendor === 'mcp') {
return { type: 'object', ...result };
}
return dereferenceLocalRefs({ type: 'object', ...result });
}

/**
Expand Down
45 changes: 25 additions & 20 deletions packages/core-internal/test/shared/inputRequired.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -270,27 +270,32 @@ describe('inputRequired() builder', () => {
expect(act).toThrow(TypeError);
expect(act).toThrow(/unsupported JSON Schema constraint\(s\).*additionalProperties/);

expect(() =>
inputRequired.elicit({
message: 'Name?',
requestedSchema: {
'~standard': {
version: 1,
vendor: 'test',
validate: (value: unknown) => ({ value }),
jsonSchema: {
input: () => ({
$defs: { Name: { type: 'string' } },
type: 'object',
properties: { name: { $ref: '#/$defs/Name' } },
required: ['name']
}),
output: () => ({})
}
// $defs/$ref no longer reach elicitation validation: standardSchemaToJsonSchema
// inlines local $ref and strips $defs, so the schema converts cleanly.
const request = inputRequired.elicit({
message: 'Name?',
requestedSchema: {
'~standard': {
version: 1,
vendor: 'test',
validate: (value: unknown) => ({ value }),
jsonSchema: {
input: () => ({
$defs: { Name: { type: 'string' } },
type: 'object',
properties: { name: { $ref: '#/$defs/Name' } },
required: ['name']
}),
output: () => ({})
}
} as never
})
).toThrow(/\$defs/);
}
} as never
});
expect((request.params as { requestedSchema: unknown }).requestedSchema).toEqual({
type: 'object',
properties: { name: { type: 'string' } },
required: ['name']
});
});

test('elicit drops annotation-only root keywords', () => {
Expand Down
Loading
Loading