Repository navigation
Clarify ModelConfig rejection behavior - #604
Conversation
Revise provider and harness docs to describe unsupported configurations as rejected or not ready, rather than implying revisions fail to compile. Add reusable snippets for OpenAI-compatible and self-hosted vLLM gateway setup. Signed-off-by: Rachael Graham <rachael.graham@solo.io>
Docs preview
Both are uploaded Worker versions and serve no production traffic. |
Signed-off-by: Rachael Graham <rachael.graham@solo.io>
Signed-off-by: Rachael Graham <rachael.graham@solo.io>
kristin-kronstain-brown
left a comment
There was a problem hiding this comment.
Thanks for taking on the rejection wording across the provider pages. Updating every internal anchor link in one pass is exactly the kind of sweep that usually gets half done.
Summary
- Blocking: the heading rename breaks the glossary link at
docs-site/data/glossary.yaml:89. The REVIEW comment onapiKeyPassthroughis still open. Several touched lines say the AgentTemplate reportsCompatible, but the condition is on the Agent. - Premise: I checked the kagent controller, and a rejected config really is a compile failure (
CompileAgentreturns an error and the reconciler records it asCompilationFailure). Saying "kagent rejects" for readers is fine, but the rest of the docs still explain revisions as compiled, and the rewrite drops the fact that the Agent keeps its last successful revision. Details inline. - Should fix: pin old anchors with explicit
{#ids}, reconsider single-use snippets, and say which resource actually reports the failure (Agent, not Harness).
Outside the diff (no inline comment possible)
byo-openai.md:98also says "The AgentTemplate reports theCompatiblecondition".byo-openai.md:100still says "so it does compile", which is the wording this PR set out to remove elsewhere. Either is fine with me, but the page should be consistent.
What I checked: controller status and reconcile logic, plus apiKeyPassthrough CRD validation, against kagent-dev/kagent main, and incoming links to the renamed anchors in this repo. I did not run anything on a cluster. CI here runs static/content checks, linkChecker, deploy, and DCO, and linkChecker did not catch the glossary anchor.
| The kagent installation creates a `default-model-config` ModelConfig from the provider API key that you supply at install time, so a first agent does not need extra setup. To use a different provider, a different model, or a different set of credentials, create additional ModelConfigs. | ||
|
|
||
| ## How a ModelConfig reaches an agent | ||
| ## Configure a ModelConfig |
There was a problem hiding this comment.
Blocking. Renaming this heading breaks an incoming link. docs-site/data/glossary.yaml:89 points the glossary entry at about-model-providers/#how-a-modelconfig-reaches-an-agent, and that anchor no longer exists after this change. I think linkChecker stayed green because it does not resolve anchors in glossary data.
Pinning the old id keeps the link working and lets the title change freely:
| ## Configure a ModelConfig | |
| ## Configure a ModelConfig {#how-a-modelconfig-reaches-an-agent} |
Alternatively, update the glossary link. I would still pin the id so external deep links keep working.
There was a problem hiding this comment.
Confirmed and fixed — glossary.yaml:89 does point at the old anchor. Pinned the id as suggested, so the title changes and the link keeps working.
|
|
||
| ### Forward the caller's token | ||
|
|
||
| Set `apiKeyPassthrough: true` to send the bearer token from the incoming agent-to-agent (A2A) request to the provider as the API key, rather than reading a key from a Secret. An organization that already issues tokens that its model provider accepts can then skip managing a second credential for kagent. A ModelConfig cannot set both `apiKeyPassthrough` and `apiKeySecret`. |
There was a problem hiding this comment.
Should fix. Two things in this new paragraph.
- It leaves out the limits that a reader hits first. The CRD rejects
apiKeyPassthrough: truewhen the provider isGemini,GeminiVertexAI, orAnthropicVertexAI(go/api/v1alpha3/modelconfig_types.go:575in kagent-dev/kagent), and thecodexandclauderuntimes reject it too (line 102 of this page). Listing those here saves a round trip to the API reference. - The second sentence is a use-case pitch that the page cannot back up, and the REVIEW comment below says the author is unsure of it. I would drop it until that is confirmed.
Possible wording:
| Set `apiKeyPassthrough: true` to send the bearer token from the incoming agent-to-agent (A2A) request to the provider as the API key, rather than reading a key from a Secret. An organization that already issues tokens that its model provider accepts can then skip managing a second credential for kagent. A ModelConfig cannot set both `apiKeyPassthrough` and `apiKeySecret`. | |
| Set `apiKeyPassthrough: true` to send the bearer token from the incoming agent-to-agent (A2A) request to the provider as the API key, rather than reading a key from a Secret. A ModelConfig cannot set both `apiKeyPassthrough` and `apiKeySecret`. The setting is not available for the `Gemini`, `GeminiVertexAI`, or `AnthropicVertexAI` providers, or on a `codex` or `claude` Harness. |
There was a problem hiding this comment.
Taken verbatim. Verified the rule at modelconfig_types.go:575 — it rejects apiKeyPassthrough for exactly those three providers. You are right that the second sentence was a pitch I could not back up, so it is gone.
|
|
||
| Set `apiKeyPassthrough: true` to send the bearer token from the incoming agent-to-agent (A2A) request to the provider as the API key, rather than reading a key from a Secret. An organization that already issues tokens that its model provider accepts can then skip managing a second credential for kagent. A ModelConfig cannot set both `apiKeyPassthrough` and `apiKeySecret`. | ||
|
|
||
| <!-- REVIEW: this paragraph follows the API, which says the token is forwarded "directly to the LLM |
There was a problem hiding this comment.
Blocking. This REVIEW comment needs an answer before merge, because it questions whether the paragraph above is true.
What I could confirm from kagent-dev/kagent main: the ADK runtime reads the caller's bearer token with models.PassthroughToken (go/adk/pkg/models/base.go:99) and uses it as the provider key. What I could not confirm is how that interacts with the Substrate egress gateway, which this page says injects a static header from a Secret. If the gateway overwrites or strips the auth header, passthrough may not work on 1.x at all. That needs someone who has run it, or an engineering answer.
There was a problem hiding this comment.
The paragraph it questioned is gone, so the page no longer asserts the use case. I rewrote the REVIEW to park only the part neither of us can answer from code — whether the egress gateway overwrites or strips the auth header, which decides if passthrough works on 1.x at all. Happy to drop the comment entirely if you would rather track it as an issue.
| Substrate matches a destination on the exact DNS hostname, without path, port, or scheme. Two credentials that target the same hostname and header are rejected, including a conflict between an agent's model, a memory embedding model, and an MCP server. Give such origins distinct DNS names. A destination given as an IP address cannot carry an injected credential at all. | ||
|
|
||
| ### Credentials that do not compile | ||
| ### Credentials that kagent rejects |
There was a problem hiding this comment.
Should fix. The PR updates every internal link to the new anchor, which is good, but any external deep link (blog posts, issues, Slack threads) to #credentials-that-do-not-compile breaks. Pinning the old id avoids that:
| ### Credentials that kagent rejects | |
| ### Credentials that kagent rejects {#credentials-that-do-not-compile} |
If you would rather the id match the new title, that works too, but then nothing protects the old links.
There was a problem hiding this comment.
Pinned. Worth flagging what this exposed: pinning orphaned the seven internal links I had already repointed at the new slug, and my anchor check missed it because its regex only matched relative hrefs while these render absolute. Fixed the check and repointed all seven back — it now covers 393 internal anchor links and finds 2 broken, both pre-existing in glossary and suspend-and-resume.
| ### Credentials that kagent rejects | ||
|
|
||
| Header injection accepts one shape of credential: a static string. A credential that requires a local signature, a token exchange, or a file mounted into the agent cannot be injected, so kagent rejects the configuration instead of passing the credential to the runtime. The Agent reports the `Compatible` condition as `False` with the reason `UnsupportedConfiguration`, and kagent does not compile a revision from it. Any Session that already exists keeps running the last revision that compiled. | ||
| Header injection accepts one shape of credential, a static string. A credential that requires a local signature, a token exchange, or a file mounted into the agent cannot be injected, so kagent rejects the configuration instead of passing the credential to the runtime. The Agent reports the `Compatible` condition as `False` with the reason `UnsupportedConfiguration`, and the Agent does not become ready. A Session that is already running keeps the revision that it started on. |
There was a problem hiding this comment.
Should fix. I checked this against the controller, and I think the PR premise is partly off. In kagent-dev/kagent main, a rejected configuration literally is a compile failure: CompileAgent returns an error, and the reconciler records it as CompilationFailure (go/core/internal/controller/reconciler.go:110-123). So the old "does not compile a revision" was accurate, and the rest of this page still explains revisions as compiled (lines 52, 56, 58, 122).
The new wording is not wrong. Ready does go False (reason Blocked, from status.go). But it drops a useful fact: the Agent keeps its latestSuccessfulRevision, so a bad edit does not take down a previously working Agent. It also says "the Agent" twice. Possible wording:
| Header injection accepts one shape of credential, a static string. A credential that requires a local signature, a token exchange, or a file mounted into the agent cannot be injected, so kagent rejects the configuration instead of passing the credential to the runtime. The Agent reports the `Compatible` condition as `False` with the reason `UnsupportedConfiguration`, and the Agent does not become ready. A Session that is already running keeps the revision that it started on. | |
| Header injection accepts one shape of credential, a static string. A credential that requires a local signature, a token exchange, or a file mounted into the agent cannot be injected, so kagent rejects the configuration instead of passing the credential to the runtime. The Agent reports the `Compatible` condition as `False` with the reason `UnsupportedConfiguration`, and `Ready` as `False`. kagent does not build a new revision, and the Agent keeps its last successful revision, so a Session that is already running is unaffected. |
Please double-check the last sentence against a running cluster. I read it from status.go but did not run it.
There was a problem hiding this comment.
You are right that "does not compile a revision" was accurate, and I took your wording. I verified the last sentence on a 1.37 cluster: breaking a working Agent left latestSuccessfulRevision unchanged while desiredRevision emptied. One caveat — I triggered a reference failure rather than an UnsupportedConfiguration one, so it is the same code path but not literally the case described.
|
|
||
| > [!NOTE] | ||
| > Bedrock is the only provider that every runtime supports. On each runtime, the Bedrock credential must be a bearer token. For a `codex` Harness, you must use OpenAI `gpt-*` model IDs. Neither a `codex` nor a `claude` Harness accepts a `bedrock` setting beyond `region`. A Harness that asks for anything else fails to compile. For more information, see [Agent harness]({{< link path="agents/agent-harness#model-provider-support" >}}). | ||
| > Bedrock is the only provider that every runtime supports. On each runtime, the Bedrock credential must be a bearer token. For a `codex` Harness, you must use OpenAI `gpt-*` model IDs. Neither a `codex` nor a `claude` Harness accepts a `bedrock` setting beyond `region`. kagent rejects a Harness that asks for anything else. For more information, see [Agent harness]({{< link path="agents/agent-harness#model-provider-support" >}}). |
There was a problem hiding this comment.
Should fix. "kagent rejects a Harness" sends the reader to the wrong resource. The Harness itself stays fine (its own Ready covers only its dependencies, per agent-harness.md:340). What fails is the Agent that pairs this Harness with the ModelConfig. The old "fails to compile" had the same vagueness, so this is not a regression, but the rewrite is a chance to fix it.
| > Bedrock is the only provider that every runtime supports. On each runtime, the Bedrock credential must be a bearer token. For a `codex` Harness, you must use OpenAI `gpt-*` model IDs. Neither a `codex` nor a `claude` Harness accepts a `bedrock` setting beyond `region`. kagent rejects a Harness that asks for anything else. For more information, see [Agent harness]({{< link path="agents/agent-harness#model-provider-support" >}}). | |
| > Bedrock is the only provider that every runtime supports. On each runtime, the Bedrock credential must be a bearer token. For a `codex` Harness, you must use OpenAI `gpt-*` model IDs. Neither a `codex` nor a `claude` Harness accepts a `bedrock` setting beyond `region`. An Agent that pairs either Harness with any other `bedrock` setting reports the `Compatible` condition as `False`. For more information, see [Agent harness]({{< link path="agents/agent-harness#model-provider-support" >}}). |
There was a problem hiding this comment.
Applied. Also fixed the AgentTemplate reference further down this page at line 40, which had the same problem.
| | ------- | ---------------------------------------------------------- | | ||
| | `claude` | The Harness fails to compile, and the controller reports `Harness env "<name>" conflicts with Claude-owned runtime configuration`. | | ||
| | `codex` | The Harness fails to compile, and the controller reports `Harness env "<name>" conflicts with Codex's compiled configuration`. | | ||
| | `claude` | kagent rejects the Harness, and the controller reports `Harness env "<name>" conflicts with Claude-owned runtime configuration`. | |
There was a problem hiding this comment.
Should fix. Same as amazon-bedrock.md:13: I think the env conflict surfaces on each Agent that uses this Harness, not on the Harness, because the check runs inside CompileAgent. "kagent rejects the Harness" suggests kubectl get harness shows the error. I am fairly but not fully sure of this, so please confirm on a cluster. If I am right, something like "Every Agent on the Harness reports Compatible as False with the message ..." for both rows.
There was a problem hiding this comment.
You are right. Both rows now read "Every Agent on the Harness reports Compatible as False". Confirmed on a 1.37 cluster that the conditions land on the Agent and that a Harness carries no such status, though I exercised it through a reference failure rather than the env conflict specifically.
| kagent rejects `AnthropicVertexAI`, `GeminiVertexAI`, and `SAPAICore` on every runtime, because each authenticates with a credential that the egress gateway cannot place in an HTTP header. No agent can use these providers. For the alternatives, see [About model providers]({{< link path="setup/model-providers/about-model-providers#credentials-that-kagent-rejects" >}}). | ||
|
|
||
| kagent rejects `Mistral` for a different reason. The controller does not resolve the provider at all, so a Mistral ModelConfig reports `unsupported model provider: Mistral` and does not compile a revision. | ||
| kagent rejects `Mistral` for a different reason. The controller does not resolve the provider at all, so the controller rejects a Mistral ModelConfig and reports `unsupported model provider: Mistral`. |
There was a problem hiding this comment.
Nit. "kagent rejects ... the controller rejects" says the same thing twice in two sentences.
| kagent rejects `Mistral` for a different reason. The controller does not resolve the provider at all, so the controller rejects a Mistral ModelConfig and reports `unsupported model provider: Mistral`. | |
| kagent rejects `Mistral` for a different reason. The controller does not resolve the provider at all, so a Mistral ModelConfig reports `unsupported model provider: Mistral`. |
| @@ -0,0 +1,3 @@ | |||
| To serve your own model, run a [vLLM](https://docs.vllm.ai/) server behind an OpenAI-compatible gateway, such as [Bifrost](https://github.com/maximhq/bifrost) or [LiteLLM](https://docs.litellm.ai/). Requests then travel from kagent to the gateway to vLLM. | |||
|
|
|||
| Configure the gateway as an OpenAI-compatible provider in the same way, and get two further things right. | |||
There was a problem hiding this comment.
Should fix. Each new snippet has exactly one consumer (byo-openai.md), so the reuse adds indirection without a second page to share it. This snippet also cannot stand alone: "in the same way" refers to the section above it on byo-openai.md, and "two further things" refers to the two ### headings that follow it there. Reused anywhere else, both phrases dangle.
Unless a second consumer is planned in a follow-up, I would inline both snippets back into byo-openai.md. If one is planned, please name it in the PR description and make the text self-contained, for example "Configure the gateway as an OpenAI-compatible provider, as described in Bring your own OpenAI-compatible provider".
There was a problem hiding this comment.
There is a second consumer, but it is not visible from this repo, so the comment is fair.
solo-io/docs renders this page through a rebase stub rather than copying it, and its assemble-assets.py mounts the enterprise assets/kagent-docs/ last so a file at the same path wins. An enterprise snippets/setup/byo-openai-intro.md therefore shadows the OSS one. Sixteen snippet paths already exist in both repos and fifteen hold different content downstream, including all six what-is-kagent-* blocks that were decomposed here for the same reason.
The enterprise review asked this page to name agentgateway and drop Bifrost and LiteLLM. Without the seam the only way to do that is to fork the whole page downstream, which then drifts from upstream silently.
Your other point stands and I have acted on it: the text has to stand alone, so "in the same way" and "two further things" are gone.
| @@ -0,0 +1,3 @@ | |||
| Many providers serve the [OpenAI API](https://platform.openai.com/docs/api-reference/introduction) rather than an API of their own. To use one, set `provider: OpenAI` and point `openAI.baseUrl` at the provider's endpoint. | |||
|
|
|||
| This is the same mechanism behind the [xAI]({{< link path="setup/model-providers/xai" >}}) and [Amazon Bedrock]({{< link path="setup/model-providers/amazon-bedrock" >}}) OpenAI-compatible paths. | |||
There was a problem hiding this comment.
Nit. "This is the same mechanism" leaves "This" without a referent, and in a snippet the preceding sentence may not be what a reader just read.
| This is the same mechanism behind the [xAI]({{< link path="setup/model-providers/xai" >}}) and [Amazon Bedrock]({{< link path="setup/model-providers/amazon-bedrock" >}}) OpenAI-compatible paths. | |
| The [xAI]({{< link path="setup/model-providers/xai" >}}) and [Amazon Bedrock]({{< link path="setup/model-providers/amazon-bedrock" >}}) OpenAI-compatible paths use the same `baseUrl` mechanism. |
There was a problem hiding this comment.
Already reworded, so the dangling "This" is gone. Your version is the better one and I will take it if you would rather not have the passive.
Revise provider and harness docs to describe unsupported configurations as rejected or not ready, rather than implying revisions fail to compile. Add reusable snippets for OpenAI-compatible and self-hosted vLLM gateway setup.