Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
To serve your own model, run a [vLLM](https://docs.vllm.ai/) server behind an OpenAI-compatible gateway, such as [Bifrost](https://github.com/maximhq/bifrost) or [LiteLLM](https://docs.litellm.ai/). Requests then travel from kagent through the gateway to vLLM, and you configure the gateway as the provider endpoint.

Launch vLLM with automatic tool choice, and set `model` to the identifier that the gateway routes on.
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
Many providers serve the [OpenAI API](https://platform.openai.com/docs/api-reference/introduction) rather than an API of their own. To use one, set `provider: OpenAI` and point `openAI.baseUrl` at the provider's endpoint. This mechanism is used by the [xAI]({{< link path="setup/model-providers/xai" >}}) and [Amazon Bedrock]({{< link path="setup/model-providers/amazon-bedrock" >}}) OpenAI-compatible paths.
8 changes: 4 additions & 4 deletions docs-site/content/kagent/1.x/agents/agent-harness.md
Original file line number Diff line number Diff line change
Expand Up @@ -223,9 +223,9 @@ The runtime that a Harness selects decides which ModelConfig an Agent that runs
| `Foundry` | ✅ | ✅ | ❌ | ❌ |
| `Mistral` | ❌ | ❌ | ❌ | ❌ |

kagent rejects `AnthropicVertexAI`, `GeminiVertexAI`, and `SAPAICore` on every runtime, because each authenticates with a credential that the egress gateway cannot place in an HTTP header. The ModelConfig never compiles, so no agent can use these providers. For the alternatives, see [About model providers]({{< link path="setup/model-providers/about-model-providers#credentials-that-do-not-compile" >}}).
kagent rejects `AnthropicVertexAI`, `GeminiVertexAI`, and `SAPAICore` on every runtime, because each authenticates with a credential that the egress gateway cannot place in an HTTP header. No agent can use these providers. For the alternatives, see [About model providers]({{< link path="setup/model-providers/about-model-providers#credentials-that-do-not-compile" >}}).

kagent rejects `Mistral` for a different reason. The controller does not resolve the provider at all, so a Mistral ModelConfig reports `unsupported model provider: Mistral` and does not compile a revision.
kagent rejects `Mistral` for a different reason. The controller does not resolve the provider at all, so a Mistral ModelConfig reports `unsupported model provider: Mistral`.

Some supported combinations still carry restrictions.

Expand Down Expand Up @@ -289,8 +289,8 @@ A runtime that meets one of these names in `spec.env` either refuses the Harness

| Runtime | Result of naming a controller-owned variable in `spec.env` |
| ------- | ---------------------------------------------------------- |
| `claude` | The Harness fails to compile, and the controller reports `Harness env "<name>" conflicts with Claude-owned runtime configuration`. |
| `codex` | The Harness fails to compile, and the controller reports `Harness env "<name>" conflicts with Codex's compiled configuration`. |
| `claude` | Every Agent on the Harness reports `Compatible` as `False`, and the controller reports `Harness env "<name>" conflicts with Claude-owned runtime configuration`. |
| `codex` | Every Agent on the Harness reports `Compatible` as `False`, and the controller reports `Harness env "<name>" conflicts with Codex's compiled configuration`. |
| `kagent` | The entry is dropped and the controller's own value is compiled in its place. No error is reported. |

Two further variables fall outside that table:
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -9,18 +9,27 @@ A `ModelConfig` is a Kubernetes custom resource that names one model at one prov

The kagent installation creates a `default-model-config` ModelConfig from the provider API key that you supply at install time, so a first agent does not need extra setup. To use a different provider, a different model, or a different set of credentials, create additional ModelConfigs.

## How a ModelConfig reaches an agent
## Configure a ModelConfig {#how-a-modelconfig-reaches-an-agent}

Every ModelConfig shares the same three parts, regardless of the provider that it names.

| Field | Description |
| ----- | ----------- |
| `provider` | The provider to use. Accepted values are `OpenAI`, `Anthropic`, `AzureOpenAI`, `Ollama`, `Gemini`, `GeminiVertexAI`, `AnthropicVertexAI`, `Bedrock`, `SAPAICore`, `Foundry`, and `Mistral`. Defaults to `OpenAI`. The API accepts `Mistral`, but the controller does not resolve it, so a Mistral ModelConfig reports `unsupported model provider: Mistral` and does not compile a revision. |
| `provider` | The provider to use. Accepted values are `OpenAI`, `Anthropic`, `AzureOpenAI`, `Ollama`, `Gemini`, `GeminiVertexAI`, `AnthropicVertexAI`, `Bedrock`, `SAPAICore`, `Foundry`, and `Mistral`. Defaults to `OpenAI`. The API accepts `Mistral`, but the controller does not resolve it, so the controller rejects a Mistral ModelConfig and reports `unsupported model provider: Mistral`. |
| `model` | The model name, as the provider spells it. |
| `stream` | Whether the agent streams its model calls. Defaults to `true`. Set `false` for an endpoint that rejects streaming, which otherwise fails every turn. |
| Provider block | A block named after the provider, such as `openAI` or `bedrock`, holding the settings that only that provider takes. An empty block is valid when the provider does not take extra settings. |

Credentials come from a Kubernetes Secret in the same namespace as the ModelConfig. The `apiKeySecret` field names the Secret, and `apiKeySecretKey` names the key within that Secret. To forward the bearer token from the incoming request to the provider instead, set `apiKeyPassthrough: true`. A ModelConfig cannot set both `apiKeyPassthrough` and `apiKeySecret`. For every ModelConfig field, including its type, default, and validation rules, see the [API reference]({{< link path="reference/api-ref#modelconfigspec" >}}).
Credentials come from a Kubernetes Secret in the same namespace as the ModelConfig. The `apiKeySecret` field names the Secret, and `apiKeySecretKey` names the key within that Secret. For every ModelConfig field, including its type, default, and validation rules, see the [API reference]({{< link path="reference/api-ref#modelconfigspec" >}}).

### Forward the caller's token

Set `apiKeyPassthrough: true` to send the bearer token from the incoming agent-to-agent (A2A) request to the provider as the API key, rather than reading a key from a Secret. A ModelConfig cannot set both `apiKeyPassthrough` and `apiKeySecret`. The setting is not available for the `Gemini`, `GeminiVertexAI`, or `AnthropicVertexAI` providers, or on a `codex` or `claude` Harness.

<!-- REVIEW: the ADK runtime reads the caller's token with models.PassthroughToken and uses it as the
provider key. What is unconfirmed is how that interacts with the egress gateway, which injects a static
header from a Secret. If the gateway overwrites or strips the auth header, passthrough does not work at
all. Needs someone who has run it. -->

### Turn off model streaming

Expand Down Expand Up @@ -65,9 +74,9 @@ When a request is made to a provider, the gateway puts the credential in the hea

Substrate matches a destination on the exact DNS hostname, without path, port, or scheme. Two credentials that target the same hostname and header are rejected, including a conflict between an agent's model, a memory embedding model, and an MCP server. Give such origins distinct DNS names. A destination given as an IP address cannot carry an injected credential at all.

### Credentials that do not compile
### Credentials that kagent rejects {#credentials-that-do-not-compile}

Header injection accepts one shape of credential: a static string. A credential that requires a local signature, a token exchange, or a file mounted into the agent cannot be injected, so kagent rejects the configuration instead of passing the credential to the runtime. The Agent reports the `Compatible` condition as `False` with the reason `UnsupportedConfiguration`, and kagent does not compile a revision from it. Any Session that already exists keeps running the last revision that compiled.
Header injection accepts one shape of credential, a static string. A credential that requires a local signature, a token exchange, or a file mounted into the agent cannot be injected, so kagent rejects the configuration instead of passing the credential to the runtime. The Agent reports the `Compatible` condition as `False` with the reason `UnsupportedConfiguration`, and `Ready` as `False`. kagent does not build a new revision, and the Agent keeps its last successful revision, so a Session that is already running is unaffected.

| Configuration | Why it cannot be injected | What to use instead |
| ------------- | ------------------------- | ------------------- |
Expand All @@ -76,23 +85,23 @@ Header injection accepts one shape of credential: a static string. A credential
| `SAPAICore` | OAuth2 client credentials are exchanged for a token before any request. | A provider that authenticates with an API key. See [SAP AI Core]({{< link path="setup/model-providers/sap-ai-core" >}}). |
| A `credentialRef` in a Harness `spec.env` entry | An arbitrary variable does not name a destination or a header to bind it to. | A ModelConfig or a RemoteMCPServer, each of which carries a destination. See [Agent harness]({{< link path="agents/agent-harness#configure-a-harness" >}}). |
| `openAI.tokenExchange` | The block reads a mounted service account file to acquire a token. | An endpoint that accepts a static API key. See [OpenAI]({{< link path="setup/model-providers/openai" >}}). |
| `tls.caCertSecretRef`, on any provider | The CA bundle is mounted as a file. | An endpoint whose certificate chains to a public CA. Setting `tls.disableVerify: true` skips certificate verification entirely and belongs only in a test environment. |
| A custom certificate authority bundle | The bundle is mounted as a file, and `tls` defines no field for one. The API server rejects `caCertSecretRef` and `caCertSecretKey` as unknown fields. | An endpoint whose certificate chains to a public CA. Setting `tls.disableVerify: true` skips certificate verification entirely and belongs only in a test environment. |

A rejected credential reports one of two messages. A credential that cannot be injected reports `cannot use gateway header injection`, and one that needs a mounted file reports `ModelConfig requires volume mounts unsupported by Substrate ActorTemplate`.

The `Ollama` provider is unaffected when it points at a local daemon, which authenticates with no credential. An Ollama Cloud model reaches `api.ollama.com` with an API key, and that key is injected as a header like any other. For when a model routes to the cloud, see [Ollama]({{< link path="setup/model-providers/ollama" >}}).

## The Harness runtime decides which providers are available
## Runtime provider support

A ModelConfig is only half of the decision. The runtime that a {{< gloss "Harness" >}}Harness{{< /gloss >}} selects also constrains which providers an agent can use, because each runtime integrates a different set.
The runtime that a {{< gloss "Harness" >}}Harness{{< /gloss >}} selects constrains which providers an agent can use, because each runtime integrates a different set.

- The **`kagent`** runtime integrates every provider, and the **`byo`** runtime integrates the same set, because both compile through the same path.
- The **`codex`** runtime integrates only `OpenAI` and `Bedrock`.
- The **`claude`** runtime integrates only `Anthropic` and `Bedrock`.

A provider is available only when the runtime integrates it and the egress gateway can inject its credential as a header. The `kagent` and `byo` runtimes integrate `AnthropicVertexAI`, `GeminiVertexAI`, and `SAPAICore`, but the gateway cannot inject a Google service account key or a set of OAuth2 client credentials. kagent 1.0 rejects a ModelConfig that names any of the three, on every runtime, so no agent can reach these providers at all. For the alternatives, see [Credentials that do not compile](#credentials-that-do-not-compile).
A provider is available only when the runtime integrates it and the egress gateway can inject its credential as a header. The `kagent` and `byo` runtimes integrate `AnthropicVertexAI`, `GeminiVertexAI`, and `SAPAICore`, but the gateway cannot inject a Google service account key or a set of OAuth2 client credentials, so kagent rejects a ModelConfig that names any of the three on every runtime. For the alternatives, see [Credentials that kagent rejects](#credentials-that-do-not-compile).

Neither `codex` nor `claude` accepts a ModelConfig that sets `defaultHeaders`, `tls`, or `apiKeyPassthrough`, and each narrows the provider settings it takes. A pair that asks for a provider its runtime does not integrate fails to compile, and the AgentTemplate reports the `Compatible` condition as `False` with the reason `UnsupportedConfiguration`.
Neither `codex` nor `claude` accepts a ModelConfig that sets `defaultHeaders`, `tls`, or `apiKeyPassthrough`, and each narrows the provider settings that it takes. When a ModelConfig names a provider that its runtime does not integrate, the Agent reports the `Compatible` condition as `False` with the reason `UnsupportedConfiguration`.

For the full matrix, including the per-combination restrictions, see [Agent harness]({{< link path="agents/agent-harness#model-provider-support" >}}).

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ Amazon Bedrock serves models from several families behind one AWS API. {{< reuse
Prefer the native provider. If you need the OpenAI request format, or an inference profile that only that endpoint exposes, use the OpenAI-compatible path.

> [!NOTE]
> Bedrock is the only provider that every runtime supports. On each runtime, the Bedrock credential must be a bearer token. For a `codex` Harness, you must use OpenAI `gpt-*` model IDs. Neither a `codex` nor a `claude` Harness accepts a `bedrock` setting beyond `region`. A Harness that asks for anything else fails to compile. For more information, see [Agent harness]({{< link path="agents/agent-harness#model-provider-support" >}}).
> Bedrock is the only provider that every runtime supports. On each runtime, the Bedrock credential must be a bearer token. For a `codex` Harness, you must use OpenAI `gpt-*` model IDs. Neither a `codex` nor a `claude` Harness accepts a `bedrock` setting beyond `region`. An Agent that pairs either Harness with any other `bedrock` setting reports the `Compatible` condition as `False`. For more information, see [Agent harness]({{< link path="agents/agent-harness#model-provider-support" >}}).

> [!IMPORTANT]
> Both paths authenticate with credentials from a Kubernetes Secret. Attaching an AWS IAM role to the agent, such as with [EKS IAM Roles for Service Accounts](https://docs.aws.amazon.com/eks/latest/userguide/iam-roles-for-service-accounts.html), is not currently supported: an agent runs as a Substrate Actor rather than as a pod that kagent controls, so there is no per-agent ServiceAccount to attach a role to.
Expand All @@ -37,7 +37,7 @@ The native provider authenticates with a Bedrock API key, which kagent sends as
```

> [!IMPORTANT]
> A Secret that holds `AWS_ACCESS_KEY_ID` and `AWS_SECRET_ACCESS_KEY` instead does not work on any runtime. IAM credentials sign each request locally, and an agent reaches its provider through an egress gateway that injects a static header, so kagent rejects the ModelConfig at compile time. The AgentTemplate reports the `Compatible` condition as `False` with the message `environment credential "AWS_ACCESS_KEY_ID" cannot use gateway header injection; local signing and arbitrary secret environment variables are unsupported`. For more information, see [About model providers]({{< link path="setup/model-providers/about-model-providers#credentials-that-do-not-compile" >}}).
> A Secret that holds `AWS_ACCESS_KEY_ID` and `AWS_SECRET_ACCESS_KEY` instead does not work on any runtime. IAM credentials sign each request locally, and an agent reaches its provider through an egress gateway that injects a static header, so kagent rejects the ModelConfig. The Agent reports the `Compatible` condition as `False` with the message `environment credential "AWS_ACCESS_KEY_ID" cannot use gateway header injection; local signing and arbitrary secret environment variables are unsupported`. For more information, see [About model providers]({{< link path="setup/model-providers/about-model-providers#credentials-that-do-not-compile" >}}).

3. Create a `ModelConfig` that uses the `Bedrock` provider.
```yaml
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -85,7 +85,7 @@ Two properties of Anthropic's pricing decide whether caching reduces cost. Neith
For the models that support caching, the current minimum prefix sizes, and the pricing, see the [Anthropic prompt caching docs](https://platform.claude.com/docs/en/build-with-claude/prompt-caching).

> [!WARNING]
> **The `claude` harness rejects `promptCaching: true`.** That harness accepts no `anthropic` settings beyond `baseUrl`, so a ModelConfig that enables caching fails to compile for it, and the AgentTemplate reports `Claude does not support Anthropic provider options beyond baseUrl yet` rather than becoming ready. Claude Code caches its own prefix on a 5-minute window regardless, so the setting gains nothing there. Where a `claude` agent and a `kagent` agent must share one installation, give the `claude` agent a ModelConfig that leaves `promptCaching` unset rather than enabling the field chart-wide. For the settings that each harness takes, see [Agent harness]({{< link path="agents/agent-harness#model-provider-support" >}}).
> **The `claude` harness rejects `promptCaching: true`.** That harness accepts no `anthropic` settings beyond `baseUrl`, so kagent rejects a ModelConfig that enables caching for it, and the Agent reports `Claude does not support Anthropic provider options beyond baseUrl yet` rather than becoming ready. Claude Code caches its own prefix on a 5-minute window regardless, so the setting gains nothing there. Where a `claude` agent and a `kagent` agent must share one installation, give the `claude` agent a ModelConfig that leaves `promptCaching` unset rather than enabling the field chart-wide. For the settings that each harness takes, see [Agent harness]({{< link path="agents/agent-harness#model-provider-support" >}}).

### Prompt caching at install time

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -131,7 +131,7 @@ EOF
| ------- | ----- |
| The ModelConfig reports `Accepted` as `False` | A referenced Secret or `endpointFrom` ConfigMap does not exist, or does not hold the named key. Run `kubectl describe modelconfig` to see which one. |
| Foundry returns `401 Unauthorized` or `403 Forbidden` | The API key does not have access to the resource. |
| The AgentTemplate reports `Compatible` as `False` | The resolved configuration cannot run on the Harness. Read the condition message for the specific reason. |
| The Agent reports `Compatible` as `False` | The resolved configuration cannot run on the Harness. Read the condition message for the specific reason. |

## Next steps

Expand Down
Loading
Loading