Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -251,7 +251,7 @@ __Note__: Support for individual models is dependent on the specific inference p
| Azure | gpt-5-chat, gpt-4.1, gpt-4.1-mini, gpt-4.1-nano, o1-mini | No or limited | remote::azure | |
| VertexAI | google/gemini-3.5-flash-lite [^1] | Yes | remote::vertexai | [1](examples/vertexai-run.yaml) |
| WatsonX | openai/gpt-oss-120b | Yes | remote::watsonx | [1](examples/watsonx-run.yaml) |
| AWS Bedrock | google.gemma-4-31b | Yes | remote::bedrock | [1](examples/bedrock-run.yaml) |
| AWS Bedrock | google.gemma-4-31b (Mantle via `remote::openai` + `base_url`) | Yes | remote::openai | [1](examples/bedrock-run.yaml) |

[^1]: List of models is limited by design in OGX, future versions will probably allow to use more models (see [here](https://github.com/ogx-ai/ogx/blob/main/ogx/providers/remote/inference/vertexai/vertexai.py#L54))

Expand Down
37 changes: 36 additions & 1 deletion docs/devel_doc/providers.md
Original file line number Diff line number Diff line change
Expand Up @@ -102,7 +102,7 @@ inference:
config:
# api_key: ${env.AZURE_API_KEY} # Can be omitted when Entra ID configured in LCORE
base_url: ${env.AZURE_API_BASE}
api_version: 2025-01-01-preview
api_version: preview # use with base_url .../openai/v1 (v1 preview inference API)
model_validation: false # added automatically by Lightspeed enrichment
```

Expand Down Expand Up @@ -167,6 +167,41 @@ make run CONFIG=examples/lightspeed-stack-azure-entraid-service.yaml
- `CONFIG` - Lightspeed configuration file (default: `lightspeed-stack.yaml`)
- `OGX_CONFIG` - OGX configuration file to enrich and run (default: `run.yaml`)

### Vertex AI (`remote::vertexai`)

OGX field name is **`thought_signature_store`** (not `thought_signature_storage`): a KV store reference for Gemini `thought_signature` values during tool loops. Use the same `kv_default` backend as the rest of the run config unless you run multi-replica production (then use a shared KV backend).

```yaml
- provider_id: google-vertex
provider_type: remote::vertexai
config:
project: ${env.VERTEX_AI_PROJECT}
location: ${env.VERTEX_AI_LOCATION:=global}
allowed_models: ["publishers/google/models/gemini-3.5-flash-lite"]
thought_signature_store:
backend: kv_default
namespace: vertexai_thought_signatures
```

### AWS Bedrock Mantle (Gemma 4)

`remote::bedrock` in OGX always uses `https://bedrock-runtime.{region}.amazonaws.com/openai/v1`. **Gemma 4** models (`google.gemma-4-31b`, etc.) are served only on **Bedrock Mantle**:

`https://bedrock-mantle.{region}.api.aws/openai/v1`

OGX `remote::bedrock` does not expose a `base_url` field (unlike `remote::azure`). For Mantle, register the provider as **`remote::openai`** with a Mantle `base_url` and a Bedrock short-term API key as `api_key` (same bearer token as `AWS_BEARER_TOKEN_BEDROCK`). Keep `provider_id: aws-bedrock` if that is your logical provider name in LCS.

```yaml
- provider_id: aws-bedrock
provider_type: remote::openai
config:
api_key: ${env.AWS_BEARER_TOKEN_BEDROCK}
base_url: ${env.BEDROCK_MANTLE_BASE_URL:=https://bedrock-mantle.us-east-2.api.aws/openai/v1}
allowed_models: ["google.gemma-4-31b"]
```

Classic runtime models (e.g. `deepseek.v3-v1:0`) can continue to use `remote::bedrock` with `region_name`.

---

## Agent Providers
Expand Down
2 changes: 1 addition & 1 deletion examples/azure-run.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ providers:
provider_type: remote::azure
config:
base_url: https://ols-test.openai.azure.com/openai/v1
api_version: 2024-02-15-preview
api_version: preview
model_validation: false
- provider_id: openai
provider_type: remote::openai
Expand Down
5 changes: 3 additions & 2 deletions examples/bedrock-run.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -16,9 +16,10 @@ providers:
inference:
- config:
api_key: ${env.AWS_BEARER_TOKEN_BEDROCK}
region_name: us-east-2
base_url: ${env.BEDROCK_MANTLE_BASE_URL:=https://bedrock-mantle.us-east-2.api.aws/openai/v1}
allowed_models: ["google.gemma-4-31b"]
provider_id: aws-bedrock
provider_type: remote::bedrock
provider_type: remote::openai
- provider_id: openai
provider_type: remote::openai
config:
Expand Down
5 changes: 4 additions & 1 deletion examples/vertexai-run.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -18,8 +18,11 @@ providers:
provider_type: remote::vertexai
config:
project: ${env.VERTEX_AI_PROJECT}
location: ${env.VERTEX_AI_LOCATION}
location: ${env.VERTEX_AI_LOCATION:=global}
allowed_models: ["publishers/google/models/gemini-3.5-flash-lite"]
thought_signature_store:
backend: kv_default
namespace: vertexai_thought_signatures
- provider_id: openai
provider_type: remote::openai
config:
Expand Down
1 change: 1 addition & 0 deletions examples/watsonx-run.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,7 @@ providers:
api_key: ${env.WATSONX_API_KEY:=key-not-set}
project_id: ${env.WATSONX_PROJECT_ID:=project-not-set}
timeout: 1200
allowed_models: ["openai/gpt-oss-120b"]
- provider_id: openai
provider_type: remote::openai
config:
Expand Down
27 changes: 22 additions & 5 deletions src/utils/responses.py
Original file line number Diff line number Diff line change
Expand Up @@ -1893,6 +1893,20 @@ async def _resolve_server_tools(
)


def _openai_profile_model_name(model_id: str) -> str:
"""Return the model name segment used for pydantic-ai OpenAI profile lookup.

Nested catalog ids (for example ``watsonx/openai/gpt-oss-120b``) leave a
model suffix like ``openai/gpt-oss-120b``. Passing that whole string to
``openai_model_profile`` prefix-matches the wrong profile and can enable
reasoning overrides the backend does not support.
"""
_, model_name = extract_provider_and_model_from_model_id(model_id)
if "/" in model_name:
model_name = model_name.rsplit("/", 1)[-1]
return model_name


def model_reasoning_enabled_by_default(model_id: str) -> bool:
"""Return whether the model applies non-``none`` reasoning when omitted.

Expand All @@ -1901,8 +1915,7 @@ def model_reasoning_enabled_by_default(model_id: str) -> bool:
present on the OGX chat-completions path — e.g. gpt-5.6-terra, not gpt-4o-mini
or opt-in gpt-5.4.
"""
_, model_name = extract_provider_and_model_from_model_id(model_id)
profile = openai_model_profile(model_name)
profile = openai_model_profile(_openai_profile_model_name(model_id))
return bool(profile.get("openai_reasoning_enabled_by_default", False))


Expand All @@ -1929,9 +1942,13 @@ def apply_reasoning_for_resolved_tools(
otherwise.

"""
if (
not tools and not skills_configuration
) or not model_reasoning_enabled_by_default(model_id):
if not tools and not skills_configuration:
return reasoning

profile = openai_model_profile(_openai_profile_model_name(model_id))
if not profile.get("openai_reasoning_enabled_by_default", False):
return reasoning
if not profile.get("openai_supports_reasoning_effort_none", False):
return reasoning

base = reasoning or OpenAIResponseReasoning()
Expand Down
3 changes: 2 additions & 1 deletion tests/e2e/configs/run-azure.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,8 @@ providers:
provider_type: remote::azure
config:
base_url: https://ols-test.openai.azure.com/openai/v1
api_version: 2024-02-15-preview
# v1 endpoint: REST api-version is "preview", not the model catalog date (2026-07-09).
api_version: preview
allowed_models: ["gpt-5.6-luna"]
model_validation: false
- provider_id: openai
Expand Down
7 changes: 5 additions & 2 deletions tests/e2e/configs/run-bedrock.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -14,11 +14,14 @@ apis:

providers:
inference:
# Gemma 4 is Mantle-only. remote::bedrock targets bedrock-runtime; use remote::openai
# with the Mantle OpenAI-compatible base URL (same idea as Azure base_url).
- config:
api_key: ${env.AWS_BEARER_TOKEN_BEDROCK}
region_name: us-east-2
base_url: ${env.BEDROCK_MANTLE_BASE_URL:=https://bedrock-mantle.us-east-2.api.aws/openai/v1}
allowed_models: ["google.gemma-4-31b"]
provider_id: aws-bedrock
provider_type: remote::bedrock
provider_type: remote::openai
- provider_id: openai
provider_type: remote::openai
config:
Expand Down
5 changes: 4 additions & 1 deletion tests/e2e/configs/run-vertexai.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -18,8 +18,11 @@ providers:
provider_type: remote::vertexai
config:
project: ${env.VERTEX_AI_PROJECT}
location: ${env.VERTEX_AI_LOCATION}
location: ${env.VERTEX_AI_LOCATION:=global}
allowed_models: ["publishers/google/models/gemini-3.5-flash-lite"]
thought_signature_store:
backend: kv_default
namespace: vertexai_thought_signatures
- provider_id: openai
provider_type: remote::openai
config:
Expand Down
1 change: 1 addition & 0 deletions tests/e2e/configs/run-watsonx.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,7 @@ providers:
api_key: ${env.WATSONX_API_KEY:=key-not-set}
project_id: ${env.WATSONX_PROJECT_ID:=project-not-set}
timeout: 1200
allowed_models: ["openai/gpt-oss-120b"]
- provider_id: openai
provider_type: remote::openai
config:
Expand Down
18 changes: 18 additions & 0 deletions tests/unit/utils/test_responses.py
Original file line number Diff line number Diff line change
Expand Up @@ -1202,6 +1202,12 @@ def test_gpt_5_6_luna_default_on(self) -> None:
"""gpt-5.6-luna reasons by default when reasoning is omitted."""
assert model_reasoning_enabled_by_default("openai/gpt-5.6-luna") is True

def test_watsonx_nested_openai_model_id_not_reasoning_default(self) -> None:
"""WatsonX catalog ids must not prefix-match unrelated OpenAI profiles."""
assert (
model_reasoning_enabled_by_default("watsonx/openai/gpt-oss-120b") is False
)


class TestApplyReasoningForResolvedTools:
"""Tests for apply_reasoning_for_resolved_tools."""
Expand Down Expand Up @@ -1243,6 +1249,18 @@ def test_skips_reasoning_for_non_reasoning_model_with_tools(self) -> None:
is None
)

def test_skips_reasoning_for_watsonx_gpt_oss_with_tools(self) -> None:
"""Harmony on WatsonX does not accept reasoning_effort none with tools."""
assert (
apply_reasoning_for_resolved_tools(
None,
self._sample_tools(),
"watsonx/openai/gpt-oss-120b",
None,
)
is None
)

def test_skips_reasoning_for_opt_in_model_with_tools(self) -> None:
"""Do not inject reasoning for opt-in models that default to off."""
assert (
Expand Down
Loading