Skip to content

fix(llm): exclude sampling params for reasoning models#1822

Merged
AAgnihotry merged 3 commits into
mainfrom
fix/reasoning-model-temperature-v2.13.13
Jul 20, 2026
Merged

fix(llm): exclude sampling params for reasoning models#1822
AAgnihotry merged 3 commits into
mainfrom
fix/reasoning-model-temperature-v2.13.13

Conversation

@AAgnihotry

@AAgnihotry AAgnihotry commented Jul 20, 2026

Copy link
Copy Markdown
Contributor

Jira

SRE-625200

Summary

  • Skip temperature, n, frequency_penalty, presence_penalty, and top_p for OpenAI reasoning models (o1, o3, o4) which don't support these parameters and return 400 errors when they're included.
  • Applies to both UiPathOpenAIService and UiPathLlmChatService.
  • Bumps uipath version to 2.13.13.

Failure Details (from SRE-625200)

Alert: agents-eu-prd-agents-evalrun-failed-urt-[Agent Evaluations]-logalertv2-jae

Description: Monitors EvalRun.Failed and EvalSetRun.Failed events with URT runtime when ErrorMessage is not null. The alert fired in the jae (Japan East) region on 2026-07-18.

Alert details:

  • Severity: 3 (Minor)
  • Region: jae (Japan East)
  • Signal type: Log
  • Condition: Count of matching events > 0 in a 15-minute window
  • Metric value: 3 failures detected between 2026-07-18T06:12:26Z and 2026-07-18T06:27:26Z
  • Component: Agent Evaluations
  • Related incident: SRE-623359

Root cause: OpenAI reasoning models (o1, o3, o4) do not support temperature and other sampling parameters (n, frequency_penalty, presence_penalty, top_p). Sending these parameters causes 400 errors, which surface as EvalRun.Failed events.

Search query that triggered the alert:

customEvents
| where name contains "EvalRun.Failed" or name contains "EvalSetRun.Failed"
| where customDimensions.Runtime contains "URT"
| extend ErrorMessage = tostring(customDimensions.ErrorMessage)
| where isnotempty(ErrorMessage)

Test plan

  • Added parametrized tests for reasoning models (o1, o3, o4 variants) verifying sampling params are excluded
  • Added test verifying non-reasoning models still include temperature and sampling params

🤖 Generated with Claude Code

image

…s (o1, o3, o4)

Bump uipath to 2.13.13.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Copilot AI review requested due to automatic review settings July 20, 2026 13:28
@github-actions github-actions Bot added test:uipath-langchain Triggers tests in the uipath-langchain-python repository test:uipath-runtime test:uipath-integrations labels Jul 20, 2026

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR updates the UiPath LLM Gateway client(s) to avoid sending OpenAI sampling parameters that cause 400s for OpenAI “reasoning” models (o1/o3/o4), and bumps the uipath package version accordingly.

Changes:

  • Filter out temperature for reasoning models in UiPathOpenAIService.chat_completions.
  • Filter out temperature, n, frequency_penalty, presence_penalty, and top_p for reasoning models in UiPathLlmChatService.chat_completions.
  • Add tests covering reasoning-model filtering for the normalized LLM service; bump uipath to 2.13.13.

Reviewed changes

Copilot reviewed 3 out of 4 changed files in this pull request and generated 1 comment.

File Description
packages/uipath-platform/src/uipath/platform/chat/_llm_gateway_service.py Adds reasoning-model detection and excludes unsupported params from request bodies.
packages/uipath-platform/tests/services/test_llm_service.py Adds tests asserting reasoning models exclude sampling params (normalized API).
packages/uipath/pyproject.toml Bumps uipath version to 2.13.13.
packages/uipath/uv.lock Updates lockfile version entry for uipath to 2.13.13.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread packages/uipath-platform/tests/services/test_llm_service.py
AAgnihotry and others added 2 commits July 20, 2026 06:34
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…ests

Covers the passthrough (non-normalized) API path that also conditionally
omits temperature for o1/o3/o4 models.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@sonarqubecloud

Copy link
Copy Markdown

# Build request body - some models don't support certain parameters
model_lower = model.lower()
is_claude_model = "claude" in model_lower
is_reasoning_model = model_lower.startswith(("o1", "o3", "o4"))

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

using model name is unreliable sadly. But I think this is a best effort we can live with.

@Chibionos Chibionos left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

closest fix as BYOM can bring custom names too.

@AAgnihotry
AAgnihotry merged commit d190868 into main Jul 20, 2026
242 of 245 checks passed
@AAgnihotry
AAgnihotry deleted the fix/reasoning-model-temperature-v2.13.13 branch July 20, 2026 23:16
milind-jain-uipath added a commit that referenced this pull request Jul 21, 2026
0.2.11 was released to PyPI from main (#1822) without this change, so this
PR ships as 0.2.12.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

test:uipath-integrations test:uipath-langchain Triggers tests in the uipath-langchain-python repository test:uipath-runtime

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants