Skip to content

qwen_chat backend can't target reasoning models: always sends max_tokens + temperature (GPT-5.x / Opus reject them) #127

Description

@chirag127

The qwen_chat backend (the generic OpenAI-compatible client) hardcodes two params that current reasoning models reject through OpenAI-compatible gateways (Azure, LiteLLM proxies, OpenAI direct):

Bug 1 — max_tokens not overridable

_chat_messages_impl always sends "max_tokens". GPT-5.x / o-series demand max_completion_tokens and 400 on max_tokens:

Unsupported parameter: 'max_tokens' is not supported with this model. Use 'max_completion_tokens' instead.

Bug 2 — temperature cannot be omitted

_role_env(..., "TEMPERATURE", "0.7") uses os.environ.get(...) or default, so an explicit empty QWEN_CHAT_TEMPERATURE="" collapses back to 0.7. Reasoning models that deprecate temperature (e.g. Claude Opus 4.8) then 400:

`temperature` is deprecated for this model.

There is no way to tell the backend "do not send temperature at all."

Repro

Point QWEN_CHAT_BASE_URL at any gateway serving a GPT-5.x model, set target backend qwen_chat, run a rollout → every target call 400s. Same for an Opus-4.8 optimizer via temperature.

Impact

The qwen_chat client is effectively the generic OpenAI-compatible path (also discussed in #115), so this blocks using SkillOpt with the newest reasoning models behind Azure / LiteLLM / OpenAI gateways for both optimizer and target roles.

Proposed fix (PR incoming)

  • Add opt-in QWEN_CHAT_USE_MAX_COMPLETION_TOKENS (+ role variants) that swaps the payload key to max_completion_tokens.
  • Treat an explicit empty / none / off temperature as "omit the param" instead of falling back to 0.7.
  • Both default to current behavior; fully backward compatible. Tests included.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions