Skip to content

fix(llms): strip matching provider prefix when an explicit provider is given - #8740

Open
heyseere wants to merge 3 commits into
crewAIInc:mainfrom
heyseere:fix/llm-explicit-provider-prefix
Open

heyseere wants to merge 3 commits into
crewAIInc:mainfrom
heyseere:fix/llm-explicit-provider-prefix

Conversation

@heyseere

@heyseere heyseere commented Oct 28, 2026 •

Copy link
Copy Markdown

Related issue

Fixes #8736

Summary

LLM(model="openai/gpt-4o", provider="openai") kept the full prefixed model string (openai/gpt-4o) and sent it verbatim to the API, which rejects it — while LLM(model="openai/gpt-4o") correctly routes with gpt-4o. The explicit_provider branch of _resolve_route now strips the prefix when it canonicalizes via PROVIDER_ALIASES to the same provider, mirroring the existing behavior of the custom_openai branch. Unrelated prefixes (e.g. a namespaced model id) are left untouched, and DeepInfra's org/model validation now applies to the stripped id.

Verification

  • Tests added or updated for the changed behavior
  • Relevant tests and quality checks pass locally

lib/crewai/tests/test_llm.py (105 passed, 3 skipped): new test_explicit_provider_strips_matching_model_prefix covers openai/anthropic/deepinfra stripping, unrelated prefixes staying intact, malformed DeepInfra ids failing loudly, and kwargs not being mutated. ruff check clean on changed files.

The rebase onto main pulled in #7459's test_llm_explicit_deepinfra_provider_rejects_malformed_model, which asserted the old behavior for model="deepinfra/deepseek-ai/...", provider="deepinfra" (and CI has not run yet, so it went unnoticed). Since stripping that redundant prefix is this PR's intent — mirroring the prefix-routing and custom_openai paths — the double-prefixed case moved into a new positive test (test_llm_explicit_deepinfra_provider_strips_matching_prefix, 7353870) and the genuinely malformed cases (no org part, trailing slash) stay in the rejection set. tests/llms/ + tests/test_llm.py: 1192 passed, 23 skipped.

Disclosure: this PR was produced with AI assistance (ZCode agent) and human-reviewed — please apply the llm-generated label.

Additional context

None

From ZCode


Note

Medium Risk
Changes core LLM factory routing for every call with an explicit provider, which determines the model string sent to provider APIs; behavior is narrowed and well tested but affects a hot path.

Overview
Fixes a routing bug where LLM(..., provider=...) kept a redundant provider prefix on the model id and sent it verbatim to native APIs (e.g. openai/gpt-4o with provider="openai"), while unprefixed routing already used gpt-4o.

The explicit_provider branch in _resolve_route now strips the first path segment when it canonicalizes (via PROVIDER_ALIASES) to the same provider as the kwarg—including alias pairs like claude/anthropic, google/gemini, aws/bedrock, and azure_openai/azure. Unrelated prefixes (e.g. namespaced ids) are unchanged. DeepInfra org/model validation runs on the stripped id.

Tests cover _resolve_route, DeepInfra integration (including moving double-prefixed deepinfra/... from malformed rejection to expected stripping), and kwargs immutability.

Reviewed by Cursor Bugbot for commit 30fd598. Bugbot is set up for automated code reviews on this repo. Configure here.

@strix-security

strix-security Bot commented Oct 28, 2026 •

Copy link
Copy Markdown

Strix Security Review

Warning

This pull request has 25 commits after the last Strix review (0f6f761). Strix has not reviewed these changes.
Automatic review on push is off for this repository. To review the latest changes, tag @strix-security in a comment, or turn on re-review on push.

✅ No open security findings.

PR summary: Fixes LLM routing so that when an explicit provider is passed alongside a model string that repeats that provider's prefix (e.g. model="openai/gpt-4o", provider="openai"), the prefix is stripped before the model id is sent to the API, instead of being forwarded verbatim.

Review summary

No security issues found. The patch only changes the model identifier sent to the already-selected provider when a model string repeats that provider's prefix; provider selection, base_url/api_key resolution, and host targeting are all keyed off the unchanged explicit provider value, and the change is strictly subtractive (it can only remove a leading alias segment, never add one). DeepInfra's org/model validation now runs on the stripped id and is stricter, not weaker. Two independent traces through the native provider classes and LiteLLM fallback found no sink where the stripped value behaves differently in a security-relevant way. One non-security note for maintainers: the comparison uses the raw provider value, so alias spellings such as claude, google, aws, and azure_openai still fail to strip, leaving the original bug only partially fixed.

Ruled out (2)
  • Can the new prefix stripping send a request to a different provider, endpoint, or credential?
    Provider selection, native class selection, and base_url/api_key resolution all key off the unchanged explicit provider value, and every alias that could collide (claude/anthropic, google/gemini, aws/bedrock, azure_openai/azure) resolves to the same native class, so stripping only affects the model…
  • Can the stripped model string reach a URL, header, or filesystem sink that the prefixed value could not?
    The change only removes a leading alias segment, model text reaches request bodies rather than hosts, and the one place a model is interpolated into a URL (Azure deployment names) ends up with fewer path segments than before.

Review 1 on this PR · 0f6f761 · requested Oct 28 16:07 UTC when the PR was opened


Reviewed by Strix
Re-run review · Configure security review settings

@cursor cursor Bot left a comment •

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Stale Bugbot comment from a previous run.

Comment thread lib/crewai/src/crewai/llm.py Resolved
@heyseere
heyseere force-pushed the fix/llm-explicit-provider-prefix branch 2 times, most recently from 4dc3d3d to b9b25e9 Compare October 29, 2026 15:42

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit b9b25e9. Configure here.

Comment thread lib/crewai/src/crewai/llm.py Resolved
@heyseere

Copy link
Copy Markdown
Author

Friendly ping for a review when you have a moment. Both Bugbot threads are resolved and the PR is mergeable; happy to adjust anything.

The prefix strip compared a canonicalized model prefix against the raw provider kwarg, so alias spellings (claude, google, aws, azure_openai) never matched and their prefixed model ids were still sent to the API verbatim. Canonicalize both sides and cover the alias routes.
@heyseere
heyseere force-pushed the fix/llm-explicit-provider-prefix branch from 7353870 to 30fd598 Compare November 1, 2026 09:58

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[BUG] LLM(provider=...) with a provider-prefixed model sends the prefixed name to the API (404)

1 participant