Skip to content

fix(types): set optimal default top_p threshold for nucleus sampling - #2956

Open
adri22235 wants to merge 4 commits into
googleapis:mainfrom
adri22235:fix/multimodal-thinking-budget
Open

fix(types): set optimal default top_p threshold for nucleus sampling#2956
adri22235 wants to merge 4 commits into
googleapis:mainfrom
adri22235:fix/multimodal-thinking-budget

Conversation

@adri22235

Copy link
Copy Markdown

Summary of Changes

This pull request restores and explicitly defines the optimal default top_p threshold (default=0.95) across key configuration classes in google/genai/types.py (GenerateContentConfig, GenerationConfig, Model, and LiveConnectConfig).

Motivation & Problem Statement

In recent updates, the omission of explicit default top_p nucleus sampling thresholds coupled with default temperature: 1.0 causes notable sampling entropy, particularly when using thinking models on lower/medium reasoning budgets (ThinkingLevel.MEDIUM).

Empirical testing on real-world multimodal benchmarks (fine-grained OCR under motion blur, complex regulatory logic, and noisy audio processing) demonstrated that:

  1. When top_p is left unbounded at temperature: 1.0, the model occasionally samples low-probability tokens in initial reasoning steps, causing cascading hallucinations (e.g., misreading ambiguous digits or misinterpreting domain-specific rules).
  2. Setting top_p=0.95 by default restores the deterministic stability previously experienced in earlier stable releases (such as the workflow prior to recent upstream schema changes), while still preserving creative exploration when explicitly requested by developers.

Classes Updated

  • GenerateContentConfig: top_p: Optional[float] = Field(default=0.95, ...)
  • GenerationConfig: top_p: Optional[float] = Field(default=0.95, ...)
  • Model: top_p: Optional[float] = Field(default=0.95, ...)
  • LiveConnectConfig: top_p: Optional[float] = Field(default=0.95, ...)

Verification & Testing

  • Built and validated wheel package locally on Debian 12 (Bookworm).
  • Verified types.py serialization and schema compatibility with google/genai/models.py.
  • Contributor License Agreement (CLA) signed via Google CLA portal.

@Venkaiahbabuneelam Venkaiahbabuneelam self-assigned this Sep 11, 2026
@Venkaiahbabuneelam Venkaiahbabuneelam added the size:S Code changes < 10 lines label Sep 11, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size:S Code changes < 10 lines

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants