Skip to content

feat(types): add presence_penalty and token breakdown details to LLM Gateway - #191

Open
pylrn wants to merge 1 commit into
AssemblyAI:mainfrom
pylrn:feat/llm-gateway-presence-penalty-token-details
Open

pylrn wants to merge 1 commit into
AssemblyAI:mainfrom
pylrn:feat/llm-gateway-presence-penalty-token-details

Conversation

@pylrn

@pylrn pylrn commented Sep 26, 2026

Copy link
Copy Markdown

Description

This PR brings the Node.js SDK's LLM Gateway types into parity with the AssemblyAI LLM Gateway API and the Python SDK (parity with Python SDK PR #261).

Changes:

  • ModelDefaultParameters: Expose presence_penalty?: number | null;
  • ChatCompletionRequest: Expose presence_penalty?: number; and frequency_penalty?: number;
  • ChatCompletionMessage: Expose thinking?: string | null; to support reasoning models returning thought processes
  • ChatCompletionUsage: Expose detailed token breakdown fields:
    • prompt_tokens_details?: ChatCompletionPromptTokensDetails | null; (including cached_tokens, audio_tokens, cache_creation with ephemeral_5m_input_tokens / ephemeral_1h_input_tokens, and cache_write_tokens)
    • completion_tokens_details?: ChatCompletionTokensDetails | null; (including reasoning_tokens, audio_tokens, accepted_prediction_tokens, rejected_prediction_tokens)
    • input_tokens?: number | null; and output_tokens?: number | null;
  • ChatCompletionResponse: Expose optional http_status_code, response_time, llm_status_code, and request
  • ChatCompletionChoice: Allow finish_reason to be null
  • Added comprehensive unit tests in tests/unit/llm-gateway.test.ts verifying parsing of detailed usage breakdowns, thinking tokens, and presence penalties
  • Updated CHANGELOG.md and bumped package.json to 4.41.6 for the Version Bump Check workflow

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant