feat(llm_gateway): Add LLM Gateway Module

CLAUDE.md

@@ -1,6 +1,6 @@
 # AssemblyAI Python SDK
 
-Speech-to-text and audio intelligence SDK. Supports pre-recorded transcription, real-time streaming, and audio analysis features.
+Speech-to-text and audio intelligence SDK. Supports pre-recorded transcription, real-time streaming, audio analysis features, and LLM access through AssemblyAI's LLM Gateway.
 
 ## Quick start
 
@@ -54,6 +54,10 @@ aai.settings.api_key = "your-key"
 - `aai.DictationResponse` — Dictation result: `.text`, `.words` (`DictationWord` with `text` + `confidence`), `.confidence`, `.llm_response`, `.llm_error`, `.audio_duration_ms`, `.session_id`, `.request_time_ms`, `.sync_time_ms`, and `.final_text` (the LLM rewrite, falling back to `.text`)
 - `assemblyai.streaming.v3.RealTimeTranscriber` — Real-time streaming with event-based API (threaded)
 - `assemblyai.streaming.v3.AsyncRealTimeTranscriber` — Asyncio-native counterpart; same options/events
+- `aai.LLMGateway` — LLM Gateway client. Resources: `models.list()`, `chat.completions.create()` (incl. `stream=True`), `understanding.create()`/`.validate()`
+- `aai.AsyncLLMGateway` — Asyncio counterpart of `LLMGateway`. Same resources, every API call a coroutine; owns an HTTP pool (`async with` or `await aclose()`)
+- `assemblyai.llm_gateway.v1` — Canonical module for both gateways, matching the gateway's `/v1` API. The top-level `aai.*` names re-export it
+- `aai.LLMGatewayMessageParam` — `TypedDict` for a chat message; pass plain dicts (`{"role": "user", "content": "..."}`)
 
 ## Common patterns
 
@@ -448,6 +452,141 @@ async with AsyncRealTimeTranscriber(RealTimeTranscriberOptions(token=token_from_
 - `format_turns=True` enables punctuation/casing on confirmed end-of-turns. Toggle mid-session via `client.set_params(RealTimeSessionParameters(format_turns=True))`.
 - `AsyncRealTimeTranscriber` used as `async with` calls `disconnect(terminate=True)` on normal block exit and `disconnect(terminate=False)` on exception — no explicit `disconnect()` needed inside the block.
 
+## LLM Gateway
+
+`aai.LLMGateway` is a client for AssemblyAI's LLM Gateway: model listing, OpenAI-shaped chat
+completions routed to OpenAI/Claude/Gemini/Bedrock models, and speech understanding over an
+existing transcript. It targets `llm-gateway.assemblyai.com` and rides on the same
+`Client`/`AsyncClient` as the rest of the SDK.
+
+```python
+import assemblyai as aai
+
+aai.settings.api_key = os.environ["ASSEMBLYAI_API_KEY"]
+
+gateway = aai.LLMGateway()
+
+for model in gateway.models.list().data:
+    print(model.id, model.context_length, model.pricing.global_.prompt)
+
+completion = gateway.chat.completions.create(
+    model="claude-sonnet-5",
+    messages=[{"role": "user", "content": "Summarize this call."}],
+)
+print(completion.choices[0].message.content)
+```
+
+**Messages are plain dicts** (`aai.LLMGatewayMessageParam`) — no message object to build.
+`role` is `"user"`, `"assistant"`, `"system"`, or `"tool"` (a tool result, paired with
+`tool_call_id`); `content` is a string or a list of content-part dicts.
+
+**Request params**: everything past `model`/`messages`/`stream` is a keyword argument forwarded
+verbatim into the request body — `max_tokens`, `temperature`, `tools`, `tool_choice`,
+`response_format`, plus the AssemblyAI extensions (`fallbacks`, `fallback_config`,
+`zero_data_retention`, `transcript_id`, `reasoning`, `model_region`, `fail_fast`, `max_timeout`,
+…). `fallbacks` is a list of dicts in the API's own shape:
+`[{"model": "gpt-5-mini", "messages": [...]}]`.
+
+```python
+completion = gateway.chat.completions.create(
+    model="claude-sonnet-5",
+    messages=[{"role": "user", "content": "..."}],
+    max_tokens=512,
+    fallbacks=[{"model": "gpt-5-mini", "messages": [{"role": "user", "content": "..."}]}],
+    zero_data_retention=True,
+    transcript_id="transcript_abc123",
+)
+```
+
+**Streaming**: `stream=True` returns an iterator of `LLMGatewayCompletionChunk`:
+```python
+stream = gateway.chat.completions.create(
+    model="claude-sonnet-5",
+    messages=[{"role": "user", "content": "..."}],
+    stream=True,
+)
+for chunk in stream:
+    if chunk.choices[0].delta.content:
+        print(chunk.choices[0].delta.content, end="")
+```
+Chunk parsing is verified for OpenAI-routed models only. Chunks can carry tool-call fragments
+(`chunk.choices[0].delta.tool_calls`), but this SDK does not reassemble them or accumulate the
+streamed text for you — collect chunks yourself, or use a non-streaming `create()`.
+
+There is no built-in tool-calling loop: `tools`/`tool_choice` are forwarded like any other
+request param (see "Request params" above), and replaying a tool result is a plain
+`{"role": "tool", "tool_call_id": ..., "content": ...}` message per
+https://www.assemblyai.com/docs/llm-gateway/tool-calling — drive the loop yourself around
+`create()`.
+
+**Understanding**: speaker identification, translation, and custom formatting over an existing
+transcript — `transcript_id` is required, there is no raw-text path.
+```python
+result = gateway.understanding.create(
+    transcript_id="transcript_abc123",
+    request={"speaker_identification": {"speaker_type": "name", "speakers": [{"name": "Ana"}, {"name": "Peter"}]}},
+)
+print(result.speech_understanding["response"], result.utterances)
+
+gateway.understanding.validate(request={"translation": {"target_languages": ["es"]}})
+```
+`request` is a plain dict — the server has no fixed schema for it. `validate()` returns `None`
+when the request is valid and raises `aai.LLMGatewayError` with `.errors` populated otherwise;
+it costs the same auth/balance as a real call, so it is not a free dry run.
+
+**Errors**: every failure raises `aai.LLMGatewayError` (an `aai.AssemblyAIError`) with
+`.status_code`, `.request_id`, and `.errors` (validation messages, when the server sent any).
+For a stream, only a failure before the first byte raises — a mid-stream failure truncates the
+iterator.
+
+**Unknown server fields survive**: response models allow extras, so a field the gateway adds is
+readable as an attribute without an SDK upgrade, and collections the server sends as `null` are
+read as empty (`completion.choices` is always a list).
+
+**Settings**: `aai.settings.llm_gateway_base_url` (default
+`https://llm-gateway.assemblyai.com`) and `aai.settings.llm_gateway_http_timeout` (default
+`30.0`), both overridable via `ASSEMBLYAI_LLM_GATEWAY_BASE_URL` /
+`ASSEMBLYAI_LLM_GATEWAY_HTTP_TIMEOUT`.
+
+## Asyncio LLM Gateway (`AsyncLLMGateway`)
+
+`aai.AsyncLLMGateway` mirrors `aai.LLMGateway` with coroutines — same resources, same request
+params, same response models. Every API call is awaited; `stream=True` resolves to an
+async iterator of `LLMGatewayCompletionChunk` you `async for` over.
+
+```python
+import asyncio
+import assemblyai as aai
+
+aai.settings.api_key = os.environ["ASSEMBLYAI_API_KEY"]
+
+async def main():
+    async with aai.AsyncLLMGateway() as gateway:
+        completion = await gateway.chat.completions.create(
+            model="claude-sonnet-5",
+            messages=[{"role": "user", "content": "Summarize this call."}],
+        )
+        print(completion.choices[0].message.content)
+
+        stream = await gateway.chat.completions.create(
+            model="claude-sonnet-5",
+            messages=[{"role": "user", "content": "..."}],
+            stream=True,
+        )
+        async for chunk in stream:
+            print(chunk.choices[0].delta.content or "", end="")
+
+asyncio.run(main())
+```
+
+**Lifecycle**: the gateway owns an HTTP connection pool — use `async with`, or call
+`await gateway.aclose()`. Pass `client=aai.AsyncClient(settings=aai.settings)` to share one
+pool; a client you pass in stays yours to close. Sync `LLMGateway` needs no such handling: it
+borrows the shared default `Client` and has no `close()`.
+
+**Per-gateway key**: both gateways take `api_key=...` instead of a client, and build their own
+client for it. Given alongside `client`, the key wins and the passed client is left untouched.
+
 ## Important gotchas
 
 - **Always check status**: `if transcript.status == aai.TranscriptStatus.error` — accessing `.text` on a failed transcript returns None, not an exception
@@ -456,6 +595,7 @@ async with AsyncRealTimeTranscriber(RealTimeTranscriberOptions(token=token_from_
 - **Streaming v3 lives in its own module**: `assemblyai.streaming.v3` (there is no other streaming API in this SDK). See the "Streaming (real-time)" section above.
 - **The SDK does not capture microphone audio**: bring your own capture (e.g. `pyaudio`, `sounddevice`) and stream the PCM chunks
 - **`transcribe_async()` returns a `concurrent.futures.Future`**, not an asyncio coroutine. In asyncio code use `aai.AsyncTranscriber` (see "Asyncio transcription" above) — or `aai.AsyncSyncTranscriber` for the sync API
+- **LLM Gateway request params are `**kwargs`**: everything past `model`/`messages`/`stream` is forwarded verbatim, so there is no client-side check on the name — spell params carefully
 - **Timestamps are in milliseconds** throughout the SDK
 - **Minimum Python**: 3.8+
 
@@ -467,4 +607,5 @@ async with AsyncRealTimeTranscriber(RealTimeTranscriberOptions(token=token_from_
 
 - [Full documentation](https://www.assemblyai.com/docs)
 - [API reference](https://www.assemblyai.com/docs/api-reference)
+- [LLM Gateway docs](https://www.assemblyai.com/docs/llm-gateway/quickstart)
 - [llms-full.txt](https://www.assemblyai.com/docs/llms-full.txt?lang=python) (Python-filtered docs for LLMs)

MIGRATION.md

@@ -6,7 +6,7 @@ Two things in one guide: what genuinely breaks moving from 0.x to 1.0.0, and the
 
 | 0.x pattern | 1.0 best practice |
 | --- | --- |
-| `aai.Lemur(...)` | Removed. Feed `transcript.text` to the LLM of your choice |
+| `aai.Lemur(...)` | Removed. Feed `transcript.text` to `aai.LLMGateway`, or to the LLM of your choice |
 | `from assemblyai.extras import MicrophoneStream` | Removed. Capture PCM yourself (`pyaudio`, `sounddevice`) and pass it to `stream(...)` |
 | `pip install "assemblyai[extras]"` | `pip install -U assemblyai` |
 | `StreamingClient`, `StreamingClientOptions` | `RealTimeTranscriber`, `RealTimeTranscriberOptions` |
@@ -23,25 +23,39 @@ Two things in one guide: what genuinely breaks moving from 0.x to 1.0.0, and the
 
 ### LeMUR support removed
 
-The LeMUR API and every `aai.Lemur*` name are gone from the SDK. There is no drop-in replacement in this package.
+The LeMUR API and every `aai.Lemur*` name are gone from the SDK. There is no drop-in replacement.
 
-A transcript is still just text, so the migration is to send it to whichever LLM you already use:
+A transcript is still just text, so the migration is to send it to an LLM yourself:
 
 ```python
-from assemblyai import TranscriptStatus
-from assemblyai.prerecorded.v2 import Transcriber
+import assemblyai as aai
 
-transcript = Transcriber(api_key="YOUR_API_KEY").transcribe(
+transcript = aai.Transcriber(api_key="YOUR_API_KEY").transcribe(
     "https://assembly.ai/wildfires.mp3"
 )
 
-if transcript.status == TranscriptStatus.error:
+if transcript.status == aai.TranscriptStatus.error:
     raise RuntimeError(transcript.error)
 
 prompt = f"Summarize this call transcript:\n\n{transcript.text}"
-# ...hand `prompt` to your LLM client of choice.
 ```
 
+`aai.LLMGateway` covers that in-package, across OpenAI/Claude/Gemini/Bedrock models — see [LLM Gateway Examples](README.md#llm-gateway-examples):
+
+```python
+gateway = aai.LLMGateway(api_key="YOUR_API_KEY")
+
+completion = gateway.chat.completions.create(
+    model="claude-sonnet-5",
+    messages=[{"role": "user", "content": prompt}],
+)
+print(completion.choices[0].message.content)
+```
+
+Speaker identification, translation, and custom formatting run off the transcript id instead, via `gateway.understanding.create(transcript_id=transcript.id, request={...})`.
+
+Or hand `prompt` to whichever LLM client you already use.
+
 ### Audio-capture extras removed
 
 `assemblyai.extras` (including `MicrophoneStream`) and the `[extras]` install option no longer exist. `pip install "assemblyai[extras]"` fails; use `pip install -U assemblyai`.