feat: add openinfer provider with model mappings - #6
Conversation
Adds OpenInfer (platform.openinfer.io) as an inference provider with mappings for Qwen 3.5 9B and Llama 3.2 1B Instruct, including new registry model entries for both. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
Hi, I ahve added this PR to integrate OpenInfer's OICloud as an infernece provider service. I have the API key generated, but wasn't sure where to add or how to share. Please help with that. |
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
Hi, @madeburo, could we get an update on merging this PR? |
|
Hi @onkarbpatil Before we can merge, please address the following:
Please do not share the API key in this PR or commit it to the repository. API credentials can be handled separately through a private channel or repository secrets when telemetry integration is configured. |
Update the OpenInfer mappings with published rate-card pricing and extend coverage to two additional served models. Pricing (USD per 1M tokens, input/output): Qwen3.5 9B 0.15 / 0.18 Qwen3.5 27B 0.72 / 0.72 Llama3.2 1B 0.02 / 0.02 Gemma4 31B 0.52 / 0.75 Adds a canonical entry for Qwen 3.5 27B; Gemma 4 31B already exists upstream, so it needs only a mapping. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Set context_window_override to the context length each model is actually served at on OpenInfer nodes: Qwen3.5 27B 32768 Llama3.2 1B 16384 Gemma4 31B 32768 Qwen3.5 9B is served at its full native 262144 window, so its override stays null. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
Hi @madeburo I have added the pricing and context size. You can get the public API to access them here "https://console.openinfer.ai/usage" |
Gemma4 31B is served at 8192 tokens on OpenInfer nodes, not 32768. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Llama3.2 1B is served at 24000 tokens, not 16384. Qwen3.5 9B is served at 65536 rather than its full native 262144 window, so it now carries an explicit override instead of null. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2 participants
Summary
Adds OpenInfer (openinfer.io) as an inference provider, with mappings for the two models it currently serves. Both models are new to the registry, so their canonical model entries are included.
Disclosure: I work at OpenInfer, so provider details (model catalog, rate limits) come from first-hand knowledge of the service.
Files added
providers/openinfer.yaml— OpenAI-compatible API athttps://api.openinfer.ai/v1, bearer auth, global routing across distributed CPU/GPU/NPU computemodels/qwen3-5-9b.yaml— Qwen 3.5 9B (Alibaba, Apache 2.0, 262K context, multimodal)models/llama-3-2-1b.yaml— Llama 3.2 1B Instruct (Meta, 128K context)mappings/openinfer/qwen3-5-9b.yaml— served as@oi/Qwen3.5-9Bmappings/openinfer/llama-3-2-1b.yaml— served as@oi/Llama-3.2-1B-InstructPricing note
OpenInfer Cloud is free through August 31, 2026 (see openinfer.io/cloud), so mappings list $0.00 input/output. We will submit an update with post-promo rates when the free period ends.
Validation
python validate_registry.pypasses — 403 files validated.🤖 Generated with Claude Code