Add Neuralwatt as a compatible provider - #3509
Open
404oops wants to merge 1 commit into
Open
Conversation
Contributor
There was a problem hiding this comment.
Pull request overview
This PR adds Neuralwatt as a new OpenAI-compatible backend provider for the ai-chat driver, including model catalog handling, metering/billing integration (cost + energy annotations), and automated tests to validate request shaping and usage accounting.
Changes:
- Added
NeuralwattProviderimplementation with model catalog + quota caching, vision-model gating, and OpenAI-compatible chat completions wiring. - Added Neuralwatt model catalog mapping utilities and comprehensive offline unit tests (including streaming + metering behavior).
- Registered the Neuralwatt provider in
ChatCompletionDriverand extended driver edge tests to cover provider registration.
Reviewed changes
Copilot reviewed 5 out of 5 changed files in this pull request and generated 1 comment.
Show a summary per file
| File | Description |
|---|---|
| src/backend/drivers/ai-chat/providers/neuralwatt/NeuralwattProvider.ts | Implements Neuralwatt chat provider with catalog/quota fetch, vision handling, and metering integration for cost/energy. |
| src/backend/drivers/ai-chat/providers/neuralwatt/NeuralwattProvider.test.ts | Adds unit tests covering catalog/quota caching, request shaping, streaming, and billing/metering behavior. |
| src/backend/drivers/ai-chat/providers/neuralwatt/models.ts | Defines Neuralwatt catalog types + mapping to IChatModel, plus helpers for vision detection and image-part detection. |
| src/backend/drivers/ai-chat/ChatCompletionDriver.ts | Registers Neuralwatt provider and updates model-map aggregator handling. |
| src/backend/drivers/ai-chat/ChatCompletionDriver.edges.test.ts | Adds Neuralwatt to edge-test provider config and stubs its model catalog for model-map building. |
馃挕 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment on lines
+1022
to
+1027
| const AGGREGATORS = new Set([ | ||
| 'together-ai', | ||
| 'openrouter', | ||
| 'infron', | ||
| 'neuralwatt', | ||
| ]); |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Hello everyone! I love Puter and all other projects built with it. With that being said, I feel like self-hosted Puter instances could use
Neuralwatt is an open-source-exclusive LLM provider (with options to host your own infrastructure, more on that later) whose main premise is charging for how much power you use, with the option to switch to token-based pricing if you don't like power-based pricing.
Now for the technical part
Its API is OpenAI-compatible, however it includes an energy field which is utilized in the billing section. I cross-referenced what the console total said to the Neuralwatt dashboard's total and found the results to be accurately represented.
I wrote a unit test to verify if everything is as it should be, according to other unit tests (particularly the OpenRouter one), and I verified that all 17 of them pass.
I wrote the provider backend, which wasn't really complicated, but I followed a convention that fetches at "boot", since Neuralwatt doesn't change very often. I do feel like that behavior should be discussed with y'all, so if you have any objections to that, do let me know.
I also wired it into the provider driver which is necessary for it to even work.
I tested it with my own keys (inside the console, unfortunately), and it works fine. However, due to the nature of Puter's dev environment, I can't test it in apps, so I'm quite unsure on how I can test it with local apps and how models are handled. But it does work.
If you have any questions, or guidance on how I can improve this feature, please let me know. I'm still learning the codebase and how it works, and I think I got a hang of this part.