VisualLLM

Build reliable OpenAI-compatible endpoints by arranging AI models visually.

Routing, made visible. The vault on the left holds the models you've kept; the endpoints on the right are live OpenAI-compatible servers. The relic at the right edge of each lane answers first — everything to its left is a fallback, tried in order.

What is VisualLLM?

VisualLLM is a visual fallback router for AI models. It's a desktop application that lets you:

Add your AI providers (OpenRouter, OpenAI, Anthropic, or any OpenAI-compatible endpoint)
Browse their model catalogs with rich filtering and sorting
Drag models from the vault (left sidebar) into any lane — each lane is an ordered fallback chain
Reorder models within a lane by dragging — the rightmost model answers first; everything to its left is a fallback
Expose lanes as local OpenAI-compatible endpoints that your tools can connect to
Integrate a lane into your editor (VS Code, VS Code Insiders, Windsurf) with one click — it shows up in the editor's own model picker
Self-update from GitHub — the app checks for a new signed release at startup and offers to restart when one is ready

When a request comes in, the rightmost model answers first. If it fails, can't serve the request, or returns unusable content, VisualLLM automatically tries the next model in line — and explains exactly what happened at each step.

A Look Inside

The Gallery. Every model your providers offer, with intelligence, coding, and agentic scores, context size, price, and measured speed. Lock the columns you care about and the list sorts by them.	Providers. OpenRouter, OpenAI, Anthropic, Groq, Together, DeepSeek, xAI, Mistral, or any OpenAI-compatible endpoint — including local servers like Ollama, LM Studio, and vLLM.
Adding a provider. Keys are stored in the OS keychain and never leave the Rust backend. Export your setup to move it between machines — keys stay behind.

The interface is an installation: a living, reaction-diffusion field rendered on the GPU (with an honest CPU fallback), slabs of smoked acrylic floating above it, and a lane's traffic rippling through the chemistry at its own position. Nothing is a static rectangle.

How It Works: A GUI for an LLM Proxy

VisualLLM is a desktop GUI that configures and runs a local LLM proxy. Under the hood:

The Rust backend runs a lightweight HTTP server on 127.0.0.1:4100 — this is the proxy.
Each lane you create becomes a distinct OpenAI-compatible endpoint at /lane/<slug>/v1.
The GUI lets you add providers, browse their catalogs, and drag models into lanes — no YAML, no code.
At request time, the proxy walks the lane right-to-left, trying each model until one succeeds (capability check → request → commit gate → forward).
Every hop is recorded — you get headers (x-visualllm-served-by, x-visualllm-passed-over, x-visualllm-trail) and a full incident log.

Think of it as: a visual control plane for a multi-provider LLM proxy. You configure the routing visually; the proxy executes it faithfully.

The Problem It Solves

Most AI gateway solutions make routing a configuration problem — editing YAML files, managing complex rules, or writing code. VisualLLM makes it visible and intuitive: you see your models, you arrange them in order, and the system does exactly what you expect.

This is especially valuable when:

You want automatic fallback when a provider is rate-limited, out of credit, or unavailable
You need reliable responses and want to understand exactly which model served each request
You use multiple providers and want to route requests based on capability, not just availability
You want transparency — knowing not just that a request succeeded, but which model answered and why

Why VisualLLM?

🎯 Visual First

Everything is arranged visually. No configuration files to edit, no YAML to learn. What you see is what you get.

🔒 Secure by Design

API keys never leave the Rust backend — the webview has no network or filesystem access
Keys are stored in the OS keychain (Linux native), not in plaintext files
All network requests flow through Rust — the renderer only receives state, never credentials
Bound to loopback (127.0.0.1) — your endpoints are local-only by default

📊 Intelligent Routing

Capability checking — models that can't serve your request are skipped, not tried
Commit gate — 200 responses are verified to contain usable content before forwarding
Loop detection — optional Loopwatch catches agents stuck in tool-call loops
Detailed receipts — every failure is recorded with evidence for debugging

🔄 Honest Fallback

A response isn't successful just because a provider returned HTTP 200. VisualLLM verifies that:

The response contains actual content (not just reasoning tokens)
The model can actually serve the request (vision, tools, context size)
The stream hasn't stalled or died mid-response

Every response includes headers telling you:

x-visualllm-served-by — which model actually answered
x-visualllm-passed-over — how many models were skipped or failed
x-visualllm-trail — the complete story of what happened

💡 Built for Developers

OpenAI-compatible — works with VS Code, Cursor, and any OpenAI-compatible client
No build step — the renderer is plain HTML/CSS/JS
Rust backend — fast, safe, and reliable
Tauri framework — lightweight, secure desktop app

Quick Start

For Users (Pre-built)

Ready-to-run installers are published to GitHub Releases:

.deb for Debian/Ubuntu
AppImage for any Linux distribution
.msi / .exe for Windows
.dmg for macOS

VisualLLM updates itself. A few seconds after launch it checks GitHub for a newer release, downloads it in the background, and offers to restart — every download is signature-verified against the key embedded in the app. No manual reinstall needed.

🧪 Calling All Testers!

We're actively looking for testers on all platforms! VisualLLM is Linux-first, but we want to ensure it works well on macOS and Windows too. If you can help test:

macOS — Test the .dmg installer, notarization, and code signing
Windows — Test the .msi and .exe installers, code signing, and WebView2 integration
Linux — Test .deb, AppImage, and Flatpak on various distributions

How to help:

Download the latest release artifacts from GitHub Releases
Install and run VisualLLM on your platform
Report any issues at GitHub Issues with:
- Your OS and version
- The artifact you tested (.dmg, .msi, .exe, .deb, AppImage, Flatpak)
- Steps to reproduce any problems
- Screenshots if applicable

Your feedback directly shapes the quality of the first public release!

For Developers (From Source)

Prerequisites

Linux (Ubuntu/Debian):

# System dependencies for WebKit and Tauri
sudo apt update && sudo apt install -y \
  libwebkit2gtk-4.1-dev libxdo-dev libayatana-appindicator3-dev librsvg2-dev \
  build-essential curl wget file libssl-dev

# Rust toolchain
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
source "$HOME/.cargo/env"

# Node.js (for build tooling)
nvm install --lts  # or: sudo apt install nodejs npm

Other distributions: See detailed instructions below.

Build and Run

# Clone the repository
cd /path/to/visualllm

# Install dependencies
npm ci

# Verify everything works
node tools/smoke.js
cargo test --manifest-path src-tauri/Cargo.toml

# Run the app in development mode
npm run dev

# Build release packages (.deb + AppImage on Linux)
npm run build

# Or run the compiled binary outside a snap-polluted terminal
# (prefers target/release if it exists, otherwise target/debug)
tools/launch-system.sh

The app will open a window and start the engine on http://127.0.0.1:4100.

Using VisualLLM

1. Add a Provider

Click Providers in the sidebar, then Add Provider. Enter:

Name — a friendly name for this provider
Kind — openrouter, openai, anthropic, or generic
Base URL — the API endpoint (defaults based on kind)
API Key — your secret key (stored securely in the OS keychain)

2. Browse Models

Click Browse to see all models from your configured providers. Use the filters and sorting options to find what you need:

Filter by capability (vision, tools, reasoning)
Sort by intelligence, coding, agentic scores (OpenRouter)
Filter by author, context size, price

3. Build a Lane

Click Pool to select models you're interested in. Then:

Click New Lane
Drag models from the pool into your lane (right to left = primary to fallback)
Give it a name and save

The lane is now a local OpenAI-compatible endpoint at:

http://127.0.0.1:4100/lane/<lane-slug>/v1

4. Connect Your Client

VS Code / Cursor / Any OpenAI-compatible client:

Base URL: http://127.0.0.1:4100/lane/<lane-slug>/v1
Model: <lane-slug> (the same as your lane name)
API Key: Leave blank (or use any placeholder if required)

Direct cURL request:

curl http://127.0.0.1:4100/lane/my-lane/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "my-lane",
    "messages": [{"role": "user", "content": "Hello"}],
    "max_tokens": 50
  }'

5. Integrate a Lane into Your Editor

Instead of wiring endpoints by hand, add a lane straight into an editor's model picker with one click. On each lane, open the editor picker and choose a target:

VS Code — the lane appears as a customendpoint model you can select in Copilot Chat / the model picker
VS Code Insiders — same integration for the Insiders build
Windsurf — same VS Code schema, Windsurf config directory

VisualLLM writes the lane into the editor's chatLanguageModels.json, derives its capabilities (vision, tools, context window) from the lane's actual members, and removes it automatically when the lane is deleted. Re-running an integration updates the existing entry rather than duplicating it.

Advanced Features

Per-Member Settings

Each model in a lane can have its own parameters:

Temperature, top-p, frequency/presence/repetition penalties
Max tokens limit
These override the client's request for that specific model

Reasoning Suppression

Enable "No thinking" on a lane to ask providers (that support it) to skip reasoning tokens. This is a preference, not a guarantee — the commit gate catches models that think anyway.

Loopwatch

Enable "Unstick" on a lane to detect and break tool-call loops:

Repeat loops: Same tool called with same arguments multiple times
Futile loops: Different arguments returning identical results
The loop is collapsed and a diagnostic note is appended to the conversation

Incident Records

Every failure is recorded with:

Timestamp and which lane/member failed
The failure kind (rate_limited, out_of_credit, capability_gap, etc.)
The evidence (provider error messages, trail notes)
Lane settings at the time (no_think, loopwatch enabled)

Click the bell icon in the status bar to view recent incidents.

Export and Import

Your lanes, pool, and provider configuration can move between machines:

Export — open the Providers panel and click Export…. This saves a JSON file containing lanes, pool, and provider settings. API keys are never included; they stay in your OS keychain.
Import — click Import… and choose a previously exported file.
- Merge — combines the file with your current state. Existing providers keep their local API keys, and lanes with the same slug are updated rather than duplicated.
- Replace — wipes your current lanes, pool, and providers and loads the file exactly. You will need to re-enter API keys afterward.

Architecture

Security Model

VisualLLM follows a strict security architecture:

┌─────────────────────────────────────────────────────────┐
│                    TAURI APPLICATION                        │
├─────────────────────────────────────────────────────────┤
│  ┌─────────────────┐      ┌─────────────────────────────┐ │
│  │   WEBVIEW       │      │         RUST BACKEND          │ │
│  │  (Renderer)      │      │  (src-tauri/src/)             │ │
│  │                 │      │                             │ │
│  │  • HTML/CSS/JS   │      │  • Provider key storage      │ │
│  │  • No network    │◄─────┤  • Catalog fetching          │ │
│  │  • No filesystem │      │  • Request routing           │ │
│  │  • Tauri commands│      │  • Fallback logic            │ │
│  │                 │      │  • Loop detection             │ │
│  │                 │      │  • Incident recording         │ │
│  └─────────────────┘      └─────────────────────────────┘ │
└─────────────────────────────────────────────────────────┘
                              │
                              ▼
                    ┌─────────────────┐
                    │  OS Keychain     │  ← API keys stored here
                    └─────────────────┘

The webview can only:

Render the UI
Call Tauri commands defined in src-tauri/src/main.rs
Receive state from the Rust backend

The webview CANNOT:

Make HTTP requests
Access the filesystem
Read API keys
Execute arbitrary code

File Structure

visualllm/
├── renderer/               # Frontend (HTML/CSS/JS)
│   ├── index.html          # Main window
│   ├── style.css           # Styles
│   └── app.js              # All UI logic
├── src-tauri/               # Backend (Rust)
│   ├── src/
│   │   ├── main.rs         # Tauri shell, commands
│   │   ├── server.rs       # HTTP engine, routing
│   │   ├── providers.rs    # Provider management, catalogs
│   │   ├── lanes.rs        # Lane storage and management
│   │   ├── incidents.rs    # Failure recording
│   │   └── loopwatch.rs    # Loop detection
│   ├── Cargo.toml          # Rust dependencies
│   └── tauri.conf.json     # Tauri configuration
├── tools/                  # Development scripts
│   ├── smoke.js            # Renderer smoke test
│   └── preview.js          # Browser preview harness
├── README.md               # This file
├── ROADMAP.md              # Public release plan
├── CONTRIBUTING.md         # Development guidelines
├── SECURITY.md             # Security policy
└── LICENSE                 # MIT License

How a Request Flows

Client Request
     │
     ▼
┌─────────────────────┐
│  Engine (server.rs)  │  ← Listens on 127.0.0.1:4100
└─────────────────────┘
     │
     ▼
┌─────────────────────┐
│  Find Lane           │  ← Looks up lane by slug
└─────────────────────┘
     │
     ▼
┌─────────────────────┐
│  Inspect Request     │  ← Checks vision, tools, token needs
└─────────────────────┘
     │
     ▼
┌─────────────────────┐
│  Walk Members        │  ← Tries each model in order
└─────────────────────┘
     │
     ├─ Can this model serve? (capability check)
     │
     ▼
┌─────────────────────┐
│  Send to Provider    │  ← With member-specific params
└─────────────────────┘
     │
     ▼
┌─────────────────────┐
│  Verify Response     │  ← Commit gate: has usable content?
└─────────────────────┘
     │
     ├─ If usable: Forward to client with headers
     │
     ▼
┌─────────────────────┐
│  Try Next Member     │  ← If failed or unusable
└─────────────────────┘
     │
     ▼
┌─────────────────────┐
│  Return Error        │  ← If all members failed
└─────────────────────┘

Detailed Installation Instructions

Linux

Ubuntu / Debian

# Install system dependencies
sudo apt update && sudo apt install -y \
  libwebkit2gtk-4.1-dev libxdo-dev libayatana-appindicator3-dev librsvg2-dev \
  build-essential curl wget file libssl-dev libgtk-3-dev

# Install Rust
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
source "$HOME/.cargo/env"
rustup default stable

# Install Node.js (via nvm recommended)
curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.39.7/install.sh | bash
source ~/.bashrc
nvm install --lts

Fedora

sudo dnf install -y \
  webkit2gtk4.1-devel libxdo-devel libayatana-appindicator-gtk3-devel librsvg2-devel \
  gcc gcc-c++ make openssl-devel gtk3-devel

# Then install Rust and Node.js as above

Arch Linux

sudo pacman -S \
  webkit2gtk-4.1 libxdo libayatana-appindicator librsvg base-devel openssl gtk3

# Then install Rust and Node.js as above

Running the App

For reliable launching on Linux (especially under Wayland or in snap environments):

# From the repo root
export PATH="$HOME/.cargo/bin:$PATH"

env -i \
  HOME="$HOME" \
  PATH="/usr/bin:/bin:$HOME/.cargo/bin" \
  DISPLAY="${DISPLAY:-:0}" \
  XAUTHORITY="${XAUTHORITY:-$HOME/.Xauthority}" \
  ./src-tauri/target/debug/visualllm

Why this works:

env -i starts with a clean environment (no snap library conflicts)
Explicitly sets the paths needed for GTK/WebKit
Works with both X11 and Wayland

Troubleshooting

Issue	Solution
`symbol lookup error: libpthread`	You're inheriting snap's `LD_LIBRARY_PATH`. Use the `env -i` launch command above.
`Failed to initialize GTK`	Missing `DISPLAY` or `XAUTHORITY`. Ensure you're on a graphical session.
`Address already in use (port 4100)`	A previous instance is still running: `pkill -f visualllm && fuser -k 4100/tcp`
Window opens but shows "Could not connect"	The engine didn't start. Check terminal for errors, usually a stale process on 4100.
Transparent window shows black/garbled	Your compositor doesn't support ARGB visuals. Set `"transparent": false` in `src-tauri/tauri.conf.json` and rebuild.

Development

Running Tests

# Rust tests (engine logic)
cargo test --manifest-path src-tauri/Cargo.toml

# Renderer smoke test
node tools/smoke.js

# Full build
npm run build

Project Structure for Contributors

renderer/app.js — All UI logic, no framework
src-tauri/src/main.rs — Tauri shell and command definitions
src-tauri/src/server.rs — HTTP engine and routing logic
src-tauri/src/providers.rs — Provider management and catalog fetching
src-tauri/src/lanes.rs — Lane storage and member management
src-tauri/src/incidents.rs — Failure recording and classification
src-tauri/src/loopwatch.rs — Tool-call loop detection

Adding a New Feature

For UI changes: Edit files in renderer/
For backend logic: Edit files in src-tauri/src/
For new Tauri commands: Add to main.rs with #[tauri::command]
For new HTTP routes: Add to server.rs router

All new commands must be added to the capabilities list in src-tauri/capabilities/default.json.

Philosophy

Design Principles

Visibility — You should be able to understand routing decisions at a glance
Honesty — A response is only successful if it contains usable content
Safety — API keys never leave the Rust backend
Simplicity — The simple path should stay simple
Compatibility — Existing lanes and clients should keep working

Why These Choices?

Why Tauri? Lightweight, secure, and Rust-based. It gives us a real desktop app without the overhead of Electron.

Why no framework in the renderer? The UI is small and focused. Vanilla JS keeps it fast, auditable, and dependency-free.

Why Rust for the backend? Memory safety, performance, and excellent HTTP/async support. Plus, it's where the keys live.

Why loopback-only by default? Security. Your API keys are valuable, and we don't want to accidentally expose them to your network.

Contributing

We welcome contributions! Please see CONTRIBUTING.md for:

Development setup
Pull request guidelines
Code style expectations
Testing requirements

Security

Please see SECURITY.md for:

Our threat model
Vulnerability reporting process
Security design decisions

License

VisualLLM is licensed under the MIT License.

Roadmap

The public release plan, milestones, and criteria are documented in ROADMAP.md.

Release checklist

Before tagging a release:

Run the full test suite:

node tools/smoke.js
cargo test --manifest-path src-tauri/Cargo.toml
cargo fmt --check --manifest-path src-tauri/Cargo.toml

Build release packages:
```
npm run build
```
Verify the built binary starts cleanly and serves lanes on 127.0.0.1:4100.
Launch a second instance and confirm it detects the first rather than spawning a duplicate engine window.
Run the AppImage on a clean Linux VM with no WebKitGTK development packages installed to confirm it bundles its dependencies.
Update CHANGELOG.md with the version, date, and summary of changes.
Bump the version to match in src-tauri/Cargo.toml, src-tauri/tauri.conf.json, and package.json — the CI release pipeline refuses a tag whose version doesn't match the source.
Tag the release (git tag vX.Y.Z && git push origin vX.Y.Z). The pipeline builds and signs the .deb, AppImage, .msi, .exe, and .dmg, uploads their .sig files and a fresh latest.json, and publishes the GitHub Release — which is also what the in-app auto-updater checks against.

Version policy

Releases follow Semantic Versioning:

MAJOR — incompatible changes to lane persistence, provider config, or the OpenAI-compatible endpoint contract.
MINOR — new user-facing features, new provider kinds, or significant UI improvements.
PATCH — bug fixes, documentation updates, and minor robustness improvements.

The first public release will be 1.0.0 once the release criteria in ROADMAP.md are met.

Credits

VisualLLM is built by Creative Systems Development.

Built with ❤️ using Rust, Tauri, and a commitment to making AI routing visible and reliable.

Name		Name	Last commit message	Last commit date
Latest commit History 112 Commits
.continue/agents		.continue/agents
.github		.github
docs/screenshots		docs/screenshots
renderer-backup-original		renderer-backup-original
renderer		renderer
src-tauri		src-tauri
tools		tools
.gitignore		.gitignore
CODE_OF_CONDUCT.md		CODE_OF_CONDUCT.md
CONTRIBUTING.md		CONTRIBUTING.md
DISTRIBUTION_SETUP.md		DISTRIBUTION_SETUP.md
LICENSE		LICENSE
README.md		README.md
RELEASE_ROADMAP.md		RELEASE_ROADMAP.md
ROADMAP.md		ROADMAP.md
SECURITY.md		SECURITY.md
git		git
handoff.md		handoff.md
neumorphic.md		neumorphic.md
package-lock.json		package-lock.json
package.json		package.json
run.sh		run.sh

Folders and files

Latest commit

History

Repository files navigation

VisualLLM

What is VisualLLM?

A Look Inside

How It Works: A GUI for an LLM Proxy

The Problem It Solves

Why VisualLLM?

🎯 Visual First

🔒 Secure by Design

📊 Intelligent Routing

🔄 Honest Fallback

💡 Built for Developers

Quick Start

For Users (Pre-built)

🧪 Calling All Testers!

For Developers (From Source)

Prerequisites

Build and Run

Using VisualLLM

1. Add a Provider

2. Browse Models

3. Build a Lane

4. Connect Your Client

5. Integrate a Lane into Your Editor

Advanced Features

Per-Member Settings

Reasoning Suppression

Loopwatch

Incident Records

Export and Import

Architecture

Security Model

File Structure

How a Request Flows

Detailed Installation Instructions

Linux

Ubuntu / Debian

Fedora

Arch Linux

Running the App

Troubleshooting

Development

Running Tests

Project Structure for Contributors

Adding a New Feature

Philosophy

Design Principles

Why These Choices?

Contributing

Security

License

Roadmap

Release checklist

Version policy

Credits

About

Topics

Resources

Code of conduct

Contributing

Security policy

Stars

Watchers

Forks

Releases

Packages

Contributors

Languages