Agentic OS - Intelligence Distribution Grid

A production-grade Multi-LLM Reinforcement Learning Router that serves as the brain of an agentic infrastructure. Routes AI inference requests to optimal providers using Q-Learning, with circuit breaker safety, multi-tenant isolation, and a CRT-styled control dashboard.

Architecture Overview

  User Request
       |
       v
  +----+----+     +-------------+     +------------------+
  |  Chat   | --> |  Router     | --> | Provider Registry |
  |  API    |     |  Engine     |     | (Groq/OpenAI/AWS) |
  +---------+     +------+------+     +------------------+
                         |
            +------------+------------+
            |                         |
     +------v------+         +-------v--------+
     | Q-Table RL  |         | Circuit Breaker |
     | (Epsilon-   |         | (Factory Safety) |
     |  Greedy)    |         +----------------+
     +------+------+
            |
     +------v------+         +----------------+
     |  Telemetry  | ----->  | Redis Streams  |
     |  Fire/Forget|         | (Tenant Isolated|
     +-------------+         +--------+-------+
                                      |
                    +-----------------+-----------------+
                    |                                   |
            +-------v-------+                  +-------v--------+
            | Epoch Worker  |                  | Global Brain   |
            | (Mini-Batch   |                  | (Anti-Whale    |
            |  Q-Learning)  |                  |  Aggregation)  |
            +---------------+                  +----------------+

Quick Start

Prerequisites

• Node.js 18+
• API keys for LLM providers (Groq, OpenAI recommended)
• Upstash Redis, Turso, QStash accounts (free tiers available)

Installation

# Clone the repository
git clone <repository-url>
cd agentic-os

# Install dependencies
npm install

# Set up environment variables
cp .env.local.example .env.local
# Edit .env.local with your credentials

# Run development server
npm run dev

Environment Variables

Variable	Description	Required
UPSTASH_REDIS_REST_URL	Upstash Redis REST URL	Yes
UPSTASH_REDIS_REST_TOKEN	Upstash Redis token	Yes
TURSO_DATABASE_URL	Turso database URL	Yes
TURSO_AUTH_TOKEN	Turso auth token	Yes
QSTASH_TOKEN	QStash token	Yes
PINECONE_API_KEY	Pinecone API key	Yes
OPENAI_API_KEY	OpenAI API key	For OpenAI provider
GROQ_API_KEY	Groq API key	For Groq provider
AWS_ACCESS_KEY_ID	AWS access key	For Bedrock provider
NVIDIA_API_KEY	NVIDIA API key	For NIM provider

API Endpoints

Core Inference

Endpoint	Method	Description
/api/chat	POST	Main chat interface with memory
/api/router	POST	Low-level routing execution
/api/inference	POST	Raw model execution (debug)

Control Plane

Endpoint	Method	Description
/api/providers	GET/POST	Provider registry management
/api/circuit	GET/POST	Circuit breaker control
/api/metrics	GET	System metrics and performance
/api/telemetry	POST	Telemetry event ingestion
/api/health	GET	Health check

Background Workers

Endpoint	Method	Description
/api/cron/epoch	GET	Local Q-table learning epoch
/api/cron/global	GET	Global brain aggregation

Dashboard

Route	Description
/dashboard	CRT terminal control panel

Chat API Example

curl -X POST http://localhost:3000/api/chat \
  -H "Content-Type: application/json" \
  -d '{
    "message": "Explain quantum computing",
    "orgId": "default",
    "taskType": "reasoning",
    "complexity": 0.8
  }'

Router API Example

curl -X POST http://localhost:3000/api/router \
  -H "Content-Type: application/json" \
  -d '{
    "prompt": "Write a Python function to sort a list",
    "orgId": "default",
    "type": "code",
    "complexity": 0.6
  }'

Supported Providers

Provider	Models	Cost/1M	Status
Groq	Llama 3.1 8B/70B, Mixtral	$0.05/$0.08	Active
OpenAI	GPT-4o, GPT-4o-mini	$0.15/$0.60	Active
AWS Bedrock	Claude 3.5, Titan, Llama 3	$3.00/$15.00	Disabled
NVIDIA NIM	Llama 3.1 70B, Nemotron	$0.50/$0.50	Disabled

How It Works

1. Task Classification

The router analyzes incoming requests to determine:

• Task Type: reasoning, fast-tool, code, creative, analysis, debugging
• Complexity: 0.0-1.0 score based on prompt analysis
• User Priority: fast, cheap, quality, balanced

2. Provider Selection

Uses Epsilon-Greedy Q-Learning:

• With probability 1-epsilon (90%): Selects provider with highest Q-value
• With probability epsilon (10%): Random exploration for learning

3. Circuit Breaker Protection

Per-provider health monitoring:

• Tracks error rate and latency
• Opens circuit after 5 consecutive failures or 20% error rate
• 30-second cooldown, then half-open testing
• Closes after 3 successful test calls

4. Reward Computation

reward = quality * 1.0 - latency_norm * 0.3 - cost_norm * 0.2

Bounded to [-1.0, 1.0] with safeguards:

• Hard veto on failures
• Quality floor of 0.4
• Anomaly smoothing for perfect scores

5. Q-Table Updates

Q_new = Q_old + alpha * (reward - Q_old)

State-action key format: {task_type}|{complexity_bucket}|{provider_id}

6. Multi-Tenant Isolation

Each tenant (org) has:

• Isolated Redis Stream for telemetry
• Independent Q-table for learning
• Trust score for weighting contributions
• Rate limiting and cost controls

7. Global Brain Aggregation

Anti-whale protection ensures fair learning:

weight = min(5.0, log10(1 + volume)) * trust_score

Reinforcement Learning Flow

Request --> Task Profile --> Q-Table Lookup --> Provider Selection
                                                  |
Execution --> Telemetry --> Reward Governor --> Q-Table Update
                                                  |
                                    (Async) --> Global Aggregation

Technology Stack

Layer	Technology
Framework	Next.js 14 (App Router)
Language	TypeScript
Styling	Tailwind CSS
Short-Term Memory	Upstash Redis
Persistent Store	Turso (SQLite)
Long-Term Memory	Pinecone (1024d vectors)
Job Queue	QStash
Runtime	Vercel Edge

Project Structure

agentic-os/
├── src/
│   ├── app/
│   │   ├── api/
│   │   │   ├── chat/           # Chat inference endpoint
│   │   │   ├── router/         # Core routing engine
│   │   │   ├── inference/      # Raw model execution
│   │   │   ├── metrics/        # System metrics
│   │   │   ├── circuit/        # Circuit breaker control
│   │   │   ├── providers/      # Provider registry
│   │   │   ├── telemetry/      # Telemetry ingestion
│   │   │   ├── cron/
│   │   │   │   ├── epoch/      # Q-table learning worker
│   │   │   │   └── global/     # Global brain aggregation
│   │   │   └── health/         # Health check
│   │   ├── dashboard/          # CRT control panel UI
│   │   ├── layout.tsx          # Root layout
│   │   ├── page.tsx            # Root (redirects to dashboard)
│   │   └── globals.css         # Global styles
│   ├── lib/
│   │   ├── core/               # Core engine
│   │   │   ├── router.ts       # Main router
│   │   │   ├── circuitBreaker.ts
│   │   │   ├── meter.ts        # Metering system
│   │   │   ├── registry.ts     # Provider registry
│   │   │   ├── rewardGovernor.ts
│   │   │   ├── qtable.ts       # Q-table engine
│   │   │   └── trustEngine.ts  # Trust scoring
│   │   ├── db/                 # Database layer
│   │   │   ├── redis.ts        # Upstash Redis client
│   │   │   ├── turso.ts        # Turso SQLite client
│   │   │   ├── pinecone.ts     # Pinecone client (Node.js)
│   │   │   └── pinecone-edge.ts # Edge-compatible interface
│   │   ├── providers/          # Provider adapters
│   │   │   └── executor.ts     # Unified provider execution
│   │   ├── types/              # TypeScript types
│   │   ├── config/             # Configuration
│   │   └── utils/              # Utilities
│   └── ...
├── .env.local                  # Environment variables
├── next.config.mjs             # Next.js config
├── tailwind.config.ts          # Tailwind config
└── package.json

Development

Running Cron Workers Locally

# Epoch worker (Q-table learning)
npm run cron:epoch

# Global aggregation worker
npm run cron:global

Database Schema

The Turso database includes:

• providers - Provider configurations
• request_logs - Detailed request logs
• provider_metrics - Aggregated metrics
• circuit_history - Circuit breaker state changes
• q_table_snapshots - Q-table checkpoints
• trust_profiles - Tenant trust scores
• tenant_configs - Tenant settings

Contributing

1. Fork the repository
2. Create a feature branch
3. Make your changes
4. Add tests
5. Submit a pull request

License

MIT

---

Built for the agentic age. Route intelligently. Learn continuously. Scale infinitely.