Agentic OS Intelligence Grid

A self-optimizing, multi-tenant AI model routing system that uses reinforcement learning to intelligently distribute inference requests across multiple LLM providers (Groq, OpenAI, AWS Bedrock, NVIDIA NIM). The system continuously learns which provider performs best for each task type, automatically handling failures, optimizing cost, and maintaining quality through a federated feedback loop.

Architecture Overview

                          ┌─────────────────────────────────────────────┐
                          │              OPERATOR DASHBOARD             │
                          │     (Grid Status · Providers · Q-Table)     │
                          └──────────────────┬──────────────────────────┘
                                             │ REST API
                          ┌──────────────────▼──────────────────────────┐
                          │              HTTP API LAYER                  │
                          │    /api/router · /api/providers · /api/*    │
                          └──────────────────┬──────────────────────────┘
                                             │
                          ┌──────────────────▼──────────────────────────┐
                          │            CORE ENGINE LAYER                │
                          │                                             │
                          │  ┌──────────────┐  ┌───────────────────┐   │
                          │  │    ROUTER     │  │ CIRCUIT BREAKER   │   │
                          │  │  Q-Table +    │  │ Failure Safety    │   │
                          │  │  ε-Greedy     │  │ System            │   │
                          │  └──────┬───────┘  └───────┬───────────┘   │
                          │         │                   │               │
                          │  ┌──────▼───────┐  ┌───────▼───────────┐   │
                          │  │   REGISTRY   │  │  REWARD GOVERNOR  │   │
                          │  │  Provider    │  │  Anti-Hacking     │   │
                          │  │  Configs     │  │  Sanitizer        │   │
                          │  └──────┬───────┘  └───────────────────┘   │
                          │         │                                   │
                          │  ┌──────▼───────┐  ┌───────────────────┐   │
                          │  │   METER      │  │  TRUST ENGINE     │   │
                          │  │  Usage       │  │  Tenant Cred.     │   │
                          │  │  Tracking    │  │  Scoring          │   │
                          │  └──────────────┘  └───────────────────┘   │
                          └──────────────────┬──────────────────────────┘
                                             │
              ┌──────────────────────────────┼──────────────────────────────┐
              │                              │                              │
   ┌──────────▼──────────┐     ┌────────────▼──────────┐    ┌─────────────▼──────────┐
   │   PROVIDER LAYER    │     │   MEMORY LAYER        │    │   RL ENGINE LAYER       │
   │                     │     │                        │    │                         │
   │  ┌───────┐ ┌──────┐│     │  ┌──────┐  ┌────────┐ │    │  ┌───────┐ ┌──────────┐│
   │  │ Groq  │ │OpenAI││     │  │ STM  │  │  LTM   │ │    │  │Q-Table│ │Telemetry ││
   │  └───┬───┘ └──┬───┘│     │  │Redis │  │Pinecone│ │    │  │Redis  │ │Streams   ││
   │  ┌───┴───┐ ┌──┴───┐│     │  │List  │  │Vector  │ │    │  │Hash   │ │XADD     ││
   │  │  AWS  │ │NVIDIA││     │  │      │  │Search  │ │    │  └───┬───┘ └────┬─────┘│
   │  │Bedrock│ │ NIM  ││     │  └──────┘  └────────┘ │    │      │         │       │
   │  └───────┘ └──────┘│     │                        │    │  ┌───▼─────────▼───┐ │
   └─────────────────────┘     └────────────────────────┘    │  │  RL WORKER      │ │
                                                            │  │  (Mini-Service) │ │
                                                            │  │  Port 3003       │ │
                                                            │  └─────────────────┘ │
              ┌─────────────────────────────────────────────┴──────────────────────────┐
              │                        DATA LAYER                                       │
              │  ┌──────────┐  ┌──────────┐  ┌──────────┐  ┌────────────────────────┐ │
              │  │  Redis   │  │  Turso   │  │ Pinecone │  │  Federated Global      │ │
              │  │ (Upstash)│  │ (libSQL) │  │ (Vector) │  │  Q-Table Prior         │ │
              │  └──────────┘  └──────────┘  └──────────┘  └────────────────────────┘ │
              └───────────────────────────────────────────────────────────────────────┘

Key Features

RL-Powered Routing — Q-learning agent automatically learns optimal provider selection per task type (reasoning, code, fast-tool, analysis, chat) using reward signals from latency, cost, and quality
Epsilon-Greedy Exploration — 10% of requests are randomly routed to maintain coverage and discover new optimal paths
Multi-Tenant Micro-Grids — Each organization gets isolated Q-tables, telemetry streams, and trust scores; federated learning aggregates learnings into a global prior
Circuit Breaker Safety — Automatic provider isolation on 5 consecutive failures, >20% error rate, or >3s avg latency with half-open recovery probing
Reward Governor — Anti-hacking safeguards: quality floor veto at 0.4, reward clipping to [-1, 1], anomaly smoothing for suspiciously perfect scores
Trust Scoring — 4-tier tenant credibility system (gold/standard/variable/risk) with weighted formula: 0.35*stability + 0.25*snr + 0.25*alignment + 0.15*(1-exploration)
Dual Memory System — Short-Term Memory (STM) via Redis sliding window for active sessions, Long-Term Memory (LTM) via Pinecone vector search for persistent knowledge
Automatic Fallback Chains — Transparent failover across all enabled providers when primary selection fails
Real-Time Dashboard — Full operational visibility: provider fleet status, circuit breaker states, Q-table visualization, metrics, trust profiles
Federated Learning with Anti-Whale — Global Q-table updated with log-scaled volume dampening to prevent high-volume tenants from dominating the shared policy

Tech Stack

Layer	Technology
Framework	Next.js 16 (App Router, TypeScript)
Frontend	React 19, Tailwind CSS 4, shadcn/ui, Framer Motion, Lucide Icons
State Management	Zustand (client), TanStack Query (server)
Hot State / Q-Tables	Upstash Redis (REST-based)
Persistent Storage	Turso (libSQL / SQLite)
Vector DB / LTM	Pinecone (1024-dim, Cohere v4 embeddings)
RL Processing	Standalone Bun mini-service (port 3003)
LLM Providers	Groq, OpenAI, AWS Bedrock (Claude), NVIDIA NIM

Quick Start

1. Install Dependencies

bun install

2. Set Environment Variables

Copy the following into your .env.local file (see the full table below):

# Redis (Upstash)
UPSTASH_REDIS_REST_URL=https://your-redis.upstash.io
UPSTASH_REDIS_REST_TOKEN=your-token

# Turso (libSQL)
TURSO_DATABASE_URL=libsql://your-db.turso.io
TURSO_AUTH_TOKEN=your-token

# Pinecone (Vector DB)
PINECONE_API_KEY=your-key
PINECONE_INDEX_NAME=agentic-os-memory

# LLM Provider API Keys
GROQ_API_KEY=your-key
OPENAI_API_KEY=your-key
AWS_BEDROCK_ACCESS_KEY=your-key
AWS_BEDROCK_SECRET_KEY=your-key
AWS_BEDROCK_REGION=us-east-1
NVIDIA_API_KEY=your-key

3. Initialize the Grid

# Start the dev server
bun run dev

# Initialize database schema and registry
curl http://localhost:3000/api/init

4. Start the RL Worker

cd mini-services/rl-worker
bun install
bun run dev  # Runs on port 3003

5. Send Your First Request

curl -X POST http://localhost:3000/api/router \
  -H "Content-Type: application/json" \
  -d '{"task": "Write a Python function to sort a list"}'

Environment Variables

Variable	Required	Default	Description
`UPSTASH_REDIS_REST_URL`	Yes	—	Upstash Redis REST endpoint URL
`UPSTASH_REDIS_REST_TOKEN`	Yes	—	Upstash Redis REST auth token
`TURSO_DATABASE_URL`	Yes	—	Turso/libSQL database URL
`TURSO_AUTH_TOKEN`	Yes	—	Turso auth token
`PINECONE_API_KEY`	Yes	—	Pinecone API key for LTM
`PINECONE_INDEX_NAME`	No	`agentic-os-memory`	Pinecone index name
`PINECONE_DIMENSION`	No	`1024`	Embedding dimension
`GROQ_API_KEY`	Yes	—	Groq API key
`OPENAI_API_KEY`	Yes	—	OpenAI API key
`AWS_BEDROCK_ACCESS_KEY`	Yes	—	AWS Bedrock access key
`AWS_BEDROCK_SECRET_KEY`	Yes	—	AWS Bedrock secret key
`AWS_BEDROCK_REGION`	No	`us-east-1`	AWS region for Bedrock
`NVIDIA_API_KEY`	No	—	NVIDIA NIM API key
`GRID_POLICY_VERSION`	No	`v1.1`	RL policy version tag
`GRID_LEARNING_RATE`	No	`0.1`	Q-learning rate (alpha)
`GRID_EXPLORATION_RATE`	No	`0.1`	Epsilon-greedy exploration rate
`GRID_REWARD_QUALITY_FLOOR`	No	`0.4`	Minimum quality for positive reward
`GRID_META_LEARNING_RATE`	No	`0.05`	Federated global update rate
`GRID_TRUST_INITIAL_SCORE`	No	`0.3`	Default trust score for new tenants
`GRID_CIRCUIT_FAILURE_THRESHOLD`	No	`5`	Consecutive failures to OPEN circuit
`GRID_CIRCUIT_ERROR_RATE_THRESHOLD`	No	`0.2`	Error rate threshold for degradation
`GRID_CIRCUIT_LATENCY_THRESHOLD`	No	`3000`	Latency threshold (ms) for degradation

API Endpoints

Inference

Method	Endpoint	Description
`POST`	`/api/router`	Route a task to the optimal provider

POST /api/router — Body:

{
  "task": "Your prompt here",
  "context": "Optional context",
  "taskType": "reasoning | code | fast-tool | analysis | chat",
  "userPriority": "fast | cheap | quality",
  "orgId": "tenant-id",
  "environment": "prod | dev",
  "sessionId": "session-id",
  "systemPrompt": "System instructions",
  "temperature": 0.7,
  "maxTokens": 4096,
  "preferProvider": "groq | openai | aws | nvidia"
}

Provider Management

Method	Endpoint	Description
`GET`	`/api/providers`	List all providers with stats
`POST`	`/api/providers`	Toggle or update a provider

Circuit Breaker

Method	Endpoint	Description
`GET`	`/api/circuit`	Get all circuit breaker states
`POST`	`/api/circuit`	Manually open/close/reset a circuit

Observability

Method	Endpoint	Description
`GET`	`/api/metrics?orgId=&period=`	Aggregated usage metrics
`GET`	`/api/qtable?orgId=&environment=`	View Q-table values
`DELETE`	`/api/qtable?orgId=&environment=`	Clear tenant Q-table
`GET`	`/api/trust?orgId=`	Tenant trust profiles
`GET`	`/api/health`	System health + DB connectivity

Memory

Method	Endpoint	Description
`GET`	`/api/memory?type=stm&sessionId=`	Read STM messages
`POST`	`/api/memory?type=stm`	Store STM message
`DELETE`	`/api/memory?type=stm&sessionId=`	Clear STM session
`GET`	`/api/memory?type=ltm&orgId=&query=`	LTM semantic search
`POST`	`/api/memory?type=ltm`	Store LTM entry

System

Method	Endpoint	Description
`GET`	`/api/init`	Initialize DB schema, registry, circuits

All responses follow a uniform envelope:

{
  "success": true,
  "data": { ... },
  "error": null,
  "timestamp": 1712345678901
}

Folder Structure

agentic-os-grid/
├── src/
│   ├── app/
│   │   ├── api/
│   │   │   ├── router/route.ts       # Main inference entry point
│   │   │   ├── providers/route.ts    # Provider CRUD
│   │   │   ├── circuit/route.ts      # Circuit breaker control
│   │   │   ├── metrics/route.ts      # Usage metrics
│   │   │   ├── qtable/route.ts       # Q-table visualization
│   │   │   ├── memory/route.ts       # STM/LTM management
│   │   │   ├── trust/route.ts        # Trust scoring
│   │   │   ├── health/route.ts       # System health
│   │   │   └── init/route.ts         # One-time initialization
│   │   ├── page.tsx                  # Operator dashboard
│   │   ├── layout.tsx                # Root layout
│   │   └── globals.css               # Global styles
│   ├── components/ui/                # shadcn/ui components
│   ├── lib/
│   │   ├── types/index.ts            # 40+ TypeScript interfaces
│   │   ├── config/index.ts           # RL params, provider configs, Redis keys
│   │   ├── core/
│   │   │   ├── router.ts             # The Intelligence Switch
│   │   │   ├── registry.ts           # Provider Registry
│   │   │   ├── circuitBreaker.ts     # Failure Safety System
│   │   │   ├── rewardGovernor.ts     # Anti-Hacking Reward Sanitizer
│   │   │   └── meter.ts              # Usage Tracking
│   │   ├── providers/
│   │   │   └── executor.ts           # Unified LLM Provider Interface
│   │   ├── rl/
│   │   │   ├── qTable.ts             # Redis Q-Table Operations
│   │   │   ├── telemetry.ts          # Redis Stream Telemetry
│   │   │   └── trustEngine.ts        # Tenant Trust Scoring
│   │   ├── memory/
│   │   │   ├── stm.ts                # Short-Term Memory (Redis List)
│   │   │   └── ltm.ts                # Long-Term Memory (Pinecone)
│   │   └── db/
│   │       ├── redis.ts              # Upstash Redis client
│   │       ├── turso.ts              # Turso/libSQL client + schema
│   │       └── pinecone.ts           # Pinecone vector client
│   └── hooks/                        # React hooks
├── mini-services/
│   └── rl-worker/
│       ├── index.ts                  # RL telemetry processor (port 3003)
│       └── package.json
├── worklog.md                        # Development worklog
├── package.json
└── next.config.ts

How RL Routing Works

The Grid uses tabular Q-learning to continuously optimize provider selection:

1. Task Classification

Incoming prompts are classified into one of 6 task types using keyword heuristics:

reasoning — think, reason, analyze, compare, evaluate
code — code, function, implement, debug, fix, build
fast-tool — quick, fast, simple, summarize
analysis — data, chart, report, metrics
chat — default fallback

2. Provider Selection (Q-Table + Epsilon-Greedy)

For each request:
  ├── With probability ε (10%): randomly select a provider (EXPLORATION)
  └── With probability (1-ε) (90%):
       For each available provider:
         Q_value = lookup("taskType:providerId") from tenant → global → 0
         stat_score = successRate*0.4 + latencyScore*0.3 + costScore*0.3
         final_score = Q_value * 0.7 + stat_score * 0.3
       Select provider with highest final_score (EXPLOITATION)

3. Q-Table Update (by RL Worker)

Q_new = Q_old + α * (Reward - Q_old)
where α = 0.1 (learning rate)

The reward is computed by the Reward Governor:

raw_reward = quality * 1.0 - latency_norm * 0.3 - cost_norm * 0.2
sanitized = clip(raw_reward, -1.0, 1.0)
           → quality floor veto if quality < 0.4
           → anomaly cap at 0.95

4. Federated Global Update

Tenant deltas are aggregated with Anti-Whale protection:

weight = min(5.0, log10(1 + volume))
global_Q += meta_lr * weighted_avg(delta)
where meta_lr = 0.05

Memory Architecture

Short-Term Memory (STM)

Storage: Redis List (stm:session:{sessionId})
Sliding Window: Keeps last 10 messages per session
TTL: 24 hours of inactivity
Token Estimation: ceil(content.length / 4)

Long-Term Memory (LTM)

Storage: Pinecone vector index
Embedding: Cohere v4, 1024 dimensions (via AWS Bedrock)
Operations: Store, Query (top-K with metadata filter), Delete, Stats
Namespace: Per-organization metadata filtering

Trust Scoring System

Each tenant is assigned a trust score (0–1) based on 4 behavioral signals:

Component	Weight	Formula
Stability	0.35	`1 - variance` of updates
SNR	0.25	`goodUpdates / totalUpdates`
Alignment	0.25	Cosine similarity with global policy
Exploration	0.15	`1 - explorationRatio`

Effective weight for federated updates: trustScore * min(5.0, log10(1 + volume))

Tier	Score Range	Price Multiplier
Gold	>= 0.85	0.9x
Standard	>= 0.60	1.0x
Variable	>= 0.30	1.5x
Risk	< 0.30	2.5x

License

This project is provided as-is for educational and research purposes. See individual provider licenses for API usage terms.

Name		Name	Last commit message	Last commit date
.zscripts		.zscripts
agent-ctx		agent-ctx
db		db
download		download
examples/websocket		examples/websocket
mini-services		mini-services
prisma		prisma
public		public
skills		skills
src		src
upload		upload
.env		.env
.gitignore		.gitignore
AGENTS.md		AGENTS.md
ARCHITECTURE.md		ARCHITECTURE.md
Caddyfile		Caddyfile
PROJECTREPORT.md		PROJECTREPORT.md
README.md		README.md
bun.lock		bun.lock
components.json		components.json
eslint.config.mjs		eslint.config.mjs
next.config.ts		next.config.ts
package-lock.json		package-lock.json
package.json		package.json
postcss.config.mjs		postcss.config.mjs
tailwind.config.ts		tailwind.config.ts
tsconfig.json		tsconfig.json
worklog.md		worklog.md

Navigation Menu

Repository navigation

About

Repository files navigation

Agentic OS Intelligence Grid

Architecture Overview

Key Features

Tech Stack

Quick Start

1. Install Dependencies

2. Set Environment Variables

3. Initialize the Grid

4. Start the RL Worker

5. Send Your First Request

Environment Variables

API Endpoints

Inference

Provider Management

Circuit Breaker

Observability

Memory

System

Folder Structure

How RL Routing Works

1. Task Classification

2. Provider Selection (Q-Table + Epsilon-Greedy)

3. Q-Table Update (by RL Worker)

4. Federated Global Update

Memory Architecture

Short-Term Memory (STM)

Long-Term Memory (LTM)

Trust Scoring System

License

About

Releases

Packages

Contributors

Languages

Latest commit

History

Folders and files

Repository files navigation

Agentic OS Intelligence Grid

Architecture Overview

Key Features

Tech Stack

Quick Start

1. Install Dependencies

2. Set Environment Variables

3. Initialize the Grid

4. Start the RL Worker

5. Send Your First Request

Environment Variables

API Endpoints

Inference

Provider Management

Circuit Breaker

Observability

Memory

System

Folder Structure

How RL Routing Works

1. Task Classification

2. Provider Selection (Q-Table + Epsilon-Greedy)

3. Q-Table Update (by RL Worker)

4. Federated Global Update

Memory Architecture

Short-Term Memory (STM)

Long-Term Memory (LTM)

Trust Scoring System

License

About

Resources

Stars

Watchers

Forks

Releases

Packages

Contributors

Languages