logprobs

Open-source alternative to TypeSafe's Jev: a System One style model layer that gives typed, calibrated decisions from any open-weights LLM in one forward pass (HF + vLLM), with honest benchmarks

Typed choices and probability distributions from text and images with llama.cpp, Ollama, and OpenRouter

Super fast Jev like server, model agnostic, working with any OpenAI compatible endpoint

Jev-shaped (TypeSafe System One) classification wrapper over OpenAI-like clients

Typed decisions from local LLMs in one forward pass, via Ollama. No generation loop, no JSON parsing.

An experimental approach for AI search visibility.

Calibrated yes/no, pick-one and scale judgments from your own LLMs, read straight off the logprobs. Runs on vLLM.

Interactive Streamlit app that visualises token-level log probabilities from Azure OpenAI

Jev-compatible /v1/systemone server reading typed decisions from LLM logits, benchmarked against TypeSafe's Jev on the same items via JevBench

Token-level entropy trajectories from LLM logprobs. Models can measure their own uncertainty — grounding it in truth requires external verification.

Jev-style /v1/systemone API in front of any OpenAI-compatible LLM server (one-token logprob scoring, MIT)

Model-agnostic substrate for running small local LLMs well — code agents, chat, image/video, speech. Small capable main model + tiny fast worker.

An interactive client for exploring token-level log probabilities from OpenAI-compatible chat models.

Self-hosted System One decision engine (TypeSafe Jev's Choice/Score/Noul contract) running locally on LM Studio, with real probabilities from token logprobs.

Minimal demo of calibrated LLM-as-a-judge scores from the single-call token-logprob distribution (Wang et al. 2025; G-Eval).

A drop-in confidence gate for LLM agents: turn token logprobs into an accept / verify / ask-a-human routing decision, with honest per-model calibration. Local & open-model first, plus an MCP server.

A demo to show confidence probabilities in AI responses

See what an LLM almost said: token-level probabilities, fork points and branching alternate completions via OpenAI logprobs. FastAPI + React.

Confidence-gated invoice extraction: per-field scores from constrained decoding, measured for calibration on DocILE

Analyze LLM token log-probabilities: perplexity, per-token confidence, low-confidence span detection, and a terminal heatmap. Zero deps, CLI + library.