ICE sits between a chat client and any OpenAI-compatible model and gives that model a persistent, structured memory of everything it has discussed — entirely on your own hardware.
client ──▶ ICE proxy ──▶ local model (Ollama, or vLLM)
│
└── PostgreSQL + pgvector
episodic · knowledge graph · procedural · documents
Every chat session starts from zero. The user re-explains who they are, what they are building, and which decisions were already ruled out. Larger context windows have not solved this: a window is a buffer, not a memory. Once a conversation outgrows it the model silently loses the thread, and the usual workarounds — exporting transcripts, pasting them into a fresh session — degrade within a dozen turns.
The instinctive fix is to retrieve more: fill the window with everything plausibly related. The central finding of this project is that this instinct is wrong. Retrieval quality is governed by what you leave out.
Each turn traverses a synchronous pre-flight and an asynchronous post-flight phase.
Pre-flight. The prompt is classified by a small PyTorch head over a frozen
Qwen3-Embedding-0.6B encoder: 27 all-sigmoid logits across three heads — 11 topic labels,
12 intent labels, and 4 independent context-reliance signals (Needs_Memory,
Temporal_Recall, Needs_Live_Info, High_Complexity). The reliance signals are deliberately
independent rather than a single choice, because a prompt can need stored memory and live
information at once; a fully self-contained prompt is the derived state where all four stay
low. A calibrated decision then combines the memory signal with a memory-pressure prior to
decide whether long-term retrieval fires at all. The hybrid orchestrator runs its legs in
parallel,
fuses them with weighted Reciprocal Rank Fusion, and post-processes the fused list with
keyword/recency/length bonuses, session diversification, deduplication, and a per-query token
budget. A prompt assembler lays the result out under a stable prefix to maximise KV-cache
reuse, and a mixture-of-experts router picks the best locally-served model.
Post-flight. Once the response has streamed, the turn is evaluated for information density, summarised if it does not earn lossless storage, mined for behavioural patterns, and — when dense enough — passed to the knowledge-graph extractor. An in-process maintenance runtime then decays, clusters, reflects on, and compacts the stores on ledger-driven cadences.
The organising principle in the code is that memory is earned: a turn is preserved losslessly only if it is dense enough to deserve it. Everything else is compressed, decayed, and eventually archived to cold storage.
| Store | Contents | Retrieval |
|---|---|---|
| Episodic | every turn, with decay scores, summaries, and access counts | BM25 and decay-weighted vector search |
| Codex | a temporally-versioned knowledge graph: entities, typed edges carrying valid_from/valid_until, an append-only event log |
graph traversal from resolved entities |
| Procedural | recurring behavioural patterns mined from interaction history | vector match behind trigger conditions |
| Documents | ingested files, chunked and embedded | chunk-level vector search |
These are complemented by persistent memory slots, topical context clusters, batch summaries, cold storage, and a timeline leg serving temporal queries.
The system has grown several capabilities that sit alongside the core loop:
- Temporal retrieval. Memory is queryable along the time axis, not just by similarity:
as-of ("what did I think about this in March?"), range, and evolution ("how did my
design for X change?") — backed by the knowledge graph's
valid_from/valid_untiledges, so a superseded fact stays retrievable as history rather than being overwritten. - Agentic maintenance. A maintenance agent reconciles graph state during idle GPU time — merging duplicate entities, resolving contradictions, and escalating anything ambiguous to a review queue rather than guessing. Deterministic checks run first; the model is consulted only for genuinely ambiguous supersessions.
- Conversation import. Exported histories from ChatGPT, Claude, and DeepSeek can be replayed through the full pipeline, reconstructing episodic, graph, procedural, and cluster state as though ICE had been present all along — rather than dumping them into a searchable archive. This is the same replay machinery the evaluation protocol uses.
- Coding mode. A project-state engine tracks code structure, decisions, and git history as first-class memory, so retrieval can surface an architectural decision alongside the file it applies to.
- MCP server. ICE exposes itself over the Model Context Protocol for headless use by agents, booting the stack itself without the HTTP proxy.
- User control. In-chat commands (
/remember,/forget,/bookmark,/scope,/search,/slots,/delete-conversation), per-conversation and project scoping, an incognito mode that writes nothing, and a real deletion cascade — memory the user can inspect, correct, and remove rather than merely accumulate.
ICE is evaluated with LSREP (Longitudinal State-Replay Evaluation Protocol), a benchmark developed for this work. LSREP replays real, long-running conversations turn by turn, reconstructs the complete memory state at each of 50 checkpoints, and scores answers against a ground truth that evolves as facts are superseded — measuring what static benchmarks cannot: memory that accumulates, decays, and is revised over months. The reported study covers 1,985 turns and 1,211 probes, judged by an independent model.
Compared against a strong vector-RAG baseline sharing the same embedder, database, and budget logic:
| Result | Finding |
|---|---|
| Answer quality | Statistical tie — paired difference +0.00 (95% CI [−0.07, +0.07]) |
| Context efficiency | 32% fewer fragments injected for that same quality |
| Head-to-head preference | 30.6% vs 21.2% of blind tournament wins, non-overlapping CIs |
| Fragment quality | ICE's fragments correlate positively with answer quality (r = +0.19); the baseline's show no relationship (r = −0.02) |
| Robustness | On 8,000+ token turns the unbudgeted baseline fails 94.2% of probes; ICE holds at a mean score of 4.33 |
A cumulative ablation isolates rank fusion as the mechanism that makes multi-signal retrieval safe: adding an unfused lexical leg is actively harmful (−0.74, 95% CI [−1.14, −0.36]), and fusion recovers it (+0.82, [+0.39, +1.24]).
Read plainly: ICE does not beat a well-built vector-RAG baseline on raw answer quality. It matches it on a third less context, is preferred in blind comparison, and survives conditions under which the baseline collapses.
A component-level fidelity audit is published alongside the paper, and it separates three things that ablation studies routinely conflate. One component was genuinely defective (a pgvector binding bug that killed procedural retrieval outright). Several were never exercised by this benchmark — document retrieval had no documents ingested, nothing decayed far enough to trigger batch summarisation or cold storage, every probe asked for current truth rather than how a fact changed, and every probe was scoped to a single conversation. And the rest worked and carried the result: the per-query token budget, rank fusion, decay-weighted episodic retrieval, post-fusion curation, and a live-but-under-weighted knowledge graph. A component that never ran is not a component that failed, and the paper is careful not to claim otherwise in either direction.
- 📄 Paper —
experiments/paper/ICE_paper_v2.pdf - 🔍 Fidelity audit —
experiments/paper/notes/FIDELITY_AUDIT.md - 🏷 Evaluated snapshot — git tag
v2-paper-eval
src/api/ FastAPI proxy, prompt assembly, configuration, routers
src/classifier/ intent / topic / context-reliance classifier and rule-based pre-pass
src/retrieval/ hybrid orchestrator — legs, RRF fusion, budgeting, post-processing
src/memory/ ORM models, shared embedder, backup / export / re-embed tooling
src/workers/ in-process maintenance runtime and the individual jobs
src/coding/ project-state engine for code-aware memory
src/ingestion/ conversation import (ChatGPT / Claude / DeepSeek exports)
src/mcp/ ICE as an MCP server, for headless use by agents
src/services/ HTTP-free service layer shared by the API and MCP surfaces
docs/ architecture reference, roadmap, provenance, cleanup ledger
experiments/ the three experiments, their harnesses, results, and the paper
tests/ standalone integration scripts plus a fast pytest smoke suite
docs/ICE_Architecture.md is the authoritative description of the
system as built and the best entry point for reading the code.
Important
ICE is not packaged or distributable software. It is a research system developed on and
for a single Arch Linux workstation with an NVIDIA GPU. setup.sh is a personal bootstrap
script, not an installer — it invokes pacman directly, assumes pyenv, and does not
provision a model server or configuration file. Expect to adapt it. A packaged application
is a roadmap item, not a current capability.
Environment: Linux, Docker, uv, Python 3.11.9+, PostgreSQL with pgvector (supplied via Docker), a running Ollama instance with at least one pulled model, and an NVIDIA GPU for background extraction work.
Bring the stack up manually:
docker compose -f docker/docker-compose.yml up -d # PostgreSQL + pgvector
uv sync # Python dependencies
uv run alembic upgrade head # database schema
uv run uvicorn src.api.main:app --host 0.0.0.0 --port 8000Configuration is read from a .env file at the repository root through Pydantic Settings
(src/api/config.py documents every field and its default, including ollama_base_url and
the model paths). That file — and every model and data path ICE loads — is resolved against
the installation directory rather than the shell's working directory, so ICE behaves the same
whatever you launch it from; set ICE_HOME to override where it looks.
Point an OpenAI-compatible client at http://localhost:8000/v1 and address the synthetic
model name ice-proxy. Send an X-ICE-Conversation-ID header to scope memory to a
conversation.
Maintenance — decay, clustering, reflection, extraction, compaction — runs in-process on an async scheduler. There is no broker and no worker fleet; PostgreSQL is the only external service.
uv run pytest tests/smoke -q # fast sanity suiteAn active research project by a single author. It is not a product, and the following limits are deliberate and documented rather than incidental:
- The evaluation is single-user. Every benchmark conversation was written by the author. The results demonstrate effectiveness across conversation types, not across a population of users. The corpora themselves are not released, because they are personal; the protocol, harness, and metrics are.
- The published numbers describe a tag, not
main. They were produced atv2-paper-eval. Since then the Celery worker fleet was replaced by an in-process runtime, embeddings moved to 1024 dimensions, the RAG leg was replaced by a document store, and temporal retrieval, a coding mode, an MCP surface, and conversation import were added. - Several components remain immature. The knowledge graph under-contributes relative to its design, and some mechanisms have not yet been measured in a fully working state. The paper and the fidelity audit identify exactly which, and why.
- Single-machine, single-user, and not hardened for deployment.
Planned work is tracked in docs/ROADMAP.md.
Memory is the most intimate thing a user can hand to an AI system. ICE keeps it on the user's own hardware, in a database they can inspect, edit, export, and delete, and requires explicit approval before high-stakes memory updates are applied. A system that faithfully models someone's beliefs and history is dual-use by nature; keeping it local, inspectable, and under the user's control is treated here as part of the contribution rather than a property to be traded away.
@software{sonar2026ice_software,
author = {Sonar, Deepesh},
title = {{ICE}: Infinite Context Engine},
year = {2026},
doi = {10.5281/zenodo.21759702},
url = {https://github.com/Deepnar/ice}
}@unpublished{sonar2026ice_paper,
author = {Sonar, Deepesh},
title = {{ICE}: A Local-First Conversational Memory System and a
Longitudinal Evaluation Protocol},
year = {2026},
note = {Submitted to ACM Transactions on Intelligent Systems and Technology}
}Licensed under the Apache License 2.0. See NOTICE for attribution.