Skip to content

configuration.md

Latest commit

 

History

History
110 lines (75 loc) · 5.3 KB

configuration.md

File metadata and controls

110 lines (75 loc) · 5.3 KB
Raw

Configuration

This document covers every environment variable and Docker Compose option.

Table of Contents

llm-service

All configuration is read from environment variables in llm-service/app/config.py. Reference copy: llm-service/.env.example.

Variable Default Description
GENERATION_MODEL qwen2.5-coder:7b Ollama model used for plan generation, code generation, and critique.
EMBEDDING_MODEL bge-m3 Embedding model requested from the embeddings endpoint for RAG.
CONFIRM_WORD CODE_OK Exact word (case-insensitive) the LLM critic must return to approve code.
MAX_RETRIES 2 Reserved; not currently wired into the pipeline.
CODE_RETRIES_COUNT 5 Reserved; not currently wired into the pipeline.
CODE_RETRIES_MODEL 2 LLM model retry budget used by GenerationPipeline.
CODE_RETRIES_SANDBOX 20 Max regeneration attempts per code validation loop (sandbox feedback).
HOST 0.0.0.0 Bind address for the FastAPI server.
PORT 8080 Bind port for the FastAPI server.
OLLAMA_URL http://ollama:11434 Ollama base URL.
SANDBOX_SERVICE_URL http://sandbox-service:6778 sandbox-service base URL.
QDRANT_URL http://qdrant:6333 Qdrant base URL (RAG).
QDRANT_COLLECTION lua_patterns Qdrant collection searched for RAG matches.
EMBEDDINGS_URL http://embeddings:8000/v1/embeddings OpenAI-compatible embeddings endpoint (POST with {"input": "...", "model": "..."}).

Not configurable via environment:

Setting Value Where
LIMIT_FOR_RAG_DOCS 1 Top-K results per plan chunk (hardcoded in app/config.py).
RAG embed model bge-m3 Hardcoded in the RAG client request body (app/clients/rag.py).

Note on retry settings: the active retry loops use CODE_RETRIES_SANDBOX (validation loop) and CODE_RETRIES_MODEL (pipeline). MAX_RETRIES and CODE_RETRIES_COUNT are defined for future use and currently have no effect.

Ollama models

The default qwen2.5-coder:7b is pulled automatically by the ollama-init container on first startup. To use a different model, pull it into Ollama yourself and set GENERATION_MODEL accordingly.

RAG prerequisites

RAG is fully optional. For it to work you need:

  1. A running Qdrant instance with a collection named per QDRANT_COLLECTION.
  2. A running embeddings service exposing an OpenAI-compatible /v1/embeddings endpoint, reachable at EMBEDDINGS_URL.

If either is unavailable, build_rag_context swallows the error and generation proceeds without RAG context.

sandbox-service

The Rust service reads only logging configuration:

Variable Default Description
RUST_LOG info tracing filter, e.g. debug, sandbox_service=debug, trace.

Runtime limits (memory, timeout) are not configurable per deployment; see sandbox-security.md for the fixed values and the per-request timeout field.

llm-tui

Variable Default Description
LLM_SERVICE_URL http://localhost:8080 Base URL of llm-service.

Docker Compose

The full stack is defined in docker-compose.yml.

Service Image / Build Ports Notes
ollama ollama/ollama:latest 11434 Persistent model storage in the ollama_data volume.
qdrant qdrant/qdrant 6333 Data persists in ./qdrant_data (bind mount).
ollama-init ollama/ollama:latest — Pulls qwen2.5-coder:7b; runs once, then exits. llm-service waits for it via service_completed_successfully.
llm-service ./llm-service 8080 Env: OLLAMA_URL=http://ollama:11434 (others default to in-network service names).
sandbox-service ./sandbox-service 6778 Runs with privileged: true; logging capped at 10 MB × 3 files.
llm-tui ./llm-tui — Mounts ./exports and lua_to_json.py; stdin_open + tty for interactive use.

Customizing defaults: create a .env file in the project root. Compose automatically substitutes ${VAR} references in docker-compose.yml. Note that the compose file currently hardcodes OLLAMA_URL for llm-service; to override it, edit docker-compose.yml to use an environment variable.

Operational notes:

  • Sessions are stored in memory in llm-service; restarting the container loses all active sessions.
  • sandbox-service is privileged — do not expose its port externally without a security review.
  • Qdrant data is persisted in ./qdrant_data (note the :Z SELinux label on some hosts).
  • The first docker compose up --build downloads the Ollama model and can take several minutes.

Configuration precedence

  1. Environment variables set on the container/process take precedence.
  2. llm-service/.env.example is a reference only — it is not loaded automatically. For local development, copy it to llm-service/.env or export variables in your shell.
  3. Defaults in code apply when nothing is set.