Search inside your videos — and cut them into a story — locally.
GhostReel watches folders of video footage and makes everything in them searchable: it transcribes the speech with Whisper, describes keyframes with a local vision model, and embeds both so you can search by what was said, shown or written on screen and jump straight to that moment. Organise footage in projects, then chat with the model to script a video from your clips, preview the cut, and hand it to Premiere Pro as an FCP XML timeline (via OpenTimelineIO) — or export an MP4 to send as a quick demo.
It runs fully standalone — models run on your GPU (CUDA or Vulkan on Linux/Windows, Metal on macOS) inside the app, nothing else to install — on Windows, Linux and macOS. If you already run model servers, it can use them instead so the same GPU never holds a model twice: highllama for vision and embeddings and GhostPen for transcription.
Built with Tauri v2 (Rust backend + React/TypeScript frontend) and a ghostreel CLI sharing
the same core.
Design docs:
.agents/plan.md(decisions, runtime profiles, milestones),.agents/TODO.md(build status).
- Folder indexing: add folders (whole tree, recursively) to a project; GhostReel finds
every video and identifies it by content hash (moved or duplicated files are indexed once).
Indexing is a resumable queue with progress and ETA;
ghostreel index --watchkeeps indexing new footage as it arrives. - Transcripts with timestamps: Whisper (tiny → large-v3-turbo), language detection, click any line to jump there.
- Keyframes described by a vision model: scene changes plus a frame at least every N seconds (configurable), each described (subject, setting, shot, on-screen text). Scene detection decodes on the GPU. Frame descriptions run with configurable sampling resolution (512px–1280px, defaulting to 768px) and concurrent multi-sequence batching in the local helper, achieving sub-second describe times (~0.75s per frame).
- Camera steadiness & shake analysis: measures camera movement and handheld tremor asynchronously, identifying shaky stretches so the script generation and editor cut around them.
- Hybrid search: keywords (SQLite FTS5) + meaning (embeddinggemma vectors, sqlite-vec), merged into moments you can play from the right second.
- Script chat: ask for "a 60 second promo about the neighbourhood using the interviews" — the model explores the footage with tools (search, transcripts, keyframes), drafts beats with clips and voice-over, and revises from your feedback. Every clip is checked against the footage; talking clips keep whole sentences with a breath before the cut; the editor instructions are editable in Settings.
- Reference edits: mark a finished video (e.g. an edit someone already made by hand) as a reference — the chat studies its length, pacing and structure but never uses it as footage. Or just remove a video from a project's library.
- Timeline preview & export: render a low-res preview of any script version, burn titles and narration, export MP4, FCP 7 XML (Premiere Pro) or .otio.
- Model manager: download and choose models per capability (speech, vision & chat, embeddings) and where each runs: Auto, This computer or Server.
- CLI for everything: index, search, transcripts, scripts, exports and models from a
terminal (
ghostreel --help).
Download the latest build from Releases:
| Platform | File | Notes |
|---|---|---|
| Windows 10/11 | GhostReel_x.y.z_x64-setup.exe |
NVIDIA driver 570+ for GPU (CUDA 12.8 runtime is bundled) |
| Linux | .AppImage |
NVIDIA driver 570+ for GPU |
| Windows 10/11 · Vulkan | GhostReel_x.y.z_x64_vulkan-setup.exe |
AMD, Intel or NVIDIA GPU with a Vulkan 1.2-capable driver |
| Linux · Vulkan | GhostReel_x.y.z_amd64_vulkan.AppImage |
AMD, Intel or NVIDIA GPU with a Vulkan 1.2-capable driver |
| macOS 14+ · Apple Silicon | GhostReel_x.y.z_aarch64.dmg |
MLX vision/chat, Metal Whisper, self-signed |
| macOS 12+ · Intel | GhostReel_x.y.z_x64.dmg |
llama.cpp / Whisper with Metal, self-signed |
| CLI only | ghostreel-cli-*.zip / .tar.gz |
same helpers and ffmpeg, no desktop app |
On macOS, open the DMG and drag GhostReel to Applications. The app is self-signed and is not notarized by Apple: after the first launch is blocked, open System Settings → Privacy & Security → Open Anyway. Subsequent releases reuse the same signing certificate.
ffmpeg/ffprobe are bundled. On first start open Models and download what you need:
| Capability | Default model | Size | Alternatives |
|---|---|---|---|
| Speech | Whisper large-v3-turbo |
1.6 GB | small (488 MB), quantized turbo q8_0 / q5_0 (874 / 574 MB), .en variants |
| Frame descriptions & script chat | Qwen3.5 9B | Native: 6.9 GB with projector; Apple Silicon: pinned MLX bundle | Qwen2.5-VL 7B/3B, Gemma 3 4B |
| Search embeddings | embeddinggemma-300M Q8_0 |
334 MB | fixed, so indexes stay compatible between computers |
GhostReel also picks up models you already have (GhostPen's whisper models, LM Studio) instead of downloading them again. The whole setup fits an 8 GB GPU (RTX 3070).
Frame descriptions and the script chat are configured separately (Models page, or ghostreel config set vision.* / chat_model.*): describing a keyframe is a short prompt plus one image, while the chat needs room for
search results and a whole draft. Defaults: 8k context for descriptions, 32k for the chat, both with a q4_0 KV
cache and flash attention, which is what fits a 32k window on an 8 GB card (an f16 cache would need 4 GB for the
cache alone).
The llama.cpp helper fits GPU layers and MoE expert placement to available device memory, leaving
headroom for the vision projector. Settings exposes flash attention (auto, on, off) and
chat_model.cpu_moe_layers: 0 keeps automatic fitting; a positive value keeps the first N blocks'
expert weights in system RAM. This only affects MoE models, and trades generation speed for GPU
memory; it does not turn a dense model into an MoE. These controls do not apply to MLX.
The helper caps the context at the model's trained limit and retries smaller windows on allocation failure. Script chat uses that actual loaded window, counts prompt tokens before generation, and drops old exchanges while retaining the rules, current request and latest evidence to reserve space for a complete draft. Text requests reuse the common prompt prefix, with a bounded snapshot of recurrent/SWA state so Qwen3.5 can reuse it too. Image requests and incompatible cache states rebuild safely.
python3 scripts/smoke-llm-cache.py --helper <ghostreel-llm> --model <model.gguf> --mmproj <projector.gguf> [--cpu] checks cached results against fresh inference without downloading
models. A local Qwen3.5 9B CPU check took 1.85 s for a growing conversation versus 14.38 s fresh
(322 reused tokens, one generated token, excluding model load). This measures prompt reuse rather
than whole-script or GPU throughput.
For NVIDIA hardware validation, build the CUDA helper with
GHOSTREEL_GPU=cuda bash scripts/build-helpers.sh llm, then run:
python3 scripts/smoke-llm-nvidia.py \
--helper target/release/ghostreel-llm \
--model /path/to/dense-model.gguf --mmproj /path/to/dense-projector.gguf \
--moe-model /path/to/moe-model.gguf --moe-mmproj /path/to/moe-projector.gguf \
--cpu-moe 32 --output-dir /tmp/ghostreel-nvidia-validationThe MoE arguments are optional. The check verifies real CUDA memory use, attention and expert
placement from runtime logs, a 32k context, and a structured script response. The helper reports
the selected expert override patterns in its ready message. Logs and report.json remain in
the output directory for review; the check does not change the app's model settings.
Validated on Linux with an RTX 4070 (12 GB), driver 615.78.08 and a CUDA 13.4 release build:
| Model / mode | Loaded context | Helper GPU memory | Result |
|---|---|---|---|
| Qwen3.5 9B, automatic flash attention + q4 KV | 32k | 7256 MiB | 11,095-token prompt and structured script passed |
| Qwen3.5 9B, flash attention off + f16 KV | 8k | 7224 MiB | Structured script passed |
| Ornith 1.5 35B A3B, automatic expert placement | 32k | 10512 MiB | Expert tensors in host RAM; structured script passed |
| Ornith 1.5 35B A3B, 32 expert blocks in RAM | 32k | 7164 MiB | Explicit host placement and structured script passed |
For an apples-to-apples decode benchmark with the same AgentWorld GGUF and expert split:
python3 scripts/bench-llm-speed.py --helper target/release/ghostreel-llm \
--model /path/to/Qwen-AgentWorld-35B-A3B-UD-Q4_K_XL.gguf \
--mmproj /path/to/mmproj-Qwen-AgentWorld-35B-A3B-BF16.gguf \
--context 131072 --cpu-moe 32 --output-dir /tmp/ghostreel-agentworld-speedOn the same RTX 4070, a 54-token prompt and 256-token response measured 43.2 → 55.3 decode tokens/s at an allocated 128k context, with 32 of 40 expert blocks in host RAM. The comparison holds the model, prompt, quantization and placement constant, changing the old 4-thread, per-decode pool/mmap settings to 8 threads, a resident thread pool and non-mapped host experts. At an allocated 16k context with 26 host expert blocks, it measured 39.6 → 57.6 tokens/s. These are short-prompt, single-run measurements, excluding model load and prompt processing; a full 128k input or a multi-pass script takes longer. The benchmark writes both reports and runtime logs. The UI's expert slider shows the share of AgentWorld's 40 blocks in RAM; a value of 0 lets automatic fitting choose placement.
A real Greet Mag script-agent trial used an initial 28,275-token prompt, 128k allocated context, 32 host expert blocks, and thinking disabled. Six model calls averaged 44.7 decode tokens/s (1,839 generated tokens); the full turn took 87.0 seconds, excluding model load. Initial prefill took 39.7 seconds; cached later prefills took 0.6–2.5 seconds. The saved 57.6-second draft passed structural validation after repairs. This checks the agent pipeline, not editorial quality.
Local script agents default to 30 research tool calls. Planning replies do not use this budget;
a separate model-call ceiling prevents endless retries. Proposed shots are checked with
get_video, spoken quotes with get_transcript, and the model reviews the resulting evidence
before finalizing. Revisions introducing new ranges are checked again within the same budget.
Missing coverage after repairs is flagged as incomplete research. Adjust the limit in Settings
or chat_model.max_tool_rounds; 0 uses script.local_tool_rounds / script.roomy_tool_rounds.
New chat answers persist the actual model name, token count, decode rate and whole-turn time. When a server supplies token usage but no decode time, the footer explicitly labels its rate as overall, including prompt processing and request overhead. Older answers and providers without counters show that metrics are unavailable.
For a larger CUDA cache check, add --context 32768 --notes 1800 to smoke-llm-cache.py.
The growing 9063-token prompt reused 9022 tokens and took 0.096 s versus 2.949 s fresh, with
matching output. This is a one-token-response prompt-reuse check, excluding model load, not a
whole-script throughput claim. Prefer automatic placement; explicit expert offload frees more
VRAM at the cost of additional work in system RAM.
In a project's Library, check videos to search just those files. Shift-click a checkbox for a range, Ctrl/⌘-click rows to add or remove files, or turn on Select videos and drag a rectangle across rows. The header checkbox selects all visible rows; category filters can hide selected rows, and the selection count makes that explicit. Clear selection returns to the folder or whole-project scope. Normal row clicks still open previews outside selection mode.
Choose a watched folder under Search in, or click Search here beside its path, to restrict both the library and search to that folder's indexed videos, including its indexed subfolders. Changing folders clears the file selection. Keyword and semantic search both honor these scopes.
| Standalone (default on a fresh machine) | Shared servers | |
|---|---|---|
| Speech | ghostreel-asr helper (whisper.cpp, CUDA) |
GhostPen STT server :8771 |
| Vision & chat | ghostreel-llm helper (llama.cpp, CUDA) |
highllama :8089 (OpenAI-compatible) |
| Embeddings | ghostreel-llm helper |
highllama embeddings :8091 |
Each capability is set to Auto, This computer or Server on the Models page (or
ghostreel config set vision.backend server). Auto uses a server only when it is reachable
and capable (vision support, timestamped transcripts, the exact embedding model) and falls
back to the local helper otherwise. ghostreel doctor shows what would be used and why.
Any OpenAI-compatible vision server works for frame descriptions and chat (URL, model and API key are configurable).
No GPU headroom for a vision model? Point a capability at a coding-agent CLI you already have — claude, agy or opencode — on the Models page (backend CLI agent) or from the terminal:
ghostreel config set vision.backend cli # frame descriptions through a CLI
ghostreel config set vision.cli.tool claude
ghostreel config set chat_model.backend local # …while the script chat stays local
ghostreel config test-cli vision # one call, prints what it answeredThe two capabilities are independent: claude for the descriptions and a local model for the script, or the other way round, or a server for one and a CLI for the other.
| calls | what it costs | |
|---|---|---|
| Frame descriptions | one per keyframe | claude bills per call (~$0.04 a frame, so ~$60 for a 1500-frame library); agy and opencode spend their own quota |
| Script chat | a handful per script | cents with claude |
Auto never picks a CLI — it spends money or quota, so it has to be chosen explicitly. ghostreel doctor
lists which CLIs it can find, and the Models page has a Test one call button that describes a generated
image so you can check the setup (and the latency) before indexing.
- New project → name, frame rate and resolution (used for timeline exports).
- Add folder → pick the footage folder; Index now. Progress shows in the project and on the Activity page. Stop any time; indexing again resumes where it left off.
- Search the project: "people talking about the park", "for sale sign", on-screen text. Click a hit to play from that moment; click a video to see its keyframes and transcript.
- Scripts tab → describe the video you want. Revise in the chat ("slower, let people finish talking") or edit beats and clips by hand; each change is a new version.
- Render preview to watch the cut, then Export for Premiere (FCP XML), Export .otio or Export MP4.
In Premiere Pro: File → Import the .xml; the sequence links to your original files.
ghostreel doctor # GPU, ffmpeg, models, servers
ghostreel project create "Greet Mag" --fps 30 --width 3840 --height 2160
ghostreel folder add ~/Videos/shoot -p "Greet Mag"
ghostreel index -p "Greet Mag" # --watch to keep indexing new files
ghostreel search -p "Greet Mag" "neighbours talking about the community"
ghostreel script chat --project "Greet Mag" "a 60 second promo using the interviews"
ghostreel script export <script-id> --format fcp_xml -o promo.xml
ghostreel models list | download <id> | use <id>
ghostreel config set frames.max_interval_s 5 && ghostreel index -p "Greet Mag" --redo framesghostreel mcp speaks the Model Context Protocol on stdio, so an
agent can index footage, search it and build an edit. Register it once:
{
"mcpServers": {
"ghostreel": { "command": "ghostreel", "args": ["mcp"] }
}
}Or let the agent's own command add it:
| Agent | Command |
|---|---|
| Claude Code | claude mcp add --scope user ghostreel -- ghostreel mcp |
| Codex | codex mcp add ghostreel -- ghostreel mcp, then set tool_timeout_sec = 1800 under [mcp_servers.ghostreel] (the 60 s default is shorter than index and draft_script take) |
| Antigravity | agy mcp add ghostreel ghostreel mcp |
| OpenCode | opencode mcp add --global ghostreel -- ghostreel mcp |
Claude Desktop, Cursor, Gemini CLI and most other clients take the JSON above in their own
settings file; give the full path to ghostreel where the client doesn't read your shell's
PATH. Add "env": {"GHOSTREEL_HOME": "/path"} to work on a different library. The
guide has the details for each.
| Tool | What it does |
|---|---|
doctor |
ffmpeg, GPU, database, models and which AI backends would be used |
list_projects · create_project · delete_project |
projects with their indexing progress; create one (fps/size = export timeline); delete one (video files are never touched) |
add_folder |
watch a footage folder (whole tree; editor render/cache folders are skipped) |
index · index_status |
run pending indexing for up to max_seconds (resumable — call again while work is pending), or just report progress |
list_videos |
one row per video: category (interview, broll, drone…), camera style, speech, keyframes and what it shows |
search |
hybrid keyword + meaning search; returns moments with video_id and timestamps |
get_transcript |
timestamped speech of one or more videos; off-mic (interviewer) lines marked [off], or dropped with speech_only |
get_video · get_frames |
each keyframe's description and on-screen text; the keyframe pictures themselves (JPEG) |
get_steadiness |
camera shake in a stretch of a video, a verdict, and the nearest steady stretch when it is shaky |
list_scripts · get_script |
script versions of a project |
draft_script |
let GhostReel's editing model draft or revise a script from the footage (minutes) |
save_script |
save your own script JSON as a new version (dry_run only checks it); returns the issues found |
preview_script |
render a preview MP4 of a script version and return its path |
export_script |
write fcp_xml (Premiere Pro) or otio |
get_settings · set_settings |
read or change the model settings (same keys as ghostreel config set) |
set_video_role |
take a video out of the library (removed) or keep it as a reference edit |
Typical run: create_project → add_folder → index until nothing is pending →
list_videos / search / get_transcript / get_video / get_frames to learn the footage →
draft_script or save_script → preview_script / export_script. The agent can write the
edit itself without GhostReel's chat model, checking its picks with get_steadiness. Clips must
lie inside footage the tools reported: save_script returns an error issue per clip that
doesn't and a warning per clip on a shaky stretch (and still saves the version, so it can be
fixed; dry_run checks without saving).
A script is plain JSON:
{
"title": "NW Hills promo",
"target_duration_s": 60,
"beats": [
{
"id": "hook",
"purpose": "Open with a resident on why the neighbourhood is special",
"narration": "",
"on_screen_text": "Northwest Hills",
"clips": [{ "video_id": 8, "in_s": 12.4, "out_s": 28.9, "audio": "source" }]
}
]
}audio is source (someone speaking) or mute (b-roll under narration). narration is the
voice-over for that beat — about 2.5 words per second of its clips.
A beat can also have a bed, { "video_id": 3, "in_s": 40.0, "out_s": 52.5 }, whose audio
runs under the whole beat: its clips should be mute and add up to the bed's length, so an
interview answer plays over b-roll (a J-cut).
Everything lives in one folder: ~/.ghostreel/ (%USERPROFILE%\.ghostreel\ on Windows).
| Path | What |
|---|---|
config.toml |
backends, models, frame interval, chat instructions |
ghostreel.db |
projects, transcripts, frame descriptions, vectors, scripts (SQLite) |
frames/ |
keyframe JPEGs (thumbnails, descriptions) |
proxies/, previews/ |
preview renders (safe to delete; regenerated) |
models/ |
downloaded models |
GHOSTREEL_HOME moves the whole folder. Your videos are only read, never modified or copied.
- In standalone mode nothing leaves your machine.
- With servers, frames, transcripts and prompts go to the URLs you configured — point them at
machines you trust. API keys are stored in plaintext in
config.toml.
GhostReel checks for new releases from github.com/highercomve/GhostReel on the Status page. Click Check for updates — if a newer version is available you will see the release notes and a Download and install button with a progress bar. Once the download finishes, Restart to finish applies the update immediately.
Updates are signed with a minisign key; the public key is embedded in the app and the
installer. The updater manifest (latest.json) is published to the GitHub release by CI
and consumed over HTTPS — no external update server required.
Requirements: Rust (stable), Node.js 22, the Tauri prerequisites
and ffmpeg on PATH for development. For GPU helpers, install the CUDA Toolkit 12.x
or the Vulkan SDK (including glslc and SPIRV-Headers); Windows also needs MSVC. On macOS, install the Xcode command-line tools (xcode-select --install)
and CMake (brew install cmake); native helpers use Metal automatically.
On Apple Silicon, also install Python 3.12 for building the frozen MLX helper:
GHOSTREEL_MLX_PYTHON=python3.12 scripts/build-mlx.sh. Installers bundle Python and MLX;
users do not need either installed.
npm install
cargo test --workspace # core tests (no GPU, no servers)
cargo run -p ghostreel-cli -- doctor
npx tauri dev # desktop app with hot reloadRelease bundles (see AGENTS.md → Packaging):
scripts/build-helpers.sh # ghostreel-asr + ghostreel-llm (CUDA if the toolkit is present)
npm run bundle:linux # AppImage (+ CLI tarball)
npm run bundle:windows # NSIS installer (on Windows)
npm run bundle:macos # .app + DMG (on macOS; ad-hoc signed locally)CI builds CUDA and Vulkan variants for Linux/Windows, plus native Apple Silicon / Intel
macOS installers on every v* tag (.github/workflows/release.yml). Vulkan assets include
_vulkan in their filenames. Updates preserve the installed backend using separate
linux-x86_64-vulkan / windows-x86_64-vulkan entries in latest.json; existing CUDA
and macOS updater targets stay compatible.
Master builds warm separate Rust and compiled-helper caches for each OS/backend. Exact helper cache hits skip native compilation and Vulkan SDK setup. Cache keys include helper sources, vendored bindings, external locked dependencies, workspace build settings, Rust/C++ toolchains, runner image, and GPU SDK version; workspace version bumps reuse the helpers. SDK downloads and npm dependencies are cached too. Tags restore the default branch's caches; master saves them for reuse across releases.
To build Vulkan helpers locally, set up the Vulkan SDK environment, then run
GHOSTREEL_GPU=vulkan scripts/build-helpers.sh before the normal bundle command.
For a Vulkan app that preserves its backend on update, also set
GHOSTREEL_UPDATER_TARGET=linux-x86_64-vulkan (or windows-x86_64-vulkan) when bundling.
Apple Silicon uses MLX for local frame descriptions and script chat. The Models page selects MLX versions of Qwen2.5-VL 3B/7B, Gemma 3 4B, and Qwen3.5 9B. Downloads are pinned to a Hugging Face revision and include every weight shard, tokenizer, and processor configuration. Interrupted downloads resume, and a model is only marked installed once every file is complete. Qwen3.5 9B is the default for both frame descriptions and script chat on every platform. Apple Silicon uses its pinned MLX bundle; Linux, Windows and Intel Macs use the GGUF model and projector. The old Bonsai selection on Apple Silicon migrates to Qwen3.5 9B because Bonsai has no matching MLX release in this catalog.
Whisper and embeddinggemma retain their existing model files and helpers. Intel Macs, Linux, and Windows retain GGUF for vision/chat. MLX processes frames serially to bound memory usage; structured JSON replies use a grammar and skip free-form reasoning. MLX selects its own attention implementation, so llama.cpp's flash-attention setting is hidden on Apple Silicon. MLX requires macOS 14+; the Intel installer still supports macOS 12+.
CI freezes scripts/ghostreel_mlx.py with Python 3.12, includes Metal shaders and processor
implementations, and runs the standalone helper's Metal smoke test and a real-model vision/chat integration test. To test it locally after
building: target/release/ghostreel-mlx --self-test. There is no claim of a speed advantage until
frame-description and chat workloads have been benchmarked on the same Mac.
Like Oriel, GhostReel uses a persistent self-signed code-signing certificate. Create it once on any host with OpenSSL, outside the repository, and save both files securely:
scripts/macos-signing.sh create ~/.config/ghostreel/keys
openssl base64 -A -in ~/.config/ghostreel/keys/ghostreel-codesign.p12 | gh secret set MACOS_CERTIFICATE
gh secret set MACOS_CERT_PASSWORD < ~/.config/ghostreel/keys/ghostreel-codesign.passwordRun the gh commands from this repository. Reuse these secrets for every release;
rotating the certificate changes the app's signing identity. CI imports it into a temporary
keychain, signs the app, its sidecars and the DMG with Tauri, verifies signatures and removes
the keychain even on failure. Release builds require both macOS secrets; master cache builds
need neither. The existing TAURI_SIGNING_PRIVATE_KEY still signs updater payloads separately.
No Apple account or notarization credentials are required. For a local build with this identity,
import the .p12 into your login keychain and set APPLE_SIGNING_IDENTITY to its SHA-1 fingerprint
before npm run bundle:macos (see Tauri signing).
macOS FFmpeg/ffprobe are pinned standalone builds from Martin Riedl (Apple Silicon) and Evermeet (Intel), verified by SHA-256. They include the software encoder, drawtext and subtitle filters used by previews. Their upstream license and source notice are included in the app's resources.
crates/ghostreel-core config, DB (SQLite + FTS5 + sqlite-vec), probing, indexing, frames,
search, models, script chat, preview, OTIO export
crates/ghostreel-cli the `ghostreel` binary
crates/ghostreel-asr whisper helper process (CUDA)
crates/ghostreel-llm vision / chat / embeddings helper process (CUDA)
src-tauri/ desktop app: commands, task queue, local media server
src/ React + TypeScript frontend
scripts/ helper builds, sidecar fetch/staging, packaging
.agents/ plan and TODO
whisper.cpp and llama.cpp run in separate helper processes because their ggml builds can't share one binary.
MIT