diff --git a/.gitattributes b/.gitattributes new file mode 100644 index 0000000..be963a1 --- /dev/null +++ b/.gitattributes @@ -0,0 +1,3 @@ +# Force LF line endings for all shell scripts — prevents CRLF breakage on Windows checkouts. +*.sh text eol=lf +git-mem text eol=lf diff --git a/.gitignore b/.gitignore index 3a10380..1dd2be2 100644 --- a/.gitignore +++ b/.gitignore @@ -1 +1 @@ -*-export.json +*-export.json diff --git a/LICENSE b/LICENSE index 2375f8e..f6d1928 100644 --- a/LICENSE +++ b/LICENSE @@ -1,21 +1,21 @@ -MIT License - -Copyright (c) 2026 cribe - -Permission is hereby granted, free of charge, to any person obtaining a copy -of this software and associated documentation files (the "Software"), to deal -in the Software without restriction, including without limitation the rights -to use, copy, modify, merge, publish, distribute, sublicense, and/or sell -copies of the Software, and to permit persons to whom the Software is -furnished to do so, subject to the following conditions: - -The above copyright notice and this permission notice shall be included in all -copies or substantial portions of the Software. - -THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR -IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, -FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE -AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER -LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, -OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE -SOFTWARE. +MIT License + +Copyright (c) 2026 cribe + +Permission is hereby granted, free of charge, to any person obtaining a copy +of this software and associated documentation files (the "Software"), to deal +in the Software without restriction, including without limitation the rights +to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +copies of the Software, and to permit persons to whom the Software is +furnished to do so, subject to the following conditions: + +The above copyright notice and this permission notice shall be included in all +copies or substantial portions of the Software. + +THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +SOFTWARE. diff --git a/MULTI-SOURCE.md b/MULTI-SOURCE.md new file mode 100644 index 0000000..639d2c1 --- /dev/null +++ b/MULTI-SOURCE.md @@ -0,0 +1,104 @@ +# Multi-Source Memory + +**Status:** ✅ Feature complete (June 19, 2026) — folder-based design + +Federated memory search across multiple git-memory repositories — search your primary memories **plus** shared team/base knowledge without merging repos. + +## Use Cases + +### 1. Agent Isolation +Each agent gets its own store; link others as read-only: +```bash +export GIT_MEMORY_DIR=~/hermes-memory +git-mem init +git-mem source add personal ~/memory-store # symlink (local) or clone (URL) +``` +Hermes writes to `~/hermes-memory`, searches both. Your personal store is never written to. + +### 2. Team Knowledge Sharing +```bash +git-mem source add team /mnt/shared/team-memories +git-mem source add daniel ~/external/daniel-devops-notes +git-mem search "kubernetes deployment" +``` +Searches your repo + team repo + Daniel's repo, shows which source each hit came from. + +### 3. Work/Personal Separation +```bash +export GIT_MEMORY_DIR=~/work-memories +git-mem source add personal ~/personal-memories +git-mem search "docker networking" +``` + +## Commands + +```bash +git-mem source add # symlink local path or clone URL +git-mem source list # show all sources (enabled/disabled) +git-mem source disable # exclude from search (renames dir) +git-mem source enable # re-include in search +git-mem source remove # unlink symlink or delete clone +git-mem source sync [name] # git pull for one or all sources +``` + +## How It Works + +### Storage (folder-based) +Sources are git repos living under `${GIT_MEMORY_DIR}-sources/`: +``` +~/memory-store-sources/ +├── team/ ← symlink to /mnt/shared/team-memories +├── hermes-base/ ← cloned from URL +└── daniel.disabled/ ← disabled (excluded from search) +``` + +No config file, no JSON. The filesystem IS the source of truth: +- Each child dir containing `.git` is a source +- Source name = folder basename +- `.disabled` suffix = excluded from search +- Local paths are symlinked; URLs are cloned + +### Search Behavior +1. **Primary repo** (`$GIT_MEMORY_DIR`) is always searched first +2. **Enabled sources** are searched in folder-name order +3. Results from sources are prefixed with `[source-name]` +4. Disabled sources (`.disabled` suffix) are skipped + +### Example Output +``` +$ git-mem search redis +Found: + a1b2c3d Redis timeout config + [team] e6f0c08 [team][gotcha] Redis needs 30s minimum timeout +``` + +## Design Principles (KISS) + +✅ **No config files** — filesystem is the source of truth +✅ **No jq dependency** — pure bash + awk +✅ **No history rewriting** — sources are read-only views +✅ **No merge conflicts** — each repo stays independent +✅ **Explicit enable/disable** — rename suffix, visible in `ls` +✅ **Source attribution** — every result shows which repo it came from + +## Environment Variables + +| Variable | Default | Purpose | +|----------|---------|---------| +| `GIT_MEMORY_DIR` | `~/memory-store` | Primary store (read-write) | +| `GIT_MEMORY_SOURCES_DIR` | `${GIT_MEMORY_DIR}-sources` | Folder of source repos (read-only search) | + +## Roadmap + +- [x] `source add/list/enable/disable/remove/sync` +- [x] Multi-source search with attribution +- [x] Folder-based storage (no JSON, no config files) +- [x] URL cloning + local symlink support +- [x] `--json` output with source field +- [x] 34 dedicated tests (test-source.sh) +- [ ] `promote --to ` — copy entry between stores + +--- + +**Implementation:** ~80 LOC (iter_sources, source_dir_for, cmd_source), pure bash, no external deps beyond git. +**Tested:** June 19, 2026 — 127/127 tests passing across 7 suites. diff --git a/README.md b/README.md index b163ee4..ea01d39 100644 --- a/README.md +++ b/README.md @@ -1,267 +1,337 @@ -# git-memory - -> An AI memory system where the emptiest repo is the fullest database. - -**The idea:** Use git empty commits as a persistent, portable, zero-dependency memory store for AI agents. No MCP server. No Node.js. No SQLite. Just `git`. - ---- - -## The Problem with Existing Memory Solutions - -| System | Lock-in | Runtime | Failure modes | -|--------|---------|---------|--------------| -| VS Code built-in `/memories/` | VS Code only | None | None | -| simple-memory MCP | Any MCP client | Node.js server | Server crash, auth, connection | -| memory-mcp (marketplace) | Any MCP client | Node.js 22+ | Same + Git credentials | -| **git-memory** | Anything with `git` | None | Disk write | - -The irony of MCP memory systems: they add a protocol layer to solve portability, but introduce server processes and runtime dependencies. `git` is already on every machine, every editor, every CI, every SAW, every phone (Termux). It needs no proxy. - ---- - -## Docs - -- [SKILL.md](SKILL.md) — agent instructions, capture heuristics, session workflow -- [docs/benchmark.md](docs/benchmark.md) — performance comparison vs simple-memory MCP (800+ memories) -- [docs/roadmap.md](docs/roadmap.md) — planned features and open design questions - ---- - -## How It Works - -The memory store is a git repository. Every memory is stored as an empty commit. The repo has no files — ever. - -``` -memory-store/ -└── .git/ ← the ENTIRE memory store lives here - (no other files) -``` - ---- - -## Install - -### One-liner - -```bash -curl -sL https://raw.githubusercontent.com/chrisribe/git-memory/main/git-mem \ - -o ~/.local/bin/git-mem && chmod +x ~/.local/bin/git-mem -``` - -### From source - -```bash -git clone https://github.com/chrisribe/git-memory.git -cd git-memory -./install.sh -``` - -The installer copies the Copilot skill to `~/.agents/skills/git-memory/SKILL.md`. -If you run the installer with `sudo`, the skill may end up under `/root/.agents/skills/git-memory` instead of your user profile and will not appear in your normal VS Code session. - -### Manual - -Copy `git-mem` anywhere on your `$PATH` and make it executable: - -```bash -mkdir -p ~/.local/bin -cp git-mem ~/.local/bin/git-mem -chmod +x ~/.local/bin/git-mem -``` - -**Windows (Git Bash):** `~/.local/bin` isn't on `$PATH` by default. Add it to your `~/.bashrc`: - -```bash -echo 'export PATH="$HOME/.local/bin:$PATH"' >> ~/.bashrc -source ~/.bashrc -``` - -### Verify - -```bash -git-mem help # standalone -git mem help # as git subcommand — both work -``` - -> Because the script is named `git-mem`, git auto-discovers it as a subcommand. `git mem add ...` and `git-mem add ...` are interchangeable. - ---- - -## Quick Start - -```bash -# Initialize memory store -git-mem init - -# Store a memory -git-mem add "[dri][cosmosdb] RU exhaustion is container ceiling not partition" - -# Store with details (opens $EDITOR) -git-mem edit - -# Search (OR — any word matches) -git-mem search cosmosdb throttle - -# Search (AND — all words must match) -git-mem search +cosmosdb +partition - -# Browse recent -git-mem recent - -# See all tags -git-mem tags - -# Stats -git-mem stats - -# Sync across machines -git-mem sync -``` - ---- - -## Commands - -| Command | What it does | -|---------|-------------| -| `git-mem init` | Initialize memory store at `~/memory-store` | -| `git-mem add "[tags] summary"` | Store a one-liner (with dedup check) | -| `git-mem add "[tags] summary" "body"` | Store with subject + body | -| `git-mem edit` | Store via `$EDITOR` (multi-line, no escaping pain) | -| `git-mem search ` | Fuzzy multi-keyword search (OR) | -| `git-mem search +word1 +word2` | AND search (all must match) | -| `git-mem show ` | Show full memory content | -| `git-mem recent [n]` | Browse recent memories (default: 20) | -| `git-mem tags` | List all tags with frequency counts | -| `git-mem stats` | Memory store statistics | -| `git-mem sync` | Safe `pull --rebase` then `push` | -| `git-mem export` | Dump all memories to stdout | - -### What the wrapper adds over raw git - -| Feature | Raw git | git-mem | -|---------|---------|---------| -| Dedup detection | ❌ | ✅ Warns before storing near-duplicates | -| Tag normalization | ❌ | ✅ Auto-lowercases `[DRI]` → `[dri]` | -| Tag validation | ❌ | ✅ Warns on missing or malformed tags | -| Multi-keyword search | One `--grep` at a time | OR and AND in one command | -| Case-insensitive search | Need `-i` flag | Always on | -| Multi-line input | Shell escaping hell | `edit` opens `$EDITOR` cleanly | -| Safe sync | Manual rebase dance | One command: `git-mem sync` | - ---- - -## Configuration - -Environment variables only. Zero config files. - -| Variable | Default | Purpose | -|----------|---------|---------| -| `GIT_MEMORY_DIR` | `~/memory-store` | Path to the memory store | -| `GIT_MEMORY_DEDUP_THRESHOLD` | `3` | Min word overlap to trigger dedup warning | - -```bash -# Use a different memory store -export GIT_MEMORY_DIR=~/work-memories -git-mem add "[dri] something work-related" -``` - ---- - -## Tag Convention - -Tags go in square brackets at the start of the subject line: - -``` -[area][subtopic] One-line summary -``` - -| Tag | Purpose | -|-----|---------| -| `[dri]` | On-call lessons | -| `[arch]` | Architecture decisions | -| `[gotcha]` | Non-obvious traps | -| `[workflow]` | Process/tooling | -| `[decision]` | Tech choices and rationale | -| `[auto]` | AI auto-captured | - -Combine freely: `[dri][cosmosdb]`, `[gotcha][build]`, `[arch][rpaas]` - ---- - -## Git Aliases (alternative to wrapper) - -If you prefer plain git over the `git-mem` script, add these to `~/.gitconfig`: - -```ini -[alias] - remember = commit --allow-empty - mem = "!f() { git commit --allow-empty -m \"$*\"; }; f" - recall = log --grep --oneline - recall-full = "!f() { git log --grep=\"$1\" --format='%C(yellow)%h%Creset %C(green)%aI%Creset%n%B%n---'; }; f" - memories = log --oneline -20 - forget = "!f() { git rebase -i \"$1\"^; }; f" - mem-stats = "!echo \"Total memories: $(git log --oneline | wc -l)\"" - mem-export = log --format="%H|%aI|%s%n%b%n---" -``` - -```bash -git mem "[dri][k8s] Pod restarts from OOM — check memory limits first" -git recall cosmosdb -git memories -``` - -> **Under the hood:** Every memory is just `git commit --allow-empty -m "..."`. Search is `git log --grep`. That's the entire system — no magic, no abstraction. The wrapper and aliases are convenience, not necessity. - ---- - -## Portability - -```bash -# Clone existing memories to a new machine -git-mem init https://github.com/you/memories.git - -# Already ran init without a URL? Re-run with the URL — it will offer to backup and replace -git-mem init https://github.com/you/memories.git - -# After setup, sync is one command -git-mem sync # pull --rebase then push - -# Offline backup (single file) -git bundle create memory-$(date +%Y%m%d).bundle --all -``` - -Works on: WSL, SAW, devbox, phone (Termux), CI, bare Linux box, anywhere. - ---- - -## For AI Agents - -See [SKILL.md](SKILL.md) for agent instructions, capture heuristics, and session workflow. - -### VS Code Copilot - -`install.sh` copies the skill to `~/.agents/skills/git-memory/SKILL.md` automatically. The skill triggers on memory-related requests. -You can also invoke it directly from Copilot Chat with `/git-memory`. - -### Claude Code - -Once the repo is public: `/plugin marketplace add chrisribe/git-memory` - -Until then, copy `SKILL.md` manually or reference it from your `CLAUDE.md`. - -### Other agents (Cursor, Windsurf, etc.) - -Copy `SKILL.md` to wherever your agent reads instructions. The file is plain markdown — works anywhere an agent can read a file. - ---- - -## Status - -Working. Wrapper script (`git-mem`), installer, 54 passing tests. See [docs/roadmap.md](docs/roadmap.md) for what's next. - ---- - -## Acknowledgments - -This project grew out of [simple-memory-mcp](https://github.com/chrisribe/simple-memory-mcp) — an MCP-based memory server I built and used as my daily driver. simple-memory taught me what mattered in a memory system (fast search, tags, dedup) and what didn't (servers, protocols, runtimes). git-memory is the "less is more" rewrite: same workflow, zero infrastructure. If you want richer features (full-text search, GraphQL API, relations), simple-memory-mcp is still the better tool. If you want something that works everywhere git does with nothing to install or keep running, this is it. +# git-memory + +> An AI memory system where the emptiest repo is the fullest database. + +**The idea:** Use git empty commits as a persistent, portable, zero-dependency memory store for AI agents. No MCP server. No Node.js. No SQLite. Just `git`. + +--- + +## The Problem with Existing Memory Solutions + +| System | Lock-in | Runtime | Failure modes | +|--------|---------|---------|--------------| +| VS Code built-in `/memories/` | VS Code only | None | None | +| simple-memory MCP | Any MCP client | Node.js server | Server crash, auth, connection | +| memory-mcp (marketplace) | Any MCP client | Node.js 22+ | Same + Git credentials | +| **git-memory** | Anything with `git` | None | Disk write | + +The irony of MCP memory systems: they add a protocol layer to solve portability, but introduce server processes and runtime dependencies. `git` is already on every machine, every editor, every CI, every SAW, every phone (Termux). It needs no proxy. + +--- + +## Docs + +- [SKILL.md](SKILL.md) — agent instructions, capture heuristics, session workflow +- [MULTI-SOURCE.md](MULTI-SOURCE.md) — federated search across multiple memory repos +- [docs/benchmark.md](docs/benchmark.md) — performance comparison vs simple-memory MCP (800+ memories) +- [docs/roadmap.md](docs/roadmap.md) — planned features and open design questions + +--- + +## How It Works + +The memory store is a git repository. Every memory is stored as an empty commit. The repo has no files — ever. + +``` +memory-store/ +└── .git/ ← the ENTIRE memory store lives here + (no other files) +``` + +--- + +## Install + +### One-liner + +```bash +curl -sL https://raw.githubusercontent.com/chrisribe/git-memory/main/git-mem \ + -o ~/.local/bin/git-mem && chmod +x ~/.local/bin/git-mem +``` + +### From source + +```bash +git clone https://github.com/chrisribe/git-memory.git +cd git-memory +./install.sh +``` + +The installer copies the Copilot skill to `~/.agents/skills/git-memory/SKILL.md`. +If you run the installer with `sudo`, the skill may end up under `/root/.agents/skills/git-memory` instead of your user profile and will not appear in your normal VS Code session. + +### Manual + +Copy `git-mem` anywhere on your `$PATH` and make it executable: + +```bash +mkdir -p ~/.local/bin +cp git-mem ~/.local/bin/git-mem +chmod +x ~/.local/bin/git-mem +``` + +**Windows (Git Bash):** `~/.local/bin` isn't on `$PATH` by default. Add it to your `~/.bashrc`: + +```bash +echo 'export PATH="$HOME/.local/bin:$PATH"' >> ~/.bashrc +source ~/.bashrc +``` + +### Verify + +```bash +git-mem help # standalone +git mem help # as git subcommand — both work +``` + +> Because the script is named `git-mem`, git auto-discovers it as a subcommand. `git mem add ...` and `git-mem add ...` are interchangeable. + +--- + +## Quick Start + +```bash +# Initialize memory store +git-mem init + +# Store a memory +git-mem add "[dri][cosmosdb] RU exhaustion is container ceiling not partition" + +# Store with details (opens $EDITOR) +git-mem edit + +# Search (OR — any word matches) +git-mem search cosmosdb throttle + +# Search (AND — all words must match) +git-mem search +cosmosdb +partition + +# Browse recent +git-mem recent + +# See all tags +git-mem tags + +# Stats +git-mem stats + +# Sync across machines +git-mem sync +``` + +--- + +## Commands + +| Command | What it does | +|---------|-------------| +| `git-mem init` | Initialize memory store at `~/memory-store` | +| `git-mem add "[tags] summary"` | Store a one-liner (with dedup check) | +| `git-mem add "[tags] summary" "body"` | Store with subject + body | +| `git-mem edit` | Store via `$EDITOR` (multi-line, no escaping pain) | +| `git-mem search ` | Fuzzy multi-keyword search (OR) | +| `git-mem search +word1 +word2` | AND search (all must match) | +| `git-mem show ` | Show full memory content | +| `git-mem recent [n]` | Browse recent memories (default: 20) | +| `git-mem tags` | List all tags with frequency counts | +| `git-mem stats` | Memory store statistics | +| `git-mem sync` | Safe `pull --rebase` then `push` | +| `git-mem source add ` | Add a read-only source (clone URL or symlink local) | +| `git-mem source list` | Show all sources with status | +| `git-mem source disable ` | Exclude source from search | +| `git-mem source enable ` | Re-include source in search | +| `git-mem source remove ` | Remove source (unlink or delete) | +| `git-mem source sync [name]` | Pull latest for sources | +| `git-mem export` | Dump all memories to stdout | + +### What the wrapper adds over raw git + +| Feature | Raw git | git-mem | +|---------|---------|---------| +| Dedup detection | ❌ | ✅ Warns before storing near-duplicates | +| Tag normalization | ❌ | ✅ Auto-lowercases `[DRI]` → `[dri]` | +| Tag validation | ❌ | ✅ Warns on missing or malformed tags | +| Multi-keyword search | One `--grep` at a time | OR and AND in one command | +| Case-insensitive search | Need `-i` flag | Always on | +| Multi-line input | Shell escaping hell | `edit` opens `$EDITOR` cleanly | +| Safe sync | Manual rebase dance | One command: `git-mem sync` | + +--- + +## Configuration + +Environment variables only. Zero config files. + +| Variable | Default | Purpose | +|----------|---------|---------| +| `GIT_MEMORY_DIR` | `~/memory-store` | Path to the primary memory store | +| `GIT_MEMORY_SOURCES_DIR` | `${GIT_MEMORY_DIR}-sources` | Folder of source repos for federated search | +| `GIT_MEMORY_DEDUP_THRESHOLD` | `3` | Min word overlap to trigger dedup warning | + +```bash +# Use a different memory store +export GIT_MEMORY_DIR=~/work-memories +git-mem add "[dri] something work-related" +``` + +--- + +## Tag Convention + +Tags go in square brackets at the start of the subject line: + +``` +[area][subtopic] One-line summary +``` + +| Tag | Purpose | +|-----|---------| +| `[dri]` | On-call lessons | +| `[arch]` | Architecture decisions | +| `[gotcha]` | Non-obvious traps | +| `[workflow]` | Process/tooling | +| `[decision]` | Tech choices and rationale | +| `[auto]` | AI auto-captured | + +Combine freely: `[dri][cosmosdb]`, `[gotcha][build]`, `[arch][rpaas]` + +--- + +## Git Aliases (alternative to wrapper) + +If you prefer plain git over the `git-mem` script, add these to `~/.gitconfig`: + +```ini +[alias] + remember = commit --allow-empty + mem = "!f() { git commit --allow-empty -m \"$*\"; }; f" + recall = log --grep --oneline + recall-full = "!f() { git log --grep=\"$1\" --format='%C(yellow)%h%Creset %C(green)%aI%Creset%n%B%n---'; }; f" + memories = log --oneline -20 + forget = "!f() { git rebase -i \"$1\"^; }; f" + mem-stats = "!echo \"Total memories: $(git log --oneline | wc -l)\"" + mem-export = log --format="%H|%aI|%s%n%b%n---" +``` + +```bash +git mem "[dri][k8s] Pod restarts from OOM — check memory limits first" +git recall cosmosdb +git memories +``` + +> **Under the hood:** Every memory is just `git commit --allow-empty -m "..."`. Search is `git log --grep`. That's the entire system — no magic, no abstraction. The wrapper and aliases are convenience, not necessity. + +--- + +## Portability + +```bash +# Clone existing memories to a new machine +git-mem init https://github.com/you/memories.git + +# Already ran init without a URL? Re-run with the URL — it will offer to backup and replace +git-mem init https://github.com/you/memories.git + +# After setup, sync is one command +git-mem sync # pull --rebase then push + +# Offline backup (single file) +git bundle create memory-$(date +%Y%m%d).bundle --all +``` + +Works on: WSL, SAW, devbox, phone (Termux), CI, bare Linux box, anywhere. + +--- + +## Multi-Source (Federated Search) + +Search your primary store plus additional memory repos — without merging them. Sources are **read-only**: `add` always writes to the primary store only. + +```bash +# Add a community knowledge base +git-mem source add sql-expert https://github.com/someone/sql-patterns-memories + +# Add a team's shared gotchas +git-mem source add team /path/to/team-memories + +# Search now hits your memories + all enabled sources +git-mem search redis +# a1b2c3d Redis timeout config +# [team] e6f0c08 [team][gotcha] Redis needs 30s minimum timeout + +# Manage sources +git-mem source list +git-mem source disable team # exclude from search (renames dir) +git-mem source enable team # re-include +git-mem source remove team # unlink symlink; original untouched +git-mem source sync # pull latest for all sources +``` + +### Use cases + +**Knowledge domains** — add curated memory repos for specific expertise: +```bash +git-mem source add sql-expert https://github.com/someone/sql-patterns +git-mem source add k8s-gotchas https://github.com/someone/k8s-learnings +``` + +**Team sharing** — share project context without exposing personal memories: +```bash +git-mem source add team /mnt/shared/team-memories +git-mem source add daniel ~/external/daniel-devops-notes +``` + +**Isolated contexts** — each agent, project, or workflow gets its own store: +```bash +# Agent with its own memory, reading yours as a source +export GIT_MEMORY_DIR=~/hermes-memory +git-mem init +git-mem source add personal ~/memory-store + +# Work vs personal separation +export GIT_MEMORY_DIR=~/work-memories +git-mem source add personal ~/memory-store +``` + +**Project archival** — keep a dying project's learnings searchable without contaminating your daily store: +```bash +# Archive a project's memories as a source (disable when noisy) +git-mem source add project-atlas ~/old-projects/atlas-memories +git-mem source disable project-atlas # stop searching, keep the repo +git-mem source enable project-atlas # bring it back when needed +``` + +Sources live in `${GIT_MEMORY_DIR}-sources/` by default. Override with `GIT_MEMORY_SOURCES_DIR`. + +--- + +## For AI Agents + +See [SKILL.md](SKILL.md) for agent instructions, capture heuristics, and session workflow. + +### VS Code Copilot + +`install.sh` copies the skill to `~/.agents/skills/git-memory/SKILL.md` automatically. The skill triggers on memory-related requests. +You can also invoke it directly from Copilot Chat with `/git-memory`. + +### Claude Code + +Once the repo is public: `/plugin marketplace add chrisribe/git-memory` + +Until then, copy `SKILL.md` manually or reference it from your `CLAUDE.md`. + +### Other agents (Cursor, Windsurf, etc.) + +Copy `SKILL.md` to wherever your agent reads instructions. The file is plain markdown — works anywhere an agent can read a file. + +--- + +## Status + +Working. Wrapper script (`git-mem`), installer, 127 passing tests. See [docs/roadmap.md](docs/roadmap.md) for what's next. + +--- + +## Acknowledgments + +This project grew out of [simple-memory-mcp](https://github.com/chrisribe/simple-memory-mcp) — an MCP-based memory server I built and used as my daily driver. simple-memory taught me what mattered in a memory system (fast search, tags, dedup) and what didn't (servers, protocols, runtimes). git-memory is the "less is more" rewrite: same workflow, zero infrastructure. If you want richer features (full-text search, GraphQL API, relations), simple-memory-mcp is still the better tool. If you want something that works everywhere git does with nothing to install or keep running, this is it. diff --git a/SKILL.md b/SKILL.md index 14fdfba..4de9082 100644 --- a/SKILL.md +++ b/SKILL.md @@ -1,186 +1,175 @@ ---- -name: git-memory -description: "Zero-dependency persistent memory using git empty commits. Use when the user wants to remember information across sessions, store learnings, save context for later, or build a personal knowledge base. Also use when: 'remember this', 'save for later', 'don't forget', memory store, knowledge persistence, or any request to preserve information beyond the current conversation. Use even when the user doesn't say 'memory' explicitly — any desire to retain knowledge across sessions qualifies. Works offline, syncs via git, requires only git CLI." -user-invocable: true ---- - -# git-memory - -Persistent memory for AI agents via git empty commits. - -## Setup - -Always use the `git-mem` CLI wrapper — it handles dedup detection, tag normalization, and consistent formatting. - -```bash -git-mem init # first time only — creates ~/memory-store -``` - -Store location: `~/memory-store` (override with `GIT_MEMORY_DIR`). - -## Commands - -### Store - -```bash -git-mem add "[tags] summary" # one-liner -git-mem add "[tags] summary" "Single-line body." # with body (single line only) -git-mem add --yes "[auto][tags] summary" # non-interactive (agent use) -echo "Multi-line body here." | git-mem add --yes --stdin "[tags] summary" # multi-line safe -``` - -> **Multi-line bodies:** Always use `--stdin` with a pipe for bodies containing newlines. The positional body argument is truncated at the first newline on Windows due to `.cmd` argument passing limitations. - -### Search - -Default to AND search — OR gets noisy with 2+ terms because it matches any word. - -```bash -git-mem search +cosmosdb +partition # AND: all words must match -git-mem search cosmosdb throttle # OR: any word matches -``` - -### Data quality check - -When retrieving memories, watch for **truncation signals** — bodies that were silently clipped: - -- Body ends with `:` (introducing a list that got cut off) -- Body is a single line when the subject implies detailed content (e.g., "full recipe", "complete checklist", "architecture") -- Body ends mid-sentence or mid-word - -If you detect truncation, **tell the user** instead of silently working with partial data: - -> "⚠️ This memory appears truncated (body ends with `:`). Want me to search sessions or the codebase for the full context?" - -### Other commands - -```bash -git-mem recent 20 # browse recent (run at session start for context) -git-mem show # full memory content -git-mem forget # retract a memory (append-only, reversible) -git-mem forget --reason "…" # retract with reason -git-mem resurface # list retracted memories -git-mem resurface cosmosdb # search retracted memories -git-mem resurface --restore # restore a retracted memory -git-mem tags # list all tags -git-mem stats # store statistics -git-mem sync # push/pull across machines -git-mem export # export all memories -``` - -## Connect an existing memories repo - -Pass the remote URL to `init` — it clones the repo directly: - -```bash -git-mem init https://github.com/you/memories.git -``` - -If you already ran `git-mem init` without a URL and want to start over: - -```bash -rm -rf ~/memory-store -git-mem init https://github.com/you/memories.git -``` - -## Fallback: raw git (only if git-mem is not on PATH) - -Only use these if `git-mem` is genuinely unavailable (e.g., not installed). Prefer `git-mem` — it adds dedup checks and tag normalization that raw git lacks. - -```bash -git -C ~/memory-store commit --allow-empty -m "[tags] summary" # store -git -C ~/memory-store log --oneline -i --grep "keyword" # search -git -C ~/memory-store log --oneline -20 # recent -git -C ~/memory-store log -1 --format="%B" # show -git -C ~/memory-store pull --rebase --autostash && git -C ~/memory-store push # sync -``` - -## Subject line format - -The subject is what appears in search results, so it must stand alone — a vague subject means the memory is effectively lost. - -``` -[tags] Keyword-rich summary that stands alone -``` - -``` -Good: [dri][cosmosdb] RU exhaustion ≠ hot partition — check autoscale ceiling -Bad: [dri] Investigation notes -``` - -## Tags - -Format: `[area][subtopic] Summary`. Combine freely. Auto-normalized to lowercase. - -| Tag | Purpose | -|-----|---------| -| `[dri]` | On-call / incident learnings | -| `[arch]` | Architecture decisions | -| `[gotcha]` | Non-obvious traps | -| `[workflow]` | Process / tooling patterns | -| `[decision]` | Tech choices with rationale | -| `[auto]` | AI auto-captured | - -## What to save vs skip - -The quality gate: "Would this save future-me 10+ minutes of investigation?" This filters out the ~80% of potential memories that are noise. - -**Save** — corrections to wrong mental models, non-obvious gotchas, architecture decisions with rationale, DRI root causes, API quirks not in docs, cost/perf numbers - -**Skip** — anything already in docs/on disk, ephemeral content (playlists, brainstorms, one-time plans), WIP snapshots (save conclusions not journeys), things that change soon, preferences already in user profile - -### Auto-capture (`[auto]` tag) - -Apply a higher bar because historically 80% of auto-captures were noise. Only auto-save when: -- User corrected a wrong assumption -- A multi-step debugging session reached resolution -- A non-obvious gotcha was discovered - -If unsure, don't save — the user can always say "remember this." - -## When to forget - -Memories are never truly deleted — `forget` appends a retraction commit. The original is hidden from search/recent but still exists in git history. `resurface --restore` brings it back. - -**Forget** — superseded knowledge (you learned the real answer), one-off incident context after resolution, noisy `[auto]` captures that failed the quality bar, wrong mental models you don't want polluting future searches - -**Don't forget** — anything you're unsure about (use `resurface` later to review), root cause learnings even if the system changed, architecture decisions (the rationale still matters) - -```bash -git-mem forget abc1234 # retract -git-mem forget abc1234 --reason "wrong — real cause was X" -git-mem resurface # browse retracted pool -git-mem resurface --restore abc1234 # bring it back -``` - -## Session workflow - -1. **Start:** `git-mem recent 20` — load context -2. **During:** Store non-obvious, reusable learnings immediately -3. **End:** Nothing needed — memories persist - -## Branches as thought spaces (optional) - -Use branches to separate knowledge by confidence level or scope: - -| Branch pattern | Purpose | Merge to main? | -|----------------|---------|-----------------| -| `main` | Verified knowledge | — | -| `brainstorm/*` | Exploratory ideas | Cherry-pick survivors | -| `incident/*` | Incident context | Merge DRI lessons only | -| `project/*` | Project-scoped | Merge learnings, archive | - -## Example - -Discover RU exhaustion ≠ hot partition during CosmosDB investigation: - -```bash -git-mem add "[dri][cosmosdb] RU exhaustion ≠ hot partition — check autoscale ceiling" \ - "100% normalized RU can mean container-level ceiling hit, not partition hotspot. Fix: increase autoScaleMaxThroughput in Bicep, not partition key redesign." -``` - -Later, user hits CosmosDB 408s: -```bash -git-mem search +cosmosdb -git-mem show -``` +--- +name: git-memory +description: "Zero-dependency persistent memory using git empty commits. Use when the user wants to remember information across sessions, store learnings, save context for later, or build a personal knowledge base. Also use when: 'remember this', 'save for later', 'don't forget', memory store, knowledge persistence, or any request to preserve information beyond the current conversation. Use even when the user doesn't say 'memory' explicitly — any desire to retain knowledge across sessions qualifies. Works offline, syncs via git, requires only git CLI." +user-invocable: true +--- + +# git-memory + +Persistent memory for AI agents via git empty commits. + +## Setup + +Always use the `git-mem` CLI wrapper — it handles dedup detection, tag normalization, and consistent formatting. + +```bash +git-mem init # first time only — creates ~/memory-store +``` + +Store location: `~/memory-store` (override with `GIT_MEMORY_DIR`). +Sources location: `~/memory-store-sources` (override with `GIT_MEMORY_SOURCES_DIR`). + +## Commands + +### Store + +```bash +git-mem add "[tags] summary" # one-liner +git-mem add "[tags] summary" "Single-line body." # with body (single line only) +git-mem add --yes "[auto][tags] summary" # non-interactive (agent use) +echo "Multi-line body here." | git-mem add --yes --stdin "[tags] summary" # multi-line safe +``` + +> **Multi-line bodies:** Always use `--stdin` with a pipe for bodies containing newlines. The positional body argument is truncated at the first newline on Windows due to `.cmd` argument passing limitations. + +### Search + +Default to AND search — OR gets noisy with 2+ terms because it matches any word. + +```bash +git-mem search +cosmosdb +partition # AND: all words must match +git-mem search cosmosdb throttle # OR: any word matches +``` + +### Other commands + +```bash +git-mem recent 20 # browse recent (run at session start for context) +git-mem show # full memory content +git-mem forget # retract a memory (append-only, reversible) +git-mem forget --reason "…" # retract with reason +git-mem resurface # list retracted memories +git-mem resurface cosmosdb # search retracted memories +git-mem resurface --restore # restore a retracted memory +git-mem tags # list all tags +git-mem stats # store statistics +git-mem sync # push/pull across machines +git-mem export # export all memories +``` + +## Connect an existing memories repo + +Pass the remote URL to `init` — it clones the repo directly: + +```bash +git-mem init https://github.com/you/memories.git +``` + +If you already ran `git-mem init` without a URL and want to start over: + +```bash +rm -rf ~/memory-store +git-mem init https://github.com/you/memories.git +``` + +> **Fallback:** If `git-mem` isn't installed, use `git -C ~/memory-store commit --allow-empty -m "[tags] summary"` — but you lose dedup checks and tag normalization. + +## Subject line format + +The subject is what appears in search results, so it must stand alone — a vague subject means the memory is effectively lost. + +``` +[tags] Keyword-rich summary that stands alone +``` + +``` +Good: [dri][cosmosdb] RU exhaustion ≠ hot partition — check autoscale ceiling +Bad: [dri] Investigation notes +``` + +## Tags + +Format: `[area][subtopic] Summary`. Combine freely. Auto-normalized to lowercase. + +| Tag | Purpose | +|-----|---------| +| `[dri]` | On-call / incident learnings | +| `[arch]` | Architecture decisions | +| `[gotcha]` | Non-obvious traps | +| `[workflow]` | Process / tooling patterns | +| `[decision]` | Tech choices with rationale | +| `[auto]` | AI auto-captured | + +## What to save vs skip + +The quality gate: "Would this save future-me 10+ minutes of investigation?" This filters out the ~80% of potential memories that are noise. + +**Save** — corrections to wrong mental models, non-obvious gotchas, architecture decisions with rationale, DRI root causes, API quirks not in docs, cost/perf numbers + +**Skip** — anything already in docs/on disk, ephemeral content (playlists, brainstorms, one-time plans), WIP snapshots (save conclusions not journeys), things that change soon, preferences already in user profile + +### Auto-capture (`[auto]` tag) + +Apply a higher bar because historically 80% of auto-captures were noise. Only auto-save when: +- User corrected a wrong assumption +- A multi-step debugging session reached resolution +- A non-obvious gotcha was discovered + +If unsure, don't save — the user can always say "remember this." + +## When to forget + +Use `git-mem forget ` to retract superseded knowledge or noisy `[auto]` captures. Don't forget architecture decisions or root causes — the rationale still matters. Retraction is reversible: `git-mem resurface --restore `. + +## Session workflow + +1. **Start:** `git-mem recent 20` — load context +2. **During:** Store non-obvious, reusable learnings immediately +3. **End:** Nothing needed — memories persist + + +## Example + +Discover RU exhaustion ≠ hot partition during CosmosDB investigation: + +```bash +git-mem add "[dri][cosmosdb] RU exhaustion ≠ hot partition — check autoscale ceiling" \ + "100% normalized RU can mean container-level ceiling hit, not partition hotspot. Fix: increase autoScaleMaxThroughput in Bicep, not partition key redesign." +``` + +Later, user hits CosmosDB 408s: +```bash +git-mem search +cosmosdb +git-mem show +``` + +## Multi-source (federated search) + +Search your primary store **plus** additional read-only memory repos. Writes always go to the primary store only — sources are never modified. + +### Commands + +```bash +git-mem source add # symlink local path or clone URL +git-mem source list # show all sources (enabled/disabled) +git-mem source disable # exclude from search (keeps repo) +git-mem source enable # re-include in search +git-mem source remove # unlink symlink or delete clone +git-mem source sync [name] # git pull for one or all sources +``` + +### Use cases + +- **Knowledge domains** — add curated repos: `git-mem source add sql-expert https://...` +- **Team sharing** — `git-mem source add team /shared/team-memories` +- **Isolated contexts** — agents/projects get their own `GIT_MEMORY_DIR`, link others as read-only +- **Project archival** — `git-mem source add old-project ~/archive/atlas-memories` then disable when noisy + +### Environment variables + +| Variable | Default | Purpose | +|----------|---------|--------| +| `GIT_MEMORY_DIR` | `~/memory-store` | Primary store (read-write) | +| `GIT_MEMORY_SOURCES_DIR` | `${GIT_MEMORY_DIR}-sources` | Folder of source repos (read-only search) | + +Sources are just git repos in the sources folder. Folder name = source name. Append `.disabled` to the folder name to exclude from search. diff --git a/docs/UBIQUITOUS_LANGUAGE.md b/docs/UBIQUITOUS_LANGUAGE.md index 830aec1..133f77f 100644 --- a/docs/UBIQUITOUS_LANGUAGE.md +++ b/docs/UBIQUITOUS_LANGUAGE.md @@ -1,56 +1,56 @@ -# Ubiquitous Language - -## Core Concepts - -| Term | Definition | Aliases to avoid | -|------|------------|------------------| -| **Memory** | A single fact, learning, or decision stored as an empty git commit | Note, entry, commit, record | -| **Memory store** | The git repository containing all memories (default: `~/memory-store`) | Database, repo, vault | -| **Decision cache** | The conceptual role of git-memory: storing conclusions and learnings, not raw information | Search index, knowledge base, RAG | - -## Memory Structure - -| Term | Definition | Aliases to avoid | -|------|------------|------------------| -| **Subject** | The first line of a memory — must stand alone and be keyword-rich for search | Summary, title, message, headline | -| **Body** | Optional detail text after the subject, separated by blank line | Details, content, description | -| **Tag** | Bracketed prefix on the subject (e.g., `[dri][auto]`) for categorization | Label, category | - -## Memory Lifecycle - -| Term | Definition | Aliases to avoid | -|------|------------|------------------| -| **Store** | Create a new memory via `git-mem add` | Save, write, commit | -| **Forget** | Soft-delete a memory via `git-mem forget` — excluded from search but recoverable | Delete, remove, retract, archive | -| **Resurface** | Search or restore retracted memories | Undelete, recover | -| **Dedup** | Check for similar existing memories before storing | Duplicate check | -| **Sync** | Pull (rebase) then push to remote | Push, backup | - -## Actors - -| Term | Definition | Aliases to avoid | -|------|------------|------------------| -| **Agent** | An AI assistant using git-memory for persistent context | LLM, AI, model, bot | -| **Session** | One continuous conversation between a user and an agent | Chat, conversation, thread | - -## Relationships - -- A **Memory store** contains zero or more **Memories** -- A **Memory** has exactly one **Subject** and optionally one **Body** -- A **Memory** has zero or more **Tags** -- A **Forgotten** memory is still a **Memory** — just excluded from normal search -- An **Agent** reads from and writes to one **Memory store** per **Session** - -## Example Dialogue - -> **Dev:** "When an agent stores a memory, does it search for dupes first?" -> -> **Domain expert:** "Yes — `git-mem add` runs a **dedup** check against existing **subjects**. If word overlap exceeds the threshold, it warns. Use `--yes` to skip the prompt." -> -> **Dev:** "What if I accidentally stored something wrong? Can I delete it?" -> -> **Domain expert:** "No deletion — we **forget** it. The **memory** stays in history but is excluded from search. You can **resurface** it later if needed." -> -> **Dev:** "So git-memory is like a search index for my notes?" -> -> **Domain expert:** "No — it's a **decision cache**. A search index finds existing information. Git-memory stores your conclusions: what you learned, what you decided, what to avoid next time. The agent doesn't search the web or codebase — it recalls what you already figured out." +# Ubiquitous Language + +## Core Concepts + +| Term | Definition | Aliases to avoid | +|------|------------|------------------| +| **Memory** | A single fact, learning, or decision stored as an empty git commit | Note, entry, commit, record | +| **Memory store** | The git repository containing all memories (default: `~/memory-store`) | Database, repo, vault | +| **Decision cache** | The conceptual role of git-memory: storing conclusions and learnings, not raw information | Search index, knowledge base, RAG | + +## Memory Structure + +| Term | Definition | Aliases to avoid | +|------|------------|------------------| +| **Subject** | The first line of a memory — must stand alone and be keyword-rich for search | Summary, title, message, headline | +| **Body** | Optional detail text after the subject, separated by blank line | Details, content, description | +| **Tag** | Bracketed prefix on the subject (e.g., `[dri][auto]`) for categorization | Label, category | + +## Memory Lifecycle + +| Term | Definition | Aliases to avoid | +|------|------------|------------------| +| **Store** | Create a new memory via `git-mem add` | Save, write, commit | +| **Forget** | Soft-delete a memory via `git-mem forget` — excluded from search but recoverable | Delete, remove, retract, archive | +| **Resurface** | Search or restore retracted memories | Undelete, recover | +| **Dedup** | Check for similar existing memories before storing | Duplicate check | +| **Sync** | Pull (rebase) then push to remote | Push, backup | + +## Actors + +| Term | Definition | Aliases to avoid | +|------|------------|------------------| +| **Agent** | An AI assistant using git-memory for persistent context | LLM, AI, model, bot | +| **Session** | One continuous conversation between a user and an agent | Chat, conversation, thread | + +## Relationships + +- A **Memory store** contains zero or more **Memories** +- A **Memory** has exactly one **Subject** and optionally one **Body** +- A **Memory** has zero or more **Tags** +- A **Forgotten** memory is still a **Memory** — just excluded from normal search +- An **Agent** reads from and writes to one **Memory store** per **Session** + +## Example Dialogue + +> **Dev:** "When an agent stores a memory, does it search for dupes first?" +> +> **Domain expert:** "Yes — `git-mem add` runs a **dedup** check against existing **subjects**. If word overlap exceeds the threshold, it warns. Use `--yes` to skip the prompt." +> +> **Dev:** "What if I accidentally stored something wrong? Can I delete it?" +> +> **Domain expert:** "No deletion — we **forget** it. The **memory** stays in history but is excluded from search. You can **resurface** it later if needed." +> +> **Dev:** "So git-memory is like a search index for my notes?" +> +> **Domain expert:** "No — it's a **decision cache**. A search index finds existing information. Git-memory stores your conclusions: what you learned, what you decided, what to avoid next time. The agent doesn't search the web or codebase — it recalls what you already figured out." diff --git a/docs/benchmark.md b/docs/benchmark.md index 894ad78..6c3b50a 100644 --- a/docs/benchmark.md +++ b/docs/benchmark.md @@ -1,176 +1,176 @@ -# git-memory Benchmark & Comparison - -> Benchmarked April 13, 2026 on Windows / Git Bash - -Validation of git-memory against simple-memory MCP — the most widely used MCP-based memory system. Tests run against a real corpus of 800+ memories migrated from simple-memory. - ---- - -## Migration - -| Metric | Result | -|--------|--------| -| Memories migrated | 800+ | -| Date range | 8 months | -| Unique tags | 1,500+ | -| Method | `git fast-import` (single process) | -| Time | **0.3s** | -| Data loss | None — all content, tags, and timestamps preserved | - -The initial approach (one `git commit` subprocess per memory) ran at ~3/s. Switching to `git fast-import` achieved 2,600+/s. - ---- - -## Search Quality - -Identical queries run against both systems with comparable result limits. - -| Query | simple-memory (limit:50) | git-memory OR | git-memory AND | -|-------|--------------------------|--------------|----------------| -| `cosmosdb` | 29 | 22 | — | -| `dri incident` | 30+ | noisy (broad matches) | 7 precise | -| `certificate tls` | 20+ | 9 broad | 9 precise | -| `memory mcp` | 20+ | 14 | — | - -### Observations - -- **simple-memory** returns relevance-ranked results. Defaults to 10; must pass `limit:50` for full recall. -- **git-memory OR** returns all matches unranked. Broad terms pull in noise. -- **git-memory AND** (`+word1 +word2`) is the precision tool. Results are clean and specific. -- **Recall quality is equivalent** when both systems are tuned (AND in git-memory, limit:50 in simple-memory). - ---- - -## Performance at Scale (800+ memories) - -| Operation | Before fixes | After fixes | Speedup | -|-----------|-------------|-------------|---------| -| AND search | 72.6s | **0.7s** | 100x | -| `add` with dedup check | hung indefinitely | **0.9s** | ∞ | -| OR search (single keyword) | 0.4s | 0.4s | — | -| OR search (rare keyword, 56 results) | 0.6s | 0.6s | — | -| Search (no results) | 0.4s | 0.4s | — | - -### Root causes fixed - -1. **AND search** spawned one `git log -1` subprocess per candidate commit. Fix: `git log --all-match --grep=X --grep=Y` — one process, sub-second. -2. **Dedup check** iterated all commits × all words in a nested bash loop. Fix: `git log --grep` per word + `awk` frequency counting. -3. **OR dedup** used O(n²) bash array scan. Fix: `awk '!seen[$1]++'` pipe. - ---- - -## Agent Round-Trip - -Store a memory, then search and retrieve it — the core agent workflow. - -### Store - -| | simple-memory MCP | git-memory | -|---|---|---| -| Mechanism | 1 MCP tool call | 1 terminal command | -| Time | instant (network-bound) | **0.8s** (local, includes dedup) | - -### Search - -| | simple-memory MCP | git-memory | -|---|---|---| -| Mechanism | GraphQL query | `git-mem search --json` | -| Time | instant | **0.17s** (text) / **0.28s** (JSON) | -| Output | `{"data":{"memories":[...]}}` | `[{"hash":"...","date":"...","subject":"..."}]` | - -### Full content fetch - -| | simple-memory MCP | git-memory | -|---|---|---| -| Mechanism | `memory(hash:"...")` | `git-mem show --json ` | -| Time | instant | **0.08s** (text) / **0.27s** (JSON) | -| Output | JSON with content, tags, createdAt | JSON with hash, date, subject, body | - -### Token efficiency - -| | simple-memory MCP | git-memory --json | -|---|---|---| -| Search (7 results) | ~820 chars | ~1,100 chars | -| Show (full body) | ~1,450 chars (title duplicated) | ~1,100 chars (no duplication) | - ---- - -## Cold Start - -Time from zero (no install) to first memory stored and searchable. - -| | simple-memory MCP | git-memory | -|---|---|---| -| Prerequisites | Node.js, npm | git, bash | -| Install steps | npm install + MCP config + VS Code restart | curl one file + chmod | -| VS Code restart | Required | Not required | -| Time to first memory | Minutes | **0.6s** | -| Works on SAW/devbox | ⚠️ Needs Node.js | ✅ git is pre-installed | -| Works in CI | Needs MCP runtime | ✅ Just bash | -| Works offline | ❌ (MCP server dependency) | ✅ | -| Works on Termux (mobile) | ⚠️ | ✅ | - ---- - -## Unit Tests - -54 tests across 4 suites, all passing. - -| Suite | Tests | Coverage | -|-------|-------|----------| -| test-basic.sh | 25 | init, add, tags, stats, export, show, recent, help, errors | -| test-dedup.sh | 6 | duplicate detection, false positives, short messages, --yes mode | -| test-search.sh | 12 | OR, AND, case insensitivity, tag search, dedup, edge cases | -| test-sync.sh | 11 | remote setup, pull/push, divergent history, data loss check | - ---- - -## Auto-Capture Quality Audit - -Audited all memories tagged `[auto]` in a production simple-memory MCP store (10 memories, one week). - -| Category | Count | % | -|----------|------:|--:| -| Genuinely valuable | 2 | 20% | -| Context dump (stale quickly) | 2 | 20% | -| Trivial/noise | 5 | 50% | -| Duplicate | 1 | 10% | - -### Pattern - -The 2 valuable auto-saves both **corrected a wrong mental model** — the kind of thing you'd want to remember. The 5 noise entries were ephemeral session activity the agent happened to observe. The heuristic doesn't distinguish "user discovered something important" from ambient activity. - -### Impact on the comparison - -Silent auto-capture (often cited as an MCP advantage) is actually a liability at this signal-to-noise ratio. Terminal-visible saves (git-memory) let you notice and reject noise in real time. SKILL.md auto-capture rules are tightened to require a higher bar for `[auto]` tagged saves. - ---- - -## Where git-memory Wins - -- **Zero dependencies** — works anywhere git exists -- **Offline-first** — no server, no network needed -- **Cold start** — 0.6s from nothing to first memory -- **Transparency** — every memory is a git commit, inspectable with any git tool -- **Portability** — SAW, devbox, Termux, CI, bare Linux -- **No compile/rebuild cycle** — edit SKILL.md or the bash script and you're done. MCP needs TypeScript rebuild + server restart. -- **Visible auto-capture** — terminal commands let you catch noise. Silent MCP saves accumulate junk unnoticed. -- **Dedup built-in** — warns before storing near-duplicates. MCP has none. -- **Cross-agent compatibility** — SKILL.md works in VS Code Copilot, Claude Code, Cursor, any agent that reads files. MCP only works in MCP-compatible clients. -- **No maintenance** — no server process to monitor, restart, or debug. - -## Where simple-memory MCP Wins - -- **Structured API** — native tool calls, no terminal parsing -- **Relevance ranking** — results scored by relevance, not just grep matches - -## Where They're Equal - -- Recall quality (when tuned: AND search vs limit:50) -- Token efficiency (~same chars per result) -- Full content retrieval (identical data) -- JSON output (git-memory `--json` matches MCP structured responses) - ---- - -**Not yet tested:** failure recovery, parallel write safety, cross-platform (macOS/Linux/WSL2). +# git-memory Benchmark & Comparison + +> Benchmarked April 13, 2026 on Windows / Git Bash + +Validation of git-memory against simple-memory MCP — the most widely used MCP-based memory system. Tests run against a real corpus of 800+ memories migrated from simple-memory. + +--- + +## Migration + +| Metric | Result | +|--------|--------| +| Memories migrated | 800+ | +| Date range | 8 months | +| Unique tags | 1,500+ | +| Method | `git fast-import` (single process) | +| Time | **0.3s** | +| Data loss | None — all content, tags, and timestamps preserved | + +The initial approach (one `git commit` subprocess per memory) ran at ~3/s. Switching to `git fast-import` achieved 2,600+/s. + +--- + +## Search Quality + +Identical queries run against both systems with comparable result limits. + +| Query | simple-memory (limit:50) | git-memory OR | git-memory AND | +|-------|--------------------------|--------------|----------------| +| `cosmosdb` | 29 | 22 | — | +| `dri incident` | 30+ | noisy (broad matches) | 7 precise | +| `certificate tls` | 20+ | 9 broad | 9 precise | +| `memory mcp` | 20+ | 14 | — | + +### Observations + +- **simple-memory** returns relevance-ranked results. Defaults to 10; must pass `limit:50` for full recall. +- **git-memory OR** returns all matches unranked. Broad terms pull in noise. +- **git-memory AND** (`+word1 +word2`) is the precision tool. Results are clean and specific. +- **Recall quality is equivalent** when both systems are tuned (AND in git-memory, limit:50 in simple-memory). + +--- + +## Performance at Scale (800+ memories) + +| Operation | Before fixes | After fixes | Speedup | +|-----------|-------------|-------------|---------| +| AND search | 72.6s | **0.7s** | 100x | +| `add` with dedup check | hung indefinitely | **0.9s** | ∞ | +| OR search (single keyword) | 0.4s | 0.4s | — | +| OR search (rare keyword, 56 results) | 0.6s | 0.6s | — | +| Search (no results) | 0.4s | 0.4s | — | + +### Root causes fixed + +1. **AND search** spawned one `git log -1` subprocess per candidate commit. Fix: `git log --all-match --grep=X --grep=Y` — one process, sub-second. +2. **Dedup check** iterated all commits × all words in a nested bash loop. Fix: `git log --grep` per word + `awk` frequency counting. +3. **OR dedup** used O(n²) bash array scan. Fix: `awk '!seen[$1]++'` pipe. + +--- + +## Agent Round-Trip + +Store a memory, then search and retrieve it — the core agent workflow. + +### Store + +| | simple-memory MCP | git-memory | +|---|---|---| +| Mechanism | 1 MCP tool call | 1 terminal command | +| Time | instant (network-bound) | **0.8s** (local, includes dedup) | + +### Search + +| | simple-memory MCP | git-memory | +|---|---|---| +| Mechanism | GraphQL query | `git-mem search --json` | +| Time | instant | **0.17s** (text) / **0.28s** (JSON) | +| Output | `{"data":{"memories":[...]}}` | `[{"hash":"...","date":"...","subject":"..."}]` | + +### Full content fetch + +| | simple-memory MCP | git-memory | +|---|---|---| +| Mechanism | `memory(hash:"...")` | `git-mem show --json ` | +| Time | instant | **0.08s** (text) / **0.27s** (JSON) | +| Output | JSON with content, tags, createdAt | JSON with hash, date, subject, body | + +### Token efficiency + +| | simple-memory MCP | git-memory --json | +|---|---|---| +| Search (7 results) | ~820 chars | ~1,100 chars | +| Show (full body) | ~1,450 chars (title duplicated) | ~1,100 chars (no duplication) | + +--- + +## Cold Start + +Time from zero (no install) to first memory stored and searchable. + +| | simple-memory MCP | git-memory | +|---|---|---| +| Prerequisites | Node.js, npm | git, bash | +| Install steps | npm install + MCP config + VS Code restart | curl one file + chmod | +| VS Code restart | Required | Not required | +| Time to first memory | Minutes | **0.6s** | +| Works on SAW/devbox | ⚠️ Needs Node.js | ✅ git is pre-installed | +| Works in CI | Needs MCP runtime | ✅ Just bash | +| Works offline | ❌ (MCP server dependency) | ✅ | +| Works on Termux (mobile) | ⚠️ | ✅ | + +--- + +## Unit Tests + +54 tests across 4 suites, all passing. + +| Suite | Tests | Coverage | +|-------|-------|----------| +| test-basic.sh | 25 | init, add, tags, stats, export, show, recent, help, errors | +| test-dedup.sh | 6 | duplicate detection, false positives, short messages, --yes mode | +| test-search.sh | 12 | OR, AND, case insensitivity, tag search, dedup, edge cases | +| test-sync.sh | 11 | remote setup, pull/push, divergent history, data loss check | + +--- + +## Auto-Capture Quality Audit + +Audited all memories tagged `[auto]` in a production simple-memory MCP store (10 memories, one week). + +| Category | Count | % | +|----------|------:|--:| +| Genuinely valuable | 2 | 20% | +| Context dump (stale quickly) | 2 | 20% | +| Trivial/noise | 5 | 50% | +| Duplicate | 1 | 10% | + +### Pattern + +The 2 valuable auto-saves both **corrected a wrong mental model** — the kind of thing you'd want to remember. The 5 noise entries were ephemeral session activity the agent happened to observe. The heuristic doesn't distinguish "user discovered something important" from ambient activity. + +### Impact on the comparison + +Silent auto-capture (often cited as an MCP advantage) is actually a liability at this signal-to-noise ratio. Terminal-visible saves (git-memory) let you notice and reject noise in real time. SKILL.md auto-capture rules are tightened to require a higher bar for `[auto]` tagged saves. + +--- + +## Where git-memory Wins + +- **Zero dependencies** — works anywhere git exists +- **Offline-first** — no server, no network needed +- **Cold start** — 0.6s from nothing to first memory +- **Transparency** — every memory is a git commit, inspectable with any git tool +- **Portability** — SAW, devbox, Termux, CI, bare Linux +- **No compile/rebuild cycle** — edit SKILL.md or the bash script and you're done. MCP needs TypeScript rebuild + server restart. +- **Visible auto-capture** — terminal commands let you catch noise. Silent MCP saves accumulate junk unnoticed. +- **Dedup built-in** — warns before storing near-duplicates. MCP has none. +- **Cross-agent compatibility** — SKILL.md works in VS Code Copilot, Claude Code, Cursor, any agent that reads files. MCP only works in MCP-compatible clients. +- **No maintenance** — no server process to monitor, restart, or debug. + +## Where simple-memory MCP Wins + +- **Structured API** — native tool calls, no terminal parsing +- **Relevance ranking** — results scored by relevance, not just grep matches + +## Where They're Equal + +- Recall quality (when tuned: AND search vs limit:50) +- Token efficiency (~same chars per result) +- Full content retrieval (identical data) +- JSON output (git-memory `--json` matches MCP structured responses) + +--- + +**Not yet tested:** failure recovery, parallel write safety, cross-platform (macOS/Linux/WSL2). diff --git a/docs/roadmap.md b/docs/roadmap.md index 063df46..39277b7 100644 --- a/docs/roadmap.md +++ b/docs/roadmap.md @@ -1,61 +1,64 @@ -# Roadmap - -## Done - -- [x] Git Bash (Windows) — 54/54 tests passing -- [x] WSL2 (Ubuntu) — 54/54 tests passing -- [x] Dedup checks subject + body, `--yes` flag for non-interactive use -- [x] `search --json` and `show --json` for agent output -- [x] `forget` / `resurface` / `resurface --restore` (append-only retraction) -- [x] `sync` with `pull --rebase --autostash`, divergent history tested - ---- - -## Up Next - -### Cross-platform testing -- [x] WSL2 — 54/54 tests passing, colors working -- [ ] macOS zsh, Termux -- [ ] Verify `$EDITOR` → `$VISUAL` → `vi` fallback chain -- [ ] Verify color output degrades when piped (`| less`, `> file`) -- **Known:** `grep -oE` (BSD vs GNU) and `wc -l` whitespace (macOS) — mitigations in place, need verification - -### `git-mem context` -- [ ] Single command: recent 10 + stats + tag list — one call for agent session start - -### `--dry-run` -- [ ] Check for dupes without storing (exit 0 = clean, 1 = dupe). Useful for CI and scripted flows. - -### `sync --status` -- [ ] Show ahead/behind count without pushing - -### Pruning -- [ ] `git-mem prune --older-than 1y --tag auto` — interactive cleanup of old auto-captured memories - ---- - -## Nice-to-haves - -| Feature | Effort | Notes | -|---------|--------|-------| -| `fzf` interactive search | Small | Optional dep, graceful fallback | -| Encryption at rest | Medium | `git-crypt` or GPG-signed commits | - ---- - -## Design Decisions - -### `git-mem` is both standalone and a git subcommand -Ship as `git-mem` (no extension). Both `git-mem add` and `git mem add` work via PATH discovery. - -### Env vars only -All config via `GIT_MEMORY_DIR` and `GIT_MEMORY_DEDUP_THRESHOLD`. No config files. Zero-config philosophy. - -### Single memory store -Multiple stores already work via env var override. No profiles, no named contexts. YAGNI. - -### Append-only retraction for `forget` -Memories are never deleted — just retracted. Retracted memories are excluded from normal search but discoverable via `resurface`. No `--hard` option. Users who want to rewrite history know `git rebase -i`. - -### README for humans, SKILL.md for agents -README covers concept, install, quick start, comparison tables. SKILL.md covers commands, heuristics, session workflow. No duplication. +# Roadmap + +## Done + +- [x] Git Bash (Windows) — 127/127 tests passing (7 suites) +- [x] WSL2 (Ubuntu) — 127/127 tests passing +- [x] Dedup checks subject + body, `--yes` flag for non-interactive use +- [x] `search --json` and `show --json` for agent output +- [x] `forget` / `resurface` / `resurface --restore` (append-only retraction) +- [x] `sync` with `pull --rebase --autostash`, divergent history tested +- [x] Multi-source federated search (folder-based, no config files) +- [x] `source add/list/enable/disable/remove/sync` +- [x] `.gitattributes` LF enforcement for cross-platform checkout + +--- + +## Up Next + +### Cross-platform testing +- [x] WSL2 — 54/54 tests passing, colors working +- [ ] macOS zsh, Termux +- [ ] Verify `$EDITOR` → `$VISUAL` → `vi` fallback chain +- [ ] Verify color output degrades when piped (`| less`, `> file`) +- **Known:** `grep -oE` (BSD vs GNU) and `wc -l` whitespace (macOS) — mitigations in place, need verification + +### `git-mem context` +- [ ] Single command: recent 10 + stats + tag list — one call for agent session start + +### `--dry-run` +- [ ] Check for dupes without storing (exit 0 = clean, 1 = dupe). Useful for CI and scripted flows. + +### `sync --status` +- [ ] Show ahead/behind count without pushing + +### Pruning +- [ ] `git-mem prune --older-than 1y --tag auto` — interactive cleanup of old auto-captured memories + +--- + +## Nice-to-haves + +| Feature | Effort | Notes | +|---------|--------|-------| +| `fzf` interactive search | Small | Optional dep, graceful fallback | +| Encryption at rest | Medium | `git-crypt` or GPG-signed commits | + +--- + +## Design Decisions + +### `git-mem` is both standalone and a git subcommand +Ship as `git-mem` (no extension). Both `git-mem add` and `git mem add` work via PATH discovery. + +### Env vars only +All config via `GIT_MEMORY_DIR` and `GIT_MEMORY_DEDUP_THRESHOLD`. No config files. Zero-config philosophy. + +### Single memory store +Multiple stores already work via env var override. No profiles, no named contexts. YAGNI. + +### Append-only retraction for `forget` +Memories are never deleted — just retracted. Retracted memories are excluded from normal search but discoverable via `resurface`. No `--hard` option. Users who want to rewrite history know `git rebase -i`. + +### README for humans, SKILL.md for agents +README covers concept, install, quick start, comparison tables. SKILL.md covers commands, heuristics, session workflow. No duplication. diff --git a/git-mem b/git-mem old mode 100644 new mode 100755 index f30de2b..8117f29 --- a/git-mem +++ b/git-mem @@ -9,6 +9,9 @@ set -euo pipefail GIT_MEM_VERSION="dev" # stamped by install.sh GIT_MEMORY_DIR="${GIT_MEMORY_DIR:-$HOME/memory-store}" GIT_MEMORY_DEDUP_THRESHOLD="${GIT_MEMORY_DEDUP_THRESHOLD:-3}" # min matching words to warn +# Sources live next to the primary store by default (e.g. ~/memory-store-sources). +# Each enabled child repo is searched; disabled ones carry a .disabled suffix. +GIT_MEMORY_SOURCES_DIR="${GIT_MEMORY_SOURCES_DIR:-${GIT_MEMORY_DIR}-sources}" COLOR_YELLOW='\033[1;33m' COLOR_GREEN='\033[0;32m' COLOR_RED='\033[0;31m' @@ -52,6 +55,7 @@ COMMANDS stats Memory statistics sync Safe pull --rebase, then push export Dump all memories to stdout + source Manage multi-source memory (see: git-mem source help) help Show this help ENVIRONMENT @@ -70,6 +74,8 @@ EXAMPLES echo "multi-line body" | git-mem add --yes --stdin "[tags] subject" # pipe body git-mem tags git-mem sync + git-mem source add hermes-base https://github.com/NousResearch/hermes-memories + git-mem source list EOF } @@ -207,50 +213,39 @@ is_hash_retracted() { # OR mode: single git-log piped to awk (avoids N git calls per search term). # AND mode: uses git's native --all-match --grep (already single-process). # \x01 delimiter separates hash/date/subject for JSON output without collisions. -cmd_search() { - ensure_repo - [[ $# -eq 0 ]] && die "Usage: git-mem search [--json] " - - local mode="or" - local json_mode=0 - local terms=() - - for arg in "$@"; do - case "$arg" in - --json) json_mode=1 ;; - +*) mode="and"; terms+=("${arg#+}") ;; - *) terms+=("$arg") ;; - esac - done - - [[ ${#terms[@]} -eq 0 ]] && die "Usage: git-mem search [--json] " - - # Single git call, then format for text or JSON - local terms_pipe - terms_pipe=$(printf '%s|' "${terms[@]}") - terms_pipe="${terms_pipe%|}" - - local grep_args=() - if [[ "$mode" == "and" ]]; then - for term in "${terms[@]}"; do - grep_args+=(--grep="$term") - done - fi - # Choose format based on output mode - local fmt="%h %s" - [[ $json_mode -eq 1 ]] && fmt="%h%x01%aI%x01%s" - - local raw_results - # Get retracted hashes once +# Search a single repo +search_repo() { + local repo_path="$1" + local source_name="$2" # "mine" for primary, or source name + local mode="$3" + local fmt="$4" + local terms_pipe="$5" + shift 5 + local grep_args=("$@") + + [[ ! -d "$repo_path/.git" && ! -f "$repo_path/.git" ]] && return 0 + + # Get retracted hashes once per repo local retracted_set - retracted_set=$(get_retracted_set) - + retracted_set=$(git -C "$repo_path" log --format="%s%n%b" --all 2>/dev/null | awk ' + /^\[retracted\]/ { next } + /^\[restored\]/ { next } + /^Retracted: / { h=substr($0,12); sub(/[ \t\r].*$/,"",h); if (h ~ /^[0-9a-f]{7,40}$/) retracted[h]++; next } + /^Restored: / { h=substr($0,11); sub(/[ \t\r].*$/,"",h); if (h ~ /^[0-9a-f]{7,40}$/) restored[h]++; next } + END { + for (h in retracted) { + if (retracted[h] > (restored[h]+0)) print h + } + }' || true) + + local raw_results + if [[ "$mode" == "or" ]]; then # Build retracted pattern for awk (pipe-separated full hashes) local retracted_pipe retracted_pipe=$(echo "$retracted_set" | paste -sd'|' 2>/dev/null || true) - raw_results=$(gc log --format="$fmt" --all 2>/dev/null | awk -v terms="$terms_pipe" -v retracted="$retracted_pipe" ' + raw_results=$(git -C "$repo_path" log --format="$fmt" --all 2>/dev/null | awk -v terms="$terms_pipe" -v retracted="$retracted_pipe" -v source="$source_name" ' BEGIN { n = split(tolower(terms), tarr, "|") } { low = tolower($0) @@ -270,11 +265,19 @@ cmd_search() { } } for (i = 1; i <= n; i++) { - if (index(low, tarr[i]) > 0) { print; next } + if (index(low, tarr[i]) > 0) { + # Prefix with source name + if (source != "mine") { + print "[" source "] " $0 + } else { + print $0 + } + next + } } }') else - raw_results=$(gc log --format="$fmt" --all -i --all-match "${grep_args[@]}" 2>/dev/null || true) + raw_results=$(git -C "$repo_path" log --format="$fmt" --all -i --all-match "${grep_args[@]}" 2>/dev/null || true) # Filter out retraction/restore meta-commits and retracted originals if [[ -n "$raw_results" ]]; then raw_results=$(echo "$raw_results" | grep -iv '\[retracted\]' | grep -iv '\[restored\]' || true) @@ -283,8 +286,68 @@ cmd_search() { filter_pattern=$(echo "$retracted_set" | cut -c1-7 | sed 's/^/^/' | paste -sd'|') raw_results=$(echo "$raw_results" | grep -ivE "$filter_pattern" || true) fi + # Prefix with source name + if [[ "$source_name" != "mine" && -n "$raw_results" ]]; then + raw_results=$(echo "$raw_results" | sed "s/^/[$source_name] /") + fi fi fi + + echo "$raw_results" +} + +cmd_search() { + ensure_repo + [[ $# -eq 0 ]] && die "Usage: git-mem search [--json] " + + local mode="or" + local json_mode=0 + local terms=() + + for arg in "$@"; do + case "$arg" in + --json) json_mode=1 ;; + +*) mode="and"; terms+=("${arg#+}") ;; + *) terms+=("$arg") ;; + esac + done + + [[ ${#terms[@]} -eq 0 ]] && die "Usage: git-mem search [--json] " + + # Single git call, then format for text or JSON + local terms_pipe + terms_pipe=$(printf '%s|' "${terms[@]}") + terms_pipe="${terms_pipe%|}" + + local grep_args=() + if [[ "$mode" == "and" ]]; then + for term in "${terms[@]}"; do + grep_args+=(--grep="$term") + done + fi + + # Choose format based on output mode + local fmt="%h %s" + [[ $json_mode -eq 1 ]] && fmt="%h%x01%aI%x01%s" + + # Search primary repo first (always "mine") + local raw_results + raw_results=$(search_repo "$GIT_MEMORY_DIR" "mine" "$mode" "$fmt" "$terms_pipe" "${grep_args[@]}") + + # Search enabled sources (folder-based; see iter_sources). + while IFS='|' read -r name path enabled; do + [[ -z "$name" ]] && continue + [[ "$enabled" == "true" ]] || continue + local source_results + source_results=$(search_repo "$path" "$name" "$mode" "$fmt" "$terms_pipe" "${grep_args[@]}") + if [[ -n "$source_results" ]]; then + if [[ -n "$raw_results" ]]; then + raw_results=$(printf "%s\n%s" "$raw_results" "$source_results") + else + raw_results="$source_results" + fi + fi + done < <(iter_sources) if [[ -z "$raw_results" ]]; then if [[ $json_mode -eq 1 ]]; then @@ -297,22 +360,42 @@ cmd_search() { echo -n "[" while IFS=$'\x01' read -r hash date subj; do [[ -z "$hash" ]] && continue + + # Strip the "[source] " prefix that search_repo glued onto the hash. + # date/subj were already split on the first read, so DON'T re-read here + # (the remaining string has no \x01 and would blank them out). + local source="mine" + if [[ "$hash" =~ ^\[([^\]]+)\]\ (.*)$ ]]; then + source="${BASH_REMATCH[1]}" + hash="${BASH_REMATCH[2]}" + fi + subj=$(json_escape_str "$subj") [[ $first -eq 0 ]] && printf "," first=0 - printf '{"hash":"%s","date":"%s","subject":"%s"}' "$hash" "$date" "$subj" + printf '{"hash":"%s","date":"%s","subject":"%s","source":"%s"}' "$hash" "$date" "$subj" "$source" done <<< "$raw_results" echo "]" else echo -e "${COLOR_GREEN}Found:${COLOR_RESET}" echo "$raw_results" | while IFS= read -r line; do - local hash="${line%% *}" - local rest="${line#* }" - echo -e " ${COLOR_CYAN}${hash}${COLOR_RESET} ${rest}" + # Parse source prefix if present + if [[ "$line" =~ ^\[([^\]]+)\]\ (.*)$ ]]; then + local source="${BASH_REMATCH[1]}" + local rest="${BASH_REMATCH[2]}" + local hash="${rest%% *}" + local subject="${rest#* }" + echo -e " ${COLOR_DIM}[$source]${COLOR_RESET} ${COLOR_CYAN}${hash}${COLOR_RESET} ${subject}" + else + local hash="${line%% *}" + local rest="${line#* }" + echo -e " ${COLOR_CYAN}${hash}${COLOR_RESET} ${rest}" + fi done fi } + # --- Commands --- cmd_init() { @@ -767,6 +850,222 @@ cmd_version() { echo "git-mem ${GIT_MEM_VERSION}" } +# --- Multi-Source Support (folder-based) --- +# +# Sources are just git-memory repos living under $GIT_MEMORY_SOURCES_DIR. +# No config file, no JSON: the filesystem IS the source of truth. +# - Each immediate child dir containing .git is a source. +# - The source name is the dir basename. +# - A ".disabled" suffix on the dir name excludes it from search. +# - Local sources are symlinks; URL sources are clones. + +# Count memories in a repo +count_memories() { + local path="$1" + [[ ! -d "$path/.git" && ! -f "$path/.git" ]] && echo "0" && return + git -C "$path" rev-list --count HEAD 2>/dev/null || echo "0" +} + +# Emit "name|path|enabled" for every source under $GIT_MEMORY_SOURCES_DIR. +# A source is any child (dir or symlink-to-dir) that resolves to a git repo. +iter_sources() { + [[ -d "$GIT_MEMORY_SOURCES_DIR" ]] || return 0 + local d name enabled + for d in "$GIT_MEMORY_SOURCES_DIR"/*; do + [[ -d "$d" ]] || continue # skips non-dirs and unmatched glob + [[ -d "$d/.git" || -f "$d/.git" ]] || continue + name="$(basename "$d")" + enabled=true + if [[ "$name" == *.disabled ]]; then + enabled=false + name="${name%.disabled}" + fi + printf '%s|%s|%s\n' "$name" "$d" "$enabled" + done +} + +# Resolve a source name to its on-disk path (enabled or disabled). Empty if absent. +source_dir_for() { + local name="$1" + if [[ -e "$GIT_MEMORY_SOURCES_DIR/$name" || -L "$GIT_MEMORY_SOURCES_DIR/$name" ]]; then + echo "$GIT_MEMORY_SOURCES_DIR/$name" + elif [[ -e "$GIT_MEMORY_SOURCES_DIR/$name.disabled" || -L "$GIT_MEMORY_SOURCES_DIR/$name.disabled" ]]; then + echo "$GIT_MEMORY_SOURCES_DIR/$name.disabled" + fi +} + +cmd_source() { + local subcmd="${1:-list}" + shift 2>/dev/null || true + + case "$subcmd" in + add) + [[ $# -lt 2 ]] && die "Usage: git-mem source add " + local name="$1" + local location="$2" + + # Validate name + [[ "$name" =~ [^a-zA-Z0-9_-] ]] && die "Source name must be alphanumeric (a-z, 0-9, -, _)" + [[ "$name" == "mine" || "$name" == "primary" ]] && die "Reserved name: $name" + + # Check if already exists (enabled or disabled) + [[ -n "$(source_dir_for "$name")" ]] && die "Source already exists: $name" + + mkdir -p "$GIT_MEMORY_SOURCES_DIR" + local dest="$GIT_MEMORY_SOURCES_DIR/$name" + local real_path + real_path=$(realpath "$location" 2>/dev/null || echo "$location") + local primary_real + primary_real=$(realpath "$GIT_MEMORY_DIR" 2>/dev/null || echo "$GIT_MEMORY_DIR") + + [[ "$real_path" == "$primary_real" ]] && die "Cannot add primary memory store as a source" + + if [[ -d "$real_path/.git" || -f "$real_path/.git" ]]; then + # Local path — symlink into sources dir + ln -s "$real_path" "$dest" || die "Symlink failed" + elif [[ "$location" =~ ^(https?|git|ssh):// || "$location" =~ ^[a-zA-Z0-9._-]+@[a-zA-Z0-9._-]+:.+ ]]; then + # URL — clone into sources dir + [[ -e "$dest" ]] && die "Directory already exists: $dest" + echo "Cloning $location..." + git clone "$location" "$dest" 2>&1 || die "Clone failed" + else + die "Not a git repository: $real_path" + fi + + local count + count=$(count_memories "$dest") + echo -e "${COLOR_GREEN}✓ Added source '${name}' ($count memories) → $dest${COLOR_RESET}" + ;; + + list) + local any=0 + while IFS='|' read -r name path enabled; do + if [[ $any -eq 0 ]]; then + echo -e "${COLOR_GREEN}SOURCES${COLOR_RESET}" + echo "" + any=1 + fi + local icon="✓" + [[ "$enabled" == "false" ]] && icon="✗" + local count + count=$(count_memories "$path") + local remote + remote=$(git -C "$path" remote get-url origin 2>/dev/null || true) + local link_indicator="" + [[ -L "$path" ]] && link_indicator=" [symlink]" + printf " %s %-20s %5s entries %s%s\n" \ + "$icon" "$name" "$count" "$path" "$link_indicator" + [[ -n "$remote" ]] && \ + printf " %s[remote: %s]%s\n" "$COLOR_DIM" "$remote" "$COLOR_RESET" + done < <(iter_sources) + + if [[ $any -eq 0 ]]; then + echo -e "${COLOR_DIM}No sources configured.${COLOR_RESET}" + echo "" + echo "Add a source:" + echo " git-mem source add hermes-base https://github.com/NousResearch/hermes-memories" + echo " git-mem source add team /path/to/team-memories" + fi + + echo "" + local primary_count + primary_count=$(count_memories "$GIT_MEMORY_DIR") + echo -e "${COLOR_GREEN}Primary:${COLOR_RESET} $GIT_MEMORY_DIR ($primary_count entries)" + ;; + + sync) + local target="${1:-}" + local synced=0 + + while IFS='|' read -r name path enabled; do + [[ -n "$target" && "$name" != "$target" ]] && continue + if [[ "$enabled" == "false" ]]; then + echo -e "${COLOR_DIM}Skipping disabled source: $name${COLOR_RESET}" + continue + fi + echo -e "${COLOR_CYAN}Syncing $name${COLOR_RESET} ($path)" + git -C "$path" pull --rebase --autostash 2>&1 \ + || echo -e "${COLOR_YELLOW} Warning: sync failed${COLOR_RESET}" + synced=$((synced + 1)) + done < <(iter_sources) + + if [[ $synced -eq 0 && -n "$target" ]]; then + die "Source not found: $target" + fi + [[ -z "$target" ]] && echo -e "${COLOR_GREEN}✓ Synced all sources${COLOR_RESET}" + ;; + + enable|disable) + [[ $# -eq 0 ]] && die "Usage: git-mem source $subcmd " + local name="$1" + local current_path + current_path=$(source_dir_for "$name") + [[ -z "$current_path" ]] && die "Source not found: $name" + + if [[ "$subcmd" == "disable" ]]; then + if [[ "$current_path" == *.disabled ]]; then + echo "Already disabled: $name" + else + mv "$current_path" "${current_path}.disabled" + echo -e "${COLOR_GREEN}✓ disabled source: $name${COLOR_RESET}" + fi + else + if [[ "$current_path" != *.disabled ]]; then + echo "Already enabled: $name" + else + mv "$current_path" "${current_path%.disabled}" + echo -e "${COLOR_GREEN}✓ enabled source: $name${COLOR_RESET}" + fi + fi + ;; + + remove) + [[ $# -eq 0 ]] && die "Usage: git-mem source remove " + local name="$1" + local path + path=$(source_dir_for "$name") + [[ -z "$path" ]] && die "Source not found: $name" + + if [[ -L "$path" ]]; then + # Symlink — remove the link only; original repo is untouched + rm "$path" + echo -e "${COLOR_GREEN}✓ Removed source: $name${COLOR_RESET} (symlink removed; original repo untouched)" + else + # Clone — offer to delete + read -rp "Delete cloned repository at $path? [y/N] " confirm + if [[ "$confirm" == "y" || "$confirm" == "Y" ]]; then + rm -rf "$path" + echo -e "${COLOR_GREEN}✓ Removed source: $name${COLOR_RESET}" + else + echo "Aborted." + fi + fi + ;; + + *) + cat <<'EOF' +git-mem source — manage additional memory sources + +COMMANDS + source add Add a source (clone if URL, symlink if local path) + source list List all sources + source sync [name] Pull latest for one or all sources + source enable Include source in search + source disable Exclude from search (renames dir, keeps repo) + source remove Remove source (symlink: unlink only; clone: offers delete) + +EXAMPLES + git-mem source add hermes-base https://github.com/NousResearch/hermes-memories + git-mem source add team /path/to/team-memories + git-mem source list + git-mem source sync + git-mem source disable hermes-base + git-mem source remove team +EOF + ;; + esac +} + # --- Dispatch --- cmd="${1:-help}" @@ -785,6 +1084,7 @@ case "$cmd" in stats) cmd_stats "$@" ;; sync) cmd_sync "$@" ;; export) cmd_export "$@" ;; + source) cmd_source "$@" ;; version|--version|-v) cmd_version ;; help|--help|-h) usage ;; *) die "Unknown command: $cmd. Run: git-mem help" ;; diff --git a/tests/test-basic.sh b/tests/test-basic.sh index e4999c8..eb4e56c 100644 --- a/tests/test-basic.sh +++ b/tests/test-basic.sh @@ -22,9 +22,9 @@ assert_exit_0 "add one-liner" bash "$GIT_MEM" add "[test] First memory for testi assert_exit_0 "add with body" bash "$GIT_MEM" add "[test][detail] Memory with body" "This is the body text with details." assert_exit_0 "add with --yes" bash "$GIT_MEM" add --yes "[test] Another memory with yes flag" -# Verify commits exist +# Verify commits exist (3 adds + 1 [meta] init = 4) count=$(git -C "$GIT_MEMORY_DIR" log --oneline | wc -l | tr -d ' ') -if [[ "$count" -eq 3 ]]; then pass "3 commits stored"; else fail "expected 3 commits, got $count"; fi +if [[ "$count" -eq 4 ]]; then pass "4 commits stored (3 adds + init)"; else fail "expected 4 commits, got $count"; fi # Verify body stored correctly last_body=$(git -C "$GIT_MEMORY_DIR" log -1 --skip=1 --format="%b") diff --git a/tests/test-source.sh b/tests/test-source.sh new file mode 100644 index 0000000..836afa9 --- /dev/null +++ b/tests/test-source.sh @@ -0,0 +1,123 @@ +#!/usr/bin/env bash +# test-source.sh — tests for git-mem source (folder-based multi-source) +set -euo pipefail +SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd)" +source "$SCRIPT_DIR/test-utils.sh" + +# Primary store + sources dir side-by-side +export GIT_MEMORY_DIR +GIT_MEMORY_DIR=$(mktemp -d "${TMPDIR:-/tmp}/git-mem-src-test-XXXXXX") +export GIT_MEMORY_SOURCES_DIR="${GIT_MEMORY_DIR}-sources" + +cleanup() { rm -rf "$GIT_MEMORY_DIR" "$GIT_MEMORY_SOURCES_DIR"; } +trap cleanup EXIT + +# Helper: create a minimal git-memory repo at $1 with one commit tagged $2 +make_repo() { + local path="$1" tag="$2" + mkdir -p "$path" + git init -q "$path" + git -C "$path" config user.email "t@t" + git -C "$path" config user.name "T" + git -C "$path" commit --allow-empty -m "[$tag] Memory from $tag repo" +} + +# --- Setup --- +bash "$GIT_MEM" init >/dev/null 2>&1 +bash "$GIT_MEM" add --yes "[mine] Primary repo memory" >/dev/null 2>&1 + +make_repo /tmp/git-mem-src-team-$$ team +make_repo /tmp/git-mem-src-hermes-$$ hermes + +cleanup_externals() { + rm -rf /tmp/git-mem-src-team-$$ /tmp/git-mem-src-hermes-$$ +} +trap "cleanup; cleanup_externals" EXIT + +# --- source add (local path → symlink) --- +echo "=== source add ===" +assert_exit_0 "add local source (team)" bash "$GIT_MEM" source add team /tmp/git-mem-src-team-$$ +assert_exit_0 "add local source (hermes)" bash "$GIT_MEM" source add hermes /tmp/git-mem-src-hermes-$$ +assert_exit_nonzero "add duplicate name fails" bash "$GIT_MEM" source add team /tmp/git-mem-src-team-$$ +assert_exit_nonzero "add primary as source fails" bash "$GIT_MEM" source add mine "$GIT_MEMORY_DIR" +assert_exit_nonzero "reserved name 'mine' rejected" bash "$GIT_MEM" source add mine /tmp/git-mem-src-team-$$ +assert_output_contains "scp-style SSH URL treated as remote clone" "Clone failed" \ + bash "$GIT_MEM" source add sshremote1 git@127.0.0.1:/tmp/git-mem-nonexistent-a-$$ +assert_output_not_contains "scp-style SSH URL not treated as local path" "Not a git repository" \ + bash "$GIT_MEM" source add sshremote2 git@127.0.0.1:/tmp/git-mem-nonexistent-b-$$ + +# Verify symlinks created +[[ -L "$GIT_MEMORY_SOURCES_DIR/team" ]] && pass "team is a symlink" || fail "team symlink missing" +[[ -L "$GIT_MEMORY_SOURCES_DIR/hermes" ]] && pass "hermes is a symlink" || fail "hermes symlink missing" + +# --- source list --- +echo "" +echo "=== source list ===" +assert_output_contains "list shows team" "team" bash "$GIT_MEM" source list +assert_output_contains "list shows hermes" "hermes" bash "$GIT_MEM" source list +assert_output_contains "list shows Primary" "Primary" bash "$GIT_MEM" source list +assert_output_contains "list shows symlink indicator" "symlink" bash "$GIT_MEM" source list + +# --- search across sources --- +echo "" +echo "=== search (multi-source) ===" +assert_output_contains "search finds primary memory" "Primary repo memory" bash "$GIT_MEM" search primary +assert_output_contains "search finds team source" "Memory from team" bash "$GIT_MEM" search team +assert_output_contains "search finds hermes source" "Memory from hermes" bash "$GIT_MEM" search hermes +assert_output_contains "search attributes team result" "[team]" bash "$GIT_MEM" search team + +# --- search --json includes source field --- +echo "" +echo "=== search --json (source attribution) ===" +json=$(bash "$GIT_MEM" search --json team 2>/dev/null) +if echo "$json" | grep -q '"source":"team"'; then + pass "--json includes source field" +else + fail "--json missing source field (got: $json)" +fi +if echo "$json" | grep -q '"date":"'; then + pass "--json includes non-empty date for source result" +else + fail "--json date empty for source result (got: $json)" +fi +if echo "$json" | grep -q '"subject":"'; then + pass "--json includes non-empty subject for source result" +else + fail "--json subject empty for source result (got: $json)" +fi + +# --- source disable --- +echo "" +echo "=== source disable ===" +assert_exit_0 "disable team" bash "$GIT_MEM" source disable team +[[ -d "$GIT_MEMORY_SOURCES_DIR/team.disabled" ]] && pass "team.disabled dir exists" || fail "team.disabled not created" +[[ ! -e "$GIT_MEMORY_SOURCES_DIR/team" ]] && pass "team (enabled) removed" || fail "old team dir still exists" + +assert_output_not_contains "disabled source excluded from search" "Memory from team" bash "$GIT_MEM" search team +assert_output_contains "primary still searched when source disabled" "Primary repo memory" bash "$GIT_MEM" search primary + +assert_exit_0 "disable idempotent (already disabled)" bash "$GIT_MEM" source disable team + +# --- source enable --- +echo "" +echo "=== source enable ===" +assert_exit_0 "enable team" bash "$GIT_MEM" source enable team +[[ -d "$GIT_MEMORY_SOURCES_DIR/team" ]] && pass "team re-enabled dir exists" || fail "team dir missing after enable" +assert_output_contains "re-enabled source appears in search" "Memory from team" bash "$GIT_MEM" search team +assert_exit_0 "enable idempotent (already enabled)" bash "$GIT_MEM" source enable team + +# --- source remove (symlink) --- +echo "" +echo "=== source remove ===" +assert_exit_0 "remove hermes (symlink)" bash "$GIT_MEM" source remove hermes +[[ ! -e "$GIT_MEMORY_SOURCES_DIR/hermes" ]] && pass "hermes symlink gone" || fail "hermes symlink still present" +[[ -d "/tmp/git-mem-src-hermes-$$" ]] && pass "original hermes repo intact" || fail "original hermes repo deleted" +assert_exit_nonzero "remove non-existent fails" bash "$GIT_MEM" source remove hermes + +# --- source list after changes --- +echo "" +echo "=== source list (post-remove) ===" +assert_output_contains "list still shows team" "team" bash "$GIT_MEM" source list +assert_output_not_contains "list no longer shows hermes" "hermes" bash "$GIT_MEM" source list + +print_summary diff --git a/tests/test-sync.sh b/tests/test-sync.sh index ce3a914..f553359 100644 --- a/tests/test-sync.sh +++ b/tests/test-sync.sh @@ -32,7 +32,7 @@ pass "machine A: init + push (branch: $BRANCH)" # Machine B: clone from remote git clone "$BARE_REPO" "$MACHINE_B" >/dev/null 2>&1 count_b=$(git -C "$MACHINE_B" log --oneline | wc -l | tr -d ' ') -if [[ "$count_b" -eq 1 ]]; then pass "machine B: cloned with 1 memory"; else fail "machine B: expected 1 memory, got $count_b"; fi +if [[ "$count_b" -eq 2 ]]; then pass "machine B: cloned with 2 commits (init + 1 add)"; else fail "machine B: expected 2, got $count_b"; fi echo "" echo "=== Sync: A adds, B pulls ===" @@ -46,7 +46,7 @@ git -C "$MACHINE_A" push >/dev/null 2>&1 export GIT_MEMORY_DIR="$MACHINE_B" bash "$GIT_MEM" sync >/dev/null 2>&1 || true count_b=$(git -C "$MACHINE_B" log --oneline | wc -l | tr -d ' ') -if [[ "$count_b" -eq 2 ]]; then pass "B synced: has 2 memories"; else fail "B sync: expected 2, got $count_b"; fi +if [[ "$count_b" -eq 3 ]]; then pass "B synced: has 3 commits (init + 2 adds)"; else fail "B sync: expected 3, got $count_b"; fi echo "" echo "=== Sync: both add, then sync (divergent history) ==="