diff --git a/AGENTS.md b/AGENTS.md index 51c9d485..c8843e34 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -24,7 +24,7 @@ dsh-mneme 是 DSH 宿主的记忆插件（蒸馏 / 注入 / 检索 / 巩固 / sc - 不 `amend` / force-push 别人的提交，不把别人未提交的改动 `stash` 走； - **提交不得有 AI 署名**（无 `Co-Authored-By`、无 "Generated with"）——见 [CONTRIBUTING](CONTRIBUTING.md)； - **本机私有信息不进公开文件**：车道分配、端口占用、记忆库路径这类只对某台机器成立的东西，放不进库的本地板（`.worktrees/coordination.md`）或 `.git/info/exclude`；本文件只放对所有人都成立的纪律（定位见开头：薄入口）； -- 两个宿主都加载本插件时：`memoryDir` 可共用（`src/store.js` 的 `busy_timeout` + WAL 已为多进程就绪），但 `externalApiEnabled`（默认端口 8790）与 `autoDream` 后台任务**只能一边开**，否则 EADDRINUSE + 重复做梦（镜像/审计双写）。 +- 两个宿主都加载本插件时：`memoryDir` 可共用（`src/store.js` 的 `busy_timeout` + WAL 已为多进程就绪），但 `externalApiEnabled`（默认端口 8790）与 `autoDream` 后台任务**只能一边开**，否则 EADDRINUSE + 重复做梦（镜像/审计双写）。独立服务（`dsh-mneme-serve`）同理：与 DSH 外部访问抢同一端口、**二选一**；它第一期无 LLM，巩固结构性只在宿主侧，不构成第二个做梦者。 ## 模块地图（按功能面） @@ -44,7 +44,7 @@ dsh-mneme 是 DSH 宿主的记忆插件（蒸馏 / 注入 / 检索 / 巩固 / sc | 复用统计 | `src/recall-stats.js` | recall_runs 只读聚合（#217，Top-N 召回 + 僵尸率） | | 冷启动 | `src/bootstrap.js` | 从仓库文件反向构建初始记忆（POST /bootstrap） | | 配置 | `src/config.js`（schema + lightMode）、`src/settings.js`（feature flags 白名单） | 一切行为开关的家 | -| API 面 | `src/api.js`（宿主内 /api/dsh-mneme/*）、`src/api-standalone.js`（Bearer 数据面）、`bin/dsh-mneme-mcp.mjs`（MCP stdio） | 对外三张脸 | +| API 面 | `src/api.js`（宿主内 /api/dsh-mneme/*）、`src/api-standalone.js`（Bearer 数据面）、`bin/dsh-mneme-mcp.mjs`（MCP stdio）、`src/serve.js` + `bin/dsh-mneme-serve.mjs`（独立服务 daemon，无 LLM 数据面） | 对外四张脸 | | 面板 | `lib/client.js` | 面板侧产物，无 src 对应物 | | 运行时 | `src/runtime/*` | 模型下载（断点续传）/ 校验 / adopt | | 命令 | `src/commands.js` | 斜杠命令注册与派发 | diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 6f3aa15c..e3e03f02 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -159,6 +159,7 @@ The standalone external API (`src/api-standalone.js`) and the `bin/cli.mjs` clie - Keep the route surface read/write on memories only; new routes need tests in `test/standalone-api.test.js` (they spin the real server on port 0). - Auth additions/changes must keep `timingSafeEqual` token comparison and the `GET /health` exception. - The CLI is dependency-free by contract - do not add imports to `bin/cli.mjs`. +- The standalone daemon (`bin/dsh-mneme-serve.mjs` → `src/serve.js`) is exempt from that contract: it mounts `lib/serve.js` by design. Keep it a pure data plane — no LLM handle, no dream/summarize path (single-writer guarantee vs. the host), and keep `strictPort` semantics (busy configured port = hard error; the in-host sidecar keeps its hop-and-fallback recovery). ## Issue Reporting Requirements (read this first) diff --git a/README.md b/README.md index 12126317..a8664900 100644 --- a/README.md +++ b/README.md @@ -10,7 +10,7 @@ license CI node - tests + tests coverage Awesome

@@ -149,12 +149,22 @@ dsh web > **旧挂载兼容**：已部署的 `dsh-mneme-mcp` + `DSH_MNEME_TOKEN` 写法继续有效（bin 与 env 变量均保留，无需迁移）。未全局安装 npm 包时，把 `command` 换成 `npx` 并追加参数 `-p mneme-memory mneme-mcp`（Claude Code/OpenCode 写进 args 数组，Codex 写 `args = ["-p", "mneme-memory", "mneme-mcp"]`）。配置细节与安全注意事项见[完整文档](dsh-mneme/README.md#mcp-server任意-mcp-客户端接入)。 +## 不开 DSH 也能服务（独立服务 daemon） + +`dsh-mneme-serve` 把记忆库跑成常驻数据面——DSH 关着，第三方集成（网页端桥接、脚本、自有面板）照样读写同一份记忆：token 与 DSH 面板/CLI 共用，路由与「外部访问 API」同源，端口被占直接报错（与 DSH 外部访问二选一）。第一期无 LLM（巩固/蒸馏仍属 DSH 宿主），检索为关键词 + BM25，向量接入在后续版本。 + +```bash +npm i -g @modusensus/dsh-mneme +dsh-mneme-serve # 默认 ~/.dsh/memory + 127.0.0.1:8790 +``` + ## 文档 | 文档 | 路径 | |------|------| | 插件完整文档（功能 / 安装 / 配置 / 架构） | [dsh-mneme/README.md](dsh-mneme/README.md) | | stdio MCP server——Claude Code / Cursor 等任意 MCP 客户端接入记忆六件套 | [dsh-mneme/README.md · MCP Server](dsh-mneme/README.md#mcp-server任意-mcp-客户端接入) | +| 独立服务 daemon（不开 DSH 常驻数据面） | [dsh-mneme/docs/DAEMON.md](dsh-mneme/docs/DAEMON.md) | | 配置说明（全键参考） | [dsh-mneme/docs/CONFIGURATION.md](dsh-mneme/docs/CONFIGURATION.md) | | 实体结构化设计 | [dsh-mneme/docs/ENTITIES.md](dsh-mneme/docs/ENTITIES.md) | | 语义架构 | [dsh-mneme/docs/SEMANTIC.md](dsh-mneme/docs/SEMANTIC.md) | @@ -183,7 +193,7 @@ dsh web ```bash cd dsh-mneme && npm install -npm test # 1529 个测试 +npm test # 1535 个测试 npm run stress # 三轴线压测 npm run sync # src → lib 同步 ``` @@ -332,12 +342,22 @@ The plugin ships a zero-dependency stdio MCP server (standalone npm package **`m > **Legacy mounts keep working**: `dsh-mneme-mcp` + `DSH_MNEME_TOKEN` remain supported (both the bin and env vars are preserved; no migration needed). If the npm package is not installed globally, use `npx` as the command with args `-p mneme-memory mneme-mcp` (an args array in Claude Code/OpenCode; `args = ["-p", "mneme-memory", "mneme-mcp"]` in Codex). Full config details and security notes: [full docs](dsh-mneme/README.md#mcp-server任意-mcp-客户端接入) (Chinese). +## Serve memories without DSH (standalone daemon) + +`dsh-mneme-serve` runs the memory store as a long-lived data plane — with DSH closed, third-party integrations (web-bridge tools, scripts, your own panels) still read and write the same memories: the Bearer token is shared with the DSH panel/CLI, routes mirror the external API, and a busy port is a hard error (pick either the daemon or DSH's external API, not both). Phase 1 is LLM-free (consolidation/distillation stay with the DSH host); retrieval is keyword + BM25, with vector search arriving in a later release. + +```bash +npm i -g @modusensus/dsh-mneme +dsh-mneme-serve # defaults: ~/.dsh/memory + 127.0.0.1:8790 +``` + ## Docs | Doc | Path | |-----|------| | Full plugin docs (features / install / config / architecture) | [dsh-mneme/README.md](dsh-mneme/README.md)（中文） | | stdio MCP server — plug the six memory tools into any MCP client (Claude Code / Cursor / …) | [dsh-mneme/README.md · MCP Server](dsh-mneme/README.md#mcp-server任意-mcp-客户端接入)（中文） | +| Standalone daemon (serve memories without DSH) | [dsh-mneme/docs/DAEMON.md](dsh-mneme/docs/DAEMON.md)（中文） | | Configuration reference (all keys) | [dsh-mneme/docs/CONFIGURATION.md](dsh-mneme/docs/CONFIGURATION.md)（中文） | | Entity structure design | [dsh-mneme/docs/ENTITIES.md](dsh-mneme/docs/ENTITIES.md) | | Semantic architecture | [dsh-mneme/docs/SEMANTIC.md](dsh-mneme/docs/SEMANTIC.md) | @@ -366,7 +386,7 @@ The plugin ships a zero-dependency stdio MCP server (standalone npm package **`m ```bash cd dsh-mneme && npm install -npm test # 1529 tests +npm test # 1535 tests npm run stress # three-axis stress test npm run sync # src → lib sync ``` diff --git a/dsh-mneme/CHANGELOG.md b/dsh-mneme/CHANGELOG.md index dc0ce968..a0ed6dc3 100644 --- a/dsh-mneme/CHANGELOG.md +++ b/dsh-mneme/CHANGELOG.md @@ -2,6 +2,9 @@ ## [Unreleased] +## 🆕 新增 + +- **独立服务 daemon（`dsh-mneme-serve`，#363）**：mneme 现在能在 DSH 宿主之外常驻——`src/serve.js` 的 `createServeRuntime` 用最小装配（store → settings → mirror → service → maintenance → standalone API，每步锚定 index.js 装配行号）把数据面跑成独立进程，第三方集成（网页端桥接等）不必为挂载记忆库而保持 DSH 开机。第一期刻意无 LLM：巩固（autoDream）与蒸馏结构性不在 daemon 内，这是与宿主「单写者」的机械保证，不靠用户自觉；检索为关键词 + BM25（向量由后续 PR 抽取 semantic 装配后接入）。token 与 DSH 面板/CLI 共用同一 kv 凭证，端口/主机解析链与外部访问一致；`createStandaloneApi` 新增 `strictPort` 选项——daemon 的配置端口被占即报错退出而非顺延（第三方把 URL 写死，静默换端口等于坏），不传该选项的宿主旁路行为不变。`/search` 照常落 recall_runs，第三方检索的复用统计不缺数。多进程共存（daemon 与宿主同库互写互读）有专门回归锁；已知限制（双进程去重竞态、镜像双写、版本偏斜）见 docs/DAEMON.md。 ## 🧹 工程 - **发布准备脚本在 CRLF 检出上不再假成功（`scripts/release-prep.mjs`）**：该脚本用 `/^(# Changelog\n\n)/` 匹配 CHANGELOG 文件头，而 Windows 检出是 CRLF——正则命中不了，`replace` 退化成空操作，**脚本却照样打印 `✓ … 占位节`**，`git status` 里看不出任何异常（CI 跑在 ubuntu 是 LF，所以只有本机发版会中招，v0.8.13 那次即如此、最后靠人工补的占位节）。规则抽成 `dsh-mneme/scripts/changelog-prep.mjs` 的纯函数：行尾两种都吃、插入内容跟随原文件行尾、带 BOM 也认；匹配不上则如实回报 `header-not-found`，入口**报错退出（exit 1）**而不是假打印成功。配 6 条回归测试（LF / CRLF / BOM / 幂等 / 回报契约 / detectEol）。 diff --git a/dsh-mneme/README.md b/dsh-mneme/README.md index 61b53a28..fd026b15 100644 --- a/dsh-mneme/README.md +++ b/dsh-mneme/README.md @@ -5,7 +5,7 @@ [![npm version](https://img.shields.io/npm/v/@modusensus/dsh-mneme?color=blue&label=npm)](https://www.npmjs.com/package/@modusensus/dsh-mneme) [![license](https://img.shields.io/badge/license-MIT-green)](LICENSE) [![Awesome](https://awesome-dsh-plugin.com/badge.svg)](https://github.com/awesome-dsh-plugin/awesome-dsh-plugin) -[![tests](https://img.shields.io/badge/tests-1529%20passed-success)](https://github.com/slow-stack/mneme) +[![tests](https://img.shields.io/badge/tests-1535%20passed-success)](https://github.com/slow-stack/mneme) [![CI](https://img.shields.io/github/actions/workflow/status/slow-stack/mneme/ci.yml)](https://github.com/slow-stack/mneme/actions) [![node](https://img.shields.io/badge/node-22%2B-blue)](https://nodejs.org) [![npm downloads](https://img.shields.io/npm/d18m/@modusensus/dsh-mneme.svg?color=blue&label=downloads)](https://www.npmjs.com/package/@modusensus/dsh-mneme) @@ -515,6 +515,22 @@ dsh-mneme config show # 查看当前配置（toke > 所有读取/写入命令支持 `--json` 输出原始 JSON；`config path` 打印配置文件路径（`~/.dsh-mneme/cli.json`）。 +### 独立服务（daemon，`dsh-mneme-serve`） + +不启动 DSH 也能让记忆库对外服务：`dsh-mneme-serve` 在宿主之外组装同一套 lib，把数据面跑成常驻进程（路由与鉴权与「独立外部 API」完全同源）。适用场景：第三方集成（如 [Mneme Bridge](https://github.com/slow-stack/mneme/discussions/363) 这类网页端桥接）需要长期挂载记忆库，而 DSH 不必一直开着。 + +```bash +dsh-mneme-serve # 默认 ~/.dsh/memory + 8790 +dsh-mneme-serve --memory-dir "D:\my mem" --port 8790 --host 127.0.0.1 +``` + +- **鉴权与端口**：Bearer token 与 DSH 面板 / CLI 共用同一份（kv `external_api`，首次启动自动生成并持久化到 `memory.db`）；端口/主机解析链与「外部访问」一致（显式参数 > 持久值 > 默认 8790/127.0.0.1）。配置端口被占会**直接报错退出**（不做端口顺延）——第三方把 URL 写死，静默换端口等于坏。因此 **daemon 与 DSH 的「外部访问」二选一**，不要同端口同开。 +- **安全**：daemon 使用明文 HTTP，不提供原生 TLS。指定非回环 `--host` 时，请勿直接把服务暴露给不可信网络；远程访问请走 TLS 终止代理或 SSH 隧道。 +- **能力边界（第一期，无 LLM）**：存储 / 检索（关键词 + BM25）/ 镜像同步与人改合并 / `POST /maintenance/reclaim` / `/bootstrap` 全可用；巩固（autoDream）与蒸馏不在 daemon 内——巩固只属于 DSH 宿主进程，这是与宿主「单写者」的机械保证。向量检索暂缺（后续版本接入），`/search` 退化为关键词 + BM25 属预期。 +- **检索回执**：daemon 的 `/search` 同样落 `recall_runs`，第三方检索的复用统计不缺数。 +- **生命周期**：stdout 仅就绪时打一行 `dsh-mneme-serve listening on http://host:port (pid N)`（供脚本解析实际端口），日志走 stderr；SIGINT/SIGTERM 优雅收库，Windows 强杀由 WAL 回放兜底。 +- **已知限制**：与 DSH 同时运行属设计内场景（WAL 多进程并发），但去重是先查后写、库层无 UNIQUE 约束，双进程并发写同一 `(type, title, scope)` 有极小概率产生重复；daemon 与插件请同版本升级。细节与坑清单见 [docs/DAEMON.md](docs/DAEMON.md)。 + ### MCP Server（任意 MCP 客户端接入） > Claude Code / Codex / Hermes / OpenCode / OpenClaw 等各客户端的最小挂载配置速查表见[根 README](../README.md#用在其他-ai-工具里mcp)；本节是完整配置与安全说明。规划中的独立分发包 `mneme-memory` 落地后，挂载命令将保持兼容（详见仓库 Discussions #300）。 @@ -584,7 +600,7 @@ src/ ├── api.js # HTTP 路由（Web 面板数据通道，含 /conflicts 冲突队列） └── index.js # 插件接线 lib/ # src 的同步分发产物（npm run sync；发布前由 root prepack 的 check-sync.js 校验一致性；唯一手写例外 lib/client.js——Web 面板 bundle，sync 不覆盖） -test/ # 1529 个 node:test 测试（审计与三轴线压测不变量；src↔lib 一致性由 scripts/check-sync.js 发布闸门校验） +test/ # 1535 个 node:test 测试（审计与三轴线压测不变量；src↔lib 一致性由 scripts/check-sync.js 发布闸门校验） scripts/ # e2e-dsh.js 端到端演示 · stress-dsh.js 三轴线压测 · sync-lib.js 同步 · check-sync.js 发布闸门 · benchmark-recall.js / benchmark-embed.js / benchmark-rerank.js 基准 · sync-test-badge.mjs 测试徽章 · build-runtime-manifest.mjs 运行时清单 ``` @@ -593,7 +609,7 @@ scripts/ # e2e-dsh.js 端到端演示 · stress-dsh.js 三轴线压 ```bash cd dsh-mneme npm install # 安装 peer 依赖（以 devDependencies 形式，用于本地测试） -npm test # 运行 1529 个测试 +npm test # 运行 1535 个测试 npm run stress # 三轴线压测：长会话检索 / 冲突仲裁 / 多 Agent 并发（离线 mock LLM） npm run sync # 把 src/ 同步到 lib/（发布时由 prepack 钩子自动执行） ``` diff --git a/dsh-mneme/bin/dsh-mneme-serve.mjs b/dsh-mneme/bin/dsh-mneme-serve.mjs new file mode 100644 index 00000000..8424b624 --- /dev/null +++ b/dsh-mneme/bin/dsh-mneme-serve.mjs @@ -0,0 +1,128 @@ +#!/usr/bin/env node +// bin/dsh-mneme-serve.mjs —— mneme 独立服务(daemon)入口。 +// +// 为什么独立成 bin 而不是 cli.mjs 的子命令:CONTRIBUTING「The CLI is dependency-free +// by contract」禁止给 bin/cli.mjs 加 import,而 serve 必须挂载 lib/serve.js;命名循 +// dsh-mneme-mcp 先例。数据面 = src/api-standalone.js 同一工厂,路由与鉴权零新面。 +// +// 生命周期:bind 失败(含 strictPort 下端口被占)→ stderr 清错 + exit 1; +// SIGINT/SIGTERM → dispose + exit 0。Windows 下 kill() 是硬终止、handler 不跑: +// 已提交事务由 WAL 回放兜底(node:sqlite 未 finalize 语句的 close 抛错同理),不丢数据。 +// +// 日志纪律:进度/错误全走 stderr;stdout 只在就绪时打一行机器可读的 listening 行 +// (脚本/测试从中解析实际端口),其余时刻保持干净——serve 是常驻进程,stdout 常被 +// 重定向进监控管道。 +import { readFileSync } from "node:fs"; +import { fileURLToPath } from "node:url"; +import { dirname, join, resolve } from "node:path"; +import { createServeRuntime } from "../lib/serve.js"; + +const BIN_NAME = "dsh-mneme-serve"; +const PKG_ROOT = join(dirname(fileURLToPath(import.meta.url)), ".."); +const PKG = JSON.parse(readFileSync(join(PKG_ROOT, "package.json"), "utf8")); + +const USAGE = `${BIN_NAME} — run the mneme data plane as a standalone service (no DSH required) + +Usage: dsh-mneme-serve [--memory-dir ] [--port ] [--host ] + +Options: + --memory-dir data directory (default: ~/.dsh/memory, same as the plugin) + --port HTTP port (default: persisted external_api port, else 8790) + --host bind address (default: persisted external_api host, else 127.0.0.1) + -h, --help show this help + -V, --version print version + +Auth: Bearer token is shared with the DSH panel / CLI (kv "external_api" in +memory.db); it is generated on first boot. A busy configured port is a hard +error — the DSH external API and this daemon must not share a port (pick one).`; + +/** 极简 argv 解析(--k=v / --k v / 旗标);够用即可,完整 CLI 在 bin/cli.mjs。 */ +function parseArgv(argv) { + const out = {}; + for (let i = 0; i < argv.length; i++) { + const a = argv[i]; + if (a === "-h" || a === "--help") out.help = true; + else if (a === "-V" || a === "--version") out.version = true; + else if (a.startsWith("--")) { + const eq = a.indexOf("="); + if (eq > -1) out[a.slice(2, eq)] = a.slice(eq + 1); + else { + const v = argv[i + 1]; + // 本 bin 的旗标全部取值（无泛用布尔旗标）：值缺失或下一个 token 是旗标 + // 都按「缺值」报错退出——不能落成 true（Number(true)=1 会把 --port 变成 + // 绑端口 1，报 EACCES 让用户查错方向）。帮助/版本在上方分支已提前返回。 + if (v !== undefined && !v.startsWith("-")) { out[a.slice(2)] = v; i++; } + else fail(`${a} 缺少参数值`); + } + } else { + fail(`未知参数: ${a}\n运行 \`${BIN_NAME} --help\` 查看用法。`); + } + } + return out; +} + +function fail(msg) { + console.error(`[${BIN_NAME}] ${msg}`); + process.exit(1); +} + +// api-standalone 的 logger 契约:info/warn/error 收单字符串。全走 stderr(见头部纪律)。 +const logger = { + info: (msg) => console.error(`[dsh-mneme] ${msg}`), + warn: (msg) => console.error(`[dsh-mneme] warn: ${msg}`), + error: (msg) => console.error(`[dsh-mneme] error: ${msg}`) +}; + +async function main(argv) { + const args = parseArgv(argv); + if (args.help) { process.stdout.write(USAGE + "\n"); return; } + if (args.version) { console.log(PKG.version); return; } + + // memoryDir:CLI > env > serve.js 缺省(~/.dsh/memory)。相对路径按 CWD 解; + // 前导 ~ 留给 serve.js 展开(与宿主 index.js:192-194 同一口径)。 + const rawDir = typeof args["memory-dir"] === "string" ? args["memory-dir"] + : (typeof process.env.DSH_MNEME_MEMORY_DIR === "string" && process.env.DSH_MNEME_MEMORY_DIR ? process.env.DSH_MNEME_MEMORY_DIR : undefined); + const memoryDir = rawDir && !rawDir.startsWith("~") ? resolve(rawDir) : rawDir; + + let port; + if (args.port !== undefined) { + port = Number(args.port); + if (!Number.isInteger(port) || port < 0 || port > 65535) { + fail(`--port 需要合法端口(0-65535),收到: ${args.port}`); + } + } + const host = typeof args.host === "string" && args.host ? args.host : undefined; + + const rt = createServeRuntime({ memoryDir, port, host, logger }); + try { + await rt.api.ready; + } catch (err) { + fail(`数据面启动失败: ${err?.message ?? err}\n` + + ` 端口 ${rt.api.port} 被占通常意味着另一个 mneme 数据面正在运行` + + `(DSH 的「外部访问」或另一份 ${BIN_NAME})——二选一,或用 --port 换端口。`); + } + const addr = rt.api.server.address(); + // stdout 唯一一行:机器可读就绪行(测试/脚本解析端口用)。 + console.log(`${BIN_NAME} listening on http://${addr?.address ?? rt.api.host}:${addr?.port ?? rt.api.port} (pid ${process.pid})`); + if (!rt.tokenExisted) { + console.error(`[dsh-mneme] 首次启动已生成 Bearer token(已持久化,与 DSH 面板/CLI 共用):\n` + + ` ${rt.api.token}`); + } + + let closing = false; + const shutdown = async (signal) => { + // 第二次信号 = 强制退出（dispose 里 server.close 等 in-flight 收尾，极端情况下会挂） + if (closing) process.exit(0); + closing = true; + console.error(`[dsh-mneme] ${signal} received, closing...`); + try { await rt.dispose(); } catch { /* dispose 各步自吞 */ } + process.exit(0); + }; + process.on("SIGINT", () => shutdown("SIGINT")); + process.on("SIGTERM", () => shutdown("SIGTERM")); +} + +main(process.argv.slice(2)).catch((err) => { + console.error(err?.stack ?? String(err)); + process.exit(1); +}); diff --git a/dsh-mneme/docs/DAEMON.md b/dsh-mneme/docs/DAEMON.md new file mode 100644 index 00000000..a5e4acc3 --- /dev/null +++ b/dsh-mneme/docs/DAEMON.md @@ -0,0 +1,44 @@ +# dsh-mneme 独立服务(daemon) + +`dsh-mneme-serve`:在 DSH 宿主之外把 mneme 跑成一个常驻数据面。#363(Mneme Bridge)确立的方向——第三方集成需要长期挂载,而 DSH 不必一直开着;这是官方推荐姿势,lib 直挂的 embedded 模式降级为无网兜底。 + +## 1. 职责边界 + +daemon 是**数据面**,不是第二个宿主: + +- **有**:存储(SQLite)、检索(关键词 + BM25,`/search` 统一召回)、镜像同步与人改合并、`/maintenance/reclaim`、`/bootstrap`、recall_runs 检索回执。 +- **没有(第一期,无 LLM)**:巩固(autoDream)、蒸馏(autoSummarize)、实体抽取、sleep、注入/工具/面板路由。前两者是**结构性缺失**而非开关——daemon 装配里没有 LLM 句柄,巩固只属于 DSH 宿主进程。这就是 daemon 与宿主「单写者」的机械保证(AGENTS.md externalApi/autoDream 单侧纪律的 daemon 版),不依赖用户自觉。 + +与宿主装配(`src/index.js` apply)的关系:`src/serve.js` 只搬数据面那一半,每步注释锚定 index.js 来源行号;刻意不抽公共装配函数(apply 其余环节与宿主 ctx 纠缠,防御段纪律「最后动或不动」)。装配漂移风险由 `test/serve-bin.test.js` 的多进程共存用例兜底(两进程真开同一个库互写互读)。 + +## 2. 对外接口 + +路由面 = `src/api-standalone.js` 全表(health/status/profile/rules/memories 读写/search/maintenance/bootstrap),鉴权同源(Bearer + timingSafeEqual,`GET /health` 免鉴权)。**零新路由**;唯一新选项是 `strictPort`(见 §4)。 + +CLI: + +```bash +dsh-mneme-serve [--memory-dir ] [--port ] [--host ] +``` + +- `memoryDir`:CLI > env `DSH_MNEME_MEMORY_DIR` > `~/.dsh/memory`(与宿主 config.js 同默认,支持前导 `~`)。 +- port/host 解析链与宿主「外部访问」一致:显式参数 > kv `external_api` 持久值 > 默认 8790 / 127.0.0.1。 +- token 与 DSH 面板 / CLI **共用同一份**(kv `external_api`,首次启动自动生成并持久化)——三方零配置互通。 +- 安全:daemon 使用明文 HTTP,不提供原生 TLS。指定非回环 `--host` 时,请勿直接把服务暴露给不可信网络;远程访问请走 TLS 终止代理或 SSH 隧道。 +- stdout 只在就绪时打一行 `dsh-mneme-serve listening on http://host:port (pid N)`(机器可读,脚本/测试解析端口用);日志全走 stderr。 +- SIGINT/SIGTERM 优雅收库后 exit 0;Windows 强杀由 WAL 回放兜底。 + +## 3. 内部文件 + +- `src/serve.js` — `createServeRuntime({memoryDir, port, host, logger, strictPort})`:装配链 createStore → createSettings → createMirror → createService(最小 config)→ recoverMirror → 人改镜像合并 → recall recorder → createMaintenance → createStandaloneApi,每步锚定 index.js 行号。返回 `{api, store, service, settings, maintenance, tokenExisted, dispose}`;第三方可 import 它自行托管生命周期(bin 只是薄壳)。 +- `bin/dsh-mneme-serve.mjs` — CLI 壳。独立成 bin 而非 cli.mjs 子命令:CONTRIBUTING 禁止给 cli.mjs 加 import;命名循 dsh-mneme-mcp 先例。 +- `src/api-standalone.js` 的 `strictPort` 选项 — 唯一的数据面改动,默认关闭。 + +## 4. 已知坑 + +1. **端口互斥,二选一**:daemon 与 DSH 的「外部访问」抢同一个默认端口。daemon 侧 strictPort 报错退出;反方向( daemon 先占 8790,DSH 后开外部访问)宿主侧会**静默顺延**到下一端口(宿主旁路的多实例恢复语义,api-standalone.js listenWithRetry)——面板显示的端口会变,别当 bug 报。 +2. **双进程写并发**:与 DSH 同时运行是设计内场景(WAL + busy_timeout 先序,store.js createStore)。但 `saveWithDedupe` 的 (type,title,scope) 去重是先查后写、库层无 UNIQUE 约束,两进程并发写同一三元组有极小概率产生重复条目——已知限制,勿当强保证宣传(要不要加 UNIQUE 索引属 schema 防御段,单独决策)。 +3. **镜像双写竞态**:daemon 与宿主都会渲镜像 .md;可再生物,失败由 recoverMirror 自愈,极端并发下单文件可能短暂脏,下次同步覆盖。 +4. **版本偏斜**:库迁移是幂等加法式(PRAGMA 检查 + ALTER),旧代码读新 schema 一般无碍,但该组合无人测过——daemon 与插件请同版本升级。 +5. **第一期不吃宿主配置**:daemon 不加载 config schema(schemastery 是宿主 peer 依赖),面板/feature_flags 对它不生效;它只有 CLI 参数 + 上述固定最小 config(`language: zh`、document 子系统关闭)。向量检索(依赖 embedder/reranker 装配)由后续 PR 接入,接入前 `/search` 退化关键词 + BM25 属预期。 +6. **验收锚点**:`test/serve.test.js`(in-process 全链路 + token 复用 + recall_runs 回执)、`test/serve-bin.test.js`(真子进程 + 多进程互写互读)、`test/standalone-api.test.js` 的 strictPort 用例(busy → reject,默认路径仍顺延)。 diff --git a/dsh-mneme/lib/api-standalone.js b/dsh-mneme/lib/api-standalone.js index 01191af5..6bccac82 100644 --- a/dsh-mneme/lib/api-standalone.js +++ b/dsh-mneme/lib/api-standalone.js @@ -30,8 +30,13 @@ const MAX_PORT_ATTEMPTS = 20; * the next MAX_PORT_ATTEMPTS-1 ports, and finally falls back to port 0 so the * OS assigns a free port. Multiple DSH profiles/instances sharing the default * port no longer leave the standalone API permanently unavailable. + * + * strictPort opts out of that recovery (dsh-mneme-serve daemon, #363): a + * long-lived third-party integration pins the URL, so silently hopping ports + * would make clients talk to nothing (or, worse, to a future different data + * plane). A busy configured port is a configuration error there — fail loud. */ -function listenWithRetry(server, startPort, host, logger) { +function listenWithRetry(server, startPort, host, logger, strictPort = false) { return new Promise((resolve, reject) => { let attempt = 0; const tryListen = (port) => { @@ -42,6 +47,10 @@ function listenWithRetry(server, startPort, host, logger) { }; const onError = (error) => { server.off("listening", onListening); + if (error?.code === "EADDRINUSE" && strictPort) { + reject(error); + return; + } if (error?.code === "EADDRINUSE" && attempt < MAX_PORT_ATTEMPTS - 1) { attempt++; const next = startPort + attempt; @@ -193,11 +202,13 @@ async function handlePutBody(res, service, logger, id, text) { * (crypto.randomBytes(24).toString("base64url")) and persisted when empty. * - port: explicit arg > persisted settings > config.externalApiPort > 8790. * - host: explicit arg > config.externalApiHost > "127.0.0.1". + * - strictPort: EADDRINUSE rejects instead of hopping ports (daemon mode; + * default false keeps the in-host sidecar recovery described above). * Returns { server, port, host, token, ready }: `port` is the effective bound * port (updated to the OS-assigned one after `ready` resolves when asked to * bind port 0), `ready` resolves once listening and rejects if the bind fails. */ -export function createStandaloneApi({ service, store, config = {}, logger, settings, port, host, maintenance }) { +export function createStandaloneApi({ service, store, config = {}, logger, settings, port, host, maintenance, strictPort = false }) { const persisted = settings?.getExternalApi?.() ?? {}; let token = typeof persisted.token === "string" ? persisted.token : ""; @@ -552,7 +563,7 @@ export function createStandaloneApi({ service, store, config = {}, logger, setti }); const ready = new Promise((resolve, reject) => { // listening handled by listenWithRetry - listenWithRetry(server, boundPort, boundHost, logger).then(resolve, reject); + listenWithRetry(server, boundPort, boundHost, logger, strictPort).then(resolve, reject); }); // server.listen is called inside listenWithRetry ready.then(() => { diff --git a/dsh-mneme/lib/serve.js b/dsh-mneme/lib/serve.js new file mode 100644 index 00000000..82ffba94 --- /dev/null +++ b/dsh-mneme/lib/serve.js @@ -0,0 +1,113 @@ +// src/serve.js —— 独立服务(daemon)运行时:mneme 在 DSH 宿主之外常驻的最小装配面 +// (#363 承诺的「官方推荐第三方挂载姿势」)。bin/dsh-mneme-serve.mjs 是它的 CLI 壳, +// Mneme Bridge 这类第三方也可直接 import 本模块自行托管生命周期。 +// +// 与宿主装配(src/index.js apply)的关系:只搬数据面那一半,每步注释锚定 index.js +// 来源行号。刻意不抽公共装配函数——apply 的其余环节(注入/工具/dream)与宿主 ctx +// 纠缠,防御段纪律是「最后动或不动」;这 60 行的漂移风险由 serve-bin 测试的多进程 +// 共存用例兜底(两侧真开同一个库互写互读)。 +// +// 第一期无 LLM:巩固(autoDream)与蒸馏(autoSummarize)结构上不在这里——巩固只 +// 属于 DSH 宿主进程,这就是 daemon 与宿主「单写者」的机械保证(AGENTS.md 的 +// externalApi/autoDream 单侧纪律),不依赖用户自觉。检索是关键词 + BM25(service +// 内建);向量管线由 PR2 的 semantic 抽取接入。 +import { mkdirSync } from "node:fs"; +import { homedir } from "node:os"; +import { join } from "node:path"; +import { createStore } from "./store.js"; +import { createSettings } from "./settings.js"; +import { createMirror, TYPE_FILE } from "./mirror.js"; +import { langOf } from "./lang.js"; +import { createService } from "./service.js"; +import { createMaintenance } from "./maintenance.js"; +import { createStandaloneApi } from "./api-standalone.js"; + +/** + * 组装并启动一个独立数据面。 + * @param {object} opts + * memoryDir — 数据目录;缺省与宿主同默认 ~/.dsh/memory(config.js:9),支持前导 ~。 + * port/host — 透传 createStandaloneApi;缺省走 kv external_api 持久值 > 8790/127.0.0.1。 + * logger — console 形状(info/warn/error,收单字符串);缺省 null(全链路容缺)。 + * strictPort — 默认 true:配置端口被占即失败(第三方把 URL 写死,顺延=静默打到 + * 错误端口)。显式 port=0(测试/OS 分配)不受影响。 + * @returns {api, store, service, settings, maintenance, tokenExisted, dispose} + * tokenExisted — 启动前 kv 里是否已有 token;false 时本次为首次生成,入口层可提示。 + */ +export function createServeRuntime({ memoryDir, port, host, logger = null, strictPort = true } = {}) { + // index.js:192-195:~ 展开只认前导;目录不存在时 node:sqlite 直接抛,先 mkdir。 + const dir = String(memoryDir || join(homedir(), ".dsh", "memory")).replace(/^~(?=$|[\\/])/, homedir()); + mkdirSync(dir, { recursive: true }); + + // 装配顺序照 index.js:197-292:store → settings → mirror → service → recoverMirror。 + const store = createStore(join(dir, "memory.db")); + const settings = createSettings(store.db); + // 第一期不吃宿主 config schema(schemastery 是宿主 peer 依赖,独立进程装不装随缘), + // 配最小集,与第三方 embedded 挂载实测同形:language 参与镜像渲染与去重标题; + // documentMemoryEnabled 关掉 document 子系统(daemon 不建 documentIndex,service + // 内部 if-guard 安全,index.js:261 的 documentIndex 仅宿主装配)。 + const cfg = { language: "zh", documentMemoryEnabled: false }; + const mirror = createMirror(dir, langOf(cfg)); + const service = createService({ store, mirror, config: cfg, logger }); + + // index.js:292:上次镜像同步失败留下的 dirty 状态,启动时安全重渲一次(内部自吞)。 + service.recoverMirror(); + + // index.js:319-332:人改镜像先合并——镜像文件里的手工编辑每次启动都赢。 + // readHumanEdits 全类型一次读齐:mergeHumanEdits 成功会重渲全部镜像,逐类型读改 + // 循环会拿没读到的类型覆盖掉未合并的编辑(index.js:324-327 注释同款坑)。 + const humanEdits = new Map(); + for (const type of Object.keys(TYPE_FILE)) humanEdits.set(type, mirror.readHumanEdits(type)); + for (const [type, edits] of humanEdits) { + if (edits.length) service.mergeHumanEdits(type, edits); + } + + // index.js:294-309:检索回执落 recall_runs(searchMemories 的 recordRecall 默认开, + // 第三方检索统计因此不缺数)。best-effort:回执写失败绝不影响检索本身。 + service.setRecallRecorder((recall) => { + try { + store.saveRecallRun({ + query: recall.query, + mode: recall.mode, + topK: recall.topK, + threshold: recall.threshold ?? null, + candidates: recall.candidates ?? [], + created_at: recall.createdAt + }); + } catch { /* non-fatal: recall recording is bookkeeping */ } + }); + + // index.js:636:#275 无损回收。daemon 带上它,POST /maintenance/reclaim 才有后端 + // (api-standalone 缺 maintenance 时该路由 503);同样不挂启动路径、不接定时器。 + const maintenance = createMaintenance({ store, config: cfg, logger }); + + // index.js:643-649 同一工厂;差别只有 strictPort 默认开(daemon 语义,见上)。 + // token 与宿主共用同一 kv 键 external_api:首次启动自动生成并持久化 + // (api-standalone.js:203-212),DSH 面板 / CLI / daemon 三方零配置共享凭证。 + const tokenExisted = Boolean(settings.getExternalApi?.()?.token); + const api = createStandaloneApi({ service, store, config: cfg, logger, settings, port, host, maintenance, strictPort }); + + return { + api, + store, + service, + settings, + maintenance, + tokenExisted, + /** + * 收尾：先停收新请求、等在途请求跑完，再关库——直接同步关库会让在途的 + * PUT/POST 撞上已关的 store（500 或丢写）。closeIdleConnections 排干 + * keep-alive 空闲连接（node ≥18.2，旧版无此 API 则跳过），否则 server.close + * 的回调要等 keep-alive 超时才触发。node:sqlite 对未 finalize 语句可能抛， + * 吞掉——WAL 会在下次打开时回放，已提交事务不丢。 + */ + async dispose() { + await new Promise((resolve) => { + try { + api.server.close(() => resolve()); + api.server.closeIdleConnections?.(); + } catch { resolve(); } + }); + try { store.close(); } catch { /* 同上 */ } + } + }; +} diff --git a/dsh-mneme/package.json b/dsh-mneme/package.json index da8f88db..6502c468 100644 --- a/dsh-mneme/package.json +++ b/dsh-mneme/package.json @@ -16,12 +16,16 @@ "icon": "assets/icon.svg", "bin": { "dsh-mneme": "bin/cli.mjs", - "dsh-mneme-mcp": "bin/dsh-mneme-mcp.mjs" + "dsh-mneme-mcp": "bin/dsh-mneme-mcp.mjs", + "dsh-mneme-serve": "bin/dsh-mneme-serve.mjs" }, "exports": { ".": { "default": "./lib/index.js" }, + "./serve": { + "default": "./lib/serve.js" + }, "./client": { "default": "./lib/client.js" }, diff --git a/dsh-mneme/src/api-standalone.js b/dsh-mneme/src/api-standalone.js index 01191af5..6bccac82 100644 --- a/dsh-mneme/src/api-standalone.js +++ b/dsh-mneme/src/api-standalone.js @@ -30,8 +30,13 @@ const MAX_PORT_ATTEMPTS = 20; * the next MAX_PORT_ATTEMPTS-1 ports, and finally falls back to port 0 so the * OS assigns a free port. Multiple DSH profiles/instances sharing the default * port no longer leave the standalone API permanently unavailable. + * + * strictPort opts out of that recovery (dsh-mneme-serve daemon, #363): a + * long-lived third-party integration pins the URL, so silently hopping ports + * would make clients talk to nothing (or, worse, to a future different data + * plane). A busy configured port is a configuration error there — fail loud. */ -function listenWithRetry(server, startPort, host, logger) { +function listenWithRetry(server, startPort, host, logger, strictPort = false) { return new Promise((resolve, reject) => { let attempt = 0; const tryListen = (port) => { @@ -42,6 +47,10 @@ function listenWithRetry(server, startPort, host, logger) { }; const onError = (error) => { server.off("listening", onListening); + if (error?.code === "EADDRINUSE" && strictPort) { + reject(error); + return; + } if (error?.code === "EADDRINUSE" && attempt < MAX_PORT_ATTEMPTS - 1) { attempt++; const next = startPort + attempt; @@ -193,11 +202,13 @@ async function handlePutBody(res, service, logger, id, text) { * (crypto.randomBytes(24).toString("base64url")) and persisted when empty. * - port: explicit arg > persisted settings > config.externalApiPort > 8790. * - host: explicit arg > config.externalApiHost > "127.0.0.1". + * - strictPort: EADDRINUSE rejects instead of hopping ports (daemon mode; + * default false keeps the in-host sidecar recovery described above). * Returns { server, port, host, token, ready }: `port` is the effective bound * port (updated to the OS-assigned one after `ready` resolves when asked to * bind port 0), `ready` resolves once listening and rejects if the bind fails. */ -export function createStandaloneApi({ service, store, config = {}, logger, settings, port, host, maintenance }) { +export function createStandaloneApi({ service, store, config = {}, logger, settings, port, host, maintenance, strictPort = false }) { const persisted = settings?.getExternalApi?.() ?? {}; let token = typeof persisted.token === "string" ? persisted.token : ""; @@ -552,7 +563,7 @@ export function createStandaloneApi({ service, store, config = {}, logger, setti }); const ready = new Promise((resolve, reject) => { // listening handled by listenWithRetry - listenWithRetry(server, boundPort, boundHost, logger).then(resolve, reject); + listenWithRetry(server, boundPort, boundHost, logger, strictPort).then(resolve, reject); }); // server.listen is called inside listenWithRetry ready.then(() => { diff --git a/dsh-mneme/src/serve.js b/dsh-mneme/src/serve.js new file mode 100644 index 00000000..82ffba94 --- /dev/null +++ b/dsh-mneme/src/serve.js @@ -0,0 +1,113 @@ +// src/serve.js —— 独立服务(daemon)运行时:mneme 在 DSH 宿主之外常驻的最小装配面 +// (#363 承诺的「官方推荐第三方挂载姿势」)。bin/dsh-mneme-serve.mjs 是它的 CLI 壳, +// Mneme Bridge 这类第三方也可直接 import 本模块自行托管生命周期。 +// +// 与宿主装配(src/index.js apply)的关系:只搬数据面那一半,每步注释锚定 index.js +// 来源行号。刻意不抽公共装配函数——apply 的其余环节(注入/工具/dream)与宿主 ctx +// 纠缠,防御段纪律是「最后动或不动」;这 60 行的漂移风险由 serve-bin 测试的多进程 +// 共存用例兜底(两侧真开同一个库互写互读)。 +// +// 第一期无 LLM:巩固(autoDream)与蒸馏(autoSummarize)结构上不在这里——巩固只 +// 属于 DSH 宿主进程,这就是 daemon 与宿主「单写者」的机械保证(AGENTS.md 的 +// externalApi/autoDream 单侧纪律),不依赖用户自觉。检索是关键词 + BM25(service +// 内建);向量管线由 PR2 的 semantic 抽取接入。 +import { mkdirSync } from "node:fs"; +import { homedir } from "node:os"; +import { join } from "node:path"; +import { createStore } from "./store.js"; +import { createSettings } from "./settings.js"; +import { createMirror, TYPE_FILE } from "./mirror.js"; +import { langOf } from "./lang.js"; +import { createService } from "./service.js"; +import { createMaintenance } from "./maintenance.js"; +import { createStandaloneApi } from "./api-standalone.js"; + +/** + * 组装并启动一个独立数据面。 + * @param {object} opts + * memoryDir — 数据目录;缺省与宿主同默认 ~/.dsh/memory(config.js:9),支持前导 ~。 + * port/host — 透传 createStandaloneApi;缺省走 kv external_api 持久值 > 8790/127.0.0.1。 + * logger — console 形状(info/warn/error,收单字符串);缺省 null(全链路容缺)。 + * strictPort — 默认 true:配置端口被占即失败(第三方把 URL 写死,顺延=静默打到 + * 错误端口)。显式 port=0(测试/OS 分配)不受影响。 + * @returns {api, store, service, settings, maintenance, tokenExisted, dispose} + * tokenExisted — 启动前 kv 里是否已有 token;false 时本次为首次生成,入口层可提示。 + */ +export function createServeRuntime({ memoryDir, port, host, logger = null, strictPort = true } = {}) { + // index.js:192-195:~ 展开只认前导;目录不存在时 node:sqlite 直接抛,先 mkdir。 + const dir = String(memoryDir || join(homedir(), ".dsh", "memory")).replace(/^~(?=$|[\\/])/, homedir()); + mkdirSync(dir, { recursive: true }); + + // 装配顺序照 index.js:197-292:store → settings → mirror → service → recoverMirror。 + const store = createStore(join(dir, "memory.db")); + const settings = createSettings(store.db); + // 第一期不吃宿主 config schema(schemastery 是宿主 peer 依赖,独立进程装不装随缘), + // 配最小集,与第三方 embedded 挂载实测同形:language 参与镜像渲染与去重标题; + // documentMemoryEnabled 关掉 document 子系统(daemon 不建 documentIndex,service + // 内部 if-guard 安全,index.js:261 的 documentIndex 仅宿主装配)。 + const cfg = { language: "zh", documentMemoryEnabled: false }; + const mirror = createMirror(dir, langOf(cfg)); + const service = createService({ store, mirror, config: cfg, logger }); + + // index.js:292:上次镜像同步失败留下的 dirty 状态,启动时安全重渲一次(内部自吞)。 + service.recoverMirror(); + + // index.js:319-332:人改镜像先合并——镜像文件里的手工编辑每次启动都赢。 + // readHumanEdits 全类型一次读齐:mergeHumanEdits 成功会重渲全部镜像,逐类型读改 + // 循环会拿没读到的类型覆盖掉未合并的编辑(index.js:324-327 注释同款坑)。 + const humanEdits = new Map(); + for (const type of Object.keys(TYPE_FILE)) humanEdits.set(type, mirror.readHumanEdits(type)); + for (const [type, edits] of humanEdits) { + if (edits.length) service.mergeHumanEdits(type, edits); + } + + // index.js:294-309:检索回执落 recall_runs(searchMemories 的 recordRecall 默认开, + // 第三方检索统计因此不缺数)。best-effort:回执写失败绝不影响检索本身。 + service.setRecallRecorder((recall) => { + try { + store.saveRecallRun({ + query: recall.query, + mode: recall.mode, + topK: recall.topK, + threshold: recall.threshold ?? null, + candidates: recall.candidates ?? [], + created_at: recall.createdAt + }); + } catch { /* non-fatal: recall recording is bookkeeping */ } + }); + + // index.js:636:#275 无损回收。daemon 带上它,POST /maintenance/reclaim 才有后端 + // (api-standalone 缺 maintenance 时该路由 503);同样不挂启动路径、不接定时器。 + const maintenance = createMaintenance({ store, config: cfg, logger }); + + // index.js:643-649 同一工厂;差别只有 strictPort 默认开(daemon 语义,见上)。 + // token 与宿主共用同一 kv 键 external_api:首次启动自动生成并持久化 + // (api-standalone.js:203-212),DSH 面板 / CLI / daemon 三方零配置共享凭证。 + const tokenExisted = Boolean(settings.getExternalApi?.()?.token); + const api = createStandaloneApi({ service, store, config: cfg, logger, settings, port, host, maintenance, strictPort }); + + return { + api, + store, + service, + settings, + maintenance, + tokenExisted, + /** + * 收尾：先停收新请求、等在途请求跑完，再关库——直接同步关库会让在途的 + * PUT/POST 撞上已关的 store（500 或丢写）。closeIdleConnections 排干 + * keep-alive 空闲连接（node ≥18.2，旧版无此 API 则跳过），否则 server.close + * 的回调要等 keep-alive 超时才触发。node:sqlite 对未 finalize 语句可能抛， + * 吞掉——WAL 会在下次打开时回放，已提交事务不丢。 + */ + async dispose() { + await new Promise((resolve) => { + try { + api.server.close(() => resolve()); + api.server.closeIdleConnections?.(); + } catch { resolve(); } + }); + try { store.close(); } catch { /* 同上 */ } + } + }; +} diff --git a/dsh-mneme/test/serve-bin.test.js b/dsh-mneme/test/serve-bin.test.js new file mode 100644 index 00000000..01c77d09 --- /dev/null +++ b/dsh-mneme/test/serve-bin.test.js @@ -0,0 +1,120 @@ +import test from "node:test"; +import assert from "node:assert/strict"; +import { spawn } from "node:child_process"; +import { mkdtempSync, rmSync } from "node:fs"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; +import { fileURLToPath } from "node:url"; +import { createStore } from "../src/store.js"; +import { createService } from "../src/service.js"; +import { createSettings } from "../src/settings.js"; + +// serve bin 冒烟 + 多进程共存锁(daemon 唯一的真新风险点):daemon 子进程与测试进程 +// 同时打开同一 memory.db(WAL + busy_timeout 本就为此设计),互写互读。spawn 必须 +// 异步(maintenance.test.js 的教训:同步 spawn 会堵住本进程事件循环);Windows 下 +// child.kill() 是硬终止,不断言信号路径,只断言可达性与退出。 + +const BIN = fileURLToPath(new URL("../bin/dsh-mneme-serve.mjs", import.meta.url)); + +function wait(ms) { + return new Promise((r) => setTimeout(r, ms)); +} + +async function waitFor(fn, timeoutMs, what) { + const deadline = Date.now() + timeoutMs; + let lastErr; + while (Date.now() < deadline) { + try { + return await fn(); + } catch (err) { + lastErr = err; + } + await wait(150); + } + throw new Error(`timeout waiting for ${what}: ${lastErr?.message ?? lastErr}`); +} + +test("serve bin: 值旗标缺值直接报错退出(--port 后无值不能落成 Number(true)=1)", { timeout: 30000 }, async () => { + // CodeRabbit on #364:--port 紧跟另一个旗标或结束时,旧解析把它存成 true, + // Number(true)=1 通过校验 → 静默改绑端口 1(EACCES 误导排错方向)。 + const child = spawn(process.execPath, [BIN, "--memory-dir", mkdtempSync(join(tmpdir(), "mneme-serve-arg-")), "--port"], { + stdio: ["ignore", "pipe", "pipe"] + }); + let stderr = ""; + child.stderr.on("data", (d) => { stderr += d; }); + const [code] = await new Promise((resolve) => { + child.on("exit", (c) => resolve([c])); + }); + assert.notEqual(code, 0, "missing value must exit non-zero"); + assert.ok(stderr.includes("缺少参数值"), `stderr should name the missing value, got: ${stderr.slice(-200)}`); +}); + +test("serve bin: spawn 冒烟;daemon 与宿主进程同库互写互读", { timeout: 120000 }, async () => { + const dir = mkdtempSync(join(tmpdir(), "mneme-serve-bin-")); + + // 预置 token:测试进程先开一次库写进 kv external_api,daemon 复用之 —— 同时验证 + // 「与宿主面板/CLI 共用同一凭证」这条零配置承诺在真子进程里成立。 + const seedStore = createStore(join(dir, "memory.db")); + const seedSettings = createSettings(seedStore.db); + const TOKEN = "serve-bin-test-token-0123456789abcdef"; + seedSettings.setExternalApi({ token: TOKEN }); + const peer = createService({ store: seedStore, mirror: null, config: {} }); + peer.saveWithDedupe({ type: "project", title: "peer 进程直写", content: "测试进程经 createStore 写入", importance: 3 }); + + const child = spawn(process.execPath, [BIN, "--memory-dir", dir, "--port", "0"], { + stdio: ["ignore", "pipe", "pipe"] + }); + let stdout = ""; + let stderr = ""; + child.stdout.on("data", (d) => { stdout += d; }); + child.stderr.on("data", (d) => { stderr += d; }); + + try { + // stdout 唯一机器可读行:listening 行(解析实际端口) + await waitFor(() => { + if (child.exitCode !== null) { + throw new Error(`daemon exited early (code ${child.exitCode}): ${stderr.slice(-400)}`); + } + if (!stdout.includes("listening on http://")) throw new Error("no listening line yet"); + return true; + }, 20000, "listening line"); + const port = Number(stdout.match(/:(\d+)/)[1]); + const base = `http://127.0.0.1:${port}`; + const auth = { authorization: `Bearer ${TOKEN}` }; + + await waitFor(async () => { + const res = await fetch(`${base}/health`); + assert.equal(res.status, 200); + return true; + }, 10000, "daemon /health"); + + // 双向可见性 ①:测试进程直写 → daemon HTTP 检索可见 + await waitFor(async () => { + const res = await fetch(`${base}/search?q=${encodeURIComponent("peer 进程直写")}`, { headers: auth }); + assert.equal(res.status, 200); + const body = await res.json(); + assert.ok((body.items ?? []).some((m) => m.title === "peer 进程直写")); + return true; + }, 10000, "peer write visible via daemon"); + + // 双向可见性 ②:daemon HTTP 写 → 测试进程 service 同步检索可见 + const save = await fetch(`${base}/memories`, { + method: "POST", + headers: { ...auth, "content-type": "application/json" }, + body: JSON.stringify({ type: "project", title: "daemon HTTP 写入", content: "serve 进程经数据面写入", importance: 3 }) + }); + assert.ok(save.status === 201 || save.status === 200, `daemon save status ${save.status}`); + assert.ok( + peer.search("daemon HTTP 写入", { limit: 5 }).some((m) => m.title === "daemon HTTP 写入"), + "daemon-side write must be visible to the other process immediately" + ); + } finally { + child.kill(); + await new Promise((r) => { + if (child.exitCode !== null) r(); + else child.on("exit", r); + }); + try { seedStore.close(); } catch { /* 同上 */ } + try { rmSync(dir, { recursive: true, force: true }); } catch { /* win 文件锁偶发,残留临时目录无害 */ } + } +}); diff --git a/dsh-mneme/test/serve.test.js b/dsh-mneme/test/serve.test.js new file mode 100644 index 00000000..5d29b987 --- /dev/null +++ b/dsh-mneme/test/serve.test.js @@ -0,0 +1,105 @@ +import test from "node:test"; +import assert from "node:assert/strict"; +import { mkdtempSync, rmSync } from "node:fs"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; +import { createServeRuntime } from "../src/serve.js"; + +// daemon 装配面锁(test/serve-bin.test.js 另有真子进程 + 多进程共存): +// in-process 起真 HTTP(port 0,OS 分配),fetch 走 health / 401 / save / search / +// token 持久化复用全链路。锁的是「daemon 数据面 = api-standalone 同一工厂」这一契约。 + +function tmpDir() { + return mkdtempSync(join(tmpdir(), "mneme-serve-")); +} + +test("serve: /health 免鉴权,业务路由无 token 401", async () => { + const dir = tmpDir(); + const rt = createServeRuntime({ memoryDir: dir, port: 0 }); + await rt.api.ready; + try { + const base = `http://127.0.0.1:${rt.api.port}`; + const health = await fetch(`${base}/health`); + assert.equal(health.status, 200); + assert.deepEqual(await health.json(), { ok: true }); + + const noAuth = await fetch(`${base}/search?q=x`); + assert.equal(noAuth.status, 401); + assert.deepEqual(await noAuth.json(), { error: "unauthorized" }); + } finally { + rt.dispose(); + try { rmSync(dir, { recursive: true, force: true }); } catch { /* win 文件锁偶发 */ } + } +}); + +test("serve: save → search 走通;token 持久化 kv;二次启动复用同一 token;检索回执落 recall_runs", async () => { + const dir = tmpDir(); + const rt = createServeRuntime({ memoryDir: dir, port: 0 }); + await rt.api.ready; + const base = `http://127.0.0.1:${rt.api.port}`; + const auth = { authorization: `Bearer ${rt.api.token}` }; + + // 全新目录:首次启动生成 token + assert.equal(rt.tokenExisted, false); + + const save = await fetch(`${base}/memories`, { + method: "POST", + headers: { ...auth, "content-type": "application/json" }, + body: JSON.stringify({ type: "preference", title: "serve 冒烟偏好", content: "回复保持简短", importance: 4 }) + }); + assert.equal(save.status, 201); + + const search = await fetch(`${base}/search?q=${encodeURIComponent("简短")}`, { headers: auth }); + assert.equal(search.status, 200); + const searchBody = await search.json(); + assert.ok((searchBody.items ?? []).some((m) => m.title === "serve 冒烟偏好")); + + // token 持久化在 kv external_api(与 DSH 面板/CLI 共用同一凭证通道) + assert.equal(rt.settings.getExternalApi().token, rt.api.token); + + // 检索回执:recorder 已接(index.js:294-309 同款),recall_runs 不缺数 —— #363 回帖 + // 向 bridge 承诺过「第三方检索的复用统计不受影响」,这条就是该承诺的回归锁。 + const recallRows = rt.store.db.prepare("SELECT COUNT(*) AS n FROM recall_runs").get(); + assert.ok(recallRows.n >= 1, "recall_runs should record the search above"); + + rt.dispose(); + + // 二次启动同目录:token 复用不重新生成(firstBoot 提示只在真正首次出现) + const rt2 = createServeRuntime({ memoryDir: dir, port: 0 }); + await rt2.api.ready; + try { + assert.equal(rt2.tokenExisted, true); + assert.equal(rt2.api.token, rt.api.token); + } finally { + rt2.dispose(); + try { rmSync(dir, { recursive: true, force: true }); } catch { /* 同上 */ } + } +}); + +test("serve: dispose 后端口可复用(同端口连起两轮不踩 strictPort)", async () => { + const dir = tmpDir(); + const rt = createServeRuntime({ memoryDir: dir, port: 0 }); + await rt.api.ready; + const port = rt.api.port; + // dispose 现为 async:等 server.close 回调(在途请求排干)后再重绑 + await rt.dispose(); + // 端口释放可能有内核级迟滞,重试绑定而不是假设立即可用 + let rebound = null; + for (let i = 0; i < 10 && !rebound; i++) { + try { + const rt2 = createServeRuntime({ memoryDir: dir, port }); + await rt2.api.ready; + rebound = rt2; + } catch (err) { + if (err?.code !== "EADDRINUSE") throw err; + await new Promise((r) => setTimeout(r, 100)); + } + } + try { + assert.ok(rebound, "port should be rebindable after dispose"); + assert.equal(rebound.api.port, port); + } finally { + await rebound?.dispose(); + try { rmSync(dir, { recursive: true, force: true }); } catch { /* 同上 */ } + } +}); diff --git a/dsh-mneme/test/standalone-api.test.js b/dsh-mneme/test/standalone-api.test.js index 0e24ba99..30d47d63 100644 --- a/dsh-mneme/test/standalone-api.test.js +++ b/dsh-mneme/test/standalone-api.test.js @@ -542,3 +542,30 @@ test("GET /search honors the occurred_at window", async () => { close(); } }); + +test("strictPort: busy port rejects instead of hopping; default path still hops", async () => { + // daemon 语义锁(#363):strictPort 下配置端口被占 = 配置错误,明确失败 —— + // 顺延会让把 URL 写死的第三方客户端静默打到错误端口。占口用裸 net server + // (OS 分配,不与并行测试抢固定号)。 + const { default: net } = await import("node:net"); + const blocker = net.createServer(); + await new Promise((resolve) => blocker.listen(0, "127.0.0.1", resolve)); + const busyPort = blocker.address().port; + + const store = createStore(":memory:"); + const service = createService({ store, mirror: null, config: {} }); + const settings = createSettings(store.db); + try { + const strict = createStandaloneApi({ service, store, config: {}, settings, logger: null, port: busyPort, strictPort: true }); + await assert.rejects(strict.ready, (err) => err?.code === "EADDRINUSE"); + + // 默认路径(不传 strictPort)行为不变:顺延成功;具体端口不断言(避免抢号)。 + const hopping = createStandaloneApi({ service, store, config: {}, settings, logger: null, port: busyPort }); + await hopping.ready; + assert.notEqual(hopping.port, busyPort, "default policy hops off a busy port"); + hopping.server.close(); + } finally { + blocker.close(); + store.close(); + } +});