Skip to content

feat(data-layout): auto-migrate a legacy .teamai into the partition (#374 P1-3) - #439

Merged
jeff-r2026 merged 3 commits into
mainfrom
worktree-issue-374-p1-3-auto-migration
Sep 8, 2026
Merged

feat(data-layout): auto-migrate a legacy .teamai into the partition (#374 P1-3)#439
jeff-r2026 merged 3 commits into
mainfrom
worktree-issue-374-p1-3-auto-migration

Conversation

@jeff-r2026

Copy link
Copy Markdown
Collaborator

What & why

P1-3 of issue #374. Earlier P1 PRs (#397/#402/#406/#414/#417) routed new
installs' machine data to the per-project partition ~/.teamai/projects/<slug>/
and read old installs via a legacy fallback. This PR is the final piece: it
automatically migrates a real legacy <repo>/.teamai/ into the partition on
the first write command, so an upgraded project ends up with zero teamai
residue
in its workspace.

How it works

Trigger — the global preAction hook (src/index.ts), narrowed twice:

  • only init / pull / push (read-only commands keep using the double-read fallback);
  • hook-dispatch excluded via TEAMAI_HOOK_SUBCOMMANDS (a high-frequency silent path must never move 12 MB);
  • --dry-run previews without writing.

Gate (planMigration, deliberately not detectProjectConfig — that
short-circuits on an existing partition and runs the self-heal bootstrap, both of
which would mask the raw legacy state): act iff git repo + legacy config.yaml
exists + scope: project + kind != self. self mode is a hard no-op — its
.teamai/ is team knowledge committed to main.

Steps (runMigration) — copy → verify → atomic rename, holding
<legacyDir>/.sync-lock (the exact lock an un-migrated pull/push contends on)
across the whole window:

  1. copy legacyDir → <partition>.staging with raw fse.copy, not copyDir (copyDir filters out .git and would corrupt the team-repo clone); skip disposable worktrees + lock files;
  2. verify staging (config parses; team-repo git rev-parse HEAD works; every entry present);
  3. rebase repo.localPath (was <legacyDir>/team-repo) onto the partition;
  4. atomic rename(staging → partition);
  5. write the anchor reverse-lookup file (the slug is a one-way sha256);
  6. release the lock, then retire legacyDir → .teamai.bak/ (never auto-deleted — the manual rollback path).

Interrupt recovery — a crash before step 4 leaves the partition absent and the
source intact (discard .staging/, retry). A crash between the partition rename
and the source retire is finished by the next write command's retire-only plan
(retire the lingering legacy dir without re-copying onto the authoritative
partition), so the legacy dir — incl. its plaintext env — never lingers forever.

Downgrade is not supported — flagged in the design doc and the bilingual usage
guides, which also correct the now-stale <cwd>/.teamai layout to the partition.

Test Plan

Unit (src/__tests__/migrate.test.ts, 19 tests) — all green

Builds a real legacy layout (real git anchors + a real nested git clone), not
an empty fixture. Covers: every plan gate (non-git / user-scope / self / malformed
/ partition-exists→retire-only); .git survival; repo.localPath rebase (inside
vs. outside legacyDir); lock not buried in backup; disposable worktrees skipped;
idempotence; retire-only finishes an interrupted run without overwriting the
partition
; corrupt staged clone aborts, source untouched, no partition/.bak/
.staging
; http-mode install migrates; leftover-staging recovery; dry-run
no-op.

Full suite

npx tsc --noEmit clean · npm run build OK · npx vitest run2747 passed.

Real-CLI end-to-end (built dist/index.js, isolated HOME + real git repos)

Scenario Result
--dry-run pull ✅ partition not created, legacy untouched, no .bak
real pull migrates ✅ config/state/index/env/managed-mcp → partition; anchor written; team-repo/.git survives, HEAD matches, git status works
repo.localPath rebased ✅ names <partition>/team-repo (resolves to a usable clone), no longer the legacy path
workspace residue ✅ legacy .teamai gone; .teamai.bak/ kept with old config; no live .sync-lock in backup
idempotent second pull ✅ no re-migration, no legacy re-created
push triggers migration
status (read-only) ✅ does not migrate
hook-dispatch ✅ does not migrate
self mode ✅ no-op — partition not created, committed knowledge untouched, no .bak
interrupted-state recovery (partition built + legacy lingering) ✅ next pull retires the leftover legacy, moves plaintext env out of the workspace, partition not overwritten

Docs

docs/designs/data-directory-layout.md (P1 marked implemented + full migration
section) and bilingual docs/usage-guide{,.zh-CN}.md (partition layout + upgrade/
migration note + downgrade warning) updated.

Out of scope (per issue)

teamai migrate/gc/--revert commands; cross-project shared clone; P2 (self
slimming); P3 (constant functionization + status --all).

@jeff-r2026

Copy link
Copy Markdown
Collaborator Author

两个问题都已本地复现并修复(commit 8f07b02),各配了真实回归测试。

1. [P1] 自动迁移永久删除已有备份
成立。retireLegacy 在 rename 前无条件 remove(backup),已有的 .teamai.bak/(上次迁移留下的或用户自己的)会被删除,普通 teamai pull 升级即造成不可恢复的数据丢失。
修复:绝不删除已有备份,改为选第一个空闲名——.teamai.bak,已存在则 .teamai.bak.1.teamai.bak.2

2. [P1] 备份改名可能让凭据进入 Git 提交
成立,且已复现:老装的 .teamai/ 仅靠仓库根 .gitignore.teamai/ 规则保护,该规则不匹配 .teamai.bak/,迁移后 git add -A + git show :.teamai.bak/env 能读出明文凭据。
修复:改名前先在源目录内写一个自包含的 .gitignore(内容 *),使备份无论最终叫什么名字、无论仓库用哪种忽略写法,都忽略自身全部内容——不存在「凭据短暂落在未忽略目录」的窗口。

验证(真实 CLI,dist/index.js):

  • 已有 .teamai.bak/only-copy.txt 迁移后保留,本次备份落 .teamai.bak.1,partition 正确,工作区零残留
  • 迁移后 git add -A.teamai.bak 无任何文件被暂存;git show :.teamai.bak/env:.teamai.bak/token 均失败;备份文件仍在磁盘上(回滚路径完好)
  • 三种仓库根忽略写法(.teamai//.teamai/.teamai)均通过

回归测试新增 2 个:「never overwrites an existing .teamai.bak」「keeps the retired backup git-ignored so a git add -A cannot leak its secrets」。全量单测 2749 passed,tsc / build 均通过。design 文档的 retire 步骤已同步这两项安全措施。

…374 P1-3)

An install created before partitioning keeps its machine data in the business
repo at <workspaceRoot>/.teamai/. P1-2 routed NEW installs to the partition and
read old installs via a legacy fallback; this moves a real legacy .teamai INTO
the partition on the next write command so the workspace ends up zero-residue.

src/migrate.ts:
- planMigration(): gate WITHOUT detectProjectConfig (which short-circuits on an
  existing partition and runs the self-heal bootstrap — both would mask the raw
  legacy state). Migrate iff git repo + legacy config.yaml exists + no partition
  config yet + scope=project + kind!=self. self mode is a hard no-op (its
  .teamai is team knowledge committed to main).
- runMigration(): copy -> verify -> atomic rename so an interruption never
  leaves data half-in-both-places. Holds <legacyDir>/.sync-lock (the exact path
  an un-migrated pull/push contends on) across the whole window; contention
  skips (idempotent). Copies with raw fse.copy NOT copyDir — copyDir filters out
  .git and would corrupt the team-repo clone. Skips disposable worktrees
  (reports-wt/knowledge-wt) and lock files. Writes an anchor reverse-lookup file
  (the slug is a one-way sha256). Retires the source to .teamai.bak (never
  auto-deleted — the manual rollback path).

index.ts: wire maybeMigrate into the global preAction hook (now async), narrowed
to init/pull/push and excluding TEAMAI_HOOK_SUBCOMMANDS (hook-dispatch must
never move 12MB). --dry-run previews without writing.

Downgrade after migration is not supported — flagged in the design doc and the
bilingual usage guides, which also correct the now-stale "<cwd>/.teamai" layout
to the partition.

Tests: src/__tests__/migrate.test.ts builds a REAL legacy layout (real anchors +
a real nested git clone) and covers every gate, .git survival, atomicity,
idempotence, interrupt recovery, and dry-run. Verified end-to-end with the built
CLI: dry-run no-op, real pull/push migrate, git clone stays usable, workspace
zero-residue, self mode / status / hook-dispatch never migrate.
…ne + clean failure (#374 P1-3 review)

Adversarial self-review of the migration found three issues; all fixed with
regression tests.

[H1] A crash between the partition rename (step 3) and the source retire (step 5)
left the partition authoritative but the legacy dir — including its plaintext env
— lingering in the workspace forever: the next run's planMigration hit the
"partition exists" check and returned null, so the legacy dir was never retired,
breaking the zero-residue guarantee. Fix: planMigration now returns a
'retire-only' plan when the partition exists but a legacy dir still lingers;
runMigration finishes the job by retiring the leftover to .teamai.bak WITHOUT
re-copying onto the authoritative partition. The same path handles the
under-lock TOCTOU case (a sibling built the partition while we waited).

[L5] verifyStaging now smoke-checks the staged team-repo clone with
`git rev-parse HEAD` (on staging, before the rename), so a partial/corrupt copy
aborts with the source untouched instead of promoting a broken clone the next
pull would choke on. Existence of .git alone is no longer taken as proof.

[M3] The async preAction hook under program.parse() would surface a migration
failure as a raw unhandled-rejection stack. maybeMigrate is now wrapped: on
failure teamai prints a clean error ("your original .teamai is unchanged, re-run
to retry") and exits non-zero, rather than crashing into the command on partial
state.

Tests: retire-only finishes an interrupted run without overwriting the
partition; a corrupt staged clone aborts leaving source + no partition/.bak/
.staging; http-mode install (no team-repo) migrates. Verified end-to-end with
the built CLI: the interrupted-state pull retires the lingering legacy dir and
moves the plaintext env out of the workspace while keeping the partition intact.
…ed (#374 P1-3 review)

Two P1 issues found in review of the retireLegacy step, both reproduced locally
and fixed with regression tests.

[P1] Existing .teamai.bak was destroyed. retireLegacy did an unconditional
remove(backup) before the rename, so a pre-existing `.teamai.bak/` (from a prior
migration or the user's own) was deleted — a plain `teamai pull` on upgrade could
cause unrecoverable data loss. Fix: never remove an existing backup; pick the
first FREE name instead (`.teamai.bak`, else `.teamai.bak.1`, …).

[P1] The renamed backup could leak credentials into git. An old install's
`.teamai/` was often protected ONLY by a repo-root `.gitignore` rule matching
`.teamai/`, which does NOT match `.teamai.bak/`. After the rename a `git add -A`
staged the plaintext env/token (confirmed: `git show :.teamai.bak/env` returned
the secret). Fix: drop a self-contained `.gitignore` (`*`) INTO the dir BEFORE
renaming, so the backup ignores its own contents regardless of its final name or
the repo's ignore rules — no window in which the credentials sit unignored.

Tests: a pre-existing .teamai.bak is preserved while the migration backup goes to
.teamai.bak.1; after migration `git add -A` stages nothing under .teamai.bak and
`git show :.teamai.bak/{env,token}` both fail, while the files remain on disk for
rollback. Verified end-to-end against the built CLI across three repo-root ignore
spellings (`.teamai/`, `/.teamai/`, `.teamai`).
@jeff-r2026
jeff-r2026 force-pushed the worktree-issue-374-p1-3-auto-migration branch from 8f07b02 to 0fa78fa Compare September 8, 2026 12:41
@jeff-r2026
jeff-r2026 merged commit 999f57c into main Sep 8, 2026
7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant