[phernandez]

Why

Files copied into a newly created project's empty folder before the next watcher reload produce no events and stay unindexed. The watcher also uses a startup config snapshot that can exclude projects added later.

Fixes #1817. Carries forward @TPSJASPER's #1861, cherry-picked with original authorship and sign-off.

What Changed

• Reload current project config when selecting local projects to watch.
• After attaching a newly seen project's watch, run a catch-up index pass. Preserve startup scan ownership and project identity across path moves.
• Retain the contributor's retry, cancellation and reload-drain coverage.
• Handle continuous ignored events during attachment: observe raw batches in catch-up cycles and apply the existing visibility filter before indexing.
• Add a production MCP lifecycle integration test with a real native watcher, database, file indexing and embeddings. It creates a project through bm_project, copies a note, checks search, then edits the file and checks search again while hidden-file activity continues.

Implementation Details

The catch-up starts only after awatch has attached its watches. Running in the event consumer prevents the scan and event batches from overlapping. Project identities track completed catch-up, so changing an existing project's path does not scan a still-empty move destination and delete its index.

The new integration test found an additional gap in the original contribution: watchfiles suppresses batches whose events are all filtered. Continuous hidden-file activity can therefore suppress both observable batches and timeout yields, preventing catch-up indefinitely. New-project cycles receive raw batches and filter them locally. Only an actual quiet batch clears the drain guard.

The integration test uses the real MCP lifespan and normal database initialization, rather than API overrides or application mocks. Its only environment overrides select production runtime behavior; a bounded poll waits for observable search results. Existing inline tests remain available for fast feedback.

Testing

• The production lifecycle test failed against the unmodified cherry-picked fix(sync): index newly watched local projects after attachment #1861 (45-second search deadline); passes with the ignored-event fix.
• uv run pytest tests/index -q -o addopts= --tb=short: 221 passed.
• uv run pytest tests/index/test_watch_new_projects.py tests/mcp/test_new_project_watching.py tests/services/test_initialization.py tests/index/test_watch_service.py -q -o addopts= --tb=short: 33 passed.
• uv run pytest test-int/mcp/test_new_project_watch_lifecycle.py -q -o addopts= --tb=short: passed on SQLite with continuous hidden-file activity.
• BASIC_MEMORY_TEST_POSTGRES=1 uv run pytest test-int/mcp/test_new_project_watch_lifecycle.py tests/index/test_watch_new_projects.py tests/services/test_initialization.py -q -o addopts= --tb=short: 24 passed.
• just fast-check: lint, format and full typecheck passed.
• just doctor: sandboxed write, index and search passed.
• git diff --check: passed.

Risks / Follow-ups

New projects receive one project-wide scan after watch attachment, which can delay event processing while that scan runs. Reload deferral remains bounded as in the contributor's implementation. This changes local watching only; hosted indexing is unaffected. Existing project path moves retain their current behavior.

[chatgpt-codex-connector]

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review	Status	Commit	Review trigger
📝 Code Review	✅ Completed 2026-10-25T16:50:11.107699Z	ed0b0c2	PR opened

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.