Skip to content

Fix source-aware subpage context traversal - #1842

Merged
laboef1900 merged 5 commits into
devfrom
feature/1791-source-aware-subpages
Nov 5, 2026
Merged

laboef1900 merged 5 commits into
devfrom
feature/1791-source-aware-subpages

Conversation

@laboef1900

Copy link
Copy Markdown
Contributor

Summary

Fixes #1791. Part of #1827.

  • Resolve the initial LLM context parent and every recursive parent key by page source: Confluence ID for Confluence pages, internal ID for standalone pages even when they retain a historical Confluence ID. No historical-ID traversal alias.
  • Preserve the existing visibility predicates, cycle guard, breadth/depth and character limits. Other hierarchy APIs and new-page parent validation are unchanged.

Evidence

  • Real PostgreSQL regression covers standalone parent/child/grandchild with null and historical IDs, wrong-key decoys, private descendants, Confluence keys, and depth, cycle, page-count and character limits. npx vitest run src/domains/confluence/services/subpage-context.standalone.test.ts — 5 passed.
  • Real listening loopback /api/llm/ask and local loopback provider, real PostgreSQL, Redis and auth; historical standalone grandchild reaches the provider prompt. Sanitized env -i; npx vitest run src/routes/llm/llm-ask-cache-authorization.integration.test.ts -t 'includes historical standalone grandchildren' — 1 passed, 6 unselected. The test setup sets its own localhost Ollama fallback; the provider used is explicitly assigned to the local fixture.
  • Focused npx vitest run src/routes/llm/llm-ask.test.ts — 102 passed; prompt-level allow/deny behavior retained.
  • Node v24.22.1. Parent's global-r2: npm run test -w backend and npm run test -w frontend passed; npx vitest run --config e2e/vitest.config.ts passed 30/30. Initial global-r1 npm run lint, npm run typecheck, npm run build and npm audit --omit=dev --audit-level=high passed and are reused: production source and dependencies were unchanged by the subsequent test-only correction. Root contract build passed separately.
  • Initial npm test failed on an obsolete mock-argument assertion caused by this change; corrected to retain prompt behavior and reran the affected suite and full backend/frontend above. Parent also tried nonexistent npm run test:e2e:unit; that harness error is retained and replaced by the actual e2e Vitest command above. Neither initial failure is represented as a pass.
  • Base revision 5edff9e08dad25e9db1c5668ff211c794fc9c2fd; global-r1 patch SHA-256 d8571504f2e9f7ed254caf210b10d4d0337969e2d98d6fb19f3f54be2a15e7fd, source manifest 542b809adc4540f2f68cef09ad7508a4c03c57763bc17a346c287a3297e58ca0; final patch SHA-256 b1fc139e5346cdd887aaeed9829ab0a27a1c964c53a408ee5fb43d0e71628550, source manifest 129fa4f4ef2c217110d71c6e76349afb2251fced5408a921974355780da70d8d. Raw logs and provenance retained outside the worktree in epic-1827-evidence/1791/.

Risk / rollback

This changes only sub-page context traversal for historical standalone IDs; unrelated hierarchy readers retain their existing behavior. Reverting the commit restores the old behavior, including the missing descendants; no schema/data migration is involved.

@laboef1900 laboef1900 left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Independent Astra-high review of published 7468ca4: correct, no actionable findings. Both initial callers and recursive queue steps use source-aware canonical parent keys; standalone historical Confluence IDs are not aliases. Visibility/cycle/depth/page/character constraints remain intact. Inspected real-PG hierarchy/visibility regressions, sanitized listening Ask/loopback provider proof, focused25+recursion5+Ask102, corrected full backend7171/frontend5799/root unit30, final lint/typecheck and source attestation. Initial stale mock-argument and nonexistent npm-script failures retained, corrected without weakening behavioral coverage. Applicable final-head CI including Playwright completed successfully. Reviewer inspected source/evidence read-only; no checks rerun or GitHub approval claimed.

@laboef1900 laboef1900 left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Independent review: PR #1842

Issue: #1791 (bug; no NO AI label). Target: dev.

Head: 7468ca417eb5c089057322a842678ddc54679ff6
Base: 5edff9e08dad25e9db1c5668ff211c794fc9c2fd

Verdict

Correct; no confirmed patch-introduced actionable findings. Confidence: 0.96. Review disposition: COMMENT (read-only source/evidence assessment, not newly executed verification or GitHub approval).

Scope and boundary review

Full eleven-file diff; changed implementation and regression contexts; all assembleSubPageContext/fetchSubPages consumers; Improve/Quality/Summarize helper use and Ask authorization/provider boundary. Source-aware traversal changes only parent keys, not route ID lookup precedence or unrelated hierarchy APIs.

Acceptance criteria

  • Canonical source-aware key at both initial callers and every recursive step — FULLY ADDRESSED. _helpers.ts:57-100,137-147 carries source into subPageParentKey; llm-ask.ts:359-375 uses it after authorization; subpage-context.ts:70-102 selects source and queues canonical key.

  • Null and historical standalone IDs without compatibility traversal alias — FULLY ADDRESSED. Real-PG standalone suite exercises both variants, historical wrong-key decoys, private descendants, Confluence keys and depth/page/character/cycle limits. Actual listening Ask regression checks grandchild marker reaches provider.

Findings

None.

Evidence inspected (reused, not executed here)

  • pr-1842-metadata.json: applicable exact-head CI SUCCESS; no review comments beyond prior COMMENT review
  • epic-1827-evidence/1791/loopback-smoke-env-i.log: one historical-grandchild HTTP integration test passed, six unselected
  • epic-1827-evidence/1791/published-source-attestation.json: final published-source manifest linkage

Evidence limits

No checks rerun. Retained initial root-test stale-mock assertion failure and nonexistent E2E-script invocation were reported corrected, not treated as passes. Shared parent global gates reused per PR provenance; no fresh provider execution claimed.

Suggested targeted verification

Parent only in disposable PostgreSQL/Redis: npx vitest run src/domains/confluence/services/subpage-context.standalone.test.ts src/domains/confluence/services/subpage-context.test.ts src/routes/llm/llm-ask-cache-authorization.integration.test.ts; include existing restriction suites.

Cross-PR interaction

1842 and 1843 both modify llm-ask.ts but different imports/sections: canonical parent key influences assembled messages; 1843 fingerprints those same final messages. Source composition has no observed semantic conflict; no merge executed.

No source edits, branch changes, services, GitHub reviews, pushes or merges performed.

Parent validation and publication

No additional actionable findings from parent triage.

Current final-head GitHub CI is passing. This is an independent source/evidence review, not a newly rerun full Docker/browser matrix or a GitHub approval. Source-only findings are identified as such. No code modifications or merges performed.

Reviewed by PR Reviewer Agent (eight independent review contexts; parent-validated publication).

@laboef1900 laboef1900 left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Automated PR Review

PR: #1842 -- Fix source-aware subpage context traversal
Branch: feature/1791-source-aware-subpages -> dev (reviewed at 7468ca41)
Diff size: 551 lines (+166/-116)
Reviewed at: 2026-11-01 10:45 UTC


Static Checks

Check Status Details
TypeScript PASS npm run typecheck
Lint PASS npm run lint
Backend Tests PASS* 6 failed
Frontend Tests PASS 5799 passed (5799)
Production Build PASS npm run build
npm audit FAIL pre-existing: 11 advisories (10 moderate, 1 high systeminformation) in the shared lockfile; package-lock.json not changed by this PR
Branch Target PASS feature/* → dev

Docker App Test

Built from source (docker-compose.yml + docker-compose.build.yml, native arch) on an isolated project pr-review-1842; API reached through the nginx proxy (the backend publishes no host port by design).

Check Status
Containers Healthy (postgres, redis, backend, frontend) PASS
Migrations (fresh DB → Migrations complete, through 129) PASS
Backend /api/health + /api/health/ready PASS
Fresh-instance setup (POST /api/setup/admin → 201) + Login PASS
API /api/pages, /api/search?q=…, /api/analytics/trending, /api/settings PASS (200)
API /api/admin/settings, /api/admin/llm-providers, /api/notifications/count PASS (200)
Frontend loads (GET / → 200) PASS
No error/fatal logs (level ≥ 50) PASS

E2E Browser Tests

CI recipe (E2E_CI=1 npx playwright test: built backend + vite preview, fresh kb_e2e database, provisioning via e2e/helpers/ci-setup.ts).

Project Status
e2e helper unit tests (vitest --config e2e/vitest.config.ts) PASS
chromium + collab PASS (68 passed (3.4m))

Code Review Findings

# Severity File:Line Finding
1 WARNING backend/src/routes/llm/llm-ask.test.ts:3005-3035 Two assertions in the includeSubPages RBAC gate suite were deleted rather than updated: the deny test lost expect(mockAssembleSubPageContext).not.toHaveBeenCalled() (the "RBAC gate short-circuits before any tree assembly" guarantee) and the allow test lost toHaveBeenCalledWith(user, parentKey, html, title). Neither was obsolete: with the seeded row now carrying source: 'confluence' (llm-ask.test.ts:2965), subPageParentKey returns 'page-abc', so both would pass unchanged. The unit suite no longer asserts the parent key the ask route hands to assembleSubPageContext (llm-ask.ts:371) — the exact value this PR changes — and no longer asserts that a denied caller triggers no tree fetch. Restore both (keep 'page-abc' as the expected key).
2 INFO backend/src/routes/llm/_helpers.ts:64-66 subPageParentKey is a third copy of the source-aware key rule already exported as parentKeyFor(source, id, confluenceId) in backend/src/domains/knowledge/services/page-relocate-service.ts:220-222, which CLAUDE.md:773-775 and docs/architecture/08-flow-sync.md:418 name as the single definition (the SQL restatement in core/services/page-subtree.ts:78-81 exists only because core may not import a domain). Routes already import from domains (llm-ask.ts imports domains/confluence/services/subpage-context.js), so _helpers.ts can import parentKeyFor directly; fetchSubPages (subpage-context.ts:97-100) inlines the same expression once more. Two more divergence points for one rule.
3 INFO CLAUDE.md:889-894 The new sub-page-context paragraph sits inside ## Standalone Trash Cascade (#1636) (heading at CLAUDE.md:757), wedged between the visiblePagesPredicate paragraph and the restore-409 paragraph. It is unrelated to trash/restore; place it next to the existing source-aware key rule (CLAUDE.md:773-777) or in the LLM/RAG section so future readers find it.
4 INFO backend/src/routes/llm/llm-ask-cache-authorization.integration.test.ts:270-273 The new test calls app.listen({ port: 0 }) on the shared app built in beforeAll and drives it with fetch, while every other test in the file uses the ask() helper (app.inject, line 118). The shared instance stays bound to a real socket for the rest of the file, and any later test that also calls listen fails with "Fastify instance is already listening". Deliberate per the PR body, but the ask() helper exercises the identical route code without the lifecycle coupling.

Acceptance Criteria (Issue #1791)

AC Criterion Status Evidence
AC-1 Initial LLM-context parent resolved by the source-aware key at every caller (confluence_id only for Confluence-origin pages, else id::text) PASS subPageParentKey (_helpers.ts:64-66) used at assembleContextIfNeeded (_helpers.ts:139, covers improve/analyze-quality/summarize) and /llm/ask (llm-ask.ts:371); lookupPageRef now selects source (_helpers.ts:79-99). Both files the issue cites are changed.
AC-2 Every recursive queue step uses the source-aware key PASS subpage-context.ts:97-100 queues row.source === 'confluence' && row.confluence_id ? row.confluence_id : String(row.id); query now selects source (subpage-context.ts:71-79). Existing visited cycle guard, MAX_DEPTH=5, MAX_SUB_PAGES=50 and visiblePagesPredicate untouched (subpage-context.ts:58-66).
AC-3 Null AND historical non-null standalone IDs covered by tests PASS subpage-context.standalone.test.ts:42-61 loops historical in [false, true] over root/child/grandchild on real PostgreSQL, asserting Child+Grandchild present via assembleContextIfNeeded and fetchSubPages(child) -> ['Grandchild']. llm-ask-cache-authorization.integration.test.ts:253-280 drives the real /api/llm/ask route with historical IDs and proves the grandchild body (SECRET) reaches the provider prompt.
AC-4 No compatibility alias: the historical Confluence ID is not traversed PASS Decoy pages Wrong alias (parent_id=historic-root) and Wrong recursion alias (parent_id=historic-child) asserted absent (standalone.test.ts:47-48, 54-55); Confluence tree asserts Wrong internal key (parent_id=String(root)) absent (standalone.test.ts:63-75). CLAUDE.md:892 and docs 09-flow-rag-chat.md:1377 record the no-alias rule.
AC-5 Existing visibility, depth, cycle, count and character limits preserved (PR scope statement) PASS standalone.test.ts:77-99: private descendant hidden, depth list [1..5] for a 6-deep chain, wasTruncated at maxChars=300, 2-node cycle terminates, 51 children capped at 50. /llm/ask cache key still hashes the full system+user prompt (llm-ask.ts:997-998), so the changed tree content is reflected in the key automatically; no cross-user replay surface added.

Story Validation (Issue #1791)

Issue intent: Sub-page LLM context keyed traversal on confluence_id ?? id and so, for a standalone page that retained a historical Confluence ID, looked up children under a key no writer stores — dropping children at the initial lookup and halting recursion below such a page.
Story completion: FULLY ADDRESSED
User journey: Before: asking/improving with includeSubPages on a relocated (historical-ID) standalone page produced a prompt with no children, or a tree that stopped at the first historical-ID descendant -> After: initial lookup and every BFS step key on source, so the whole visible standalone subtree reaches the prompt; Confluence trees, visibility, cycle/depth/count/char limits behave as before.
Gaps: None functional. Test-quality regression in llm-ask.test.ts (finding 1); rule duplicated instead of importing parentKeyFor (finding 2); CLAUDE.md paragraph misplaced (finding 3).
Evidence: backend/src/domains/confluence/services/subpage-context.ts:26-31,71-100; backend/src/routes/llm/_helpers.ts:57-99,137-141; backend/src/routes/llm/llm-ask.ts:48,369-374; backend/src/domains/confluence/services/subpage-context.standalone.test.ts:1-99; backend/src/routes/llm/llm-ask-cache-authorization.integration.test.ts:253-280; backend/src/routes/llm/improve-page-id.test.ts:174-275; CHANGELOG.md:10-13; CLAUDE.md:889-894; docs/architecture/09-flow-rag-chat.md:1374-1378.

Suggested Actions

  1. [SHOULD FIX] Restore the two deleted assertions in backend/src/routes/llm/llm-ask.test.ts:3005-3035 — expect(mockAssembleSubPageContext).not.toHaveBeenCalled() in the deny test and expect(mockAssembleSubPageContext).toHaveBeenCalledWith('test-user-123', 'page-abc', '<p>secret</p>', 'Doc') in the allow test; both pass with the seeded source: 'confluence' row.
  2. [CONSIDER] Replace subPageParentKey (backend/src/routes/llm/_helpers.ts:64-66) with an import of parentKeyFor from domains/knowledge/services/page-relocate-service.ts, and reference it from the comment at subpage-context.ts:97 so the rule has one owner as CLAUDE.md:773-775 states.
  3. [OPTIONAL] Move the new CLAUDE.md:889-894 paragraph out of ## Standalone Trash Cascade (#1636) to sit beside the source-aware key rule at CLAUDE.md:773-777.
  4. [OPTIONAL] Use the file's ask() helper instead of app.listen + fetch in backend/src/routes/llm/llm-ask-cache-authorization.integration.test.ts:270-273.

Verdict: COMMENT

Reason: Static checks, Docker smoke and the Playwright suite pass; issue #1791 is fully addressed (source-aware parent key at every initial lookup and every BFS step, with real-Postgres tests for null and historical standalone IDs and an end-to-end /api/llm/ask proof). One WARNING: llm-ask.test.ts lost the not.toHaveBeenCalled() RBAC-short-circuit assertion and the toHaveBeenCalledWith(..., 'page-abc', ...) parent-key assertion — the exact value this PR changes — although both still pass with the seeded source: 'confluence' row; restore them. INFO: subPageParentKey duplicates the documented single definition parentKeyFor.

Posted as a comment: GitHub rejects APPROVE/REQUEST_CHANGES from the PR author's own account, so the verdict above is advisory.


Reviewed by PR Reviewer Agent | 2026-11-01

…suite (#1791)

Restore the deny-path not.toHaveBeenCalled() and allow-path
toHaveBeenCalledWith(..., 'page-abc', ...) assertions the PR dropped, and
add a standalone-row case pinning that subPageParentKey hands the internal
id as a string even when a historical confluence_id is set.
@laboef1900

Copy link
Copy Markdown
Contributor Author

Addressed the review WARNING in 70b0b365: restored not.toHaveBeenCalled() on the deny path and toHaveBeenCalledWith('test-user-123', 'page-abc', '<p>secret</p>', 'Doc') on the allow path, and added a standalone-row case asserting the parent key is the internal id '1' despite a historical confluence_id. Full backend suite: 7171 passed (only the pre-existing local backup-stream mismatch), typecheck/lint/build clean.

laboef1900 added 3 commits November 1, 2026 12:46
CLAUDE.md was split into docs/claude/*.md on dev (#1877); this branch's
guidance paragraph moves to the matching split file.
@laboef1900
laboef1900 merged commit 8f95194 into dev Nov 5, 2026
8 checks passed
@laboef1900
laboef1900 deleted the feature/1791-source-aware-subpages branch November 5, 2026 18:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[P2] Use source-aware hierarchy keys when assembling historical standalone subpage context

1 participant