[anhnh2002]

Large repositories could skip clustering because the split decision counted only selected leaf snippets. For goja, 1,632 leaf tokens represented 392,218 tokens of indexed implementation, and initial clustering returned an empty tree.

The split decision now counts every component in the current file scope, including methods omitted by leaf selection. Source files without indexed symbols are retained with full-source file nodes. Each file has one leaf-module owner; parent modules may retain aggregate component lists.

Changes

• Use the same component-source size metric at repository, recursive-module, delegation, and whole-repository updater checks. Log code tokens, leaf tokens, threshold, and original/expanded file coverage.
• Keep existing leaves and add file representatives instead of sending every component to clustering. Goja uses 187 candidate IDs rather than all 4,526 indexed nodes. Existing node-count and output-token batching limits remain in force.
• Reconcile missing, duplicated, invalid, and out-of-scope LLM assignments by whole file. Empty, unusable, or single-group responses fall back to a deterministic partition by path and code size. Stop recursion for an indivisible oversized file.
• Preserve complete file ownership through both documentation backends and their delegation tools. Small multi-file repositories keep one-page mode and can optionally delegate, subject to the depth limit.
• Preserve existing file exclusions. A capped artifact representation of a supported source file does not replace its full-source file representative.

Measurements without LLM calls

Both clustering implementations were exercised with the same deterministic completer. Before uses base commit 364b67a. These are initial clustering trees, before documentation agents optionally add modules. Artifacts are disabled and the threshold is unchanged at 36,369 tokens.

Repository	Component code tokens before → after	Leaf tokens	Splits before → after	Files without a leaf owner before → after
goja	392,218 → 416,319	1,632	no → yes	78 → 0
goldmark	100,041 → 164,084	11,864	no → yes	61 → 0
lua	180,762 → 209,765	47,904	yes → yes	26 → 0
lz4	120,602 → 123,155	31,338	no → yes	45 → 0
cJSON	24,603 → 25,532	26,041	no → no	8 → 8*
inih	4,632 → 6,365	3,145	no → no	5 → 5*

The larger post-change totals include previously absent symbol-free source files. Goja's selected inventory includes 77 non-test Go files and one JavaScript file. Every non-test Go file has exactly one module owner after clustering. All split repositories have zero duplicate file owners.

* Small repositories intentionally retain an empty module tree for one-page documentation. Their documentation agent receives all 8/8 and 5/5 selected files; neither invokes the clustering completer. With default artifact analysis enabled, cJSON and inih also remain below threshold at 30,928 and 6,894 tokens.

Validation

• 527 Python tests passed, also with tree_sitter_php unavailable to reproduce the CI dependency set. Source-file exclusions are shared with the analyzers and do not import parser grammars.
• Full suite command: pytest -p no:cacheprovider -o addopts="" tests/ -q.
• uvx ruff@latest check and uvx ruff@latest format --check passed on all 22 changed Python files; git diff --check passed.
• Added a real parsed Go fixture with 406 structs, 810 methods, sparse leaves, methods in separate files, and a constants-only file. It crosses the true-code threshold and produces complete, disjoint module ownership. A small fixture stays in one-page mode without calling the clustering completer.
• Added parser coverage for all 16 supported languages, recursive-scope isolation, batching, malformed/partial LLM responses, oversized single files, both delegation backends, capped source artifacts, and updater growth hidden outside leaf entry points.
• Ran real codewiki generate --no-cache --no-artifacts --max-depth 2 --verbose on a scratch copy of goja using the configured Codex backend. The initial saved tree has 7 top-level groups and 27 leaf modules; after documentation-agent delegation it has 30 leaves. Both saved trees cover all 78 selected files, including all 77 non-test Go files, exactly once. The smoke run was intentionally interrupted after validating clustering and delegation; full wiki generation was not completed. Scratch copies and measurement outputs were removed.

Limits

The chosen metric deliberately sums component source, so class/method source may overlap. Source outside indexed components is counted in full only when the file has no code symbols. Ignored, unsupported, and unreadable files remain outside coverage, and dependency resolution itself is unchanged.

Files remain indivisible: goja's vm.go is a 37,272-token single-file leaf and is logged as oversized. Existing cached partial trees are not automatically rewritten; regenerate them with a fresh output or --no-cache. Small one-page repositories may still have an empty tree if their agent chooses not to delegate.