Skip to content

Enforce minimum module sizes and reject non-splits - #152

Merged
anhnh2002 merged 1 commit into
mainfrom
fix/minimum-module-size
Nov 1, 2026
Merged

anhnh2002 merged 1 commit into
mainfrom
fix/minimum-module-size

Conversation

@anhnh2002

Copy link
Copy Markdown
Collaborator

Small modules and whole-parent wrappers create unnecessary documentation pages and downstream agent tasks. Add a configurable 1,000-code-token minimum and reject partitions that leave fewer than two viable children.

Changes

  • Share delegation eligibility and size checks across Pydantic AI and caw. Count all indexed components in owned files, including methods omitted from entry-point lists.
  • Coalesce undersized groups by directory proximity before creating pages or running child agents. Apply the same rule to clustering, structural fallback, and artifact insertion while preserving complete, disjoint file ownership.
  • Report merged and rejected requests to the parent agent. A partition that collapses to one child stays in its parent; indivisible oversized files remain supported.
  • Raise the default delegation base to 8,000 tokens across backend and CLI configuration. Subsequent delegation levels require 16,000, 24,000, etc. Keep the clustering threshold at 36,369 tokens; retain explicit overrides and existing cached trees.
  • Expose --min-token-per-module through configuration, generation, and CLI help, with regression coverage for both backends and language-independent sizing.

Validation

  • Full Python suite: 654 passed (.venv/bin/python -m pytest -o addopts= -q).
  • Ruff check and format check passed for changed Python files; git diff --check passed.
  • Fresh tinydb generation completed with 18/18 files covered and no undersized modules or single-child wrappers.
  • Fresh cmark generation using DeepSeek V4 Flash, excluding build/: identical 152,232 code tokens, 403 components, and 785 dependency edges to the saved baseline. Baseline delegation base was 4,000; the new run used 8,000.
cmark metric Baseline New
Modules 60 27
Agent-created modules 36 8
Depth 5 4
Leaves below 1,000 tokens, counting complete file scopes 21 0
Single-child nodes 2 0
API requests 673 353
Recorded cost, USD 0.699 0.394

The new cmark run took 59m49s, covered all 51 files, and produced all 28 expected Markdown pages with no missing local Markdown links. The 12,992-token inline module and the 71,524-token scanner module remain leaves. One leaf lists 960 representative-component tokens but owns 1,061 tokens across its complete files, satisfying the minimum under the existing size calculation.

Remote Mermaid validation timed out during generation, so diagrams were not fully verified remotely. Source repositories and saved baseline trees were not modified; generated scratch outputs were removed.

@anhnh2002
anhnh2002 merged commit 80f6cc3 into main Nov 1, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant