Skip to content

Land Phase 7.3b WP3: Qwen3-VL prompt expander and three-axis planner - #53

Merged
perNyfelt merged 1 commit into
mainfrom
phase7-3b-wp3
Oct 19, 2026
Merged

perNyfelt merged 1 commit into
mainfrom
phase7-3b-wp3

Conversation

@perNyfelt

Copy link
Copy Markdown
Member
  • Package-private Qwen3VlPromptExpander plus position planner family
    (Qwen3VlPromptTokens/ImagePlacement/RopePlan/PositionPlanner/PromptPlan),
    a faithful port of the pinned get_rope_index image path (transformers
    4.57.6 modular_qwen3_vl.py:858-955) with checked arithmetic that
    rejects overflow and past-the-prompt grids before any array is written.
  • Token contract: the four real Qwen sentinels (hex-verified against the
    pinned 2B artifact); Qwen3VlPromptTokens.resolve takes each sentinel
    name to its ID through the tokenizer and cross-checks the outer
    config.json ID. No hard-coded numeric IDs in production code.
  • Golden tools/hf-reference/goldens/qwen3-vl-wp3-prompt.json (8 cases x 2
    add_generation_prompt variants, generated by qwen3vl_wp3_prompt.py),
    digest-pinned in tools/hf-reference/provenance.json and enforced by
    verifyHfReferenceGoldens; WP1-consistent expanded lengths 82/-56 and
    90/-60.
  • 33 pure-Java tests (Qwen3VlPromptExpanderTest) over the golden, the
    resolve contract, the hand-computed position cases and the exact
    grammar-rejection messages, plus the qwen3vl-tokens fixture.
  • Findings gain section 6.8 (grammar, rejection table, jmlx-tokenizer
    three-directions finding, planner port, evidence) and correct the
    earlier template-description line; plan section 5 records WP3
    acceptance. Corrects the earlier sentinel assumption, which was an
    artifact of terminal display stripping the 0x7c byte.

- Package-private Qwen3VlPromptExpander plus position planner family
  (Qwen3VlPromptTokens/ImagePlacement/RopePlan/PositionPlanner/PromptPlan),
  a faithful port of the pinned get_rope_index image path (transformers
  4.57.6 modular_qwen3_vl.py:858-955) with checked arithmetic that
  rejects overflow and past-the-prompt grids before any array is written.
- Token contract: the four real Qwen sentinels (hex-verified against the
  pinned 2B artifact); Qwen3VlPromptTokens.resolve takes each sentinel
  name to its ID through the tokenizer and cross-checks the outer
  config.json ID. No hard-coded numeric IDs in production code.
- Golden tools/hf-reference/goldens/qwen3-vl-wp3-prompt.json (8 cases x 2
  add_generation_prompt variants, generated by qwen3vl_wp3_prompt.py),
  digest-pinned in tools/hf-reference/provenance.json and enforced by
  verifyHfReferenceGoldens; WP1-consistent expanded lengths 82/-56 and
  90/-60.
- 33 pure-Java tests (Qwen3VlPromptExpanderTest) over the golden, the
  resolve contract, the hand-computed position cases and the exact
  grammar-rejection messages, plus the qwen3vl-tokens fixture.
- Findings gain section 6.8 (grammar, rejection table, jmlx-tokenizer
  three-directions finding, planner port, evidence) and correct the
  earlier template-description line; plan section 5 records WP3
  acceptance. Corrects the earlier sentinel assumption, which was an
  artifact of terminal display stripping the 0x7c byte.
@perNyfelt
perNyfelt merged commit 3cfb6ef into main Oct 19, 2026
3 checks passed
@perNyfelt
perNyfelt deleted the phase7-3b-wp3 branch October 19, 2026 11:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant