Skip to content

Preserve prompt padding masks in cross-attention - #3

Open
zjn20030811 wants to merge 1 commit into
OpenMOSS:mainfrom
zjn20030811:fix-prompt-padding-mask
Open

Preserve prompt padding masks in cross-attention#3
zjn20030811 wants to merge 1 commit into
OpenMOSS:mainfrom
zjn20030811:fix-prompt-padding-mask

Conversation

@zjn20030811

Copy link
Copy Markdown

Summary

  • retain the tokenizer padding mask in the MoT, Hidden, and Unified prompt encoders
  • centralize padding cleanup in a vectorized helper that zeroes padded embeddings without mutating encoder output
  • add CPU regressions for padded-token masking, non-contiguous masks, input validation, and non-mutating behavior

Why

All three prompt paths zeroed padded embeddings and then replaced the tokenizer mask with an all-valid mask. Zero vectors still participate in attention normalization, so the amount of padding could change text conditioning. Retaining the original mask excludes padded keys from cross-attention.

Validation

  • 4 CPU regression cases passed
  • syntax checks passed for all changed model modules and the regression file
  • whitespace validation passed

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant