Skip to content

fix: align contiguous Chinese clipping with token timestamps - #219

Open
Minh3132 wants to merge 4 commits into
modelscope:mainfrom
Minh3132:fix/sensevoice-chinese-clipping-215
Open

Minh3132 wants to merge 4 commits into
modelscope:mainfrom
Minh3132:fix/sensevoice-chinese-clipping-215

Conversation

@Minh3132

@Minh3132 Minh3132 commented Sep 14, 2026

Copy link
Copy Markdown

Summary

Fix incorrect clipping behavior for contiguous and repeated Chinese text in VideoClipper.

Previously, matching contiguous Chinese text could produce overlapping duplicate matches. For repeated text such as 哈哈哈哈, this could cause the same middle audio segment to be appended more than once, producing an incorrectly extended output clip.

Changes

  • Fix matching of contiguous Chinese text against token timestamps.
  • Advance by the full matched span after a successful match to prevent overlapping duplicate matches.
  • Preserve support for spaced and contiguous Chinese text.
  • Prevent repeated text from appending the same audio region multiple times.
  • Add regression tests covering repeated and contiguous Chinese clipping behavior.

Validation

Regression coverage includes:

  • contiguous Chinese text
  • spaced Chinese text
  • repeated Chinese characters
  • non-overlapping match behavior
  • VideoClipper.clip() output waveform length

Reviewer validation on the current head reported:

  • 4 new regression tests passed
  • 7 existing duplicate-text tests passed
  • 11/11 relevant tests passed

The bounded recheck found no additional functional blocker in the affected clipping behavior.

This validation is scoped to the duplicate/contiguous-text clipping regression and does not claim full end-to-end ASR, browser, UI, or video-export validation.

Fixes #215

@LauraGPT LauraGPT left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for addressing #215. The three submitted matcher tests plus seven existing duplicate-matching tests pass on exact head 4715d53. However, an independent comparison using already-working token-spaced Chinese found a user-visible regression: overlapping matches repeat the same source audio. Details and exact sample counts are inline.

Please preserve non-overlapping repeated-match behavior for both contiguous and token-spaced raw text, and add the actual clip waveform/sample-count regression rather than only asserting timestamp lists. The new repeated-match test currently codifies the changed overlapping behavior and should be revised.

Please also replace the untouched TODO/template PR body with the mapping approach, scope, validation commands and relation to #215. Another contributor previously volunteered on that issue; this review does not assign ownership or approve a broader matching-policy change.

Validation used the exact candidate helper, hash-verified base VideoClipper/support files, real clip execution and synthetic saved recognition state. No model inference/download, browser/video export, acoustic accuracy or full-suite validation is claimed.

Comment thread funclip/utils/trans_utils.py Outdated

ts = []
match_len = len(dest_tokens)
for start in range(0, len(raw_tokens) - match_len + 1):

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[P2] Keep repeated matches non-overlapping. Advancing start by one token after a match changes an already-working input: raw_text="哈 哈 哈 哈", timestamps=[[0,100],[100,200],[200,300],[300,400]], and pre_proc("哈哈"). Base main returns [[0,3200],[3200,6400]], while this head returns [[0,3200],[1600,4800],[3200,6400]]. Running actual VideoClipper.clip with a 6,400-sample synthetic input consequently returns 9,600 samples instead of 6,400, duplicating the middle source audio. This is independent of the contiguous-Chinese bug and is not merely a different textual search result. Preserve the previous advance-by-match-length behavior after a successful match (or obtain an explicit separate compatibility decision for overlapping output), and cover both spaced/contiguous transcripts plus emitted waveform length.

@LauraGPT LauraGPT left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Rechecked dbc949e: the overlapping-match/audio-duplication regression from my earlier review is fixed in this revision. The four submitted tests plus seven existing duplicate-text tests pass (11 total). Running those same four submitted tests against previous 4715d53 gives two assertion failures and two passes: the repeated-match case still adds the overlapping middle span there, and the waveform test produces 9600 instead of 6400 samples. The new revision returns the expected non-overlapping spans and exact 6400-sample waveform.

Validation used the exact candidate matching module, hash-verified base VideoClipper/helpers and real CPU numerical dependencies, with synthetic state and no ASR/model inference. This confirms the specific regression correction, not full-suite, browser/video, model accuracy or performance acceptance. No additional functional blocker was found in this bounded recheck.

@Minh3132 Minh3132 changed the title Fix/sensevoice chinese clipping 215 fix: align contiguous Chinese clipping with token timestamps Sep 16, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug] SenseVoice Chinese text clipping misses multi-character queries and uses incorrect timestamp indices

2 participants