Skip to content

fix(mem_os): split references on mixed comma styles - #2418

Open
simpleqt wants to merge 2 commits into
MemTensor:mainfrom
simpleqt:sq/ref-split-mixed-sep
Open

simpleqt wants to merge 2 commits into
MemTensor:mainfrom
simpleqt:sq/ref-split-mixed-sep

Conversation

@simpleqt

Copy link
Copy Markdown

Description

Fixes #2417

split_continuous_references turned [1:aaa, 4:bbb] into [1:aaa][4:bbb] via two sequential str.replace passes — one for ", ", one for ",". The first matching pass rewrote the content substring, so the second pass could no longer find the original: a tag mixing both styles kept its earlier references merged, and the streaming chat path (api/handlers/chat_handler.py → process_streaming_references_complete) then rendered them as literal text instead of reference tags.

split_continuous_references("[1:92ff35fb,4:bfe6f044, 7:abcd1234]")
# before: '[1:92ff35fb,4:bfe6f044][7:abcd1234]'
# after:  '[1:92ff35fb][4:bfe6f044][7:abcd1234]'

Implementation: split the bracket content with re.sub(r",\\s*", "][", ...) in a single pass; the redundant inner import re in process_streaming_references_complete moved to module level. No dependency changes.

Type of change

  • Bug fix (non-breaking change which fixes an issue)

How Has This Been Tested?

  • Unit Test

New tests/mem_os/test_reference_utils.py covers spaced, tight, and mixed separators, surrounding-text preservation, and the non-reference early returns. Red/green verified: test_mixed_separators fails against unpatched main and passes with the fix; the other four pin unchanged behavior.

  • Test Script Or Test Steps
poetry run pytest tests/mem_os/test_reference_utils.py -v

ruff check and ruff format --check pass on both touched files.

Checklist

  • I have performed a self-review of my own code | 我已自行检查了自己的代码
  • I have commented my code in hard-to-understand areas | 我已在难以理解的地方对代码进行了注释
  • I have added tests that prove my fix is effective or that my feature works | 我已添加测试以证明我的修复有效或功能正常
  • I have linked the issue to this PR (if applicable) | 我已将 issue 链接到此 PR(如果适用)

split_continuous_references applied two sequential str.replace passes,
one for ', ' and one for ','. The first matching pass rewrote the
content substring, so the second pass found nothing: a tag mixing both
styles ('[a,b, c]') kept its earlier references merged on the streaming
chat path. Split on every comma with re.sub(r',\\s*', ...)' instead.

Red/green verified: the mixed-separator test fails against the old
implementation and passes with the fix.
Copilot AI lite review requested due to automatic review settings September 28, 2026 03:20
@Memtensor-AI Memtensor-AI added area:core MOS 编排层 / 框架底座 / 跨模块问题 status:in-progress Someone or AI is working on it | 人工或 AI 正在处理 labels Sep 28, 2026

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

@Memtensor-AI

Memtensor-AI commented Sep 28, 2026 •

Copy link
Copy Markdown
Collaborator

🤖 Open Code Review

Target: PR #2418
Task: 5d9195458007c54b
Base: main
Head: sq/ref-split-mixed-sep

✅ OpenCodeReview: Review complete: 0 finding(s) across 2 selected item(s).

Generated by cloud-assistant via Open Code Review.

@Memtensor-AI

Copy link
Copy Markdown
Collaborator

✅ Automated Test Results: PASSED

All tests passed (5/5 executed). memos_python_core/changed-repo-python: 5/5. Duration: 9s [advisory, non-gating] AI-generated tests on branch test/auto-gen-ff4aba06e30b05df-20260928112516: 96/96 passed — these do NOT affect the PR verdict; review the branch manually.

Branch: sq/ref-split-mixed-sep

@Memtensor-AI Memtensor-AI added status:ready Ready for implementation; waiting for assignee or AI dispatch | 可进入实现,等待认领或派发 and removed status:in-progress Someone or AI is working on it | 人工或 AI 正在处理 labels Sep 28, 2026
split_continuous_references rewrote ANY single bracketed comma list on
the streaming chat path: mathematical or plain text like 'interval
[x, y]' came out as 'interval [x][y]'. Restrict the split to content
with the reference-tag shape (a numeric id before a colon in every
element), which is the documented input format.

Red/green verified: the new test fails against the old implementation
([x, y] rewritten to [x][y]) and passes with the fix.

Signed-off-by: simpleqt <89645338+simpleqt@users.noreply.github.com>
@simpleqt

Copy link
Copy Markdown
Author

Added a second commit (same theme): the split previously rewrote any bracketed comma list, so ordinary text like interval [x, y] was corrupted into interval [x][y] on the streaming chat path. The split now requires the reference-tag shape (a numeric id before a colon in every element), which is the documented input format. New test test_ordinary_bracketed_text_untouched is red/green verified.

@Memtensor-AI Memtensor-AI added status:in-progress Someone or AI is working on it | 人工或 AI 正在处理 and removed status:ready Ready for implementation; waiting for assignee or AI dispatch | 可进入实现,等待认领或派发 labels Sep 28, 2026
@Memtensor-AI

Copy link
Copy Markdown
Collaborator

✅ Automated Test Results: PASSED

All tests passed (6/6 executed). memos_python_core/changed-repo-python: 6/6. Duration: 8s [advisory, non-gating] AI-generated tests on branch test/auto-gen-5d9195458007c54b-20260928145304: 84/84 passed — these do NOT affect the PR verdict; review the branch manually.

Branch: sq/ref-split-mixed-sep

@Memtensor-AI Memtensor-AI added status:ready Ready for implementation; waiting for assignee or AI dispatch | 可进入实现,等待认领或派发 and removed status:in-progress Someone or AI is working on it | 人工或 AI 正在处理 labels Sep 28, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:core MOS 编排层 / 框架底座 / 跨模块问题 status:ready Ready for implementation; waiting for assignee or AI dispatch | 可进入实现,等待认领或派发

Projects

None yet

Development

Successfully merging this pull request may close these issues.

split_continuous_references leaves references merged when comma styles are mixed

4 participants