docs(self-hosting): document pure-CJK and non-ASCII retrieval bug and workarounds (#1740)

[Adityakk9031]

Resolves #1740.

Description

Documents the root cause, behavior, and validated workarounds for the pure-CJK and non-ASCII memory search retrieval issue in self-hosted v0.0.7 and v0.0.8. Adds unit test coverage asserting that non-ASCII and CJK memories are preserved during deduplication.

Root Cause

In self-hosted releases v0.0.7 and v0.0.8, the internal deduplication helper used by search and profile generation normalizes candidate memory strings using an ASCII-only regex:
text.toLowerCase().replace(/[^\w\s]/g, "").replace(/\s+/g, " ").trim()

• In JavaScript RegExp, \w without Unicode property escapes only matches ASCII word characters ([A-Za-z0-9_]).
• For pure-CJK, Cyrillic, Arabic, or other non-ASCII content without ASCII alphanumeric characters or underscores, the normalized key is stripped down to an empty string ("").
• The deduplication loop checks if (!key) continue;, which evaluates to true and silently drops the candidate item before results are returned.
• The row and embedding vector are successfully written to the database; the drop occurs solely during post-search result deduplication.

Workaround & Notes

• Appending an ASCII anchor (such as a trailing underscore _ or alphanumeric ID) prevents the key from collapsing to an empty string, allowing the item to pass deduplication.
• Clarifies that POST /v4/search enforces 1 <= limit <= 100 via schema validation; values above 100 fail validation rather than clamping.