Repository navigation
Support independently versioned full Japanese pronunciation resources - #250
Merged
Merged
Conversation
Lucas1479
changed the base branch from
main
to
codex/headless-inference-security
October 21, 2026 12:21
Lucas1479
marked this pull request as ready for review
October 21, 2026 16:39
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Result and acceptance boundary
Keep the complete 134,201-row Japanese pronunciation dictionary as an independently versioned optional resource, with reviewed corrections and explicit runtime deferrals. This PR fixes the personal-dictionary, casing and numeric-hyphen regressions from independent review. The maintainer authorized merging the application code after the final head passed CI. Resource publication, remote data-repository creation and live installation remain outside this authorization.
Base: main
879a8fbd, containing #252, #249 and #253. Integration head3e06932aincludes #253 without conflicts; Japanese implementation hashes remain identical to qualified518475d2. Inherited Qwen/provenance conflicts use exact main versions; no Qwen, AEC, root dependency or lockfile differences remain.Follow-up to independent review
Independent review confirmed the previous four findings on
916fa472and the scaling/width fixes on69368c19. The final follow-up addresses the remaining title-case personal-entry mismatch:Collects personal keys and subtracts them once from the official vocabulary, avoiding one full-set copy per personal row.
Normalizes personal surfaces with the same case rule as text input, then native width normalization, so
Menu/Existing/Okabeentries match lower/title/uppercase inputs under official aliases and formats 1/2 native rows. Case-ownedDisneyremains uppercase. Source CSV bytes remain intact; native/audio identity is revised to v11. Previously ineffective personal entries can start taking effect after upgrade.Removes dead numeric-prefix handling without changing the reviewed casing policy.
Windows fresh-process configuration measured for the prior scaling fix (single measurements, not end-to-end TTS):
| Personal rows | Before | After |
|---|---:|---:|
| 200 | 2.92 s | 2.21 s |
| 1,000 | 5.91 s | 2.22 s |
| 3,000 | 14.93 s | 2.20 s |
The regression suite uses a deterministic vocabulary-copy work bound instead of a machine-speed timeout. The new tests reproduce three failures on the unmodified old implementation (halfwidth overrides in both formats and repeated copies), and pass after the fix.
The final case-alignment patch passes six additional native cases that all fail on
69368c19, covering half/fullwidth entries, alias/native partitions, exact base names and cached reloads. With the final code, rc2 plus 10,000 synthetic personal rows configured in 2.25 seconds in a fresh Windows process with precompiled dictionaries (one sample).Handremains a low-priority base-segmentation exception. General lowercasing of equal-reading base spellings was tested and not included: base equivalence alone does not preserve selection with the legacy CSV loaded. There is no new word-specific exception or unverified resource mapping.Resource accounting and selection
Unchanged candidate: 2026.10.22.1-rc2, 37,386,096 bytes.
ZIP SHA-256:
a935b3adfded9022afe3b2384cebc2ef1e4023f54f48df1f9ded9ff45b896989.134,201 original/master rows retained: 462 reviewed corrections, 133,739 original-unverified, zero source deletion.
Supplement: 4,539 rows / 4,537 words. Runtime: 121,912 native rows and 4,411 aliases, not unique newly corrected-word counts.
7,989 source exclusions: 4,920 unresolved alternatives, 1,300 CMU flags, 1,038 base-owned rows, 399 pending pronunciation normalization, 236 short tokens, 77 contextual rows and 19 known rejected rows. Overlay replacement is separate.
Reviewed defaults supersede unverified alternatives. Existing CMU final-phone screening is reused. Personal keys take precedence over official aliases and native-word boundary insertion.
Exact base spelling ownership applies with no pack, format 1, legacy CSV and format 2; unknown spellings use the established lowercase path. Numeric-adjacent hyphens remain boundaries; alphabetic compounds remain whole.
JMdict attribution is included; Alkana comparison readings stay in research data outside runtime metadata. The unused format-1 alias prototype is removed.
Unchanged, single-reading, unflagged original rows remain active and unverified. This policy (#244) still requires explicit maintainer acceptance. Independent review reported remaining errors and deferred common words; the follow-up receipt records a frequency-priority review backlog without approving speculative readings.
Build inputs remain data commit
4f490be3f96aabde99885e3eea0e17d7e0a39d04. Full source ZIP and history bundle remain at data snapshot6afbd8a8583046b3150cc578aec104a753452ddf. All resources, source archives, corpus and recordings remain local and outside the application diff.Validation
After integrating fix(vn): qualify macOS Tk/ImageTk and document interpreter migration #253, 154 related local tests passed, including its six VN environment contracts.
148 related tests passed (145 Japanese/resource tests and 3 main Qwen contracts), with two existing deprecation warnings; Ruff and diff checks passed.
Four fresh production profiles, 26,552 records each: no pack, format 1, legacy CSV and full candidate. Every phone/target record equals reviewed
69368c19(and unchanged916fa472); the prior seven-profile comparison against main8de4c958remains reproducible. This is regression evidence, not daily-error-rate measurement.All 127 published readings retain their three original contexts (381/381). The previously disclosed Main reading difference and unscored accompaniments spans are unchanged.
The prior actual format-1 installation, full-candidate update, rollback and restoration remain recorded. The final related suite again passes lifecycle, protected personal/legacy files, identity restoration, idempotence and failed-update recovery contracts. Builder/installer code and the reviewed resource ZIP remain unchanged.
Existing four GSV A/B pairs remain historical listening evidence, not whole-vocabulary certification. No new audio or ASR-count claims. The current changes do not alter the reviewed no-personal corpus sequences.
All 17 checks passed on personal-case head
518475d2155607f07632c7d835eeade966ce1788. All 17 fresh CI checks passed on integration head3e06932a3ed38fe127be6b4d5d66e0dd356b01afagainst main879a8fbd. The maintainer authorized code merge on this qualified head; the resource candidate remains unpublished. Full resource/corpus and performance validation is Windows-local; native model CI verifies dictionary hashes and regression contracts on Windows/Linux/macOS, not full acoustic validation on every platform.References: resource contract, current case-alignment receipt, prior scaling receipt, prior integration receipt.