Skip to content

fix(i18n): match browser locale by language subtag instead of exact code - #174

Merged
iuyo5678 merged 2 commits into
Tencent:mainfrom
iAstro:fix/extension-locale-normalization
Sep 4, 2026
Merged

fix(i18n): match browser locale by language subtag instead of exact code#174
iuyo5678 merged 2 commits into
Tencent:mainfrom
iAstro:fix/extension-locale-normalization

Conversation

@iAstro

@iAstro iAstro commented Sep 2, 2026

Copy link
Copy Markdown
Contributor

Problem

The extension ships en-US and zh-CN, but only en-US was ever selected - every other browser language fell through to fallbackLng: "zh-CN", so users on en, en-GB, en-CN, en-AU, ja-JP, fr-FR saw a Chinese UI.

Fix

Match on the BCP 47 language subtag instead of the full locale code:

  • en* -> en-US (every English variant)
  • zh-Hans* / zh-CN / zh-SG / zh -> zh-CN
  • zh-Hant* / zh-TW / zh-HK / zh-MO -> zh-TW
  • anything else -> unchanged, resolved by fallbackLng

Three layers, all in packages/i18n:

  1. Custom detector preferring chrome.i18n.getUILanguage() over navigator.language, which is unreliable on the chrome-extension:// origin the popup runs on.
  2. convertDetectedLanguage applying the table above.
  3. fallbackLng is now an object: { en: ["en-US"], zh: ["zh-CN"], default: ["en-US"] } - unmatched languages land on English, not Chinese.

Simplified vs Traditional

Decided by the script subtag (Hans/Hant) when present, then by the region subtag, since Chrome reports zh-CN/zh-TW rather than script subtags.

The zh-TW branch is written now, so adding a Traditional translation later needs no code change - only a new zh-TW resource. Until then i18next falls back to zh-CN (graceful degradation, not a misroute).

Tests

Adds packages/i18n/tests/locale-resolution.test.ts (28 cases) locking the table in as regression tests; the package previously had no tests.

28/28 tests pass; tsc --noEmit, biome check, and pnpm ext:build are all clean.

Closes #168

@iuyo5678

iuyo5678 commented Sep 3, 2026

Copy link
Copy Markdown
Collaborator

简单说:需求对,但是实现方式有问题,如下:

问题

当前 normalizeLanguageCode 采用硬编码的 if/else 逐语言处理:

if (primary === "en") return "en-US";
if (primary === "zh") { /* ... */ }
return code;

这等于后面每新增一种语言,都必须回来修改这个函数#161 正在添加韩语 (ko-KR),合并后如果浏览器语言是 bare konormalizeLanguageCode 会原样返回 ko,匹配不到 ko-KR resource,韩语用户会看到英文而非韩语——归一化白做了。后续再加日语、法语等也都有同样的问题。

建议

normalizeLanguageCode 改为由 resources 的 key 自动驱动的工厂函数,不用手动维护映射表:

export function createLanguageNormalizer(
  resourceKeys: string[],
): (code: string) => string {
  const keySet = new Set(resourceKeys);
  const index = buildPrimarySubtagIndex(resourceKeys);
  // index: Map { "en" → ["en-US"], "zh" → ["zh-CN"], "ko" → ["ko-KR"], ... }
  return (code: string): string => {
    // 1. 精确匹配 → 直接返回
    if (keySet.has(code)) return code;
    const parts = code.split(/[-_]/).map((p) => p.toLowerCase());
    const primary = parts[0];
    const candidates = index.get(primary);
    if (!candidates || candidates.length === 0) return code;
    // 2. 一个 primary subtag 只对应一个 resource → 自动映射
    //    ko → ko-KR, en → en-US, ja → ja-JP ...
    if (candidates.length === 1) return candidates[0];
    // 3. 一个 primary subtag 对应多个 resource → 消歧(目前只有 zh)
    if (primary === "zh") return resolveChineseVariant(parts, candidates);
    return candidates[0];
  };
}

调用侧传入 resource keys:

// i18n.ts
const resourceKeys = Object.keys(resources);
// ...
detection: getLanguageDetectionOptions(resourceKeys),

getLanguageDetectionOptions 签名相应调整为接收 resourceKeys 参数:

export function getLanguageDetectionOptions(resourceKeys: string[]) {
  const normalize = createLanguageNormalizer(resourceKeys);
  return {
    order: ["chromeUILanguage", "navigator"],
    caches: [],
    convertDetectedLanguage: normalize,
  };
}

中文简繁消歧(resolveChineseVariant)保留为唯一的特殊逻辑——这确实是特殊情况(一个语言两种书写系统),其他语言的映射完全自动推导。

收益

场景 当前方案 改进后
新增韩语 ko-KR 需改 normalizeLanguageCodeif 分支 只需注册到 resources零改动
新增日语 ja-JP 同上 同上
新增繁体 zh-TW 不需要改(已预写) 也不需要改(candidate 数 > 1 时自动触发消歧)
映射表与 resources 不一致的风险 存在(两处手动维护) 不存在(单一事实来源)

这样也能避免和 #161 之间的合并顺序依赖——无论谁先合,后合的只需要在 resources 里注册翻译文件就行。

测试方面,现有 28 个 case 全部兼容,建议额外补两组验证扩展性的 case:

it("auto-resolves new languages from resource keys", () => {
  const normalize = createLanguageNormalizer(["en-US", "zh-CN", "ko-KR"]);
  expect(normalize("ko")).toBe("ko-KR");
  expect(normalize("ko-KR")).toBe("ko-KR");
});
it("disambiguates zh when multiple zh resources exist", () => {
  const normalize = createLanguageNormalizer(["en-US", "zh-CN", "zh-TW"]);
  expect(normalize("zh-Hant")).toBe("zh-TW");
  expect(normalize("zh-HK")).toBe("zh-TW");
  expect(normalize("zh")).toBe("zh-CN");
});

其余(chromeUiLanguageDetectorfallbackLng 对象化、vitest 配置)暂时没有发现问题。

The extension ships en-US and zh-CN resources, but only en-US was ever
selected: every other browser language fell through to fallbackLng
("zh-CN"), so users on en, en-GB, en-CN, en-AU, ja-JP, fr-FR saw Chinese.

Match on the BCP 47 language subtag and normalise it to a resource key:

  en*                                -> en-US
  zh-Hans* / zh-CN / zh-SG / zh      -> zh-CN
  zh-Hant* / zh-TW / zh-HK / zh-MO   -> zh-TW
  anything else                      -> unchanged (fallbackLng default: en-US)

Simplified vs Traditional is decided by the script subtag (Hans/Hant) when
present, then by the region subtag, since Chrome reports zh-CN / zh-TW
rather than script subtags. The zh-TW branch is written now so adding a
Traditional translation later needs no code change.

chrome.i18n.getUILanguage() is now preferred over navigator.language, which
is unreliable on the chrome-extension:// origin the popup runs on.

Adds a locale-resolution regression matrix (28 cases).

Closes Tencent#168
Per review feedback on Tencent#174: the hardcoded if/else in
normalizeLanguageCode required touching the function for every new
language, so a bare `ko` would miss the `ko-KR` resource Tencent#161 is about
to add and Korean users would land on English despite the fix.

Replace it with createLanguageNormalizer(resourceKeys), a factory that
derives its mapping from the resource keys i18next is configured with:

  1. exact resource key              -> unchanged
  2. one resource for the language   -> that key (ko -> ko-KR)
  3. several (zh-CN / zh-TW)         -> script subtag, then region
  4. nothing shipped                 -> unchanged, fallbackLng applies

getLanguageDetectionOptions now takes the resource keys and i18n.ts
passes Object.keys(resources). Chinese disambiguation stays as the only
special case - one language, two scripts; every other mapping derives
from the index automatically.

Test expectations for the zh-TW family change from "zh-TW" to "zh-CN":
only zh-CN ships today and the factory maps to shipped keys only, so
Traditional resolves directly to Simplified instead of leaking into the
fallback chain. The rendered UI is identical either way, and the new
extensibility cases lock in the automatic disambiguation once a zh-TW
bundle registers - plus the auto-resolution of a newly registered
language like ko-KR.

28 -> 31 tests; extension suite 832/832; tsc, biome and ext:build clean.
@iAstro
iAstro force-pushed the fix/extension-locale-normalization branch from c9ab9e7 to 93853eb Compare September 4, 2026 07:52
@iAstro

iAstro commented Sep 4, 2026

Copy link
Copy Markdown
Contributor Author

简单说:需求对,但是实现方式有问题,如下:

问题

当前 normalizeLanguageCode 采用硬编码的 if/else 逐语言处理:

if (primary === "en") return "en-US";
if (primary === "zh") { /* ... */ }
return code;

这等于后面每新增一种语言,都必须回来修改这个函数#161 正在添加韩语 (ko-KR),合并后如果浏览器语言是 bare konormalizeLanguageCode 会原样返回 ko,匹配不到 ko-KR resource,韩语用户会看到英文而非韩语——归一化白做了。后续再加日语、法语等也都有同样的问题。

建议

normalizeLanguageCode 改为由 resources 的 key 自动驱动的工厂函数,不用手动维护映射表:

export function createLanguageNormalizer(
  resourceKeys: string[],
): (code: string) => string {
  const keySet = new Set(resourceKeys);
  const index = buildPrimarySubtagIndex(resourceKeys);
  // index: Map { "en" → ["en-US"], "zh" → ["zh-CN"], "ko" → ["ko-KR"], ... }
  return (code: string): string => {
    // 1. 精确匹配 → 直接返回
    if (keySet.has(code)) return code;
    const parts = code.split(/[-_]/).map((p) => p.toLowerCase());
    const primary = parts[0];
    const candidates = index.get(primary);
    if (!candidates || candidates.length === 0) return code;
    // 2. 一个 primary subtag 只对应一个 resource → 自动映射
    //    ko → ko-KR, en → en-US, ja → ja-JP ...
    if (candidates.length === 1) return candidates[0];
    // 3. 一个 primary subtag 对应多个 resource → 消歧(目前只有 zh)
    if (primary === "zh") return resolveChineseVariant(parts, candidates);
    return candidates[0];
  };
}

调用侧传入 resource keys:

// i18n.ts
const resourceKeys = Object.keys(resources);
// ...
detection: getLanguageDetectionOptions(resourceKeys),

getLanguageDetectionOptions 签名相应调整为接收 resourceKeys 参数:

export function getLanguageDetectionOptions(resourceKeys: string[]) {
  const normalize = createLanguageNormalizer(resourceKeys);
  return {
    order: ["chromeUILanguage", "navigator"],
    caches: [],
    convertDetectedLanguage: normalize,
  };
}

中文简繁消歧(resolveChineseVariant)保留为唯一的特殊逻辑——这确实是特殊情况(一个语言两种书写系统),其他语言的映射完全自动推导。

收益

场景 当前方案 改进后
新增韩语 ko-KR 需改 normalizeLanguageCodeif 分支 只需注册到 resources零改动
新增日语 ja-JP 同上 同上
新增繁体 zh-TW 不需要改(已预写) 也不需要改(candidate 数 > 1 时自动触发消歧)
映射表与 resources 不一致的风险 存在(两处手动维护) 不存在(单一事实来源)
这样也能避免和 #161 之间的合并顺序依赖——无论谁先合,后合的只需要在 resources 里注册翻译文件就行。

测试方面,现有 28 个 case 全部兼容,建议额外补两组验证扩展性的 case:

it("auto-resolves new languages from resource keys", () => {
  const normalize = createLanguageNormalizer(["en-US", "zh-CN", "ko-KR"]);
  expect(normalize("ko")).toBe("ko-KR");
  expect(normalize("ko-KR")).toBe("ko-KR");
});
it("disambiguates zh when multiple zh resources exist", () => {
  const normalize = createLanguageNormalizer(["en-US", "zh-CN", "zh-TW"]);
  expect(normalize("zh-Hant")).toBe("zh-TW");
  expect(normalize("zh-HK")).toBe("zh-TW");
  expect(normalize("zh")).toBe("zh-CN");
});

其余(chromeUiLanguageDetectorfallbackLng 对象化、vitest 配置)暂时没有发现问题。

已按建议重构完成(93853eb)。

实现与您的伪代码逐行一致:

  • createLanguageNormalizer(resourceKeys):精确匹配 → primary 索引单候选自动映射 → 多候选仅 zhresolveChineseVariant → 无候选原样返回交给 fallbackLng
  • buildPrimarySubtagIndex 产出与您注释中的 Map { "en" → ["en-US"], "ko" → ["ko-KR"], ... } 结构一致
  • getLanguageDetectionOptions 已改为接收 resourceKeysi18n.ts 传入 Object.keys(resources)
  • 中文简繁消歧保留为唯一硬编码分支,其余语言映射完全由索引自动推导

测试:建议的两组 case 均已补上(ko → ko-KR 自动解析、["en-US","zh-CN","zh-TW"] 简繁消歧,后者为超集,另含 zh-MO/zh-Hant-TW 等)。另补了一组未注册语言原样返回的 case,覆盖 !candidates 分支。

一处期望值变化说明zh-TW 家族 7 条用例的期望从 zh-TW 改为 zh-CN——当前仅 zh-CN 发货,工厂函数只映射到已发货键。旧实现输出 zh-TW 后经 fallbackLng.zh 回退,渲染结果相同,但归一化器返回了不存在的键。disambiguates zh once a Traditional bundle ships 已锁定繁体包注册后的自动消歧,届时零改动生效。

chromeUiLanguageDetectorfallbackLng、vitest 配置未动。分支已 rebase 到最新 main;extension 832/832、i18n 31/31,tsc / biome / build 全绿。

@iuyo5678

iuyo5678 commented Sep 4, 2026

Copy link
Copy Markdown
Collaborator

LGTM

@iuyo5678
iuyo5678 merged commit 47ac947 into Tencent:main Sep 4, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Extension UI falls back to Chinese for any locale other than en-US | 除 en-US 外所有语言环境下扩展界面均显示中文

2 participants