Skip to content

Commit 4006a92

Browse files
committed
feat(audio): 进程环回默认启用 + 设置页音频源选择(Phase 3)
ADR-001 Phase 3:进程环回从灰度转默认启用,并补齐用户可控与可归因链路。 - flag 反转为默认开启,ENTROPY_PROCESS_LOOPBACK=0 保留为应急总闸 (现场排障可不重新发版即关闭) - audioSourcePreference:偏好持久化(localStorage),读写全程静默降级, 绝不因偏好读取失败阻断采集启动;6 例单测覆盖非法值与存储异常 - 设置页新增「课堂音频采集」区块:自动 / 仅目标窗口声音 / 系统全部声音, 各选项标注取舍(干净但可能漏采 vs 不漏采但含杂音) - audio_capture_start 双向打通:入参接收 preference(主进程读不到 localStorage),返回值回传 sourceKind + sourceReason - useAudioRecovery 按源分支:对进程环回不再提示'检查系统默认输出设备' (该源不受输出设备与系统音量影响,属误导),改为提示确认目标窗口在播放; devicechange 自动重启仅端点环回保留 - 生效源经 useClassroomCapture 暴露,供 UI 展示与内测归因 验证:client 479 tests、lint 0 errors、渲染与主进程 tsc 均 0。
1 parent bc37a96 commit 4006a92

10 files changed

Lines changed: 311 additions & 28 deletions

‎client/electron/audioCapture.ts‎

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -32,13 +32,13 @@ import {
3232
} from '../src/lib/capture/audioSourceStrategy.js';
3333

3434
/**
35-
* 进程环回特性开关(Phase 2:默认关)。
35+
* 进程环回特性开关(Phase 3:默认开)。
3636
*
37-
* 开启方式:环境变量 ENTROPY_PROCESS_LOOPBACK=1。
38-
* Phase 3 经内测灰度后改为默认开并改由设置页控制。
37+
* 默认开启后用户仍可在设置页选“系统全部声音”强制用端点环回;
38+
* ENTROPY_PROCESS_LOOPBACK=0 作为应急总闸(现场排障时无需重新发版即可关闭)。
3939
*/
4040
function isProcessLoopbackEnabled(): boolean {
41-
return process.env.ENTROPY_PROCESS_LOOPBACK === '1';
41+
return process.env.ENTROPY_PROCESS_LOOPBACK !== '0';
4242
}
4343

4444
// 保持既有导出路径不变(mediaCaptureHandlers 等调用方无需改动)
@@ -132,7 +132,7 @@ export class AudioCapture {
132132
if (this.capturing || this.disposed) return;
133133

134134
const resolvedSourceId = sourceId ?? null;
135-
// 特性开关关闭时能力恒为不可用,选源必为端点环回(行为与 Phase 0 一致)
135+
// 应急总闸关闭时能力恒为不可用,选源退回端点环回
136136
const processAvailable = isProcessLoopbackEnabled() && isProcessLoopbackAvailable();
137137
this.decision = selectAudioSource({
138138
capabilities: { processLoopbackAvailable: processAvailable },

‎client/electron/mediaCaptureHandlers.ts‎

Lines changed: 19 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -10,6 +10,7 @@
1010
import { BrowserWindow, ipcMain } from 'electron';
1111
import { AudioCapture, listAudioSources } from './audioCapture.js';
1212
import type { AudioCaptureOptions, AudioChunk } from './audioCapture.js';
13+
import type { AudioSourcePreference } from '../src/lib/capture/audioSourceStrategy.js';
1314
import { VideoRecorder } from './videoRecorder.js';
1415
import type { VideoRecordOptions } from './videoRecorder.js';
1516
import { safeHandle, getMainWindowId } from './ipcUtils.js';
@@ -55,7 +56,14 @@ export function registerMediaCaptureHandlers(): void {
5556

5657
safeHandle(
5758
'audio_capture_start',
58-
async (event, options?: Partial<AudioCaptureOptions> & { sourceId?: string }) => {
59+
async (
60+
event,
61+
options?: Partial<AudioCaptureOptions> & {
62+
sourceId?: string;
63+
/** 用户在设置页选择的音频源偏好(主进程读不到 localStorage,由渲染进程传入) */
64+
preference?: AudioSourcePreference;
65+
},
66+
) => {
5967
if (activeAudioCapture) {
6068
activeAudioCapture.dispose();
6169
activeAudioCapture = null;
@@ -79,9 +87,17 @@ export function registerMediaCaptureHandlers(): void {
7987
});
8088

8189
try {
82-
await activeAudioCapture.start(senderWin, options?.sourceId);
90+
await activeAudioCapture.start(senderWin, options?.sourceId, {
91+
preference: options?.preference,
92+
});
93+
const decision = activeAudioCapture.sourceDecision;
8394
logger.info('[IPC] audio_capture_start 已启动');
84-
return { success: true };
95+
// 回传生效源:渲染进程据此分支诊断文案并写入会话元数据(内测归因)
96+
return {
97+
success: true,
98+
sourceKind: activeAudioCapture.activeSourceKind,
99+
sourceReason: decision?.reason,
100+
};
85101
} catch (err) {
86102
const message = err instanceof Error ? err.message : String(err);
87103
logger.error('[IPC] audio_capture_start failed:', message);

‎client/src/features/classroom/hooks/useAudioRecovery.ts‎

Lines changed: 30 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -2,14 +2,18 @@
22
* 课堂音频自动恢复 hook(静音诊断 / 设备变更重启)
33
*
44
* @ai-context: 从 useClassroomAudio 拆出的独立恢复层。两条恢复路径:
5-
* ①输出设备不匹配诊断——系统环回只录默认输出设备,视频声音若输出到其他
6-
* 设备(HDMI/蓝牙)会表现为"音频块正常但持续静音",检出后提示用户核对;
7-
* ②设备变更重启——默认输出设备切换后环回仍绑定旧设备,devicechange 时
8-
* 自动 stop→start 重新绑定。
5+
* ①静音诊断——音频块正常但持续静音,成因随音频源不同(见下);
6+
* ②设备变更重启——仅端点环回需要(它绑定系统默认输出设备),进程环回
7+
* 与输出设备无关,切设备时不必重启。
8+
* @ai-context: 诊断文案必须按 sourceKind 分支(ADR-001)——对进程环回说
9+
* "请检查默认输出设备"是误导,它根本不受输出设备与系统音量影响,此时
10+
* 真实成因是目标窗口没在发声或声音来自别的应用。
911
* @ai-context: 仅依赖 refs 的稳定回调,重启经 restartingRef 互斥防并发。
1012
*/
1113
import { useEffect, useRef, useCallback } from 'react';
1214
import type { CaptureMode, SessionStatus } from '@/lib/capture';
15+
import type { AudioSourceKind } from '@/lib/capture/audioSourceStrategy';
16+
import { getAudioSourcePreference } from '@/lib/capture/audioSourcePreference';
1317
import {
1418
computeChunkRms, SilenceTracker,
1519
getDefaultOutputDeviceLabel, subscribeDeviceChange,
@@ -25,17 +29,25 @@ interface UseAudioRecoveryOptions {
2529
mode: CaptureMode;
2630
/** 会话选中窗口的源 ID,重启时沿用同一源 */
2731
audioSourceId?: string | null;
32+
/** 本次会话实际生效的音频源(由 audio_capture_start 回传) */
33+
sourceKind?: AudioSourceKind | null;
2834
onNotify: (type: 'warning' | 'error', message: string) => void;
2935
}
3036

31-
export function useAudioRecovery({ status, mode, audioSourceId, onNotify }: UseAudioRecoveryOptions) {
37+
export function useAudioRecovery({
38+
status, mode, audioSourceId, sourceKind, onNotify,
39+
}: UseAudioRecoveryOptions) {
3240
const notifyRef = useRef(onNotify);
3341
notifyRef.current = onNotify;
3442
const effectiveSourceRef = useRef<string | undefined>(audioSourceId ?? undefined);
3543
const silenceTrackerRef = useRef(new SilenceTracker());
3644
const restartingRef = useRef(false);
45+
// 生效源用 ref 桥接:静音诊断的监听器依赖数组不含它,避免重订阅丢失计数
46+
const sourceKindRef = useRef<AudioSourceKind | null>(sourceKind ?? null);
47+
sourceKindRef.current = sourceKind ?? null;
3748

3849
const audioEnabled = status === 'capturing' && (mode === 'audio' || mode === 'mixed');
50+
const isProcessSource = sourceKind === 'process_loopback';
3951

4052
// 会话开始时重置恢复状态(audioSourceId 取会话启动瞬间的快照)
4153
useEffect(() => {
@@ -53,7 +65,7 @@ export function useAudioRecovery({ status, mode, audioSourceId, onNotify }: UseA
5365
await window.electronAPI.invoke('audio_capture_stop');
5466
await new Promise((r) => setTimeout(r, RESTART_CLEANUP_DELAY_MS));
5567
const result = await window.electronAPI.invoke('audio_capture_start', {
56-
...AUDIO_START_OPTIONS, sourceId,
68+
...AUDIO_START_OPTIONS, sourceId, preference: getAudioSourcePreference(),
5769
}) as { success: boolean; error?: string };
5870
if (result.success) silenceTrackerRef.current.reset();
5971
else console.warn('[useAudioRecovery] 音频捕获重启失败:', result.error);
@@ -66,12 +78,20 @@ export function useAudioRecovery({ status, mode, audioSourceId, onNotify }: UseA
6678
}
6779
}, []);
6880

69-
// 静音诊断:音频块正常但持续无声 → 提示核对系统默认输出设备
81+
// 静音诊断:音频块正常但持续无声 → 按生效源给出对应成因
7082
useEffect(() => {
7183
if (!audioEnabled || !window.electronAPI) return;
7284
const off = window.electronAPI.on('audio_capture_chunk', (...args: unknown[]) => {
7385
const chunk = args[0] as { audioBuffer: ArrayBuffer };
7486
if (!silenceTrackerRef.current.push(computeChunkRms(chunk.audioBuffer))) return;
87+
88+
if (sourceKindRef.current === 'process_loopback') {
89+
// 进程环回不受系统音量/输出设备影响,成因只能是目标窗口没在发声
90+
notifyRef.current('warning',
91+
'持续收到静音音频:当前只采集目标窗口的声音,请确认该窗口正在播放,' +
92+
'或在设置中改为采集「系统全部声音」');
93+
return;
94+
}
7595
void getDefaultOutputDeviceLabel().then((label) => {
7696
notifyRef.current('warning',
7797
`持续收到静音音频:请确认视频声音正在播放,且输出到系统默认设备${label ? `「${label}」` : ''}(系统音频捕获只能录到默认输出设备的声音)`);
@@ -80,9 +100,9 @@ export function useAudioRecovery({ status, mode, audioSourceId, onNotify }: UseA
80100
return off;
81101
}, [audioEnabled]);
82102

83-
// 设备变更自动重启:重新绑定新的默认输出设备
103+
// 设备变更自动重启:仅端点环回需要(它绑定系统默认输出设备)
84104
useEffect(() => {
85-
if (!audioEnabled || !window.electronAPI) return;
105+
if (!audioEnabled || !window.electronAPI || isProcessSource) return;
86106
const unsubscribe = subscribeDeviceChange(() => {
87107
console.info('[useAudioRecovery] 检测到音频设备变更,自动重启音频捕获');
88108
void restartCapture(effectiveSourceRef.current).then((ok) => {
@@ -91,5 +111,5 @@ export function useAudioRecovery({ status, mode, audioSourceId, onNotify }: UseA
91111
});
92112
});
93113
return unsubscribe;
94-
}, [audioEnabled, restartCapture]);
114+
}, [audioEnabled, isProcessSource, restartCapture]);
95115
}

‎client/src/features/classroom/hooks/useClassroomCapture.ts‎

Lines changed: 10 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -12,6 +12,7 @@
1212
import { useState, useCallback, useEffect, useRef, useMemo } from 'react';
1313
import { useToast } from '@/components/ui/Toast';
1414
import { CaptureManager } from '@/lib/capture';
15+
import type { AudioSourceKind } from '@/lib/capture/audioSourceStrategy';
1516
import type {
1617
CaptureMode,
1718
CaptureSidebarConfig,
@@ -50,6 +51,10 @@ export function useClassroomCapture() {
5051
// ── 课中重点标记 ──
5152
const [bookmarks, setBookmarks] = useState<{ timestamp: number; label?: string }[]>([]);
5253

54+
// 本次会话实际生效的音频源(ADR-001):由主进程选源后回传,
55+
// 用于诊断文案分支与 UI 展示,避免对进程环回给出“检查输出设备”类误导提示
56+
const [audioSourceKind, setAudioSourceKind] = useState<AudioSourceKind | null>(null);
57+
5358
const notify = useCallback((type: 'success' | 'warning' | 'error' | 'info', message: string) => {
5459
toast({ type, message });
5560
}, [toast]);
@@ -107,9 +112,9 @@ export function useClassroomCapture() {
107112
captureManager, status, mode, onNotify: notify,
108113
});
109114

110-
// 音频自动恢复:静音诊断 / 窗口源回退环回 / 设备变更重启
115+
// 音频自动恢复:静音诊断(文案按生效源分支)/ 设备变更重启
111116
useAudioRecovery({
112-
status, mode, audioSourceId: selectedWindow?.id, onNotify: notify,
117+
status, mode, audioSourceId: selectedWindow?.id, sourceKind: audioSourceKind, onNotify: notify,
113118
});
114119

115120
const analysis = useClassroomAnalysis({
@@ -150,6 +155,7 @@ export function useClassroomCapture() {
150155
onAnalyzeFull: analysis.handleAnalyze,
151156
onMergePartials: analysis.mergePartialNotes,
152157
onNotify: (type, message) => notify(type, message),
158+
onAudioSourceResolved: setAudioSourceKind,
153159
});
154160

155161
const handleModeChange = useCallback((newMode: CaptureMode) => {
@@ -214,6 +220,8 @@ export function useClassroomCapture() {
214220
liveTranscripts: events.liveTranscripts,
215221
// 音频健康 + VAD
216222
audioHealth, vadStats: events.vadStats,
223+
// 本次会话生效的音频源(UI 可见,供内测归因)
224+
audioSourceKind,
217225
// 课程上下文
218226
courseMeta, setCourseMeta, aiDetectEnabled, setAiDetectEnabled,
219227
// 录制

‎client/src/features/classroom/hooks/useSessionControl.ts‎

Lines changed: 20 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -12,6 +12,8 @@ import { useCallback } from 'react';
1212
import { requireGatewayUrl } from '@/lib/ai/config';
1313
import { soundPlayer } from '@/lib/audio/SoundPlayer';
1414
import { analyzePartial } from '@/lib/ai/sessionAnalyzer';
15+
import { getAudioSourcePreference } from '@/lib/capture/audioSourcePreference';
16+
import type { AudioSourceKind } from '@/lib/capture/audioSourceStrategy';
1517
import type {
1618
CaptureManager,
1719
CaptureMode,
@@ -28,6 +30,10 @@ import type {
2830
interface IPCAudioStartResult {
2931
success: boolean;
3032
error?: string;
33+
/** 实际生效的音频源(ADR-001 双源选择结果) */
34+
sourceKind?: AudioSourceKind;
35+
/** 选源理由(含降级说明),供内测问题归因 */
36+
sourceReason?: string;
3137
}
3238

3339
interface UseSessionControlOptions {
@@ -56,12 +62,14 @@ interface UseSessionControlOptions {
5662
onAnalyzeFull: () => void;
5763
onMergePartials: (partials: string[], durationMs: number, keyframeCount: number) => Promise<void>;
5864
onNotify: (type: 'warning', message: string) => void;
65+
/** 音频源定下后回报(供诊断文案分支与 UI 展示) */
66+
onAudioSourceResolved?: (kind: AudioSourceKind | null) => void;
5967
}
6068

6169
export function useSessionControl({
6270
captureManager, selectedWindow, status, setStatus, mode, capturePath, config, courseMeta,
6371
frameRestartRef, audioCleanupRef, session,
64-
onAnalyzeVideo, onAnalyzeFull, onMergePartials, onNotify,
72+
onAnalyzeVideo, onAnalyzeFull, onMergePartials, onNotify, onAudioSourceResolved,
6573
}: UseSessionControlOptions) {
6674
/** 预检 AI 网关连通性(不可用仅提示,不阻断采集) */
6775
const probeGateway = useCallback(async () => {
@@ -130,14 +138,22 @@ export function useSessionControl({
130138

131139
if (audioEnabled) {
132140
try {
133-
// 方案A:优先以选中窗口为音频源(直采 B站客户端/浏览器等目标应用声音),
134-
// 窗口级捕获不受支持时主进程会下发环回降级候选,由渲染端自动回退
141+
// 选源由主进程的 selectAudioSource 决定(ADR-001):锁定具体窗口时
142+
// 优先进程环回(隔离其他应用杂音),否则用端点环回(不漏采);
143+
// 主进程读不到 localStorage,故偏好由渲染进程传入
135144
const audioResult = await window.electronAPI.invoke('audio_capture_start', {
136145
chunkDurationMs: 5000, sampleRate: 16000, channels: 1,
137146
sourceId: selectedWindow.id,
147+
preference: getAudioSourcePreference(),
138148
}) as IPCAudioStartResult;
139149
if (!audioResult.success) {
140150
console.warn('[useClassroomCapture] Audio start failed:', audioResult.error);
151+
} else {
152+
console.info(
153+
`[useClassroomCapture] 音频源=${audioResult.sourceKind ?? 'unknown'}` +
154+
`(${audioResult.sourceReason ?? '-'})`,
155+
);
156+
onAudioSourceResolved?.(audioResult.sourceKind ?? null);
141157
}
142158
} catch (audioErr) {
143159
console.warn('[useClassroomCapture] Audio unavailable:', audioErr);
@@ -147,7 +163,7 @@ export function useSessionControl({
147163
setStatus('error');
148164
console.error('[useClassroomCapture] Start failed:', err);
149165
}
150-
}, [selectedWindow, setStatus, session, probeGateway, capturePath, captureManager, config, mode, courseMeta]);
166+
}, [selectedWindow, setStatus, session, probeGateway, capturePath, captureManager, config, mode, courseMeta, onAudioSourceResolved]);
151167

152168
const handlePause = useCallback(() => {
153169
if (status === 'capturing') {
Lines changed: 60 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,60 @@
1+
/**
2+
* @ai-context: 音频源偏好持久化单测。重点覆盖"读取失败/非法值必须回落
3+
* 到 auto"——偏好读取绝不能阻断采集启动(见 audioSourcePreference 头注)。
4+
*/
5+
import { describe, it, expect, beforeEach, vi, afterEach } from 'vitest';
6+
import {
7+
getAudioSourcePreference,
8+
setAudioSourcePreference,
9+
AUDIO_SOURCE_PREFERENCE_KEY,
10+
AUDIO_SOURCE_PREFERENCE_LABELS,
11+
} from './audioSourcePreference';
12+
13+
describe('audioSourcePreference', () => {
14+
beforeEach(() => {
15+
localStorage.clear();
16+
});
17+
18+
afterEach(() => {
19+
vi.restoreAllMocks();
20+
});
21+
22+
it('未设置时默认 auto', () => {
23+
expect(getAudioSourcePreference()).toBe('auto');
24+
});
25+
26+
it('可写入并读回三种合法值', () => {
27+
for (const value of ['auto', 'force_process', 'force_endpoint'] as const) {
28+
setAudioSourcePreference(value);
29+
expect(getAudioSourcePreference()).toBe(value);
30+
}
31+
});
32+
33+
it('存储中的非法值回落到 auto', () => {
34+
localStorage.setItem(AUDIO_SOURCE_PREFERENCE_KEY, 'force_microphone');
35+
expect(getAudioSourcePreference()).toBe('auto');
36+
});
37+
38+
it('localStorage 读取抛错时回落到 auto 而非抛出', () => {
39+
vi.spyOn(Storage.prototype, 'getItem').mockImplementation(() => {
40+
throw new Error('storage unavailable');
41+
});
42+
expect(() => getAudioSourcePreference()).not.toThrow();
43+
expect(getAudioSourcePreference()).toBe('auto');
44+
});
45+
46+
it('localStorage 写入抛错时静默降级而非抛出', () => {
47+
vi.spyOn(Storage.prototype, 'setItem').mockImplementation(() => {
48+
throw new Error('quota exceeded');
49+
});
50+
expect(() => setAudioSourcePreference('force_process')).not.toThrow();
51+
});
52+
53+
it('每种偏好都有非空的标签与说明(设置页依赖)', () => {
54+
for (const value of ['auto', 'force_process', 'force_endpoint'] as const) {
55+
const { label, hint } = AUDIO_SOURCE_PREFERENCE_LABELS[value];
56+
expect(label.length).toBeGreaterThan(0);
57+
expect(hint.length).toBeGreaterThan(0);
58+
}
59+
});
60+
});

0 commit comments

Comments
 (0)