webrtc: audio processing modules suite (AEC, NS, VAD, RNNoise, AGC, HPF) by lgirdwood · Pull Request #11314 · thesofproject/sof

lgirdwood · 2026-10-15T15:57:29Z

Overview

This is PR 2 in the series splitting monolithic PR #10990 into modular, reviewable PRs rebased on main.

This PR introduces the WebRTC & Neural Audio Processing Suite, enabling real-time on-DSP speech enhancement, acoustic echo cancellation, voice activity detection, and noise suppression across SOF hardware platforms (including Panther Lake, Tiger Lake, Lunar Lake, Meteor Lake, and Arrow Lake).

All third-party library dependencies are managed via West (west.yml) pointing to official thesofproject forks, with complete pass-through stub backends (CONFIG_COMP_WEBRTC_*_STUB=y) to guarantee 100% test pass rates in CI without external checkouts.

Audio Processing Modules in this PR

1. `webrtc_vad` (Voice Activity Detection)

Algorithm: Gaussian Mixture Model (GMM) voice activity classifier extracted from WebRTC via thesofproject/libfvad (pure C).
Features: 8/16/32/48 kHz sample rates, 10/20/30 ms frame analysis, 4 operational modes (Quality, Low Bitrate, Aggressive, Very Aggressive).
Notifier Integration: Emits real-time binary decisions (1 speech / 0 silence) over NOTIFIER_ID_VAD for downstream processing without polling.
UUID: c790b11d-5d14-e54e-be36ba4ad732cc14

2. `webrtc_ns` (Classic Spectral Noise Suppression)

Algorithm: Fixed-point spectral subtraction / Wiener filter from thesofproject/webrtc-audio-processing (0.3.1 pure-C core).
Features: 8 and 16 kHz operation on 10 ms frames, 4 suppression levels (Mild to Very Aggressive), mono/stereo support.
Self-Contained Shims: Includes isolated soft-float math shims (fabsf, fabs, expf, exp, tanh, atanf) to eliminate host libm dependencies in minimal libc.
UUID: 0fc8faef-945f-004b-8d5f315047f1136a

3. `webrtc_aec` (Acoustic Echo Cancellation)

Architecture: Dual-source capture effect (pin 0 = microphone capture, pin 1 = playback echo reference), single-sink cancelled output.
Three Selectable Backends:
- AECm: Pure fixed-point Q15 echo canceller optimized for embedded/low-power DSP execution.
- Classic Float AEC: Single-precision float AEC with soft-float polynomial math lowering.
- Modern AEC3 (C++17): Stereo 2x2 MIMO echo canceller with native hardware FPU acceleration on Panther Lake (ACE 3.0).
UUID: e5da7b5b-133a-ba46-b517651d6300bb83

4. `webrtc_ns2` (RNNoise Deep Learning Noise Suppression)

Algorithm: Recurrent neural network (GRU-based) noise suppressor from thesofproject/rnnoise (pure C, BSD-3-Clause).
Features: 48 kHz locked operation, 10 ms (480-sample) framing, model weights mapped to Flash/ROM .rodata.
Integrated VAD: Computes per-frame speech probability and broadcasts NOTIFIER_ID_VAD events above configurable threshold.
UUID: eacfacdc-2a87-c942-97e894c917a740db

5. `webrtc_agc` (Digital Automatic Gain Control)

Algorithm: Fixed-point adaptive digital AGC and peak limiter from WebRTC APM (digital_agc.c).
Features: Target level (+9 dB compression gain) and peak limiting to prevent digital clipping.
UUID: 0af7b529-9dc7-4bee-a8b159e8fbe414f3

6. `webrtc_hpf` (Cascaded Biquad High-Pass Filter)

Algorithm: Fixed-point Q12/Q13 cascaded 2nd-order biquad IIR filter (<80/100 Hz cutoff).
Features: Eliminates DC offset and mechanical speaker rumble (<0.35% DSP load).
UUID: 6dfb1bea-d858-4ef7-92abcf5b4ee48159

West Manifest Integration (`west.yml`)

The module suite sources dependencies from official thesofproject forks:

libfvad at branch sof-hifi (e4b0c4d00b3db23f5825631ae8f84058e7a1b366)
webrtc-apm at branch sof-hifi (3e9c41a268fbeb36a7de2c5cb2f1b02bc7fef098)
rnnoise at branch sof-hifi (24dbb35115c6d27f9f14cc654bf0a32417f70bf9)

Topology2 & Pipelines

Component Widgets: webrtc-vad.conf, webrtc-ns.conf, webrtc-aec.conf, webrtc-ns2.conf, webrtc-agc.conf, webrtc-hpf.conf
Capture Pipelines: webrtc-ns-capture.conf, webrtc-aec-capture.conf, webrtc-ns2-capture.conf
Development Topologies: cavs-nocodec-webrtc-ns.conf, cavs-nocodec-webrtc-aec.conf, cavs-nocodec-webrtc-ns2.conf
Build Targets: Registered in tplg-targets.cmake for TGL nocodec.

Verification & Build Results

Panther Lake (PTL / ACE 3.0) Baseline:
- xtensa-build-zephyr.py -z -p ptl: PASS (code 0)
Panther Lake All WebRTC Stubs (webrtc_all_stubs.conf):
- xtensa-build-zephyr.py -z -p -o webrtc_all_stubs.conf ptl: PASS (code 0)
Panther Lake All Real WebRTC Backends (webrtc_real_y.conf):
- xtensa-build-zephyr.py -z -p -o webrtc_real_y.conf ptl: PASS (code 0)
Tiger Lake (TGL / CAVS 2.5) Stubs:
- xtensa-build-zephyr.py -z -p -o webrtc_all_stubs.conf tgl: PASS (code 0)
Cmocka Native Unit Tests:
- 32/32 tests passing: 100% PASS

Copilot

🟡 Changes recommended

Unresolved build, packaging, manifest, userspace, memory-safety, audio-processing, and topology defects block approval.

27 open findings

Honor the selected AEC backend in LLEXT builds · New
Avoid global approximate sin/cos replacements · New
Route WebRTC allocations through userspace module syscalls · New
Fix AECm include paths and architecture definitions · New
Route aligned AEC3 allocations through module syscalls · New
Guard undefined CONFIG_WEBRTC_AGC_LIMITER usage · New
Add the missing AGC LLEXT manifest wrapper · New
Reject unsupported sample rates or implement resampling · New
Add the missing HPF LLEXT manifest wrapper · New
Ensure HPF stub output is valid passthrough PCM · New
Handle circular source and sink buffer wrapping · New
Validate switch payload size before dereferencing · New
Add WebRTC root includes for noise suppression · New
Use userspace allocation hooks for noise suppression · New
Reject unsupported noise suppression rates above 16 kHz · New
Guard or independently define the VAD threshold macro · New
Move userspace scratch buffers into per-instance storage · New
Provide resample dependency for AGC and VAD builds · New
Restrict VAD frame duration to supported values · New
Avoid VAD double-free during reset failure · New

And 7 more that still need to be addressed.

What changed in this PR

Introduces six WebRTC/RNNoise audio-processing modules for SOF: VAD, noise suppression, echo cancellation, neural denoising, automatic gain control, and high-pass filtering.

Changes:

Adds processing backends, stubs, configuration, manifests, and documentation.
Pins external dependencies and registers module UUIDs and VAD notifications.
Adds Topology2 widgets, capture pipelines, and development targets.

File	Description
`west.yml`	Pins three audio dependencies.
`uuid-registry.txt`	Registers six module UUIDs.
`tools/topology/topology2/include/pipelines/cavs/webrtc-ns2-capture.conf`	Defines RNNoise capture pipeline.
`tools/topology/topology2/include/pipelines/cavs/webrtc-ns-capture.conf`	Defines VAD/NS capture pipeline.
`tools/topology/topology2/include/pipelines/cavs/webrtc-aec-capture.conf`	Defines AEC capture routing.
`tools/topology/topology2/include/components/webrtc-vad.conf`	Defines VAD widget.
`tools/topology/topology2/include/components/webrtc-ns2.conf`	Defines RNNoise widget.
`tools/topology/topology2/include/components/webrtc-ns.conf`	Defines NS widget.
`tools/topology/topology2/include/components/webrtc-hpf.conf`	Defines HPF widget and control.
`tools/topology/topology2/include/components/webrtc-agc.conf`	Defines AGC widget and control.
`tools/topology/topology2/include/components/webrtc-aec.conf`	Defines AEC widget and controls.
`tools/topology/topology2/development/tplg-targets.cmake`	Registers development topology targets.
`tools/topology/topology2/development/cavs-nocodec-webrtc-ns2.conf`	Adds RNNoise loopback topology.
`tools/topology/topology2/development/cavs-nocodec-webrtc-ns.conf`	Adds VAD/NS loopback topology.
`src/include/sof/lib/notifier.h`	Adds VAD notification ID.
`src/audio/webrtc_vad/webrtc_vad.toml`	Defines VAD manifest metadata.
`src/audio/webrtc_vad/webrtc_vad.h`	Defines VAD backend interface.
`src/audio/webrtc_vad/webrtc_vad.cmake`	Cross-builds libfvad.
`src/audio/webrtc_vad/webrtc_vad-stub.c`	Adds stub classifier.
`src/audio/webrtc_vad/webrtc_vad-shims.c`	Supplies libc shims.
`src/audio/webrtc_vad/webrtc_vad-fvad.c`	Integrates libfvad backend.
`src/audio/webrtc_vad/README.md`	Documents VAD integration.
`src/audio/webrtc_vad/llext/llext.toml.h`	Wraps loadable manifest.
`src/audio/webrtc_vad/llext/CMakeLists.txt`	Configures loadable VAD builds.
`src/audio/webrtc_vad/Kconfig`	Adds VAD configuration.
`src/audio/webrtc_vad/CMakeLists.txt`	Selects VAD build sources.
`src/audio/webrtc_ns2/webrtc_ns2.toml`	Defines RNNoise manifest metadata.
`src/audio/webrtc_ns2/webrtc_ns2.h`	Defines RNNoise interface and buffers.
`src/audio/webrtc_ns2/webrtc_ns2.cmake`	Cross-builds RNNoise.
`src/audio/webrtc_ns2/webrtc_ns2-stub.c`	Adds pass-through backend.
`src/audio/webrtc_ns2/webrtc_ns2-rnn.c`	Integrates RNNoise processing.
`src/audio/webrtc_ns2/README.md`	Documents RNNoise integration.
`src/audio/webrtc_ns2/llext/llext.toml.h`	Wraps loadable manifest.
`src/audio/webrtc_ns2/llext/CMakeLists.txt`	Configures loadable RNNoise builds.
`src/audio/webrtc_ns2/Kconfig`	Adds RNNoise configuration.
`src/audio/webrtc_ns2/CMakeLists.txt`	Selects RNNoise build sources.
`src/audio/webrtc_ns/webrtc_ns.toml`	Defines NS manifest metadata.
`src/audio/webrtc_ns/webrtc_ns.h`	Defines NS backend interface.
`src/audio/webrtc_ns/webrtc_ns.cmake`	Cross-builds WebRTC NS.
`src/audio/webrtc_ns/webrtc_ns-webrtc.c`	Integrates WebRTC NS backend.
`src/audio/webrtc_ns/webrtc_ns-stub.c`	Adds pass-through backend.
`src/audio/webrtc_ns/webrtc_ns-shims.c`	Supplies libc and math shims.
`src/audio/webrtc_ns/README.md`	Documents NS integration.
`src/audio/webrtc_ns/llext/llext.toml.h`	Wraps loadable manifest.
`src/audio/webrtc_ns/llext/CMakeLists.txt`	Configures loadable NS builds.
`src/audio/webrtc_ns/Kconfig`	Adds NS configuration.
`src/audio/webrtc_ns/CMakeLists.txt`	Selects NS build sources.
`src/audio/webrtc_hpf/webrtc_hpf.toml`	Defines HPF manifest metadata.
`src/audio/webrtc_hpf/webrtc_hpf.h`	Defines HPF state and interface.
`src/audio/webrtc_hpf/webrtc_hpf.c`	Implements lifecycle, streaming, and controls.
`src/audio/webrtc_hpf/webrtc_hpf-stub.c`	Adds HPF stub backend.
`src/audio/webrtc_hpf/webrtc_hpf-filter.c`	Implements fixed-point filtering.
`src/audio/webrtc_hpf/README.md`	Documents HPF integration.
`src/audio/webrtc_hpf/llext/CMakeLists.txt`	Configures loadable HPF builds.
`src/audio/webrtc_hpf/Kconfig`	Adds HPF configuration.
`src/audio/webrtc_hpf/CMakeLists.txt`	Selects HPF build sources.
`src/audio/webrtc_agc/webrtc_agc.toml`	Defines AGC manifest metadata.
`src/audio/webrtc_agc/webrtc_agc.h`	Defines AGC state and interface.
`src/audio/webrtc_agc/webrtc_agc-stub.c`	Adds fallback backend.
`src/audio/webrtc_agc/webrtc_agc-agc.c`	Implements framed digital AGC.
`src/audio/webrtc_agc/README.md`	Documents AGC integration.
`src/audio/webrtc_agc/llext/CMakeLists.txt`	Configures loadable AGC builds.
`src/audio/webrtc_agc/Kconfig`	Adds AGC configuration.
`src/audio/webrtc_agc/CMakeLists.txt`	Selects AGC build sources.
`src/audio/webrtc_aec/webrtc_aec3_wrapper.h`	Declares AEC3 C interface.
`src/audio/webrtc_aec/webrtc_aec3_wrapper.cpp`	Wraps C++ AEC3 processing.
`src/audio/webrtc_aec/webrtc_aec3_shims.cpp`	Supplies C++ runtime adaptations.
`src/audio/webrtc_aec/webrtc_aec.toml`	Defines AEC manifest metadata.
`src/audio/webrtc_aec/webrtc_aec.h`	Defines AEC interfaces and state.
`src/audio/webrtc_aec/webrtc_aec.cmake`	Cross-builds AECm.
`src/audio/webrtc_aec/webrtc_aec-stub.c`	Adds microphone pass-through backend.
`src/audio/webrtc_aec/webrtc_aec-shims.c`	Supplies allocator wrappers.
`src/audio/webrtc_aec/webrtc_aec-math.c`	Supplies approximate math functions.
`src/audio/webrtc_aec/webrtc_aec-aecm.c`	Integrates AECm backend.
`src/audio/webrtc_aec/llext/llext.toml.h`	Wraps loadable manifest.
`src/audio/webrtc_aec/llext/CMakeLists.txt`	Configures loadable AEC builds.
`src/audio/webrtc_aec/Kconfig`	Adds AEC backend selection and options.
`src/audio/Kconfig`	Includes module configurations.
`src/audio/CMakeLists.txt`	Registers module build directories.

🧠 Review effort: Balanced

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

lgirdwood · 2026-10-15T16:42:13Z

Thanks for the thorough review! All 27 review comments have been addressed, and all 6 modules have been updated to strictly adhere to the Pipeline 2.0 Sink/Source circular buffer API conventions as described in #11297.

Summary of Changes

1. Sink/Source API Conformance (per #11297)

Across all 6 modules (webrtc_hpf, webrtc_agc, webrtc_ns, webrtc_ns2, webrtc_vad, webrtc_aec):

Replaced linear buffer pointer arithmetic with struct cir_buf_source and struct cir_buf_sink.
Evaluated contiguous fragment boundaries before wrapping using circ_buf_frames_without_wrap().
Utilized source_cir_buf_wrap() and cir_buf_wrap() for rollover handling.
Employed cir_buf_copy() for all pass-through and fallback buffer copying.

2. Detailed Review Fixes

Module 1 (webrtc_hpf):
- Created missing loadable manifest wrapper src/audio/webrtc_hpf/llext/llext.toml.h (Comment 9).
- Fixed webrtc_hpf-stub.c to perform pass-through audio copy via cir_buf_copy() (Comment 10).
- Rewrote webrtc_hpf_process() using circular buffer wrapping helpers (Comment 11).
- Added strict payload size validation for IPC4 switch controls in webrtc_hpf_set_config() (Comment 12).
- Standardized SPDX license comment style to // across HPF files.
Module 2 (webrtc_agc):
- Created missing loadable manifest wrapper src/audio/webrtc_agc/llext/llext.toml.h (Comment 7).
- Fixed webrtc_agc-stub.c to perform pass-through audio copy via cir_buf_copy().
- Replaced manual #ifdef with IS_ENABLED(CONFIG_WEBRTC_AGC_LIMITER) (Comment 6).
- Restricted webrtc_agc_prepare() to supported sample rates (8 kHz and 16 kHz) (Comment 8).
- Updated webrtc_agc_process() with circular buffer wrapping and added payload validation in set_config().
Module 3 (webrtc_ns):
- Added ${SOF_WEBRTC_APM_SRC_DIR} to INCLUDES in llext/CMakeLists.txt and installed transitive webrtc/typedefs.h in webrtc_ns.cmake (Comment 13).
- Bound malloc/free/calloc in webrtc_ns-shims.c to mod_alloc/mod_free/mod_zalloc via active_ns_mod (Comment 14).
- Restricted webrtc_ns_prepare() to 8 kHz and 16 kHz (Comment 15).
- Rewrote webrtc_ns_process() for circular buffer rollover and zero-fill.
- Updated src/audio/webrtc_ns/README.md to document the single-precision float Wiener filter (ns_core.c) (Comment 27).
Module 4 (webrtc_ns2 / RNNoise):
- Removed depends on WEBRTC_NS2_VAD_NOTIFY from WEBRTC_NS2_VAD_THRESHOLD_PCT in Kconfig and guarded notification logic at runtime (Comment 16).
- Moved writable static globals rnn_in and rnn_out into heap-allocated struct struct webrtc_ns2_rnn_data (Comment 17).
- Rewrote webrtc_ns2_process() with circular buffer wrapping.
Module 5 (webrtc_vad):
- Separated resample_48khz.c from the AGC build condition in CMakeLists.txt (Comment 18).
- Enforced validation of frame_ms (must be 10, 20, or 30 ms) in webrtc_vad_prepare() (Comment 19).
- Used in-place fvad_reset(fvad) in webrtc_vad_fvad_reset() instead of free + new (Comment 20).
- Routed memory allocations in webrtc_vad-shims.c via active_vad_mod and mod_alloc/mod_free (Comment 21).
- Rewrote webrtc_vad_process() with circular buffer accumulation and pass-through copy.
Module 6 (webrtc_aec):
- Constrained LLEXT backend choices in Kconfig to COMP_WEBRTC_AEC = y for Float and AEC3 (Comment 1).
- Replaced the 16-point LUT in webrtc_aec-math.c with high-accuracy polynomial functions with range reduction (Comment 2).
- Bound memory allocations to mod_alloc/mod_free via active_aec_mod in webrtc_aec-shims.c and webrtc_aec3_shims.cpp (AlignedMalloc/AlignedFree) (Comments 3 & 5).
- Added ${SOF_WEBRTC_APM_SRC_DIR} to INCLUDES and installed webrtc/typedefs.h in webrtc_aec.cmake (Comment 4).
- Changed num_input_pins from 1 to 2 in webrtc-aec.conf (Comment 23).
- Updated src_to_s16() and s16_to_sink() in webrtc_aec.c with circular buffer wrapping.
Platform Manifests & Topologies:
- Added conditional #include for all 6 modules in ptl.toml.h, lnl.toml.h, mtl.toml.h, tgl.toml.h, tgl-h.toml.h, wcl.toml.h, nvl.toml.h (Comment 25).
- Fixed "ptl" "platform/intel/ptl.conf" in cavs-nocodec-webrtc-ns.conf, cavs-nocodec-webrtc-ns2.conf, and cavs-nocodec-webrtc-aec.conf (Comments 22 & 26).
- Aligned capture rate to 16 kHz in webrtc-ns-capture.conf and cavs-nocodec-webrtc-ns.conf (Comment 24).

3. Verification

Built PTL with all 6 modules in stub mode (xtensa-build-zephyr.py -z -p ptl): PASS
Built TGL with all 6 modules in stub mode (xtensa-build-zephyr.py -z -p tgl): PASS
Built PTL with all 6 modules in real backend mode (xtensa-build-zephyr.py -z -p ptl): PASS
Verified commit style and SPDX headers with checkpatch.pl: PASS

intel-sofci · 2026-10-20T18:57:47Z

PR 11314: test results

Run date: 2026-10-21 20:08 UTC

Tested commit: dcfe5ac39a4675bdd0399c50ffdbfd194ad24f17

Copilot

🟡 Changes recommended

Unresolved linkage, userspace-access, audio-processing, configuration, and topology defects block approval.

12 open issues

27 resolved since last review

And 7 more resolved.

_{Review effort: Balanced}

lgirdwood · 2026-10-21T18:53:22Z

Addressed all 12 new review comments and resolved the 3 CI build failures in the updated series:

CI Fixes

Zephyr LLEXT/build (mtl) (rimage SIGSEGV):
- Adding the 6 WebRTC modules to tools/rimage/config/mtl.toml.h increased the total [[module.entry]] count under LLEXT_FORCE_ALL_MODULAR from 44 to 50, exceeding FW_MAX_EXT_MODULE_NUM (48) in tools/rimage/src/include/rimage/cavs/cavs_ext_manifest.h and overflowing ext_mod_config_array in adsp_config.c.
- Added commit rimage: raise maximum number of modules increasing FW_MAX_EXT_MODULE_NUM from 48 to 64 (matching commit 9b55f480262f).
testbench/build-and-test & top-level_default_CMake_target_ALL (alsatplg SIGSEGV):
- tools/topology/topology2/include/pipelines/cavs/webrtc-ns-capture.conf and webrtc-ns2-capture.conf contained index $index inside child Object.Widget blocks (webrtc-vad."1", webrtc-ns."1", webrtc-ns2."1"). Because alsatplg automatically inherits index from the parent Class.Pipeline, index $index caused infinite recursion in tplg_elem_Search during compilation of sof-tgl-nocodec-webrtc-ns.tplg and sof-tgl-nocodec-webrtc-ns2.tplg.
- Removed the redundant self-referential index $index lines; all three development topologies now compile cleanly with alsatplg.

Review Comments Addressed

src/audio/webrtc_aec/llext/CMakeLists.txt (comment):
- Added ../webrtc_aec-shims.c to SOURCES for the non-stub LLEXT build and added "${SOF_WEBRTC_APM_SRC_DIR}/webrtc/common_audio/signal_processing/include" to INCLUDES.
src/audio/webrtc_aec/webrtc_aec-shims.c (comment):
- Annotated active_aec_mod with APP_SYSUSER_BSS when !defined(LLEXT_MODULE) so built-in CONFIG_USERSPACE=y builds can access it from unprivileged user-mode threads, and removed the redundant webrtc_aec_set_active_mod(mod) call from webrtc_aec_process().
src/audio/webrtc_aec/webrtc_aec.cmake (comment):
- Added the missing AECm dependencies (aecm_core_c.c, ring_buffer.c, delay_estimator.c, delay_estimator_wrapper.c, randomization_functions.c, spl_init.c, spl_sqrt.c, spl_sqrt_floor.c) and passed -DWEBRTC_ARCH_32_BITS= -DWEBRTC_ARCH_LITTLE_ENDIAN= -Dmalloc=webrtc_malloc -Dcalloc=webrtc_calloc -Drealloc=webrtc_realloc -Dfree=webrtc_free in CFLAGS.
src/audio/webrtc_agc/webrtc_agc-agc.c (comment):
- Primed out_fifo with one 10 ms frame of silence (cd->out_fifo_count = cd->frame_samples) in webrtc_agc_real_prepare() and webrtc_agc_real_reset(), compacted out_fifo with memmove on partial drains, and zero-filled underflow remainder instead of replaying raw src samples. Also fixed DEFINITIONS -> CFLAGS in src/audio/webrtc_agc/llext/CMakeLists.txt.
src/audio/webrtc_ns/webrtc_ns-shims.c & CMakeLists.txt (comment):
- Annotated active_ns_mod (and active_vad_mod in webrtc_vad-shims.c) with APP_SYSUSER_BSS when !defined(LLEXT_MODULE), mapped -Dmalloc=webrtc_ns_malloc;-Dcalloc=webrtc_ns_calloc;-Dfree=webrtc_ns_free in src/audio/webrtc_ns/CMakeLists.txt and webrtc_ns.cmake, renamed the LLEXT static archive to libwebrtc_ns_core.a to avoid colliding with the webrtc_ns custom target, and removed webrtc_ns_set_active_mod(mod) from webrtc_ns_process().
src/audio/webrtc_ns2/llext/CMakeLists.txt & webrtc_ns2-shims.c (comment):
- Added src/audio/webrtc_ns2/webrtc_ns2-shims.c providing webrtc_ns2_set_active_mod (with APP_SYSUSER_BSS for built-in), allocator shims (malloc, calloc, free), string helpers (memset, memcpy, memmove), and freestanding math functions (sin, cos, sinf, cosf, sqrt, sqrtf, log10, log10f, floor, floorf, ceilf, fabs, fabsf, exp, expf, exp2f); wired it into CMakeLists.txt, llext/CMakeLists.txt, and webrtc_ns2.cmake (with -DFLOAT_APPROX).
src/audio/webrtc_vad/CMakeLists.txt (comment):
- Updated the duplicate-symbol guards to check CONFIG_COMP_WEBRTC_AEC STREQUAL "y" and CONFIG_COMP_WEBRTC_AGC STREQUAL "y" so built-in webrtc_vad retains the SPL resampling/division sources when webrtc_aec or webrtc_agc is built as =m.
tools/topology/topology2/development/cavs-nocodec-webrtc-ns.conf (comment):
- Configured duplex SSP0 (bclk_freq 1024000, fsync_freq 16000), playback host-copier, playback dai-copier, and pcm_caps.1 consistently at 16000 Hz.
tools/topology/topology2/include/components/webrtc-agc.conf (comment):
- Replaced the unsupported 48 kHz formats with supported 16 kHz and 8 kHz S16/S32 formats.
tools/topology/topology2/include/pipelines/cavs/webrtc-aec-capture.conf (comment):
- Added webrtc-aec."1" {} inside Object.Widget so the webrtc-aec.1 widget referenced by Object.Base.route is instantiated.
src/audio/webrtc_aec/Kconfig (comment):
- Moved config WEBRTC_AEC_PERF_TELEMETRY outside choice WEBRTC_AEC_BACKEND ... endchoice so it remains an independent boolean option.
src/audio/webrtc_hpf/webrtc_hpf-filter.c (comment):
- Added kFilterCoefficients32kHz and kFilterCoefficients48kHz (~100 Hz 2nd-order Butterworth Q12 coefficients) and selected them in webrtc_hpf_real_prepare() for 32 kHz and 48 kHz streams.

Add official SOF fork projects for WebRTC audio processing: - libfvad: pure-C BSD-3 extraction of WebRTC GMM VAD for webrtc_vad (thesofproject/libfvad at sof-hifi) - webrtc-apm: WebRTC audio processing with AEC3 C++17 core, AECm, NS, and AGC (thesofproject/webrtc-audio-processing at sof-hifi) - rnnoise: xiph/rnnoise deep-learning noise suppressor for webrtc_ns2 (thesofproject/rnnoise at sof-hifi) Signed-off-by: Liam Girdwood <liam.r.girdwood@linux.intel.com>

Raise FW_MAX_EXT_MODULE_NUM from 48 to 64 to accommodate the new WebRTC modules in LLEXT all-modular builds. Signed-off-by: Liam Girdwood <liam.r.girdwood@linux.intel.com>

Add the webrtc_vad SOF module wrapping the WebRTC Gaussian Mixture Model (GMM) Voice Activity Detector via libfvad. Module characteristics: - Single-source, single-sink PCM effect with pass-through audio - Supports 8, 16, 32, and 48 kHz sample rates - 10, 20, or 30 ms frame analysis - 4 operating modes (Quality, Low Bitrate, Aggressive, Very Aggressive) - Emits NOTIFIER_ID_VAD events when speech is detected - Backends: real (libfvad static library) and pass-through stub for CI/LLEXT - LLEXT packaging supported UUID: c790b11d-5d14-e54e-be36ba4ad732cc14 Signed-off-by: Liam Girdwood <liam.r.girdwood@linux.intel.com>

Add the webrtc_ns SOF module wrapping the classic WebRTC Noise Suppression algorithm (spectral Wiener filter, fixed-point) from the webrtc-audio-processing library. Module characteristics: - Single-source, single-sink PCM effect - Fixed-point implementation; no FPU required - Operates at 8 or 16 kHz on 10 ms frames - Supports S16_LE and S32_LE, mono or stereo - Four suppression levels: Mild/Medium/Aggressive/VeryAggressive - Two backends: real (WebRTC NS) and pass-through stub for CI/LLEXT - LLEXT packaging supported UUID: 0fc8faef-945f-004b-8d5f315047f1136a Signed-off-by: Liam Girdwood <liam.r.girdwood@linux.intel.com>

…/ AEC3) Add the webrtc_aec SOF module wrapping WebRTC Acoustic Echo Cancellation. Module characteristics: - Dual-source (pin 0 = mic capture, pin 1 = playback echo reference) - Single-sink: echo-cancelled microphone output - Three selectable backends: * AECm: lightweight fixed-point Q15 for embedded low-power DSP * Classic Float AEC: single-precision floating-point with soft-float lowering * Modern AEC3 (C++17): stereo 2x2 MIMO pipeline with native hardware FPU acceleration on Panther Lake (ACE 3.0) - Configurable filter length, suppression levels, and sample rates (8/16/32/48 kHz) - Pass-through stub backend for CI validation - LLEXT packaging supported UUID: e5da7b5b-133a-ba46-b517651d6300bb83 Signed-off-by: Liam Girdwood <liam.r.girdwood@linux.intel.com>

Add the webrtc_ns2 SOF module wrapping the RNNoise deep-learning noise suppressor (GRU recurrent neural network) from xiph/rnnoise. Module characteristics: - Single-source, single-sink PCM effect - Operates at 48000 Hz on 10 ms (480-sample) frames - Supports S16_LE and S32_LE up to 2 channels - Integrated speech probability detector emitting NOTIFIER_ID_VAD - Statically linked GRU weights in .rodata - Pass-through stub backend for CI validation - LLEXT packaging supported UUID: eacfacdc-2a87-c942-97e894c917a740db Signed-off-by: Liam Girdwood <liam.r.girdwood@linux.intel.com>

Add the webrtc_agc SOF module wrapping the WebRTC Digital Automatic Gain Control (AGC) and limiter implementation from webrtc-audio-processing. Module characteristics: - Single-source, single-sink PCM effect - Fixed-point digital AGC (digital_agc.c) - Target level and compression gain control via IPC4 / Kconfig - Digital limiter preventing hard clipping - Pass-through stub backend for CI validation - LLEXT packaging supported UUID: 0af7b529-9dc7-4bee-a8b159e8fbe414f3 Signed-off-by: Liam Girdwood <liam.r.girdwood@linux.intel.com>

Add the webrtc_hpf SOF module wrapping the WebRTC cascaded 2nd-order biquad IIR high-pass filter (<80/100 Hz cutoff). Module characteristics: - Eliminates DC offset and rejects speaker rumble / mechanical vibration - Fixed-point Q12/Q13 arithmetic with 32-bit accumulators - Supports S16 and S32 formats across up to 4 channels - Measured <0.35% CPU load on Panther Lake DSP - Pass-through stub backend for CI validation - LLEXT packaging supported UUID: 6dfb1bea-d858-4ef7-92abcf5b4ee48159 Signed-off-by: Liam Girdwood <liam.r.girdwood@linux.intel.com>

Add Topology 2.0 component widgets, capture pipeline templates, and development topologies for WebRTC audio modules: Component widgets (include/components/): - webrtc-vad.conf: pass-through VAD (S16/S32, any sample rate) - webrtc-ns.conf: spectral noise suppressor (8/16 kHz, S16/S32) - webrtc-aec.conf: AECm echo canceller (dual input pins: mic + echo ref) - webrtc-ns2.conf: RNNoise GRU neural noise suppressor (48 kHz locked) - webrtc-agc.conf: WebRTC digital automatic gain control & limiter - webrtc-hpf.conf: cascaded 2nd-order biquad IIR high-pass filter Capture pipeline templates (include/pipelines/cavs/): - webrtc-ns-capture.conf: DAI copier -> VAD -> NS -> host copier - webrtc-aec-capture.conf: 2-pin capture copier -> AEC -> host copier - webrtc-ns2-capture.conf: RNNoise neural suppressor -> host copier Development topologies (development/): - cavs-nocodec-webrtc-ns.conf - cavs-nocodec-webrtc-aec.conf - cavs-nocodec-webrtc-ns2.conf Registered development targets in tplg-targets.cmake for TGL nocodec SSP. Signed-off-by: Liam Girdwood <liam.r.girdwood@linux.intel.com>

Copilot AI balanced review requested due to automatic review settings October 15, 2026 15:57

lgirdwood requested review from dbaluta, jsarha, kv2019i, lbetlej, mmaka1, plbossart and ranj063 as code owners October 15, 2026 15:57

Copilot started reviewing on behalf of lgirdwood October 15, 2026 16:00

View session

Copilot AI reviewed Oct 15, 2026

View reviewed changes

lgirdwood force-pushed the topic/webrtc-modules branch from 661613d to dc86942 Compare October 15, 2026 16:42

lgirdwood requested a balanced review from Copilot October 21, 2026 17:04

Copilot started reviewing on behalf of lgirdwood October 21, 2026 17:06

View session

Copilot AI reviewed Oct 21, 2026

View reviewed changes

lgirdwood force-pushed the topic/webrtc-modules branch from dc86942 to dcfe5ac Compare October 21, 2026 18:52

lrgirdwo added 9 commits October 21, 2026 22:02

rimage: raise maximum number of modules

2c0b091

Raise FW_MAX_EXT_MODULE_NUM from 48 to 64 to accommodate the new WebRTC modules in LLEXT all-modular builds. Signed-off-by: Liam Girdwood <liam.r.girdwood@linux.intel.com>

lgirdwood force-pushed the topic/webrtc-modules branch from dcfe5ac to a3f89dd Compare October 21, 2026 21:04

kv2019i approved these changes Oct 22, 2026

View reviewed changes

lgirdwood merged commit 590d34f into thesofproject:main Oct 22, 2026
46 checks passed

lgirdwood deleted the topic/webrtc-modules branch October 22, 2026 12:02

Conversation

lgirdwood commented Oct 15, 2026

Overview

Audio Processing Modules in this PR

1. webrtc_vad (Voice Activity Detection)

2. webrtc_ns (Classic Spectral Noise Suppression)

3. webrtc_aec (Acoustic Echo Cancellation)

4. webrtc_ns2 (RNNoise Deep Learning Noise Suppression)

5. webrtc_agc (Digital Automatic Gain Control)

6. webrtc_hpf (Cascaded Biquad High-Pass Filter)

West Manifest Integration (west.yml)

Topology2 & Pipelines

Verification & Build Results

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

🟡 Changes recommended

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

lgirdwood commented Oct 15, 2026

Summary of Changes

1. Sink/Source API Conformance (per #11297)

2. Detailed Review Fixes

3. Verification

Uh oh!

intel-sofci commented Oct 20, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

PR 11314: test results

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

🟡 Changes recommended

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

lgirdwood commented Oct 21, 2026

CI Fixes

Review Comments Addressed

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

1. `webrtc_vad` (Voice Activity Detection)

2. `webrtc_ns` (Classic Spectral Noise Suppression)

3. `webrtc_aec` (Acoustic Echo Cancellation)

4. `webrtc_ns2` (RNNoise Deep Learning Noise Suppression)

5. `webrtc_agc` (Digital Automatic Gain Control)

6. `webrtc_hpf` (Cascaded Biquad High-Pass Filter)

West Manifest Integration (`west.yml`)

intel-sofci commented Oct 20, 2026 •

edited

Loading