Skip to content

vpage: fix free tracking and region sizing, enable vpages on NVL-S - #11303

Open
lgirdwood wants to merge 4 commits into
thesofproject:mainfrom
lgirdwood:fix/vpage-region-underflow
Open

lgirdwood wants to merge 4 commits into
thesofproject:mainfrom
lgirdwood:fix/vpage-region-underflow

Conversation

@lgirdwood

@lgirdwood lgirdwood commented Oct 13, 2026 •

Copy link
Copy Markdown
Member

Fixes the NVL-S vpage_alloc failed -22 ... total 1048294 free 1048294 seen in the daily-nvl-s CI run, plus related vpage bugs. Respun as separate commits:

  1. vpage: fix element tracking when freeing a non-last allocation - velems[num_elems_in_use] (one past the last element) was moved over the released slot, orphaning the last element and leaking its pages. Adds an out-of-order free boot test.
  2. vpage: release pages mapped by a failed multi-page map - the mm driver leaves already mapped pages (and their physical pages) mapped on failure.
  3. vpage: size the vpage region without underflow, support small L2 - the region size was an unsigned subtraction that underflows on NVL-S (2MB L2) into a ~4GB region whose end wraps, so every map failed with -EINVAL. When the virtual heap and library regions do not fit in L2 (they are only virtual address space), place the vpage region after the virtual heap, end it at the library base (or end of virtual memory) and cap it by physical L2 left after the image. If nothing is left, vpages are disabled and alloc/free return -ENODEV.
  4. DNM: west.yml - pulls in drivers: mm: intel_adsp: tlb: map uncached alias in the MMU zephyrproject-rtos/zephyr#122801 (one commit on top of the current pin) so CI can test. Needed because with working vpages beyond physical L2, coherent (uncached alias) accesses fault without the Zephyr fix (EXCCAUSE 29 seen earlier). Drop and bump the pin once the Zephyr PR merges.

Built for nvl-s (compile of vpage.c only with the new commits); not yet run on hardware - needs the CI run with the west.yml change. Draft.

🤖 Generated with Claude Code

@lgirdwood

Copy link
Copy Markdown
Member Author

@harajend fyi - pls let me know if this helps.

@lgirdwood
lgirdwood marked this pull request as ready for review October 13, 2026 17:10
Copilot AI balanced review requested due to automatic review settings October 13, 2026 17:10

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Region boundary handling still has edge-case bugs, and the new regression test does not detect the original leak.

3 open findings
What changed in this PR

Fixes vpage sizing and allocation tracking, including NVL-S support.

Changes:

  • Prevents virtual-region size underflow.
  • Cleans up partially mapped pages and fixes out-of-order frees.
  • Adds an out-of-order free boot test.
File Description
zephyr/​lib/​vpage.c Updates region sizing, cleanup, and allocation tracking.
zephyr/​test/​vpage.c Adds an out-of-order free regression test.

🧠 Review effort: Balanced


💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread zephyr/lib/vpage.c Outdated
Comment thread zephyr/lib/vpage.c
Comment thread zephyr/test/vpage.c Outdated
@lgirdwood
lgirdwood force-pushed the fix/vpage-region-underflow branch from 6115723 to b6f127a Compare October 13, 2026 18:17
@intel-sofci

intel-sofci commented Oct 13, 2026 •

Copy link
Copy Markdown

PR 11303: test results

Run date: 2026-10-21 17:14 UTC

Tested commit: 81400fb02b5e15afd8849bd03c8941913067dde6

mtl pass rate lnl pass rate ptl pass rate wcl pass rate nvl pass rate

@harajend

harajend commented Oct 15, 2026 •

Copy link
Copy Markdown

@harajend fyi - pls let me know if this helps.

@lgirdwood

Tested commit b6f127a46 via intel-innersource CI run
38693960124
on both NVL-S DUTs (ba-nvls-rvp-01, ba-nvls-rvp-02).

This issue is not fixed. All playback/capture tests still fail on both
boards with a DSP firmware crash (EXCCAUSE 29 store prohibited) triggered
on the first pipeline-create IPC, halting DSP core 0 with no recovery.

Detailed analysis:

CI failure: DSP firmware crash on NVL-S (regression vs. previously-reported bug)

Tested commit b6f127a46 via intel-innersource CI run
38693960124
on both NVL-S DUTs (ba-nvls-rvp-01 SoundWire, ba-nvls-rvp-02 HDA).

Result: All playback/capture-dependent test cases fail on both boards with an
identical, reproducible DSP crash triggered by the very first pipeline-create IPC:

<inf> [core 0] vregion.vregion_create: new at base 0xa023a000 size 0x12000 pages 18 metadata at 0x400fdd80
<inf> [core 0] ipc.ipc_cmd: rx: 0x40000004|0x15
<err> [core 0] os.print_fatal_exception:  ** FATAL EXCEPTION
<err> [core 0] os.print_fatal_exception:  ** CPU 0 EXCCAUSE 29 (store prohibited)
<err> [core 0] os.print_fatal_exception:  **  PC 0xa007ef38 VADDR 0xa0031050
...
<err> [core 0] os.z_fatal_error: >>> ZEPHYR FATAL ERROR 0: CPU exception on CPU 0
<err> [core 0] zephyr.k_sys_fatal_error_handler: Halting system

Identical PC/backtrace/address on both boards → deterministic firmware bug, not
hardware/board-specific noise. DSP core 0 halts and never recovers (unlike a
suspend/resume cycle, which does not bring it back here).

This looks like a regression against a previously-filed bug, not a fix

We have an existing internal bug report (BUG_REPORT_NVL-S_vpage_alloc.md,
CI run 38194969857, firmware beeac4752) describing a failure at the
exact same virtual region 0xa023a000:

<err> [core 0] vpage.vpages_alloc_and_map: error: failed to map virtual region 0xa023a000 ..., error -22
<err> [core 0] vpage.vpage_alloc: vpage_alloc failed -22 for N pages, total 1048294 free 1048294
<err> [core 0] pipe.pipeline_new: Failed to create pipeline vregion of X bytes, using heap

In that prior report, the failure was graceful: vpage_alloc() returned
-22 (EINVAL), the firmware fell back to heap allocation, and the DSP stayed
alive (recoverable via a suspend/resume D0→D3→D0 cycle).

With this PR, the same 0xa023a000 region now produces a hard CPU
exception (store prohibited) and a full core halt
, with no recovery path
short of a host reboot.

Hypothesis: this PR enables actually mapping/using the vpage region
instead of returning EINVAL, but something writes into 0xa023a000 /
VADDR 0xa0031050 without the mapping having succeeded (or before it's
committed), so the previously-tolerated error case is now a hard store fault
instead of a handled error return.

Suggested next step

Please check the map-then-use ordering / error handling around
vregion_create() / vpages_alloc_and_map() for the small-L2 (NVL-S) path —
specifically whether the metadata write (metadata at 0x400fdd80) or first
use of the new vregion happens before/without confirming the map succeeded.

@kv2019i kv2019i left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like a good and important fix, but the commits are mixing multiple issues and as @harajend is reporting functional issues as well, marking it with -1 for now.

Comment thread zephyr/lib/vpage.c
* Keep the elements dense: move the last in-use element (num_elems_in_use - 1)
* over the released one and clear the last slot.
*/
vpage_ctx.num_elems_in_use--;

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should be in a separate commit.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done, the free-tracking fix is now its own commit (with the boot test).

Comment thread zephyr/lib/vpage.c
if (!sys_mm_drv_page_phys_get(va, NULL))
sys_mm_drv_unmap_page(va);
}

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should be a separate commit as well (or even a PR).

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done, the rollback of partially mapped pages is now a separate commit.

Comment thread zephyr/lib/vpage.c
end = L2_VIRTUAL_SRAM_BASE + L2_VIRTUAL_SRAM_SIZE;
if (IS_ENABLED(CONFIG_LIBRARY_MANAGER) && CONFIG_LIBRARY_BASE_ADDRESS < end)
end = CONFIG_LIBRARY_BASE_ADDRESS;
}

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is rewriting the change in first commit? Why not make the correct fix in the first place?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed. The commits are restructured: the sizing logic is now written once in its final form in a single commit instead of rewriting an earlier one.

@kv2019i

kv2019i commented Oct 20, 2026

Copy link
Copy Markdown
Collaborator

This PR fails without Zephyr zephyrproject-rtos/zephyr#122801 .

@lgirdwood I pushed a minimal/subset of this PR as #11323 -> just fixes the overflow and limits SOF to what is safe to do with current version of Zephyr drivers.

@lgirdwood

Copy link
Copy Markdown
Member Author

This PR fails without Zephyr zephyrproject-rtos/zephyr#122801 .

@lgirdwood I pushed a minimal/subset of this PR as #11323 -> just fixes the overflow and limits SOF to what is safe to do with current version of Zephyr drivers.

Will get Claude to respin once Zephyr part merged.

lrgirdwo and others added 4 commits October 21, 2026 17:40
vpages_free_and_unmap() moved velems[num_elems_in_use], which is one past
the last in-use element, over the released slot. Freeing a non-last
allocation therefore left the last element outside the in-use count, so
the next allocation overwrote its tracking entry and that allocation could
never be freed, leaking its virtual and physical pages.

Decrement the count first so the real last element is moved. Add a boot
test that frees allocations out of order and checks that the pages can be
reused.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Signed-off-by: Liam Girdwood <liam.r.girdwood@linux.intel.com>
The mm driver maps page by page and leaves the pages already mapped, and
their physical pages, in place when a later page fails, e.g. when physical
L2 is exhausted. vpage then freed the virtual blocks without unmapping
them, leaking physical L2.

Unmap any page of the failed range that is still mapped before freeing the
virtual blocks.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Signed-off-by: Liam Girdwood <liam.r.girdwood@linux.intel.com>
vpage_init() computed the region size as an unsigned subtraction without
checking that the image, virtual heap and library regions leave any L2
for vpages. On NVL-S (2MB L2) this underflows into a ~4GB region whose
end wraps the 32-bit address space, so sys_mm_drv_map_region_safe()
rejects every mapping with -EINVAL and every vpage_alloc() fails.

The virtual heap and library regions are only virtual address space
backed on demand, so they need not fit in L2 together with the vpages.
When they fit, keep the existing layout. Otherwise place the vpage
region after the virtual heap, end it at the library base address (or
the end of virtual memory), and cap it by the physical L2 left after the
image. If no page is left, log an error and leave vpages disabled;
vpage_alloc() and vpage_free() then fail with -ENODEV.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Signed-off-by: Liam Girdwood <liam.r.girdwood@linux.intel.com>
Temporary: pull in "drivers: mm: intel_adsp: tlb: map uncached alias in
the MMU" (zephyrproject-rtos/zephyr#122801), one commit on top of the
currently pinned beeac475263e. Without it, coherent (uncached) accesses
to vpages beyond the end of physical L2 fault on NVL-S. Drop this commit
and bump the zephyr pin once the Zephyr PR is merged.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Signed-off-by: Liam Girdwood <liam.r.girdwood@linux.intel.com>
@lgirdwood lgirdwood changed the title vpage: fix region size underflow, free tracking, and enable vpages on NVL-S vpage: fix free tracking and region sizing, enable vpages on NVL-S Oct 21, 2026
@lgirdwood
lgirdwood force-pushed the fix/vpage-region-underflow branch from b6f127a to 81400fb Compare October 21, 2026 16:40
@lgirdwood

Copy link
Copy Markdown
Member Author

Respun onto current main (commits split per review) and added a DNM west.yml commit pinning zephyr to kv2019i's #122801 head (7280c15b8cff, one commit on top of the current pin) so CI picks up the uncached-alias fix via west. Will drop it and bump the pin when the Zephyr PR merges. Related: #11323 is the minimal subset.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants