Skip to content

llext: make large C++ relocatable modules loadable and faster to load - #11316

Open
lgirdwood wants to merge 3 commits into
thesofproject:mainfrom
lgirdwood:llext-relocatable-merge
Open

lgirdwood wants to merge 3 commits into
thesofproject:mainfrom
lgirdwood:llext-relocatable-merge

Conversation

@lgirdwood

@lgirdwood lgirdwood commented Oct 15, 2026 •

Copy link
Copy Markdown
Member

Large C++ LLEXT modules (e.g. anything built with TensorFlow Lite Micro) can't be loaded with CONFIG_LLEXT_TYPE_ELF_RELOCATABLE. When they can be loaded, llext_load() is slow. This PR fixes both problems in llext_link_helper.py. Nothing changes in the firmware or the loader.

Problem

The partial (-r) link keeps every C++ COMDAT group (inline functions, template instances, vtables) as its own section. A wake-word module built with TFLM ends up with 5721 sections and 1273 relocation sections, and then:

  • every executable section except .text is linked at address 0, i.e. as a DRAM section, so 1639 of its 1640 code sections, including the hot TFLM kernels, would run from IMR
  • sections pulled in from libraries at the final link aren't in the object the helper inspects, so they get no address and also stay at 0 (vtables, rodata)
  • the LLEXT section map alone needs 45KiB, so llext_load() fails with -ENOMEM on the default 8KiB LLEXT heap; with a bigger heap, llext_manager_load_module() rejects the module because .bss isn't adjacent to .data any more

Changes

  1. Align the first section of each region. The Zephyr LLEXT loader merges sections into regions and pads each region's start down to the largest section alignment, but for relocatable (ET_REL) objects only the file offset gets padded, not the address. If a region's first section isn't at a file offset with that alignment, addresses in the region map to wrong file offsets. One effect: llext_export_symbols() reads the wrong names, so modules can't link against symbols other modules export. After linking, the helper now finds such regions and raises the alignment of their first section, which already sits at a page-aligned address, so objcopy places it at an aligned file offset. This only applies to -r builds. main already has such misaligned regions in 11 of 22 PTL modules, which is harmless only because none of them export symbols.
  2. Merge sections of relocatable modules. Before calculating addresses, the helper first partially links all objects and libraries into one object, using --force-group-allocation and a small linker script (scripts/llext_merge_sections.ld) that folds .text.*, .rodata.*, .data.* and .bss.* into one section each. .cold, .coldrodata and other specially named sections are untouched. If the linker rejects the option, the helper warns and falls back to the old behaviour.
  3. Drop relocations that need no run-time action. About 85% of the relocations in a partially linked Xtensa object are no-ops for the loader: R_XTENSA_NONE, R_XTENSA_ASM_EXPAND, and R_XTENSA_SLOT0_OP for PC-relative references within a section, which the linker has already resolved. The loader still reads each entry, its symbol and the instruction from the module image in IMR, at about 1.7µs per cache miss on PTL. The helper now removes them after linking, using the same check as the loader's r_xtensa_slot0_op_relocate(), so the result is identical.

Results

TFLM wake-word module on PTL (aphid), GCC (Zephyr SDK 1.0.1), first instantiation:

before after
sections 5721 (1685 run from IMR) 25 (only .cold in IMR)
relocations 14764 2162
.llext size 1028KiB 657KiB merged, 527KiB also pruned¹
llext_load() fails (-ENOMEM); ~208ms with a 64KiB heap, then mapping fails ~20ms
inference — 698k cycles (768k when built into the base firmware)

¹ including ~10KiB of libc/libgcc code that the module links in on the branch below.

The relocation pass alone went from 73ms to 21ms. Of the remaining ~20ms total, about half was building ext->sym_tab, which a Zephyr change removes (see below). Mapping and copying into SRAM adds another 14–34ms, limited by IMR read bandwidth (~15MB/s).

For C modules nothing much changes: they mostly have a handful of sections already, and the pruning only removes entries the loader ignores. I checked that volume builds into the same layout either way.

Dependencies / not in this PR

Loading C++ modules end to end on Xtensa also needs these Zephyr LLEXT fixes, on lgirdwood/zephyr llext-cpp-modules (not sent upstream yet):

  • dispatch STB_WEAK symbols
  • resolve symbols defined in the extension directly
  • weak GOT addend fix
  • handle R_XTENSA_PLT
  • resolve undefined weak symbols to 0
  • build ext->sym_tab only when it's used
  • export snprintk/vsnprintk, sys_clock_cycle_get_32 and xtensa_arch_except

These are independent of this PR. The module-specific changes (extra exports, linking libc/libgcc into the module) are on lgirdwood/sof llext-cpp-modules.

Testing

  • PTL: built with GCC, loaded and run on silicon; inference runs with stable cycle counts over repeated runs and module reloads. Model outputs weren't compared against a built-in reference.
  • The helper from this branch was run against the objects of a PTL build: for the C++ module and for volume it produces the same sections and relocations as the in-tree build.
  • xt-clang (RI-2022.10), which is when the build uses -r modules: PTL build on aphid with sof-hda-generic-ace3-4ch.tplg; EQ_IIR (linking against aux1_iir), EQ_FIR, DRC, PEAKVOL and MIXIN load and run on every PCM, same as on main. Without the alignment commit, EQ_IIR failed to link (cannot find iir_df1, IPC error 104), the failure the first CI run of this PR hit. After the change, no module in the PTL build has a misaligned region.
  • Not tested: LLVM linkers. If a linker rejects --force-group-allocation, the helper prints a warning and falls back to the old behaviour.

🤖 Generated with Claude Code

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

Two-stage linking and direct ELF rewriting need human validation beyond the tested GCC/PTL configuration.

No open issues 🎉

What changed in this PR

Updates build-time LLEXT processing to make large C++ relocatable modules loadable and faster to load, without changing firmware or loader code.

Changes:

  • Pre-links objects and libraries, merging standard sections with a fallback if pre-linking fails.
  • Removes Xtensa relocations that require no runtime action.
File Description
scripts/​llext_merge_sections.ld Merges standard sections while preserving specially named sections.
scripts/​llext_link_helper.py Adds pre-linking, merged-section address calculation, and relocation pruning.

Review effort: Balanced

@lgirdwood
lgirdwood marked this pull request as ready for review October 21, 2026 15:54
@intel-sofci

Copy link
Copy Markdown

PR 11316: test results

Run date: 2026-10-21 16:53 UTC

Tested commit: 3b188233a1c24f16f177e213c541fe215b7056fd

mtl pass rate lnl pass rate ptl pass rate wcl pass rate nvl pass rate

lrgirdwo and others added 3 commits October 22, 2026 13:34
The Zephyr LLEXT loader merges the sections of an extension into regions
by type: code, read-only data and writable data, and pads the start of
each region down to the largest alignment of its sections. For
relocatable (ET_REL) objects it only applies that padding to the file
offset of the region, not to its address. So if the first section of a
region isn't at a file offset with that alignment, every address within
the region is translated to a file offset, which is off by the padding.

llext_export_symbols() uses that translation for the names of symbols,
which pre-located extensions export. With wrong names, other modules
can't link against them, e.g. eq_iir against the IIR functions of
aux1_iir, which fails with -ENOENT.

This happens as soon as the first read-only section isn't the most
aligned one, for example an alignment 1 string section followed by the
16 byte aligned .module section. Find such regions after linking and
raise the alignment of their first section, which is at a page aligned
address anyway, so that objcopy places it at an aligned file offset too.
Shared (ET_DYN) objects aren't affected, so only do this for relocatable
ones.

Signed-off-by: Liam Girdwood <liam.r.girdwood@linux.intel.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
With CONFIG_LLEXT_TYPE_ELF_RELOCATABLE modules are partially linked
(-r). A partial link keeps every C++ COMDAT group (inline functions,
template instances, vtables) as a separate section, so C++ modules end
up with thousands of sections: a wake-word module built with TensorFlow
Lite Micro has 5721 sections and 1273 relocation sections.

That breaks loading in several ways:
- all executable sections except .text are linked at address 0 to run
  from DRAM, so 1639 of its 1640 code sections, including the hot TFLM
  convolution kernels, would execute from IMR
- the section addresses are calculated from the module object only, so
  sections pulled in from libraries at the final link aren't assigned
  an address at all and are left at 0, also as "DRAM" sections, among
  them C++ vtables and read-only data
- the LLEXT section map alone needs 45KiB, so llext_load() fails with
  -ENOMEM with the default 8KiB LLEXT heap, and with a bigger heap
  the module can't be mapped because .bss isn't next to .data any more

First partially link all objects and libraries of a relocatable module
into one object, with --force-group-allocation to dissolve the section
groups and a linker script to merge all .text.*, .rodata.*, .data.* and
.bss.* sections into one each, the same as a final link does. Then
calculate addresses for that object. Specially named sections like
.cold and .coldrodata aren't affected. If the linker doesn't support
this, fall back to the previous behaviour with a warning.

That module goes down to 25 sections, with only the .cold section in
DRAM, and its file size from 1028KiB to 657KiB. Together with further
fixes in the Zephyr LLEXT loader and in the module build, it then loads
and runs on PTL, without any inference slowdown compared to building it
into the base firmware. C modules mostly have a handful of sections
anyway.

Signed-off-by: Liam Girdwood <liam.r.girdwood@linux.intel.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A partially linked Xtensa object keeps relocation entries which the
LLEXT loader processes but doesn't act on: R_XTENSA_NONE and
R_XTENSA_ASM_EXPAND ones, and R_XTENSA_SLOT0_OP ones for PC-relative
L32R and CALLn references within a section, which the linker has
already resolved, so the loader leaves the instruction unchanged.

These are the vast majority: 12600 of 14764 relocations in a C++ module
built with TensorFlow Lite Micro. Still for each of them the loader
reads the relocation entry, the symbol and the instruction from the
module image in IMR, where each cache miss costs about 1.7us on PTL.
That took 73ms of the 86ms llext_load() time of that module.

Remove them from the relocation sections after linking. For
R_XTENSA_SLOT0_OP this uses the same check as the loader, so the result
is identical. With this, the relocation pass of that module takes 21ms.

Signed-off-by: Liam Girdwood <liam.r.girdwood@linux.intel.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@lgirdwood
lgirdwood force-pushed the llext-relocatable-merge branch from 3b18823 to 6fdacc4 Compare October 22, 2026 12:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants