Skip to content

[Bug] nvme: kernel heap memory leak per inactive namespace in nvme_scan_device() exhausts system heap and can brick boot #11919

Open

Description

@XueDugu

Title: [Bug] nvme: kernel heap memory leak per inactive namespace in nvme_scan_device() exhausts system heap and can brick boot

RT-Thread Version
v5.2.0, v5.2.1, v5.2.2, v5.3.0 and current master (introduced by commit 945114f, 2024-11-24, "[DM/FEATURE] Support NVME (#9591)")

Affected area
Device drivers (components/drivers/nvme/nvme.c)

Hardware-BSP vendor
QEMU (qemu-virt64-aarch64 BSP, default .config already enables RT_USING_NVME + RT_NVME_PCI); any board with an NVMe controller

Architecture
ARM64 (also affects any arch using this driver)

Board and hardware details
QEMU virt machine (-M virt) with a stock nvme PCI device (QEMU 9.2.3, unmodified). No special hardware is needed: any NVMe controller whose Identify Controller nn (Number of Namespaces) field is larger than its count of active namespaces triggers the leak. Note QEMU itself always reports nn = 256 while exposing 1 namespace, which is legal per the NVMe spec, so the leak reproduces with a fully compliant device response.

Develop Toolchain
GCC (aarch64 toolchain), scons, QEMU 9.2.3

Describe the bug

nvme_scan_device() (components/drivers/nvme/nvme.c) iterates nsid = 1 .. nn using the device-controlled nn field from the Identify Controller response, and allocates a 4480-byte struct rt_nvme_device (rt_calloc) before issuing the per-namespace Identify command:

for (rt_uint32_t nsid = 1; nsid <= number_of_ns; ++nsid)
{
    struct rt_nvme_device *ndev = rt_calloc(1, sizeof(*ndev));   /* 4480 B */
    ...
    if ((err = nvme_identify(nvme, nsid, 0, id)))
    {
        goto _free_res;      /* leak: ndev not freed */
    }
    if (!id->nsze)
    {
        continue;            /* leak: ndev not freed */   <-- main path
    }
    ...

Every namespace that reports nsze == 0 (inactive/allocated-but-not-active namespaces — a perfectly legal response; e.g. QEMU returns an all-zero structure for every non-attached nsid), and every failed identify, leaks 4480 bytes (4536 bytes with the memheap header) of the kernel system heap. The leaked memory is never recovered; the reference is simply dropped.

Measured impact, full-chain on the official qemu-virt64-aarch64 BSP with the default config (free via the free msh command):

device response kernel heap available after boot
no NVMe device 66,941,784 B
nn = 1 (single active ns) 66,931,592 B (disk nvme0n1 registered, healthy)
stock QEMU (nn = 256, 1 active ns) 65,774,912 B — 1,156,872 B (≈1.1 MiB) leaked = 255 × 4536 B

So with the stock, unmodified QEMU NVMe device (a legal nn = 256 response), every boot loses ~1.8 % of a 64 MiB kernel heap. Repeated controller re-registration (hot-plug / surprise removal + rescan) accumulates the leak linearly.

With a larger device-controlled nn (malformed or malicious controller firmware, e.g. nn = 0xFFFFFF), the loop keeps allocating until the kernel heap is fully exhausted, which happens deterministically before main() can start the shell — the system never reaches the msh prompt (boot bricked):

=== OOM POINT === nsid=14758 avail=600 maxused=67108264      (64 MiB heap)
#0  nvme_scan_device (nvme=0xffff000000263b78, number_of_ns=16384) at components/drivers/nvme/nvme.c:1079
#1  rt_nvme_controller_register (nvme=0xffff000000263b78) at components/drivers/nvme/nvme.c:1216
#2  pci_nvme_probe (pdev=0xffff000000263838) at components/drivers/nvme/nvme-pci.c:95
#3  pci_probe (dev=0xffff00000000038) at components/drivers/pci/pci.c:1050

(serial console output stops permanently after hello rt-thread; no msh /> prompt is ever printed)

To reproduce

  1. Build the official qemu-virt64-aarch64 BSP (default .config, RT_USING_NVME is already on).
  2. Create a raw disk image, boot with a stock QEMU nvme device (QEMU reports nn=256 for 1 active namespace — no device model modification needed):
    dd if=/dev/zero of=/tmp/nvme.img bs=1M count=16
    qemu-system-aarch64 -M virt,gic-version=2 -cpu max -m 128M \
        -kernel rtthread.bin -serial <console> \
        -drive file=/tmp/nvme.img,if=none,id=nvme0 \
        -device nvme,drive=nvme0
    
  3. Run free in msh and compare with a boot without the nvme device: ~1.1 MiB of the kernel heap is gone after a single boot; list_blk shows the disk otherwise working normally.
  4. GDB confirmation (gdb-multiarch): break at the if (!id->nsze) continue; branch (nvme.c:1091) — it is hit once per inactive nsid (2..256 with stock QEMU), and system_heap->available_size drops by exactly 4536 bytes on each hit.

Expected behaviour
ndev must be allocated only after the namespace is known to be active (nsze != 0 and identify succeeded), or freed on the nsze == 0 / identify-error paths; nn should also not be trusted as a loop bound without an upper limit (e.g. cap by mnan or a sane maximum, and/or use the Active Namespace ID list, CNS 02h).

Actual behaviour
4480 bytes leaked per inactive namespace per controller registration; kernel heap exhaustion and an unbootable system with a large device-reported nn.

Other additional context

Activity

  1. github-actions commented on Oct 22, 2026

    @github-actions

    如需认领并处理此 Issue,请评论 /claim。

    To claim and work on this issue, comment /claim.

  2. usualtreee commented on Oct 23, 2026

    @usualtreee

    /claim

  3. usualtreee commented on Oct 23, 2026

    @usualtreee

    确认复现(带 NVMe 时 available 少 1.17MB / 1,169,600 字节),已修复验证归零,见 PR #11920

  4. github-actions commented on Oct 23, 2026

    @github-actions

    @usualtreee has claimed this issue.

  5. Yunlk commented on Oct 23, 2026

    @Yunlk
    Contributor

    在 QEMU 上复现了,并验证了最小补丁能把缺口收回。三步测量用的是同一个固件、同一套启动参数,只切换 NVMe 设备的有无。

    测量方法(与建议步骤的一处偏差)

    建议里是用 msh 的 free 取值。我这边没有可脚本化的串口输入,所以改成开机时由应用层打印一次内存信息(INIT_APP_EXPORT 里延时 300ms 后调 rt_memory_info()),效果等价,且可重复:

    c rt_memory_info(&total, &used, &max_used); rt_kprintf("[MEM] total=%u used=%u available=%u\n", total, used, total - used);

    同样用这个探针输出 nvme0n1 是否存在,用来确认设备真的被枚举。

    三组数字

    配置 used available
    无 NVMe 控制器 5,055,752 62,053,112
    有 NVMe,未修复 7,011,800 60,097,064
    有 NVMe,已修复 5,859,200 61,249,664

    推导:

    • 未修复时的总缺口 = 7,011,800 − 5,055,752 = 1,956,048 B ≈ 1.87 MB
    • 修复后的增量(NVMe 控制器 + 1 个活动命名空间的合法占用)= 5,859,200 − 5,055,752 = 803,448 B ≈ 785 KB
    • 补丁回收 = 7,011,800 − 5,859,200 = 1,152,600 B ≈ 1.10 MB
    • 1,152,600 / 4536 ≈ 254 个对象

    也就是说这个缺口里约 1.10 MB 是泄漏、约 785 KB 是正常占用。1.10 MB 与 issue 里报告的"约 1.15 MB"吻合(255×4536 = 1,156,680,差异来自分配器对齐与首个对象的边界)。

    顺带说明:总缺口 1.87 MB 不能直接当成泄漏量,如果只看"有/无设备"的差值会高估约 785 KB。

    最小补丁

    components/drivers/nvme/nvme.c,把分配挪到 nsze 判断之后,并补上注册失败路径的释放:

    `diff
    for (rt_uint32_t nsid = 1; nsid <= number_of_ns; ++nsid)
    {

    •    struct rt_nvme_device *ndev = rt_calloc(1, sizeof(*ndev));
      
    •    if (!ndev)
      
    •    {
      
    •        err = -RT_ENOMEM;
      
    •        goto _free_res;
      
    •    }
      
    •    struct rt_nvme_device *ndev;
      
         rt_memset(id, 0, sizeof(*id));
         if ((err = nvme_identify(nvme, nsid, 0, id)))
         {
             goto _free_res;
         }
      
    •    /* 不活动的命名空间会返回 nsze == 0 并被跳过,因此必须等确认它可用
      
    •     * 之后再分配设备对象;否则每个不活动的 nsid 泄漏一个
      
    •     * struct rt_nvme_device,而 nn 由控制器决定(QEMU 报 256,只暴露 1 个) */
         if (!id->nsze)
         {
             continue;
         }
      
    •    ndev = rt_calloc(1, sizeof(*ndev));
      
    •    if (!ndev)
      
    •    {
      
    •        err = -RT_ENOMEM;
      
    •        goto _free_res;
      
    •    }
      

    `

    另外 rt_hw_blk_disk_register() 失败时也是 goto _free_res,那条路径同样会漏 ndev,补丁里一并补了 rt_free(ndev)。

    这条改法相比"在 continue 前加 rt_free"的优点是:identify 失败路径自动不再需要释放(那时还没分配),只需处理注册失败一处。

    没做的部分

    • 没跑 gdb 断点那一步(在 if (!id->nsze) continue 处确认每个 nsid 命中一次、每次 -4536 字节)。上面的三组数字已经能从总量上等价证明这一点:回收量 / 4536 ≈ 254,与 nn = 256、1 个活动命名空间完全对得上。
    • 启动参数与建议略有出入:我用的是 BSP 自带的 qemu.py(内部为 -M virt,acpi=on,its=on,gic-version=3),而不是建议里的 -M virt,gic-version=2 -cpu max -m 128M。原因是手工那条命令在本机完全没有串口输出(GICv2 与 GICv3 两种都试过),而 qemu.py 的 -serial mon:stdio 组合可以正常拿到 console。这一点如果你们验证时用的是 gic-version=2,结果应当一致(本 issue 只涉及 nvme.c 的堆分配,与中断控制器无关)。

    环境:bsp/qemu-virt64-aarch64 默认 .config(RT_USING_NVME + RT_NVME_PCI),QEMU 11.0.0,xpack aarch64-none-elf-gcc 15.2.1,master 1c801efa。

    需要的话我可以把补丁提成 PR,或者把那个开机内存探针改成正式 utest(不过它依赖 NVMe 设备存在,进 CI 可能不合适)。

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions