Skip to content

[kernel/mutex] fix: reject deleted mutex waiters - #11730

Open
liulangrenaaa wants to merge 1 commit into
RT-Thread:masterfrom
liulangrenaaa:20260821-fix/lifetime/mutex-delete-waiter-uaf
Open

[kernel/mutex] fix: reject deleted mutex waiters#11730
liulangrenaaa wants to merge 1 commit into
RT-Thread:masterfrom
liulangrenaaa:20260821-fix/lifetime/mutex-delete-waiter-uaf

Conversation

@liulangrenaaa

@liulangrenaaa liulangrenaaa commented Aug 21, 2026

Copy link
Copy Markdown
Contributor

A mutex waiter could continue accessing a mutex after the object had been
deleted, detached, or otherwise lost its lifetime guarantee. In particular,
the waiter still relied on the mutex after returning from rt_schedule() to
perform priority-inheritance and wait-list cleanup.

This created several lifetime races:

  • rt_mutex_delete()/rt_mutex_detach() could wake a suspended waiter and free
    or reuse the mutex before the waiter resumed.
  • A waiter could time out, become READY, but not run immediately. The mutex
    could then be deleted before the waiter resumed from rt_schedule().
  • delete/detach or rt_mutex_release() could race with an in-flight timeout
    that already owned the waiter's timer wakeup.

Move mutex-specific waiter cleanup before the waiter can resume without a
valid mutex lifetime.

Introduce rt_mutex_cleanup_waiter() to remove mutex wait-list state, update
mutex/owner priority inheritance state, and clear pending_object while the
scheduler lock is held.

The timeout path now performs this cleanup before inserting the waiter into
the ready queue. The delete/detach and release paths use
rt_sched_thread_ready() to determine whether they or the timeout path own the
wakeup. If the timeout already owns the wakeup, only the mutex-specific state
is cleaned and the timeout callback remains responsible for making the thread
READY.

After rt_schedule() returns, a waiter whose pending_object has already been
cleared can return its timeout or deletion error without dereferencing the
mutex again.

Reproduction with the original regression test applied to the parent version:

  • UP QEMU reused the deleted mutex address and filled it with 0xA5.
  • core.mutex/test_mutex_delete_waiter then reported a data abort with
    pc=0x6007a7bc and r00/r03=0xa5a5a5a5.
  • The symbolized stack was:
    rt_sched_thread_get_curr_prio() [src/scheduler_comm.c:125]
    _rt_mutex_take() [src/ipc.c:1489]
    rt_mutex_take() [src/ipc.c:1542]
    mutex_waiter_entry() [src/utest/mutex_tc.c:919]

Add regression coverage for:

  • dynamic mutex deletion followed by memory reuse;
  • static mutex detachment followed by object memory overwrite;
  • timeout -> READY -> mutex detach while the waiter has not run yet;
  • delete/detach racing with timeout wakeup ownership;
  • rt_mutex_release() skipping a timeout-owned head waiter and waking the next
    valid waiter.

Validation:

  • Parent UP QEMU:
    core.mutex/test_mutex_delete_waiter triggered a data abort.
  • Fixed UP QEMU:
    core.mutex passed, including the mutex lifetime regression tests.
  • scons -j$(nproc) --strict -C bsp/qemu-vexpress-a9 passed.

为什么提交这份PR (why to submit this PR)

修复 mutex waiter 在 mutex 生命周期结束后仍可能继续访问 mutex 的问题。

原实现中,waiter 从 rt_schedule() 恢复后,错误路径仍需要访问 mutex,
用于更新等待队列、mutex priority 以及 owner 的优先级继承状态。因此存在以下
生命周期竞态:

  1. rt_mutex_delete() / rt_mutex_detach() 唤醒 waiter 后,在 waiter
    真正恢复执行之前释放或复用 mutex,waiter 随后继续访问失效对象。

  2. waiter timeout 后已经离开 mutex suspend list 并进入 READY,但尚未得到
    CPU。此时其他线程删除 mutex,waiter 随后恢复时仍可能访问已经释放的
    mutex。

  3. delete/detach 或 rt_mutex_release() 与 timeout callback 同时竞争同一个
    waiter 的 wakeup ownership 时,如果 mutex cleanup 与 READY 状态转换没有
    正确协调,仍可能留下 stale pending_object 或错误的 PI 状态。

这些情况都可能导致 use-after-free、data abort,或者 mutex priority
inheritance 状态不一致。

你的解决方案是什么 (what is your solution)

  1. 新增 rt_mutex_cleanup_waiter(),在持有 scheduler lock 的情况下统一完成
    mutex waiter 的清理:

    • 从 mutex suspend list 中移除 waiter(需要时);
    • 更新 mutex pending priority;
    • 恢复/更新 mutex owner 的 priority inheritance 状态;
    • 清除 thread->pending_object
  2. 将 timeout 的 mutex-specific cleanup 前移到 waiter 进入 READY 之前。

    _thread_timeout() 在将线程插入 ready queue 前调用
    rt_mutex_cleanup_waiter()。因此 timeout waiter 一旦进入 READY,就已经不再
    需要访问原 mutex。

  3. 在 delete/detach 路径使用 rt_sched_thread_ready() 判断 wakeup ownership。

    • 如果 rt_sched_thread_ready() 成功,说明 delete/detach 赢得 wakeup race,
      在同一个 scheduler-lock 状态转换中完成 mutex cleanup,并设置
      RT_ERROR
    • 如果失败,说明 timeout 已经取得 timer wakeup ownership。此时
      delete/detach 只清理 mutex-specific 状态和 pending_object,仍由 timeout
      callback 负责最终将线程变为 READY。
  4. 调整 rt_mutex_release() 的 waiter 选择逻辑。

    当队首 waiter 的 timeout 已经取得 wakeup ownership 时,不再错误地把它当作
    正常 mutex recipient,而是先清理该 waiter 的 mutex 状态,再继续选择下一个
    有效 waiter。

  5. _rt_mutex_take()rt_schedule() 返回后,如果发现
    pending_object == RT_NULL 且错误为删除或 timeout 错误,则直接返回对应
    error,不再重新解引用 mutex。

  6. 补充回归测试,覆盖:

    • 动态 mutex delete + 内存复用;
    • 静态 mutex detach + 对象内存覆写;
    • waiter timeout -> READY -> 尚未运行 -> mutex detach;
    • delete/detach 与 timeout wakeup ownership 竞争;
    • mutex release 遇到 timeout-owned 队首 waiter 后继续唤醒下一个 waiter。

请提供验证的bsp和config (provide the config and bsp)

  • BSP:

    • bsp/qemu-vexpress-a9
  • .config:

    • 默认 QEMU 配置
    • 需开启 RT_USING_HEAP 以运行动态 mutex 相关测试
    • 需开启 mutex utest
  • Validation:

    • 修复前:
      core.mutex/test_mutex_delete_waiter 可触发 data abort,
      PC 位于 rt_sched_thread_get_curr_prio(),复用后的 mutex 内容为
      0xa5a5a5a5
    • 修复后:
      core.mutex 通过,包括 delete/detach、timeout READY lifetime race 和
      timeout/release ownership 相关回归测试。
    • 编译:
      scons -j$(nproc) --strict -C bsp/qemu-vexpress-a9 通过。
  • action:

当前拉取/合并请求的状态 Intent for your PR

必须选择一项 Choose one (Mandatory):

  • 本拉取/合并请求是一个草稿版本 This PR is for a code-review and is intended to get feedback
  • 本拉取/合并请求是一个成熟版本 This PR is mature, and ready to be integrated into the repo

代码质量 Code Quality:

我在这个拉取/合并请求中已经考虑了 As part of this pull request, I've considered the following:

  • 已经仔细查看过代码改动的对比 Already check the difference between PR and old code
  • 代码风格正确,包括缩进空格,命名及其他风格 Style guide is adhered to, including spacing, naming and other styles
  • 没有垃圾代码,代码尽量精简,不包含#if 0代码,不包含已经被注释了的代码 All redundant code is removed and cleaned up
  • 所有变更均有原因及合理的,并且不会影响到其他软件组件代码或BSP All modifications are justified and not affect other components or BSP
  • 对难懂代码均提供对应的注释 I've commented appropriately where code is tricky
  • 代码是高质量的 Code in this PR is of high quality
  • 已经使用clang-format 源码格式化工具确保格式符合RT-Thread代码规范 This PR has been formatted with clang-format and complies with RT-Thread code specification
  • 如果是新增bsp, 已经添加ci检查到.github/ALL_BSP_COMPILE.json 详细请参考链接BSP自查

@github-actions

Copy link
Copy Markdown

👋 感谢您对 RT-Thread 的贡献!Thank you for your contribution to RT-Thread!

为确保代码符合 RT-Thread 的编码规范,请在你的仓库中执行以下步骤运行代码格式化工作流(如果格式化CI运行失败)。
To ensure your code complies with RT-Thread's coding style, please run the code formatting workflow by following the steps below (If the formatting of CI fails to run).


🛠 操作步骤 | Steps

  1. 前往 Actions 页面 | Go to the Actions page
    点击进入工作流 → | Click to open workflow →

  2. 点击 Run workflow | Click Run workflow

  • Use workflow from 保持默认分支(通常为 master
    Keep the default branch (usually master) in Use workflow from
  • branch 输入框填写 PR 分支 20260821-fix/lifetime/mutex-delete-waiter-uaf
    Enter PR branch 20260821-fix/lifetime/mutex-delete-waiter-uaf in the branch field
  • 设置需排除的文件/目录(目录请以"/"结尾)
    Set files/directories to exclude (directories should end with "/")
  1. 等待工作流完成 | Wait for the workflow to complete
    格式化后的代码将作为独立提交推送至你的分支。
    The formatting changes will be pushed to your branch as a separate commit.

完成后,提交将自动更新至 20260821-fix/lifetime/mutex-delete-waiter-uaf 分支,关联的 Pull Request 也会同步更新。
Once completed, commits will be pushed to the 20260821-fix/lifetime/mutex-delete-waiter-uaf branch automatically, and the related Pull Request will be updated.

如有问题欢迎联系我们,再次感谢您的贡献!💐
If you have any questions, feel free to reach out. Thanks again for your contribution!

@github-actions github-actions Bot added the Kernel PR has src relate code label Aug 21, 2026
@github-actions

github-actions Bot commented Aug 21, 2026

Copy link
Copy Markdown

📌 Code Review Assignment

🏷️ Tag: kernel

Reviewers: @GorrayLi @ReviewSun @hamburger-os @lianux-mm @wdfk-prog @xu18838022837

Changed Files (Click to expand)
  • src/ipc.c
  • src/thread.c
  • src/utest/mutex_tc.c

📊 Current Review Status (Last Updated: 2026-08-23 01:43 CST)


📝 Review Instructions

  1. 维护者可以通过单击此处来刷新审查状态: 🔄 刷新状态
    Maintainers can refresh the review status by clicking here: 🔄 Refresh Status

  2. 确认审核通过后评论 LGTM/lgtm
    Comment LGTM/lgtm after confirming approval

  3. PR合并前需至少一位维护者确认
    PR must be confirmed by at least one maintainer before merging

ℹ️ 刷新CI状态操作需要具备仓库写入权限。
ℹ️ Refresh CI status operation requires repository Write permission.

@github-actions

Copy link
Copy Markdown

MemBrowse Memory Report

at32f415-start

  • ROM: .text +24 B (+0.0%, 65,392 B / 262,144 B, total: 25% used)

esp32-c3

  • drom0_0_seg: .eh_frame +8 B (+0.0%, 410,996 B / 8,388,576 B, total: 5% used)

gd32105r-start

  • CODE: .text +24 B (+0.0%, 69,308 B / 262,144 B, total: 26% used)

hc32f334

  • FLASH: .text +24 B (+0.0%, 99,520 B / 131,072 B, total: 76% used)

hpmicro-hpm5301evklite

  • ILM: .fast +112 B (+0.3%, 38,320 B / 131,072 B, total: 29% used)
  • XPI0: .fast +112 B (+0.1%, 125,104 B / 1,048,576 B, total: 12% used)

infineon-psoc6

  • flash: .text +24 B (+0.0%, 116,164 B / 262,144 B, total: 44% used)

loongson-ls1cdev

  • Code: .text +152 B (+0.0%, 363,400 B)

nordic-nrf51822

  • FLASH: .text +20 B (+0.0%, 52,028 B / 262,144 B, total: 20% used)

nuvoton-m487

  • CODE: .text +40 B (+0.0%, 355,972 B / 524,288 B, total: 68% used)

qemu-virt64-aarch64

  • Code: .text +128 B (+0.0%, 1,070,340 B)

raspberry-pico-rp2040

  • FLASH: .text +24 B (+0.0%, 114,128 B / 2,097,152 B, total: 5% used)

renesas-ra2l1

  • FLASH: .text +24 B (+0.0%, 98,300 B / 262,144 B, total: 37% used)

simulator

  • Code: .rela.dyn +1,296 B, .rodata +224 B, .text +964 B (+0.1%, 2,675,173 B)
  • Data: .data +1,536 B (+0.1%, 1,611,012 B)

stm32f407-rt-spark

  • CODE: .text +24 B (+0.0%, 85,788 B / 1,048,576 B, total: 8% used)

stm32l475-atk-pandora-llvm

  • ROM: .text +28 B (+0.0%, 83,456 B / 524,288 B, total: 16% used)

wch-ch32v208w-r0

  • FLASH: .text +112 B (+0.1%, 160,032 B / 491,520 B, total: 33% used)

x86

  • Code: .text +128 B (+0.1%, 199,005 B)

xuantie-e901plus

  • ISRAM: .rodata +32 B, .text +32 B (+0.1%, 46,424 B / 131,072 B, total: 35% used)
    No memory changes detected for:
  • k230
  • nxp-lpc1114

@wdfk-prog

Copy link
Copy Markdown
Contributor

我看了一下这个修复,当前 delete/detach 直接唤醒仍挂在 mutex suspend list 上的 waiter 场景应该可以覆盖,不过这里可能还有两个和 timeout 相关的生命周期竞态需要考虑。

  1. waiter 已经 timeout,但还没有真正恢复执行

复现时序大致如下:

Thread A 持有 mutex
    |
Thread B rt_mutex_take(mutex, timeout)
    |
    +--> B suspend
         pending_object = mutex
    |
timer timeout
    |
_thread_timeout(B)
    +--> B->error = -RT_ETIMEOUT
    +--> 从 mutex suspend list 删除
    +--> B 进入 READY
    |
    |  B 此时还没有得到 CPU
    |
Thread A / Thread C
    +--> rt_mutex_delete(mutex)
    +--> mutex memory free/reuse
    |
B finally resumes from rt_schedule()
    |
    +--> 当前 _rt_mutex_take() 仍继续访问 mutex
         -> potential UAF

这里的问题是:timeout 已经把 B 从 mutex->parent.suspend_thread 中移除了,因此 rt_mutex_delete() 当前新增的遍历无法再找到 B,也就无法清理它的 pending_object

而 B 从 rt_schedule() 返回以后,目前还需要重新访问 mutex 做 priority / suspend-list 等处理,所以 mutex 如果在 READY → RUNNING 这段时间被释放,就仍然存在 UAF。

建议考虑把 mutex waiter 的 timeout cleanup 前移到 waiter 变为 READY 之前,即由“将 waiter 从 SUSPEND 转为 READY 的路径”负责完成 mutex-specific cleanup:

timeout
  -> mutex PI / priority cleanup
  -> remove waiter
  -> pending_object = NULL
  -> error = -RT_ETIMEOUT
  -> READY

最理想的结果是 _rt_mutex_take()rt_schedule() 返回后,在错误路径不再需要重新解引用 mutex。

  1. clear pending_objectresume waiter 之间仍存在 timeout IRQ 窗口

当前 patch 的 delete/detach 路径大致是:

rt_sched_lock()
    |
clear waiter->pending_object
    |
rt_sched_unlock()
    |
    | <--- timer IRQ may timeout waiter here
    |
rt_susp_list_resume_all(..., RT_ERROR)

如果 timeout IRQ 恰好发生在这里,timeout 路径可能先把 waiter 从 suspend list 移除,并设置:

pending_object = NULL
error = -RT_ETIMEOUT
state = READY

随后 rt_susp_list_resume_all() 已经无法再处理这个 waiter。

waiter 恢复后,当前新增判断:

if (thread->pending_object == RT_NULL &&
    thread->error == RT_ERROR)
{
    return -RT_ERROR;
}

也不会命中,因为此时 error == -RT_ETIMEOUT,之后仍可能继续访问已经删除的 mutex。

这里建议不要先单独遍历清 pending_object,再调用 rt_susp_list_resume_all(),而是让:

stop timeout timer
+ remove from suspend list
+ clear pending_object
+ set RT_ERROR
+ READY

成为同一个 scheduler-lock 临界区内的状态转换。

类似:

rt_sched_lock(&slvl);

ret = rt_sched_thread_ready(thread);
if (ret == RT_EOK)
{
    /* delete path wins the wakeup race */
    thread->pending_object = RT_NULL;
    thread->error = RT_ERROR;
}

rt_sched_unlock(slvl);

rt_sched_thread_ready() 本身已经包含 timer stop,并且明确考虑了 timeout ISR racing 的情况,因此我觉得这里可以利用它来决定到底是 delete 路径还是 timeout 路径取得 waiter 的 wakeup ownership,而不是先修改 pending_object 再尝试唤醒。

总体上,我认为当前 PR 对“delete/detach 直接唤醒 suspended waiter”的原始 crash 是有效的,但 mutex object lifetime 的边界可能还需要覆盖:

SUSPEND -> timeout -> READY -> 尚未运行 -> mutex delete

以及 delete 与 timeout 同时竞争 waiter wakeup 的情况。建议再补一个 deterministic testcase:让 waiter timeout 后保持 READY 但暂时不能运行,此时由高优先级线程 delete/free/reuse mutex,最后再让 waiter 恢复执行。

Move mutex-specific waiter cleanup before a waiter enters the READY state.

Whichever path wins wakeup ownership—delete/detach, timeout, or normal mutex release—cleans the mutex wait list and priority-inheritance state, then drops pending_object before the waiter can resume without the mutex lifetime being guaranteed.

Return timeout and deletion errors without dereferencing a stale mutex, retry later waiters when a timeout-owned waiter is encountered, and add deterministic regression coverage for both timeout ownership states.
@liulangrenaaa
liulangrenaaa force-pushed the 20260821-fix/lifetime/mutex-delete-waiter-uaf branch from 11e636e to cbde064 Compare August 22, 2026 17:43
@liulangrenaaa

Copy link
Copy Markdown
Contributor Author

我看了一下这个修复,当前 delete/detach 直接唤醒仍挂在 mutex suspend list 上的 waiter 场景应该可以覆盖,不过这里可能还有两个和 timeout 相关的生命周期竞态需要考虑。

都改了, 也增加了新的 测试, 新版本 可以再review下

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Kernel PR has src relate code

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants