Skip to content

drivers/hv/mshv_vtl: fix fabricated sidecar VP run completions - #157

Merged
Naman Jain (namancse) merged 1 commit into
product/hcl-main/6.18from
user/smalis/port-sidecar-race
Aug 5, 2026
Merged

drivers/hv/mshv_vtl: fix fabricated sidecar VP run completions#157
Naman Jain (namancse) merged 1 commit into
product/hcl-main/6.18from
user/smalis/port-sidecar-race

Conversation

@smalis-msft

Copy link
Copy Markdown
Contributor

sidecar_scan_vps() treats cpu_status == CPU_STATUS_IDLE together with vp_state == VP_STATE_ASYNC as an async run that has completed. But a VP is equally idle before its run has been started, so that pair only means "completed" if the IDLE was observed after the start.

sidecar_ioctl_start() published the claim first:

sidecar_claim(dev, cpu, VP_STATE_ASYNC);
sidecar_start(dev, cpu);

Between those two statements the VP is idle but owned, so a concurrent scan reports a completion for a run that never happened. Userspace then consumes a command page the AP never wrote, replaying the previous intercept message against register state it has already emulated. This shows up as a mismatch between the rax in the intercept message and the VP's register state on the first IO port exit after the VP is moved back to Linux.

Claim into a new VP_STATE_ASYNC_STARTING, which the scan does not accept, and publish VP_STATE_ASYNC only once sidecar_start() has set cpu_status to CPU_STATUS_RUN. Publish with xchg() so the store is ordered against both the preceding cmpxchg() and the cpu_status load that follows; a release store would let that load be hoisted, losing the re-arm described below. Invert the load order in the scan to match, so that observing VP_STATE_ASYNC implies observing CPU_STATUS_RUN.

The AP can run and go idle while the state is still VP_STATE_ASYNC_STARTING, in which case a scan in that window skips the VP and consumes its attention. Re-arm the scan after publishing so the completion is not lost.

Auditing the surrounding completion-reporting path turned up three more defects, fixed here as well. None of them is required to close the race above, so the ASYNC_STARTING change alone is self-contained if a minimal backport is wanted.

First, move the locking out of the scan and into its callers, renaming it sidecar_scan_vps_locked(). The claim loop in
sidecar_scan_next_stopped() ran outside scan_mutex, so two readers could claim the same VP_STATE_ASYNC_STOPPED entry, report the same CPU twice and underflow num_vps_stopped, after which its zero fast path never short-circuits again. Holding the mutex across both the scan and the claim makes the counter exact and removes the need to update it atomically.

Second, wake the wait queue from whichever path marks entries stopped, after the counter has been incremented. sidecar_scan_vps_locked() consumes needs_vp_scan, but sidecar_poll() never claims the entries it marks, so a reader woken by the ISR that set needs_vp_scan could find both that flag and num_vps_stopped clear and go back to sleep with a completion pending.

Third, return the pending count from sidecar_scan_vps_locked() instead of having both callers re-read num_vps_stopped under the same lock. The early return has to report the count as well: when needs_vp_scan is clear the scan does nothing, but entries marked by an earlier scan may still be waiting to be claimed.

Clean cherry-pick of #154.

sidecar_scan_vps() treats cpu_status == CPU_STATUS_IDLE together with
vp_state == VP_STATE_ASYNC as an async run that has completed. But a VP
is equally idle before its run has been started, so that pair only means
"completed" if the IDLE was observed after the start.

sidecar_ioctl_start() published the claim first:

	sidecar_claim(dev, cpu, VP_STATE_ASYNC);
	sidecar_start(dev, cpu);

Between those two statements the VP is idle but owned, so a concurrent
scan reports a completion for a run that never happened. Userspace then
consumes a command page the AP never wrote, replaying the previous
intercept message against register state it has already emulated. This
shows up as a mismatch between the rax in the intercept message and the
VP's register state on the first IO port exit after the VP is moved back
to Linux.

Claim into a new VP_STATE_ASYNC_STARTING, which the scan does not
accept, and publish VP_STATE_ASYNC only once sidecar_start() has set
cpu_status to CPU_STATUS_RUN. Publish with xchg() so the store is
ordered against both the preceding cmpxchg() and the cpu_status load
that follows; a release store would let that load be hoisted, losing the
re-arm described below. Invert the load order in the scan to match, so
that observing VP_STATE_ASYNC implies observing CPU_STATUS_RUN.

The AP can run and go idle while the state is still
VP_STATE_ASYNC_STARTING, in which case a scan in that window skips the
VP and consumes its attention. Re-arm the scan after publishing so the
completion is not lost.

Auditing the surrounding completion-reporting path turned up three more
defects, fixed here as well. None of them is required to close the race
above, so the ASYNC_STARTING change alone is self-contained if a minimal
backport is wanted.

First, move the locking out of the scan and into its callers, renaming
it sidecar_scan_vps_locked(). The claim loop in
sidecar_scan_next_stopped() ran outside scan_mutex, so two readers could
claim the same VP_STATE_ASYNC_STOPPED entry, report the same CPU twice
and underflow num_vps_stopped, after which its zero fast path never
short-circuits again. Holding the mutex across both the scan and the
claim makes the counter exact and removes the need to update it
atomically.

Second, wake the wait queue from whichever path marks entries stopped,
after the counter has been incremented. sidecar_scan_vps_locked()
consumes needs_vp_scan, but sidecar_poll() never claims the entries it
marks, so a reader woken by the ISR that set needs_vp_scan could find
both that flag and num_vps_stopped clear and go back to sleep with a
completion pending.

Third, return the pending count from sidecar_scan_vps_locked() instead
of having both callers re-read num_vps_stopped under the same lock. The
early return has to report the count as well: when needs_vp_scan is
clear the scan does nothing, but entries marked by an earlier scan may
still be waiting to be claimed.

Signed-off-by: John Starks <jostarks@microsoft.com>
Copilot AI lite review requested due to automatic review settings August 4, 2026 17:08
@smalis-msft

Copy link
Copy Markdown
Contributor Author

Naman Jain (@namancse) Please review

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Fixes a race in the Hyper-V VTL sidecar VP completion detection path where CPU_STATUS_IDLE + VP_STATE_ASYNC could be misinterpreted as a completed async run even when the VP had not actually started running yet, leading userspace to consume stale command-page contents.

Changes:

  • Introduces VP_STATE_ASYNC_STARTING so the scan logic does not treat a claimed-but-not-started VP as a completed run.
  • Reorders/pairs state publishing and scanning (publish VP_STATE_ASYNC only after cpu_status is set to CPU_STATUS_RUN, and scan loads vp_state before cpu_status) to prevent fabricated completions.
  • Refactors completion scanning/claiming to hold scan_mutex across both scanning and claiming, adjusts wakeups, and returns the pending count from sidecar_scan_vps_locked().

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@namancse
Naman Jain (namancse) merged commit f5c38dc into product/hcl-main/6.18 Aug 5, 2026
12 checks passed
@smalis-msft
Steven Malis (smalis-msft) deleted the user/smalis/port-sidecar-race branch August 5, 2026 16:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants