Skip to content

tests: the strand drain-race case's rendezvous depends on spinning, and starves under Valgrind #826

Description

@christianparpart

tests/test_strand_race.cpp's case "StrandExecutor keeps one strand per key when a post races the drain" (line 334) synchronises its threads by spinning.

  • Four chaser threads spin on an atomic gate, yielding once every 4096 spins (around line 403).
  • The producer yields in a loop on the completed counter (around line 434).
  • Nothing blocks, so a round only advances when the scheduler happens to run the pool worker that executes that round's task.

That works on a native build with enough cores. Under Valgrind, which runs one thread at a time and by default lets a yielding thread take the lock straight back, it degrades badly:

Where Time for this one case
Locally (WSL, valgrind 3.26, gcc-debug) 1982 s
master's CI Valgrind job (run 36115939775, 64d339a) about 45 of the job's 54 minutes
#806 CI Valgrind job (e984ebe) more than 3 hours; the job was cancelled as hung

The rest of morph_tests takes about 3 minutes under Valgrind.

#806 works around it in CI (c9394a0, ci: valgrind schedules spinning threads fairly): the Valgrind step now passes --fair-sched=yes. That brings the case to 63 s (4 of 4 runs) and the whole of morph_tests to 217 s with 0 errors. The test's own rendezvous still depends on how threads get scheduled.

Follow-up (low priority): make the rendezvous not depend on spinning. For example, chasers could block on a condition variable until the gate opens, and spin only across the few instructions the race window needs. It must keep the case's measured ability to catch the pre-fix drain race under TSan, which is why it was not changed in #806.

Note: the core-cpp 0.4.0 switch replaces morph's strand code, and this test file with it. If that lands first, this issue closes with it. The replacement tests must not spin-wait either.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    area: ciSubsystem: citriage: validWell-framed; implement as written

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions