Make dparse() safe to call concurrently: thread-local first path (rxode2#1427)

[mattfidler]

Fixes the dparser side of nlmixr2/rxode2#1427, the root cause of the CRAN r-devel clang segfault in nlmixr2/nlmixr2est#1154.

Problem

new_VecZNode() in src/parse.c handed the first reduction path to every parse as the single process-wide static VecZNode path1, and grew it with vec_add(). rxode2 5.1.7 runs dparse() on several statements at once, one D_Parser per thread (its rxOptExpr() CSE pass and its symengine translation batch). Those parses share and reallocate path1, which corrupts it. ThreadSanitizer reports new_VecZNode() (parse.c:1332) racing free_paths() (parse.c:1365) on path1, and clang builds segfault (address 0x4, "tracked pnodes").

Fix

The first path is now a local VecZNode in reduce_one(), passed through build_paths() → build_paths_internal() → new_VecZNode() and compared in free_paths(). That keeps the no-malloc fast path, shares nothing between threads, and needs no _Thread_local (dparser still allows R >= 3.3 toolchains). No path pointer outlives reduce_one(): make_PNode() and PNode_equal() only read the path while building children.

• Version bumped to 1.3.2, matching NEWS. rxode2 keys its parallel parsing on dparser >= 1.3.2.
• NEWS states what concurrent callers must do:
  • resolve the dparser.h entry points on the main thread first;
  • avoid inputs that hit syntax-error, ambiguity or R-callback paths, which still call the R API.

Regression test / reprex

tests/testthat/test-concurrent-dparse.R builds a small grammar with long rules and a pthreads driver (concurrent_parser.c). It parses a 200-statement input from several threads at once, one parser per thread, after one serial parse on the main thread. The test is skipped on CRAN and on Windows. If the driver fails to build, the test fails instead of skipping.

dparser build	concurrent test
CRAN 1.3.1-13, clang 21	segfault
this branch, clang 21	pass
CRAN 1.3.1-13, clang 21 + ThreadSanitizer	36 data-race reports, all on path1
this branch, clang 21 + ThreadSanitizer	0 data-race reports
CRAN 1.3.1-13, gcc 13	passes (even at 16 threads x 500 parses)
this branch, gcc 13	pass

The race went unseen because gcc's timing hides it, while TSan shows it regardless of compiler. clang inlines build_paths() into reduce_one() and writes path1 directly, so every run crashes. On this repo's CI the test catches the bug on the clang (macOS) build and is a passing guard on gcc runners.

The dparser test suite passes: 106 tests, 0 failures, 4 skipped.

Reviews

• antigravity (gemini-3.1-pro-high), triaged:
  • the test driver now joins started threads before Rf_error();
  • _Thread_local portability was resolved by the stack-owned design;
  • the lazy R_GetCCallable() concern does not apply to rxode2, which fills a pointer table at load.
• codex (gpt, codex exec -s read-only), round 1:
  • TLS portability: fixed, design changed as above;
  • lazy-resolve and R-API-on-error limits: pre-existing, now documented in NEWS;
  • test skipped on build failure: fixed, it now fails.
• codex round 2: NO ISSUES.

Companion rxode2 PR: nlmixr2/rxode2#1428 (gates its parallel parsing on this version, safe with either dparser).

🤖 Generated with Claude Code

[mattfidler]

Codex review (codex exec -s read-only, codex-cli 0.155.1)

Round 1 (at 2aa2a5c) — 4 findings

1. src/parse.c:1323 — _Thread_local is an undeclared compiler requirement. dparser allows R (>= 3.3), and GCC only added _Thread_local in 4.9. Suggested: make the first vector automatic storage owned by reduce_one().
   Fixed in 9a3c31f. path1 is now a local in reduce_one(), threaded through build_paths()/build_paths_internal()/new_VecZNode() and compared in free_paths(). No TLS, still no malloc. I checked that make_PNode()/PNode_equal() only read the path during the call, so nothing outlives reduce_one().

2. src/dparser.h:22 (also :28, :34, :46) — cold wrapper calls race (pre-existing). Concurrent first calls write the same static fun pointer and call R_GetCCallable() from workers.
   Documented in NEWS: callers must resolve the entry points on the main thread first. The test warms up on the main thread. rxode2 is unaffected because it fills a dparserPtr.h pointer table at load.

3. src/parse.c:1709 — default diagnostics call the R API (pre-existing). Syntax errors with recovery enabled reach Rprintf(), and unresolved ambiguities reach Rf_error().
   Documented in NEWS: only error-free parses without R callbacks are safe off the main thread.

4. tests/testthat/test-concurrent-dparse.R:34 — a build failure became a skip, which silently drops coverage.
   Fixed in 9a3c31f: the test now stop()s with the R CMD SHLIB output.

Codex also confirmed it found no other shared scratch state mutated by independent parses in parse.c, scan.c, util.c or dsymtab.c.

Round 2 (at 9a3c31f)

NO ISSUES.

Verification after the fixes

• clang 21 + ThreadSanitizer, concurrent driver: 0 data-race reports (stock 1.3.1-13: 36, all on path1).
• Concurrent test: stock clang 21 build segfaults; this branch passes on gcc 13 and clang 21.
• Full dparser suite: 106 tests, 0 failures, 4 skipped.
• rxode2's CSE and symengine-batch reproducers pass at 2 and 8 threads against this build.