4.0: one lock per container tree; drop use_lock

[lesnik512]

Observation

Every child container allocates its own threading.RLock. On this machine that is 107 ns of an 829 ns child build (13%), paid per request. The lock guards only singleton creation on a cache miss, which is rare after warm-up and already double-checked.

Proposal

One lock per tree, created at the root and shared by children (a child stores the reference, as it already does for the shared registries). use_lock goes away; a tree is always locked, and the cost is a pointer copy. Contention: concurrent cold misses across requests would serialise on one lock instead of per child, which is what the lock is for anyway.

Needs before it is ready

An end-to-end measurement on G6 (child build) and G15 (concurrent first-resolve, the contention scenario), plus a pass over the free-threading tests. The 107 ns figure is a microbenchmark of RLock() alone.

Out of scope

The _scope_map copy (another ~95 ns per child): walking parent_container instead moves the cost onto every cross-scope hop, and the trade is workload-dependent.

[lesnik512]

Prior measurement worth keeping here (it lived in a decision record that is being removed). Lazy allocation of the per-child lock and registries was measured and declined:

• Isolated child build: the RLock allocation is ≈195 ns per child (py3.10 at the time; 107 ns on an M2 with 3.14).
• Realistic caching request cycle (build child → resolve a REQUEST-cached resource → close, the C4/G7 shape): lazy saves ≈0.4%, because a caching child uses its lock, so laziness only defers the allocation and adds a None check on the hot path.
• Narrow no-cache child (transient/APP deps only): ≈67 ns, 3.5%, the ceiling.
• A lazily created lock must itself be created atomically, which reintroduces the creation race the lock exists to prevent.

So laziness is out; this issue's design (one lock per tree, allocated once at the root) is the one that removes the per-child cost without the race, and it is the one to measure on G6 and G15.

[lesnik512]

This was generated by AI during triage.

Measured: branch bench/478-per-tree-lock (296a7de)

Apple arm64, CPython 3.14.7 and 3.14.7t. Paired runs, alternating conditions, stash-swapping only container.py between them so the benchmark file is identical on both sides. The change shares the root's RLock with every child and leaves use_lock in place, so these numbers isolate the sharing from the API removal.

The win, end to end

Median of 4 alternating pairs, GIL build:

Scenario	Base	After	Delta
G6 child build	826.7 ns	694.6 ns	−16.0%
G6b auto-scope	878.3 ns	744.5 ns	−15.2%
G7 request cycle (÷100)	2.436 us	2.306 us	−5.4%
G13 teardown at scale	10.9 us	10.8 us	−1.0%

The issue predicted 13% on G6 from the 107 ns RLock() microbenchmark. End to end it is ~132 ns and 16%, so the microbenchmark was if anything conservative.

G7 is the number to quote, not G6. The same ~130 ns is 16% of a bare child build but 5.4% of a realistic build/resolve/close cycle. "13% of an 829 ns child build, paid per request" overstates what a request actually saves.

G15 cannot see the trade this makes

G15 races N threads on a single root container resolving APP-scoped providers. That cache item lives on the root, so target._lock is the root's lock under both designs. G15 is structurally blind to the regression and, as expected, moved only within noise.

The scenario that actually changes is concurrent cold misses in sibling children, which no benchmark covered. The branch adds test_g15b_concurrent_first_resolve_sibling_children: each thread builds its own REQUEST child and first-resolves 50 REQUEST-scoped cached providers in it, so every creation is a cold miss in a container no other thread touches. Per-container locks never contend there; one lock per tree serializes them.

Under the GIL every concurrency scenario is within noise (G14 −3.3% to +0.1%, G15 −3.2% to +6.7%, G15b −2.9% to +0.1%) — expected, since creation is already serialized.

Free-threaded 3.14t, 5 alternating pairs on G15b:

Threads	Base	After	Delta
1	514.8 us	533.2 us	+3.6%
2	8684.7 us	9097.7 us	+4.8%
4	43024.3 us	44730.4 us	+4.0%

This is a weak signal, not a measurement. Intra-condition spread swamps the deltas: base at 2 threads ranged 6769-10876 us across its own 5 runs. What makes it worth recording is that the sign is consistent across all three thread counts and reproduced across two sampling sessions. Treat "roughly 4%, direction confirmed, magnitude not" as the honest reading.

Two caveats on G15b itself:

• Its creators are trivial classes, so lock hold time is near-minimal and contention cost is understated for a graph whose singletons do real work on construction.
• Free-threaded first-resolve is already 25x the GIL cost at 2 threads (8.7 ms vs 0.35 ms), consistent with the finding in benchmarks/README.md that resolution throughput does not scale. The 4% sits on top of a path that is already the documented weak spot.

Correctness

• Full suite passes on both builds: 540 passed, 8 skipped, on 3.14 and 3.14t.
• just lint clean (ruff + ty).
• No test asserted that a child owns a distinct lock, so the free-threading pass the issue asks for turned up nothing to rewrite. test_build_child_container_propagates_use_lock_false and test_use_lock_false_yields_no_private_lock both still pass: a use_lock=False root propagates None to children by inheritance rather than by forwarding the flag.
• The change adds an 11th SLF001 suppression (parent_container._lock, on the line above the existing _scope_map one). Feeds Move three seams so the data stops leaving its owner #474.

What is now decidable

The measurement gate is satisfied. What remains is the judgment the issue names: is a 16% child build and 5.4% request cycle worth roughly 4% on free-threaded sibling contention, plus removing use_lock from the public constructor?

Two things worth separating in that answer:

1. Sharing the lock is what these numbers cover. It is not breaking on its own.
2. Removing use_lock is the 4.0 part. Once the lock is per-tree, use_lock=False means "this whole tree is unlocked", which is a different and narrower promise than today's per-container flag. If the argument stays, it needs that reinterpretation documented; if it goes, that is the migration note.

The branch is a measurement artifact, not a proposed PR: it carries the noqa, no doc changes, no migration note, and no decision on use_lock.

[lesnik512]

Prototype commit (branch deleted, commit preserved here)

A working prototype of this change exists as commit 296a7defdd96932fd322ca379089bf0945ed33f4,
formerly the branch bench/478-per-tree-lock. The branch has been deleted as cleanup; recover the
work with:

git fetch origin 296a7defdd96932fd322ca379089bf0945ed33f4
git checkout -b bench/478-per-tree-lock FETCH_HEAD

What it does

Two changes in modern_di/container.py:

• A child takes parent_container._lock instead of allocating its own, so one RLock serves the
  whole container tree. The root still honours use_lock.
• build_child_container stops threading use_lock=self._lock is not None through to the child,
  since the lock is now inherited directly.

What already landed

The commit also added the G15b benchmark (sibling REQUEST children racing on cold cached
providers, which G15 cannot see because it races threads on a single root whose lock is shared
either way). That half is already on main as #486, so only the container.py change above is
unmerged.

One note for whoever picks this up

The prototype adds a # noqa: SLF001 on parent_container._lock. That is the same
Container-reads-Container pattern annotated at modern_di/container.py:120 in #487: SLF001
exempts self/cls only, so it flags a same-class read even though no boundary is crossed. Give it
the same one-line comment rather than a bare suppression.

[lesnik512]

This was generated by AI during triage.

Triage: needs-triage → ready-for-human, enhancement

The readiness gate in the body is satisfied

"Needs before it is ready" asked for three things. All three are answered:

• G6 (child build): measured, −16.0%, in the measurement comment above.
• G15 (contention): measured, and more usefully, shown to be structurally blind to this change
  (it races threads on a single root, whose lock is shared under both designs). The scenario that
  does change, concurrent cold misses in sibling children, is now covered by
  test_g15b_concurrent_first_resolve_sibling_children, which landed on main in test(benchmarks): cover concurrent first-resolve in sibling children #486. So the
  benchmark the decision needs is in the repo, not only on a branch.
• Free-threading test pass: done, and it found nothing to rewrite, because no test asserted that
  a child owns a distinct lock. test_build_child_container_propagates_use_lock_false and
  test_use_lock_false_yields_no_private_lock pass under inheritance as well as forwarding.

The 107 ns microbenchmark in the body is superseded: end to end it is ~132 ns and 16% of G6, so the
body's 13% estimate was conservative. The number worth quoting is G7's −5.4%, not G6's −16%,
since the same ~130 ns is a much smaller slice of a realistic build/resolve/close cycle.

Why ready-for-human and not ready-for-agent

The remaining work is not specification, it is a judgment on public API, which is exactly what the
ready-for-human role covers. Two things are bundled here and only one of them is measured:

1. Sharing the lock across the tree. Non-breaking, fully measured, positive except for a weak
   ~4% regression on free-threaded sibling contention whose direction is confirmed and magnitude is
   not.
2. Removing use_lock from the constructor. Breaking, unmeasured (there is nothing to measure),
   and semantically different: once the lock is per-tree, use_lock=False means "this whole tree is
   unlocked", a narrower promise than today's per-container flag. Either the argument stays with
   that reinterpretation documented, or it goes with a migration note.

No agent should pick (2) up from this issue as written, and (1) cannot ship on its own without
someone deciding whether it is worth the free-threaded trade.

Suggestion, not an action taken

Consider splitting: (1) as its own non-breaking issue that could be ready-for-agent once the
free-threaded trade is accepted, and (2) staying here as the 4.0 API decision. Leaving that to you
rather than splitting the issue during triage.

Not changed

Milestone stays 4.0, which is correct for (2) and is the reason (1) is currently blocked behind
it. No assignee. The prototype commit is preserved and recoverable per
this comment.