Skip to content

gh-159564: Make weakref dereference lock-free. - #159578

Draft
nascheme wants to merge 1 commit into
python:mainfrom
nascheme:gh-159564-lock-free-weakref-get-ref
Draft

nascheme wants to merge 1 commit into
python:mainfrom
nascheme:gh-159564-lock-free-weakref-get-ref

Conversation

@nascheme

@nascheme nascheme commented Oct 16, 2026 •

Copy link
Copy Markdown
Member

In the free-threaded build, change _PyWeakref_GET_REF() so it no longer takes the striped weakref list lock. The referent's memory stays readable if it is freed concurrently because object heaps delay page reuse with QSBR, and wr_object only ever transitions from the referent to Py_None, so _Py_TryIncrefCompare() is sufficient.

Use release store for clear_weakref_lock_held(), instead of sequentially consistent stores. The weakref list lock orders the list updates, and the lock-free read only requires that the store of Py_None happens before the referent's memory can be reused.

The benchmark results below are based on a C extension implementing a weak-map using either PyUnstable_TryIncRef() (borrowed refereces) vs using weak references. Thread scaling of "get, N threads, private maps" (ns per op, lower is better):

Threads tryincref weakref weakref opt
1 11.1 20.8 11.7
4 4.9 21.1 5.4
8 3.0 23.0 3.2

In the free-threaded build, _PyWeakref_GET_REF() no longer takes the
striped weakref list lock.  The referent's memory stays readable if it
is freed concurrently because object heaps delay page reuse with QSBR,
and wr_object only ever transitions from the referent to Py_None, so
_Py_TryIncrefCompare() is sufficient.

clear_weakref_lock_held() now uses release stores instead of
sequentially consistent stores.  The weakref list lock orders the list
updates, and the lock-free read only requires that the store of Py_None
happens before the referent's memory can be reused.
@nascheme nascheme added performance Performance or resource usage topic-free-threading labels Oct 16, 2026
@bedevere-app bedevere-app Bot added the type-feature A feature request or enhancement label Oct 16, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

performance Performance or resource usage topic-free-threading type-feature A feature request or enhancement

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant