Reduce critical section boilerplate in type slot implementations

@erlend-aasland

Feature or enhancement

PEP 703 in large part relies on replacing the GIL with fine grained per-object locks. The primary way to acquire and release these locks is through the critical section API (#111569). In #111903, we added support for generating the necessary critical section calls in Argument Clinic wrapped methods. In #112205, this was extended to getters and setters as well.

However, some object behavior is implemented using "tp slots", which are not supported by Argument Clinic. These functions often need critical sections for thread-safety; that means more boilerplate code.

Here are some possible ideas on how to address this:

Add support for "tp slots" to Argument Clinic

We could add support for certain tp slots functions to Argument Clinic. This would provide a consistent way to add critical sections across both slots, getters/setters, and "regular" methods. The disadvantage is that "tp slots" don't need much argument parsing, so they may otherwise not be a great fit for Argument Clinic

Use C macros to wrap slot implementations with critical sections

We could implement an internal-only header that provides macros to wrap a given "tp slot" function with a critical section. For example, something like:

static PyObject *
list_richcompare_impl(PyObject *v, PyObject *w, int op)
{
   // implement richcompare unchanged
}

// generates a list_richcompare() function with the correct Py_BEGIN/END_CRITICAL_SECTION()
_Py_CRITICAL_SECTION_TP_RICHCOMPARE(list_richcompare, list_richcompare_impl);

I think this would not be too difficult to implement -- probably easier than modifying Argument Clinic. The disadvantage is the lack of uniformity around how we add critical sections to functions.

Keep the status quo

We can always just keep adding the Py_BEGIN/END_CRITICAL_SECTION() calls manually for tp slots.

cc @erlend-aasland, @corona10

Linked PRs

pythongh-114258: Argument Clinic: refactor getset implementation (pyt…

@ZeroIntensity

cc @ZeroIntensity

@corona10

Peter and I discussed during the sprint a simple plan like this:

Phase 1: single `@critical_section` + plain Clinic declaration (53)

Status	dunder	C slot	Return	Functions	Assignee
✅	`__next__`	tp_iternext	PyObject *	chain_next, product_next, combinations_next, cwr_next, permutations_next, accumulate_next, zip_longest_next, bytesio_iternext, stringio_iternext, textiowrapper_iternext, Reader_iternext, odictiter_iternext, longrangeiter_next	@corona10
✅	`__repr__`	tp_repr	PyObject *	list_repr, dict_repr, set_repr, bytearray_repr, mmap__repr__method	@ZeroIntensity
✅	`__len__`	mp_length, sq_length	Py_ssize_t	dbm_length, gdbm_length, mmap_length	@corona10
✅	`__bool__`	nb_bool	int	dbm_bool, gdbm_bool	@corona10
✅	`__hash__`	tp_hash	Py_hash_t	weakref_hash	@corona10
✅	`__getitem__` (`Py_ssize_t`)	sq_item	PyObject *	deque_item, Array_item, Pointer_item, mmap_item, bytearray_getitem	@corona10
✅	`__getitem__` (`object`)	mp_subscript	PyObject *	Array_subscript, dbm_subscript, gdbm_subscript, mmap_subscript	@corona10
✅	`__setitem__` (`Py_ssize_t`)	sq_ass_item	int	deque_ass_item, Array_ass_item, Pointer_ass_item, mmap_ass_item, bytearray_setitem, list_ass_item	@ZeroIntensity
✅	`__setitem__` (`object`)	mp_ass_subscript	int	Array_ass_subscript, dbm_ass_sub, gdbm_ass_sub, mmap_ass_subscript	@ZeroIntensity
✅	`__contains__`	sq_contains	int	deque_contains, dbm_contains, gdbm_contains	@corona10
✅	`__mul__`	sq_repeat	PyObject *	bytearray_repeat, list_repeat	@ZeroIntensity
✅	`__imul__`	sq_inplace_repeat	PyObject *	deque_inplace_repeat, bytearray_irepeat, list_inplace_repeat	@ZeroIntensity
✅	`__add__`	sq_concat	PyObject *	deque_concat	@ZeroIntensity
✅	`__iadd__`	sq_inplace_concat	PyObject *	bytearray_iconcat	@ZeroIntensity

Phase 2: `Py_BEGIN_CRITICAL_SECTION2` + plain Clinic declaration (2)

Status	dunder	C slot	Signature	Functions	Assignee
⬜	`__richcompare__` (name TBD)	tp_richcompare	(self, other, op) -> PyObject *	list_richcompare, odict_richcompare	@ZeroIntensity

@critical_section self other already works; only the dunder name and
the op parameter need a decision.

Phase 3: small migration first, then Clinic (16)

Status	Work needed	dunder	Functions	Assignee
✅	Extract the locked body into an `_impl`	`__repr__`	BaseException_repr, frame_repr	@corona10
✅	Extract the locked body into an `_impl`	`__str__`	BaseException_str	@corona10
✅	Extract the locked body into an `_impl`	`__next__`	tokenizeriter_next	@corona10
✅	Extract the locked body into an `_impl`	`__bool__`	Pointer_bool, Simple_bool	@corona10
⬜	Extract the locked body into an `_impl`	`__contains__`	bytearray_contains	@ZeroIntensity
⬜	Extract the locked body into an `_impl`	`__release_buffer__`	mmap_buffer_releasebuf, Xxo_releasebuffer	@ZeroIntensity
⬜	Second lock target is a field of self	`__next__`	dequeiter_next, dequereviter_next	@ZeroIntensity
⬜	Buffer slot, Py_buffer * parameter	`__buffer__`	mmap_buffer_getbuf, bytesiobuf_getbuffer	@ZeroIntensity

Out of scope for now: keep manual (25)

The rest of the case would be handled manually.

pythongh-114258: Update AC to support @critical_section for tp_next

pythongh-114258: Update AC to support @critical_section for tp_iternext

gh-114258: Update AC to support @critical_section for tp_iternext (gh…

gh-114258: Use Argument Clinic for `__len__` (GH-160012)

gh-114258: Use Argument Clinic for `__setitem__` (GH-160048)

Merge remote-tracking branch 'upstream/main' into pythongh-114258-get…

gh-114258: Use Argument Clinic for `__repr__` (GH-160011)

Merge remote-tracking branch 'upstream/main' into pythongh-114258-con…

gh-114258: Use Argument Clinic for `__mul__`/`__imul__` (GH-160049)

gh-114258: Use Argument Clinic for `__add__`/`__iadd__` (GH-160051)

Merge remote-tracking branch 'upstream/main' into pythongh-114258-con…

gh-114258: Update Argument Clinic for __contain__ slots (gh-160093)

Merge remote-tracking branch 'upstream/main' into pythongh-114258-get…

Merge remote-tracking branch 'upstream/main' into pythongh-114258-hash

gh-114258: Use Argument Clinic for ``__getitem__`` (gh-160068)

Merge branch 'main' of https://github.com/python/cpython into pythong…

gh-114258: Use Argument Clinic for ``__bool__`` (GH-160057)

Merge remote-tracking branch 'upstream/main' into pythongh-114258-hash

gh-114258: Use Argument Clinic for ``__hash__`` (gh-160059)

gh-114258: Update Argument Clinic for ``__str__``. (gh-160402)

gh-114258: Update _ctypes and tokenize to use AC where it can be. (gh…

Navigation Menu

Repository navigation

Reduce critical section boilerplate in type slot implementations #114258

Feature or enhancement

Add support for "tp slots" to Argument Clinic

Use C macros to wrap slot implementations with critical sections

Keep the status quo

Linked PRs

Phase 1: single `@critical_section` + plain Clinic declaration (53)

Phase 2: `Py_BEGIN_CRITICAL_SECTION2` + plain Clinic declaration (2)

Phase 3: small migration first, then Clinic (16)

Out of scope for now: keep manual (25)

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Reduce critical section boilerplate in type slot implementations #114258

Description

Feature or enhancement

Add support for "tp slots" to Argument Clinic

Use C macros to wrap slot implementations with critical sections

Keep the status quo

Linked PRs

Activity

corona10 commented on Oct 20, 2026

corona10 commented on Oct 20, 2026

Phase 1: single @critical_section + plain Clinic declaration (53)

Phase 2: Py_BEGIN_CRITICAL_SECTION2 + plain Clinic declaration (2)

Phase 3: small migration first, then Clinic (16)

Out of scope for now: keep manual (25)

Metadata

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Issue actions

Phase 1: single `@critical_section` + plain Clinic declaration (53)

Phase 2: `Py_BEGIN_CRITICAL_SECTION2` + plain Clinic declaration (2)