Merge branch 'main' into unignore-error-port-unreachable-in-proactor

Author: graingert

.github/workflows/reusable-san.yml

@@ -72,6 +72,8 @@ jobs:
     - name: MSan option setup
       if: inputs.sanitizer == 'MSan'
       run: |
+        sudo sysctl -w vm.mmap_rnd_bits=28  # Reduce ASLR to avoid MSan re-executing
+
         echo "MSAN_OPTIONS=${SAN_LOG_OPTION} allocator_may_return_null=1 handle_segv=0" >> "$GITHUB_ENV"
         # MSan reports false positives for memory initialized by libraries
         # that are not built with MSan, so disable modules that use them.

Doc/data/stable_abi.dat

@@ -169,14 +169,22 @@ or :c:member:`PyTypeObject.tp_itemsize`), it cannot be ported to
 ``abi3t`` 3.15.
 
 
+.. _abi3t-migration-build:
+
 Setting up the build
 ====================
 
-If you use a build tool (such as setuptools, meson-python, scikit-build-core),
-search its documentation for a way to select ``abi3t``.
-At the time of writing, not all of them have this; but if your tool does,
-use it.
-You may want to verify that it set the right flag by temporarily adding the
+If you use a build tool, search its documentation for "``abi3t``", and follow
+any instructions to select the ABI.
+For reference, here are direct links for several popular build tools:
+
+- `meson-python
+  <https://mesonbuild.com/meson-python/how-to-guides/limited-api.html#the-abi3t-stable-abi>`__
+- `scikit-build-core
+  <https://scikit-build-core.readthedocs.io/en/stable/configuration/#customizing-the-output-wheel>`__
+- `Maturin <https://www.maturin.rs/bindings#py_limited_apiabi3>`__
+
+You may want to verify that the tool set the right flag by temporarily adding the
 following just after ``#include <Python.h>``::
 
    #if Py_TARGET_ABI3T+0 <= 0x30f0000
@@ -185,6 +193,13 @@ following just after ``#include <Python.h>``::
 
 This should result in a different error than "``abi3t`` define is not set".
 
+.. seealso::
+
+   `Building and distributing abi3t extensions
+   <https://py-free-threading.github.io/abi3t/>`__:
+   Build configuration and wheel testing examples in the community-maintained
+   Python Free-Threading Guide.
+
 .. note::
 
    If your build tool doesn't support ``abi3t`` yet, set the following macro

Doc/howto/abi3t-migration.rst

@@ -2113,7 +2113,7 @@ always available. Unless explicitly noted otherwise, all variables are read-only
    See :ref:`remote-debugging` for more information about the remote debugging
    mechanism.
 
-   .. audit-event:: sys.remote_exec pid script_path
+   .. audit-event:: sys.remote_exec pid,script_path
 
       When the code is executed in the remote process, an
       :ref:`auditing event <auditing>` ``sys.remote_exec`` is raised with

Doc/library/sys.rst

@@ -1022,6 +1022,19 @@ Debug options
 
    .. versionadded:: 3.6
 
+.. option:: --with-hwaddress-sanitizer
+
+   Enable HWAddressSanitizer memory error detector, ``hwasan`` (default is no).
+   Note that on x86-64 this uses `page aliasing
+   <https://clang.llvm.org/docs/HardwareAssistedAddressSanitizerDesign.html#supported-architectures>`_,
+   which only tags heap allocations and is unsafe for programs that ``fork()``,
+   including much of the test suite.
+   See the `LLVM HWASan design documentation
+   <https://clang.llvm.org/docs/HardwareAssistedAddressSanitizerDesign.html>`_
+   for more information.
+
+   .. versionadded:: next
+
 .. option:: --with-memory-sanitizer
 
    Enable MemorySanitizer allocation error detector, ``msan`` (default is no).

Doc/using/configure.rst

@@ -572,16 +572,19 @@ the existing Stable ABI (``abi3``) and the version-specific ABI for
 free-threading (``cp315t``) separately.
 
 Stable ABI for Free-Threaded Builds should typically
-be selected in a build tool (such as, for example, Setuptools, meson-python,
-scikit-build-core, or Maturin).
-At the time of writing, these tools do **not** support ``abi3t``.
-If this is the case for your tool, compile for ``cp315t`` separately.
+be selected in a build tool. Tools with ``abi3t`` support include
+meson-python, scikit-build-core, and Maturin; see
+:ref:`abi3t-migration-build` for links to their documentation on ``abi3t`` builds.
+If your build tool does not support ``abi3t``, compile for ``cp315t`` separately.
 If not using a build tool -- or when writing such a tool -- you can select
 ``abi3t`` by setting the macro :c:macro:`!Py_TARGET_ABI3T` as discussed
 in :ref:`abi3-compiling`.
 
 A practical :ref:`migration guide <abi3t-migration-howto>` for switching to
 ``abi3t`` is available.
+The community-maintained `Python Free-Threading Guide
+<https://py-free-threading.github.io/abi3t/>`__ provides examples of building
+and testing ``abi3t`` wheels.
 
 .. seealso:: :pep:`803` for further details.
 

Doc/whatsnew/3.15.rst

@@ -1171,6 +1171,11 @@ Build changes
 
   (Contributed by Stan Ulbrych in :gh:`139314`.)
 
+* Add the :option:`--with-hwaddress-sanitizer` :program:`configure` option to
+  build with `HWAddressSanitizer <https://clang.llvm.org/docs/HardwareAssistedAddressSanitizerDesign.html>`_.
+
+  (Contributed by Stan Ulbrych, Florian Mayer and AnnaAr321 in :gh:`156049`.)
+
 
 C API changes
 =============
@@ -1181,6 +1186,10 @@ New features
 * Add :c:func:`Py_HashBuffer` to the limited C API.
   (Contributed by Victor Stinner in :gh:`148233`.)
 
+* Add :c:macro:`Py_SETREF` and :c:macro:`Py_XSETREF` macros to the limited C
+  API.
+  (Contributed by Nathan Goldbaum and Victor Stinner in :gh:`158842`.)
+
 Porting to Python 3.16
 ----------------------
 

Doc/whatsnew/3.16.rst

@@ -326,87 +326,6 @@ _PyObject_GenericSetAttrWithDict(PyObject *, PyObject *,
 
 PyAPI_FUNC(PyObject *) _PyObject_FunctionStr(PyObject *);
 
-/* Safely decref `dst` and set `dst` to `src`.
- *
- * As in case of Py_CLEAR "the obvious" code can be deadly:
- *
- *     Py_DECREF(dst);
- *     dst = src;
- *
- * The safe way is:
- *
- *      Py_SETREF(dst, src);
- *
- * That arranges to set `dst` to `src` _before_ decref'ing, so that any code
- * triggered as a side-effect of `dst` getting torn down no longer believes
- * `dst` points to a valid object.
- *
- * Temporary variables are used to only evaluate macro arguments once and so
- * avoid the duplication of side effects. _Py_TYPEOF(), C++ auto, or memcpy()
- * is used to avoid a miscompilation caused by type punning. See Py_CLEAR()
- * comment for implementation details about type punning.
- *
- * The memcpy() implementation does not emit a compiler warning if 'src' has
- * not the same type than 'src': any pointer type is accepted for 'src'.
- */
-#ifdef _Py_TYPEOF
-#define Py_SETREF(dst, src) \
-    do { \
-        _Py_TYPEOF(&(dst)) _tmp_dst_ptr = &(dst); \
-        _Py_TYPEOF(dst) _tmp_old_dst = (*_tmp_dst_ptr); \
-        *_tmp_dst_ptr = (src); \
-        Py_DECREF(_tmp_old_dst); \
-    } while (0)
-#elif defined(__cplusplus) && (__cplusplus >= 201103L ||  _MSVC_LANG >= 201103L)
-#define Py_SETREF(dst, src) \
-    do { \
-        auto _tmp_dst_ptr = &(dst); \
-        auto _tmp_old_dst = (*_tmp_dst_ptr); \
-        *_tmp_dst_ptr = (src); \
-        Py_DECREF(_tmp_old_dst); \
-    } while (0)
-#else
-#define Py_SETREF(dst, src) \
-    do { \
-        PyObject **_tmp_dst_ptr = _Py_CAST(PyObject**, &(dst)); \
-        PyObject *_tmp_old_dst = (*_tmp_dst_ptr); \
-        PyObject *_tmp_src = _PyObject_CAST(src); \
-        memcpy(_tmp_dst_ptr, &_tmp_src, sizeof(PyObject*)); \
-        Py_DECREF(_tmp_old_dst); \
-    } while (0)
-#endif
-
-/* Py_XSETREF() is a variant of Py_SETREF() that uses Py_XDECREF() instead of
- * Py_DECREF().
- */
-#ifdef _Py_TYPEOF
-#define Py_XSETREF(dst, src) \
-    do { \
-        _Py_TYPEOF(&(dst)) _tmp_dst_ptr = &(dst); \
-        _Py_TYPEOF(dst) _tmp_old_dst = (*_tmp_dst_ptr); \
-        *_tmp_dst_ptr = (src); \
-        Py_XDECREF(_tmp_old_dst); \
-    } while (0)
-#elif defined(__cplusplus) && (__cplusplus >= 201103L ||  _MSVC_LANG >= 201103L)
-#define Py_XSETREF(dst, src) \
-    do { \
-        auto _tmp_dst_ptr = &(dst); \
-        auto _tmp_old_dst = (*_tmp_dst_ptr); \
-        *_tmp_dst_ptr = (src); \
-        Py_XDECREF(_tmp_old_dst); \
-    } while (0)
-#else
-#define Py_XSETREF(dst, src) \
-    do { \
-        PyObject **_tmp_dst_ptr = _Py_CAST(PyObject**, &(dst)); \
-        PyObject *_tmp_old_dst = (*_tmp_dst_ptr); \
-        PyObject *_tmp_src = _PyObject_CAST(src); \
-        memcpy(_tmp_dst_ptr, &_tmp_src, sizeof(PyObject*)); \
-        Py_XDECREF(_tmp_old_dst); \
-    } while (0)
-#endif
-
-
 /* Define a pair of assertion macros:
    _PyObject_ASSERT_FROM(), _PyObject_ASSERT_WITH_MSG() and _PyObject_ASSERT().
 

Include/cpython/object.h

@@ -57,12 +57,6 @@ extern PyObject* _PyObject_Call(
     PyObject *args,
     PyObject *kwargs);
 
-extern PyObject * _PyObject_CallMethodFormat(
-    PyThreadState *tstate,
-    PyObject *callable,
-    const char *format,
-    ...);
-
 // Export for 'array' shared extension
 PyAPI_FUNC(PyObject*) _PyObject_CallMethod(
     PyObject *obj,

Include/internal/pycore_call.h

@@ -468,7 +468,7 @@ extern void _PyUnicode_InternStatic(PyInterpreterState *interp, PyObject **);
 extern void _PyUnicode_ClearInterned(PyInterpreterState *interp);
 
 // Like PyUnicode_AsUTF8(), but check for embedded null characters.
-// Export for '_sqlite3' shared extension.
+// Export for '_sqlite3' shared extension, and for Argument Clinic code.
 PyAPI_FUNC(const char *) _PyUnicode_AsUTF8NoNUL(PyObject *);