Bug report

Bug description:

After a JVM is created in-process on the main thread (as JPype and r5py do), freeing a deeply nested list segfaults on 3.14 and 3.15. The same script runs cleanly on 3.13.

This reproducer uses only ctypes and a JDK, with no third-party Python packages:

"""Free a deeply nested list after starting a JVM in-process (JNI via ctypes)."""

import ctypes
import pathlib
import subprocess
import sys

java_home = subprocess.run(
    ["java", "-XshowSettings:properties", "-version"], capture_output=True, text=True
).stderr.split("java.home = ")[1].split()[0]
libjvm = ctypes.CDLL(str(pathlib.Path(java_home, "lib", "server", "libjvm.so")))


class JavaVMOption(ctypes.Structure):
    _fields_ = [("optionString", ctypes.c_char_p), ("extraInfo", ctypes.c_void_p)]


class JavaVMInitArgs(ctypes.Structure):
    _fields_ = [
        ("version", ctypes.c_int),
        ("nOptions", ctypes.c_int),
        ("options", ctypes.POINTER(JavaVMOption)),
        ("ignoreUnrecognized", ctypes.c_ubyte),
    ]


flags = [arg.encode() for arg in sys.argv[1:]]
options = (JavaVMOption * max(len(flags), 1))(*[JavaVMOption(f, None) for f in flags])
args = JavaVMInitArgs(0x00010008, len(flags), options, 0)  # JNI_VERSION_1_8
vm, env = ctypes.c_void_p(), ctypes.c_void_p()
assert libjvm.JNI_CreateJavaVM(ctypes.byref(vm), ctypes.byref(env), ctypes.byref(args)) == 0
print(sys.version.split()[0], "JVM started with", sys.argv[1:] or "default options", flush=True)

value = None
for _ in range(100_000):
    value = [value]
print("built", flush=True)
del value
print("freed", flush=True)

Results on Linux x86_64 (Ubuntu 24.04, glibc 2.39, OpenJDK 21.0.12, ulimit -s = 8192). The Python builds are the python-build-standalone interpreters installed by uv:

Other observations:

• On 3.14.6, the crash happens when freeing the list. Nesting depths of 1,000 to 10,000 free cleanly; 30,000 and above crash.
• Without the JVM, 3.14 frees the 100,000-deep list without a problem.
• With the JVM loaded, unbounded Python recursion (def f(): return f()) still raises RecursionError normally. Only the deallocation path crashes.
• No Python traceback or HotSpot hs_err crash log is produced. That fits the main thread's stack running into the JVM's guard pages.

My guess is that this comes from 3.14 checking C stack depth against the thread's actual stack bounds, which the trashcan deallocation path relies on. HotSpot adds guard pages to the primordial thread's stack, and CPython may not account for them. DisablePrimordialThreadGuardPages stops HotSpot from adding those guard pages, and the crash goes away. I haven't confirmed the mechanism in CPython's source.

We hit this in a pytest suite where some tests import r5py, which starts a JVM through JPype, and other tests build deeply nested values to check that the code under test doesn't recurse. The pytest-xdist workers died with "node down: Not properly terminated". As a workaround we now set JAVA_TOOL_OPTIONS="-XX:+UnlockExperimentalVMOptions -XX:+DisablePrimordialThreadGuardPages".

CPython versions tested on:

3.13, 3.14, 3.15

Operating systems tested on:

Linux