Repository navigation
gh-159437: Support frozen objects in combined GC reference passes - #159597
mfalco-oai wants to merge 7 commits into
Conversation
Count the stable unreachable set while preparing callbacks and retain a conservative cleanup flag. Reuse the count only when late weakref clearing has no work, and force fresh cleanup and counting after resurrection analysis. Add a regression for a callback-free weakref observed when clearing a young cycle releases an older finalizer.
Count tp_del candidates during the existing reference traversals so zero can skip legacy classification. Reuse callback preparation to clear initial unreachable tags, while preserving normalized links before DEBUG_COLLECTABLE output can invoke Python. Cover debug visibility of weakrefs with callbacks and legacy classification during a generation 0 collection.
There was a problem hiding this comment.
Let's leave frozen support out of this series for now. The persistent header flag needs updating during freeze, unfreeze, untracking and weakref moves, so there is quite a bit more state to maintain here.
An earlier reference pass change I worked on kept the ordinary path when frozen objects were present. Keeping that fallback would let us handle frozen support separately once the application benefit is clearer.
I ran a little experiment keeping a cache of parsed SQL queries, freezing the runtime, then parsing and rendering more queries. Repeated runs on a busy machine didn't show a clear overall gain over the preceding change. The same work in a child after fork looked more promising, but there weren't enough runs to draw a conclusion.
Freezing and unfreezing that retained heap took much longer than with the fallback. Unfreezing in the child after fork was more expensive again. We need to include those costs when deciding whether this pays off.
@nascheme, any thoughts on the extra work in freeze() and unfreeze() here?
| if (wrcb_to_call->_gc_next == (uintptr_t)gc) { | ||
| #ifdef _PyGC_PREV_MASK_PERM | ||
| // The old generation must not retain permanent markers. | ||
| gc->_gc_prev &= ~_PyGC_PREV_MASK_PERM; |
There was a problem hiding this comment.
The weakref has already left the permanent generation when it enters wrcb_to_call. Why keep PERM set until after the callback? Clearing it when we queue the weakref would keep the flag tied to list membership.
|
|
||
| gc.disable() | ||
| first = Node() | ||
| first.link = first |
There was a problem hiding this comment.
I tried leaving the frozen flag set after unfreeze to see whether this test caught it. These separate self cycles still passed. Making first and second point at each other after the frozen setup left the objects alive and caught the missing collection. We should use that cycle here.
2 participants
With this optimization the GIL collector can use the combined reference pass during the initial reachability analysis of full collections with frozen objects, on builds with 64-bit pointers. Bit 2 of
_gc_previdentifies objects owned by the permanent generation, so the pass leaves their temporary reference state untouched.The marker requires a minimum alignment of 8 bytes for exposed object layouts; opaque headers keep
_PyObject_MIN_ALIGNMENTat 4.gc.freeze()andgc.unfreeze()walk their lists to set and clear the marker. Frozen weakrefs retain it while queued for callbacks, and surviving weakrefs have it cleared before moving to the old generation.The guard for subinterpreters and runtime reinitialization still applies.