Repository navigation
Preserve container exits and OOM backoff - #2045
Conversation
jackf-beamcloud
left a comment
There was a problem hiding this comment.
Looks right: tini re-exec only when the worker is PID 1 (tini is already in the worker image), final-metrics evidence covers the lost wait RPC case including init vs guest OOM, and the terminalOOM flag stops later cleanup stops from reclassifying an OOM. Non-blocking: Wait does an unbounded <-done after forceDelete with a WithoutCancel ctx and no cmd.WaitDelay on the restore command, so a stray inherited stdout fd would hang it; and restoreOutput entries leak if a restored container never reaches Wait.
|
Addressed both follow-ups in 6b06ffa: bounded post-exit draining (including detached Run), automatic restore-tracking cleanup, and joining pending output after deletion. Orphan-FD regressions cover Run/Wait/Delete/no Wait and fail without the fixes. Full Linux/race and staging checks passed again. Restored stdout/stderr remained live after 9.18s, with all 130 final-tail lines preserved. |
jackf-beamcloud
left a comment
There was a problem hiding this comment.
6b06ffa covers both earlier notes: WaitDelay on the restore command plus the read deadline on the output pipe bound post-exit draining, and restore tracking now removes itself via CompareAndDelete when the restore CLI exits, so containers that never reach Wait don't leak entries. Orphan-fd tests for Run/Wait/Delete/no-Wait look right. Non-blocking: once the 5s deadline hits, output an orphan writes after that is dropped, which seems like the right trade.
Reap detached runsc children, preserve terminal exit/OOM evidence through cleanup, and bound final log drains without changing failure backoff.
Validated Linux/race tests and staging exits/OOMs/stops, functions/endpoints, sandbox survival, logging/checkpoint/restore, and fresh image builds. Pins the approved merged gVisor runtime.