Skip to content

feat(bigtable): sweep stale accelerator daemon tempdirs at startup - #18688

Merged
mutianf merged 3 commits into
bigtable-accelfrom
accel8
Oct 23, 2026
Merged

mutianf merged 3 commits into
bigtable-accelfrom
accel8

Conversation

@mutianf

@mutianf mutianf commented Oct 13, 2026

Copy link
Copy Markdown
Contributor

Thank you for opening a Pull Request! Before submitting your PR, there are a few things you can do to make sure it goes smoothly:

  • Make sure to open an issue as a bug/issue before writing your code! That way we can discuss the change, evaluate designs, and agree on the general idea
  • Ensure the tests and linter pass
  • Code coverage does not decrease (if any source code was changed)
  • Appropriate docs were updated (if necessary)

Fixes #<issue_number_goes_here> 🦕


Stack created with GitHub Stacks CLI • Give Feedback 💬

@mutianf
mutianf requested a review from a team as a code owner October 13, 2026 19:56
@mutianf
mutianf added this pull request to stack #18689 October 13, 2026 19:56
@mutianf
mutianf requested a review from sushanb October 13, 2026 19:57

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces automatic cleanup of stale accelerator daemon temporary directories at startup by scanning for leftover directories, checking if their sockets are active, and removing them if they are older than a threshold. The review feedback correctly identifies potential compatibility issues on non-Unix platforms (such as Windows) where Unix domain sockets are unsupported. It is recommended to add a defensive guard in the sweep function to avoid AttributeErrors and to skip the new unit tests on unsupported platforms.

Comment thread packages/google-cloud-bigtable/google/cloud/bigtable/data/_accelerator/_daemon.py Outdated
@mutianf
mutianf removed this pull request from stack #18689 October 13, 2026 20:06
@mutianf
mutianf added this pull request to stack #18691 October 13, 2026 20:07
Base automatically changed from accel7 to accel6 October 13, 2026 20:34
Base automatically changed from accel6 to bigtable-accel October 21, 2026 19:40
Change-Id: Icd8495d9c7f1bdcf39ae984b0037f813ef4c4011
Comment thread packages/google-cloud-bigtable/google/cloud/bigtable/data/_accelerator/_daemon.py Outdated
Comment thread packages/google-cloud-bigtable/google/cloud/bigtable/data/_accelerator/_daemon.py Outdated
Comment thread packages/google-cloud-bigtable/google/cloud/bigtable/data/_accelerator/_daemon.py Outdated
@mutianf

mutianf commented Oct 22, 2026

Copy link
Copy Markdown
Contributor Author

Note to reviewer: logic changes are in daemon.py. Rest are formatting changes

…ilure

Change-Id: Ie5c45b75e0785a284a0d241415cd5b8492e14e9c
try:
with open(pid_path) as _f:
pid = int(_f.read().strip())
os.kill(pid, 0)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PID reuse makes stale tempdirs potentially immortal. Once the OS recycles this PID onto any long-lived service (common on Linux after wrap-around), os.kill(pid, 0) returns True forever and the dir is never reaped. Cheap mitigation: also read /proc/<pid>/stat field 22 (start_time, in clock ticks since boot) and compare to the tempdir's mtime — a mismatch means PID reuse and the dir is safe to sweep. Not a blocker (worst case is leaked /tmp entries), but worth tracking as a follow-up.

pid_path = os.path.join(path, _PID_FILENAME)
try:
with open(pid_path) as _f:
pid = int(_f.read().strip())

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If start() crashes between the deadline-file write and the pid-file write, the pid file ends up empty / truncated. int(_f.read().strip()) then raises ValueError, the outer except OSError doesn't catch it, and (depending on where the exception goes) the dir gets skipped and lives forever. Two options: (a) write to daemon.pid.tmp and os.rename into place atomically, or (b) broaden the except to also catch ValueError and treat an unparseable pid as "deadline-only, fall back to the age heuristic".

@sushanb sushanb left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. All three prior findings addressed (FD leak with-block, blank-line nit, scope confirmed). Two new follow-up suggestions above on PID reuse and partial-pid-write robustness — both can land as follow-ups, not blockers for this PR.

Change-Id: I2dcd04d457e90e628ebd17d69987174a58489379
@mutianf
mutianf merged commit 3bd2e23 into bigtable-accel Oct 23, 2026
16 checks passed
@mutianf
mutianf deleted the accel8 branch October 23, 2026 16:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants