Skip to content

Drop the unreachable re-entry guard from _repo_lock - #326

Open
anandhu-eng wants to merge 1 commit into
docs/repo-lock-timeoutfrom
refactor/single-repo-lock-guard
Open

anandhu-eng wants to merge 1 commit into
docs/repo-lock-timeoutfrom
refactor/single-repo-lock-guard

Conversation

@anandhu-eng

@anandhu-eng anandhu-eng commented Sep 23, 2026 •

Copy link
Copy Markdown
Contributor

Stack: #265 ← #323 ← #325 ← #326. Merge in that order: #323 into #265's branch first, then each next PR. This PR targets #325's branch. All 125 tests pass at the top of the stack.

Why

#265 has two thread-local sets that both mean "this thread already has this repo":

_held_repo_locks (in _repo_lock) _pulling_repos (in _repo_pull_lock)
On re-entry Continues without locking, and redoes the work Skips the work and returns {'return': 0}
Reached today No Yes, on dependency cycles

Both came from #303. The first fixed a self-deadlock: register_repo pulls each dependency while the parent repo's lock is held, and a second FileLock on the same file blocks even within one thread. But that fix turned a dependency cycle into infinite recursion, so the second set was added to skip the nested pull.

The first set is now unreachable:

  • _repo_pull_lock checks _pulling_repos before it calls _repo_lock, and a path is only in _held_repo_locks while it is also in _pulling_repos.
  • _repo_lock's only other caller, RepoAction.rm(), never runs inside a pull.
  • Removing the check fails only the one test that called _repo_lock directly. No pull or rm test notices.

Changes

  • _repo_lock is now a plain lock. _held_repo_locks and _repo_locks_held_by_this_thread are removed.
  • The note about two threads pulling repos that depend on each other (a real lock-order inversion that still ends only on timeout) moves onto _pulling_repos. Comments that described same-thread re-entry as a no-op are corrected.
  • Tests:
    • test_same_thread_reentry_does_not_self_deadlock is removed. It only exercised the deleted branch.
    • test_dependency_cycle_does_not_recurse now also asserts that the nested entry doesn't block. With the remaining guard removed it fails with a lock Timeout, which I checked.

No behaviour change. Locally: 120 unit tests pass, plus .github/scripts/test_repo_pull_force.py.

🤖 Generated with Claude Code

@anandhu-eng
anandhu-eng requested a review from a team as a code owner September 23, 2026 22:47
@github-actions

Copy link
Copy Markdown

MLCommons CLA bot All contributors have signed the MLCommons CLA ✍️ ✅

@github-actions

Copy link
Copy Markdown

🤖 AI PR Review Summary

This PR removes the previously used thread-local set for tracking held repo locks and the reentrancy logic in _repo_lock, simplifying the locking mechanism. Instead, it relies on _repo_pull_lock to guard against same-thread recursive pulls. The comments have been updated to clarify the locking behavior and limitations. The test for same-thread reentry deadlock was removed, reflecting the change in locking strategy. Overall, the changes reduce complexity but remove the within-thread reentrancy in _repo_lock, which may affect code relying on that behavior. The tests verify that locks are released properly and that dependency cycles are guarded by _repo_pull_lock. No new inline actionable issues found in the diff.

@anandhu-eng
anandhu-eng force-pushed the refactor/single-repo-lock-guard branch from 54f451e to 2689a33 Compare September 23, 2026 23:11
@anandhu-eng
anandhu-eng changed the base branch from copilot/fix-mlc-pull-repo-thread-safety to docs/repo-lock-timeout September 23, 2026 23:11
@anandhu-eng
anandhu-eng added this pull request to stack #324 September 29, 2026 13:28
@anandhu-eng
anandhu-eng force-pushed the refactor/single-repo-lock-guard branch from 2689a33 to fe5e5e4 Compare September 29, 2026 13:52
_repo_lock kept a thread-local set (_held_repo_locks) so a thread could
re-enter a repo lock it already held. That case can no longer happen:
_repo_pull_lock checks _pulling_repos first and skips the nested pull, and
a path is in _held_repo_locks only while it is also in _pulling_repos. The
other caller, RepoAction.rm(), never runs inside a pull.

Remove the set and make _repo_lock a plain lock, leaving _pulling_repos as
the single same-thread guard. Move the note about crossed dependencies
between threads onto _pulling_repos, and fix comments that described
re-entry as a no-op.

Tests: drop test_same_thread_reentry_does_not_self_deadlock, which only
exercised the removed branch. test_dependency_cycle_does_not_recurse now
also asserts the nested entry does not block; it fails with a lock
Timeout if the remaining guard is removed.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@anandhu-eng
anandhu-eng force-pushed the refactor/single-repo-lock-guard branch from fe5e5e4 to 25106f4 Compare September 29, 2026 14:08

This branch was successfully deployed

1 active (outdated) deployment
ai-review — 54f451ef Deployed Sep 23, 2026 by anandhu-eng via ai-review #326
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants