Repository navigation
Conversation
Removing root.json before creating the new link means a failure leaves no root.json at all, and bootstrap=None can no longer be used. Use a temp name + os.replace(), like _persist_file() does. Signed-off-by: Sachith Reddy <sachith.24bcs10403@sst.scaler.com>
|
@jku could you please have a look when you get a chance . |
|
This also matters for concurrent processes sharing the metadata directory (#2836), but the fixed A temp name unique to the process makes this step concurrency-safe on POSIX: temp_name = f"{linkname}.{os.getpid()}.tmp"With that change, the same test had no failures (0 out of 15 trials, vs 15 out of 15 with this PR and 18 out of 18 without it). The other metadata writes already go through Context: since kernels 0.18, every Hugging Face kernel load verifies a sigstore signature, so the ranks of a multi-GPU launch hit this on a fresh machine: huggingface/kernels#970 |
_update_root_symlink()deletesroot.jsonbefore creating the new link, so if the process dies in that window or the link can't be created,root.jsonjust stays gone:Since that's the trust anchor for
Updater(bootstrap=None), the client can't start after that. It also runs from afinallyblock in_load_root(), so this can happen on failure paths too.Fixed this by creating the link under a temp name and using
os.replace()to put it in place, same idea as_persist_file(). I used a fixed.tmpname instead oftempfilesinceos.symlink()needs a path that doesn't exist yet, and multiple Updater instances aren't supported anyway.Added a test that fails on develop.