Give devices without PCI topology a stable cache identity - #68
Merged
Conversation
The kernel cache never hit on an Apple GPU. getUniqueDeviceIdentifier asks for CL_DEVICE_PCI_BUS_ID_NV and CL_DEVICE_PCI_SLOT_ID_NV, which Apple's runtime does not implement, and clGetWrapper returned the value it declared without initializing it and without looking at the error code. The identifier was therefore whatever happened to be on the stack: three consecutive runs on an M4 Max wrote three cache files and every one of them reported "precompiled = no". Queries now go through clGetWrapperChecked, which zeroes the value and reports whether the runtime answered. Where neither vendor extension is available the identifier is an FNV-1a hash of the device name, vendor and driver version. Two identical GPUs hash to one entry, which is what you want: the binary they need is the same. Devices that do report PCI topology keep the identifier they had, so caches already on disk stay valid. The same missing check made clGetWrapperString size its buffer from an uninitialized length whenever a query failed; it now returns an empty string. Also drops the container smoke test's blind spot: without a GPU the program exits before loading a kernel, so nothing verified the kernels were in the image at all, and points the two bench examples at pr/57 rather than a branch that only ever existed on one laptop. Co-authored-by: Cursor <cursoragent@cursor.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
The compiled-kernel cache never hit on an Apple GPU.
getUniqueDeviceIdentifierasks forCL_DEVICE_PCI_BUS_ID_NVandCL_DEVICE_PCI_SLOT_ID_NV, which Apple's runtime does not implement, andclGetWrapperreturned the value it declared without initializing it and without looking at the error code:So the identifier was whatever was on the stack. Three consecutive runs on an M4 Max wrote three different cache files and every one reported
precompiled = no, meaning the kernel was recompiled on every start.Queries now go through
clGetWrapperChecked, which zeroes the value and reports whether the runtime answered. Where neither vendor extension is available, the identifier is an FNV-1a hash of the device name, vendor and driver version. Two identical GPUs hash to one entry, which is correct: the binary they need is the same. Devices that do report PCI topology keep the identifier they had, so caches already on disk stay valid and NVIDIA and AMD see no change.The same missing check made
clGetWrapperStringsize its buffer from an uninitialized length whenever a query failed.Two smaller things ride along. The container smoke test had a blind spot: without a GPU the program exits before loading a kernel, so nothing verified the kernels were in the image at all - it now checks they sit next to the binary. And the two
benchexamples pointed atpr57-head, a branch that only ever existed on one laptop, rather than the portablepr/57.Test plan
precompiled = yes; before the change it was three files andprecompiled = noevery timemakeandmake -C testsbuild without warnings,test_correctness_pr49.x64passesMade with Cursor