Skip to content

feat: support rust gpu - #907

Closed
drbh wants to merge 5 commits into
mainfrom
rust-gpu-support
Closed

drbh wants to merge 5 commits into
mainfrom
rust-gpu-support

Conversation

@drbh

@drbh drbh commented Oct 15, 2026 •

Copy link
Copy Markdown
Collaborator

this pr builds on the recent rust cpu support and extends support to rust gpu

related prior work

@HuggingFaceDocBuilderDev

Copy link
Copy Markdown

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

@github-actions

github-actions Bot commented Oct 15, 2026 •

Copy link
Copy Markdown

Coverage report — kernels/

Measured on: Python 3.10 / Torch 2.13.0.
Other CI configurations are not included in this number.
Hardware-gated code paths (ROCm/XPU/NPU/Darwin/Windows) are excluded or unreachable on the Linux+CUDA runner.

Total coverage: 87.8% — threshold: 80% — ✅

Per-file breakdown
Name Stmts Miss Cover Missing
src/kernels/__init__.py 14 0 100%
src/kernels/_system.py 6 1 83% 10
src/kernels/_versions.py 130 14 89% 53, 59-60, 63-64, 102, 165-170, 199, 219
src/kernels/archs.py 56 1 98% 94
src/kernels/backends.py 215 63 71% 42, 46, 50-53, 70, 92, 110, 119, 123, 127-129, 150, 159, 163, 167-169, 190, 201, 203, 210-213, 222, 226, 230-255, 263, 286-306
src/kernels/compat.py 9 1 89% 5
src/kernels/deps.py 70 1 99% 56
src/kernels/hf_hub.py 63 2 97% 21, 23
src/kernels/importer.py 50 5 90% 95, 99, 102, 116-117
src/kernels/install.py 21 7 67% 76-100
src/kernels/layer/__init__.py 6 0 100%
src/kernels/layer/_interval_tree.py 103 4 96% 23, 52, 147, 150
src/kernels/layer/device.py 48 14 71% 42, 47-49, 91, 96-98, 101, 149, 152, 155-157
src/kernels/layer/func.py 85 6 93% 90, 115, 191, 311, 338, 368
src/kernels/layer/globals.py 5 0 100%
src/kernels/layer/kernelize.py 82 8 90% 259, 297, 305-306, 312, 316, 332-334
src/kernels/layer/layer.py 309 15 95% 211, 258, 285, 416, 556-557, 578, 586, 597, 637, 641, 654, 708, 738, 813
src/kernels/layer/mode.py 14 0 100%
src/kernels/layer/repos.py 144 42 71% 27, 33, 36-43, 63-64, 70, 73-76, 90, 94, 103-104, 110, 113-116, 123-124, 130, 133-136, 143-144, 150, 153-156, 163-164, 170, 173-176, 257
src/kernels/load.py 71 2 97% 338, 378
src/kernels/locking.py 89 64 28% 35-83, 91-98, 102-125, 137, 152-159, 165-175, 179-186
src/kernels/python_deps.py 58 6 90% 59-60, 64-65, 101, 104
src/kernels/resolver.py 162 1 99% 248
src/kernels/status.py 50 2 96% 25, 79
src/kernels/validate.py 88 5 94% 9, 100, 167, 190-191
src/kernels/variants.py 278 17 94% 65, 96, 117, 147, 256-257, 299-302, 304, 388-394, 400-406, 455-461
src/kernels/verify.py 127 6 95% 46, 202-204, 318-319
TOTAL 2353 287 88%

Updated by the Test kernels workflow on commit 1cfb75aaa9a574693e52624ba8b73c3d0df03d0f.

@danieldk
danieldk self-requested a review October 19, 2026 08:29

@danieldk danieldk left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice! Added a bunch of comments.

Comment thread kernels-common/src/config/mod.rs Resolved
Comment thread kernels-common/src/config/mod.rs Resolved
Comment thread kernels-common/src/config/mod.rs Outdated

RustCuda {
cargo_manifest: String,
device_manifest: Option<String>,

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I looked at some of the example code in cuda-oxide and I couldn't find separate device manifests? Do you have an example? (The PR doesn't seem to have an example.)

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it's cuda-oxide's interop mode, the host crate lists its device crates in [[package.metadata.cuda-oxide.device-crates]]. we need the split for now since a single crate pulls in bindgen via cuda-core and generic kernels can't load from a .so yet. it was duplicated in build.toml though so i removed it there and we read the cargo metadata instead. also added an example in examples/kernels/relu-rust-cuda

Comment thread kernels-common/src/config/mod.rs Resolved
@drbh
drbh marked this pull request as draft October 22, 2026 00:17
@drbh

drbh commented Oct 22, 2026

Copy link
Copy Markdown
Collaborator Author

note: moving back to draft since I'm updating the approach to avoid bundling ptx in the kernel and improving the overall build process

@drbh drbh mentioned this pull request Oct 22, 2026
@drbh

drbh commented Oct 28, 2026

Copy link
Copy Markdown
Collaborator Author

closing since #928 supersedes

@drbh drbh closed this Oct 28, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants