From b0c3846b70f108665d2af826da166b1402798a99 Mon Sep 17 00:00:00 2001 From: "riseproject-dev[bot]" <330740410+riseproject-dev[bot]@users.noreply.github.com> Date: Thu, 22 Oct 2026 12:15:00 +0000 Subject: [PATCH 1/4] vllm: Add versions 0.31.0, 0.32.0 Signed-off-by: riseproject-dev[bot] <330740410+riseproject-dev[bot]@users.noreply.github.com> --- docs/packages/vllm.yaml | 2 ++ 1 file changed, 2 insertions(+) diff --git a/docs/packages/vllm.yaml b/docs/packages/vllm.yaml index a65f2d9f66bb..f0ede378f265 100644 --- a/docs/packages/vllm.yaml +++ b/docs/packages/vllm.yaml @@ -16,3 +16,5 @@ versions: - filename: vllm-0.30.0-cp312-abi3-manylinux_2_39_riscv64.whl sha256: 8981c8e48b92e6c891543d2b85c203f98569f65366bd40afbbdf8970881d92e1 requires-python: <3.15,>=3.10 +- version: 0.31.0 +- version: 0.32.0 From 46d131c923017e5732295d81770f8056a7c08f38 Mon Sep 17 00:00:00 2001 From: Ludovic Henry Date: Fri, 23 Oct 2026 17:14:29 +0800 Subject: [PATCH 2/4] vllm: carry patches over to 0.31.0 and 0.32.0 (0001 rebased onto the unconditional numba requirement) --- ...ents-do-not-require-numba-on-riscv64.patch | 42 +++++++++++ ...-probe-the-torch-openblas-for-sbgemm.patch | 70 +++++++++++++++++++ ...ents-do-not-require-numba-on-riscv64.patch | 42 +++++++++++ ...-probe-the-torch-openblas-for-sbgemm.patch | 70 +++++++++++++++++++ 4 files changed, 224 insertions(+) create mode 100644 patches/vllm/0.31.0/0001-requirements-do-not-require-numba-on-riscv64.patch create mode 100644 patches/vllm/0.31.0/0002-cpu-probe-the-torch-openblas-for-sbgemm.patch create mode 100644 patches/vllm/0.32.0/0001-requirements-do-not-require-numba-on-riscv64.patch create mode 100644 patches/vllm/0.32.0/0002-cpu-probe-the-torch-openblas-for-sbgemm.patch diff --git a/patches/vllm/0.31.0/0001-requirements-do-not-require-numba-on-riscv64.patch b/patches/vllm/0.31.0/0001-requirements-do-not-require-numba-on-riscv64.patch new file mode 100644 index 000000000000..605ce99489ea --- /dev/null +++ b/patches/vllm/0.31.0/0001-requirements-do-not-require-numba-on-riscv64.patch @@ -0,0 +1,42 @@ +From 0000000000000000000000000000000000000001 Mon Sep 17 00:00:00 2001 +From: RISE Project CI +Date: Sat, 19 Sep 2026 00:00:00 +0000 +Subject: [PATCH] [CPU] Do not require numba on riscv64 + +requirements/cpu.txt requires numba on every CPU architecture except +s390x. numba has no riscv64 wheel on any index, and it cannot be built +from its sdist either: it hard-depends on llvmlite, whose build links a +patched LLVM distributed only as a conda package on a channel that +publishes no linux-riscv64 subdir. An unpatched riscv64 CPU wheel is +therefore buildable but not installable. + +numba is optional to vLLM by construction - vllm/utils/import_utils.py +declares is_numba_available() "Whether the optional `numba` package is +available" and the Kimi-K2.5 fused vision processor falls back to the +remote HF processor when it is absent. Excluding it costs riscv64 the +same features s390x already goes without (n-gram speculative decoding +and the fused vision/inkling processors), which is exactly the +configuration upstream already ships for s390x. + +Upstream-Status: To upstream [vLLM publishes no riscv64 wheels and runs no riscv64 CI, so there is no upstream build this marker would affect yet; resubmit once riscv64 joins docs/getting_started/installation/cpu.md and the CPU release matrix] + +Signed-off-by: RISE Project CI +--- + requirements/cpu.txt | 2 +- + 1 file changed, 1 insertion(+), 1 deletion(-) + +diff --git a/requirements/cpu.txt b/requirements/cpu.txt +index b4867a0..63c68b1 100644 +--- a/requirements/cpu.txt ++++ b/requirements/cpu.txt +@@ -3,7 +3,7 @@ + + setuptools==77.0.3 # this version can reuse CMake build dir + +-numba == 0.65.0 # Required for N-gram speculative decoding ++numba == 0.65.0; platform_machine != "riscv64" # Required for N-gram speculative decoding + + # Dependencies for CPUs + torch==2.13.0+cpu; platform_machine == "x86_64" or platform_machine == "s390x" or platform_machine == "aarch64" +-- +2.51.0 diff --git a/patches/vllm/0.31.0/0002-cpu-probe-the-torch-openblas-for-sbgemm.patch b/patches/vllm/0.31.0/0002-cpu-probe-the-torch-openblas-for-sbgemm.patch new file mode 100644 index 000000000000..33bed09ffa22 --- /dev/null +++ b/patches/vllm/0.31.0/0002-cpu-probe-the-torch-openblas-for-sbgemm.patch @@ -0,0 +1,70 @@ +From 0000000000000000000000000000000000000002 Mon Sep 17 00:00:00 2001 +From: RISE Project CI +Date: Sun, 20 Sep 2026 00:00:00 +0000 +Subject: [PATCH] [CPU] Probe the torch wheel's OpenBLAS for sbgemm_ + +cpu_extension.cmake decides whether to compile the OpenBLAS bf16 GEMM path +by globbing the installed torch wheel for libopenblas*.so*, and defines +VLLM_HAS_OPENBLAS when the glob hits. Presence of the library is not the +same thing as presence of the symbol: blas_gemm() in +csrc/cpu/sgl-kernels/blas_gemm.h calls sbgemm_, and OpenBLAS only builds +sbgemm_ for the architectures it has BFLOAT16 kernels for (x86_64 +Cooper Lake and later, ARM64 Neoverse, POWER10, z14 and later). RISC-V is +not one of them. + +The miss cannot be caught at link time, because the comment right above +the glob is accurate - _C deliberately does not link OpenBLAS and expects +libtorch.so to have loaded it RTLD_GLOBAL - so a shared object with an +unresolved sbgemm_ links happily and the build succeeds. It fails on the +first "import vllm._C" instead: + + ImportError: .../vllm/_C.abi3.so: undefined symbol: sbgemm_ + +blas_gemm.h already carries the right fallback for this case: its #else +branch dispatches through at::native::cpublas::gemm_no_downcast_stub, +which libtorch_cpu.so exports on every architecture. So ask the library +whether it actually provides sbgemm_ and take that fallback when it does +not, rather than assuming every non-x86 torch wheel ships a bf16-capable +OpenBLAS. When the probe itself cannot run the previous behaviour is kept, +so no platform that works today changes. + +Upstream-Status: To upstream [Architecture-agnostic: the probe is a no-op wherever OpenBLAS does export sbgemm_, and it fixes any platform whose torch wheel ships an OpenBLAS built without BUILD_BFLOAT16, riscv64 being the one this repo builds for] + +Signed-off-by: RISE Project CI +--- + cmake/cpu_extension.cmake | 20 ++++++++++++++++++-- + 1 file changed, 18 insertions(+), 2 deletions(-) + +diff --git a/cmake/cpu_extension.cmake b/cmake/cpu_extension.cmake +index 92c3daa..a7a229c 100644 +--- a/cmake/cpu_extension.cmake ++++ b/cmake/cpu_extension.cmake +@@ -430,8 +430,24 @@ if (NOT ENABLE_X86_ISA) + "${TORCH_INSTALL_PREFIX}/lib/libopenblas*.so*") + # Note: we don't link openblas directly to _C extension, as it's available through libtorch.so + if (_VLLM_TORCH_OPENBLAS_LIBS) +- list(GET _VLLM_TORCH_OPENBLAS_LIBS 0 VLLM_OPENBLAS_LIB) +- message(STATUS "CPU OpenBLAS library: ${VLLM_OPENBLAS_LIB}") ++ list(GET _VLLM_TORCH_OPENBLAS_LIBS 0 _VLLM_TORCH_OPENBLAS_LIB) ++ # blas_gemm() calls OpenBLAS' bf16 GEMM, which OpenBLAS only builds for ++ # the architectures it has BFLOAT16 kernels for. Elsewhere the library is ++ # present but sbgemm_ is not, and since _C resolves it through ++ # libtorch.so instead of linking OpenBLAS itself, the miss is not ++ # reported until the extension is imported. Probe for the symbol and keep ++ # the PyTorch reference BLAS path when it is absent. ++ execute_process( ++ COMMAND ${CMAKE_NM} --dynamic --defined-only "${_VLLM_TORCH_OPENBLAS_LIB}" ++ OUTPUT_VARIABLE _VLLM_TORCH_OPENBLAS_SYMBOLS ++ ERROR_QUIET) ++ if (_VLLM_TORCH_OPENBLAS_SYMBOLS STREQUAL "" OR ++ _VLLM_TORCH_OPENBLAS_SYMBOLS MATCHES "[ \t]sbgemm_[\r\n]") ++ set(VLLM_OPENBLAS_LIB "${_VLLM_TORCH_OPENBLAS_LIB}") ++ message(STATUS "CPU OpenBLAS library: ${VLLM_OPENBLAS_LIB}") ++ else() ++ message(STATUS "CPU OpenBLAS library ${_VLLM_TORCH_OPENBLAS_LIB} provides no sbgemm_, using the PyTorch reference BLAS path") ++ endif() + endif() + endif() + +-- +2.51.0 diff --git a/patches/vllm/0.32.0/0001-requirements-do-not-require-numba-on-riscv64.patch b/patches/vllm/0.32.0/0001-requirements-do-not-require-numba-on-riscv64.patch new file mode 100644 index 000000000000..8e6a65265aba --- /dev/null +++ b/patches/vllm/0.32.0/0001-requirements-do-not-require-numba-on-riscv64.patch @@ -0,0 +1,42 @@ +From 0000000000000000000000000000000000000001 Mon Sep 17 00:00:00 2001 +From: RISE Project CI +Date: Sat, 19 Sep 2026 00:00:00 +0000 +Subject: [PATCH] [CPU] Do not require numba on riscv64 + +requirements/cpu.txt requires numba on every CPU architecture except +s390x. numba has no riscv64 wheel on any index, and it cannot be built +from its sdist either: it hard-depends on llvmlite, whose build links a +patched LLVM distributed only as a conda package on a channel that +publishes no linux-riscv64 subdir. An unpatched riscv64 CPU wheel is +therefore buildable but not installable. + +numba is optional to vLLM by construction - vllm/utils/import_utils.py +declares is_numba_available() "Whether the optional `numba` package is +available" and the Kimi-K2.5 fused vision processor falls back to the +remote HF processor when it is absent. Excluding it costs riscv64 the +same features s390x already goes without (n-gram speculative decoding +and the fused vision/inkling processors), which is exactly the +configuration upstream already ships for s390x. + +Upstream-Status: To upstream [vLLM publishes no riscv64 wheels and runs no riscv64 CI, so there is no upstream build this marker would affect yet; resubmit once riscv64 joins docs/getting_started/installation/cpu.md and the CPU release matrix] + +Signed-off-by: RISE Project CI +--- + requirements/cpu.txt | 2 +- + 1 file changed, 1 insertion(+), 1 deletion(-) + +diff --git a/requirements/cpu.txt b/requirements/cpu.txt +index 0570517..8037ccd 100644 +--- a/requirements/cpu.txt ++++ b/requirements/cpu.txt +@@ -3,7 +3,7 @@ + + setuptools==78.1.1 # this version can reuse CMake build dir; >=78.1.1 for GHSA-5rjg-fvgr-3xxf + +-numba == 0.65.0 # Required for N-gram speculative decoding ++numba == 0.65.0; platform_machine != "riscv64" # Required for N-gram speculative decoding + + # Dependencies for CPUs + torch==2.13.0+cpu; platform_machine == "x86_64" or platform_machine == "s390x" or platform_machine == "aarch64" +-- +2.51.0 diff --git a/patches/vllm/0.32.0/0002-cpu-probe-the-torch-openblas-for-sbgemm.patch b/patches/vllm/0.32.0/0002-cpu-probe-the-torch-openblas-for-sbgemm.patch new file mode 100644 index 000000000000..33bed09ffa22 --- /dev/null +++ b/patches/vllm/0.32.0/0002-cpu-probe-the-torch-openblas-for-sbgemm.patch @@ -0,0 +1,70 @@ +From 0000000000000000000000000000000000000002 Mon Sep 17 00:00:00 2001 +From: RISE Project CI +Date: Sun, 20 Sep 2026 00:00:00 +0000 +Subject: [PATCH] [CPU] Probe the torch wheel's OpenBLAS for sbgemm_ + +cpu_extension.cmake decides whether to compile the OpenBLAS bf16 GEMM path +by globbing the installed torch wheel for libopenblas*.so*, and defines +VLLM_HAS_OPENBLAS when the glob hits. Presence of the library is not the +same thing as presence of the symbol: blas_gemm() in +csrc/cpu/sgl-kernels/blas_gemm.h calls sbgemm_, and OpenBLAS only builds +sbgemm_ for the architectures it has BFLOAT16 kernels for (x86_64 +Cooper Lake and later, ARM64 Neoverse, POWER10, z14 and later). RISC-V is +not one of them. + +The miss cannot be caught at link time, because the comment right above +the glob is accurate - _C deliberately does not link OpenBLAS and expects +libtorch.so to have loaded it RTLD_GLOBAL - so a shared object with an +unresolved sbgemm_ links happily and the build succeeds. It fails on the +first "import vllm._C" instead: + + ImportError: .../vllm/_C.abi3.so: undefined symbol: sbgemm_ + +blas_gemm.h already carries the right fallback for this case: its #else +branch dispatches through at::native::cpublas::gemm_no_downcast_stub, +which libtorch_cpu.so exports on every architecture. So ask the library +whether it actually provides sbgemm_ and take that fallback when it does +not, rather than assuming every non-x86 torch wheel ships a bf16-capable +OpenBLAS. When the probe itself cannot run the previous behaviour is kept, +so no platform that works today changes. + +Upstream-Status: To upstream [Architecture-agnostic: the probe is a no-op wherever OpenBLAS does export sbgemm_, and it fixes any platform whose torch wheel ships an OpenBLAS built without BUILD_BFLOAT16, riscv64 being the one this repo builds for] + +Signed-off-by: RISE Project CI +--- + cmake/cpu_extension.cmake | 20 ++++++++++++++++++-- + 1 file changed, 18 insertions(+), 2 deletions(-) + +diff --git a/cmake/cpu_extension.cmake b/cmake/cpu_extension.cmake +index 92c3daa..a7a229c 100644 +--- a/cmake/cpu_extension.cmake ++++ b/cmake/cpu_extension.cmake +@@ -430,8 +430,24 @@ if (NOT ENABLE_X86_ISA) + "${TORCH_INSTALL_PREFIX}/lib/libopenblas*.so*") + # Note: we don't link openblas directly to _C extension, as it's available through libtorch.so + if (_VLLM_TORCH_OPENBLAS_LIBS) +- list(GET _VLLM_TORCH_OPENBLAS_LIBS 0 VLLM_OPENBLAS_LIB) +- message(STATUS "CPU OpenBLAS library: ${VLLM_OPENBLAS_LIB}") ++ list(GET _VLLM_TORCH_OPENBLAS_LIBS 0 _VLLM_TORCH_OPENBLAS_LIB) ++ # blas_gemm() calls OpenBLAS' bf16 GEMM, which OpenBLAS only builds for ++ # the architectures it has BFLOAT16 kernels for. Elsewhere the library is ++ # present but sbgemm_ is not, and since _C resolves it through ++ # libtorch.so instead of linking OpenBLAS itself, the miss is not ++ # reported until the extension is imported. Probe for the symbol and keep ++ # the PyTorch reference BLAS path when it is absent. ++ execute_process( ++ COMMAND ${CMAKE_NM} --dynamic --defined-only "${_VLLM_TORCH_OPENBLAS_LIB}" ++ OUTPUT_VARIABLE _VLLM_TORCH_OPENBLAS_SYMBOLS ++ ERROR_QUIET) ++ if (_VLLM_TORCH_OPENBLAS_SYMBOLS STREQUAL "" OR ++ _VLLM_TORCH_OPENBLAS_SYMBOLS MATCHES "[ \t]sbgemm_[\r\n]") ++ set(VLLM_OPENBLAS_LIB "${_VLLM_TORCH_OPENBLAS_LIB}") ++ message(STATUS "CPU OpenBLAS library: ${VLLM_OPENBLAS_LIB}") ++ else() ++ message(STATUS "CPU OpenBLAS library ${_VLLM_TORCH_OPENBLAS_LIB} provides no sbgemm_, using the PyTorch reference BLAS path") ++ endif() + endif() + endif() + +-- +2.51.0 From dea706621548b779d53098702b3900c429b4ae37 Mon Sep 17 00:00:00 2001 From: Ludovic Henry Date: Sun, 25 Oct 2026 19:02:45 +0100 Subject: [PATCH 3/4] vllm: provide a Rust toolchain for the test stage --- .github/workflows/build-vllm.yml | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/.github/workflows/build-vllm.yml b/.github/workflows/build-vllm.yml index 70971056cabb..ca7e1428db0d 100644 --- a/.github/workflows/build-vllm.yml +++ b/.github/workflows/build-vllm.yml @@ -117,9 +117,18 @@ jobs: --exclude libtorch_global_deps.so --exclude libc10.so --exclude libgomp.so.1 + # Installing the vllm wheel pulls oss-harmony, which has no riscv64 + # wheel and is built from its Rust sdist. BEFORE_ALL installs rustup + # with `--default-toolchain none` (the build needs no compiler), so + # cargo has no toolchain to run ("rustup could not choose a version of + # cargo"): install one for the test stage and put cargo on PATH. + CIBW_BEFORE_TEST: >- + rustup toolchain install stable --profile minimal + && rustup default stable CIBW_TEST_ENVIRONMENT: >- PIP_EXTRA_INDEX_URL=https://pypi.riseproject.dev/simple/ VLLM_TARGET_DEVICE=cpu + PATH="$PATH:$HOME/.cargo/bin" CIBW_TEST_COMMAND: >- python -c "import vllm, vllm._C, torch; from vllm.platforms import current_platform; From f3723cddb41ce848f1830675d1cc4b796f73ece0 Mon Sep 17 00:00:00 2001 From: Ludovic Henry Date: Mon, 26 Oct 2026 09:23:48 +0100 Subject: [PATCH 4/4] vllm: install pillow wheel-only in the test stage Installing the vllm wheel pulls pillow, whose newest PyPI release has no riscv64 wheel; pip fell back to the sdist and failed for lack of jpeg headers. The registry has the wheel, so only-binary keeps pip on it. --- .github/workflows/build-vllm.yml | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/.github/workflows/build-vllm.yml b/.github/workflows/build-vllm.yml index ca7e1428db0d..fea69b926359 100644 --- a/.github/workflows/build-vllm.yml +++ b/.github/workflows/build-vllm.yml @@ -125,8 +125,12 @@ jobs: CIBW_BEFORE_TEST: >- rustup toolchain install stable --profile minimal && rustup default stable + # Installing the vllm wheel also pulls pillow, wheel-only: its newest PyPI + # release has no riscv64 wheel (the registry has one), and the sdist can't + # build without jpeg headers. CIBW_TEST_ENVIRONMENT: >- PIP_EXTRA_INDEX_URL=https://pypi.riseproject.dev/simple/ + PIP_ONLY_BINARY=pillow VLLM_TARGET_DEVICE=cpu PATH="$PATH:$HOME/.cargo/bin" CIBW_TEST_COMMAND: >-