Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
185 changes: 185 additions & 0 deletions .github/workflows/build-daal.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,185 @@
# SPDX-FileCopyrightText: 2026 The RISE Project
# SPDX-License-Identifier: MIT
---
# PyPI's daal wheel is Intel's own prebuilt x86_64/win_amd64-only binary (Intel Simplified
# Software License, no sdist) that statically bakes in closed oneMKL kernels
# (mkl_sparse_*_avx512 symbols in libonedal_core.so.4, built with the closed Intel oneAPI
# DPC++/C++ Compiler). Unlike the parked onemkl-sycl-*/mkl family, oneDAL itself is
# Apache-2.0 and has a real non-x86 "ref" backend with no SIMD/MKL dependency: a RISCV64
# CpuType in its CPU-dispatch layer (cpp/daal/AGENTS.md), dedicated
# dev/make/function_definitions/{riscv64,lnxriscv64}.mk and
# compiler_definitions/clang.ref.riscv64.mk files, and an exercised upstream Azure Pipelines
# job (LinuxMakeLLVM_OpenBLAS_rv64 in .ci/pipeline/ci.yml) that cross-compiles+QEMU-tests
# `make daal` and `make onedal_c` with BACKEND_CONFIG=ref against OpenBLAS+oneTBB. This
# workflow runs that same recipe natively (we have real riscv64 runners, so no
# cross-compile/sysroot/QEMU) instead of redistributing Intel's x86_64 binary.
# Based on: https://github.com/uxlfoundation/oneDAL (no upstream CI builds this *wheel*, but
# .ci/pipeline/ci.yml's rv64 job builds this same `daal`+`onedal_c` library pair).
name: Build daal wheels (riscv64)

on:
workflow_dispatch:
inputs:
version:
description: 'Version glob to (re)build; empty builds every version of docs/packages/daal.yaml not released yet'
required: false
default: ''
pull_request:
branches: [main]
paths:
- '.github/workflows/build-daal.yml'
- 'docs/packages/daal.yaml'
- 'patches/daal/**'
push:
branches: [main]
paths:
- '.github/workflows/build-daal.yml'
- 'docs/packages/daal.yaml'
- 'patches/daal/**'

concurrency:
group: ${{ github.workflow }}-${{ github.head_ref || github.run_id }}
cancel-in-progress: true

permissions:
contents: read # to fetch code (actions/checkout)

env:
OPENBLAS_VERSION: v0.3.29
TBB_VERSION: v2023.0.0
MANYLINUX_RISCV64_IMAGE: quay.io/pypa/manylinux_2_39_riscv64

jobs:
setup:
uses: $/.github/workflows/_setup.yml
with:
package: daal
version: ${{ inputs.version }}

build_wheels:
needs: [setup]
if: needs.setup.outputs.versions != '[]'
strategy:
fail-fast: false
matrix:
version: ${{ fromJSON(needs.setup.outputs.versions) }}
name: Build daal ${{ matrix.version }} py2.py3-none-manylinux_riscv64
runs-on: ubuntu-24.04-riscv
timeout-minutes: 300

env:
DAAL_VERSION: ${{ matrix.version }}

steps:
- name: Checkout oneDAL ${{ env.DAAL_VERSION }}
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
repository: uxlfoundation/oneDAL
ref: ${{ env.DAAL_VERSION }}
persist-credentials: false

- name: Checkout python-wheels
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
path: python-wheels
persist-credentials: false

- name: Patch oneDAL source
run: git apply -v python-wheels/patches/daal/${{ env.DAAL_VERSION }}/*.patch

- name: Pull manylinux_riscv64 image
run: docker pull "${MANYLINUX_RISCV64_IMAGE}"

# The `ref` backend statically copies OpenBLAS's symbols into libonedal_core
# (dev/make/deps.ref.mk) and needs a static libopenblas.a with the ILP64 interface;
# NO_FORTRAN=1 matches oneDAL's own .ci/env/openblas.sh (oneDAL never calls into
# OpenBLAS's Fortran/LAPACK layer, so this drops the libgfortran dependency entirely).
# TBB is linked dynamically (dev/make/function_definitions/lnxriscv64.mk) and is not
# bundled: the wheel declares Requires-Dist tbb==2023.* like Intel's own, satisfied by
# this registry's existing tbb port.
- name: Build OpenBLAS, oneTBB, then daal+onedal_c, and package the wheel
run: |
docker run --rm -v "$(pwd):/work" -w /work -e DAAL_VERSION -e OPENBLAS_VERSION -e TBB_VERSION "${MANYLINUX_RISCV64_IMAGE}" bash -c '
set -euxo pipefail
dnf -y --enablerepo=crb install clang

git clone --depth 1 --branch "${OPENBLAS_VERSION}" https://github.com/OpenMathLib/OpenBLAS /work/openblas-src
make -C /work/openblas-src -j"$(nproc)" NO_FORTRAN=1 USE_OPENMP=0 USE_THREAD=0 USE_LOCKING=1 BINARY=64 INTERFACE64=1
make -C /work/openblas-src install PREFIX=/work/openblas-install
cp /work/openblas-src/LICENSE /work/LICENSE-openblas.txt

# -DCMAKE_INSTALL_LIBDIR=lib: GNUInstallDirs defaults to lib64 on this Rocky-based
# image (gotcha 377), but the oneDAL makefile (TBBDIR.libia.prefix) hardcodes "lib".
git clone --depth 1 --branch "${TBB_VERSION}" https://github.com/uxlfoundation/oneTBB /work/tbb-src
cmake -S /work/tbb-src -B /work/tbb-build -DCMAKE_BUILD_TYPE=Release -DTBB_TEST=OFF -DTBB_STRICT_PROTOTYPES=OFF -DCMAKE_INSTALL_LIBDIR=lib -DCMAKE_INSTALL_PREFIX=/work/tbb-install
cmake --build /work/tbb-build -j"$(nproc)"
cmake --install /work/tbb-build

export CC=clang CXX=clang++
.ci/scripts/build.sh --compiler clang --optimizations rv64 --target daal --backend-config ref --plat lnxriscv64 --blas-dir /work/openblas-install --tbb-dir /work/tbb-install --jobs "$(nproc)"
.ci/scripts/build.sh --compiler clang --optimizations rv64 --target onedal_c --backend-config ref --plat lnxriscv64 --blas-dir /work/openblas-install --tbb-dir /work/tbb-install --jobs "$(nproc)"

RELEASE_LIB_DIR=__release_lnx_clang/daal/latest/lib/riscv64
ls -la "${RELEASE_LIB_DIR}"
mkdir -p dist_libs
cp -L "${RELEASE_LIB_DIR}"/libonedal*.so* dist_libs/
strip dist_libs/*.so*
file dist_libs/*.so*
export LD_LIBRARY_PATH=/work/tbb-install/lib:/work/tbb-install/lib64
for f in dist_libs/*.so*; do ldd "$f"; done
cat > setup.py <<PYEOF
import glob
import os
from setuptools import setup
setup(
name="daal",
version=os.environ["DAAL_VERSION"],
description="Intel(R) oneAPI Data Analytics Library",
long_description="oneDAL riscv64 dynamic libraries for Linux, built from the Apache-2.0 licensed oneDAL source using its ref/OpenBLAS backend (dev/make/function_definitions/lnxriscv64.mk), not Intel'"'"'s closed-MKL x86_64 binary.",
url="https://github.com/uxlfoundation/oneDAL",
license="Apache-2.0",
license_files=["LICENSE", "third-party-programs.txt", "LICENSE-openblas.txt"],
classifiers=[
"Operating System :: POSIX :: Linux",
"Topic :: Software Development :: Libraries",
],
install_requires=["tbb==2023.*"],
packages=[],
data_files=[("lib", sorted(glob.glob("dist_libs/*.so*")))],
)
PYEOF
pybin=/opt/python/cp312-cp312/bin
"$pybin/pip" install -q -U setuptools wheel
"$pybin/python3" setup.py bdist_wheel --python-tag py2.py3 --plat-name manylinux_2_39_riscv64
PIP_EXTRA_INDEX_URL=https://pypi.riseproject.dev/simple/ PIP_ONLY_BINARY=tbb "$pybin/pip" install -q dist/*.whl
"$pybin/python3" - <<PYEOF
import ctypes, os, sysconfig
libdir = os.path.join(sysconfig.get_path("data"), "lib")
for name in ("libonedal_core.so.4", "libonedal_thread.so.4", "libonedal.so.4", "libonedal_parameters.so.4"):
ctypes.CDLL(os.path.join(libdir, name))
print("loaded", name)
PYEOF
'

- uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: daal-${{ env.DAAL_VERSION }}-py2.py3-none-manylinux_riscv64
path: dist/*.whl
if-no-files-found: error

publish:
name: Publish daal ${{ matrix.version }}
needs: [setup, build_wheels]
if: needs.setup.outputs.versions != '[]'
strategy:
fail-fast: false
matrix:
version: ${{ fromJSON(needs.setup.outputs.versions) }}
permissions:
contents: write
pull-requests: write
uses: $/.github/workflows/_publish-wheel.yml
secrets:
app-private-key: ${{ secrets.RISEPROJECT_APP_PRIVATE_KEY }}
with:
artifact-pattern: daal-${{ matrix.version }}-*-manylinux_riscv64
5 changes: 5 additions & 0 deletions docs/packages/daal.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
package-name: daal
source-code: https://github.com/uxlfoundation/oneDAL
license: Apache-2.0
versions:
- version: 2026.1.0
Original file line number Diff line number Diff line change
@@ -0,0 +1,62 @@
From 0000000000000000000000000000000000000000 Mon Sep 17 00:00:00 2001
From: Ludovic Henry <git@ludovic.dev>
Date: Fri, 23 Oct 2026 00:00:00 +0000
Subject: [PATCH] identify native riscv64 hosts as lnxriscv64

dev/make/identify_os.sh only knows x86_64 and aarch64, so on a native
riscv64 Linux host it prints "Unknown architecture: riscv64". The
lnxriscv64 build is otherwise complete (dev/make/function_definitions/
lnxriscv64.mk, compiler_definitions/clang.ref.riscv64.mk), but upstream
CI only builds it cross-compiled from x86_64, where the host identifies
as lnx32e and the gap is never hit. Natively, two things break:

- `make daal PLAT=lnxriscv64` fails in _release_nuspec, because
deploy/nuget/prepare_dal_nuget.sh calls identify_os.sh itself and has
no lnxriscv64 branch either:

./deploy/nuget/prepare_dal_nuget.sh: line 56: [: too many arguments
Unknown platform Unknown architecture: riscv64
make: *** [makefile:1109: _release_nuspec] Error 1

- the makefile compares PLAT against IDENTIFIED_PLAT to set
COMPILER_is_cross, so a native build is treated as a cross build and
clang is given --target=riscv64-linux-gnu instead of its host triple.

Map riscv64 to lnxriscv64 in identify_os.sh, and give
prepare_dal_nuget.sh a lnxriscv64 branch mirroring lnxarm with the
lib/riscv64 release directory the makefile already uses
(ARCH_DIR_ONEDAL = riscv64).

Upstream-Status: To upstream [not yet submitted; upstream CI only cross-compiles lnxriscv64 from x86_64, so this native-host fix needs raising on its own]

Signed-off-by: Ludovic Henry <git@ludovic.dev>
---
diff --git a/deploy/nuget/prepare_dal_nuget.sh b/deploy/nuget/prepare_dal_nuget.sh
--- a/deploy/nuget/prepare_dal_nuget.sh
+++ b/deploy/nuget/prepare_dal_nuget.sh
@@ -67,6 +67,13 @@ create_package() {
dynamic_lib_path=lib/arm
static_lib_path=lib/arm
lib_prefix=libonedal
+ elif [ ${platform} = "lnxriscv64" ]; then
+ platform=linux
+ tbb_platform=linux
+ rls_prefix=${rls_dir}/daal/latest
+ dynamic_lib_path=lib/riscv64
+ static_lib_path=lib/riscv64
+ lib_prefix=libonedal

elif [ ${platform} = "mac32e" ]; then
platform=osx-x64
diff --git a/dev/make/identify_os.sh b/dev/make/identify_os.sh
--- a/dev/make/identify_os.sh
+++ b/dev/make/identify_os.sh
@@ -22,6 +22,8 @@ if [ "${os}" = "Linux" ]; then
echo lnx32e
elif [ "${ARCH}" = "aarch64" ]; then
echo lnxarm
+ elif [ "${ARCH}" = "riscv64" ]; then
+ echo lnxriscv64
else
echo "Unknown architecture: ${ARCH}"
exit 1
Original file line number Diff line number Diff line change
@@ -0,0 +1,42 @@
From 0000000000000000000000000000000000000000 Mon Sep 17 00:00:00 2001
From: Ludovic Henry <git@ludovic.dev>
Date: Sun, 25 Oct 2026 00:00:00 +0000
Subject: [PATCH] silence unused --gcc-toolchain flag on native riscv64 clang
builds

The top-level makefile always appends `--gcc-toolchain=$(GCC_TOOLCHAIN_PATH)`
to every C.COMPILE invocation (dev/make/common.mk), with
GCC_TOOLCHAIN_PATH derived from `which gcc` (resolving to /usr in the
manylinux_riscv64 image). On this native riscv64 host, with no
`--target=` passed (clang.ref.riscv64.mk only adds one when
COMPILER_is_cross is "yes"), clang's riscv64 driver never consults the
GCC installation for the compile-only (`-c`) step the flag is passed
on, so it is flagged as unused:

clang++: error: argument unused during compilation:
'--gcc-toolchain=/usr' [-Werror,-Wunused-command-line-argument]
make: *** [makefile:567: .../daal_strings.o] Error 1

-Werror (set right alongside in COMPILER.lnx.clang) turns that into a
hard build failure for every translation unit. Upstream already works
around the identical situation for icx's -static-libstdc++ flag
(COMPILER.lnx.icx in dev/make/compiler_definitions/icx.mkl.32e.mk also
pairs -Werror with a `-Wno-unused-command-line-argument` for a flag
that is unused at compile time but still needed at link time); do the
same here instead of dropping -Werror or the toolchain flag.

Upstream-Status: Inappropriate [this build-system flag combination is only exercised by a native (non-cross) riscv64 host, which upstream CI never runs]

Signed-off-by: Ludovic Henry <git@ludovic.dev>
---
diff --git a/dev/make/compiler_definitions/clang.ref.riscv64.mk b/dev/make/compiler_definitions/clang.ref.riscv64.mk
--- a/dev/make/compiler_definitions/clang.ref.riscv64.mk
+++ b/dev/make/compiler_definitions/clang.ref.riscv64.mk
@@ -37,6 +37,7 @@ COMPILER.sysroot = $(if $(SYSROOT),--sysroot $(SYSROOT))

COMPILER.lnx.clang= clang++ \
-DDAAL_REF -DONEDAL_REF -DDAAL_CPU=rv64 -Werror -Wreturn-type \
+ -Wno-unused-command-line-argument \
$(COMPILER.lnx.clang.target) \
$(COMPILER.sysroot)

Loading