Skip to content

daal: Add version 2026.1.0 - #2971

Draft
luhenry wants to merge 5 commits into
mainfrom
daal
Draft

luhenry wants to merge 5 commits into
mainfrom
daal

Conversation

@luhenry

@luhenry luhenry commented Oct 13, 2026 •

Copy link
Copy Markdown
Member

Compiles oneDAL's daal+onedal_c libraries against OpenBLAS and oneTBB using its own "ref" CPU-dispatch backend (no SIMD/MKL). Upstream publishes no riscv64 wheel.

Mirrors oneDAL's own riscv64 Azure Pipelines job (LinuxMakeLLVM_OpenBLAS_rv64 in .ci/pipeline/ci.yml).

Differs from upstream

  • Builds natively instead of cross-compiling with a QEMU-tested sysroot - this runner fleet is riscv64 already.
  • Uses source-built OpenBLAS (NO_FORTRAN=1), not Intel's closed oneMKL; PyPI's own wheel bundles closed MKL with no riscv64 build.

Testing

  • loads each built .so with ctypes instead of upstream's QEMU-run C++ examples (no Python bindings in this wheel).

License: Statically links OpenBLAS (BSD-3-Clause); its LICENSE ships alongside oneDAL's Apache-2.0 one.

Built on cp312 (library-only wheel, no interpreter matrix).

PyPI's daal wheel is Intel's own prebuilt x86_64/win_amd64-only binary
under the Intel Simplified Software License with no sdist, and
statically bakes in closed oneMKL kernels. oneDAL itself is Apache-2.0
and ships a real non-x86 "ref" backend (RISCV64 CpuType, dedicated
dev/make/function_definitions/{riscv64,lnxriscv64}.mk and
clang.ref.riscv64.mk, an exercised upstream Azure Pipelines rv64 job)
with no SIMD/MKL dependency, built here against OpenBLAS (statically
linked, NO_FORTRAN=1) and oneTBB (dynamically linked, satisfied at
install time by this registry's existing tbb==2023.* port) instead.
@github-actions

Copy link
Copy Markdown
Contributor
PR Preview Action v1.8.1

QR code for preview link

🚀 View preview at
https://riseproject-dev.github.io/python-wheels/pr-preview/pr-2971/

Built to branch gh-pages at 2026-10-13 14:10 UTC.
Preview will be ready when the GitHub Pages deployment is complete.

The riscv64 CI job failed with "Can't find TBB runtimes nether in
/work/tbb-install/lib not in ...": oneTBB's CMake build installs under
GNUInstallDirs' CMAKE_INSTALL_LIBDIR, which defaults to lib64 on the
Rocky-based manylinux_2_39_riscv64 image (gotcha 377), while oneDAL's
makefile hardcodes TBBDIR.libia.prefix to "$TBBROOT/lib". Pinning
-DCMAKE_INSTALL_LIBDIR=lib on the oneTBB configure matches OpenBLAS's
own install layout and what oneDAL's build looks for.

luhenry commented Oct 16, 2026 •

Copy link
Copy Markdown
Member Author

CI root-caused and fixed.

What failed: Build daal 2026.1.0 py2.py3-none-manylinux_riscv64 died during make daal with:

makefile:254: *** Can`t find TBB runtimes nether in /work/tbb-install/lib not in /work/tbb-install/lib.. Stop.

Root cause: the oneTBB build step configures with plain CMake and no CMAKE_INSTALL_LIBDIR override. On the Rocky-based manylinux_2_39_riscv64 image, GNUInstallDirs defaults that to lib64, so cmake --install put the TBB shared libs under /work/tbb-install/lib64 (confirmed in the log: -- Installing: /work/tbb-install/lib64/libtbb.so...). oneDAL's own top-level makefile hardcodes TBBDIR.libia.prefix := $(TBBDIR.2)/lib when resolving TBBROOT, so it never looked in lib64 and bailed. This is the same class of issue as gotcha 377 in this repo's porting skill (Rocky's lib64 default biting a hardcoded lib elsewhere) — not an upstream oneDAL bug and not something that would affect other architectures (our own OpenBLAS build already installs to lib on this same image).

Fix: pin -DCMAKE_INSTALL_LIBDIR=lib on the oneTBB cmake -S ... -B ... configure line in build-daal.yml, matching OpenBLAS's own install layout and what oneDAL's makefile expects. Pushed as fb82dd6020d.

The oneTBB step failed with "fatal: could not create leading
directories of '/work/tbb-src': Permission denied". The comment added
in the previous commit sits inside the single-quoted `bash -c '...'`
script passed to docker run, and its apostrophe ("oneDAL's") closed
that quote early. The container script therefore ended right after the
OpenBLAS build, and the runner's own shell ran the oneTBB `git clone`
on the host, where /work does not exist. Reword the comment without
apostrophes so the whole script is passed to the container again.
The build failed in the release step of `make daal`, before compiling
anything: dev/make/identify_os.sh only maps x86_64 and aarch64, so on the
native riscv64 runner it prints "Unknown architecture: riscv64", and the
_release_nuspec target runs deploy/nuget/prepare_dal_nuget.sh, which
exits with "Unknown platform Unknown architecture: riscv64". Upstream
never hits this because its riscv64 CI cross-compiles from x86_64.

Carry a patch that makes identify_os.sh print lnxriscv64 on riscv64 and
gives prepare_dal_nuget.sh a matching lnxriscv64 branch, and apply it
before the container build. It also makes the makefile treat the build
as native (PLAT == IDENTIFIED_PLAT), so clang no longer gets
--target=riscv64-linux-gnu.

Also pass packages=[] to the generated setup.py: without it setuptools
auto-discovery runs in the oneDAL checkout root and stops with "Multiple
top-level packages discovered in a flat-layout: ['dev', 'cpp', 'data',
'cmake', 'deploy', 'samples', 'dist_libs']".
The build got past the identify_os.sh fix and into the real compile
step, then failed on every core_static translation unit:

    clang++: error: argument unused during compilation:
    '--gcc-toolchain=/usr' [-Werror,-Wunused-command-line-argument]
    make: *** [makefile:567: .../daal_strings.o] Error 1

oneDAL's top-level makefile always appends
`--gcc-toolchain=$(GCC_TOOLCHAIN_PATH)` to every compile invocation
(dev/make/common.mk), with GCC_TOOLCHAIN_PATH derived from `which gcc`
(/usr in the manylinux_riscv64 image). Now that the host is treated as
native rather than cross (previous commit), clang.ref.riscv64.mk no
longer adds a `--target=` flag, and clang's riscv64 driver never
consults the GCC installation during the compile-only (-c) step the
flag is passed on, so it flags the argument as unused; COMPILER.lnx.clang
pairs that with -Werror, turning it into a hard failure.

Carry a patch adding -Wno-unused-command-line-argument next to
-Werror in clang.ref.riscv64.mk, mirroring how icx.mkl.32e.mk already
silences the identical situation for its own -static-libstdc++ flag
(also unused at compile time, needed at link time), rather than
dropping -Werror or the toolchain flag.
luhenry added a commit that referenced this pull request Oct 25, 2026
…arning

The build_onedal job failed in the Build oneDAL (riscv64) check: the very
first clang++ invocations (cpp/daal/src/services/*.cpp) errored with:

    clang++: error: argument unused during compilation:
    '--gcc-toolchain=/usr' [-Werror,-Wunused-command-line-argument]

dev/make/common.mk derives --gcc-toolchain from `which gcc` and passes it
to every clang compile, and clang.ref.riscv64.mk's COMPILER.lnx.clang
turns -Werror on. On this riscv64 manylinux image, clang's
GCCInstallation probe does not treat that hint as applicable to every
translation unit (at least the services/*.cpp ones, which pull in no
libstdc++ headers), so the otherwise-harmless "unused argument" notice
becomes a hard build failure. The same --gcc-toolchain mechanism runs
unconditionally for every clang platform (x86_64, arm, ...); only
riscv64 hits this, consistent with clang's RISC-V GCCInstallation
detection being newer/less-trodden than the other architectures'.

Carry a second patch adding -Wno-unused-command-line-argument next to
the existing -Werror, so this one diagnostic stays non-fatal without
changing which headers/libraries get searched or used.

PR #2971 (daal) builds oneDAL the same way and hits the identical
failure; this only fixes this port's own copy of the recipe.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant