Repository navigation
Conversation
|
Before you submit for review:
If you did not complete any of these, then please explain below. |
r-devulap
left a comment
There was a problem hiding this comment.
LGTM. FYI, There are 5 kernels (hamming distance, min, max, sub and scale) that are currently in Panama but not in native, we should port them to native for the sake of completeness even though they never showed up as hotspots. I will have a PR out for that shortly.
|
|
||
| On Java 22+ with the Vector API enabled, JVector automatically prefers its Google | ||
| Highway native provider when the native library is available. Native libraries are | ||
| bundled for Linux x86-64 (SSE4.2 or newer) and Linux AArch64. Use these JVM options: |
There was a problem hiding this comment.
Just a quick note: native x86 support spent at least one release as an experimental feature before being enabled in production. In contrast, for AArch64 we're enabling it by default in production in the same release that introduces the feature.
|
See #753 |
MarkWolters
left a comment
There was a problem hiding this comment.
Willing to consider approval but at the least the exception handling strategy for failure to load the native library needs to be discussed.
| String maxIsa = (String) support.getClass().getMethod("getMaxIsaEnv").invoke(support); | ||
| LOG.info("Highway ISA tier: " + isa + ", Env variable JVECTOR_MAX_ISA: " + (maxIsa != null ? maxIsa : "(not set)")); | ||
| } catch (Exception e) { | ||
| LOG.info("Native Vector API enabled. Using NativeVectorizationProvider. (Unable to determine Highway ISA tier)"); |
There was a problem hiding this comment.
This becomes dangerous with the change in default behavior. An AI-assisted investigation (TL;DR we should be catching multiple Exception types here and behaving appropriately based on the Exception rather than just blindly returning the native provider):
getActiveIsa probe failures (VectorizationProvider.java:132-141)
getActiveIsa() is the first FFM downcall into libjvector. The provider constructor only checks os.arch and loads the library; it never calls into it. So an exception at line 134 means the library loaded but can't be called. It doesn't mean the ISA tier is unknown, because the C function just returns a static string. The catch (Exception e) logs at INFO and returns the native provider anyway.
Exception (wrapped in InvocationTargetException) |
Cause | Do the native kernels work? |
|---|---|---|
ExceptionInInitializerError caused by IllegalCallerException |
JDK 22/23 with --enable-native-access naming modules that don't include jvector's. The restricted call is in NativeSimdOps's static initializer (C_POINTER = ADDRESS.withTargetLayout(...)). |
No. The whole NativeSimdOps class fails to initialize, so every later native call throws NoClassDefFoundError. |
UnsatisfiedLinkError: unresolved symbol |
The library that loaded lacks the symbol, e.g. an older libjvector.so on java.library.path, which LibraryLoader tries before the bundled one. |
Maybe. Other symbols may be present, missing, or have different signatures. |
Reproduction
Built from the PR head on aarch64 Linux (Docker, eclipse-temurin:22-jdk and 25-jdk). The native library is a stub libjvector.so that exports jvector_simd_get_active_isa, jvector_simd_get_max_isa_env and dot_product_f32, loaded via -Djava.library.path. The probe program calls VectorizationProvider.getInstance(), then VectorUtil.dotProduct on two 4-element vectors.
| Scenario | JDK 22 | JDK 25 |
|---|---|---|
| A. Default flags | Native, ISA stub, dot = 10 ✅ |
Native, ISA stub, dot = 10 ✅ |
B. --enable-native-access=java.base |
Native provider returned; dot throws NoClassDefFoundError: Could not initialize class NativeSimdOps (cause: IllegalCallerException: Illegal native access from: unnamed module) ❌ |
Native, dot = 10 ✅ (JDK 24+ warns instead of throwing) |
C. Same as B + -Djvector.enable_native_vectorization=false |
Panama, dot = 10 ✅ | Panama, dot = 10 ✅ |
D. Library without jvector_simd_get_active_isa (stale library) |
Native, "Unable to determine Highway ISA tier", dot = 10 |
Same |
E. --illegal-native-access=deny |
n/a (option added in JDK 24) | AssertionError from VectorizationProvider.lookup (System.loadLibrary throws IllegalCallerException in LibraryLoader.loadJvector:53, which only catches UnsatisfiedLinkError) ❌ |
JDK 22, scenario B:
INFO: Native Vector API enabled. Using NativeVectorizationProvider. (Unable to determine Highway ISA tier)
PROVIDER = NativeVectorizationProvider
DOT FAILED: java.lang.NoClassDefFoundError: Could not initialize class io.github.jbellis.jvector.vector.cnative.NativeSimdOps
caused by: java.lang.ExceptionInInitializerError: Exception java.lang.IllegalCallerException: Illegal native access from: unnamed module
Takeaways
- B: the probe failure is real and leaves a provider that can't run any native call, while Panama works under the same flags (C). This was tolerable when native was opt-in; with native as the default, it affects users who never asked for it. Suggest treating a failed probe like a constructor
UnsupportedOperationException: log a warning and fall through to Panama. Keep the hard failure for-Djvector.vectorization_provider=native. - E: on JDK 24+, strict native access turns into an
AssertionErrorduring provider init, so jvector can't be used at all, even though Panama would work. JEP 472 says a future JDK will deny by default, so this will eventually happen without any flag.LibraryLoadershould handleIllegalCallerException(orlookupshould fall back on any non-forced native init failure). - D: the probe is a rough check. A missing probe symbol doesn't always mean the kernels are broken, but falling back is still the safe default.
Summary
Prefer the existing Google Highway native provider automatically on Java 22+ when the Vector API and native library are available. This changes provider selection, using the same native implementation that currently requires opt-in.
Closes #750.
Selection and dispatch
jvector.enable_native_vectorizationproperty totrue.jvector.experimental.enable_native_vectorizationas a deprecated compatibility alias; the new property takes precedence when both are set.jvector.vectorization_provider=native|panama|defaultprecedence and fail-fast behavior.-Djvector.enable_native_vectorization=falseas the automatic-native opt-out; the deprecated alias remains supported.The dispatch audit found that
NativeVectorUtilSupportalready calls all 22 exported native kernels. Operations without native implementations use inherited Panama code with native vector storage. No kernel implementation or CPU ISA selection changes are needed.JVECTOR_MAX_ISAkeeps its existing meaning; AVX3_DL/SPR currently inherit AVX3 kernel entries on main.README guidance documents selection precedence and the required Vector API/native-access JVM options. This PR is based on main and is independent of #744, #747, and the additional Highway optimizations.
Validation
Coverage includes default selection, both new and deprecated true/false properties, conflict precedence, forced provider precedence, absent native artifact, unsupported architecture, explicit native failure, and absent Vector API module. Each successful subprocess checks a distance calculation and inherited sum operation.
This draft makes no new performance or recall claims. The native route is unchanged; existing native-versus-Panama studies remain evidence for those implementations. Hosted CI and review remain required before merging.