Skip to content

Prefer Google Highway native SIMD by default - #751

Open
tlwillke wants to merge 3 commits into
mainfrom
native-by-default
Open

tlwillke wants to merge 3 commits into
mainfrom
native-by-default

Conversation

@tlwillke

@tlwillke tlwillke commented Oct 12, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

Prefer the existing Google Highway native provider automatically on Java 22+ when the Vector API and native library are available. This changes provider selection, using the same native implementation that currently requires opt-in.

Closes #750.

Selection and dispatch

  • Default the supported jvector.enable_native_vectorization property to true.
  • Retain jvector.experimental.enable_native_vectorization as a deprecated compatibility alias; the new property takes precedence when both are set.
  • Attempt automatic native selection only on Java 22+, matching the native provider's FFM requirement.
  • Preserve explicit jvector.vectorization_provider=native|panama|default precedence and fail-fast behavior.
  • Use -Djvector.enable_native_vectorization=false as the automatic-native opt-out; the deprecated alias remains supported.
  • Preserve Panama/scalar fallback when the native provider, library, runtime, or Vector API is unavailable.

The dispatch audit found that NativeVectorUtilSupport already calls all 22 exported native kernels. Operations without native implementations use inherited Panama code with native vector storage. No kernel implementation or CPU ISA selection changes are needed. JVECTOR_MAX_ISA keeps its existing meaning; AVX3_DL/SPR currently inherit AVX3 kernel entries on main.

README guidance documents selection precedence and the required Vector API/native-access JVM options. This PR is based on main and is independent of #744, #747, and the additional Highway optimizations.

Validation

Check Result
Java 23: fourteen fresh-JVM selection tests plus three existing provider/numeric tests 17 passed
Java 17, Java 11-compatible profile 13 passed; 4 SIMD-only cases skipped
Installed Java 21 runtime without the Vector API module Scalar fallback verified

Coverage includes default selection, both new and deprecated true/false properties, conflict precedence, forced provider precedence, absent native artifact, unsupported architecture, explicit native failure, and absent Vector API module. Each successful subprocess checks a distance calculation and inherited sum operation.

This draft makes no new performance or recall claims. The native route is unchanged; existing native-versus-Panama studies remain evidence for those implementations. Hosted CI and review remain required before merging.

@github-actions

github-actions Bot commented Oct 12, 2026 •

Copy link
Copy Markdown
Contributor

Before you submit for review:

  • Does your PR follow guidelines from CONTRIBUTIONS.md?
  • Did you summarize what this PR does clearly and concisely?
  • Did you include performance data for changes which may be performance impacting?
  • Did you include useful docs for any user-facing changes or features?
  • Did you include useful javadocs for developer oriented changes, explaining new concepts or key changes?
  • Did you rebase your branch onto the latest main for regression testing and PR submission?
  • Did you trigger regression testing via Run Bench Main and review results?
  • Did you adhere to the code formatting guidelines (TBD)
  • Did you group your changes for easy review, providing meaningful descriptions for each commit?
  • Did you ensure that all files contain the correct copyright header?
  • Did you add documentation for this feature to the release notes directory?

If you did not complete any of these, then please explain below.

@tlwillke tlwillke self-assigned this Oct 12, 2026
@tlwillke
tlwillke requested a review from r-devulap October 12, 2026 22:50
@tlwillke
tlwillke marked this pull request as ready for review October 12, 2026 22:50
@tlwillke tlwillke added the 4.0.2 Release label label Oct 12, 2026

@r-devulap r-devulap left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. FYI, There are 5 kernels (hamming distance, min, max, sub and scale) that are currently in Panama but not in native, we should port them to native for the sake of completeness even though they never showed up as hotspots. I will have a PR out for that shortly.

Comment thread README.md

On Java 22+ with the Vector API enabled, JVector automatically prefers its Google
Highway native provider when the native library is available. Native libraries are
bundled for Linux x86-64 (SSE4.2 or newer) and Linux AArch64. Use these JVM options:

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just a quick note: native x86 support spent at least one release as an experimental feature before being enabled in production. In contrast, for AArch64 we're enabling it by default in production in the same release that introduces the feature.

@r-devulap

Copy link
Copy Markdown
Contributor

See #753

@MarkWolters MarkWolters left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Willing to consider approval but at the least the exception handling strategy for failure to load the native library needs to be discussed.

String maxIsa = (String) support.getClass().getMethod("getMaxIsaEnv").invoke(support);
LOG.info("Highway ISA tier: " + isa + ", Env variable JVECTOR_MAX_ISA: " + (maxIsa != null ? maxIsa : "(not set)"));
} catch (Exception e) {
LOG.info("Native Vector API enabled. Using NativeVectorizationProvider. (Unable to determine Highway ISA tier)");

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This becomes dangerous with the change in default behavior. An AI-assisted investigation (TL;DR we should be catching multiple Exception types here and behaving appropriately based on the Exception rather than just blindly returning the native provider):

getActiveIsa probe failures (VectorizationProvider.java:132-141)

getActiveIsa() is the first FFM downcall into libjvector. The provider constructor only checks os.arch and loads the library; it never calls into it. So an exception at line 134 means the library loaded but can't be called. It doesn't mean the ISA tier is unknown, because the C function just returns a static string. The catch (Exception e) logs at INFO and returns the native provider anyway.

Exception (wrapped in InvocationTargetException) Cause Do the native kernels work?
ExceptionInInitializerError caused by IllegalCallerException JDK 22/23 with --enable-native-access naming modules that don't include jvector's. The restricted call is in NativeSimdOps's static initializer (C_POINTER = ADDRESS.withTargetLayout(...)). No. The whole NativeSimdOps class fails to initialize, so every later native call throws NoClassDefFoundError.
UnsatisfiedLinkError: unresolved symbol The library that loaded lacks the symbol, e.g. an older libjvector.so on java.library.path, which LibraryLoader tries before the bundled one. Maybe. Other symbols may be present, missing, or have different signatures.

Reproduction

Built from the PR head on aarch64 Linux (Docker, eclipse-temurin:22-jdk and 25-jdk). The native library is a stub libjvector.so that exports jvector_simd_get_active_isa, jvector_simd_get_max_isa_env and dot_product_f32, loaded via -Djava.library.path. The probe program calls VectorizationProvider.getInstance(), then VectorUtil.dotProduct on two 4-element vectors.

Scenario JDK 22 JDK 25
A. Default flags Native, ISA stub, dot = 10 ✅ Native, ISA stub, dot = 10 ✅
B. --enable-native-access=java.base Native provider returned; dot throws NoClassDefFoundError: Could not initialize class NativeSimdOps (cause: IllegalCallerException: Illegal native access from: unnamed module) ❌ Native, dot = 10 ✅ (JDK 24+ warns instead of throwing)
C. Same as B + -Djvector.enable_native_vectorization=false Panama, dot = 10 ✅ Panama, dot = 10 ✅
D. Library without jvector_simd_get_active_isa (stale library) Native, "Unable to determine Highway ISA tier", dot = 10 ⚠️ Same ⚠️
E. --illegal-native-access=deny n/a (option added in JDK 24) AssertionError from VectorizationProvider.lookup (System.loadLibrary throws IllegalCallerException in LibraryLoader.loadJvector:53, which only catches UnsatisfiedLinkError) ❌

JDK 22, scenario B:

INFO: Native Vector API enabled. Using NativeVectorizationProvider. (Unable to determine Highway ISA tier)
PROVIDER = NativeVectorizationProvider
DOT FAILED: java.lang.NoClassDefFoundError: Could not initialize class io.github.jbellis.jvector.vector.cnative.NativeSimdOps
  caused by: java.lang.ExceptionInInitializerError: Exception java.lang.IllegalCallerException: Illegal native access from: unnamed module

Takeaways

  • B: the probe failure is real and leaves a provider that can't run any native call, while Panama works under the same flags (C). This was tolerable when native was opt-in; with native as the default, it affects users who never asked for it. Suggest treating a failed probe like a constructor UnsupportedOperationException: log a warning and fall through to Panama. Keep the hard failure for -Djvector.vectorization_provider=native.
  • E: on JDK 24+, strict native access turns into an AssertionError during provider init, so jvector can't be used at all, even though Panama would work. JEP 472 says a future JDK will deny by default, so this will eventually happen without any flag. LibraryLoader should handle IllegalCallerException (or lookup should fall back on any non-forced native init failure).
  • D: the probe is a rough check. A missing probe symbol doesn't always mean the kernels are broken, but falling back is still the safe default.

@jshook jshook mentioned this pull request Oct 14, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

4.0.2 Release label

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Prefer Google Highway native SIMD by default

3 participants