Ship the quantized kernels as their own library

[shoumikhin]

A quantized model uses smaller numbers than a normal one, so the tensors take less memory.
Running one needs the quantized operator kernels.

The only copy the wheel shipped is the one torch loads to export a model, which a C++ application
cannot use. Such an application links the runtime, loads a quantized model, and the model fails at
run time with a missing operator, which looks like a model problem rather than a packaging one.

Build the quantized kernels as their own shared library and name it as a CMake component, the same
way the other kernel sets are named.

find_package(executorch REQUIRED COMPONENTS kernels_quantized)
target_link_libraries(my_app PRIVATE executorch::runtime
                                     executorch::kernels_quantized)

The wheel now ships lib/libexecutorch_kernels_quantized.so.

Note that the wheel also ships a second copy of these kernels, inside the library torch loads when
you export a model. That copy is built into the plugin rather than resolved from the shared library,
so a process holding both registers the same operators twice, and the runtime treats that as fatal:

Re-registering quantized_decomposed::add.out

This affects only a process that does both, for example an application that embeds a Python
interpreter. A plain C++ application can link the component freely.

Because of that, this is the one component EXECUTORCH_LIBRARIES does not include, so an
application that links whatever the package offers cannot end up in that position without asking.
A consumer that wants the quantized kernels names the component, or on CMake older than 3.28, where
no component targets exist, links EXECUTORCH_QUANTIZED_KERNELS_LIBRARY as well. That variable is
now populated on both CMake routes, so a consumer that adopts the older-CMake recipe and later
upgrades keeps the library on their link line instead of silently losing it.

Built the wheel, installed it into a clean environment, and:

• exported a quantized model and ran it from Python, matching eager PyTorch to within the
  quantization step (measured worst difference 0.0048 against a tolerance of 0.02).
• built a C++ application that links executorch::kernels_quantized, ran the same program, and got
  the same output as Python, byte for byte.
• confirmed the Python extension does not depend on the run-time copy, and that a process holding
  the shipped library and the export plugin aborts in either load order.
• checked every shipped library the same way, to establish that this is the only pair that
  collides: the CPU kernels, the delegate, the thread pool, the profiler and the runtime all
  coexist with both the extension and the export plugin.
• an application linking only EXECUTORCH_LIBRARIES does not depend on the quantized library while
  still depending on the CPU kernels, on CMake 3.28 and on real CMake 3.24. A new check asserts
  this, and it fails on the previous behaviour.
• EXECUTORCH_QUANTIZED_KERNELS_LIBRARY resolves to the shipped library on both the modern-CMake
  route (as the imported target) and the pre-3.28 route (as a file path).
• a missing quantized library now fails the checks instead of skipping them. The preset that builds
  the wheel enables these kernels unconditionally, so their absence is a regression rather than a
  configuration to tolerate, and both the ownership table and the C++ check previously treated it as
  an acceptable state and reported coverage they had not run.

Ran on Linux x86_64 and aarch64.

[shoumikhin]

Stack from ghstack (oldest at bottom):

• Ship linkable libraries in the macOS wheel #21771
• Ship the OpenVINO delegate as its own library #21770
• Build and publish CUDA wheels #21668
• Ship the CUDA delegate in the wheel #21645
• -> Ship the quantized kernels as their own library #21642
• Ship a C++ SDK in the wheel #21639
• Split the runtime out of the Python bindings extension #21610

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/21642

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❌ 1 New Failure, 92 Pending, 1 Unrelated Failure, 8 Unclassified Failures

As of commit de203b1 with merge base 43f89fb ():

NEW FAILURE - The following job has failed:

• pull / test-arduino-library / run / arduino-library (gh)
  RuntimeError: Command docker exec -t 8caa43dca0ac4dd954bc662d128ebd3893c4ebec3e7d58eb55e28abbad76f70c /exec failed with exit code 1

UNCLASSIFIED FAILURES - DrCI could not classify the following jobs because the workflow did not run on the merge base. The failures may be pre-existing on trunk or introduced by this PR:

• Build Aarch64 Linux Wheels / pytorch/executorch / build-wheel-py3_14-cpu-aarch64 (gh) (this job did not run on the merge base, so DrCI cannot tell whether the failure is pre-existing)
  pyarrow
• Build Aarch64 Linux Wheels / pytorch/executorch / upload / upload-wheel-py3_14-cpu-aarch64 (gh) (this job did not run on the merge base, so DrCI cannot tell whether the failure is pre-existing)
  Unable to download artifact(s): Artifact not found
• Build Linux Wheels / pytorch/executorch / build-manywheel-py3_14-cpu (gh) (this job did not run on the merge base, so DrCI cannot tell whether the failure is pre-existing)
  pyarrow
• Build Linux Wheels / pytorch/executorch / upload / upload-manywheel-py3_14-cpu (gh) (this job did not run on the merge base, so DrCI cannot tell whether the failure is pre-existing)
  Unable to download artifact(s): Artifact not found
• Build macOS Wheels / pytorch/executorch / build-wheel-py3_14-cpu (gh) (this job did not run on the merge base, so DrCI cannot tell whether the failure is pre-existing)
  pyarrow
• Build macOS Wheels / pytorch/executorch / upload / upload-wheel-py3_14-cpu (gh) (this job did not run on the merge base, so DrCI cannot tell whether the failure is pre-existing)
  Unable to download artifact(s): Artifact not found
• Build Windows Wheels / pytorch/executorch / build-wheel-py3_14-cpu (gh) (this job did not run on the merge base, so DrCI cannot tell whether the failure is pre-existing)
  pyarrow
• Build Windows Wheels / pytorch/executorch / upload / upload-wheel-py3_14-cpu (gh) (this job did not run on the merge base, so DrCI cannot tell whether the failure is pre-existing)
  Unable to download artifact(s): Artifact not found

BROKEN TRUNK - The following job failed but were present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

• Apple / build-demo-ios / macos-job (gh) (trunk failure)
  RuntimeError: Command bash /Users/runner/work/_temp/exec_script failed with exit code 65

This comment was automatically generated by Dr. CI and updates every 15 minutes.