More updates for MKL installations

software/numpy/README.md

@@ -38,7 +38,7 @@ The speedup arrives in two parts that activate differently, and the distinction
 
 ### Installation
 
-There are two practical ways to get a oneMKL-backed NumPy. conda is recommended because it also lets you control the OpenMP runtime (see [Threads and NUMA](#threads-and-numa)). [Miniforge](https://github.com/conda-forge/miniforge) distribution is recommended.
+There are three practical ways to get a oneMKL-backed NumPy. conda is recommended because it also lets you control the OpenMP runtime (see [Threads and NUMA](#threads-and-numa)). [Miniforge](https://github.com/conda-forge/miniforge) distribution is recommended.
 
 **conda.** A single command installs NumPy, SciPy, the three extension packages (mkl_fft, mkl_random, mkl_umath), and the runtime libraries. The BLAS/LAPACK backend routes to oneMKL automatically; the extensions are installed but still need explicit activation (the activation process is shown in the [Optimization Levers](#optimization-levers) section).
 
@@ -49,7 +49,7 @@ conda create -n idp_env python intelpython3_full \
   conda activate idp_env
 ```
 
-> **Threading layer for this environment.** `intelpython3_full` is a metapackage that also brings in `scikit-learn`, which is built against GNU OpenMP. In this mixed environment set `MKL_THREADING_LAYER=GNU` so oneMKL and those packages share one OpenMP runtime (see [Threads and NUMA](#threads-and-numa)). If you only need oneMKL-backed NumPy, prefer the targeted install below, which stays on Intel OpenMP (`MKL_THREADING_LAYER=INTEL`) with no such tradeoff.
+> **Threading layer for this environment.** `intelpython3_full` is a metapackage that also brings in `scikit-learn`, which is built against GNU OpenMP. In this mixed environment set `MKL_THREADING_LAYER=GNU` so oneMKL and those packages share one OpenMP runtime (see [Threads and NUMA](#threads-and-numa)) - or alternatively, install the LLVM OpenMP runtime (`conda install _openmp_mutex=*=*llvm*`) which is binary compatible with both. If you only need oneMKL-backed NumPy, prefer the targeted install below, which stays on Intel OpenMP (`MKL_THREADING_LAYER=INTEL`) with no such tradeoff.
 
 Pin python version to match your project if you need a specific interpreter. NumPy comes from conda-forge; the Intel channel supplies Intel's latest oneMKL builds. The `mkl_fft`/`mkl_random`/`mkl_umath` extensions are available from both conda-forge and the Intel channel, so either channel works for them; the command below keeps both channels enabled. To add oneMKL to an *existing* environment that already has conda-forge NumPy installed, swap its BLAS to the MKL variant and add the extensions in place (this re-links the NumPy you already have, it does not reinstall NumPy).
 
@@ -74,6 +74,8 @@ pip install --index-url https://software.repos.intel.com/python/pypi \
 
 Use `--index-url`, not `--extra-index-url`: Intel's index is a partial mirror, and with `--extra-index-url` pip would see PyPI's higher-numbered OpenBLAS wheel and install that instead. Packages Intel does not mirror (for example `threadpoolctl`, used for [verification](#verifying-onemkl-is-active)) install normally from PyPI in a separate step. The Intel wheels target Linux and Windows; if `pip` reports no matching distribution, check that your platform and Python version are covered on the index.
 
+**apt.** On Debian and Debian-based distributions such as Ubuntu, system-level installs of NumPy (`apt install python3-numpy`) take their BLAS and LAPACK from system-level `libblas` and `liblapack`, which can be switched among different providers through the Debian alternatives system. See this [link section](../R/README.md#linux) for full instructions on how to set MKL as system provider for BLAS and LAPACK, which apt-installed NumPy will use.
+
 Whichever path you take, choose the OpenMP threading layer and set it **before anything imports NumPy, SciPy or MKL**. The variable is read once at MKL load time, so exporting it after the import has no effect. Intel OpenMP is the fastest on Intel hardware and is oneMKL's default, so setting the variable explicitly documents intent and guarantees the choice rather than changing behavior, especially useful in an all-Intel environment where Intel OpenMP is the only runtime present. In a mixed environment where other packages bring GNU's `libgomp`, you may instead set `MKL_THREADING_LAYER=GNU` so the process shares one runtime; that tradeoff and the other layer values are detailed under [Threads and NUMA](#threads-and-numa):
 
 ```bash

software/scikit-learn/README.md

@@ -106,8 +106,7 @@ Note that, if these were previously installed from PyPI in a given environment,
 pip install -U --force-reinstall --index-url https://software.repos.intel.com/python/pypi numpy scipy mkl-service
 ```
 
-If NumPy and SciPy are installed as system packages from APT (not recommended as versions will be out of date), similar system libraries `libblas` and `liblapack` can be made to be backed by oneMKL through the Debian alternatives system, after [installing oneMKL through APT](https://www.intel.com/content/www/us/en/developer/tools/oneapi/onemkl-download.html?operatingsystem=linux&linux-install=apt):
-https://www.intel.com/content/www/us/en/developer/articles/technical/using-onemkl-with-r.html#inpage-nav-2-undefined
+If NumPy and SciPy are installed as system packages from APT (not recommended as versions will be out of date), similar system libraries `libblas` and `liblapack` can be made to be backed by oneMKL through the Debian alternatives system, after [installing oneMKL through APT](https://www.intel.com/content/www/us/en/developer/tools/oneapi/onemkl-download.html?operatingsystem=linux&linux-install=apt) - see this [link section](../R/README.md#linux) for full instructions.
 
 Backends for BLAS and LAPACK mostly affect procedures from scikit-learn that rely on linear algebra, such as linear models and procedures involving covariances, distances, and similar (e.g. [LinearRegression](https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.LinearRegression.html), [LogisticRegression](https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.LogisticRegression.html), [KMeans](https://scikit-learn.org/stable/modules/generated/sklearn.cluster.KMeans.html), [PCA](https://scikit-learn.org/stable/modules/generated/sklearn.decomposition.PCA.html), [EmpiricalCovariance](https://scikit-learn.org/stable/modules/generated/sklearn.covariance.EmpiricalCovariance.html), etc.), but do not have any effect on tree-based models (e.g. [RandomForestClassifier](https://scikit-learn.org/stable/modules/generated/sklearn.ensemble.RandomForestClassifier.html), [HistGradientBoostingRegressor](https://scikit-learn.org/stable/modules/generated/sklearn.ensemble.HistGradientBoostingRegressor.html), etc.), nor on meta-estimators (e.g. [GridSearchCV](https://scikit-learn.org/stable/modules/generated/sklearn.model_selection.GridSearchCV.html)).
 
@@ -132,7 +131,17 @@ conda create -n intelenv -c conda-forge \
 
 On Windows, switching of OpenMP backends in conda environments is unfortunately not possible.
 
-When packages are installed through `pip` or APT, switching OpenMP backends is unfortunately not as easy unless packages are compiled from source, and the default choice for backend in those channels is usually GNU's LibGOMP, which is not as performant on Intel hardware. Thus, it is recommended to use a conda environment to manage the Python installation, where the OpenMP backend can be easily changed as needed.
+When packages are installed through Intel's pip index or through APT (e.g. `sudo apt install python3-sklearn`, switching OpenMP backends is unfortunately not as easy unless packages are compiled from source, and the default choice for backend in those channels is usually GNU's LibGOMP, which is not as performant on Intel hardware. Thus, it is recommended to use a conda environment to manage the Python installation (including the NumPy, SciPy, and scikit-learn installations), where the OpenMP backend can be easily changed as needed.
+
+Note that, on Linux, if packages like NumPy and SciPy are not installed in a conda environment but are using MKL through either a system-level library or through the pip-managed Intel builds, then there might be runtime conflicts due to scikit-learn trying to load LibGOMP while MKL tries to load Intel's OMP. In those cases (non-conda-managed MKL), to ensure that a single OpenMP runtime is loaded, MKL can be configured to use LibGOMP by setting an environment variable as follows:
+```shell
+export MKL_THREADING_LAYER=GNU
+```
+
+To make this permanent, it can be added to a file like `/etc/environment` as follows:
+```shell
+printf "MKL_THREADING_LAYER=GNU\n" | sudo tee -a /etc/environment
+```
 
 ### Verifying backends
 
@@ -168,7 +177,7 @@ If LLVM's LibOMP is being used, it will show an entry like the following:
 
 If the `prefix` entry mentions something different, such as `libgomp`, then it means another backend is in usage. Alternatively, if Intel's OpenMP is being used, it will show as `libiomp`.
 
-Note that the command above might return multiple backends - if that happens, the entry that appears first in the list is most likely to be used in practice by scikit-learn.
+Note that the command above might return multiple backends - if that happens, the entry that appears first in the list is most likely to be used in practice by scikit-learn. However, in such cases, it is advisable to set the environment variable `$MKL_THREADING_LAYER` to avoid loading potentially incompatible backends.
 
 ## Parallelism in scikit-learn