Add NumPy optimization guide#36
Conversation
Add intel-numpy mkl extension optimizations readme
|
Overall comment: this guide recommends setting the IOMP threading layter for MKL, but pretty much every other PyPI package outside of Intel-distributed NumPy will bundle LibGOMP and could potentially cause incompatibilities. Perhaps it could recommend setting |
|
Comment again that the guide specifically mentions AVX-512 as the highest level of SIMD instructions, but that will become outdated soon as hardware with avx10.2 gets released. |
| print(f"speedup : {stock_ms / mkl_ms:.1f}x") | ||
| ``` | ||
|
|
||
| Measured on AWS, Intel® Xeon® 6975P-C, 16 cores / 32 threads (HT on), 1 socket, Ubuntu 26.04 LTS. Numbers vary by hardware. |
There was a problem hiding this comment.
Have these numbers been through PDT?
There was a problem hiding this comment.
These are example snippets and freshly measured (FFT on the AWS Xeon 6975P-C) but have not been through PDT.
where as the benchmarking results section displayed numbers and speedups are been through the PCR review process and approved and published on Intel Distribution for Python page - https://www.intel.com/content/www/us/en/developer/tools/oneapi/distribution-for-python.html
|
|
||
| --- | ||
|
|
||
| ## Benchmark results |
There was a problem hiding this comment.
Have these numbers all been through PDT?
|
|
||
| ```bash | ||
| export MKL_THREADING_LAYER=GNU # share one OpenMP runtime (libgomp) with other packages | ||
| export MKL_THREADING_LAYER=INTEL # Intel OpenMP (libiomp5): best oneMKL performance |
There was a problem hiding this comment.
I think this is the default and setting it like this doesn't change anything. CC @vmalia .
There was a problem hiding this comment.
Yes, Intel value is the default. It is good to mention it here in the documentation so the users know about this variable.
| Alternatively, to partition a single generator instead of using the `MT2203` family, `skipahead(n)` advances a stream by `n` steps so each worker starts at a non-overlapping offset: | ||
|
|
||
| ```python | ||
| block = 1_000_000 |
There was a problem hiding this comment.
I think something like 2^63-1 (INT64_MAX) would be a more reasonable number. Can also be obtained programmatically from numpy np.iinfo(np.int64).
There was a problem hiding this comment.
Good point on not hard-coding a magic stride. One caveat: skipahead takes a 64-bit offset, so 2^63-1 (np.iinfo(np.int64).max) works as a single stride but overflows once multiplied by the worker index (for worker_id >= 2), worker_id * INT64_MAX
| The one situation that hurts performance is loading two OpenMP runtimes in the same process, for example oneMKL on Intel OpenMP (`libiomp5`) alongside a package that bundles GNU's `libgomp`. Two runtimes over-subscribe the cores. `MKL_THREADING_LAYER` controls oneMKL's runtime only; it does not stop another package from loading its own. When your environment already standardizes on one OpenMP runtime, match oneMKL to it: | ||
|
|
||
| In conda, the runtime is selected through the `_openmp_mutex` metapackage. To pin LLVM's LibOMP (the upstreamed continuation of Intel OpenMP) on Linux: | ||
| - With LLVM's LibOMP present, use `MKL_THREADING_LAYER=INTEL`: oneMKL runs its Intel threading layer on LibOMP and the `KMP_*` controls apply. |
There was a problem hiding this comment.
Shouldn't it also set MKL_THREADING_LAYER=GNU when coupled with LLVM-OMP libraries? CC @vmalia
There was a problem hiding this comment.
oneMKL using libomp is not the expected behavior when setting MKL_THREADING_LAYER=INTEL. Even if it works now, it cannot be guaranteed to work in the future because LLVM OpenMP is not officially supported by oneMKL as of yet.
The expected behavior when setting MKL_THREADING_LAYER=INTEL and not finding libiomp is a runtime error.
The expected behavior when setting MKL_THREADING_LAYER=GNU and not finding libgomp is a runtime error.
There was a problem hiding this comment.
@vmalia In the case of a conda OMP mutex set to LLVM, on linux, the file is still named libgomp.so.1, same as GNU's. So in that case, it would find GOMP, but it would be LLVM's behind the scenes.
| - With LLVM's LibOMP present, use `MKL_THREADING_LAYER=INTEL`: oneMKL runs its Intel threading layer on LibOMP and the `KMP_*` controls apply. | ||
| - If other packages in the process pull in `libgomp` (some pip-installed scientific packages do), `MKL_THREADING_LAYER=GNU` selects oneMKL's GNU threading layer so the process shares a single runtime. | ||
|
|
||
| Platform defaults differ. Linux pip stacks often bundle `libgomp`; conda environments select the runtime through the `_openmp_mutex` metapackage (default LLVM LibOMP on recent conda-forge). On Windows conda, NumPy/SciPy/MKL resolve `llvm-openmp`, and oneMKL there reports the `intel` threading layer. To pin LLVM's LibOMP on Linux: |
There was a problem hiding this comment.
Default on linux is GOMP, unless MKL from conda-forge is installed, which switches it to LLVM-OMP. Not sure if the packages from the intel channel also set the OMP mutex though.
There was a problem hiding this comment.
All correct, but Intel-channel packages do not seem to set the mutex.
|
@jharlow-intel @vchamarthi If following the advise from this guide with intel packages, it ends up pulling GOMP instead of LLVM-OMP. Is that intended behavior? Are the intel packages meant to set the OMP mutex to LibOMP like the conda-forge ones do? See log below Details |
I want to reiterate that Intel's Optimization and Tuning guides should recommend Intel libraries and runtimes. If the package-native default runtime is set to a different vendor, the Intel guide should show how to manually configure the environment to use Intel's runtime libraries for the customer to get the best performance for their Intel hardware. |
Please see my above comment. |
Correct that it's expected to pull To your question, are the Intel packages meant to set it? I do not think so I am not aware of strong reason if intel packages should do the mutex pinning to an llvm/gnu specific. In the environment you created it isn't broken either way, at runtime oneMKL loads Intel OpenMP by default (threading_layer: intel, libiomp loaded); the libgomp on disk stays inert unless another GNU-linked lib calls it. |
5 participants
Adds a new tuning guide documenting how to run NumPy with Intel® oneMKL-backed performance (BLAS/LAPACK plus optional FFT/random/umath patching), and links it from the repository’s main README
Changes:
CC @xaleryb @jharlow-intel @napetrov for addition review