Skip to content

Add libomp/libiomp compatibility experiment scripts#14

Closed
ogrisel wants to merge 1 commit into
masterfrom
cursor/libomp-libiomp-experiments-36c4
Closed

Add libomp/libiomp compatibility experiment scripts#14
ogrisel wants to merge 1 commit into
masterfrom
cursor/libomp-libiomp-experiments-36c4

Conversation

@ogrisel

@ogrisel ogrisel commented Jul 10, 2026

Copy link
Copy Markdown
Owner

Adds reproducible experiment scripts for follow-up investigation from PR joblib#222 about whether the Linux libiomp + libomp warning is still needed with recent library versions.

Findings (Linux, Jul 2026)

Tested with intel-openmp 2025.0.0 (libiomp perf lib 5.0.20240820) and LLVM libomp 18.1.8 (Clang 18.1 build), matching the pylatest_conda_mkl_clang_gcc CI setup.

The warning is still needed on Linux. Recent versions remain incompatible:

Test Result
C reproducer: call omp_get_max_threads from both runtimes Deadlock
ctypes calls into both loaded runtimes OMP Error joblib#15 → SIGABRT
threadpool_limits() after nested prange+BLAS Hangs on Python exit
KMP_DUPLICATE_LIB_OK=TRUE Suppresses abort but still unreliable

Loading both libraries without calling into both simultaneously can appear fine; active use of both OpenMP APIs still crashes or deadlocks.

Scripts

  • experiments/libomp_libiomp_repro.c — minimal C reproducer (LLVM bug #43565)
  • experiments/libomp_libiomp_variants.c — load-only / intel-first / llvm-first variants
  • experiments/run_experiments.py — Python integration tests mirroring threadpoolctl usage

How to run

# Setup: conda env with MKL numpy/scipy + LLVM 18 libomp (see CI install.sh)
export CC_OUTER_LOOP=clang-18 CC_INNER_LOOP=gcc
bash continuous_integration/build_test_ext.sh

# C reproducer
gcc -o repro experiments/libomp_libiomp_repro.c -ldl
LD_LIBRARY_PATH=$CONDA_PREFIX/lib:/usr/lib/llvm-18/lib:$LD_LIBRARY_PATH timeout 15 ./repro

# Python experiments
PYTHONPATH=tests/_openmp_test_helper:. python experiments/run_experiments.py

Recommendation

Keep the Linux-only warning from PR joblib#222. Do not remove the warning on any platform without new evidence that the runtimes have become compatible.

Open in Web Open in Cursor 

Scripts to reproduce and test the Linux-specific incompatibility between
Intel OpenMP (libiomp) and LLVM OpenMP (libomp) with recent library
versions. Supports follow-up investigation from PR joblib#222.

Co-authored-by: Olivier Grisel <olivier.grisel@ensta.org>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants