Skip to content

Use khiops-core and khiops-driver-* pip packages containing binaries only#582

Merged
tramora merged 2 commits into
mainfrom
core-and-drivers-as-pip-packages
Jun 19, 2026
Merged

Use khiops-core and khiops-driver-* pip packages containing binaries only#582
tramora merged 2 commits into
mainfrom
core-and-drivers-as-pip-packages

Conversation

@tramora

@tramora tramora commented May 26, 2026

Copy link
Copy Markdown
Collaborator

Fixes #572 and #578


TODO Before Asking for a Review

  • Rebase your branch to the latest version of main (or main-v10)
  • Make sure all CI workflows are green
  • When adding a public feature/fix: Update the Unreleased section of CHANGELOG.md (no date)
  • Self-Review: Review "Files Changed" tab and fix any problems you find
  • API Docs (only if there are changes in docstrings, rst files or samples):
    • Check the docs build without warning: see the log of the API Docs workflow
    • Check that your changes render well in HTML: download the API Docs artifact and open index.html
    • If there are any problems it is faster to iterate by building locally the API Docs

@tramora tramora marked this pull request as draft May 26, 2026 16:23
@tramora tramora force-pushed the core-and-drivers-as-pip-packages branch 9 times, most recently from caa2f18 to fd7c550 Compare May 28, 2026 10:32
@tramora tramora marked this pull request as ready for review May 28, 2026 13:44
@tramora tramora requested a review from popescu-v May 28, 2026 13:44
@tramora tramora force-pushed the core-and-drivers-as-pip-packages branch 2 times, most recently from 394d474 to 5312426 Compare May 28, 2026 21:33
@tramora

tramora commented May 28, 2026

Copy link
Copy Markdown
Collaborator Author

False sentiment of success : we simulate multiple Python environments using conda (#573). This is very unfortunate as we are not able yet to detect automatically an issue with the new PyPI packages regarding the remote drivers khiops-core:1041. In my opinion this one is a blocking issue.

@tramora tramora force-pushed the core-and-drivers-as-pip-packages branch from 5312426 to b0223fa Compare May 29, 2026 00:37
@popescu-v

Copy link
Copy Markdown
Collaborator

False sentiment of success : we simulate multiple Python environments using conda (#573). This is very unfortunate as we are not able yet to detect automatically an issue with the new PyPI packages regarding the remote drivers khiops-core:1041. In my opinion this one is a blocking issue.

Hence, to properly test this, issue #573 must be addressed first. Right, @tramora ?

@tramora tramora force-pushed the core-and-drivers-as-pip-packages branch from 95a1d21 to cfd1802 Compare May 29, 2026 09:43
Comment thread .github/workflows/api-docs.yml Outdated
Comment thread .github/workflows/dev-docker.yml Outdated
Comment thread .github/workflows/pip.yml Outdated
Comment thread .github/workflows/quick-checks.yml Outdated
Comment thread .github/workflows/quick-checks.yml Outdated
Comment thread .github/workflows/tests.yml
Comment thread .github/workflows/tests.yml Outdated
Comment thread .github/workflows/tests.yml Outdated
Comment thread khiops/core/internals/filesystems.py Outdated
Comment thread khiops/core/internals/filesystems.py Outdated
Comment thread khiops/core/internals/filesystems.py Outdated
@tramora

tramora commented May 29, 2026

Copy link
Copy Markdown
Collaborator Author

False sentiment of success : we simulate multiple Python environments using conda (#573). This is very unfortunate as we are not able yet to detect automatically an issue with the new PyPI packages regarding the remote drivers khiops-core:1041. In my opinion this one is a blocking issue.

Hence, to properly test this, issue #573 must be addressed first. Right, @tramora ?

I changed the $CONDA install -y -n env ... to $CONDA run -n env pip install ... in order to really use the PyPI packages.
But it is not perfect yet as the run will set the CONDA_PREFIX environment variable that is checked by khiops-python to determine if the installation is done in a 'conda' method or not. 'conda' method and 'pip' one are close but not totally identical

Comment thread khiops/core/internals/runner.py Outdated
Comment thread khiops/core/internals/runner.py Outdated
Comment thread khiops/core/internals/runner.py Outdated
Comment thread khiops/core/internals/runner.py Outdated
Comment thread khiops/core/internals/runner.py Outdated
Comment thread khiops/core/internals/runner.py Outdated
Comment thread packaging/docker/khiopspydev/Dockerfile.ubuntu-debian Outdated
Comment thread packaging/docker/khiopspydev/Dockerfile.ubuntu-debian Outdated
Comment thread packaging/docker/khiopspydev/Dockerfile.ubuntu-debian Outdated

@popescu-v popescu-v left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See pending comments.

@tramora tramora force-pushed the core-and-drivers-as-pip-packages branch from 541a1e8 to 61f07d7 Compare June 17, 2026 17:52
@tramora tramora requested a review from popescu-v June 17, 2026 17:56
Comment thread .github/instructions/docker-changes.instructions.md Outdated
Comment thread packaging/docker/khiopspydev/Dockerfile.ubuntu-debian Outdated
Comment thread .github/workflows/pip.yml Outdated
Comment thread khiops/core/internals/runner.py Outdated

@popescu-v popescu-v left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just a few loose ends, among which a rather important one IMHO.

@tramora tramora force-pushed the core-and-drivers-as-pip-packages branch from 61f07d7 to 875c51d Compare June 18, 2026 14:44
@tramora tramora requested a review from popescu-v June 18, 2026 14:58
@tramora tramora force-pushed the core-and-drivers-as-pip-packages branch from 875c51d to 3f53564 Compare June 18, 2026 15:42
Comment thread khiops/core/internals/runner.py Outdated
Comment thread khiops/core/internals/runner.py Outdated

@popescu-v popescu-v left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just a text to remove (mention of Khiops binaries in the Docker file), and some unused code to remove from the runner (see the pending comments).

Thierry RAMORASOAVINA added 2 commits June 19, 2026 13:13
…ries only

- these packages become mandatory ("khiops-core") or optional ("khiops-driver-*") dependencies dragged during the installation process
- these packages must be installed in the Conda environments used to simulate multiple environs (the Conda packages MUST not be used)
- in order to avoid distorting usage statistics the test workflows will always use khiops packages from TestPypi
- the khiops-server inside the dev docker image will not use any longer the Khiops binaries from the native packages : the `KHIOPS_CMD` and `KHIOPS_COCLUSTERING_CMD` env variables will be set instead
@tramora tramora force-pushed the core-and-drivers-as-pip-packages branch from 3f53564 to 235f573 Compare June 19, 2026 11:13
@tramora tramora requested a review from popescu-v June 19, 2026 11:14

@popescu-v popescu-v left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@tramora tramora merged commit 360d541 into main Jun 19, 2026
24 checks passed
@tramora tramora deleted the core-and-drivers-as-pip-packages branch June 19, 2026 11:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Depend on the khiops-core Pip Package

2 participants