-
Notifications
You must be signed in to change notification settings - Fork 3.3k
Enhanced setup.py with dual setup.py and pyproject.toml package support #44810
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Changes from 4 commits
f249b3f
cb8ff0b
881eb2f
9bf77bf
3d2b7a8
9f9c4ea
377e4f1
46fb773
aa0c7ba
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,40 @@ | ||
| # -------------------------------------------------------------------------- | ||
| # Copyright (c) Microsoft Corporation. All rights reserved. | ||
| # Licensed under the MIT License. See License.txt in the project root for | ||
| # license information. | ||
| # -------------------------------------------------------------------------- | ||
|
|
||
| [build-system] | ||
| requires = ["setuptools>=77.0.3", "wheel"] | ||
| build-backend = "setuptools.build_meta" | ||
|
|
||
| [project] | ||
| name = "azure-sdk-for-python" | ||
| version = "1.0.0" | ||
| description = "Azure SDK for Python - Meta package for installing all Azure SDK packages" | ||
| readme = "README.md" | ||
| license = {text = "MIT"} | ||
| authors = [ | ||
| {name = "Microsoft Corporation", email = "azpysdkhelp@microsoft.com"} | ||
| ] | ||
| requires-python = ">=3.9" | ||
| classifiers = [ | ||
| "Development Status :: 5 - Production/Stable", | ||
| "Programming Language :: Python", | ||
| "Programming Language :: Python :: 3 :: Only", | ||
| "Programming Language :: Python :: 3.9", | ||
| "Programming Language :: Python :: 3.10", | ||
| "Programming Language :: Python :: 3.11", | ||
| "Programming Language :: Python :: 3.12", | ||
| "Programming Language :: Python :: 3.13", | ||
| "License :: OSI Approved :: MIT License", | ||
| ] | ||
| keywords = ["azure", "azure sdk"] | ||
|
|
||
| [project.urls] | ||
| Homepage = "https://github.com/Azure/azure-sdk-for-python" | ||
| Repository = "https://github.com/Azure/azure-sdk-for-python" | ||
| Documentation = "https://docs.microsoft.com/python/azure/" | ||
|
|
||
| [tool.setuptools] | ||
| py-modules = [] | ||
| Original file line number | Diff line number | Diff line change | ||||||||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
|
|
@@ -5,19 +5,39 @@ | |||||||||||||||||||||||||||
| # Licensed under the MIT License. See License.txt in the project root for | ||||||||||||||||||||||||||||
| # license information. | ||||||||||||||||||||||||||||
| #-------------------------------------------------------------------------- | ||||||||||||||||||||||||||||
| # | ||||||||||||||||||||||||||||
| # This setup.py supports discovery and installation of Azure SDK packages | ||||||||||||||||||||||||||||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. @copilot look for packages with setup and then only packages that have pyproject.toml with a [project] section
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Updated package discovery to find setup.py packages first, then only pyproject.toml packages with [project] section in commit 377e4f1. Now discovers:
The filtering happens during discovery using a helper function that checks for the [project] section via TOML parser. |
||||||||||||||||||||||||||||
| # from both traditional setup.py and modern pyproject.toml configurations. | ||||||||||||||||||||||||||||
| # | ||||||||||||||||||||||||||||
| # Package Discovery: | ||||||||||||||||||||||||||||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. @copilot I want the pyproject.toml to do this and I want to remove the setup.py
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Done! setup.py has been removed and the logic moved to install_packages.py, which is now managed by pyproject.toml. Changes in commit 3d2b7a8:
Usage: # Run directly
python install_packages.py install
# Or install root package and use CLI
pip install .
install-azure-sdk-packages installThe pyproject.toml now fully manages the package configuration, and install_packages.py handles the orchestration. |
||||||||||||||||||||||||||||
| # - Finds packages with setup.py at root or in sdk/*/ directories | ||||||||||||||||||||||||||||
| # - Finds packages with pyproject.toml at root or in sdk/*/ directories | ||||||||||||||||||||||||||||
| # | ||||||||||||||||||||||||||||
| # Installation Logic: | ||||||||||||||||||||||||||||
| # - Packages with pyproject.toml containing [project] section: | ||||||||||||||||||||||||||||
| # Installed using pip (PEP 621 compliant) | ||||||||||||||||||||||||||||
| # - Packages with setup.py: | ||||||||||||||||||||||||||||
| # Installed using traditional setuptools via runpy | ||||||||||||||||||||||||||||
| # - Packages with both setup.py and pyproject.toml (without [project]): | ||||||||||||||||||||||||||||
| # Prefer setup.py for backward compatibility | ||||||||||||||||||||||||||||
| #-------------------------------------------------------------------------- | ||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||
| import os.path | ||||||||||||||||||||||||||||
| import glob | ||||||||||||||||||||||||||||
| import copy | ||||||||||||||||||||||||||||
| import sys | ||||||||||||||||||||||||||||
| import runpy | ||||||||||||||||||||||||||||
| import subprocess | ||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||
| root_folder = os.path.abspath(os.path.dirname(__file__)) | ||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||
| # pull in any packages that exist in the root directory | ||||||||||||||||||||||||||||
| # Support both setup.py and pyproject.toml packages | ||||||||||||||||||||||||||||
| packages = {('.', os.path.dirname(p)) for p in glob.glob('azure*/setup.py')} | ||||||||||||||||||||||||||||
| packages.update({('.', os.path.dirname(p)) for p in glob.glob('azure*/pyproject.toml')}) | ||||||||||||||||||||||||||||
| # Handle the SDK folder as well | ||||||||||||||||||||||||||||
| packages.update({tuple(os.path.dirname(f).rsplit(os.sep, 1)) for f in glob.glob('sdk/*/azure*/setup.py')}) | ||||||||||||||||||||||||||||
| packages.update({tuple(os.path.dirname(f).rsplit(os.sep, 1)) for f in glob.glob('sdk/*/azure*/pyproject.toml')}) | ||||||||||||||||||||||||||||
| # [(base_folder, package_name), ...] to {package_name: base_folder, ...} | ||||||||||||||||||||||||||||
| packages = {package_name: base_folder for (base_folder, package_name) in packages} | ||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||
|
|
@@ -32,8 +52,9 @@ | |||||||||||||||||||||||||||
| content_package = sorted([p for p in packages.keys() if p not in meta_package+nspkg_packages]) | ||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||
| # Move azure-common at the beginning, it's important this goes first | ||||||||||||||||||||||||||||
| content_package.remove("azure-common") | ||||||||||||||||||||||||||||
| content_package.insert(0, "azure-common") | ||||||||||||||||||||||||||||
| if "azure-common" in content_package: | ||||||||||||||||||||||||||||
| content_package.remove("azure-common") | ||||||||||||||||||||||||||||
| content_package.insert(0, "azure-common") | ||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||
| # Package final: | ||||||||||||||||||||||||||||
| if "install" in sys.argv: | ||||||||||||||||||||||||||||
|
|
@@ -44,6 +65,7 @@ | |||||||||||||||||||||||||||
| for pkg_name in packages_for_installation: | ||||||||||||||||||||||||||||
| pkg_setup_folder = os.path.join(root_folder, packages[pkg_name], pkg_name) | ||||||||||||||||||||||||||||
| pkg_setup_path = os.path.join(pkg_setup_folder, 'setup.py') | ||||||||||||||||||||||||||||
| pkg_pyproject_path = os.path.join(pkg_setup_folder, 'pyproject.toml') | ||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||
| try: | ||||||||||||||||||||||||||||
| saved_dir = os.getcwd() | ||||||||||||||||||||||||||||
|
|
@@ -52,8 +74,48 @@ | |||||||||||||||||||||||||||
| os.chdir(pkg_setup_folder) | ||||||||||||||||||||||||||||
| sys.path = [pkg_setup_folder] + copy.copy(saved_syspath) | ||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||
| print("Start ", pkg_setup_path) | ||||||||||||||||||||||||||||
| result = runpy.run_path(pkg_setup_path) | ||||||||||||||||||||||||||||
| # Determine which file to use: pyproject.toml with [project] or setup.py | ||||||||||||||||||||||||||||
| use_pyproject = False | ||||||||||||||||||||||||||||
| if os.path.exists(pkg_pyproject_path): | ||||||||||||||||||||||||||||
| # Check if pyproject.toml has [project] section using TOML parser | ||||||||||||||||||||||||||||
| try: | ||||||||||||||||||||||||||||
| # Use tomllib for Python 3.11+, fallback to tomli for older versions | ||||||||||||||||||||||||||||
| try: | ||||||||||||||||||||||||||||
| import tomllib as toml | ||||||||||||||||||||||||||||
| except ImportError: | ||||||||||||||||||||||||||||
| import tomli as toml | ||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||
| with open(pkg_pyproject_path, 'rb') as f: | ||||||||||||||||||||||||||||
| pyproject_data = toml.load(f) | ||||||||||||||||||||||||||||
| if 'project' in pyproject_data: | ||||||||||||||||||||||||||||
| use_pyproject = True | ||||||||||||||||||||||||||||
| except Exception: | ||||||||||||||||||||||||||||
| # If parsing fails, fallback to setup.py if available | ||||||||||||||||||||||||||||
| pass | ||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||
| if use_pyproject: | ||||||||||||||||||||||||||||
| # Use pip to install pyproject.toml-based packages | ||||||||||||||||||||||||||||
| # Map setup.py commands to pip commands | ||||||||||||||||||||||||||||
| if "install" in sys.argv: | ||||||||||||||||||||||||||||
| print("Start ", pkg_pyproject_path) | ||||||||||||||||||||||||||||
| cmd = [sys.executable, '-m', 'pip', 'install', '.'] | ||||||||||||||||||||||||||||
|
||||||||||||||||||||||||||||
| elif "develop" in sys.argv or any(arg in sys.argv for arg in ['-e', '--editable']): | ||||||||||||||||||||||||||||
|
||||||||||||||||||||||||||||
| elif "develop" in sys.argv or any(arg in sys.argv for arg in ['-e', '--editable']): | |
| elif "develop" in sys.argv: |
Copilot
AI
Jan 23, 2026
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
When commands other than "install" or "develop" are used, pyproject.toml packages are skipped entirely (line 101). This means commands like bdist_wheel, sdist, or other setup.py commands won't work for pyproject.toml packages. If these commands need to be supported, consider mapping them to equivalent pip/build commands or documenting this limitation clearly.
| # For other commands like --version, --help, etc., skip pyproject.toml packages | |
| # These commands are meant for the root setup.py, not individual packages | |
| # For other commands (e.g., sdist, bdist_wheel, --version, --help, etc.), | |
| # pyproject.toml-based packages are not handled per-package by this script. | |
| # These commands are meant for the root setup.py, not individual packages. | |
| print( | |
| f"Skipping pyproject.toml-based package '{pkg_name}' for setup.py " | |
| f"command(s) {sys.argv[1:]}. Only 'install' and 'develop' (or " | |
| f"editable variants) are supported for per-package operations. " | |
| "Use 'pip install' or 'python -m build' in the package directory " | |
| "to build distribution artifacts for this package.", | |
| file=sys.stderr, | |
| ) |
Copilot
AI
Jan 23, 2026
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The subprocess.run call uses capture_output=False, which means stdout and stderr will be printed to the console. However, the function doesn't check result.returncode to propagate failures. If a package installation fails, the script continues installing other packages and only prints a warning. This could lead to incomplete installations going unnoticed. Consider either:
- Raising an exception on non-zero return codes to halt the installation
- Collecting failures and reporting them at the end
- At minimum, using a more severe logging mechanism than just a warning
| print(f"Warning: Package {pkg_name} installation returned non-zero exit code", file=sys.stderr) | |
| print(f"Error: Package {pkg_name} installation returned non-zero exit code {result.returncode}", file=sys.stderr) | |
| sys.exit(result.returncode) |
Copilot
AI
Jan 23, 2026
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The logic for handling packages with both setup.py and pyproject.toml is unclear. According to the PR description, "Packages with both → prefers setup.py for backward compatibility", but the code at line 87 prefers pyproject.toml over setup.py (using use_pyproject as the first condition). The conditions should be reordered to check for setup.py first if backward compatibility is the goal, or the PR description should be updated to match the actual behavior.
Copilot
AI
Jan 23, 2026
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The broad exception handler catches all exceptions (including KeyboardInterrupt and SystemExit) and only prints them to stderr without re-raising. This means if a user tries to cancel installation with Ctrl+C, the script will continue processing other packages. Consider either:
- Catching specific exceptions (e.g.,
subprocess.CalledProcessError,OSError) - Re-raising KeyboardInterrupt and SystemExit
- Using
except Exception as e:instead of bareexceptto exclude system-exiting exceptions
| print(f"Warning: No setup.py or pyproject.toml found for {pkg_name}", file=sys.stderr) | |
| print(f"Warning: No setup.py or pyproject.toml found for {pkg_name}", file=sys.stderr) | |
| except KeyboardInterrupt: | |
| # Allow user-initiated interruption (e.g., Ctrl+C) to stop the script | |
| raise |
Uh oh!
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@copilot remove this
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Removed copyright header from pyproject.toml in commit 377e4f1.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@copilot no remove the file
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Removed pyproject.toml and restored original setup.py in commit 46fb773.
The repository is now back to its original state with only setup.py (no pyproject.toml).