Skip to content

Enhanced setup.py with dual setup.py and pyproject.toml package support #44810

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
Copilot wants to merge 9 commits into
base: main
Choose a base branch
from
Open

Enhanced setup.py with dual setup.py and pyproject.toml package support #44810

Changes from all commits
Commits
Show all changes
9 commits
Select commit Hold shift + click to select a range
f249b3f
Initial plan
Copilot Jan 22, 2026
cb8ff0b
Add pyproject.toml and update setup.py to support both setup.py and p…
Copilot Jan 22, 2026
881eb2f
Add documentation and improve setup.py comments
Copilot Jan 22, 2026
9bf77bf
Address code review feedback: use TOML parser, improve command handli…
Copilot Jan 22, 2026
3d2b7a8
Remove setup.py and move logic to install_packages.py with pyproject.…
Copilot Jan 22, 2026
9f9c4ea
Restore setup.py and remove install_packages.py and INSTALL.md
Copilot Jan 22, 2026
377e4f1
Remove copyright header from pyproject.toml and filter packages to on…
Copilot Jan 22, 2026
46fb773
Remove pyproject.toml and restore original setup.py
Copilot Jan 22, 2026
aa0c7ba
Add support for pyproject.toml packages with [project] section in set…
Copilot Jan 22, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
67 changes: 61 additions & 6 deletions setup.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,13 +11,41 @@
import copy
import sys
import runpy
import subprocess

# Use tomllib for Python 3.11+, fallback to tomli for older versions
try:
import tomllib as toml
except ImportError:
import tomli as toml
Copy link

Copilot AI Jan 23, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The fallback import of tomli will fail if the package is not installed. For Python versions 3.8-3.10, the tomli package needs to be available as a dependency or the script will crash with an ImportError. Consider either:

  1. Adding tomli to a requirements file or as a conditional dependency (for Python < 3.11)
  2. Handling the ImportError and providing a more helpful error message instructing users to install tomli
  3. Using a try/except block that provides fallback behavior if neither tomllib nor tomli is available
Suggested change
import tomli as toml
try:
import tomli as toml
except ImportError as exc:
raise ImportError(
"Unable to import a TOML parser. This script requires either "
"'tomllib' (Python 3.11+) or the 'tomli' package to parse "
"pyproject.toml. For Python versions earlier than 3.11, "
"install 'tomli', for example:\n\n"
" pip install tomli\n"
) from exc

Copilot uses AI. Check for mistakes.

root_folder = os.path.abspath(os.path.dirname(__file__))

# pull in any packages that exist in the root directory
# Helper function to check if pyproject.toml has [project] section
def has_project_section(pyproject_path):
"""Check if a pyproject.toml file has a [project] section."""
try:
with open(pyproject_path, 'rb') as f:
pyproject_data = toml.load(f)
return 'project' in pyproject_data
except Exception:
Copy link

Copilot AI Jan 23, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The broad exception handler silently swallows all errors including file not found, permission errors, and invalid TOML syntax. This makes debugging difficult. Consider either:

  1. Catching specific exceptions (e.g., FileNotFoundError, tomllib.TOMLDecodeError or tomli.TOMLDecodeError)
  2. Logging the exception for debugging purposes
  3. At minimum, narrowing to OSError and TOML-specific exceptions rather than catching all exceptions
Suggested change
except Exception:
except (OSError, toml.TOMLDecodeError):

Copilot uses AI. Check for mistakes.
return False

# Discover packages with setup.py
packages = {('.', os.path.dirname(p)) for p in glob.glob('azure*/setup.py')}
# Handle the SDK folder as well
packages.update({tuple(os.path.dirname(f).rsplit(os.sep, 1)) for f in glob.glob('sdk/*/azure*/setup.py')})

# Discover packages with pyproject.toml that have [project] section
for pyproject_file in glob.glob('azure*/pyproject.toml'):
pyproject_path = os.path.join(root_folder, pyproject_file)
if has_project_section(pyproject_path):
packages.add(('.', os.path.dirname(pyproject_file)))

for pyproject_file in glob.glob('sdk/*/azure*/pyproject.toml'):
pyproject_path = os.path.join(root_folder, pyproject_file)
if has_project_section(pyproject_path):
packages.add(tuple(os.path.dirname(pyproject_file).rsplit(os.sep, 1)))

# [(base_folder, package_name), ...] to {package_name: base_folder, ...}
packages = {package_name: base_folder for (base_folder, package_name) in packages}

Expand All @@ -32,8 +60,9 @@
content_package = sorted([p for p in packages.keys() if p not in meta_package+nspkg_packages])

# Move azure-common at the beginning, it's important this goes first
content_package.remove("azure-common")
content_package.insert(0, "azure-common")
if "azure-common" in content_package:
content_package.remove("azure-common")
content_package.insert(0, "azure-common")

# Package final:
if "install" in sys.argv:
Expand All @@ -44,6 +73,7 @@
for pkg_name in packages_for_installation:
pkg_setup_folder = os.path.join(root_folder, packages[pkg_name], pkg_name)
pkg_setup_path = os.path.join(pkg_setup_folder, 'setup.py')
pkg_pyproject_path = os.path.join(pkg_setup_folder, 'pyproject.toml')

try:
saved_dir = os.getcwd()
Expand All @@ -52,8 +82,33 @@
os.chdir(pkg_setup_folder)
sys.path = [pkg_setup_folder] + copy.copy(saved_syspath)

print("Start ", pkg_setup_path)
result = runpy.run_path(pkg_setup_path)
# Determine which file to use: pyproject.toml with [project] or setup.py
# Since we already filtered during discovery, if pyproject.toml exists it has [project]
Comment on lines +85 to +86
Copy link

Copilot AI Jan 23, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The comment is misleading. During discovery (lines 39-47), packages are only added if their pyproject.toml has a [project] section. However, a package could also be added via setup.py discovery (lines 35-36). So when checking during installation, a pyproject.toml file might exist without a [project] section, or it might have a [project] section but the package was discovered via setup.py. The comment should be clarified or removed.

Suggested change
# Determine which file to use: pyproject.toml with [project] or setup.py
# Since we already filtered during discovery, if pyproject.toml exists it has [project]
# Determine which file to use: prefer pyproject.toml when it exists and has a [project] section,
# otherwise fall back to setup.py or skip if neither is usable.

Copilot uses AI. Check for mistakes.
use_pyproject = os.path.exists(pkg_pyproject_path) and has_project_section(pkg_pyproject_path)
Copy link

Copilot AI Jan 23, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The has_project_section() function is called redundantly during installation. Packages are already filtered to have a [project] section during discovery (lines 41, 46), but the function is called again here. This is inefficient and causes double file I/O. Since packages were already validated during discovery, you can simplify this to just use_pyproject = os.path.exists(pkg_pyproject_path) or remove the redundant check entirely if setup.py should take precedence when both files exist.

Suggested change
use_pyproject = os.path.exists(pkg_pyproject_path) and has_project_section(pkg_pyproject_path)
use_pyproject = os.path.exists(pkg_pyproject_path)

Copilot uses AI. Check for mistakes.

if use_pyproject:
# Use pip to install pyproject.toml-based packages
# Map setup.py commands to pip commands
if "install" in sys.argv:
print("Start ", pkg_pyproject_path)
cmd = [sys.executable, '-m', 'pip', 'install', '.']
Copy link

Copilot AI Jan 23, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The code assumes pip is available by calling sys.executable -m pip. If pip is not installed or not available as a module, this will fail with a non-zero exit code. While the code prints a warning (line 105), it doesn't provide any guidance to the user about installing pip. Consider adding a more helpful error message when pip is not available, or checking for pip availability before attempting to use it.

Copilot uses AI. Check for mistakes.
elif "develop" in sys.argv or any(arg in sys.argv for arg in ['-e', '--editable']):
Copy link

Copilot AI Jan 23, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The command mapping logic checks for "develop" in sys.argv but also checks for '-e' and '--editable' flags. However, when using the root setup.py (e.g., python setup.py develop), sys.argv will contain "develop" but not '-e' or '--editable'. The check for these flags appears redundant and potentially confusing since they are pip-specific flags that wouldn't be passed to setup.py. Consider removing the editable flag checks or clarifying when they would be used.

Suggested change
elif "develop" in sys.argv or any(arg in sys.argv for arg in ['-e', '--editable']):
elif "develop" in sys.argv:

Copilot uses AI. Check for mistakes.
print("Start ", pkg_pyproject_path)
cmd = [sys.executable, '-m', 'pip', 'install', '-e', '.']
else:
# For other commands like --version, --help, etc., skip pyproject.toml packages
# These commands are meant for the root setup.py, not individual packages
Comment on lines +99 to +100
Copy link

Copilot AI Jan 23, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When commands other than "install" or "develop" are used, pyproject.toml packages are skipped entirely (line 101). This means commands like bdist_wheel, sdist, or other setup.py commands won't work for pyproject.toml packages. If these commands need to be supported, consider mapping them to equivalent pip/build commands or documenting this limitation clearly.

Suggested change
# For other commands like --version, --help, etc., skip pyproject.toml packages
# These commands are meant for the root setup.py, not individual packages
# For other commands (e.g., sdist, bdist_wheel, --version, --help, etc.),
# pyproject.toml-based packages are not handled per-package by this script.
# These commands are meant for the root setup.py, not individual packages.
print(
f"Skipping pyproject.toml-based package '{pkg_name}' for setup.py "
f"command(s) {sys.argv[1:]}. Only 'install' and 'develop' (or "
f"editable variants) are supported for per-package operations. "
"Use 'pip install' or 'python -m build' in the package directory "
"to build distribution artifacts for this package.",
file=sys.stderr,
)

Copilot uses AI. Check for mistakes.
continue

result = subprocess.run(cmd, cwd=pkg_setup_folder, capture_output=False)
if result.returncode != 0:
print(f"Warning: Package {pkg_name} installation returned non-zero exit code", file=sys.stderr)
Copy link

Copilot AI Jan 23, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The subprocess.run call uses capture_output=False, which means stdout and stderr will be printed to the console. However, the function doesn't check result.returncode to propagate failures. If a package installation fails, the script continues installing other packages and only prints a warning. This could lead to incomplete installations going unnoticed. Consider either:

  1. Raising an exception on non-zero return codes to halt the installation
  2. Collecting failures and reporting them at the end
  3. At minimum, using a more severe logging mechanism than just a warning
Suggested change
print(f"Warning: Package {pkg_name} installation returned non-zero exit code", file=sys.stderr)
print(f"Error: Package {pkg_name} installation returned non-zero exit code {result.returncode}", file=sys.stderr)
sys.exit(result.returncode)

Copilot uses AI. Check for mistakes.
elif os.path.exists(pkg_setup_path):
# Use the traditional setup.py approach
print("Start ", pkg_setup_path)
result = runpy.run_path(pkg_setup_path)
Comment on lines +87 to +109
Copy link

Copilot AI Jan 23, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The logic for handling packages with both setup.py and pyproject.toml is unclear. According to the PR description, "Packages with both → prefers setup.py for backward compatibility", but the code at line 87 prefers pyproject.toml over setup.py (using use_pyproject as the first condition). The conditions should be reordered to check for setup.py first if backward compatibility is the goal, or the PR description should be updated to match the actual behavior.

Copilot uses AI. Check for mistakes.
else:
print(f"Warning: No setup.py or pyproject.toml found for {pkg_name}", file=sys.stderr)
Copy link

Copilot AI Jan 23, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The broad exception handler catches all exceptions (including KeyboardInterrupt and SystemExit) and only prints them to stderr without re-raising. This means if a user tries to cancel installation with Ctrl+C, the script will continue processing other packages. Consider either:

  1. Catching specific exceptions (e.g., subprocess.CalledProcessError, OSError)
  2. Re-raising KeyboardInterrupt and SystemExit
  3. Using except Exception as e: instead of bare except to exclude system-exiting exceptions
Suggested change
print(f"Warning: No setup.py or pyproject.toml found for {pkg_name}", file=sys.stderr)
print(f"Warning: No setup.py or pyproject.toml found for {pkg_name}", file=sys.stderr)
except KeyboardInterrupt:
# Allow user-initiated interruption (e.g., Ctrl+C) to stop the script
raise

Copilot uses AI. Check for mistakes.
except Exception as e:
print(e, file=sys.stderr)
finally:
Expand Down
Loading