CP-13290 Migrate Python packages from setup.cfg/setup.py to pyproject.toml

[SumoSourabh]

Problem

All five packages in this repo (common, dvp, libs, platform, tools) used the legacy setup.cfg + setup.py + requirements.txt packaging layout. This had a few real costs:

• Metadata fragmentation. Package metadata lived in setup.cfg, install deps in setup.py, dev deps in requirements.txt, and the version in a separate VERSION file. Five places to look, five places to drift.
• Non-standard install bootstrap. Each requirements.txt used local-path entries (./../common, etc.) to install sibling packages from source. This worked but is not a standard, portable pattern — it tied developer workflow to the repo layout.
• No PEP 621 metadata. Without pyproject.toml's [project] table, modern tooling (uv, hatch, Dependabot's pyproject support, IDEs) couldn't read the project metadata reliably. The package metadata couldn't be expressed in declarative form.
• Legacy LICENSE format. Apache LICENSE files still had the Copyright [yyyy] [name of copyright owner] placeholder text from the upstream template — never filled in for Delphix/Perforce.
• Inconsistent copyright headers. A number of config files and a few source files lacked the standard Delphix copyright header.
• build_project.sh was CI-hostile. It called tput unconditionally (fails on non-TTY runners), aborted on the first module failure, and didn't clean .egg-info directories between builds.

Solution

Packaging migration (per package)

For all five packages:

• Replaced setup.cfg + setup.py + requirements.txt with a single pyproject.toml using PEP 621 [project] metadata.
• Removed the VERSION files; the version now lives as a static string in [project].version. tools/package_util.get_version() was refactored to use importlib.metadata.version("dvp-tools") so it reads the installed-package version rather than a source-tree file.
• Inlined dependencies in [project.dependencies] and [project.optional-dependencies].dev. Dev deps that previously lived in requirements.txt are now published as a [dev] extra — pip install dvp-tools[dev] is now the standard way to install dev tooling.
• Sibling-package pins (e.g. dvp-common == 5.1.0) are static strings in the dependency list. .bumpversion.cfg was extended with stanzas for each pyproject.toml, so a single bumpversion run updates [project].version and every sibling pin atomically.
• License metadata uses the PEP 639 SPDX form: license = "Apache-2.0" with license-files = ["LICENSE"]. Requires setuptools >= 77, which is pinned in each [build-system].requires.

LICENSE files

All six LICENSE files (root + per-package) had the Apache template placeholder filled in with Copyright 2026 Delphix Corp., a Perforce company.

Copyright headers

Added or year-bumped the standard # Copyright (c) YYYY by Delphix. All rights reserved. header on config files that were missing it (.bumpversion.cfg, .github/dependabot.yml, .github/workflows/publish-python-packages.yml, docs/Pipfile, docs/build.sh, linkcheck-skip.txt) and on source files modified by this commit.

.bumpversion.cfg

• Bumped current_version from 5.0.1 to 5.1.0 (pre-existing drift; the VERSION files were already at 5.1.0).
• Removed the five VERSION-file stanzas (those files no longer exist).
• Added a stanza per pyproject.toml, so the default {current_version} search updates both [project].version and all sibling pins in one pass.

bin/build_project.sh

• Guarded tput calls with a TTY check (CI now runs cleanly).
• Added a failed_steps accumulator so all configured steps (build, flake8, pytest, coverage) run to completion and the script reports a summary instead of aborting on the first failure.
• Extended the clean step to remove *.egg-info directories alongside build/.
• Removed the -i flag from coverage report (coverage is now scoped via [tool.coverage.run] in tools/pyproject.toml).

CI workflow (.github/workflows/pre-commit.yml)

• Collapsed the per-package matrix into a single OS-only matrix; install all four packages once per OS in dependency order, then run pytest and flake8 per package as separate steps.
• Each test/lint step shows up as a named, independently red/green step in the GitHub UI, preserving failure attribution without 12-way matrix fan-out.

Dependency version pin

Bumped the dvp-api pin from 1.10.0.dev0 to 1.10.0.dev3 to align with the latest published TestPyPI dev release.

Testing

Sdist parity check (vs 5.1.0.dev0 on TestPyPI)

Built local sdists for all five packages and diffed against 5.1.0.dev0 (the last pre-migration sdist published).

• File-list diff: identical apart from + pyproject.toml, - setup.py, - src/.../VERSION — exactly the migration's intent.
• Install-dep name sets: exact match across all five packages. No accidental drops, no accidental additions.
• Package-data parity: tools/plugin_template/, tools/validation_schemas/, tools/codegen/ all present in the new sdist (verified via MANIFEST.in round-trip).
• Metadata format upgrade: as expected — License-Expression: Apache-2.0 replaces the Classifier: License :: ... form, Author-email: Delphix <…> replaces the legacy separate Author:/Author-email: fields, Project-URL: Homepage, … replaces the legacy Home-page: field.
• Improvement: the new sdists publish Provides-Extra: dev, which the old ones did not. Dev tooling is now installable via the standard [dev] extra mechanism instead of a separate requirements.txt.

Sdist build → wheel build round-trip

Built each package's sdist with python -m build --sdist, then ran pip wheel --no-deps against the resulting tarball. All five wheels build cleanly. This was specifically the failure mode of the earlier deps.toml-based design (sdists couldn't build into wheels because deps.toml wasn't packaged) and is now fixed.

Local dry-run of the new CI workflow

Created a fresh Python 3.11 virtualenv and ran the workflow steps end-to-end:

Step	Result
Install all four packages (editable, dependency order)	✅
pytest common (37 tests)	✅ All pass
pytest libs (56 tests)	✅ All pass
pytest platform (282 tests)	✅ All pass
pytest tools (385 tests, 1 skipped)	⚠ 1 pre-existing failure (see below)
flake8 common / libs / platform / tools (main + test)	✅ All clean

The single failing test (tools/.../test_cli.py::TestInitCli::test_name_required) passes in isolation but fails when run alongside its sibling tests in the same class. The test file hasn't been modified since 2019 and the migration touches no code under test. This is pre-existing test-class pollution that the new workflow surfaces; it does not block this PR.

bumpversion config validation

current_version = 5.1.0 now matches the static version strings in all five pyproject.toml files. The default {current_version} search in the new stanzas matches the version = "5.1.0" line and every sibling pin in one pass; a future bump will update all of them atomically.

Runtime version read

package_util.get_version() was rewritten to use importlib.metadata.version("dvp-tools"). After pip install -e ./tools, calling get_version() returns '5.1.0' — the same return shape as before. Both callers (cli.py:__version__ for dvp --version and plugin_dependency_util.py for plugin scaffolding) consume only the string value and are unaffected.

Bonus: stale test fix

Bundled in a second commit, CP-13290 Replace stale test_name_required with test_name_optional:

TestInitCli::test_name_required was asserting that dvp init fails without --name — pre-PYT-536 behavior. Commit 8f19557 made --plugin-name optional (the auto-generated plugin id falls back as the display name), but this test was never updated. It happened to pass on dev machines where dvp init fails for unrelated reasons (no Java on PATH, write permission, etc.) and only surfaced in CI now that Ubuntu runners have Java available and the migration's install-once workflow runs the tools suite.

Replaced with test_name_optional that captures the current contract: dvp init succeeds without --name, and initialize.init() is invoked with name=None.

[Copilot]

Pull request overview

Migrates the repo’s Python packaging for all SDK modules from legacy setup.py/setup.cfg/requirements.txt to PEP 621 pyproject.toml, while updating build/test automation and housekeeping (licenses/headers) to match the new packaging model.

Changes:

• Replaced legacy packaging files with pyproject.toml across common, libs, platform, tools, and dvp, and removed VERSION files.
• Updated tooling/CI scripts and workflows to build/test/install using the new packaging layout.
• Updated LICENSE placeholders and refreshed/added copyright headers; updated a tools CLI test to reflect --plugin-name being optional.

Reviewed changes

Copilot reviewed 47 out of 48 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
tools/src/test/python/dlpx/virtualization/_internal/test_cli.py	Updates CLI init test to assert --plugin-name is optional and initialize.init() is invoked with name=None.
tools/src/test/python/dlpx/virtualization/_internal/engine_version.cfg	Bumps copyright year in test config.
tools/src/main/python/dlpx/virtualization/_internal/VERSION	Removes legacy version file for tools package.
tools/src/main/python/dlpx/virtualization/_internal/package_util.py	Switches version lookup to installed distribution metadata (importlib.metadata).
tools/setup.py	Removes legacy setuptools build script for tools.
tools/setup.cfg	Removes legacy setuptools metadata config for tools.
tools/requirements.txt	Removes legacy dev requirements file for tools.
tools/pyproject.toml	Adds PEP 621 metadata, deps, dev extras, entrypoint, setuptools config, and coverage config for tools.
tools/MANIFEST.in	Stops packaging removed VERSION file.
tools/LICENSE	Fills in Apache 2.0 template copyright placeholder.
platform/src/test/python/dlpx/virtualization/fake_generated_definitions.py	Adds missing standard header to test helper.
platform/src/main/python/dlpx/virtualization/platform/VERSION	Removes legacy version file for platform package.
platform/setup.py	Removes legacy setuptools build script for platform.
platform/setup.cfg	Removes legacy setuptools metadata config for platform.
platform/requirements.txt	Removes legacy dev requirements file for platform.
platform/pyproject.toml	Adds PEP 621 metadata/deps/dev extras and setuptools config for platform.
platform/LICENSE	Fills in Apache 2.0 template copyright placeholder.
linkcheck-skip.txt	Adds standard header to linkcheck skip list.
LICENSE	Fills in Apache 2.0 template copyright placeholder at repo root.
libs/src/test/python/dlpx/virtualization/_engine/init.py	Adds package marker with standard header for tests.
libs/src/main/python/dlpx/virtualization/libs/VERSION	Removes legacy version file for libs package.
libs/setup.py	Removes legacy setuptools build script for libs.
libs/setup.cfg	Removes legacy setuptools metadata config for libs.
libs/requirements.txt	Removes legacy dev requirements file for libs.
libs/pyproject.toml	Adds PEP 621 metadata/deps/dev extras and setuptools config for libs.
libs/LICENSE	Fills in Apache 2.0 template copyright placeholder.
dvp/src/test/python/test_not_used.py	Updates header years and removes trailing whitespace line.
dvp/src/main/python/dlpx/virtualization/VERSION	Removes legacy version file for dvp package.
dvp/src/main/python/dlpx/virtualization/init.py	Updates header years.
dvp/src/main/python/dlpx/init.py	Updates header years.
dvp/setup.py	Removes legacy setuptools build script for dvp.
dvp/setup.cfg	Removes legacy setuptools metadata config for dvp.
dvp/requirements.txt	Removes legacy dev requirements file for dvp.
dvp/pyproject.toml	Adds PEP 621 metadata/deps/dev extras and setuptools config for dvp.
dvp/LICENSE	Fills in Apache 2.0 template copyright placeholder.
docs/Pipfile	Adds standard header.
docs/build.sh	Adds standard header.
common/src/main/python/dlpx/virtualization/common/VERSION	Removes legacy version file for common package.
common/setup.py	Removes legacy setuptools build script for common.
common/setup.cfg	Removes legacy setuptools metadata config for common.
common/requirements.txt	Removes legacy dev requirements file for common.
common/pyproject.toml	Adds PEP 621 metadata/deps/dev extras and setuptools config for common.
common/LICENSE	Fills in Apache 2.0 template copyright placeholder.
bin/build_project.sh	Makes build/test/lint script more CI-friendly and aggregates failures; updates install/clean behavior for pyproject-based builds.
.github/workflows/publish-python-packages.yml	Updates publishing workflow to use python -m build and newer GitHub Actions versions.
.github/workflows/pre-commit.yml	Simplifies CI by installing all packages once per OS and running per-package test/lint steps.
.github/dependabot.yml	Adds standard header.
.bumpversion.cfg	Updates version bump configuration to target pyproject.toml instead of removed VERSION files.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

Pull request overview

Copilot reviewed 47 out of 48 changed files in this pull request and generated no new comments.