diff --git a/docs/proposals/global-build-system-dependencies-hook.md b/docs/proposals/global-build-system-dependencies-hook.md new file mode 100644 index 00000000..53c2e588 --- /dev/null +++ b/docs/proposals/global-build-system-dependencies-hook.md @@ -0,0 +1,188 @@ +# Global hook for build-system dependency post-processing + +- Author: Vikash Shaw +- Created: 2026-07-24 +- Status: Proposed +- GitHub issue: [#1263](https://github.com/python-wheel-build/fromager/issues/1263) +- Implementation PR: [#1271](https://github.com/python-wheel-build/fromager/pull/1271) +- Proposal PR: [#1272](https://github.com/python-wheel-build/fromager/pull/1272) + +## What + +This proposal suggests adding `get_build_system_dependencies` as a new +global hook point under `fromager.hooks`, so that downstream projects +can register hooks to post-process the build-system dependencies list +for all packages without needing per-package plugins. + +## Why + +Fromager currently provides two extension mechanisms: + +1. **Per-package plugins** (`fromager.project_overrides`): Override a + hook for a single package. When present, the plugin replaces the + default implementation entirely. + +2. **Global hooks** (`fromager.hooks`): Run for every package. Currently + support `post_build`, `post_bootstrap`, and `prebuilt_wheel`, which + are event callbacks that fire after an action has completed. + +Currently, no global hook runs during dependency resolution. When a +cross-cutting concern affects build dependencies for many packages, the +only option today is to write identical per-package plugins for each one. + +### Motivating example + +setuptools 81 removed support for the `setup.py --dry-run` option and +changed some related distutils/setuptools signatures. setuptools 82 +removed `pkg_resources` entirely. Many PyPI packages still reference +these removed APIs in their `setup.py`, causing build failures when +Fromager resolves an uncapped setuptools. + +In one downstream project, this led to **22 identical per-package +plugins**, each scanning `setup.py` to detect removed API usage and +appending a setuptools version cap. Every time a new package hits the +same incompatibility, another identical plugin must be added. This does +not scale well. + +### Why not `update_build_requires`? + +Fromager's YAML settings support `update_build_requires` for statically +adding build dependencies. However, the setuptools cap is conditional +and depends on what APIs a given `setup.py` actually uses. A static YAML +entry would either over-constrain all packages or still require +per-package entries, which reduces but does not eliminate the maintenance +overhead. + +## How + +### Execution order + +The hook runs inside `dependencies.get_build_system_dependencies()`, +after the per-package override (or default) returns and before marker +filtering: + +``` +1. Check for cached requirements file (early return if exists) +2. overrides.find_and_invoke() <-- per-package plugin or default +3. hooks.run_get_build_system_dependencies_hooks() <-- NEW +4. _filter_requirements() <-- marker evaluation +5. Write requirements cache file +``` + +Per-package plugins still produce the initial dependency list. Global +hooks can then augment it. Marker filtering happens last, so hooks do +not need to handle markers themselves. The result is cached, so hooks +run only once per package per build. + +### Input materialization + +`overrides.find_and_invoke()` returns `typing.Iterable[str]`. Before +invoking the first global hook, Fromager materializes this into a +`list[str]` (via `list(orig_deps)` in the implementation). This ensures +that hooks always receive a concrete list, so expressions like +`requirements + ["setuptools<82"]` work regardless of whether the +per-package plugin returned a generator, tuple, or set. + +### Hook signature + +```python +def get_build_system_dependencies( + *, + ctx: context.WorkContext, + req: Requirement, + sdist_root_dir: pathlib.Path, + build_dir: pathlib.Path, + requirements: list[str], +) -> list[str]: + ... +``` + +The hook receives the current requirements list and must return a +(possibly modified) `list[str]`. + +### Chaining and ordering + +Unlike the existing global hooks (`post_build`, `post_bootstrap`, +`prebuilt_wheel`), which are fire-and-forget callbacks that return +nothing, this hook chains: each hook receives the previous hook's output +and returns a modified list. + +Hook execution order follows stevedore's `HookManager` discovery order. +Because hooks chain, the order can matter if two hooks modify the same +requirement. Hooks should be designed to be order-independent where +possible (e.g., appending constraints rather than replacing entries). + +### Error handling + +If a hook raises an exception, the chain stops immediately and the +exception propagates up. The build for that package fails. Subsequent +hooks in the chain are not called. This matches the behavior of the +existing global hooks, where an exception in any hook is not swallowed. + +### Cache invalidation + +The cached `build-system-requirements.txt` is written after global hooks +run. If a global hook is installed, removed, or changed after a previous +build has cached requirements, the cached file must be cleared for the +hook to take effect. In practice, this means clearing the work directory +between builds when hook configuration changes. + +### Registration + +Hooks are registered as entry points under the `fromager.hooks` +namespace, the same way `post_build` and other existing hooks work: + +```toml +[project.entry-points."fromager.hooks"] +get_build_system_dependencies = "my_package.hooks:get_build_system_dependencies" +``` + +The hook could also be released as a standalone installable package +(e.g. `fromager-setuptools-hook`) so any Fromager user can opt in by +simply installing it, without writing any hook code themselves. + +### Example hook + +A minimal hook that appends a constraint: + +```python +def get_build_system_dependencies( + *, + ctx: context.WorkContext, + req: Requirement, + sdist_root_dir: pathlib.Path, + build_dir: pathlib.Path, + requirements: list[str], +) -> list[str]: + # Inspect sdist content and conditionally add constraints + if needs_constraint(build_dir): + return requirements + ["setuptools<82"] + return requirements +``` + +## Interaction with existing mechanisms + +| Mechanism | Scope | Relationship | +| -- | -- | -- | +| `update_build_requires` (YAML) | Per-package, static | Runs during `prepare_source`, before this hook. | +| `remove_build_requires` (YAML) | Per-package, static | Same as above. | +| Per-package plugin | Per-package, dynamic | Runs first. Global hooks receive its output. | +| Cached `build-system-requirements.txt` | Per-package | If cache exists, function returns early. Hooks do not run. | +| **Global hooks (this proposal)** | All packages, dynamic | Runs after per-package plugin, before marker filtering. | + +## Alternatives considered + +### Core logic in Fromager (PR [#1264](https://github.com/python-wheel-build/fromager/pull/1264)) + +An earlier approach proposed adding setuptools detection logic directly +into `default_get_build_system_dependencies`. This would have required +zero downstream changes, but was rejected because it makes Fromager +opinionated about a specific problem. Different downstream projects have +different needs, and the maintainer preferred keeping Fromager generic +with an opt-in hook mechanism instead. + +The global hook approach is more general-purpose: it can be used for +any cross-cutting build dependency concern, not just setuptools capping. +The same hook point could also be extended to other dependency types +(`get_build_backend_dependencies`, `get_build_sdist_dependencies`) in +the future if there is interest.