BUG: ensure build options are preserved from an existing build directory#852
BUG: ensure build options are preserved from an existing build directory#852Alvaro-Kothe wants to merge 2 commits into
Conversation
rgommers
left a comment
There was a problem hiding this comment.
Thanks @Alvaro-Kothe, this PR is helpful, it looks like a clean solution. It shows that this can be done without anything special, and it makes these build options behave like all other options on a reconfigure, so they're less special while keeping their intended defaults.
This is a behavior change, but I think we can get away with it. The one relevant situation we had before is building invoking meson setup separately to pass build options, before a subsequent pip install command. That was necessary in some cases in the early days, because IIRC there were some limitations on what could be passed directly via pip or pypa/build, especially relevant for cross compilation. Here is an example I remember:
- https://github.com/conda-forge/scipy-feedstock/blob/06d0da3ce0e753836a6cce9cc6db2f515c41ae74/recipe/build.sh
- conda-forge/scipy-feedstock#205 (comment)
It's possible that on less actively maintained distros, such recipes are still in place. If they don't pass -Dbuildtype=release and the other options themselves (-Dn_debug=if-release is likely to be missing), then that will be broken by this PR, possibly silently at build time.
I think the change is a good one, but we'll have to do some more digging to find cases like the one above that could be affected, and see if they can be updated.
| setup_args.insert(0, '--reconfigure') | ||
| self._run(self._meson + ['setup', *setup_args]) | ||
|
|
||
| if not reconfigure: |
There was a problem hiding this comment.
This is the only change, the rest is just a reshuffle. It would be useful to expand the code comment, something like:
# default build options, overriding Meson's defaults for them.
# on a reconfigure, the values will persist unless the user explicitly
# passes new values for them.|
|
||
| # reconfiguration with a new option | ||
| project = mesonpy.Project(package_simple, tmp_path, meson_args={ | ||
| 'setup': ['-Doptimization=2'], |
There was a problem hiding this comment.
This test is correct but a little obscure, given that it doesn't directly checks default/updated values of any of the build options that meson-python actually overrides. Perhaps also assert that the buildtype is and remains debug.
While we're at it here, it looks like there is no test that checks that the default buildtype is release nor the default for b_vscrt. There is an test for ndebug in test_options.py. Would you mind adding a new test there checking that the defaults are as expected on a clean build?
There was a problem hiding this comment.
Perhaps also assert that the buildtype is and remains debug.
I added some assertions in the initial configuration and in the reconfiguration, but I feel like most of them were redundant, considering the last assertion initial_options == reconfigure_options
There is an test for ndebug in test_options.py. Would you mind adding a new test there checking that the defaults are as expected on a clean build?
Created a new test in test_options.py, but I am not sure if the vscrt assertion is correct, since I am not on Windows and couldn't test it.
I checked these distros: conda-forge, Spack, Nix, OpenEmbedded/Yocto, Buildroot, Arch, Alpine, Void, Gentoo, Debian, Fedora. None would be affected. There are some usages in individual packages and end user dev recipes with the That all uses patterns that are no longer needed, but used to be - so if we want to make this change, we should probably at a minimum emit clear warnings for one release cycle. |
| setup_args = [ | ||
| setup_cmd = self._meson + ['setup'] | ||
|
|
||
| if reconfigure: | ||
| setup_cmd.append('--reconfigure') | ||
|
|
||
| setup_cmd += [ | ||
| os.fspath(self._source_dir), | ||
| os.fspath(self._build_dir), | ||
| # default build options | ||
| '-Dbuildtype=release', | ||
| '-Db_ndebug=if-release', | ||
| '-Db_vscrt=md', | ||
| # user build options | ||
| *self._meson_args['setup'], | ||
| # pass native file last to have it override the python | ||
| # interpreter path that may have been specified in user | ||
| # provided native files | ||
| f'--native-file={os.fspath(self._meson_native_file)}', | ||
| ] | ||
| if reconfigure: | ||
| setup_args.insert(0, '--reconfigure') | ||
| self._run(self._meson + ['setup', *setup_args]) | ||
|
|
||
| if not reconfigure: | ||
| # default build options | ||
| setup_cmd += [ | ||
| '-Dbuildtype=release', | ||
| '-Db_ndebug=if-release', | ||
| '-Db_vscrt=md', | ||
| ] | ||
|
|
||
| # user build options | ||
| setup_cmd += self._meson_args['setup'] | ||
|
|
||
| # pass native file last to have it override the python | ||
| # interpreter path that may have been specified in user | ||
| # provided native files | ||
| setup_cmd.append(f'--native-file={os.fspath(self._meson_native_file)}') | ||
|
|
||
| self._run(setup_cmd) |
There was a problem hiding this comment.
I haven't thought much about the implications of this change. However, the implementation can be much more linear:
# Override Meson default build options to adjust the build
# process to the task of building Python wheels. These
# defaults can be overridden by user options. To preserve
# user specified build options, do not pass these defaults
# when reconfiguring an existing build directory.
defaults = [
'-Dbuildtype=release',
'-Db_ndebug=if-release',
'-Db_vscrt=md',
] if not reconfigure else []
setup_args = [
os.fspath(self._source_dir),
os.fspath(self._build_dir),
# default build options
*defaults
# user build options
*self._meson_args['setup'],
# pass native file last to have it override the python
# interpreter path that may have been specified in user
# provided native files
f'--native-file={os.fspath(self._meson_native_file)}',
]and the rest remains the same.
There was a problem hiding this comment.
Changed. I opted to not use the ternary, since the list is a little big.
|
I don't think this satisfies the user requirement in all cases. Let's assume that the [tool.meson-python.args]
setup = ['-Dbuildtype=release']If the user installs the package like this |
|
I think that's a different requirement, and I'm not sure how valid it is (it's certainly more niche). I'd expect that package authors only hardcode build options in It could be addressed, but wouldn't that require keeping track of state explicitly? Right now everything is stateless, aside from what's in |
| @@ -899,23 +899,33 @@ def _run(self, cmd: Sequence[str]) -> None: | |||
|
|
|||
There was a problem hiding this comment.
There is also an imprecision in the commit message: the reconfigure argument is passed when the specified build directory is detected to contain a Meson build dir, not when it merely exist. And, as @rgommers already pointed out (IIRC) buildtype is not the only default argument passed. I would reword the commit message like this
BUG: do no pass default options when reconfiguring
This ensures build options specified by the user when the build was
first initialized are preserved, unless explicitly overridden.
Fixes #851.
There was a problem hiding this comment.
Updated. Do I need to change the PR description too?
There was a problem hiding this comment.
Thanks. Updating the PR description as well may be useful - at least for me, this captured the essence of the change, and I didn't quite get it from the initial issue or PR description.
This ensures build options specified by the user when the build was first initialized are preserved, unless explicitly overridden. Fixes mesonbuild#851.
210b727 to
8538505
Compare
|
From #852 (comment)
From #852 (comment)
How do I emit warnings? I tried to use Also, should the warning be a deprecation warning, i.e., create another PR/commit just for the warning, or should it warn on the current behaviour change? The first link seems a little bit problematic with this change... It seems that they will end up creating |
Let's first check with @dnicolodi whether he agrees with a one-release warning cycle or not. One could also make a case about that probably not reaching the people who need to see it most.
That's unavoidable I think, since it swallows all output. |
8538505 to
59cbf82
Compare
59cbf82 to
1af9044
Compare
This ensures build options specified by the user when the build was first initialized are preserved, unless explicitly overridden.
Fixes #851.