Skip to content

move the nightly env into its own file - #11481

Merged
keewis merged 25 commits into
pydata:mainfrom
keewis:separate-nightly
Aug 4, 2026
Merged

move the nightly env into its own file#11481
keewis merged 25 commits into
pydata:mainfrom
keewis:separate-nightly

Conversation

@keewis

@keewis keewis commented Jul 28, 2026

Copy link
Copy Markdown
Collaborator

This avoids having to build binary dependencies on e.g. RTD in order to lock the dependencies.

This won't actually do anything besides accelerating the locking, until we change the upstream-dev ci. For that we need to have parcels-code/pixi-lock support the manifest-path or working-directory options of setup-pixi (cc @VeckoTheGecko), and we also need to change the commands we use to run the tests. I may be able look into both this week.

Ultimately we're working around two restrictions in pixi with this:

  • conda dependencies are resolved first and cannot be changed when resolving pypi dependencies
  • source dependencies are built into wheels when locking to figure out dependencies, which means this always happens first, regardless of whether the environment is then actually used.

For reference, I saw @TomNicholas did something very similar for virtualizarr a couple of months ago, except that is using pip to modify the environment.

This avoids having to build binary dependencies on e.g. RTD in order to
lock the dependencies.
@github-actions github-actions Bot added the CI Continuous Integration tools label Jul 28, 2026
@github-actions github-actions Bot added the Automation Github bots, testing workflows, release automation label Jul 29, 2026
@keewis keewis added the run-upstream Run upstream CI label Aug 2, 2026
Comment thread .github/workflows/upstream-dev-ci.yaml Fixed
@keewis

keewis commented Aug 2, 2026

Copy link
Copy Markdown
Collaborator Author

this appears to be ready now. The failing upstream-dev tests are also visible on main (I'll bump the version of the reporting action in a separate PR so we actually get an issue for these).

Edit: not sure what to do about the mypy internal error. I'd probably investigate this in a separate PR.

Note that this depends on Parcels-code/pixi-lock#21, so we should merge only after that was merged and the references to my fork were removed (cc @VeckoTheGecko)

@keewis

keewis commented Aug 4, 2026

Copy link
Copy Markdown
Collaborator Author

this should be ready for reviews now (the nightly CI fails with the same errors on main, too)

@keewis

keewis commented Aug 4, 2026

Copy link
Copy Markdown
Collaborator Author

actually, I'll go ahead and merge this now. Any DX improvements (like creating a pixi task to run the nightly tests) can be separate PRs.

As usual, if this breaks anything we can revert and I'll investigate afterwards.

@keewis
keewis merged commit be846c4 into pydata:main Aug 4, 2026
29 of 31 checks passed
@keewis
keewis deleted the separate-nightly branch August 4, 2026 18:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Automation Github bots, testing workflows, release automation CI Continuous Integration tools run-upstream Run upstream CI

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants