Skip to content

fix: run of circular-dependencies with python3.15#7396

Open
knst wants to merge 2 commits into
dashpay:developfrom
knst:fix-multiprocess-3.15
Open

fix: run of circular-dependencies with python3.15#7396
knst wants to merge 2 commits into
dashpay:developfrom
knst:fix-multiprocess-3.15

Conversation

@knst

@knst knst commented Jul 1, 2026

Copy link
Copy Markdown
Collaborator

Issue being fixed or feature implemented

Multiprocess uses dill to pickle the nested handle_module2 closure and dill's Python 3.15 support is broken (co_lnotab was removed from code objects).

What was done?

Moved some functions and variables to global namespace.

How Has This Been Tested?

Run test/lint/lint-circular-dependencies.py with python3.15
Review hint: use git show -w --color-moved=dimmed-zebra

Breaking Changes

Please describe any breaking changes your code introduces

Checklist:

  • I have performed a self-review of my own code
  • I have commented my code, particularly in hard-to-understand areas
  • I have added or updated relevant unit/integration/functional/e2e tests
  • I have made corresponding changes to the documentation
  • I have assigned this pull request to a milestone (for repository code-owners and collaborators only)

@knst knst added this to the 24 milestone Jul 1, 2026
@thepastaclaw

thepastaclaw commented Jul 1, 2026

Copy link
Copy Markdown

✅ Review complete (commit 7cb0c29)

@github-actions

github-actions Bot commented Jul 1, 2026

Copy link
Copy Markdown

⚠️ Potential Merge Conflicts Detected

This PR has potential conflicts with the following open PRs:

Please coordinate with the authors of these PRs to avoid merge conflicts.

@knst knst requested review from PastaPastaPasta and UdjinM6 July 1, 2026 07:45

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 0fe8e0553b

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

with Pool(8) as pool:
# Use fork so workers inherit the populated `deps` global without
# having to pickle it for every task.
with multiprocessing.get_context("fork").Pool(8) as pool:

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Avoid requiring the fork start method

When this devtool is run with a Windows Python or any other platform that lacks the fork start method, this line raises before any dependency checking runs, which also breaks test/lint/lint-circular-dependencies.py for those environments. The previous multiprocess.Pool path could serialize the nested closure under spawn, so this Python 3.15 fix regresses non-fork platforms; consider passing/initializing deps for workers instead of hard-coding fork.

Useful? React with 👍 / 👎.

@coderabbitai

coderabbitai Bot commented Jul 1, 2026

Copy link
Copy Markdown

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro

Run ID: e0ec1644-82cf-4db3-a366-b173898e282c

📥 Commits

Reviewing files that changed from the base of the PR and between fa97d7a and 7cb0c29.

📒 Files selected for processing (2)
  • contrib/containers/ci/ci-slim.Dockerfile
  • contrib/devtools/circular-dependencies.py
💤 Files with no reviewable changes (1)
  • contrib/containers/ci/ci-slim.Dockerfile
🚧 Files skipped from review as they are similar to previous changes (1)
  • contrib/devtools/circular-dependencies.py

Walkthrough

contrib/devtools/circular-dependencies.py now uses the standard library multiprocessing module, with files, deps, and handle_module2 moved to module scope. The cycle search uses a fork-based process pool when available, and falls back to serial mapping otherwise. The CI slim Dockerfile also removes the pinned multiprocess package from its install list.

Estimated code review effort: 2 (Simple) | ~10 minutes

Related PRs: None identified.

Suggested labels: contrib, devtools, refactor

Suggested reviewers: None identified.

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Title check ✅ Passed The title clearly points to the circular-dependencies Python 3.15 compatibility fix.
Description check ✅ Passed The description matches the change by explaining the multiprocess/dill issue and the move to global scope.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@thepastaclaw thepastaclaw left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

Small dev-tool fix replacing third-party multiprocess (broken on Python 3.15 due to dill/co_lnotab) with stdlib multiprocessing using an explicit fork context, hoisting handle_module2/files/deps to module scope. Behavior is preserved — the pool is recreated inside the outer loop, so each forked worker set inherits an up-to-date deps. Affects only contrib/devtools/circular-dependencies.py; no consensus or runtime code touched.

@UdjinM6

UdjinM6 commented Jul 3, 2026

Copy link
Copy Markdown

pls consider adeb8dd

knst and others added 2 commits July 3, 2026 17:46
Multiprocess uses dill to pickle the nested handle_module2 closure
and dill's Python 3.15 support is broken (co_lnotab was removed from code objects).

Review hint: use `git show -w --color-moved=dimmed-zebra`
…ess dep

get_context("fork") raises ValueError on platforms without fork (Windows),
so guard it and run handle_module2 serially there. Also remove the now-unused
multiprocess pip package, whose only consumer was this script.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@knst knst force-pushed the fix-multiprocess-3.15 branch from fa97d7a to 7cb0c29 Compare July 3, 2026 10:47

@thepastaclaw thepastaclaw left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

Small, well-scoped dev-tool fix replacing the third-party multiprocess package with stdlib multiprocessing to unblock Python 3.15 (dill's 3.15 support is broken due to co_lnotab removal). Second commit adds a serial-map fallback for platforms without fork (Windows) and drops the now-unused pip package from the slim CI image. Both changes are correct: handle_module2 and its deps/files globals are hoisted to module scope so fork workers inherit the populated dependency graph, and the serial path runs in the same process where the globals are already populated. No carried-forward findings from the prior 0fe8e05 review (which was clean) and no new blocking or suggestion-level issues in the latest delta.

Source: reviewers opus/opus, sonnet5/claude-sonnet-5, codex/gpt-5.5[high] (failed: initialize stall) for general + dash-core-commit-history; verifier opus/opus.

@thepastaclaw

Copy link
Copy Markdown

Thanks. The current PR head 7cb0c292f3325a619e1dc04d63237c993f74a025 already contains the same patch as adeb8dda470fbf3c48d099068800052d71cf0cf5 (same diff/message); I re-reviewed and approved that head.

@UdjinM6 UdjinM6 left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

utACK 7cb0c29

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants