fix: resolve Windows single-GPU training crash and matplotlib compati…#2794
Open
unknown7751 wants to merge 4 commits into
Open
fix: resolve Windows single-GPU training crash and matplotlib compati…#2794unknown7751 wants to merge 4 commits into
unknown7751 wants to merge 4 commits into
Conversation
…bility - Skip mp.Process subprocess for single-GPU runs; call run() directly. On Windows, PyTorch spawn-based child processes silently crash during CUDA context initialization, preventing any training from starting. - Guard dist.init_process_group and DDP wrapping behind n_gpus > 1. - Set num_workers=0 on Windows to avoid nested subprocess spawning. Remove persistent_workers and prefetch_factor which require num_workers > 0. - Replace fig.canvas.tostring_rgb() with fig.canvas.buffer_rgba() in plot_spectrogram_to_numpy(). tostring_rgb() was removed in matplotlib 3.8. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
fairseq checkpoint_utils.load_model_ensemble_and_task calls torch.load without weights_only=False, which raises UnpicklingError in PyTorch 2.6+ where weights_only defaults to True. Apply the same patch used in train.py. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
…bility
Pull request checklist
The PR has a proper title. Use Semantic Commit Messages. (No more branch-name title please)
Make sure this is ready to be merged into the relevant branch. Please don't create a PR and let it hang for a few days.
Ensure you can run the codes you submitted succesfully. These submissions will be prioritized for review:
Introduce improvements in program execution speed;
Introduce improvements in synthesis quality;
Fix existing bugs reported by user feedback (or you met);
Introduce more convenient user operations.
PR type
Description
Screenshot