[TTS][Docs][MagpieTTS] Add MagpieTTS finetuning docs by subhankar-ghosh · Pull Request #15546 · NVIDIA-NeMo/NeMo

subhankar-ghosh · 2026-03-25T00:06:19Z

Important

The Update branch button must only be pressed in very rare occassions.
An outdated branch is never blocking the merge of a PR.
Please reach out to the automation team before pressing that button.

What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

Collection: [Note which collection this PR will affect]

Changelog

Add specific line by line info of high level changes in this PR.

Usage

You can potentially add a usage example below

# Add a code snippet demonstrating how to use this

GitHub Actions CI

The Jenkins CI system has been replaced by GitHub Actions self-hosted runners.

The GitHub Actions CI will run automatically when the "Run CICD" label is added to the PR.
To re-run CI remove and add the label again.
To run CI on an untrusted fork, a NeMo user with write access must first click "Approve and run".

Before your PR is "Ready for review"

Pre checks:

Make sure you read and followed Contributor guidelines
Did you write any new necessary tests?
Did you add or update any necessary documentation?
Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
- Reviewer: Does the PR have correct import guards for all optional libraries?

PR Type:

New Feature
Bugfix
Documentation

If you haven't finished some of the above items you can still open "Draft" PR.

Who can review?

Anyone in the community is free to review the PR once the checks have passed.
Contributor guidelines contains specific people who can review PRs to various areas.

Additional Information

Related to # (issue)

Signed-off-by: subhankar-ghosh <subhankar2321@gmail.com>

subhankar-ghosh · 2026-03-25T00:06:42Z

@shehzeen Can you please check the GRPO part of the docs please?

github-actions · 2026-03-25T06:15:30Z

[🤖]: Hi @subhankar-ghosh 👋,

We wanted to let you know that a CICD pipeline for this PR just finished successfully.

So it might be time to merge this PR or get some approvals.

docs/source/tts/magpietts-finetuning.rst

Signed-off-by: subhankar-ghosh <subhankar2321@gmail.com>

* Test no extras docs build Signed-off-by: Charlie Truong <chtruong@nvidia.com> * ci: Update docs job to use 0.83.0 templates Signed-off-by: Charlie Truong <chtruong@nvidia.com> * Uncomment push cases for build-docs github action Signed-off-by: Charlie Truong <chtruong@nvidia.com> --------- Signed-off-by: Charlie Truong <chtruong@nvidia.com>

Signed-off-by: Subhankar Ghosh <subhankarg@nvidia.com>

Signed-off-by: Vladimir Bataev <vbataev@nvidia.com>

Upgrades actions to versions compatible with the Node.js 24 runtime: - actions/checkout: → v6 - actions/upload-artifact: → v6 - actions/download-artifact: → v7 - actions/github-script: → v8 - actions/setup-python: → v6 Mirrors: NVIDIA/Megatron-LM@1d5e68b Signed-off-by: oliver könig <okoenig@nvidia.com>

* Update README.md Signed-off-by: Jason <jasoli@nvidia.com> * Revise Nemotron VoiceChat release details in README Updated the release information for Nemotron VoiceChat and added details about its features and early access. Signed-off-by: zhehuaichen <139396994+zhehuaichen@users.noreply.github.com> --------- Signed-off-by: Jason <jasoli@nvidia.com> Signed-off-by: zhehuaichen <139396994+zhehuaichen@users.noreply.github.com> Co-authored-by: zhehuaichen <139396994+zhehuaichen@users.noreply.github.com>

* initial commit for end-of-utterance detection Signed-off-by: Weiqing Wang <weiqingw@nvidia.com> * change targets to long() type Signed-off-by: Weiqing Wang <weiqingw@nvidia.com> * change output_types() Signed-off-by: Weiqing Wang <weiqingw@nvidia.com> * add random padding and refactor for multiple utterances per sample Signed-off-by: stevehuang52 <heh@nvidia.com> * add handling multiple text groundtruth Signed-off-by: stevehuang52 <heh@nvidia.com> * update and add eval scripts Signed-off-by: stevehuang52 <heh@nvidia.com> * drop sou label and add eob label Signed-off-by: stevehuang52 <heh@nvidia.com> * update hybrid-rnnt-ctc and rnnt models to use eou dataset Signed-off-by: stevehuang52 <heh@nvidia.com> * set default return eou frame label to false Signed-off-by: stevehuang52 <heh@nvidia.com> * handle empty utterance Signed-off-by: stevehuang52 <heh@nvidia.com> * add script for injecting special eou tokens into SPE tokenizer Signed-off-by: stevehuang52 <heh@nvidia.com> * refactor eou eval utils Signed-off-by: stevehuang52 <heh@nvidia.com> * add eou rnnt training Signed-off-by: stevehuang52 <heh@nvidia.com> * update doc Signed-off-by: stevehuang52 <heh@nvidia.com> * update data augmentation Signed-off-by: stevehuang52 <heh@nvidia.com> * update data related functions Signed-off-by: stevehuang52 <heh@nvidia.com> * fix tokenizer with eou tokens Signed-off-by: stevehuang52 <heh@nvidia.com> * adding eou force aligner Signed-off-by: Weiqing Wang <weiqingw@nvidia.com> * update for eou Signed-off-by: stevehuang52 <heh@nvidia.com> * fix the case when 'segments_level_ctm_filepath' is not produced Signed-off-by: Weiqing Wang <weiqingw@nvidia.com> * fix force aligner Signed-off-by: stevehuang52 <heh@nvidia.com> * fix aligner Signed-off-by: stevehuang52 <heh@nvidia.com> * update for asr-eou Signed-off-by: stevehuang52 <heh@nvidia.com> * clean up and update infer Signed-off-by: stevehuang52 <heh@nvidia.com> * update Signed-off-by: stevehuang52 <heh@nvidia.com> * update Signed-off-by: stevehuang52 <heh@nvidia.com> * fix rnnt_decoding for empty string Signed-off-by: stevehuang52 <heh@nvidia.com> * update cfg Signed-off-by: stevehuang52 <heh@nvidia.com> * update cfg Signed-off-by: stevehuang52 <heh@nvidia.com> * update padding augment Signed-off-by: stevehuang52 <heh@nvidia.com> * update Signed-off-by: stevehuang52 <heh@nvidia.com> * update Signed-off-by: stevehuang52 <heh@nvidia.com> * update cfg Signed-off-by: stevehuang52 <heh@nvidia.com> * fix eob metric logging Signed-off-by: stevehuang52 <heh@nvidia.com> * refactor and add hybrid model Signed-off-by: stevehuang52 <heh@nvidia.com> * update cfg Signed-off-by: stevehuang52 <heh@nvidia.com> * update EOU models Signed-off-by: stevehuang52 <heh@nvidia.com> * update cfg Signed-off-by: stevehuang52 <heh@nvidia.com> * update Signed-off-by: stevehuang52 <heh@nvidia.com> * refactor percentile calculation Signed-off-by: stevehuang52 <heh@nvidia.com> * update augmentation Signed-off-by: stevehuang52 <heh@nvidia.com> * update cfg Signed-off-by: stevehuang52 <heh@nvidia.com> * update model and cfg Signed-off-by: stevehuang52 <heh@nvidia.com> * update frame eou Signed-off-by: stevehuang52 <heh@nvidia.com> * update cfg Signed-off-by: stevehuang52 <heh@nvidia.com> * add adapter to eou Signed-off-by: stevehuang52 <heh@nvidia.com> * remove pdb Signed-off-by: stevehuang52 <heh@nvidia.com> * update cfg Signed-off-by: stevehuang52 <heh@nvidia.com> * update cfg Signed-off-by: stevehuang52 <heh@nvidia.com> * update cfg Signed-off-by: stevehuang52 <heh@nvidia.com> * update Signed-off-by: stevehuang52 <heh@nvidia.com> * add cfg Signed-off-by: stevehuang52 <heh@nvidia.com> * fix eou metric Signed-off-by: stevehuang52 <heh@nvidia.com> * update adapter Signed-off-by: stevehuang52 <heh@nvidia.com> * add scripts Signed-off-by: stevehuang52 <heh@nvidia.com> * update docstring Signed-off-by: stevehuang52 <heh@nvidia.com> * update Signed-off-by: stevehuang52 <heh@nvidia.com> * update Signed-off-by: stevehuang52 <heh@nvidia.com> * update generate eval data Signed-off-by: stevehuang52 <heh@nvidia.com> * update eou val Signed-off-by: stevehuang52 <heh@nvidia.com> * update Signed-off-by: stevehuang52 <heh@nvidia.com> * add drop_pnc=true as default for dataloading Signed-off-by: stevehuang52 <heh@nvidia.com> * update Signed-off-by: stevehuang52 <heh@nvidia.com> * update cfg Signed-off-by: stevehuang52 <heh@nvidia.com> * update Signed-off-by: stevehuang52 <heh@nvidia.com> * update Signed-off-by: stevehuang52 <heh@nvidia.com> * update Signed-off-by: stevehuang52 <heh@nvidia.com> * fix miss rate Signed-off-by: stevehuang52 <heh@nvidia.com> * update Signed-off-by: stevehuang52 <heh@nvidia.com> * add ignore_eob_label Signed-off-by: stevehuang52 <heh@nvidia.com> * fix and update Signed-off-by: stevehuang52 <heh@nvidia.com> * improve lhotse augmentation Signed-off-by: stevehuang52 <heh@nvidia.com> * update cfg Signed-off-by: stevehuang52 <heh@nvidia.com> * update cfg Signed-off-by: stevehuang52 <heh@nvidia.com> * update Signed-off-by: stevehuang52 <heh@nvidia.com> * update Signed-off-by: stevehuang52 <heh@nvidia.com> * add debug info Signed-off-by: stevehuang52 <heh@nvidia.com> * improve data augmentation Signed-off-by: stevehuang52 <heh@nvidia.com> * update utils Signed-off-by: stevehuang52 <heh@nvidia.com> * update Signed-off-by: stevehuang52 <heh@nvidia.com> * update Signed-off-by: stevehuang52 <heh@nvidia.com> * update dataloader Signed-off-by: stevehuang52 <heh@nvidia.com> * update oomptimizer Signed-off-by: stevehuang52 <heh@nvidia.com> * update oomptimizer Signed-off-by: stevehuang52 <heh@nvidia.com> * update eou model Signed-off-by: stevehuang52 <heh@nvidia.com> * update eou model Signed-off-by: stevehuang52 <heh@nvidia.com> * update eou model Signed-off-by: stevehuang52 <heh@nvidia.com> * update augmentation Signed-off-by: stevehuang52 <heh@nvidia.com> * update aug Signed-off-by: stevehuang52 <heh@nvidia.com> * update augment Signed-off-by: stevehuang52 <heh@nvidia.com> * update Signed-off-by: stevehuang52 <heh@nvidia.com> * update drop pnc func Signed-off-by: stevehuang52 <heh@nvidia.com> * update eou finetune Signed-off-by: stevehuang52 <heh@nvidia.com> * update transcribe Signed-off-by: stevehuang52 <heh@nvidia.com> * update cfg Signed-off-by: stevehuang52 <heh@nvidia.com> * fix cfg Signed-off-by: stevehuang52 <heh@nvidia.com> * clean up for PR Signed-off-by: stevehuang52 <heh@nvidia.com> * clean up Signed-off-by: stevehuang52 <heh@nvidia.com> * Potential fix for code scanning alert no. 16191: Explicit returns mixed with implicit (fall through) returns Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com> Signed-off-by: He Huang (Steve) <105218074+stevehuang52@users.noreply.github.com> * Potential fix for code scanning alert no. 16190: Explicit returns mixed with implicit (fall through) returns Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com> Signed-off-by: He Huang (Steve) <105218074+stevehuang52@users.noreply.github.com> * Apply isort and black reformatting Signed-off-by: stevehuang52 <stevehuang52@users.noreply.github.com> * Potential fix for code scanning alert no. 16185: File is not always closed Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com> Signed-off-by: He Huang (Steve) <105218074+stevehuang52@users.noreply.github.com> * clean up Signed-off-by: stevehuang52 <heh@nvidia.com> * clean up Signed-off-by: stevehuang52 <heh@nvidia.com> * fix pylint&flake8 Signed-off-by: stevehuang52 <heh@nvidia.com> * fix pylint Signed-off-by: stevehuang52 <heh@nvidia.com> * refactor Signed-off-by: stevehuang52 <heh@nvidia.com> * update pr Signed-off-by: stevehuang52 <heh@nvidia.com> * update adapter Signed-off-by: stevehuang52 <heh@nvidia.com> * clean up Signed-off-by: stevehuang52 <heh@nvidia.com> * update readme, test, etc Signed-off-by: He Huang <heh@nvidia.com> * Apply isort and black reformatting Signed-off-by: stevehuang52 <stevehuang52@users.noreply.github.com> * update doc Signed-off-by: He Huang <heh@nvidia.com> * clean up Signed-off-by: He Huang <heh@nvidia.com> * fix and rename Signed-off-by: He Huang <heh@nvidia.com> * update doc Signed-off-by: He Huang <heh@nvidia.com> * clean up Signed-off-by: He Huang <heh@nvidia.com> * move all length aug to invalid Signed-off-by: He Huang <heh@nvidia.com> * fix typo Signed-off-by: He Huang <heh@nvidia.com> * rename and move to scripts/asr_eou Signed-off-by: He Huang <heh@nvidia.com> * fix ci Signed-off-by: He Huang <heh@nvidia.com> * fix ci Signed-off-by: He Huang <heh@nvidia.com> * clean up Signed-off-by: He Huang <heh@nvidia.com> * clean up Signed-off-by: He Huang <heh@nvidia.com> * fix linting Signed-off-by: He Huang <heh@nvidia.com> * fix ci Signed-off-by: He Huang <heh@nvidia.com> * Apply isort and black reformatting Signed-off-by: stevehuang52 <stevehuang52@users.noreply.github.com> * Potential fix for code scanning alert no. 17270: Explicit export is not defined Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com> Signed-off-by: He Huang (Steve) <105218074+stevehuang52@users.noreply.github.com> * Potential fix for code scanning alert no. 17271: Explicit export is not defined Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com> Signed-off-by: He Huang (Steve) <105218074+stevehuang52@users.noreply.github.com> * Potential fix for code scanning alert no. 17272: Explicit export is not defined Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com> Signed-off-by: He Huang (Steve) <105218074+stevehuang52@users.noreply.github.com> --------- Signed-off-by: Weiqing Wang <weiqingw@nvidia.com> Signed-off-by: stevehuang52 <heh@nvidia.com> Signed-off-by: He Huang (Steve) <105218074+stevehuang52@users.noreply.github.com> Signed-off-by: stevehuang52 <stevehuang52@users.noreply.github.com> Signed-off-by: He Huang <heh@nvidia.com> Co-authored-by: Weiqing Wang <weiqingw@nvidia.com> Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com> Co-authored-by: stevehuang52 <stevehuang52@users.noreply.github.com>

docs/source/tts/magpietts-finetuning.rst

Signed-off-by: subhankar-ghosh <subhankar2321@gmail.com>

Add MagpieTTS finetuning docs

d2457dd

Signed-off-by: subhankar-ghosh <subhankar2321@gmail.com>

subhankar-ghosh requested review from blisc and shehzeen March 25, 2026 00:06

subhankar-ghosh added documentation Improvements or additions to documentation TTS Run CICD labels Mar 25, 2026

chtruong814 added Run CICD and removed Run CICD labels Mar 25, 2026

chtruong814 temporarily deployed to test March 25, 2026 00:07 — with GitHub Actions Inactive

subhankar-ghosh enabled auto-merge (squash) March 25, 2026 00:10

github-actions bot removed the Run CICD label Mar 25, 2026

Merge branch 'main' into magpietts_finetuning

7853ff0

blisc requested changes Mar 30, 2026

View reviewed changes

subhankar-ghosh and others added 7 commits March 31, 2026 09:06

Finetuning docs review changes

7b4fc36

Signed-off-by: subhankar-ghosh <subhankar2321@gmail.com>

Rename index for attention prior weights (#15551)

7dd32cf

Signed-off-by: Subhankar Ghosh <subhankarg@nvidia.com>

Ignore PnC for WER calculation: streaming ASR inference (#15550)

15249c0

Signed-off-by: Vladimir Bataev <vbataev@nvidia.com>

github-actions bot added ASR CI common labels Mar 31, 2026

subhankar-ghosh closed this Mar 31, 2026

auto-merge was automatically disabled March 31, 2026 16:18
Pull request was closed

subhankar-ghosh reopened this Mar 31, 2026

github-actions bot removed ASR CI labels Mar 31, 2026

github-actions bot removed the common label Mar 31, 2026

subhankar-ghosh enabled auto-merge (squash) March 31, 2026 16:22

subhankar-ghosh requested a review from blisc March 31, 2026 16:24

blisc reviewed Apr 1, 2026

View reviewed changes

docs/source/tts/magpietts-finetuning.rst Show resolved Hide resolved

subhankar-ghosh requested a review from blisc April 1, 2026 16:58

blisc previously approved these changes Apr 1, 2026

View reviewed changes

subhankar-ghosh added 2 commits April 2, 2026 09:15

Fix freesound url

3784ec7

Signed-off-by: subhankar-ghosh <subhankar2321@gmail.com>

Merge branch 'main' of github.com:NVIDIA/NeMo into magpietts_finetuning

5fc635a

subhankar-ghosh dismissed blisc’s stale review via 5fc635a April 2, 2026 16:16

subhankar-ghosh force-pushed the magpietts_finetuning branch from 9f3931a to 5fc635a Compare April 2, 2026 16:16

github-actions bot added the ASR label Apr 2, 2026

Fix freesound url

fee82bb

Signed-off-by: subhankar-ghosh <subhankar2321@gmail.com>

blisc approved these changes Apr 2, 2026

View reviewed changes

subhankar-ghosh added Run CICD labels Apr 3, 2026

subhankar-ghosh temporarily deployed to test April 3, 2026 14:16 — with GitHub Actions Inactive

github-actions bot removed the Run CICD label Apr 3, 2026

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[TTS][Docs][MagpieTTS] Add MagpieTTS finetuning docs#15546

[TTS][Docs][MagpieTTS] Add MagpieTTS finetuning docs#15546
subhankar-ghosh wants to merge 12 commits intomainfrom
magpietts_finetuning

subhankar-ghosh commented Mar 25, 2026

Uh oh!

subhankar-ghosh commented Mar 25, 2026

Uh oh!

github-actions bot commented Mar 25, 2026

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

Conversation

subhankar-ghosh commented Mar 25, 2026

What does this PR do ?

Changelog

Usage

GitHub Actions CI

Before your PR is "Ready for review"

Who can review?

Additional Information

Uh oh!

subhankar-ghosh commented Mar 25, 2026

Uh oh!

github-actions bot commented Mar 25, 2026

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants