Skip to content

Fix HuBERT positional batch norm padding handling - #47765

Open
kunwar-vikrant wants to merge 2 commits into
huggingface:mainfrom
kunwar-vikrant:fix/hubert-pos-conv-padding
Open

Fix HuBERT positional batch norm padding handling#47765
kunwar-vikrant wants to merge 2 commits into
huggingface:mainfrom
kunwar-vikrant:fix/hubert-pos-conv-padding

Conversation

@kunwar-vikrant

@kunwar-vikrant kunwar-vikrant commented Aug 4, 2026

Copy link
Copy Markdown

CI

What does this PR do?

When conv_pos_batch_norm=True, HuBERT includes padded time steps in BatchNorm statistics. BatchNorm may also turn zero-padded positions into non-zero values, which then affect valid boundary tokens through the positional convolution.

  • Passes the original 2D attention mask to HubertPositionalConvEmbedding.
  • Excludes padded positions from BatchNorm statistics during training.
  • Zeros padded positions before the positional convolution.
  • Supports both standard and stable-layer-norm HuBERT encoders.
  • Makes the changes in modular_hubert.py and regenerates modeling_hubert.py.
  • Adds regression coverage for both training and evaluation modes.

Tests

  • HuBERT test suite: 188 passed, 195 skipped.
  • Modular conversion consistency check passed.
  • Code style checks passed.

Fixes #47739

@github-actions

github-actions Bot commented Aug 4, 2026

Copy link
Copy Markdown
Contributor

[For maintainers] Suggested jobs to run (before merge)

run-slow: hubert

@github-actions

github-actions Bot commented Aug 4, 2026

Copy link
Copy Markdown
Contributor

CI recap

Dashboard: View test results in Grafana
Latest run: 30899056454
Result: success | Grafana metrics are not available yet.

@Abdulrhman-shaheen

Copy link
Copy Markdown

please, read my comment on #47763

@kunwar-vikrant

kunwar-vikrant commented Aug 4, 2026

Copy link
Copy Markdown
Author

@Abdulrhman-shaheen here modeling_hubert.py is generated using modular_hubert.py which is the source of truth.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

HuBERT positional convolution is sensitive to padding when conv_pos_batch_norm=True

2 participants