Skip to content

Sparse or zero-valid windows still participate in training #78

@jeffreyHoelzel

Description

@jeffreyHoelzel

The dataset yields all windows, including windows with no valid labeled residues. The trainer handles zero-valid masks by producing zero loss, but the optimizer still performs a step.

Evidence:

  • src/pepseqpred/core/data/proteindataset.py
    • ProteinDataset.__iter__ yields every window from _iter_windows.
    • The final mask may be all zero when a window contains only uncertain or padded residues.
  • src/pepseqpred/core/train/trainer.py
    • _batch_step creates zero loss when mask.sum() == 0.
    • Training still calls zero_grad, loss.backward(), and optimizer.step().

Why this can hurt:

  • Adam step counters still advance on no-information batches.
  • Learning-rate dynamics and optimizer moments can be affected without a meaningful gradient.
  • If some pathogens or proteins have many uncertain/unlabeled regions, they can consume training steps without learning signal.

Planning direction:

  • Add logging for zero-valid windows and zero-valid batches by split, fold, rank, and source.
  • Skip optimizer steps when a batch has zero valid residues.
  • Optionally filter zero-valid windows in ProteinDataset for training.

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions