Skip to content

Cosmos3 Distilled support#14177

Open
yzhautouskay wants to merge 3 commits into
huggingface:mainfrom
yzhautouskay:yzhautouskay/cosmos3_distilled
Open

Cosmos3 Distilled support#14177
yzhautouskay wants to merge 3 commits into
huggingface:mainfrom
yzhautouskay:yzhautouskay/cosmos3_distilled

Conversation

@yzhautouskay

Copy link
Copy Markdown
Contributor

What does this PR do?

Cosmos3 distilled (few-step) support

Adds inference support for the distilled few-step Cosmos3 checkpoints through the
modular pipeline (Cosmos3OmniModularPipeline).

Checkpoints supported

  • nvidia/Cosmos3-Super-Text2Image-4Step (distilled T2I)
  • nvidia/Cosmos3-Super-Image2Video-4Step (distilled I2V)

Both ship a FlowMatchEulerDiscreteScheduler with a fixed sigma schedule
(scheduler.config.fixed_step_sampler_config.t_list) and bake classifier-free
guidance into the weights. They are only supported via the modular pipeline; the
task-based Cosmos3OmniPipeline does not implement the distilled contract.

Before submitting

Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

@github-actions github-actions Bot added size/L PR with diff > 200 LOC documentation Improvements or additions to documentation modular-pipelines and removed size/L PR with diff > 200 LOC labels Jul 13, 2026

@yiyixuxu yiyixuxu left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks!
i left some comments

Comment thread src/diffusers/modular_pipelines/cosmos/before_denoise.py
Comment thread src/diffusers/modular_pipelines/cosmos/modular_pipeline.py Outdated
Comment thread src/diffusers/modular_pipelines/cosmos/modular_pipeline.py Outdated
Comment thread src/diffusers/modular_pipelines/cosmos/modular_pipeline.py Outdated
Comment thread src/diffusers/modular_pipelines/cosmos/denoise.py Outdated
@github-actions

Copy link
Copy Markdown
Contributor

Hi @yzhautouskay, thanks for the PR! It does not appear to link an issue it fixes. If this PR addresses an existing issue, please add a closing keyword (e.g. Fixes #1234) to the PR description so the issue is linked. See the contribution guide for more details. If this PR intentionally does not fix a tracked issue, a maintainer can add the no-issue-needed label to silence this reminder.

@yzhautouskay

Copy link
Copy Markdown
Contributor Author

thanks @yiyixuxu! I've refactored it to have separate Cosmos3DistilledModularPipeline

@yiyixuxu

Copy link
Copy Markdown
Collaborator

@yzhautouskay can you resolve the conflict?

@yzhautouskay yzhautouskay force-pushed the yzhautouskay/cosmos3_distilled branch from 177ee1e to 3ac7ec0 Compare July 15, 2026 09:43
@yzhautouskay yzhautouskay force-pushed the yzhautouskay/cosmos3_distilled branch from 3ac7ec0 to abb37e7 Compare July 15, 2026 09:48

@yiyixuxu yiyixuxu left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks, I left some comments

Comment thread docs/source/en/api/pipelines/cosmos3.md Outdated
Comment thread src/diffusers/modular_pipelines/modular_pipeline.py Outdated
Comment thread src/diffusers/modular_pipelines/cosmos/before_denoise.py Outdated
Comment thread src/diffusers/modular_pipelines/cosmos/before_denoise.py Outdated
Comment thread src/diffusers/modular_pipelines/cosmos/encoders.py Outdated
]


def _fake_distilled_components(t_list=(1.0, 0.75, 0.5, 0.25)):

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done, distilled tests now use ModularPipelineTesterMixin. For now tiny fixture lives under my personal account at yzhautouskay/tiny-cosmos3-distilled-modular-pipe
Could you create hf-internal-testing/tiny-cosmos3-distilled-modular-pipe and copy it over ? I'll then flip the TINY_DISTILLED_REPO constant in the test (and pretrained_model_name_or_path will need to be changed for components in modular_model_index.json)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation modular-pipelines size/L PR with diff > 200 LOC tests utils

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants