[docs] update philosophy.md (finally)#13808
Open
yiyixuxu wants to merge 1 commit into
Open
Conversation
Update the framing to match how Diffusers actually works now: models follow the single-file policy (with UNet/older transformers as legacy exceptions, not the model), add a Modular Pipelines section parallel to standard Pipelines, and slim Pipelines/Models bullets to remove stale guidance (per-task pipeline subclassing, generic-class checkpoint adaptation, dead links to deprecated repos). Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
github-actions
Bot
added
documentation
|
The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update. |
sayakpaul
approved these changes
May 26, 2026
| ### Pipelines | ||
|
|
||
| Pipelines are designed to be easy to use (therefore do not follow [*Simple over easy*](#simple-over-easy) 100%), are not feature complete, and should loosely be seen as examples of how to use [models](#models) and [schedulers](#schedulers) for inference. | ||
| Pipelines are intended **only** for inference. Pipelines are designed to be easy to use (therefore do not follow [*Simple over easy*](#simple-over-easy) 100%) — they should be readable, self-explanatory, easy to tweak, and loosely seen as examples of how to use [models](#models) and [schedulers](#schedulers). Pipelines are not feature complete, to build feature-complete user interfaces on top of Diffusers, consider using [Modular Diffusers](../modular_diffusers/overview). |
Member
There was a problem hiding this comment.
Suggested change
| Pipelines are intended **only** for inference. Pipelines are designed to be easy to use (therefore do not follow [*Simple over easy*](#simple-over-easy) 100%) — they should be readable, self-explanatory, easy to tweak, and loosely seen as examples of how to use [models](#models) and [schedulers](#schedulers). Pipelines are not feature complete, to build feature-complete user interfaces on top of Diffusers, consider using [Modular Diffusers](../modular_diffusers/overview). | |
| Pipelines, either standard or modular, are intended **only** for inference. Pipelines are designed to be easy to use (therefore do not follow [*Simple over easy*](#simple-over-easy) 100%) — they should be readable, self-explanatory, easy to tweak, and loosely seen as examples of how to use [models](#models) and [schedulers](#schedulers). Pipelines are not feature complete, to build feature-complete user interfaces on top of Diffusers, consider using [Modular Diffusers](../modular_diffusers/overview). |
stevhliu
approved these changes
May 28, 2026
| ### Pipelines | ||
|
|
||
| Pipelines are designed to be easy to use (therefore do not follow [*Simple over easy*](#simple-over-easy) 100%), are not feature complete, and should loosely be seen as examples of how to use [models](#models) and [schedulers](#schedulers) for inference. | ||
| Pipelines are intended **only** for inference. Pipelines are designed to be easy to use (therefore do not follow [*Simple over easy*](#simple-over-easy) 100%) — they should be readable, self-explanatory, easy to tweak, and loosely seen as examples of how to use [models](#models) and [schedulers](#schedulers). Pipelines are not feature complete, to build feature-complete user interfaces on top of Diffusers, consider using [Modular Diffusers](../modular_diffusers/overview). |
Member
There was a problem hiding this comment.
Suggested change
| Pipelines are intended **only** for inference. Pipelines are designed to be easy to use (therefore do not follow [*Simple over easy*](#simple-over-easy) 100%) — they should be readable, self-explanatory, easy to tweak, and loosely seen as examples of how to use [models](#models) and [schedulers](#schedulers). Pipelines are not feature complete, to build feature-complete user interfaces on top of Diffusers, consider using [Modular Diffusers](../modular_diffusers/overview). | |
| Pipelines, standard or modular, are intended only for inference. They're designed to be easy to use (so they don't follow [Simple over easy](#simple-over-easy) 100%): readable, self-explanatory, easy to tweak, and best seen as examples of how to use [models](#models) and [schedulers](#schedulers). They aren't feature complete. To build feature-complete user interfaces on top of Diffusers, use [Modular Diffusers](../modular_diffusers/overview). |
|
|
||
| ### Modular Pipelines | ||
|
|
||
| Modular Pipelines is the composable alternative to standard pipelines: workflows are built from reusable **pipeline blocks** that can be mixed, matched, swapped, and shared. Where standard pipelines are loose reference examples of how to use models and schedulers, Modular Pipelines is the recommended path for building feature-complete user interfaces on top of Diffusers, and for the community to build and share new pipelines in a decentralized way. |
Member
There was a problem hiding this comment.
Suggested change
| Modular Pipelines is the composable alternative to standard pipelines: workflows are built from reusable **pipeline blocks** that can be mixed, matched, swapped, and shared. Where standard pipelines are loose reference examples of how to use models and schedulers, Modular Pipelines is the recommended path for building feature-complete user interfaces on top of Diffusers, and for the community to build and share new pipelines in a decentralized way. | |
| Modular Diffusers is the composable alternative to standard pipelines. You build a workflow from reusable *pipeline blocks* that you can mix, match, swap, and share. Standard pipelines are loose reference examples of how to use models and schedulers. Modular Diffusers is the recommended path for building feature-complete user interfaces on top of Diffusers, and for the community to build and share new pipelines in a decentralized way. |
|
|
||
| Now, let's look a bit into the nitty-gritty details of the design philosophy. Diffusers essentially consists of three major classes: [pipelines](https://github.com/huggingface/diffusers/tree/main/src/diffusers/pipelines), [models](https://github.com/huggingface/diffusers/tree/main/src/diffusers/models), and [schedulers](https://github.com/huggingface/diffusers/tree/main/src/diffusers/schedulers). | ||
| Let's walk through more in-detail design decisions for each class. | ||
| Now, let's look a bit into the nitty-gritty details of the design philosophy. Diffusers consists of [models](https://github.com/huggingface/diffusers/tree/main/src/diffusers/models), [schedulers](https://github.com/huggingface/diffusers/tree/main/src/diffusers/schedulers), and two ways to compose them into a runnable workflow: standard [pipelines](https://github.com/huggingface/diffusers/tree/main/src/diffusers/pipelines) (monolithic, one task per pipeline class) and [Modular Diffusers](../modular_diffusers/overview) (composable, block-based). Let's walk through more in-detail design decisions for each. |
Member
There was a problem hiding this comment.
Suggested change
| Now, let's look a bit into the nitty-gritty details of the design philosophy. Diffusers consists of [models](https://github.com/huggingface/diffusers/tree/main/src/diffusers/models), [schedulers](https://github.com/huggingface/diffusers/tree/main/src/diffusers/schedulers), and two ways to compose them into a runnable workflow: standard [pipelines](https://github.com/huggingface/diffusers/tree/main/src/diffusers/pipelines) (monolithic, one task per pipeline class) and [Modular Diffusers](../modular_diffusers/overview) (composable, block-based). Let's walk through more in-detail design decisions for each. | |
| Now for the nitty-gritty details of the design philosophy. Diffusers gives you two ways to compose [models](https://github.com/huggingface/diffusers/tree/main/src/diffusers/models) and [schedulers](https://github.com/huggingface/diffusers/tree/main/src/diffusers/schedulers) into a runnable workflow: standard [pipelines](https://github.com/huggingface/diffusers/tree/main/src/diffusers/pipelines), which are monolithic with one task per pipeline class, and Modular Diffusers, which is composable and block-based. The sections below cover pipelines, modular pipelines, models, and schedulers in turn. |
| Modular Pipelines is the composable alternative to standard pipelines: workflows are built from reusable **pipeline blocks** that can be mixed, matched, swapped, and shared. Where standard pipelines are loose reference examples of how to use models and schedulers, Modular Pipelines is the recommended path for building feature-complete user interfaces on top of Diffusers, and for the community to build and share new pipelines in a decentralized way. | ||
|
|
||
| The following design principles are followed: | ||
| - Modular pipelines also follow the single-file policy. Each modular pipeline lives in its own folder under [`src/diffusers/modular_pipelines/`](https://github.com/huggingface/diffusers/tree/main/src/diffusers/modular_pipelines). A modular pipeline folder contains multiple files corresponding to each stage of the workflow — `encoders.py`, `before_denoise.py`, `denoise.py`, `decoders.py`, `modular_blocks_<model>.py` for assembly, and `modular_pipeline.py` for the per-model `ModularPipeline` subclass. Modular pipelines don't cross-import each other. |
Member
There was a problem hiding this comment.
Suggested change
| - Modular pipelines also follow the single-file policy. Each modular pipeline lives in its own folder under [`src/diffusers/modular_pipelines/`](https://github.com/huggingface/diffusers/tree/main/src/diffusers/modular_pipelines). A modular pipeline folder contains multiple files corresponding to each stage of the workflow — `encoders.py`, `before_denoise.py`, `denoise.py`, `decoders.py`, `modular_blocks_<model>.py` for assembly, and `modular_pipeline.py` for the per-model `ModularPipeline` subclass. Modular pipelines don't cross-import each other. | |
| - Modular pipelines follow the single-file policy. Each one lives in its own folder under [src/diffusers/modular_pipelines/](https://github.com/huggingface/diffusers/tree/main/src/diffusers/modular_pipelines), and the folder splits the workflow across one file per stage: `encoders.py`, `before_denoise.py`, `denoise.py`, and `decoders.py`. Two more files complete the pipeline. `modular_blocks_<model>.py` assembles the stages, and `modular_pipeline.py` defines the per-model [`ModularPipeline`] subclass. Modular pipelines don't cross-import each other. |
|
|
||
| The following design principles are followed: | ||
| - Modular pipelines also follow the single-file policy. Each modular pipeline lives in its own folder under [`src/diffusers/modular_pipelines/`](https://github.com/huggingface/diffusers/tree/main/src/diffusers/modular_pipelines). A modular pipeline folder contains multiple files corresponding to each stage of the workflow — `encoders.py`, `before_denoise.py`, `denoise.py`, `decoders.py`, `modular_blocks_<model>.py` for assembly, and `modular_pipeline.py` for the per-model `ModularPipeline` subclass. Modular pipelines don't cross-import each other. | ||
| - Each modular pipeline is defined as a set of `ModularPipelineBlocks` — leaf blocks in the stage files (`encoders.py`, `before_denoise.py`, `denoise.py`, `decoders.py`), then assembled into the full workflow in `modular_blocks_<model>.py` using container classes like `SequentialPipelineBlocks` and `AutoPipelineBlocks`. Unlike `DiffusionPipeline`, which both defines the computation logic and runs it, modular splits these into two concepts: blocks are pure definitions (declare inputs, outputs, and component dependencies — they do not hold weights and cannot be executed), and a `ModularPipeline` is the runnable counterpart, created via `.init_pipeline(repo_id)`. Keeping blocks stateless and weight-free is what makes them freely composable, swappable, and shareable across workflows. |
Member
There was a problem hiding this comment.
Suggested change
| - Each modular pipeline is defined as a set of `ModularPipelineBlocks` — leaf blocks in the stage files (`encoders.py`, `before_denoise.py`, `denoise.py`, `decoders.py`), then assembled into the full workflow in `modular_blocks_<model>.py` using container classes like `SequentialPipelineBlocks` and `AutoPipelineBlocks`. Unlike `DiffusionPipeline`, which both defines the computation logic and runs it, modular splits these into two concepts: blocks are pure definitions (declare inputs, outputs, and component dependencies — they do not hold weights and cannot be executed), and a `ModularPipeline` is the runnable counterpart, created via `.init_pipeline(repo_id)`. Keeping blocks stateless and weight-free is what makes them freely composable, swappable, and shareable across workflows. | |
| - Each modular pipeline is defined as a set of [`ModularPipelineBlocks`]. Leaf blocks live in the stage files (`encoders.py`, `before_denoise.py`, `denoise.py`, `decoders.py`), and `modular_blocks_<model>.py` assembles them into the full workflow with container classes like [`SequentialPipelineBlocks`] and [`AutoPipelineBlocks`]. This splits apart two concepts that [`DiffusionPipeline combines`]. A block is a pure definition. It declares inputs, outputs, and component dependencies, but holds no weights and can't run. A [`ModularPipeline`], created with `.init_pipeline(repo_id)`, is the runnable counterpart. Keeping blocks stateless and weight-free is what makes them freely composable, swappable, and shareable across workflows. |
| The following design principles are followed: | ||
| - Modular pipelines also follow the single-file policy. Each modular pipeline lives in its own folder under [`src/diffusers/modular_pipelines/`](https://github.com/huggingface/diffusers/tree/main/src/diffusers/modular_pipelines). A modular pipeline folder contains multiple files corresponding to each stage of the workflow — `encoders.py`, `before_denoise.py`, `denoise.py`, `decoders.py`, `modular_blocks_<model>.py` for assembly, and `modular_pipeline.py` for the per-model `ModularPipeline` subclass. Modular pipelines don't cross-import each other. | ||
| - Each modular pipeline is defined as a set of `ModularPipelineBlocks` — leaf blocks in the stage files (`encoders.py`, `before_denoise.py`, `denoise.py`, `decoders.py`), then assembled into the full workflow in `modular_blocks_<model>.py` using container classes like `SequentialPipelineBlocks` and `AutoPipelineBlocks`. Unlike `DiffusionPipeline`, which both defines the computation logic and runs it, modular splits these into two concepts: blocks are pure definitions (declare inputs, outputs, and component dependencies — they do not hold weights and cannot be executed), and a `ModularPipeline` is the runnable counterpart, created via `.init_pipeline(repo_id)`. Keeping blocks stateless and weight-free is what makes them freely composable, swappable, and shareable across workflows. | ||
| - To support a new workflow/task, write the task-specific block(s), compose them with existing ones, and declare the new workflow in `_workflow_map` on the top-level block assembly. Unlike `DiffusionPipeline`, each `ModularPipeline` can support many workflows (text2img, img2img, inpaint, etc.). |
Member
There was a problem hiding this comment.
Suggested change
| - To support a new workflow/task, write the task-specific block(s), compose them with existing ones, and declare the new workflow in `_workflow_map` on the top-level block assembly. Unlike `DiffusionPipeline`, each `ModularPipeline` can support many workflows (text2img, img2img, inpaint, etc.). | |
| - To support a new task, write the task-specific blocks, compose them with existing ones, and register the workflow in `_workflow_map` on the top-level block assembly. A single `ModularPipeline` can support many workflows, such as text-to-image, image-to-image, and inpainting, whereas a `DiffusionPipeline` runs only one. |
| ### Models | ||
|
|
||
| Models are designed as configurable toolboxes that are natural extensions of [PyTorch's Module class](https://pytorch.org/docs/stable/generated/torch.nn.Module.html). They only partly follow the **single-file policy**. | ||
| Models are designed as configurable toolboxes that are natural extensions of [PyTorch's Module class](https://pytorch.org/docs/stable/generated/torch.nn.Module.html). They should follow the **single-file policy**. Some older models predate this convention and are kept as-is as legacy exceptions — not patterns to follow for new models. E.g. the original [`UNet2DConditionModel`] class was used for different UNet variations. |
Member
There was a problem hiding this comment.
Suggested change
| Models are designed as configurable toolboxes that are natural extensions of [PyTorch's Module class](https://pytorch.org/docs/stable/generated/torch.nn.Module.html). They should follow the **single-file policy**. Some older models predate this convention and are kept as-is as legacy exceptions — not patterns to follow for new models. E.g. the original [`UNet2DConditionModel`] class was used for different UNet variations. | |
| Models are designed as configurable toolboxes that are natural extensions of [PyTorch's Module class](https://pytorch.org/docs/stable/generated/torch.nn.Module.html). They should follow the *single-file policy*. Some older models predate this convention and are kept as-is. Treat them as legacy exceptions, not patterns to follow for new models. For example, the original [`UNet2DConditionModel`] class was used for several UNet variations. |
| - Models correspond to **a type of model architecture**. *E.g.* the [`UNet2DConditionModel`] class is used for all UNet variations that expect 2D image inputs and are conditioned on some context. | ||
| - All models can be found in [`src/diffusers/models`](https://github.com/huggingface/diffusers/tree/main/src/diffusers/models) and every model architecture shall be defined in its file, e.g. [`unets/unet_2d_condition.py`](https://github.com/huggingface/diffusers/blob/main/src/diffusers/models/unets/unet_2d_condition.py), [`transformers/transformer_2d.py`](https://github.com/huggingface/diffusers/blob/main/src/diffusers/models/transformers/transformer_2d.py), etc... | ||
| - Models **do not** follow the single-file policy and should make use of smaller model building blocks, such as [`attention.py`](https://github.com/huggingface/diffusers/blob/main/src/diffusers/models/attention.py), [`resnet.py`](https://github.com/huggingface/diffusers/blob/main/src/diffusers/models/resnet.py), [`embeddings.py`](https://github.com/huggingface/diffusers/blob/main/src/diffusers/models/embeddings.py), etc... **Note**: This is in stark contrast to Transformers' modeling files and shows that models do not really follow the single-file policy. | ||
| - Each model architecture type lives in its own folder under [`src/diffusers/models`](https://github.com/huggingface/diffusers/tree/main/src/diffusers/models) (e.g. [`transformers/`](https://github.com/huggingface/diffusers/tree/main/src/diffusers/models/transformers), [`autoencoders/`](https://github.com/huggingface/diffusers/tree/main/src/diffusers/models/autoencoders), [`unets/`](https://github.com/huggingface/diffusers/tree/main/src/diffusers/models/unets)), and each model family has its own file within that folder (e.g. [`transformer_flux.py`](https://github.com/huggingface/diffusers/blob/main/src/diffusers/models/transformers/transformer_flux.py), [`transformer_wan.py`](https://github.com/huggingface/diffusers/blob/main/src/diffusers/models/transformers/transformer_wan.py)). |
Member
There was a problem hiding this comment.
Suggested change
| - Each model architecture type lives in its own folder under [`src/diffusers/models`](https://github.com/huggingface/diffusers/tree/main/src/diffusers/models) (e.g. [`transformers/`](https://github.com/huggingface/diffusers/tree/main/src/diffusers/models/transformers), [`autoencoders/`](https://github.com/huggingface/diffusers/tree/main/src/diffusers/models/autoencoders), [`unets/`](https://github.com/huggingface/diffusers/tree/main/src/diffusers/models/unets)), and each model family has its own file within that folder (e.g. [`transformer_flux.py`](https://github.com/huggingface/diffusers/blob/main/src/diffusers/models/transformers/transformer_flux.py), [`transformer_wan.py`](https://github.com/huggingface/diffusers/blob/main/src/diffusers/models/transformers/transformer_wan.py)). | |
| - Each model architecture type lives in its own folder under [src/diffusers/models](https://github.com/huggingface/diffusers/tree/main/src/diffusers/models), for example [transformers/](https://github.com/huggingface/diffusers/tree/main/src/diffusers/models/transformers), [autoencoders/](https://github.com/huggingface/diffusers/tree/main/src/diffusers/models/autoencoders), or [unets/](https://github.com/huggingface/diffusers/tree/main/src/diffusers/models/unets). Each model family has its own file within that folder, such as [transformer_flux.py](https://github.com/huggingface/diffusers/blob/main/src/diffusers/models/transformers/transformer_flux.py) and [transformer_wan.py](https://github.com/huggingface/diffusers/blob/main/src/diffusers/models/transformers/transformer_wan.py). |
| - All models can be found in [`src/diffusers/models`](https://github.com/huggingface/diffusers/tree/main/src/diffusers/models) and every model architecture shall be defined in its file, e.g. [`unets/unet_2d_condition.py`](https://github.com/huggingface/diffusers/blob/main/src/diffusers/models/unets/unet_2d_condition.py), [`transformers/transformer_2d.py`](https://github.com/huggingface/diffusers/blob/main/src/diffusers/models/transformers/transformer_2d.py), etc... | ||
| - Models **do not** follow the single-file policy and should make use of smaller model building blocks, such as [`attention.py`](https://github.com/huggingface/diffusers/blob/main/src/diffusers/models/attention.py), [`resnet.py`](https://github.com/huggingface/diffusers/blob/main/src/diffusers/models/resnet.py), [`embeddings.py`](https://github.com/huggingface/diffusers/blob/main/src/diffusers/models/embeddings.py), etc... **Note**: This is in stark contrast to Transformers' modeling files and shows that models do not really follow the single-file policy. | ||
| - Each model architecture type lives in its own folder under [`src/diffusers/models`](https://github.com/huggingface/diffusers/tree/main/src/diffusers/models) (e.g. [`transformers/`](https://github.com/huggingface/diffusers/tree/main/src/diffusers/models/transformers), [`autoencoders/`](https://github.com/huggingface/diffusers/tree/main/src/diffusers/models/autoencoders), [`unets/`](https://github.com/huggingface/diffusers/tree/main/src/diffusers/models/unets)), and each model family has its own file within that folder (e.g. [`transformer_flux.py`](https://github.com/huggingface/diffusers/blob/main/src/diffusers/models/transformers/transformer_flux.py), [`transformer_wan.py`](https://github.com/huggingface/diffusers/blob/main/src/diffusers/models/transformers/transformer_wan.py)). | ||
| - Models follow the **single-file policy** — each model file should be self-contained, except for a small number of standard modules that every model uses identically (e.g. timestep embeddings, normalization layers), which can be imported from [`embeddings.py`](https://github.com/huggingface/diffusers/blob/main/src/diffusers/models/embeddings.py) and [`normalization.py`](https://github.com/huggingface/diffusers/blob/main/src/diffusers/models/normalization.py). |
Member
There was a problem hiding this comment.
Suggested change
| - Models follow the **single-file policy** — each model file should be self-contained, except for a small number of standard modules that every model uses identically (e.g. timestep embeddings, normalization layers), which can be imported from [`embeddings.py`](https://github.com/huggingface/diffusers/blob/main/src/diffusers/models/embeddings.py) and [`normalization.py`](https://github.com/huggingface/diffusers/blob/main/src/diffusers/models/normalization.py). | |
| - Models follow the single-file policy. Each model file should be self-contained, except for a small number of standard modules that every model uses identically, such as timestep embeddings and normalization layers. Import those from [embeddings.py](https://github.com/huggingface/diffusers/blob/main/src/diffusers/models/embeddings.py) and [normalization.py](https://github.com/huggingface/diffusers/blob/main/src/diffusers/models/normalization.py). |
| - Pipelines should be named after the task they are intended to solve. | ||
| - In almost all cases, novel diffusion pipelines shall be implemented in a new pipeline folder/file. | ||
|
|
||
| ### Modular Pipelines |
Member
There was a problem hiding this comment.
Suggested change
| ### Modular Pipelines | |
| ### Modular Diffusers |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
in this PR, I