feat(anima): Anima Controlnets + Inpainting Adapter#9284
Open
kappacommit wants to merge 8 commits into
Open
Conversation
Port of kohya-ss ControlNet-LLLite v2 for the Anima DiT (from the Apache-2.0 ComfyUI-Anima-LLLite reference), restructured to build from a safetensors state dict + metadata without a transformer instance so it can live in the model cache. Binds to transformer Linears by path regex with guaranteed restore, supports the 4-channel inpainting conditioning (masked RGB + binary mask), and includes the patch-padding-aware cond sizing helpers. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
New ControlNet_Checkpoint_Anima_Config probed on the lllite_conditioning1./ lllite_dit_blocks_ key namespace (no overlap with SDXL LLLite or Z-Image control), a loader that builds the adapter from metadata at the Anima inference dtype, and an Anima LLLite Inpainting starter model entry for kohya-ss/Anima-LLLite. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
New anima_lllite invocation (image + optional mask + model + weight + step range) and an optional control_lllite input on anima_denoise (v1.7.0). The conditioning image is built once per generation from the actual latent dims, the adapter binds after LoRA patching, the multiplier is step-range gated in both the Euler and scheduler-driver paths, and restore/clear always run on exit since the adapter instance is shared via the model cache. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Adds an Inpaint Adapter model picker + weight control to the Anima advanced settings, and wires anima_lllite into the canvas inpaint and outpaint graphs (all scaled/unscaled branches). The canvas denoise-limit mask is white = keep, while LLLite expects white = inpaint, so the mask is inverted via img_lerp before it reaches the adapter. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
ControlNet_Checkpoint_Anima_Config records cond_in_channels at probe time (metadata with conv1-shape fallback) so the UI can route inpaint (4ch) and general control (3ch) variants to different surfaces; null means installed before this field existed. Adds starter entries for the remaining LLLite variants: Sketch (any-test-like-v2) plus the Preview3-era depth, scribble, lineart and pose adapters. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
control_lllite now takes a single field or a list (v1.8.0). Adapters are sorted by model key for deterministic composition, applied in order after LoRA patching, gated per-step by their own weight and step range, and restored in reverse order with per-adapter exception isolation since the bind chain is only safe to unwind LIFO. Duplicate adapter models are rejected because the model cache shares one instance per key. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Adds the anima_lllite control-adapter type to control layers, lifts the hard unsupported-base block for Anima (t2i/control-lora still warn), and routes models by conditioning channels: 3ch adapters appear in control layers, 4ch and legacy installs in the Inpaint Adapter picker. Control layers and the inpaint adapter feed a shared collect node into the denoise list input across all generation modes, and duplicate adapter models block Invoke with a per-layer warning. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
kappacommit
requested review from
JPPhoto,
Pfannkuchensack,
blessedcoolant,
dunkeroni and
lstein
as code owners
github-actions
Bot
added
python
kappacommit
changed the title
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds support for kohya-ss's ControlNet-LLLite adapters for Anima — small add-on models (8–66 MB) that bring two user-facing improvements:
All variants are available as one-click starter model installs, work with existing Anima setups, and add no measurable generation time. The inpaint adapter applies automatically when inpainting/outpainting on Canvas; control adapters are picked per control layer.
Under the hood: a new backend module (ported with attribution from kohya's Apache-2.0 reference implementation), model-manager registration that routes inpaint vs. control variants to the right pickers, an
anima_lllitenode, and canvas graph wiring for both surfaces.Related Issues / Discussions
N/A — integrates https://huggingface.co/kohya-ss/Anima-LLLite (reference implementations: kohya-ss/ComfyUI-Anima-LLLite, kohya-ss/sd-scripts).
QA Instructions
Requires the Anima starter models (Base 1.0, Qwen3 0.6B encoder, QwenImage VAE).
Merge Plan
No DB schema changes. Redux changes are additive with defaults (no slice migration needed). The 7 commits are two logical stacks (inpainting adapter, then control layers) and can be reviewed in order.
Checklist
What's Newcopy (if doing a release after this PR)🤖 Generated with Claude Code