feat(lingbot): add split async vae streaming

[youngmagician114514]

Description

Add a LingBot-World v2 7-GPU split streaming configuration with async VAE decode and condition prefetch. The runtime can place condition/T5 encoding, VAE decoding, and DiT ranks on separate GPUs.

Motivation

LingBot-World v2 streaming was bottlenecked by serialized condition encode, DiT, and VAE decode stages. Splitting VAE decode onto a separate GPU and prefetching the next condition chunk allows condition encode and VAE decode to overlap with DiT work, improving steady-state streaming throughput.

Type of Change

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to not work as expected)
• Performance improvement
• Code refactoring
• Documentation update
• Other (please describe):

Changes Made

• Add an async LingBot VAE decode worker with ordered chunk handles, cancellation, cleanup, and profiling metadata.
• Add optional separate async_vae_config so VAE encode and VAE decode can use different GPU/runtime placements.
• Add condition prefetch support to encode the next condition chunk ahead of DiT consumption.
• Add a 7-GPU LingBot-World v2 streaming config: GPU0 condition/T5, GPU1 async VAE decode, GPU2-6 DiT.
• Add unit coverage for async VAE submit/wait, separate VAE decode placement, condition prefetch, cleanup, and v2 split config mapping.

Testing

• Unit tests pass (pytest tests/)
• Manual testing performed
• Benchmarks added/updated (if applicable)

Test commands:

• python -m py_compile telefuser/pipelines/lingbot_world_fast/async_vae.py telefuser/pipelines/lingbot_world_fast/denoising.py telefuser/pipelines/lingbot_world_fast/pipeline.py telefuser/pipelines/lingbot_world_fast/service.py telefuser/pipelines/lingbot_world_fast/session.py examples/lingbot/stream_lingbot_world_v2_7gpu_split_async_vae.py
• python -m ruff check telefuser/pipelines/lingbot_world_fast/async_vae.py telefuser/pipelines/lingbot_world_fast/denoising.py telefuser/pipelines/lingbot_world_fast/pipeline.py telefuser/pipelines/lingbot_world_fast/service.py telefuser/pipelines/lingbot_world_fast/session.py examples/lingbot/stream_lingbot_world_v2_7gpu_split_async_vae.py tests/unit/pipelines/lingbot_world_fast/test_pipeline_call.py tests/unit/pipelines/lingbot_world_fast/test_runtime_baseline.py tests/unit/pipelines/lingbot_world_fast/test_service_action_loop.py tests/unit/pipelines/lingbot_world_v2/test_service.py
• python -m ruff format --check telefuser/pipelines/lingbot_world_fast/async_vae.py telefuser/pipelines/lingbot_world_fast/denoising.py telefuser/pipelines/lingbot_world_fast/pipeline.py telefuser/pipelines/lingbot_world_fast/service.py telefuser/pipelines/lingbot_world_fast/session.py examples/lingbot/stream_lingbot_world_v2_7gpu_split_async_vae.py tests/unit/pipelines/lingbot_world_fast/test_pipeline_call.py tests/unit/pipelines/lingbot_world_fast/test_runtime_baseline.py tests/unit/pipelines/lingbot_world_fast/test_service_action_loop.py tests/unit/pipelines/lingbot_world_v2/test_service.py
• git diff --check

pytest was not run because the current environment does not have pytest installed.

Manual benchmark summary:

• 7-GPU split async VAE + condition prefetch, 20 chunks
• FPS: 17.068
• generation_s: 18.572
• median interval: 0.889 s
• p95 interval: 0.946 s
• condition_prefetch_wait steady mean: 0.006 ms

Checklist

• Code follows the project's coding standards (ruff)
• Pre-commit hooks pass (pre-commit run --all-files)
• All tests pass (pytest tests/)
• New tests added for new functionality
• Documentation updated (README, CLAUDE.md, docstrings)
• Commit messages are clear and descriptive
• PR title follows the convention: [TYPE] Brief description

Related Issues

Fixes #

Additional Notes

This PR does not add or modify CUDA/Triton kernels. The async VAE worker is Python/PyTorch runtime orchestration only.

The benchmark was run on H100 GPUs with the final 7-GPU layout:

• GPU0: condition VAE encode + T5/text encode
• GPU1: async VAE decode
• GPU2-6: DiT ranks 0-4

GPU Architecture Support

• SM80 (Ampere, Ada Lovelace)
• SM90 (Hopper H100)
• SM100+ (Blackwell)

Performance Impact

The 7-GPU split async VAE configuration reached 17.068 FPS in the measured 20-chunk benchmark. Compared with the prior 8-GPU synchronous measurement of 10.38 FPS, this improves steady-state throughput by approximately 64.4%, while using one fewer GPU.

[lzx1413]

The current LingBot v2 realtime path already uses an actor-based three-stage pipeline:

• VAE encode: dedicated actor
• DiT denoise: dedicated multi-GPU actor (FSDP/Ulysses)
• VAE decode: dedicated actor

The stages are connected through bounded edges, so encode, DiT, and decode are not invoked serially from the service loop. The realtime ingress now prefetches subsequent control chunks whenever the actor graph has capacity and the control is already known. It does not speculate about
controls that are still being held by the user.

This lets encode process the next input while the previous chunk is running in DiT, while decode can overlap with later encode/DiT work. This matches the goal of this PR—splitting VAE streaming and allowing condition / DiT / decode to form an asynchronous pipeline.

The main difference is that the current implementation reuses TeleFuser’s generic actor scheduler and bounded queues instead of maintaining a separate async VAE manager inside the service. LiveKit is not involved in this path.

We validated this with a real WebRTC session on 4×H100: later encode_condition_chunk work completed before the previous denoise_and_update_cache finished, and decode continued to interleave with subsequent encode/DiT work. The scheduler also reported a continuous DiT interval with
idle=0.000s.

It would be useful to add a reproducible A/B benchmark:

1. Use a fixed image, seed, and prerecorded control sequence for 20+ chunks.
2. Compare the previous “submit only after the current chunk is consumed” behavior with the current bounded-prefetch behavior.
3. Collect DiT stage_idle_intervals, chunk compute latency, end-to-end first/subsequent-frame latency, and peak memory.
4. Verify output ordering and deterministic equivalence for fixed controls.

The expected outcome is lower DiT idle time and smoother output cadence, while preserving bounded per-session queue capacity.