[Example]: Qwen3-8B online DSpark launcher example (+ DSpark recipe)#1959
[Example]: Qwen3-8B online DSpark launcher example (+ DSpark recipe)#1959h-guo18 wants to merge 1 commit into
Conversation
Adds the DSpark speculative-decoding recipe (modelopt_recipes/general/speculative_decoding/dspark.yaml) and an online-mode launcher example for Qwen3-8B (tools/launcher/examples/Qwen/Qwen3-8B/hf_online_dspark.yaml). DSpark reuses the DFlash online mode/pipeline and adds a lightweight sequential (Markov) head plus an optional confidence head, selected via dflash_architecture_config.projector_type=dspark. The Qwen3-8B example mirrors hf_online_dflash.yaml and only swaps the recipe to dspark.yaml. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Signed-off-by: h-guo18 <67671475+h-guo18@users.noreply.github.com>
📝 WalkthroughWalkthroughAdds a DSpark speculative-decoding recipe and a Qwen3-8B launcher pipeline for online training followed by vLLM smoke testing. ChangesDSpark speculative decoding
Estimated code review effort: 3 (Moderate) | ~20 minutes Sequence Diagram(s)sequenceDiagram
participant Launcher
participant DFlashOnlineTraining
participant TrainingCheckpoint
participant vLLM
Launcher->>DFlashOnlineTraining: start DSpark online training
DFlashOnlineTraining->>TrainingCheckpoint: export training checkpoint
Launcher->>vLLM: start DSpark smoke test
vLLM->>TrainingCheckpoint: load draft checkpoint
vLLM-->>Launcher: report speculative-token acceptance
🚥 Pre-merge checks | ✅ 6✅ Passed checks (6 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Warning
CodeRabbit couldn't request changes on this pull request because it doesn't have sufficient GitHub permissions.
Please grant CodeRabbit Pull requests: Read and write permission and re-run the review.
Actionable comments posted: 3
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@modelopt_recipes/general/speculative_decoding/dspark.yaml`:
- Around line 74-75: Explicitly set dflash_loss_objective to decay in the dspark
recipe so dflash_loss_decay_factor: 7.0 is applied, and update the corresponding
launcher override to use the same objective; otherwise remove or revise the
decay factor and comment.
In `@tools/launcher/examples/Qwen/Qwen3-8B/hf_online_dspark.yaml`:
- Around line 34-35: Define a single shared checkpoint path in global_vars, then
reference that variable for both the training output directory and
DRAFT_CKPT_DIR. Update all corresponding occurrences, including the locations
noted in the review, so both tasks always use the same checkpoint source.
- Around line 23-26: The comment above the launcher overrides is inaccurate
because dflash_block_size, dflash_mask_token_id, and num_hidden_layers are also
overridden. Update the documentation near the YAML override section to
explicitly list these model/generation overrides alongside the data and schedule
fields, and remove the claim that they come from dspark.yaml.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Enterprise
Run ID: bb764e85-6b49-4476-a258-119ba6e60db7
📒 Files selected for processing (2)
modelopt_recipes/general/speculative_decoding/dspark.yamltools/launcher/examples/Qwen/Qwen3-8B/hf_online_dspark.yaml
| # gamma for exponential loss decay (block_size=16 -> 7). | ||
| dflash_loss_decay_factor: 7.0 |
There was a problem hiding this comment.
🎯 Functional Correctness | 🟠 Major | ⚡ Quick win
Select the loss-weighting objective explicitly.
dflash_loss_decay_factor is only used when dflash_loss_objective=decay, but DFlashConfig defaults that field to dpace and this recipe does not override it. Therefore, 7.0 currently has no effect, despite the comment describing exponential decay. Set dflash_loss_objective: decay if that is intended; otherwise remove or update this setting and comment. The launcher repeats the same ineffective override at Line 57.
Proposed fix
dflash:
+ dflash_loss_objective: decay
dflash_loss_decay_factor: 7.0📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| # gamma for exponential loss decay (block_size=16 -> 7). | |
| dflash_loss_decay_factor: 7.0 | |
| # gamma for exponential loss decay (block_size=16 -> 7). | |
| dflash_loss_objective: decay | |
| dflash_loss_decay_factor: 7.0 |
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@modelopt_recipes/general/speculative_decoding/dspark.yaml` around lines 74 -
75, Explicitly set dflash_loss_objective to decay in the dspark recipe so
dflash_loss_decay_factor: 7.0 is applied, and update the corresponding launcher
override to use the same objective; otherwise remove or revise the decay factor
and comment.
| # The draft dims, block_size, mask token and loss weights all come from | ||
| # dspark.yaml (Qwen3-tuned defaults); only the commonly-tuned data/schedule fields | ||
| # are overridden below. dflash_block_size=16 is the semi-AR generation block | ||
| # (training_seq_len must be divisible by it). |
There was a problem hiding this comment.
📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick win
Keep the launcher documentation aligned with its overrides.
This comment says the block size, mask token, and draft dimensions come from dspark.yaml and only data/schedule fields are overridden. However, Lines 55-60 override dflash_block_size, dflash_mask_token_id, and num_hidden_layers. Update the comment to describe the actual override set.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@tools/launcher/examples/Qwen/Qwen3-8B/hf_online_dspark.yaml` around lines 23
- 26, The comment above the launcher overrides is inaccurate because
dflash_block_size, dflash_mask_token_id, and num_hidden_layers are also
overridden. Update the documentation near the YAML override section to
explicitly list these model/generation overrides alongside the data and schedule
fields, and remove the claim that they come from dspark.yaml.
| global_vars: | ||
| hf_model: /hf-local/Qwen/Qwen3-8B |
There was a problem hiding this comment.
🗄️ Data Integrity & Integration | 🟠 Major | ⚡ Quick win
Use one shared checkpoint path for both tasks.
The training output directory and DRAFT_CKPT_DIR duplicate /scratchspace/dspark_bs16. If either value changes independently, the smoke test can read a stale or missing checkpoint. Define the path once under global_vars and interpolate it in both locations.
As per coding guidelines, “Don't repeat yourself; keep a single source of truth.”
Proposed fix
global_vars:
hf_model: /hf-local/Qwen/Qwen3-8B
+ dspark_output_dir: /scratchspace/dspark_bs16
...
- - training.output_dir=/scratchspace/dspark_bs16
+ - training.output_dir=<<global_vars.dspark_output_dir>>
...
- - DRAFT_CKPT_DIR: /scratchspace/dspark_bs16
+ - DRAFT_CKPT_DIR: <<global_vars.dspark_output_dir>>Also applies to: 46-46, 74-75
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@tools/launcher/examples/Qwen/Qwen3-8B/hf_online_dspark.yaml` around lines 34
- 35, Define a single shared checkpoint path in global_vars, then reference that
variable for both the training output directory and DRAFT_CKPT_DIR. Update all
corresponding occurrences, including the locations noted in the review, so both
tasks always use the same checkpoint source.
Source: Coding guidelines
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #1959 +/- ##
=======================================
Coverage 77.80% 77.80%
=======================================
Files 519 519
Lines 57960 57960
=======================================
Hits 45095 45095
Misses 12865 12865
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
1 participant
What does this PR do?
Type of change: New example (launcher example + recipe)
Adds an online-mode DSpark speculative-decoding launcher example for Qwen3-8B, plus the DSpark training recipe it depends on.
Files added (2, both additive — no code changes):
modelopt_recipes/general/speculative_decoding/dspark.yaml— the DSpark training recipe. Selects the DSpark head viadflash_architecture_config.projector_type=dspark(DFlash backbone + lightweight sequential/Markov head + optional confidence head) and sets the three-term loss weights (dflash_ce_loss_alpha/dflash_l1_loss_alpha/dflash_confidence_head_alpha). Online training (data.mode=online) is the default path.tools/launcher/examples/Qwen/Qwen3-8B/hf_online_dspark.yaml— online DSpark training + vLLM smoke-test pipeline for Qwen3-8B, mirroring the existinghf_online_dflash.yaml. DSpark reuses the DFlash online training script (common/specdec/dflash_online_training.sh); only the--configrecipe changes todspark.yaml. The example drops the HF AR-eval step because DSpark keepsestimate_ar=false(training-time eval runs the DFlash backbone only, without the Markov head), and measures acceptance via the vLLM smoke test instead.Notes
dspark.yamlrecipe is the same file introduced in [Examples]: Kimi-K2.6/K2.7-Code Dflash/Dspark #1934. This PR includes it so the Qwen3-8B example is self-contained and independently mergeable. If this lands first, [Examples]: Kimi-K2.6/K2.7-Code Dflash/Dspark #1934 should drop its copy to avoid a duplicate-add conflict.method="dspark").Testing
dspark.yamlpasses thevalidate modelopt recipesschema check.hf_online_dspark.yamlpasses the launcher YAML reference check (check-launcher-yaml).Backward compatible?: ✅ (additive recipe + example files only)
🤖 Generated with Claude Code
Summary by CodeRabbit