example(eagle3): Qwen3-0.6B EAGLE3 online/offline launcher examples#1954
example(eagle3): Qwen3-0.6B EAGLE3 online/offline launcher examples#1954ChenhanYu wants to merge 1 commit into
Conversation
Adds examples/Qwen/Qwen3-0.6B/hf_online_eagle3.yaml + hf_offline_eagle3.yaml, adapted from the Qwen3-8B EAGLE3 examples for the small, ungated Qwen3-0.6B model on a single GPU (make_dataset/query -> dump -> train_eagle -> common/specdec_bench VLLM benchmark). Uses the co-versioned examples/specdec_bench and a VLLM backend. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Signed-off-by: Chenhan Yu <chenhany@nvidia.com>
|
Auto-sync is disabled for draft pull requests in this repository. Workflows must be run manually. Contributors can view more details about this message here. |
|
Important Review skippedDraft detected. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: CHILL Plan: Enterprise Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
✨ Finishing Touches🧪 Generate unit tests (beta)
Comment |
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #1954 +/- ##
=======================================
Coverage 77.76% 77.76%
=======================================
Files 519 519
Lines 57955 57955
=======================================
Hits 45067 45067
Misses 12888 12888
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
What does this PR do?
Type of change: new example
Adds EAGLE3 speculative-decoding launcher examples for Qwen3-0.6B (small, ungated):
examples/Qwen/Qwen3-0.6B/hf_online_eagle3.yaml+hf_offline_eagle3.yaml, adapted from the existingQwen/Qwen3-8B/hf_{online,offline}_eagle3.yamlfor the smaller model on a single GPU (make_dataset/query → dump hidden states →train_eagle→common/specdec_benchVLLM benchmark, viaexamples/specdec_bench).Usage
Testing
tools/launcher/tests/test_examples_resolve.py(parses + validates the new examples; all referencedcommon/*scripts exist).Before your PR is "Ready for review"
Additional Information
Motivation: nmm-sandbox uses these to replace internal Llama-3.2-1B EAGLE3 specdec tests that hit a TRT-LLM fused-weight loader assertion; the ungated Qwen3-0.6B + co-versioned
examples/specdec_bench(VLLM) is a smaller, maintained path.