Feature/support vlm #2

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Open

Varian-cym wants to merge 38 commits into justbin-coder:main from Varian-cym:feature/support-vlm

Varian-cym commented Nov 7, 2025

Description

Motivation and Context

How has this been tested?

Additional information (optional, e.g., figures and logs):

Types of changes

Bug fix (non-breaking change which fixes an issue)
New feature (non-breaking change which adds functionality)
Breaking change (fix or feature that would cause existing functionality to not work as expected)

Checklist:

My code follows the code style of this project.
My change requires a change to the documentation.
I have updated the documentation accordingly.
I have added tests to cover my changes.
All new and existing tests passed.

anHappyDog and others added 30 commits

October 12, 2025 18:07


          Merge pull request justbin-coder#1 from anHappyDog/feature/weight_con…

461e4e6

…vertor

feat(weight): refactor and add qwen2.5-vl mg2hf convertor


          feat(rollout_mm): add multimodal input/output for rollout backend (ju…

5db6fab

…stbin-coder#2)

* Merge pull request justbin-coder#1 from anHappyDog/feature/weight_convertor

feat(weight): refactor and add qwen2.5-vl mg2hf convertor

* feat(mm_input): add basic vision-language dataset processor and yaml config

Signed-off-by: Bo Dai <daibo@infini-ai.com>

* feat(mm_input): add vLLM multimodal support

Signed-off-by: Bo Dai <daibo@infini-ai.com>

---------

Signed-off-by: Bo Dai <daibo@infini-ai.com>


          feat(vlm): support VLM sglang rollout and fsdp training (RLinf#6)

90a1308

Signed-off-by: guozhen1997 <2997871698@qq.com>
Signed-off-by: Bo Dai <daibo@infini-ai.com>
Co-authored-by: Bo Dai <daibo@infini-ai.com>


          feat(dataset): refactor and add lazy loader process

7c382c8

Signed-off-by: Bo Dai <daibo@infini-ai.com>


          fix(vllm): fix wrong image_data param when running vlm in vllm

fa0fc75

Signed-off-by: Bo Dai <daibo@infini-ai.com>


          feat: add vqa reward function, unify math and vqa reward

6a7d4bc

Signed-off-by: guozhen1997 <2997871698@qq.com>


          feat: add reward worker

7100e6b

Signed-off-by: guozhen1997 <2997871698@qq.com>


          fix: fix vqa reward bugs and ruff format

f7c2fba

Signed-off-by: guozhen1997 <2997871698@qq.com>


          feat: rename and reorganize example config

0c38831

Signed-off-by: guozhen1997 <2997871698@qq.com>


          fix: fix ruff, fix merge bugs

e6ebd60

Signed-off-by: guozhen1997 <2997871698@qq.com>


          fix: fix multi modal inputs

dc446fc

Signed-off-by: guozhen1997 <2997871698@qq.com>


          fix(math): fix some bugs when running math model

a04f2d0

Signed-off-by: Bo Dai <daibo@infini-ai.com>


          fix(math): fix some merge_batch when item is not tensor,add support f…

a7df8fc

…or special prefix

Signed-off-by: Bo Dai <daibo@infini-ai.com>


          chore: add corresponding changes to yaml because of RewardModel and o…

14cbdf0

…ther configurations

Signed-off-by: Bo Dai <daibo@infini-ai.com>


          fix(megatron): apply corresponding changes due to fsdp

Signed-off-by: Bo Dai <daibo@infini-ai.com>


          fix(reward): change math_verify_call's result from {0,1} to {-1,1}

4bf2d81

Signed-off-by: Bo Dai <daibo@infini-ai.com>


          feat(ci): change corresponding ci config for refactored code

fc77e2a

Signed-off-by: Bo Dai <daibo@infini-ai.com>


          chore: refactor dataset parts

9d40cb4

Signed-off-by: Bo Dai <daibo@infini-ai.com>


          fix(mm_data): unify vllm/sglang's mm_data passing

ecb1ed0

Signed-off-by: Bo Dai <daibo@infini-ai.com>


          fix(rollout): fix some problems in sglang/vllm, now both are ok

582a438

Signed-off-by: Bo Dai <daibo@infini-ai.com>


          fix(ci): add ci for vqa

fa5b861

Signed-off-by: Bo Dai <daibo@infini-ai.com>


          fix(ci): fix some bugs in ci

74535e6

Signed-off-by: Bo Dai <daibo@infini-ai.com>


          fix(fsdp): add forgotten backward and optimizer step

62df313

Signed-off-by: Bo Dai <daibo@infini-ai.com>


          fix(collocated): fix inference/rollout do jobs parallelly which cause…

1ec7e54

…s oom in collocated mode

Signed-off-by: Bo Dai <daibo@infini-ai.com>


          fix(sync_weight): fix oom bugs

e57d10d

Signed-off-by: Bo Dai <daibo@infini-ai.com>


          fix(vlm): in torch260's image, transformers version is 4.51.1 and it'…

d0edcd0

…s ok, use 4.51.1 rather than 4.56.1

Signed-off-by: Bo Dai <daibo@infini-ai.com>


          fix(fsdp): use bf16 instead of fp16 for training

19f2a27

Signed-off-by: Bo Dai <daibo@infini-ai.com>


          feat(ci): add fsdp ci

d67365c

Signed-off-by: Bo Dai <daibo@infini-ai.com>


          feat(fsdp): fix ci, add fsdp optimizations like overlap and gradient …

01f95ff

…accumulation

Signed-off-by: Bo Dai <daibo@infini-ai.com>


          fix(ci): add fsdp's run_inference, fix ci

a8023c8

Signed-off-by: Bo Dai <daibo@infini-ai.com>

anHappyDog and others added 8 commits

October 12, 2025 18:07


          fix(ci): fix some errors

803c4c6

Signed-off-by: Bo Dai <daibo@infini-ai.com>


          feat(ci): fix ci

c1a74b0

Signed-off-by: Bo Dai <daibo@infini-ai.com>


          fix(reward): remove redundant reward definitions

2d51313

Signed-off-by: Bo Dai <daibo@infini-ai.com>


          fix(lock): set fsdp's recompute_logprobs True for lock competition sa…

fbc9be7

…fety

Signed-off-by: Bo Dai <daibo@infini-ai.com>


          chore: remove useless code, add correct dp_group param for mg

Signed-off-by: Bo Dai <daibo@infini-ai.com>


          fix(reward): move reward worker's timer to where reward computation r…

9dae32e

…eally happens

Signed-off-by: Bo Dai <daibo@infini-ai.com>


          ADD:support npu

5935efb

73fb07e

justbin-coder force-pushed the main branch from ffba7e6 to ee9a5b2 Compare

November 29, 2025 07:43

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet