Issues when deploying spatial forcing on libero-plus

[ZeyuanHe]

Hi, thank you for open sourcing this great work.
I am trying to deploy pi0.5 version of spatial forcing on libero-plus. I first test the official pi0.5-libero on libero-10 subset of libero-plus, and got a SR of 65%.
Then I trained pi0.5-sf based on official pi0.5-based model on original libero dataset and tested on libero-10 subset of libero-plus, resulting an SR of 23% which is much lower.

I wonder whether I train successfully.

The training is done on 3 NVIDIA GH200 120GB and training config is below:
TrainConfig( name="pi05_libero", model=pi0_config.Pi0Config(pi05=True, action_horizon=10, discrete_state_input=False), data=LeRobotLiberoDataConfig( repo_id="/scratch/u6ac/hezeyuan.u6ac/libero_lerobot", base_config=DataConfig(prompt_from_task=True), extra_delta_transform=False, assets=AssetsConfig( assets_dir="/scratch/u6ac/hezeyuan.u6ac/Spatial-Forcing/openpi-SF/checkpoints/pi0.5_base_pytorch/assets", asset_id="physical-intelligence/libero", ), ), batch_size=48, lr_schedule=_optimizer.CosineDecaySchedule( warmup_steps=10_000, peak_lr=5e-5, decay_steps=1_000_000, decay_lr=5e-5, ), optimizer=_optimizer.AdamW(clip_gradient_norm=1.0), ema_decay=0.999, weight_loader=weight_loaders.CheckpointWeightLoader("gs://openpi-assets/checkpoints/pi05_base/params"), pytorch_weight_path="/scratch/u6ac/hezeyuan.u6ac/Spatial-Forcing/openpi-SF/checkpoints/pi0.5_base_pytorch", num_train_steps=20_000, save_interval=5000, #VGGT vggt_weight_path='/scratch/u6ac/hezeyuan.u6ac/Spatial-Forcing/openpi-SF/checkpoints/vggt', vla_layers_align=12, vggt_layers_align=-1, pooling_func='bilinear', use_vggt_pe=True, use_vlm_norm=True, align_loss_coeff=0.5, )

Below are the losses:

Thank you so much!

[haofuly]

Hi, @ZeyuanHe , the order of magnitude of these curves seems normal, and there seem to be no error in this config.
I am sorry that I have not tried to integrate SF on LIBERO-plus. However, #6 (comment) share his LIBERO-plus config, which achieved a good improvement in success rate. Could you try to use his exact config to train again.
If you cannot reproduce his results, please share the training curves of both openpi-baseline and openpi-SF for further analysis.

[ZeyuanHe]

Thank you for your reply! It seems that #6 trained with libero-plus data and tested on libero-plus, resulting a good SR. However it's different from my setting (train on original libero data and test on libero-plus, which evaluates the generalization ability of the model). I am trying to switch bfloat32 training to float32, and also train pi0.5 base by myself to further analyze the issue.