Does LeRobot have a program schedule that supports multi-GPUs parallel training?

[YushunXiang]

I have seen a lot of issues and PRs link #558 #317 #956 #876 #778 about data parallel training with multi-GPU. This shows that multi-GPU training is what this community as well as I need. Does LeRobot have a schedule that supports multi-GPU training? Or how can I improve on these two PRs #317 #778 to merge into the main branch?

[chenkang455]

Same qs here.

[AdilZouitine]

Hello, the answer is yes 😄
You can see as dependency accelerate for smolvla.
cc @danaaubakirova

[danaaubakirova]

We used accelerate as well as the multi-dataset setup for pre-training only, and for quick fine-tuning on multitask setup. For the worldwide hackathon, we decided that the cleanest and most stable approach would be to train/fine-tune on a single GPU, as we don't expect big datasets or multitask setups used for pre-training phase.

Once the major PRs regarding refactoring, other features complete, enabling multi-GPU training and multi-dataset support is definitely the next step.

We'll keep you updated, thanks for the ping!

[YushunXiang]

Thanks!