ALoHa Training Tips for Engineers (Based on My Experience)

[aoml3245]

🎯 Task

Lower a cube off a Post-it and then place another cube onto it.

---

🤖 Robot & Data

• Robot: OpenManipulator Sara (6-DOF)
• Camera/Robot Pose: Not fixed → I modified the diffusion code so data transforms run continuously.

---

🔧 What I Tried

1. ALoHa with default params
   – Ran training script → agent froze in one posture.
2. VQ Encoder
   – Difficult to integrate, only 1 fps output → impractical.
   – (ALoHa team: clearer docs would help.)
3. Temporal Embedding at Inference
   – Without it → jerky, inconsistent motions (needs formal testing).
4. Hyperparameter Sweeps
   – LR, scheduler, batch size, chunk size, KL weight, data volume, training steps, encoder on/off.
   – All variants eventually got “stuck” in a single pose.
5. Simple Task Baseline
   – Cube→bin pick-and-place learned perfectly under constrained environment.
   – Used that as a starting point for harder Post-it task.

---

🚀 Breakthrough

• Hidden Dim increase: 512 → 1024
  • Result: significant improvement
  • More pick/place behaviors, proper sequencing (lower first cube before touching the next)
• Lesson: Larger hidden dimensions help handle complex tasks.

---

🔜 Next Steps

• Co-train with varied datasets
• Revisit the native ALoHa platform for better data-edit workflow

---

✉️ Contact: aoml3245@knu.ac.kr
Feel free to reach out for questions or tips!

[yunho-c]

Hi, thank you for sharing the tips! Since the GitHub issue tracker is primarily used for troubleshooting, it might be better if you shared the results someplace else—maybe in a blog post, or the LeRobot community page on HuggingFace.