Skip to content

Issues: huggingface/trl

Beta
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

398 Open 1,408 Closed
398 Open 1,408 Closed
Author
Filter by author
Loading
Label
Filter by label
Loading
Use alt + click/return to exclude labels
or ⇧ + click/return for logical OR
Projects
Filter by project
Loading
Milestones
Filter by milestone
Loading
Assignee
Filter by who’s assigned
Assigned to nobody Loading
Sort
Sort by
Newest Oldest Most commented Least commented Recently updated Least recently updated Best match
Most reactions
👍 👎 😄 🎉 😕 ❤️ 🚀 👀

Issues list

Converting a conversational dataset into a standard dataset [not working] 🐛 bug Something isn't working
#3490 opened May 23, 2025 by nbasyl
5 tasks done
1
RuntimeError: NCCL error: invalid usage when only one GPU ⚡accelerate Related to accelerate 🐛 bug Something isn't working
#3487 opened May 23, 2025 by wa008
5 tasks done
Completions Only Loss is incompatible with use_liger_kernel set as true 🐛 bug Something isn't working 🏋 SFT Related to SFT
#3484 opened May 22, 2025 by arashpreetsinghmor
Vision Fine Tuning Gemma 3 takes Impossiblily High VRam (OOM Error 8xH200) ⚡accelerate Related to accelerate 🐛 bug Something isn't working ⚡ PEFT Related to PEFT
#3481 opened May 22, 2025 by amanmehra89
5 tasks done
【GRPO】Why are some batches of prompts not involved in training?
#3477 opened May 22, 2025 by moguizhizi
5 tasks done
3
[GRPO] completion lengths are incorrectly logged when mask_truncated_completions=True 🐛 bug Something isn't working 🏋 GRPO Related to GRPO
#3476 opened May 22, 2025 by edbeeching
5 tasks done
1
@shirinyamani
3
Is it possible to make prompts dynamic (or iterable datasets) in GRPO training? ✨ enhancement New feature or request 🏋 GRPO Related to GRPO
#3474 opened May 21, 2025 by onlyjokers
1
[GPG][new trainer] Add support to new GPG method ✨ enhancement New feature or request
#3472 opened May 20, 2025 by lerogo
3 tasks done
[SFT] Can't apply_chat_template prompt-completion dataset. so, training result always bad. 🐛 bug Something isn't working 🏋 SFT Related to SFT
#3468 opened May 19, 2025 by HERIUN
5 tasks done
6
trl vllm server generating stuck 🐛 bug Something isn't working 🏋 GRPO Related to GRPO
#3467 opened May 19, 2025 by AdaChambers
5 tasks done
7
[GRPO] bnb quantization + vllm 🐛 bug Something isn't working 🏋 GRPO Related to GRPO ⚡ PEFT Related to PEFT
#3466 opened May 18, 2025 by shon-otmazgin-wix
5 tasks done
3
PPO Training does not improve SFT model outputs (Metrics identical before and after PPO) 🏋 PPO Related to PPO 🏋 SFT Related to SFT
#3464 opened May 18, 2025 by xmriz
Turn off Accelerate acceleration ⚡accelerate Related to accelerate 🏋 GRPO Related to GRPO
#3461 opened May 17, 2025 by seTalent
5
Out of Memory when GRPO fine-tune Qwen3 4B model on 80G A100 GPU 🐛 bug Something isn't working 🏋 GRPO Related to GRPO
#3456 opened May 16, 2025 by wa008
5 tasks done
2
Reward_model error in cli GRPO train when I only use reward function rather than reward_model 🐛 bug Something isn't working 📱 cli Related to the Command-line interface 🏋 GRPO Related to GRPO 🏋 Reward Related to Reward modelling
#3455 opened May 16, 2025 by wa008
5 tasks done
PPO training fails when used with accelerate ⚡️ and Deepspeed 🚀 ⚡accelerate Related to accelerate 🚀 deepspeed Related to deepspeed 🏋 PPO Related to PPO 🏋 SFT Related to SFT
#3453 opened May 16, 2025 by marcellobullo
5 tasks done
1
GRPO reward=0 and loss=0 🏋 GRPO Related to GRPO 🏋 Reward Related to Reward modelling
#3452 opened May 15, 2025 by LIUyizheSDU
1
torch distributed training with multi gpus errors in GRPOtrainer 🐛 bug Something isn't working 🏋 GRPO Related to GRPO
#3451 opened May 15, 2025 by jinhonglu
5 tasks done
4
trl vllm-serve not working on latest. 🐛 bug Something isn't working 🏋 GRPO Related to GRPO
#3450 opened May 15, 2025 by tcapelle
5 tasks done
3
Rewards functions for Command Line Interfaces GRPO trainer 📱 cli Related to the Command-line interface ✨ enhancement New feature or request 🏋 GRPO Related to GRPO 🏋 Reward Related to Reward modelling
#3448 opened May 15, 2025 by wa008
1
6
VLLM Integeration cannot run with data parrallel alone ⚡accelerate Related to accelerate 🐛 bug Something isn't working ⚡ PEFT Related to PEFT
#3446 opened May 14, 2025 by MAOJIASONG
5 tasks done
4
[GRPO] num_generations 🏋 GRPO Related to GRPO ❓ question Seeking clarification or more information
#3443 opened May 13, 2025 by shon-otmazgin-wix
5 tasks done
7
Unstructured data grpo training 🐛 bug Something isn't working 🏋 GRPO Related to GRPO
#3441 opened May 13, 2025 by yuyuhua918
7
Logging docs appear to be out of date 📚 documentation Improvements or additions to documentation
#3437 opened May 12, 2025 by ezyang
5 tasks done
3
FSDP SFT deepseekv2-prover7B OOM on 8*RTX3080
#3436 opened May 12, 2025 by LIUyizheSDU
1
ProTip! Add no:assignee to see everything that’s not assigned.