Skip to content

I hope grpo supports visual modeling #3232

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
chaodreaming opened this issue Apr 4, 2025 · 3 comments · May be fixed by #3460
Open

I hope grpo supports visual modeling #3232

chaodreaming opened this issue Apr 4, 2025 · 3 comments · May be fixed by #3460
Labels
✨ enhancement New feature or request 🏋 GRPO Related to GRPO

Comments

@chaodreaming
Copy link

Feature request

I see that trl is just one data processing away from supporting vision

Motivation

Visual models can also be enhanced themselves using grpo, and other libraries have implemented visual

Your contribution

[no](https://github.com/Deep-Agent/R1-V)
@github-actions github-actions bot added 🏋 GRPO Related to GRPO ✨ enhancement New feature or request labels Apr 4, 2025
@Aravind-LatentForce
Copy link

Vision support(especially for Qwen 2.5 3B and 7B Instruct) is going to be extremely helpful. Anyone working on this?

@hellopahe hellopahe mentioned this issue May 7, 2025
Open
@antichristHater
Copy link

antichristHater commented May 15, 2025 •
edited
Loading

Anyone working on this?

It would be really helpful if they were!

@Aravind-LatentForce
Copy link

Anyone working on this?

It would be really helpful if they were!

I'm still facing issues with Qwen 2.5 VL training using GRPO. It'd be such a nice addition to have when it works.

@hellopahe hellopahe linked a pull request May 16, 2025 that will close this issue
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
✨ enhancement New feature or request 🏋 GRPO Related to GRPO
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

add support for image inputs in GRPO hellopahe/trl
3 participants
@chaodreaming @antichristHater @Aravind-LatentForce