Skip to content

[Bug]: o4-mini and o3-mini display no thoughts #8572

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
1 task done
neubig opened this issue May 19, 2025 · 4 comments
Open
1 task done

[Bug]: o4-mini and o3-mini display no thoughts #8572

neubig opened this issue May 19, 2025 · 4 comments
Labels
bug Something isn't working llm Related to specific LLMs

Comments

@neubig
Copy link
Contributor

neubig commented May 19, 2025

Is there an existing issue for the same bug? (If one exists, thumbs up or comment on the issue instead).

  • I have checked the existing issues.

Describe the bug and reproduction steps

When using OpenHands with o4-mini or o3-mini, they display no thoughts in the frontend.

Image

This is confusing to users, who can not tell why the agent did what it did.

Thanks @kentyman23 for pointing this out.

OpenHands Installation

Docker command in README

OpenHands Version

No response

Operating System

None

Logs, Errors, Screenshots, and Additional Context

No response

@neubig neubig added the bug Something isn't working label May 19, 2025
neubig added a commit that referenced this issue May 19, 2025
Codeact is sufficiently communicative with Claude, but with o4 mini it is not. This PR is an attempt to fix this.

Fixes: #8572
@erkinalp
Copy link
Contributor

NOTABUG: OpenAI Responses API (unlike the ChatGPT Web App API) does not expose chains of thoughts in the returned response.

@neubig
Copy link
Contributor Author

neubig commented May 19, 2025

I know the openAI API does not reveal the internal thoughts of the model, but from a user experience perspective we want to have an explanation of what the model is doing so the users can follow along. We need to find a way to fix this with o4 mini

@erkinalp
Copy link
Contributor

erkinalp commented May 19, 2025

The only way would be to use ChatGPT Web App API, but not even that shows raw CoT (OpenAI o-series models aren't trained to produce human-readable chain of thought responses, the reasoning trace shown in the Web UI is generated by a separate language model translating from ChatGPT gibberish into your language)

@erkinalp
Copy link
Contributor

Soon-to-be-released open-weights release of the o3 series will have a human-readable chain of thought, but expect it to perform worse than the hosted version due to the post training required to make the chain of thought readable and policy-following.

@mamoodi mamoodi added the llm Related to specific LLMs label May 20, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working llm Related to specific LLMs
Projects
None yet
Development

No branches or pull requests

3 participants