Skip to content

[Bug]: when try to test the benchmark swe-bench_Lite, I'm getting a running error with docker.py but I don't know how to fix it. #8570

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
1 task
Succoney opened this issue May 18, 2025 · 3 comments
Labels
bug Something isn't working evaluation Related to running evaluations with OpenHands

Comments

@Succoney
Copy link

Succoney commented May 18, 2025

Is there an existing issue for the same bug? (If one exists, thumbs up or comment on the issue instead).

  • I have checked the existing issues.

Describe the bug and reproduction steps

When I wanted to verify the test results of OpenHands on Swe-bench, I followed the instructions in the repository to install the environment and configure the llm parameters. But when I started reasoning, an error was unexpectedly reported. It reported an error when running docker.py, saying that apt-get update had no permissions. But when I changed the code here to sudo apt-get update, another error was reported. In short, the error should be reported when downloading docker.

OpenHands Installation

Docker command in README

OpenHands Version

No response

Operating System

Linux

Logs, Errors, Screenshots, and Additional Context

web:
https://github.com/All-Hands-AI/OpenHands/tree/main/evaluation/benchmarks/swe_bench

bash:
./evaluation/benchmarks/swe_bench/scripts/run_infer.sh llm.deepseek-chat HEAD CodeActAgent 10 10 1 princeton-nlp/SWE-bench_Lite test

error:

Image

@Succoney Succoney added the bug Something isn't working label May 18, 2025
@Succoney
Copy link
Author

Image

@csmith49
Copy link
Collaborator

Can't say I've seen this exact behavior, but 9/10 times errors when building the Docker image are caused by running out of storage. Probably worth trying a docker system prune and seeing if that fixes anything.

@Succoney
Copy link
Author

Thank you for answering my question. This problem has been solved. I think it should be that the terminal running permissions of Ubuntu are low. When I re-run the code on Mac OS, there is no problem at all.

@mamoodi mamoodi added the evaluation Related to running evaluations with OpenHands label May 20, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working evaluation Related to running evaluations with OpenHands
Projects
None yet
Development

No branches or pull requests

3 participants