Receiving `You may use wrong User-Agent` when submitting to Codeforces

[hotarunw]

Description

When I try to submit to Codeforces, it receives wrong User-Agent error message and this submission failed.

[WARNING] You may use wrong User-Agent: 'online-judge-tools/11.1.1 (+https://github.com/online-judge-tools/oj)'
[FAILURE] submission failed

Error log

I have already signed into Codeforces using CUI-based login.

$ oj login https://codeforces.com/contest/1447/problem/A
[INFO] online-judge-tools 11.1.1 (+ online-judge-api-client 10.5.0)
[INFO] service recognized: CodeforcesService.from_url('https://codeforces.com/'): https://codeforces.com/contest/1447/problem/A
[NETWORK] GET: https://codeforces.com/enter
[NETWORK] 200 OK
[FAILURE] You are not signed in.
[INFO] Trying to open Chrome via WebDriver...
[ERROR] Message: 'chromedriver' executable needs to be in PATH. Please see https://sites.google.com/a/chromium.org/chromedriver/home

[INFO] Trying to open Firefox via WebDriver...
[ERROR] Message: 'geckodriver' executable needs to be in PATH.

[INFO] Trying to open Edge via WebDriver...
[ERROR] Message: 'MicrosoftWebDriver.exe' executable needs to be in PATH. Please download from http://go.microsoft.com/fwlink/?LinkId=619687

[INFO] Trying to open Internet Explorer via WebDriver...
[ERROR] Message: 'IEDriverServer.exe' executable needs to be in PATH. Please download from http://selenium-release.storage.googleapis.com/index.html and read up at https://github.com/SeleniumHQ/selenium/wiki/InternetExplorerDriver

[INFO] Trying to open Safari via WebDriver...
[ERROR] SafariDriver was not found; are you running Safari 10 or later? You can download Safari at https://developer.apple.com/safari/download/.
[INFO] Trying to open Opera via WebDriver...
[ERROR] Message: 'operadriver' executable needs to be in PATH. Please see https://sites.google.com/a/chromium.org/chromedriver/home

[ERROR] No WebDriver is available.
[HINT] Please install a WebDriver.
See https://www.selenium.dev/documentation/en/webdriver/driver_requirements/

Detailed instructions:
    If you use Ubuntu:
        1. Run $ sudo apt install chromium-chromedriver firefox-geckodriver
    If you use Ubuntu under Windows Subsystem for Linux:
        1. Make a symbolic link from cookie.jar in WSL to cookie.jar out of WSL. For example, run $ ln -s /mnt/c/Users/%USERNAME%/AppData/Local/online-judge-tools/online-judge-tools/cookie.jar /home/ubuntu/.local/share/online-judge-tools/cookie.jar
        2. Use `oj login` outside of WSL
    If you use Windows:
        1. Install Chocolatey. See https://chocolatey.org/
        2. Run > choco install selenium-all-drivers

[WARNING] Switch to use CUI-based login instead of Selenium
[NETWORK] GET: https://codeforces.com/enter
[NETWORK] 200 OK
Username: seiran
Password:
[NETWORK] POST: https://codeforces.com/enter
[NETWORK] redirected to: https://codeforces.com/
[NETWORK] 200 OK
[INFO] Welcome, seiran.
[NETWORK] GET: https://codeforces.com/enter
[NETWORK] 302 Found
[SUCCESS] You have already signed in.
[INFO] save cookie to: /home/hotaru/.local/share/online-judge-tools/cookie.jar

$ oj s submit.cpp
[INFO] online-judge-tools 11.1.1 (+ online-judge-api-client 10.5.0)
[INFO] read history from: /home/hotaru/.cache/online-judge-tools/download-history.jsonl
[INFO] found urls in history:
https://codeforces.com/contest/1447/problem/A
[INFO] guessed problem: https://codeforces.com/contest/1447/problem/A
[INFO] problem recognized: CodeforcesProblem.from_url('https://codeforces.com/contest/1447/problem/A'): https://codeforces.com/contest/1447/problem/A
[INFO] code (1239 byte):

** MY SOLUTION **

[INFO] load cookie from: /home/hotaru/.local/share/online-judge-tools/cookie.jar
[NETWORK] GET: https://codeforces.com/enter
[NETWORK] 302 Found
[INFO] You are logged in.
[NETWORK] GET: https://codeforces.com/contest/1447/problem/A
[NETWORK] 200 OK
[INFO] both GCC and Clang are available for C++ compiler
[INFO] use: GCC
[INFO] chosen language: 61 (GNU G++17 9.2.0 (64 bit, msys 2))
[INFO] sleep(3.00)
Are you sure? [y/N] y
[NETWORK] GET: https://codeforces.com/contest/1447/problem/A
[NETWORK] 200 OK
[NETWORK] POST: https://codeforces.com/contest/1447/problem/A
[NETWORK] 403 Forbidden
[ERROR] 403 Client Error:  for url: https://codeforces.com/contest/1447/problem/A
Traceback (most recent call last):
  File "/home/hotaru/.local/lib/python3.8/site-packages/onlinejudge/service/codeforces.py", line 389, in submit_code
    resp.raise_for_status()
  File "/home/hotaru/.local/lib/python3.8/site-packages/requests/models.py", line 943, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 403 Client Error:  for url: https://codeforces.com/contest/1447/problem/A
[WARNING] You may use wrong User-Agent: 'online-judge-tools/11.1.1 (+https://github.com/online-judge-tools/oj)'
[FAILURE] submission failed

Environment

Ubuntu under Windows Subsystem for Linux

Windows

• エディション: Windows 10 Pro
• バージョン: 20H2
• OS ビルド: 19042.630

WSL

• WSL version: WSL2
• Linux distribution: Ubuntu 20.04.1 LTS

Python and oj

$ python3 --version
Python 3.8.5
$ oj --version
online-judge-tools 11.1.1 (+ online-judge-api-client 10.5.0)

Other notes

• Will you try to create a pull request?
  • no

[kmyk]

@hotaroon Thank you for reporting! (This is well-written bugreport 😄)

Conclusion: Let me wait and see how it goes.

While waiting, the same issue will reported at cf-tool, or Codeforces may change the logic.
I'll investigate further when this is not fixed even after a while (in this weekend?)

Observation

I could reproduce the issue, but the precise condition is still unknown.
My observations are:

1. At first, the submissions succeeded regardless the User-Agent. (User-Agent is irreverent)
2. The first few times submissions succeeded, but after all submissions failed. (it seems "banned")
3. Submissions from my GUI browser (Google Chrome) succeeds even after the "ban". (the "ban" is not associated with IP address nor account)
4. However, after removing the cookie ~/.local/share/online-judge-tools/cookie.jar and re-login, submissions from online-judge-tools still fails. (the "ban" is not associated with the session)

There are many difference between HTTP requests from online-judge-tools and from GUI browsers (e.g. referer, CSRF token, etc.), so probably one of them is the cause.

Logs

User-Agent requests/2.24.0, success

$ oj-api submit-code https://codeforces.com/contest/1336/problem/A --file foo.cpp --language 61
INFO:onlinejudge_api.main:online-judge-api-client 10.5.0
INFO:onlinejudge_api.main:sleep 1.000000 sec
INFO:onlinejudge.dispatch:problem recognized: CodeforcesProblem.from_url('https://codeforces.com/contest/1336/problem/A'): https://codeforces.com/contest/1336/problem/A
INFO:onlinejudge.dispatch:contest recognized: <onlinejudge.service.codeforces.CodeforcesContest object at 0x7fa26bafd828>: https://codeforces.com/contest/1336/problem/A
INFO:onlinejudge.dispatch:service recognized: CodeforcesService.from_url('https://codeforces.com/'): https://codeforces.com/contest/1336/problem/A
INFO:onlinejudge.utils:load cookie from: /home/user/.local/share/online-judge-tools/cookie.jar
INFO:onlinejudge._implementation.utils:network: GET: https://codeforces.com/contest/1336/problem/A
INFO:onlinejudge._implementation.utils:network: 200 OK
INFO:onlinejudge._implementation.utils:network: POST: https://codeforces.com/contest/1336/problem/A
INFO:onlinejudge._implementation.utils:network: redirected to: https://codeforces.com/contest/1336/my
INFO:onlinejudge._implementation.utils:network: 200 OK
INFO:onlinejudge.service.codeforces:success: result: https://codeforces.com/contest/1336/my
INFO:onlinejudge.utils:save cookie to: /home/user/.local/share/online-judge-tools/cookie.jar
{"status": "ok", "messages": [], "result": {"url": "https://codeforces.com/contest/1336/my"}}

User-Agent online-judge-tools/11.1.1 (+https://github.com/online-judge-tools/oj), success

$ oj s https://codeforces.com/contest/1447/problem/A foo.cpp
[INFO] online-judge-tools 11.1.1 (+ online-judge-api-client 10.5.0)
[INFO] read history from: /home/user/.cache/online-judge-tools/download-history.jsonl
[INFO] found urls in history:

[INFO] problem recognized: CodeforcesProblem.from_url('https://codeforces.com/contest/1447/problem/A'): https://codeforces.com/contest/1447/problem/A
[INFO] code (12 byte):
hello world

[INFO] load cookie from: /home/user/.local/share/online-judge-tools/cookie.jar
[NETWORK] GET: https://codeforces.com/enter
[NETWORK] 302 Found
[INFO] You are logged in.
[NETWORK] GET: https://codeforces.com/contest/1447/problem/A
[NETWORK] 200 OK
[INFO] both GCC and Clang are available for C++ compiler
[INFO] use: GCC
[INFO] chosen language: 61 (GNU G++17 9.2.0 (64 bit, msys 2))
[WARNING] the problem "https://codeforces.com/contest/1447/problem/A" is specified to submit, but no samples were downloaded in this directory. this may be mis-operation
[INFO] sleep(3.00)
Are you sure? Please type "A" A
[NETWORK] GET: https://codeforces.com/contest/1447/problem/A
[NETWORK] 200 OK
[NETWORK] POST: https://codeforces.com/contest/1447/problem/A
[NETWORK] redirected to: https://codeforces.com/contest/1447/my
[NETWORK] 200 OK
[SUCCESS] result: https://codeforces.com/contest/1447/my
[INFO] open the submission page with browser: <webbrowser.BackgroundBrowser object at 0x7f9cbf231240>
[INFO] save cookie to: /home/user/.local/share/online-judge-tools/cookie.jar

User-Agent online-judge-tools/11.1.1 (+https://github.com/online-judge-tools/oj), failure

$ oj s https://codeforces.com/contest/1447/problem/A foo.cpp                                             
[INFO] online-judge-tools 11.1.1 (+ online-judge-api-client 10.5.0)
[INFO] read history from: /home/user/.cache/online-judge-tools/download-history.jsonl
[INFO] found urls in history:

[INFO] problem recognized: CodeforcesProblem.from_url('https://codeforces.com/contest/1447/problem/A'): https://codeforces.com/contest/1447/problem/A
[INFO] code (24 byte):
hello hello world hello

[INFO] load cookie from: /home/user/.local/share/online-judge-tools/cookie.jar
[NETWORK] GET: https://codeforces.com/enter
[NETWORK] 302 Found
[INFO] You are logged in.
[NETWORK] GET: https://codeforces.com/contest/1447/problem/A
[NETWORK] 200 OK
[INFO] both GCC and Clang are available for C++ compiler
[INFO] use: GCC
[INFO] chosen language: 61 (GNU G++17 9.2.0 (64 bit, msys 2))
[WARNING] the problem "https://codeforces.com/contest/1447/problem/A" is specified to submit, but no samples were downloaded in this directory. this may be mis-operation
[INFO] sleep(3.00)
Are you sure? Please type "A" A
[NETWORK] GET: https://codeforces.com/contest/1447/problem/A
[NETWORK] 200 OK
[NETWORK] POST: https://codeforces.com/contest/1447/problem/A
[NETWORK] 403 Forbidden
[ERROR] 403 Client Error:  for url: https://codeforces.com/contest/1447/problem/A
Traceback (most recent call last):
  File "/home/user/.local/lib/python3.6/site-packages/onlinejudge/service/codeforces.py", line 389, in submit_code
    resp.raise_for_status()
  File "/home/user/.local/lib/python3.6/site-packages/requests/models.py", line 943, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 403 Client Error:  for url: https://codeforces.com/contest/1447/problem/A
[WARNING] You may use wrong User-Agent: 'online-judge-tools/11.1.1 (+https://github.com/online-judge-tools/oj)'
[FAILURE] submission failed

User-Agent Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36, failure

$ oj-api --user-agent 'Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36' submit-code https://codeforces.com/contest/1336/problem/A --file foo.cpp --language 61 INFO:onlinejudge_api.main:online-judge-api-client 10.5.0 INFO:onlinejudge_api.main:sleep 1.000000 sec INFO:onlinejudge.dispatch:problem recognized: CodeforcesProblem.from_url('https://codeforces.com/contest/1336/problem/A'): https://codeforces.com/contest/1336/problem/A INFO:onlinejudge.dispatch:contest recognized: : https://codeforces.com/contest/1336/problem/A INFO:onlinejudge.dispatch:service recognized: CodeforcesService.from_url('https://codeforces.com/'): https://codeforces.com/contest/1336/problem/A INFO:onlinejudge.utils:load cookie from: /home/user/.local/share/online-judge-tools/cookie.jar INFO:onlinejudge._implementation.utils:network: GET: https://codeforces.com/contest/1336/problem/A INFO:onlinejudge._implementation.utils:network: 200 OK INFO:onlinejudge._implementation.utils:network: POST: https://codeforces.com/contest/1336/problem/A INFO:onlinejudge._implementation.utils:network: 403 Forbidden ERROR:onlinejudge.service.codeforces:403 Client Error: for url: https://codeforces.com/contest/1336/problem/A Traceback (most recent call last): File "/home/user/.local/lib/python3.6/site-packages/onlinejudge/service/codeforces.py", line 389, in submit_code resp.raise_for_status() File "/home/user/.local/lib/python3.6/site-packages/requests/models.py", line 943, in raise_for_status raise HTTPError(http_error_msg, response=self) requests.exceptions.HTTPError: 403 Client Error: for url: https://codeforces.com/contest/1336/problem/A WARNING:onlinejudge.service.codeforces:You may use wrong User-Agent: 'Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36' ERROR:onlinejudge_api.main:You may use wrong User-Agent: 'Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36': 403 Client Error: for url: https://codeforces.com/contest/1336/problem/A Traceback (most recent call last): File "/home/user/.local/lib/python3.6/site-packages/onlinejudge/service/codeforces.py", line 389, in submit_code resp.raise_for_status() File "/home/user/.local/lib/python3.6/site-packages/requests/models.py", line 943, in raise_for_status raise HTTPError(http_error_msg, response=self) requests.exceptions.HTTPError: 403 Client Error: for url: https://codeforces.com/contest/1336/problem/A

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/home/user/.local/lib/python3.6/site-packages/onlinejudge_api/main.py", line 288, in main
result = submit_code.main(problem, file=parsed.file, language_id=parsed.language, session=session)
File "/home/user/.local/lib/python3.6/site-packages/onlinejudge_api/submit_code.py", line 31, in main
submission = problem.submit_code(code, language_id=language_id, filename=str(file), session=session)
File "/home/user/.local/lib/python3.6/site-packages/onlinejudge/service/codeforces.py", line 394, in submit_code
raise SubmissionError('You may use wrong User-Agent: {}: {}'.format(repr(session.headers.get('User-Agent')), e))
onlinejudge.type.SubmissionError: You may use wrong User-Agent: 'Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36': 403 Client Error: for url: https://codeforces.com/contest/1336/problem/A
{"status": "error", "messages": ["onlinejudge.type.SubmissionError: You may use wrong User-Agent: 'Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36': 403 Client Error: for url: https://codeforces.com/contest/1336/problem/A"], "result": null}

[kmyk]

I merged a commit #120 to fix this issue.

I think this is fixed, but it's difficult to prove that this types of problems are truly fixed.
Please re-open or comment to this issue when this issue remains.

[peroon]

同じ問題が私にも今起こりました。ojは最新です。

提出用formタグを見てみるとcsrf_tokenが設定されているようで、
一方ojのcodeforces.pyにはその記述はありません。

これは関係ないでしょうか？
（的はずれな指摘でしたらすみません・・・）

[R1B3n-13]

The issue remains after the update. 😭😭

[kmyk]

Okay. I'll investigate this issue again 🙇

[kmyk]

I cannot reproduce this error now. All of consecutive eight submissions succeeded. It seems to be fixed, at least on my environment.

@peroon 分かりにくいんですが csrf_token は実はすでに含まれているはずです。<form> タグが解析され自動で送信されています。(リンク)
@peroon @Tard1grad3 Can you give me the detailed log ($ oj --verbose s https://... ... or something) with contexts (e.g. the first and second submission succeeded but the third submission failed.)? For this, please be careful not to include secret values like CSRF tokens. Thank you!