Open
Description
When running a LLVM 18.1.0 build on FreeBSD 14.0/amd64 with test-release.sh
, llvm-lit
didn't properly summarize the test results. The build log shows
[...]
PASS: llvm-libc++-shared.cfg.in :: std/utilities/variant/variant.visit/visit_return_type.pass.cpp (97112 of 97112)
Traceback (most recent call last):
File "/var/llvm/reltest/llvm-18.1.0/final/llvm-project/llvm/utils/lit/lit/formats/googletest.py", line 304, in post_process_shard_results
testsuites = json.load(f)["testsuites"]
File "/usr/local/lib/python3.9/json/__init__.py", line 293, in load
return loads(fp.read(),
File "/usr/local/lib/python3.9/json/__init__.py", line 346, in loads
return _default_decoder.decode(s)
File "/usr/local/lib/python3.9/json/decoder.py", line 337, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "/usr/local/lib/python3.9/json/decoder.py", line 355, in raw_decode
raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/var/llvm/reltest/llvm-18.1.0/final/Phase3/Release/llvmCore-18.1.0-final.obj/./bin/llvm-lit", line 65, in <module>
main(builtin_parameters)
File "/var/llvm/reltest/llvm-18.1.0/final/llvm-project/llvm/utils/lit/lit/main.py", line 129, in main
selected_tests, discovered_tests = GoogleTest.post_process_shard_results(
File "/var/llvm/reltest/llvm-18.1.0/final/llvm-project/llvm/utils/lit/lit/formats/googletest.py", line 306, in post_process_shard_results
raise RuntimeError(
RuntimeError: Failed to parse json file: /var/llvm/reltest/llvm-18.1.0/final/Phase3/Release/llvmCore-18.1.0-final.obj/runtimes/runtimes-bins/compiler-rt/lib/msan/tests/./Msan-x86_64-Test-MemorySanitizer-Unit-37844-0-9.json
FAILED: CMakeFiles/check-all /var/llvm/reltest/llvm-18.1.0/final/Phase3/Release/llvmCore-18.1.0-final.obj/CMakeFiles/check-all
I could trace this to two msan tests producing empty *.json
files (Issue #84482), which googletest.py
chokes on. This must not happen: llvm-lit
needs to be robust enough to cope with such input, invalid or no.