Please download summary.xlsx to view the most recent results.
[2025-08-07 20:39:33,249 submission_checker1.py:3491 INFO] Results=32, NoResults=0, Power Results=0
[2025-08-07 20:39:33,249 submission_checker1.py:3498 INFO] ---
[2025-08-07 20:39:33,249 submission_checker1.py:3499 INFO] Closed Results=0, Closed Power Results=0
[2025-08-07 20:39:33,249 submission_checker1.py:3504 INFO] Open Results=32, Open Power Results=0
[2025-08-07 20:39:33,249 submission_checker1.py:3509 INFO] Network Results=0, Network Power Results=0
[2025-08-07 20:39:33,249 submission_checker1.py:3514 INFO] ---
[2025-08-07 20:39:33,249 submission_checker1.py:3516 INFO] Systems=25, Power Systems=0
[2025-08-07 20:39:33,249 submission_checker1.py:3520 INFO] Closed Systems=0, Closed Power Systems=0
[2025-08-07 20:39:33,249 submission_checker1.py:3525 INFO] Open Systems=25, Open Power Systems=0
[2025-08-07 20:39:33,249 submission_checker1.py:3530 INFO] Network Systems=0, Network Power Systems=0
[2025-08-07 20:39:33,249 submission_checker1.py:3535 INFO] ---
[2025-08-07 20:39:33,249 submission_checker1.py:3540 INFO] SUMMARY: submission looks OK
[2025-08-07 20:39:34,442 module.py:5388 INFO] - ! call "postprocess" from /home/runner/MLC/repos/mlcommons@mlperf-automations/script/run-mlperf-inference-submission-checker/customize.py
| Organization | Availability | Division | SystemType | SystemName | Platform | Model | MlperfModel | Scenario | Result | Accuracy | number_of_nodes | host_processor_model_name | host_processors_per_node | host_processor_core_count | accelerator_model_name | accelerators_per_node | Location | framework | operating_system | notes | compliance | errors | version | inferred | has_power | Units | weight_data_types | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 0 | MLCommons | available | open | edge | gh_ubuntu-latest | gh_ubuntu-latest-reference-cpu-tvm-onnx_v1.19.2-default_config | resnet50 | resnet | Offline | 1.69947 | acc: 80.000 | 1 | AMD EPYC 7763 64-Core Processor | 1 | 2 | nan | 0 | open/MLCommons/results/gh_ubuntu-latest-reference-cpu-tvm-onnx_v1.19.2-default_config/resnet50/offline | tvm-onnx v1.19.2 | Ubuntu 24.04 (linux-6.11.0-1018-azure-glibc2.39) | nan | 1 | 0 | v5.0 | 0 | False | Samples/s | fp32 |
| 1 | MLCommons | available | open | edge | gh_ubuntu-latest x86 | gh_ubuntu-latest_x86-reference-cpu-onnxruntime_v1.22.1-default_config | resnet50 | resnet | Offline | 23.2033 | acc: 76.000 | 1 | AMD EPYC 7763 64-Core Processor | 1 | 2 | nan | 0 | open/MLCommons/results/gh_ubuntu-latest_x86-reference-cpu-onnxruntime_v1.22.1-default_config/resnet50/offline | onnxruntime v1.22.1 | Ubuntu 24.04 (linux-6.11.0-1018-azure-glibc2.39) | nan | 1 | 0 | v5.0 | 0 | False | Samples/s | fp32 |
| 2 | MLCommons | available | open | edge | gh_action | gh_action-reference-gpu-pytorch_v2.6.0-cu124 | stable-diffusion-xl | stable-diffusion-xl | Offline | 0.342746 | CLIP_SCORE: 16.368897303938866 FID_SCORE: 237.82578527053812 | 1 | Intel(R) Xeon(R) w7-2495X | 1 | 24 | NVIDIA GeForce RTX 4090 | 2 | open/MLCommons/results/gh_action-reference-gpu-pytorch_v2.6.0-cu124/stable-diffusion-xl/offline | pytorch v2.6.0 | Ubuntu 22.04 (linux-6.8.0-52-generic-glibc2.35) | nan | 1 | 0 | v5.0 | 0 | False | Samples/s | fp32 |
| 3 | MLCommons | available | open | edge | gh_ubuntu-latest x86 | gh_ubuntu-latest_x86-mlcommons_cpp-cpu-onnxruntime-default_config | retinanet | retinanet | Offline | 0.413064 | mAP: 76.951 | 1 | AMD EPYC 7763 64-Core Processor | 1 | 2 | nan | 0 | open/MLCommons/results/gh_ubuntu-latest_x86-mlcommons_cpp-cpu-onnxruntime-default_config/retinanet/offline | onnxruntime | Ubuntu 24.04 (linux-6.11.0-1018-azure-glibc2.39) | nan | 1 | 0 | v5.0 | 0 | False | Samples/s | fp32 |
| 4 | MLCommons | available | open | edge | gh_ubuntu-latest x86 | gh_ubuntu-latest_x86-mlcommons_cpp-cpu-onnxruntime-default_config | resnet50 | resnet | Offline | 23.146 | acc: 76.000 | 1 | AMD EPYC 7763 64-Core Processor | 1 | 2 | nan | 0 | open/MLCommons/results/gh_ubuntu-latest_x86-mlcommons_cpp-cpu-onnxruntime-default_config/resnet50/offline | onnxruntime | Ubuntu 24.04 (linux-6.11.0-1018-azure-glibc2.39) | nan | 1 | 0 | v5.0 | 0 | False | Samples/s | fp32 |
| 5 | MLCommons | available | open | edge | gh_macos-latest x86 | gh_macos-latest_x86-reference-cpu-onnxruntime_v1.22.0-default_config | resnet50 | resnet | Offline | 15.9553 | acc: 76.000 | 1 | undefined | 1 | 3 | nan | 0 | open/MLCommons/results/gh_macos-latest_x86-reference-cpu-onnxruntime_v1.22.0-default_config/resnet50/offline | onnxruntime v1.22.0 | (darwin-23.6.0) | nan | 1 | 0 | v5.0 | 0 | False | Samples/s | fp32 |
| 6 | MLCommons | available | open | edge | gh_macos-latest x86 | gh_macos-latest_x86-mlcommons_cpp-cpu-onnxruntime-default_config | resnet50 | resnet | Offline | 9.63746 | acc: 76.000 | 1 | undefined | 1 | 3 | nan | 0 | open/MLCommons/results/gh_macos-latest_x86-mlcommons_cpp-cpu-onnxruntime-default_config/resnet50/offline | onnxruntime | (darwin-23.6.0) | nan | 1 | 0 | v5.0 | 0 | False | Samples/s | fp32 |
| 7 | MLCommons | available | open | edge | gh_ubuntu-latest x86 | gh_ubuntu-latest_x86-reference-cpu-tf_v2.18.0-default_config | resnet50 | resnet | Offline | 20.9828 | acc: 76.000 | 1 | AMD EPYC 7763 64-Core Processor | 1 | 2 | nan | 0 | open/MLCommons/results/gh_ubuntu-latest_x86-reference-cpu-tf_v2.18.0-default_config/resnet50/offline | tf v2.18.0 | Ubuntu 24.04 (linux-6.8.0-1021-azure-glibc2.39) | nan | 1 | 0 | v5.0 | 0 | False | Samples/s | fp32 |
| 8 | MLCommons | available | open | edge | gh_macos-latest_x86 | gh_macos-latest_x86-reference-cpu-pytorch_v2.7.0-default_config | retinanet | retinanet | Offline | 0.438894 | mAP: 76.951 | 1 | undefined | 1 | 3 | nan | 0 | open/MLCommons/results/gh_macos-latest_x86-reference-cpu-pytorch_v2.7.0-default_config/retinanet/offline | pytorch v2.7.0 | (darwin-23.6.0) | nan | 1 | 0 | v5.0 | 0 | False | Samples/s | fp32 |
| 9 | MLCommons | available | open | edge | gh_ubuntu-latest x86 | gh_ubuntu-latest_x86-reference-cpu-onnxruntime_v1.22.0-default_config | resnet50 | resnet | Offline | 23.1581 | acc: 76.000 | 1 | AMD EPYC 7763 64-Core Processor | 1 | 2 | nan | 0 | open/MLCommons/results/gh_ubuntu-latest_x86-reference-cpu-onnxruntime_v1.22.0-default_config/resnet50/offline | onnxruntime v1.22.0 | Ubuntu 24.04 (linux-6.11.0-1015-azure-glibc2.39) | nan | 1 | 0 | v5.0 | 0 | False | Samples/s | fp32 |
| 10 | MLCommons | available | open | edge | gh_macos-latest x86 | gh_macos-latest_x86-reference-cpu-onnxruntime_v1.20.1-default_config | retinanet | retinanet | Offline | 0.353561 | mAP: 76.951 | 1 | undefined | 1 | 3 | nan | 0 | open/MLCommons/results/gh_macos-latest_x86-reference-cpu-onnxruntime_v1.20.1-default_config/retinanet/offline | onnxruntime v1.20.1 | (darwin-23.6.0) | nan | 1 | 0 | v5.0 | 0 | False | Samples/s | fp32 |
| 11 | MLCommons | available | open | edge | gh_macos-latest x86 | gh_macos-latest_x86-reference-cpu-onnxruntime_v1.20.1-default_config | resnet50 | resnet | Offline | 17.8765 | acc: 76.000 | 1 | undefined | 1 | 3 | nan | 0 | open/MLCommons/results/gh_macos-latest_x86-reference-cpu-onnxruntime_v1.20.1-default_config/resnet50/offline | onnxruntime v1.20.1 | (darwin-23.6.0) | nan | 1 | 0 | v5.0 | 0 | False | Samples/s | fp32 |
| 12 | MLCommons | available | open | edge | gh_ubuntu-latest x86 | gh_ubuntu-latest_x86-reference-cpu-onnxruntime_v1.21.1-default_config | retinanet | retinanet | Offline | 0.431769 | mAP: 76.951 | 1 | AMD EPYC 7763 64-Core Processor | 1 | 2 | nan | 0 | open/MLCommons/results/gh_ubuntu-latest_x86-reference-cpu-onnxruntime_v1.21.1-default_config/retinanet/offline | onnxruntime v1.21.1 | Ubuntu 24.04 (linux-6.11.0-1013-azure-glibc2.39) | nan | 1 | 0 | v5.0 | 0 | False | Samples/s | fp32 |
| 13 | MLCommons | available | open | edge | gh_ubuntu-latest x86 | gh_ubuntu-latest_x86-reference-cpu-onnxruntime_v1.21.1-default_config | resnet50 | resnet | Offline | 23.0274 | acc: 76.000 | 1 | AMD EPYC 7763 64-Core Processor | 1 | 2 | nan | 0 | open/MLCommons/results/gh_ubuntu-latest_x86-reference-cpu-onnxruntime_v1.21.1-default_config/resnet50/offline | onnxruntime v1.21.1 | Ubuntu 24.04 (linux-6.11.0-1013-azure-glibc2.39) | nan | 1 | 0 | v5.0 | 0 | False | Samples/s | fp32 |
| 14 | MLCommons | available | open | edge | gh_ubuntu-latest_x86 | gh_ubuntu-latest_x86-reference-cpu-onnxruntime_v1.20.1-default_config | retinanet | retinanet | Offline | 0.436959 | mAP: 76.951 | 1 | AMD EPYC 7763 64-Core Processor | 1 | 2 | nan | 0 | open/MLCommons/results/gh_ubuntu-latest_x86-reference-cpu-onnxruntime_v1.20.1-default_config/retinanet/offline | onnxruntime v1.20.1 | Ubuntu 24.04 (linux-6.8.0-1021-azure-glibc2.39) | nan | 1 | 0 | v5.0 | 0 | False | Samples/s | fp32 |
| 15 | MLCommons | available | open | edge | gh_ubuntu-latest_x86 | gh_ubuntu-latest_x86-reference-cpu-onnxruntime_v1.20.1-default_config | resnet50 | resnet | Offline | 22.8483 | acc: 76.000 | 1 | AMD EPYC 7763 64-Core Processor | 1 | 2 | nan | 0 | open/MLCommons/results/gh_ubuntu-latest_x86-reference-cpu-onnxruntime_v1.20.1-default_config/resnet50/offline | onnxruntime v1.20.1 | Ubuntu 24.04 (linux-6.8.0-1021-azure-glibc2.39) | nan | 1 | 0 | v5.0 | 0 | False | Samples/s | fp32 |
| 16 | MLCommons | available | open | edge | gh_ubuntu-latest_x86 | gh_ubuntu-latest_x86-reference-cpu-onnxruntime_v1.21.0-default_config | retinanet | retinanet | Offline | 0.437938 | mAP: 76.951 | 1 | AMD EPYC 7763 64-Core Processor | 1 | 2 | nan | 0 | open/MLCommons/results/gh_ubuntu-latest_x86-reference-cpu-onnxruntime_v1.21.0-default_config/retinanet/offline | onnxruntime v1.21.0 | Ubuntu 24.04 (linux-6.8.0-1021-azure-glibc2.39) | nan | 1 | 0 | v5.0 | 0 | False | Samples/s | fp32 |
| 17 | MLCommons | available | open | edge | gh_ubuntu-latest_x86 | gh_ubuntu-latest_x86-reference-cpu-onnxruntime_v1.21.0-default_config | resnet50 | resnet | Offline | 23.2491 | acc: 76.000 | 1 | AMD EPYC 7763 64-Core Processor | 1 | 2 | nan | 0 | open/MLCommons/results/gh_ubuntu-latest_x86-reference-cpu-onnxruntime_v1.21.0-default_config/resnet50/offline | onnxruntime v1.21.0 | Ubuntu 24.04 (linux-6.8.0-1021-azure-glibc2.39) | nan | 1 | 0 | v5.0 | 0 | False | Samples/s | fp32 |
| 18 | MLCommons | available | open | datacenter | gh_ubuntu-latest_x86 | gh_ubuntu-latest_x86-reference-cpu-pytorch_v2.4.0-default_config | rgat | rgat | Offline | 10.1918 | acc: 76.000 | 1 | AMD EPYC 7763 64-Core Processor | 1 | 2 | nan | 0 | open/MLCommons/results/gh_ubuntu-latest_x86-reference-cpu-pytorch_v2.4.0-default_config/rgat/offline | pytorch v2.4.0 | Ubuntu 24.04 (linux-6.8.0-1021-azure-glibc2.39) | nan | 1 | 0 | v5.0 | 0 | False | Samples/s | fp32 |
| 19 | MLCommons | available | open | edge | gh_ubuntu-latest x86 | gh_ubuntu-latest_x86-reference-cpu-tf_v2.19.0-default_config | resnet50 | resnet | Offline | 21.0975 | acc: 76.000 | 1 | AMD EPYC 7763 64-Core Processor | 1 | 2 | nan | 0 | open/MLCommons/results/gh_ubuntu-latest_x86-reference-cpu-tf_v2.19.0-default_config/resnet50/offline | tf v2.19.0 | Ubuntu 24.04 (linux-6.11.0-1018-azure-glibc2.39) | nan | 1 | 0 | v5.0 | 0 | False | Samples/s | fp32 |
| 20 | MLCommons | available | open | edge | gh_macos-latest x86 | gh_macos-latest_x86-reference-cpu-onnxruntime_v1.22.1-default_config | resnet50 | resnet | Offline | 18.3366 | acc: 76.000 | 1 | undefined | 1 | 3 | nan | 0 | open/MLCommons/results/gh_macos-latest_x86-reference-cpu-onnxruntime_v1.22.1-default_config/resnet50/offline | onnxruntime v1.22.1 | (darwin-23.6.0) | nan | 1 | 0 | v5.0 | 0 | False | Samples/s | fp32 |
| 21 | MLCommons | available | open | edge | gh_macos-latest x86 | gh_macos-latest_x86-reference-cpu-onnxruntime_v1.21.0-default_config | retinanet | retinanet | Offline | 0.384271 | mAP: 76.951 | 1 | undefined | 1 | 3 | nan | 0 | open/MLCommons/results/gh_macos-latest_x86-reference-cpu-onnxruntime_v1.21.0-default_config/retinanet/offline | onnxruntime v1.21.0 | (darwin-23.6.0) | nan | 1 | 0 | v5.0 | 0 | False | Samples/s | fp32 |
| 22 | MLCommons | available | open | edge | gh_macos-latest x86 | gh_macos-latest_x86-reference-cpu-onnxruntime_v1.21.0-default_config | resnet50 | resnet | Offline | 19.7056 | acc: 76.000 | 1 | undefined | 1 | 3 | nan | 0 | open/MLCommons/results/gh_macos-latest_x86-reference-cpu-onnxruntime_v1.21.0-default_config/resnet50/offline | onnxruntime v1.21.0 | (darwin-23.6.0) | nan | 1 | 0 | v5.0 | 0 | False | Samples/s | fp32 |
| 23 | MLCommons | available | open | edge | gh_macos-latest x86 | gh_macos-latest_x86-reference-cpu-onnxruntime_v1.21.1-default_config | retinanet | retinanet | Offline | 0.290026 | mAP: 76.951 | 1 | undefined | 1 | 3 | nan | 0 | open/MLCommons/results/gh_macos-latest_x86-reference-cpu-onnxruntime_v1.21.1-default_config/retinanet/offline | onnxruntime v1.21.1 | (darwin-23.6.0) | nan | 1 | 0 | v5.0 | 0 | False | Samples/s | fp32 |
| 24 | MLCommons | available | open | edge | gh_macos-latest x86 | gh_macos-latest_x86-reference-cpu-onnxruntime_v1.21.1-default_config | resnet50 | resnet | Offline | 15.6951 | acc: 76.000 | 1 | undefined | 1 | 3 | nan | 0 | open/MLCommons/results/gh_macos-latest_x86-reference-cpu-onnxruntime_v1.21.1-default_config/resnet50/offline | onnxruntime v1.21.1 | (darwin-23.6.0) | nan | 1 | 0 | v5.0 | 0 | False | Samples/s | fp32 |
| 25 | MLCommons | available | open | edge | gh_ubuntu-latest_x86 | gh_ubuntu-latest_x86-reference-cpu-pytorch_v2.7.0-default_config | retinanet | retinanet | Offline | 0.413749 | mAP: 76.951 | 1 | AMD EPYC 7763 64-Core Processor | 1 | 2 | nan | 0 | open/MLCommons/results/gh_ubuntu-latest_x86-reference-cpu-pytorch_v2.7.0-default_config/retinanet/offline | pytorch v2.7.0 | Ubuntu 24.04 (linux-6.11.0-1013-azure-glibc2.39) | nan | 1 | 0 | v5.0 | 0 | False | Samples/s | fp32 |
| 26 | MLCommons | available | open | edge | gh_ubuntu-latest_x86 | gh_ubuntu-latest_x86-reference-cpu-deepsparse_v1.8.0-default_config | bert-99 | bert-99 | Offline | 6.28304 | F1: 80.0 | 1 | AMD EPYC 7763 64-Core Processor | 1 | 2 | nan | 0 | open/MLCommons/results/gh_ubuntu-latest_x86-reference-cpu-deepsparse_v1.8.0-default_config/bert-99/offline | deepsparse v1.8.0 | Ubuntu 24.04 (linux-6.11.0-1014-azure-glibc2.39) | nan | 1 | 0 | v5.0 | 0 | False | Samples/s | fp32 |
| 27 | MLCommons | available | open | edge | gh_macos-latest_x86 | gh_macos-latest_x86-reference-cpu-deepsparse_v1.8.0-default_config | bert-99 | bert-99 | Offline | 4.48923 | F1: 80.0 | 1 | undefined | 1 | 3 | nan | 0 | open/MLCommons/results/gh_macos-latest_x86-reference-cpu-deepsparse_v1.8.0-default_config/bert-99/offline | deepsparse v1.8.0 | (darwin-23.6.0) | nan | 1 | 0 | v5.0 | 0 | False | Samples/s | fp32 |
| 28 | MLCommons | available | open | edge | gh_ubuntu-latest_x86 | gh_ubuntu-latest_x86-reference-cpu-pytorch_v2.6.0-default_config | retinanet | retinanet | Offline | 0.405224 | mAP: 76.951 | 1 | AMD EPYC 7763 64-Core Processor | 1 | 2 | nan | 0 | open/MLCommons/results/gh_ubuntu-latest_x86-reference-cpu-pytorch_v2.6.0-default_config/retinanet/offline | pytorch v2.6.0 | Ubuntu 24.04 (linux-6.8.0-1021-azure-glibc2.39) | nan | 1 | 0 | v5.0 | 0 | False | Samples/s | fp32 |
| 29 | MLCommons | available | open | edge | default | default-mlcommons_cpp-cpu-onnxruntime-default_config | retinanet | retinanet | Offline | 0.42074 | mAP: 49.593 | 1 | AMD EPYC 7763 64-Core Processor | 1 | 2 | nan | 0 | open/MLCommons/results/default-mlcommons_cpp-cpu-onnxruntime-default_config/retinanet/offline | onnxruntime | Ubuntu 24.04 (linux-6.11.0-1018-azure-glibc2.39) | nan | 1 | 0 | v5.0 | 0 | False | Samples/s | fp32 |
| 30 | MLCommons | available | open | edge | gh_macos-latest_x86 | gh_macos-latest_x86-reference-cpu-pytorch_v2.6.0-default_config | retinanet | retinanet | Offline | 0.457477 | mAP: 76.951 | 1 | undefined | 1 | 3 | nan | 0 | open/MLCommons/results/gh_macos-latest_x86-reference-cpu-pytorch_v2.6.0-default_config/retinanet/offline | pytorch v2.6.0 | (darwin-23.6.0) | nan | 1 | 0 | v5.0 | 0 | False | Samples/s | fp32 |
| 31 | MLCommons | available | open | datacenter | gh_action | gh_action-reference-cpu-pytorch_v2.6.0-default_config | mixtral-8x7b | mixtral-8x7b | Offline | 0.0148696 | ROUGE1: 31.4286 ROUGE2: 5.8824 ROUGEL: 28.5714 TOKENS_PER_SAMPLE: 152.0 gsm8k_accuracy: 100.0 mbxp_accuracy: 0.0 | 1 | Intel(R) Xeon(R) w7-2495X | 1 | 24 | nan | 0 | open/MLCommons/results/gh_action-reference-cpu-pytorch_v2.6.0-default_config/mixtral-8x7b/offline | pytorch v2.6.0 | Ubuntu 22.04 (linux-6.8.0-52-generic-glibc2.35) | nan | 1 | 0 | v5.0 | 0 | False | Tokens/s | fp32 |