Skip to content

Commit

Permalink
Fix NULL accuracy for open division results (#1481)
Browse files Browse the repository at this point in the history
* Fix bug in submission preprocessor script for extra_model_mapping

* Fixes to submission generation for gptj

* Update mlperf.conf

* Update README.md for submission scripts

* Fix submission checker for TEST01 baseline check

* Fix loadgen version to 3.1

* Fix accuracy check for open division
  • Loading branch information
arjunsuresh authored Aug 15, 2023
1 parent 728680b commit 127b761
Showing 1 changed file with 2 additions and 2 deletions.
4 changes: 2 additions & 2 deletions tools/submission/submission_checker.py
Original file line number Diff line number Diff line change
Expand Up @@ -1592,11 +1592,11 @@ def check_accuracy_dir(config, model, path, verbose):
if acc is not None and float(acc) >= acc_target:
all_accuracy_valid &= True
acc_seen[i] = True
if i == 0:
result_acc = acc
elif acc is not None:
all_accuracy_valid = False
log.warning("%s accuracy not met: expected=%f, found=%s", path, acc_target, acc)
if i == 0 and acc:
result_acc = acc
acc = None
if all(acc_seen) and hash_val:
break;
Expand Down

0 comments on commit 127b761

Please sign in to comment.