Skip to content

Commit

Permalink
Update README.md for submission scripts
Browse files Browse the repository at this point in the history
  • Loading branch information
arjunsuresh authored Jul 16, 2023
1 parent e2d8589 commit 3794a72
Showing 1 changed file with 14 additions and 8 deletions.
22 changes: 14 additions & 8 deletions tools/submission/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -27,16 +27,16 @@ Creates an encrypted tarball and generate the SHA1 of the tarball. Currently sub
### Inputs
Takes as input the path of the directory to run the checks on.
### Summary
Checks that a directory containing one or several submission is able to be uploaded to github. This script can be used by running the following command:
Checks that a directory containing one or several submissions is able to be uploaded to github. This script can be used by running the following command:
```
./repository_checks.sh <path-to-folder>
```
### Outputs
Logs in console the errors that could cause problem uploading the submission to github.
Logs in the console the errors that could cause problems uploading the submission to github.

## `submission_checker.py`
### Inputs
**input**: Path to directory containing one or several submissions.<br>
**input**: Path to the directory containing one or several submissions.<br>
**version**: Checker version. E.g v1.1, v2.0, v2.1, v3.0, v3.1. <br>
**submitter**: Filter submitters and only run the checks for some specific submitter. <br>
**csv**: Output path where the csv with the results will be stored. E.g `results/summary.csv`. <br>
Expand All @@ -60,7 +60,6 @@ python3 submission_checker.py --input <path-to-folder>
[--skip_compliance]
[--extra-model-benchmark-map <extra-mapping-string>]
[--submission-exceptions]
[--more-power-check]
```

### Outputs
Expand All @@ -70,11 +69,11 @@ python3 submission_checker.py --input <path-to-folder>
## `truncate_accuracy_log.py`
### Inputs
**input**: Path to directory containing your submission <br>
**output**: Path to directory to output the submission with truncated files <br>
**output**: Path to the directory to output the submission with truncated files <br>
**submitter**: Organization name <br>
**backup**: Path to directory to store an unmodified copy of the truncated files <br>
**backup**: Path to the directory to store an unmodified copy of the truncated files <br>
### Summary
Takes a directory containing a submission and trucates `mlperf_log_accuracy.json` files. There are two ways to use this script. First, we could create a new submission directory with the truncated files by running:
Takes a directory containing a submission and truncates `mlperf_log_accuracy.json` files. There are two ways to use this script. First, we could create a new submission directory with the truncated files by running:
```
python truncate_accuracy_log.py --input <original_submission_directory> --submitter <organization_name> --output <new_submission_directory>
```
Expand All @@ -83,4 +82,11 @@ Second, we could truncate the desired files and place and store a copy of the un
python tools/submission/truncate_accuracy_log.py --input <original_submission_directory> --submitter <organization_name> --backup <safe_directory>
```
### Outputs
Output directory with submission with trucated `mlperf_log_accuracy.json` files
Output directory with submission with truncated `mlperf_log_accuracy.json` files

## `preprocess_submission.py`
### Inputs
**input**: Path to directory containing your submission <br>
**submitter**: Organization name <br>
### Summary
The input submission directory is modified with empty directories removed and low accuracy results inferred and also multistream and offline scenario results wherever possible. The original input directory is saved in a timestamped directory.

0 comments on commit 3794a72

Please sign in to comment.