Updated docs

mlcommons · Jan 29, 2024 · 808ed7a · 808ed7a
1 parent 03d0786
commit 808ed7a
Show file tree

Hide file tree

Showing 10 changed files with 110 additions and 27 deletions.
diff --git a/cm-mlops/script/app-image-classification-onnx-py/README.md b/cm-mlops/script/app-image-classification-onnx-py/README.md
@@ -103,10 +103,14 @@ ___
 
 #### Variations
 
-  * *No group (any variation can be selected)*
+  * Group "**target**"
     <details>
     <summary>Click here to expand this section.</summary>
 
+    * **`_cpu`** (default)
+      - Environment variables:
+        - *USE_CPU*: `True`
+      - Workflow:
     * `_cuda`
       - Environment variables:
         - *USE_CUDA*: `True`
@@ -115,6 +119,10 @@ ___
     </details>
 
 
+#### Default variations
+
+`_cpu`
+
 #### Input description
 
 * --**input** Path to JPEG image to classify

diff --git a/cm-mlops/script/get-conda/README.md b/cm-mlops/script/get-conda/README.md
@@ -116,17 +116,13 @@ ___
       - Environment variables:
         - *CM_CONDA_PYTHON_VERSION*: `3.#`
       - Workflow:
-    * **`_python-3.8`** (default)
+    * `_python-3.8`
       - Environment variables:
         - *CM_CONDA_PYTHON_VERSION*: `3.8`
       - Workflow:
 
     </details>
 
-
-#### Default variations
-
-`_python-3.8`
 #### Default environment
 
 <details>
@@ -149,6 +145,7 @@ ___
   1. ***Run "preprocess" function from [customize.py](https://github.com/mlcommons/ck/tree/master/cm-mlops/script/get-conda/customize.py)***
   1. Read "prehook_deps" on other CM scripts from [meta](https://github.com/mlcommons/ck/tree/master/cm-mlops/script/get-conda/_cm.json)
   1. ***Run native script if exists***
+     * [run.bat](https://github.com/mlcommons/ck/tree/master/cm-mlops/script/get-conda/run.bat)
      * [run.sh](https://github.com/mlcommons/ck/tree/master/cm-mlops/script/get-conda/run.sh)
   1. Read "posthook_deps" on other CM scripts from [meta](https://github.com/mlcommons/ck/tree/master/cm-mlops/script/get-conda/_cm.json)
   1. ***Run "postrocess" function from [customize.py](https://github.com/mlcommons/ck/tree/master/cm-mlops/script/get-conda/customize.py)***

diff --git a/cm-mlops/script/get-dataset-cnndm/README.md b/cm-mlops/script/get-dataset-cnndm/README.md
@@ -96,6 +96,20 @@ ___
 
 #### Variations
 
+  * *No group (any variation can be selected)*
+    <details>
+    <summary>Click here to expand this section.</summary>
+
+    * `_intel`
+      - Workflow:
+    * `_intel,validation`
+      - Environment variables:
+        - *CM_CNNDM_INTEL_VARIATION*: `yes`
+      - Workflow:
+
+    </details>
+
+
   * Group "**dataset-type**"
     <details>
     <summary>Click here to expand this section.</summary>
@@ -139,6 +153,7 @@ ___
        * CM names: `--adr.['python', 'python3']...`
        - CM script: [get-python3](https://github.com/mlcommons/ck/tree/master/cm-mlops/script/get-python3)
      * mlperf,inference,source
+       * `if (CM_CNNDM_INTEL_VARIATION  != yes)`
        * CM names: `--adr.['inference-src']...`
        - CM script: [get-mlperf-inference-src](https://github.com/mlcommons/ck/tree/master/cm-mlops/script/get-mlperf-inference-src)
      * get,generic-python-lib,_package.simplejson
@@ -152,6 +167,7 @@ ___
   1. ***Run "preprocess" function from [customize.py](https://github.com/mlcommons/ck/tree/master/cm-mlops/script/get-dataset-cnndm/customize.py)***
   1. Read "prehook_deps" on other CM scripts from [meta](https://github.com/mlcommons/ck/tree/master/cm-mlops/script/get-dataset-cnndm/_cm.json)
   1. ***Run native script if exists***
+     * [run-intel.sh](https://github.com/mlcommons/ck/tree/master/cm-mlops/script/get-dataset-cnndm/run-intel.sh)
      * [run.sh](https://github.com/mlcommons/ck/tree/master/cm-mlops/script/get-dataset-cnndm/run.sh)
   1. Read "posthook_deps" on other CM scripts from [meta](https://github.com/mlcommons/ck/tree/master/cm-mlops/script/get-dataset-cnndm/_cm.json)
   1. ***Run "postrocess" function from [customize.py](https://github.com/mlcommons/ck/tree/master/cm-mlops/script/get-dataset-cnndm/customize.py)***
@@ -163,14 +179,8 @@ ___
 `cmr "get dataset gpt-j cnndm cnn-dailymail original[,variations]"  -j`
 #### New environment keys (filter)
 
-* `CM_CALIBRATION_DATASET_PATH`
-* `CM_DATASET_EVAL_PATH`
-* `CM_DATASET_PATH`
 #### New environment keys auto-detected from customize
 
-* `CM_CALIBRATION_DATASET_PATH`
-* `CM_DATASET_EVAL_PATH`
-* `CM_DATASET_PATH`
 ___
 ### Maintainers
 

diff --git a/cm-mlops/script/get-dataset-coco/README.md b/cm-mlops/script/get-dataset-coco/README.md
@@ -155,6 +155,7 @@ ___
 <summary>Click here to expand this section.</summary>
 
 * `--from=value`  &rarr;  `CM_FROM=value`
+* `--home=value`  &rarr;  `CM_HOME_DIR=value`
 * `--store=value`  &rarr;  `CM_STORE=value`
 * `--to=value`  &rarr;  `CM_TO=value`
 

diff --git a/cm-mlops/script/get-generic-python-lib/README.md b/cm-mlops/script/get-generic-python-lib/README.md
@@ -148,6 +148,8 @@ ___
         - *CM_GENERIC_PYTHON_PACKAGE_NAME*: `colored`
         - *CM_GENERIC_PYTHON_PIP_EXTRA_INDEX_URL*: `https://pypi.ngc.nvidia.com`
       - Workflow:
+    * `_conda.#`
+      - Workflow:
     * `_cupy`
       - Environment variables:
         - *CM_GENERIC_PYTHON_PACKAGE_NAME*: `cupy`
@@ -641,7 +643,7 @@ ___
   1. ***Read "prehook_deps" on other CM scripts from [meta](https://github.com/mlcommons/ck/tree/master/cm-mlops/script/get-generic-python-lib/_cm.json)***
      * install,onnxruntime,from.src,_cuda
        * `if (CM_INSTALL_ONNXRUNTIME_GPU_FROM_SRC  == yes)`
-       - CM script: [install-onnxruntime-from.src](https://github.com/mlcommons/ck/tree/master/cm-mlops/script/install-onnxruntime-from.src)
+       - CM script: [install-onnxruntime-from-src](https://github.com/mlcommons/ck/tree/master/cm-mlops/script/install-onnxruntime-from-src)
   1. ***Run native script if exists***
      * [run.bat](https://github.com/mlcommons/ck/tree/master/cm-mlops/script/get-generic-python-lib/run.bat)
      * [run.sh](https://github.com/mlcommons/ck/tree/master/cm-mlops/script/get-generic-python-lib/run.sh)

diff --git a/cm-mlops/script/get-ml-model-gptj/README.md b/cm-mlops/script/get-ml-model-gptj/README.md
@@ -115,6 +115,33 @@ ___
         - *CM_DOWNLOAD_CHECKSUM_NOT_USED*: `e677e28aaf03da84584bb3073b7ee315`
         - *CM_PACKAGE_URL*: `https://cloud.mlcommons.org/index.php/s/QAZ2oM94MkFtbQx/download`
       - Workflow:
+    * `_pytorch,int4,intel`
+      - Workflow:
+    * `_pytorch,int8,intel`
+      - Workflow:
+    * `_pytorch,intel`
+      - Environment variables:
+        - *CM_GPTJ_INTEL_MODEL*: `yes`
+      - Workflow:
+        1. ***Read "deps" on other CM scripts***
+           * get,mlperf,inference,results
+             - CM script: [get-mlperf-inference-results](https://github.com/mlcommons/ck/tree/master/cm-mlops/script/get-mlperf-inference-results)
+           * get,ml-model,gpt-j,_fp32,_pytorch
+             - CM script: [get-ml-model-gptj](https://github.com/mlcommons/ck/tree/master/cm-mlops/script/get-ml-model-gptj)
+           * get,conda,_name.gptj-pt
+             - CM script: [get-conda](https://github.com/mlcommons/ck/tree/master/cm-mlops/script/get-conda)
+           * get,python,_conda.gptj-pt
+             - CM script: [get-python3](https://github.com/mlcommons/ck/tree/master/cm-mlops/script/get-python3)
+           * get,generic,conda-package,_package.intel-openmp,_source.intel
+             * CM names: `--adr.['conda-package', 'intel-openmp']...`
+             - CM script: [install-generic-conda-package](https://github.com/mlcommons/ck/tree/master/cm-mlops/script/install-generic-conda-package)
+           * get,generic,conda-package,_package.jemalloc,_source.conda-forge
+             * CM names: `--adr.['conda-package', 'jemalloc']...`
+             - CM script: [install-generic-conda-package](https://github.com/mlcommons/ck/tree/master/cm-mlops/script/install-generic-conda-package)
+           * install,ipex,from.src,_for-intel-mlperf-inference-v3.1-gptj
+             - CM script: [install-ipex-from-src](https://github.com/mlcommons/ck/tree/master/cm-mlops/script/install-ipex-from-src)
+           * get,dataset,cnndm,_calibration
+             - CM script: [get-dataset-cnndm](https://github.com/mlcommons/ck/tree/master/cm-mlops/script/get-dataset-cnndm)
 
     </details>
 
@@ -133,16 +160,33 @@ ___
     </details>
 
 
+  * Group "**model-provider**"
+    <details>
+    <summary>Click here to expand this section.</summary>
+
+    * `_intel`
+      - Workflow:
+    * **`_mlcommons`** (default)
+      - Workflow:
+
+    </details>
+
+
   * Group "**precision**"
     <details>
     <summary>Click here to expand this section.</summary>
 
-    * **`_fp32`** (default)
+    * `_fp32`
       - Environment variables:
         - *CM_ML_MODEL_INPUT_DATA_TYPES*: `fp32`
         - *CM_ML_MODEL_PRECISION*: `fp32`
         - *CM_ML_MODEL_WEIGHT_DATA_TYPES*: `fp32`
       - Workflow:
+    * `_int4`
+      - Environment variables:
+        - *CM_ML_MODEL_INPUT_DATA_TYPES*: `int4`
+        - *CM_ML_MODEL_WEIGHT_DATA_TYPES*: `int4`
+      - Workflow:
     * `_int8`
       - Environment variables:
         - *CM_ML_MODEL_INPUT_DATA_TYPES*: `int8`
@@ -161,7 +205,7 @@ ___
 
 #### Default variations
 
-`_fp32,_pytorch`
+`_mlcommons,_pytorch`
 
 #### Script flags mapped to environment
 <details>
@@ -201,6 +245,8 @@ ___
        * CM names: `--adr.['dae']...`
        - CM script: [download-and-extract](https://github.com/mlcommons/ck/tree/master/cm-mlops/script/download-and-extract)
   1. ***Run native script if exists***
+     * [run-int4-calibration.sh](https://github.com/mlcommons/ck/tree/master/cm-mlops/script/get-ml-model-gptj/run-int4-calibration.sh)
+     * [run-intel.sh](https://github.com/mlcommons/ck/tree/master/cm-mlops/script/get-ml-model-gptj/run-intel.sh)
   1. Read "posthook_deps" on other CM scripts from [meta](https://github.com/mlcommons/ck/tree/master/cm-mlops/script/get-ml-model-gptj/_cm.json)
   1. ***Run "postrocess" function from [customize.py](https://github.com/mlcommons/ck/tree/master/cm-mlops/script/get-ml-model-gptj/customize.py)***
   1. Read "post_deps" on other CM scripts from [meta](https://github.com/mlcommons/ck/tree/master/cm-mlops/script/get-ml-model-gptj/_cm.json)
@@ -217,6 +263,7 @@ ___
 
 * `CM_ML_MODEL_FILE`
 * `CM_ML_MODEL_FILE_WITH_PATH`
+* `CM_ML_MODEL_WEIGHT_DATA_TYPES`
 ___
 ### Maintainers
 

diff --git a/cm-mlops/script/get-python3/README.md b/cm-mlops/script/get-python3/README.md
@@ -103,6 +103,18 @@ ___
     <details>
     <summary>Click here to expand this section.</summary>
 
+    * `_conda.#`
+      - Environment variables:
+        - *CM_PYTHON_CONDA*: `yes`
+      - Workflow:
+        1. ***Read "deps" on other CM scripts***
+           * get,generic,conda-package,_name.#,_package.python
+             * CM names: `--adr.['conda-package', 'conda-python']...`
+             - CM script: [install-generic-conda-package](https://github.com/mlcommons/ck/tree/master/cm-mlops/script/install-generic-conda-package)
+    * `_custom-path.#`
+      - Environment variables:
+        - *CM_PYTHON_BIN_WITH_PATH*: `#`
+      - Workflow:
     * `_lto`
       - Workflow:
     * `_optimized`
@@ -159,6 +171,7 @@ ___
 
 * `CM_PYTHON_BIN`
 * `CM_PYTHON_BIN_PATH`
+* `CM_PYTHON_BIN_WITH_PATH`
 * `CM_PYTHON_CACHE_TAGS`
 * `CM_PYTHON_MAJOR_VERSION`
 * `CM_PYTHON_MINOR_VERSION`

diff --git a/cm-mlops/script/install-llvm-prebuilt/README.md b/cm-mlops/script/install-llvm-prebuilt/README.md
@@ -107,7 +107,7 @@ These keys can be updated via `--env.KEY=VALUE` or `env` dictionary in `@input.j
 </details>
 
 #### Versions
-Default version: `14.0.0`
+Default version: `15.0.6`
 
 ___
 ### Script workflow, dependencies and native scripts

diff --git a/cm-mlops/script/print-hello-world-py/README.md b/cm-mlops/script/print-hello-world-py/README.md
@@ -115,6 +115,7 @@ ___
        * CM names: `--adr.['python', 'python3']...`
        - CM script: [get-python3](https://github.com/mlcommons/ck/tree/master/cm-mlops/script/get-python3)
      * print,python-version
+       * `if (CM_SKIP_PRINT  != True OR CM_SKIP_PRINT2  != True)`
        - CM script: [print-python-version](https://github.com/mlcommons/ck/tree/master/cm-mlops/script/print-python-version)
   1. Run "preprocess" function from customize.py
   1. Read "prehook_deps" on other CM scripts from [meta](https://github.com/mlcommons/ck/tree/master/cm-mlops/script/print-hello-world-py/_cm.json)

diff --git a/cm-mlops/script/reproduce-mlperf-inference-intel/README.md b/cm-mlops/script/reproduce-mlperf-inference-intel/README.md
@@ -168,11 +168,11 @@ ___
              * CM names: `--adr.['conda-package', 'jemalloc']...`
              - CM script: [install-generic-conda-package](https://github.com/mlcommons/ck/tree/master/cm-mlops/script/install-generic-conda-package)
            * get,pytorch,from.src,_for-intel-mlperf-inference-v3.1-bert
-             - CM script: [install-pytorch-from.src](https://github.com/mlcommons/ck/tree/master/cm-mlops/script/install-pytorch-from.src)
+             - CM script: [install-pytorch-from-src](https://github.com/mlcommons/ck/tree/master/cm-mlops/script/install-pytorch-from-src)
            * install,onednn,from.src,_for-intel-mlperf-inference-v3.1-bert
-             - CM script: [install-onednn-from.src](https://github.com/mlcommons/ck/tree/master/cm-mlops/script/install-onednn-from.src)
+             - CM script: [install-onednn-from-src](https://github.com/mlcommons/ck/tree/master/cm-mlops/script/install-onednn-from-src)
            * install,transformers,from.src,_for-intel-mlperf-inference-v3.1-bert
-             - CM script: [install-transformers-from.src](https://github.com/mlcommons/ck/tree/master/cm-mlops/script/install-transformers-from.src)
+             - CM script: [install-transformers-from-src](https://github.com/mlcommons/ck/tree/master/cm-mlops/script/install-transformers-from-src)
     * `_bs.#`
       - Environment variables:
         - *ML_MLPERF_MODEL_BATCH_SIZE*: `#`
@@ -182,9 +182,8 @@ ___
         1. ***Read "deps" on other CM scripts***
            * get,conda,_name.gptj-pt
              - CM script: [get-conda](https://github.com/mlcommons/ck/tree/master/cm-mlops/script/get-conda)
-           * get,generic,conda-package,_package.python
-             * CM names: `--adr.['conda-package', 'python']...`
-             - CM script: [install-generic-conda-package](https://github.com/mlcommons/ck/tree/master/cm-mlops/script/install-generic-conda-package)
+           * get,python,_conda.gptj-pt
+             - CM script: [get-python3](https://github.com/mlcommons/ck/tree/master/cm-mlops/script/get-python3)
            * install,llvm,src,_tag.llvmorg-16.0.6,_clang,_release,_for-intel-mlperf-inference-v3.1-gptj
              - CM script: [install-llvm-src](https://github.com/mlcommons/ck/tree/master/cm-mlops/script/install-llvm-src)
            * get,generic,conda-package,_package.ncurses,_source.conda-forge
@@ -196,8 +195,9 @@ ___
              * CM names: `--adr.['conda-package', 'jemalloc']...`
              - CM script: [install-generic-conda-package](https://github.com/mlcommons/ck/tree/master/cm-mlops/script/install-generic-conda-package)
            * install,ipex,from.src,_for-intel-mlperf-inference-v3.1-gptj
-             - CM script: [install-ipex-from.src](https://github.com/mlcommons/ck/tree/master/cm-mlops/script/install-ipex-from.src)
-           * get,generic-python-lib,_custom-python,_package.transformers
+             - CM script: [install-ipex-from-src](https://github.com/mlcommons/ck/tree/master/cm-mlops/script/install-ipex-from-src)
+           * get,generic-python-lib,_package.transformers
+             * CM names: `--adr.['pip-package']...`
              - CM script: [get-generic-python-lib](https://github.com/mlcommons/ck/tree/master/cm-mlops/script/get-generic-python-lib)
            * get,mlcommons,inference,src
              * CM names: `--adr.['inference-src']...`
@@ -208,9 +208,11 @@ ___
            * get,ml-model,large-language-model,gptj
              * CM names: `--adr.['ml-model', 'gptj-model', 'gpt-j-model']...`
              - CM script: [get-ml-model-gptj](https://github.com/mlcommons/ck/tree/master/cm-mlops/script/get-ml-model-gptj)
-           * get,generic-python-lib,_custom-python,_package.datasets
+           * get,generic-python-lib,_package.datasets
+             * CM names: `--adr.['pip-package']...`
              - CM script: [get-generic-python-lib](https://github.com/mlcommons/ck/tree/master/cm-mlops/script/get-generic-python-lib)
-           * get,generic-python-lib,_custom-python,_package.accelerate
+           * get,generic-python-lib,_package.accelerate
+             * CM names: `--adr.['pip-package']...`
              - CM script: [get-generic-python-lib](https://github.com/mlcommons/ck/tree/master/cm-mlops/script/get-generic-python-lib)
     * `_loadgen-batch-size.#`
       - Environment variables:
@@ -502,11 +504,13 @@ ___
        * CM names: `--adr.['openimages-preprocessed', 'dataset-preprocessed']...`
        - CM script: [get-preprocessed-dataset-openimages](https://github.com/mlcommons/ck/tree/master/cm-mlops/script/get-preprocessed-dataset-openimages)
      * get,mlperf,inference,results
+       * CM names: `--adr.inference-results...`
        - CM script: [get-mlperf-inference-results](https://github.com/mlcommons/ck/tree/master/cm-mlops/script/get-mlperf-inference-results)
   1. ***Run "preprocess" function from [customize.py](https://github.com/mlcommons/ck/tree/master/cm-mlops/script/reproduce-mlperf-inference-intel/customize.py)***
   1. Read "prehook_deps" on other CM scripts from [meta](https://github.com/mlcommons/ck/tree/master/cm-mlops/script/reproduce-mlperf-inference-intel/_cm.yaml)
   1. ***Run native script if exists***
-     * [run_harness.sh](https://github.com/mlcommons/ck/tree/master/cm-mlops/script/reproduce-mlperf-inference-intel/run_harness.sh)
+     * [run_bert_harness.sh](https://github.com/mlcommons/ck/tree/master/cm-mlops/script/reproduce-mlperf-inference-intel/run_bert_harness.sh)
+     * [run_gptj_harness.sh](https://github.com/mlcommons/ck/tree/master/cm-mlops/script/reproduce-mlperf-inference-intel/run_gptj_harness.sh)
   1. Read "posthook_deps" on other CM scripts from [meta](https://github.com/mlcommons/ck/tree/master/cm-mlops/script/reproduce-mlperf-inference-intel/_cm.yaml)
   1. ***Run "postrocess" function from [customize.py](https://github.com/mlcommons/ck/tree/master/cm-mlops/script/reproduce-mlperf-inference-intel/customize.py)***
   1. ***Read "post_deps" on other CM scripts from [meta](https://github.com/mlcommons/ck/tree/master/cm-mlops/script/reproduce-mlperf-inference-intel/_cm.yaml)***