Added cm set cfg automation

CHANGES.md

@@ -1,3 +1,10 @@
+### 20240429
+ * Added `cm set cfg` automation. For example, we can set default CM script to silent
+   using `cm set cfg default --key.script.silent`
+
+### 20240427
+ * Added cm run script --print_versions to print resolved versions of dependencies at the end of the run
+
 ### 20240426
  * Improved cm run script --repro function to dump dependencies, versions and generate README
 

CONTRIBUTING.md

@@ -1,77 +1,2 @@
-# Collective Mind project aka CM
-
-## Contributing
-
-The best way to contribute to the MLCommons is to get involved with one of our many project communities. You find more information about getting involved with MLCommons [here](https://mlcommons.org/en/get-involved/#getting-started). 
-
-Generally we encourage people to become a MLCommons member if they wish to contribute to MLCommons projects, but outside pull requests are very welcome too.
-
-Regardless of if you are a member, your organization needs to sign the MLCommons CLA. Please fill out this [CLA sign up form](https://forms.gle/Ew1KkBVpyeJDuRw67) form to get started.
-
-MLCommons project work is tracked with issue trackers and pull requests. Modify the project in your own fork and issue a pull request once you want other developers to take a look at what you have done and discuss the proposed changes. Ensure that cla-bot and other checks pass for your Pull requests.
-
-## CM project coordinator
-
-* [Grigori Fursin](https://cKnowledge.org/gfursin)
-
-## CM authors
-
-* [Grigori Fursin](https://cKnowledge.org/gfursin) (CM core and CM scripts for MLOps)
-* [Arjun Suresh](https://www.linkedin.com/in/arjunsuresh) (CM scripts for MLPerf and MLOps)
-
-## CM contributors in alphabetical order (suggestions and/or scripts)
-
-* Resmi Arjun
-* Alice Cheng (Nvidia)
-* Jiahao Chen (MIT)
-* Ramesh N Chukka (Intel)
-* Ray DeMoss (One Stop Systems)
-* Ryan T DeRue (Purdue University)
-* Himanshu Dutta (Indian Institute of Technology)
-* Justin Faust (One Stop Systems)
-* Diane Feddema (Red Hat)
-* Leonid Fursin (United Silicon Carbide)
-* Anirban Ghosh (Nvidia)
-* Michael Goin (Neural Magic)
-* Jose Armando Hernandez (Paris Saclay University)
-* Mehrdad Hessar (OctoML)
-* Miro Hodak (AMD)
-* Sachin Idgunji (Nvidia)
-* Tom Jablin (Google)
-* Nino Jacob
-* David Kanter (MLCommons)
-* Jason Knight (OctoML)
-* Ilya Kozulin (Deelvin)
-* @makaveli10 (Collabora)
-* Steve Leak(NERSC) 
-* Amija Maji (Purdue University)
-* Peter Mattson (Google, MLCommons)
-* Kasper Mecklenburg (Arm)
-* Pablo Gonzalez Mesa
-* Thierry Moreau (OctoML)
-* Sachin Mudaliyar
-* Stanley Mwangi (Microsoft)
-* Ashwin Nanjappa (Nvidia)
-* Hai Ah Nam (NERSC)
-* Nandeeka Nayak (UIUC)
-* Datta Nimmaturi (Nutanix)
-* Lakshman Patel
-* Arun Tejusve Raghunath Rajan (Cruise)
-* Vijay Janapa Reddi (Harvard University)
-* Andrew Reusch (OctoML)
-* Anandhu S (Kerala Technical University)
-* Sergey Serebryakov (HPE)
-* Warren Schultz (Principled Technologies)
-* Amrutha Sheleenderan (Kerala Technical University)
-* Byoungjun Seo (TTA)
-* Aditya Kumar Shaw (Indian Institute of Science)
-* Ilya Slavutin (Deelvin)
-* Jinho Suh (Nvidia)
-* Badhri Narayanan Suresh (Intel)
-* David Tafur (MLCommons)
-* Chloe Tessier
-* Gaurav Verma (Stony Brook University)
-* Scott Wasson (MLCommons)
-* Haoyang Zhang  (UIUC)
-* Bojian Zheng (University of Toronto)
-* Thomas Zhu (Oxford University)
+You can find the guidelines to contribute to the MLCommons CM project 
+at the [CM GitHub page](https://github.com/mlcommons/ck/blob/master/CONTRIBUTING.md).

COPYRIGHT.txt

@@ -0,0 +1,3 @@
+Copyright (c) 2021-2024 MLCommons
+
+The cTuning foundation donated this project to MLCommons in 2021 to benefit everyone.

README.md

@@ -38,3 +38,14 @@ See the automatically generated catalog [online](https://access.cknowledge.org/p
 ## Copyright
 
 2022-2024 [MLCommons](https://mlcommons.org)
+
+## Acknowledgments
+
+This open-source technology is being developed by the [MLCommons Task Force on Automation and Reproducibility](https://github.com/mlcommons/ck/blob/master/docs/taskforce.md)
+as a community effort based on user feedback. 
+
+We would like to thank all [volunteers, collaborators and contributors](CONTRIBUTING.md) 
+for their support, fruitful discussions, and useful feedback! 
+
+We thank the [cTuning foundation](https://cTuning.org), [cKnowledge.org](https://cKnowledge.org)
+and [MLCommons](https://mlcommons.org) for sponsoring this project!

automation/cfg/README-extra.md

@@ -0,0 +1,8 @@
+Examples:
+
+```bash
+cm set cfg default 
+cm set cfg default --key.script.silent
+cm set cfg default --key.script.silent-
+
+```

automation/cfg/_cm.json

@@ -1,4 +1,7 @@
 {
+  "action_substitutions": {
+    "set":"xset"
+  },
   "alias": "cfg",
   "automation_alias": "automation",
   "automation_uid": "bbeb15d8f0a944a4",

automation/cfg/module.py

@@ -1,3 +1,7 @@
+# Universal cfg for CM automations
+#
+# Written by Grigori Fursin
+
 import os
 
 from cmind.automation import Automation
@@ -50,3 +54,188 @@ def test(self, i):
         print (json.dumps(i, indent=2))
 
         return {'return':0}
+
+    ############################################################
+    def xset(self, i):
+        """
+        Set keys in configuration
+
+        Args:
+          (CM input dict): 
+
+            (out) (str): if 'con', output to console
+
+            (artifact) (str): CM artifact with configuration
+            (tags) (str): list of tags to find CM artifact with configuration
+
+            (key) (dict): updating config
+          ...
+
+        Returns:
+          (CM return dict):
+
+          * return (int): return code == 0 if no error and >0 if error
+          * (error) (str): error string if return>0
+
+          * Output from this automation action
+
+        """
+
+        import json
+
+        r = self._find_cfg_artifact(i)
+        if r['return']>0: return r
+
+        # Path to cfg
+        path = r['path']
+        path_to_config = r['path_to_config']
+        config = r['config']
+
+        # Clean input to leave only keys for the configuration
+        new_config = i.get('key',{})
+
+        # If new config is empty, just print existing config
+        if len(new_config) >0 :
+            # Check if need to delete some
+            def check_to_delete(d):
+
+                for k in list(d.keys()):
+                    v = d[k]
+                    if isinstance(v, dict):
+                        check_to_delete(v)
+                    else:
+                        if k.endswith('-'):
+                           if k[:-1] in d:
+                               del(d[k[:-1]])
+                           del(d[k])
+                        else:
+                           vsl = str(v).lower()
+                           if vsl == 'none': v = None
+                           elif vsl == 'false': v = False
+                           elif vsl == 'true': v = True
+
+                           d[k]=v
+
+            utils.merge_dicts({'dict1':config, 'dict2':new_config, 'append_lists':True, 'append_unique':True})
+
+            check_to_delete(config)
+
+            r = utils.save_json(path_to_config, config)
+            if r['return']>0: return r
+
+        # Print config
+        print ('Config:')
+        print ('')
+        print (json.dumps(config, indent=2))
+
+        return {'return':0}
+
+    ############################################################
+    def load(self, i):
+        """
+        Load configuration
+
+        Args:
+          (CM input dict): 
+
+            (out) (str): if 'con', output to console
+
+            (artifact) (str): CM artifact with configuration
+            (tags) (str): list of tags to find CM artifact with configuration
+          ...
+
+        Returns:
+          (CM return dict):
+
+          * return (int): return code == 0 if no error and >0 if error
+          * (error) (str): error string if return>0
+
+          * Output from this automation action
+
+        """
+
+        return self._find_cfg_artifact(i)
+
+    ############################################################
+    def _find_cfg_artifact(self, i):
+        """
+        Args:
+          (CM input dict): 
+
+            (out) (str): if 'con', output to console
+
+            (artifact) (str): CM artifact with configuration
+            (tags) (str): list of tags to find CM artifact with configuration
+          ...
+
+        Returns:
+          (CM return dict):
+
+          * return (int): return code == 0 if no error and >0 if error
+          * (error) (str): error string if return>0
+
+          * Output from this automation action
+
+        """
+
+        # Clean input to find artifact
+        ii = utils.sub_input(i, self.cmind.cfg['artifact_keys']+['tags'])
+
+        parsed_artifact = i.get('parsed_artifact',[])
+
+        artifact_obj = parsed_artifact[0] if len(parsed_artifact)>0 else None
+        artifact_repo = parsed_artifact[1] if len(parsed_artifact)>1 else None
+
+        artifact = i.get('artifact', '')
+
+        if artifact == '': 
+            ii['artifact'] = 'default'
+
+        tags = ii.get('tags', '')
+
+        if 'cm-universal-cfg' not in tags:
+            if tags!='': tags+=','
+            tags+='cm-universal-cfg'
+
+        ii['tags'] = tags
+
+        automation = ii['automation']
+        if automation!='.' and ',' not in automation: 
+            ii['automation'] = automation + ',' + self.meta['uid']
+
+        # Add placeholder (use common action)
+
+        ii['action']='find'
+        ii['out']=''
+        ii['common']=True # Avoid recursion - use internal CM add function to add the script artifact
+
+        r=self.cmind.access(ii)
+        if r['return']>0: return r
+
+        lst = r['list']
+
+        if len(lst) == 0:
+            ii['action'] = 'add'
+            ii['meta'] = {}
+
+            # Tags must be unique for default
+            r=self.cmind.access(ii)
+            if r['return']>0: return r
+
+            path = r['path']
+        elif len(lst)>1:
+            return {'return':1, 'error':'ambiguity in cfg name - more than 1 CM artifact found'}
+        else:
+            path = lst[0].path
+
+        # Check if has config
+        path_to_cfg = os.path.join(path, 'config.json')
+
+        config = {}
+        if os.path.isfile(path_to_cfg):
+            r = utils.load_json(path_to_cfg)
+            if r['return']>0: return r
+
+            config = r['meta']
+
+        return {'return':0, 'path':path, 'path_to_config':path_to_cfg, 'config':config}

automation/experiment/README-extra.md

@@ -46,12 +46,12 @@ and improve them by the community via [public benchmarking, optimization and rep
 
 ## Installing CM with ResearchOps/DevOps/MLOps automations
 
-This CM automation is available in the most commonly used `mlcommons@ck` repository. 
+This CM automation is available in the most commonly used `mlcommons@cm4mlops` repository. 
 
 First, install CM automation language as described [here](https://github.com/mlcommons/ck/blob/master/docs/installation.md).
 Then, install or update this repository as follows:
 ```bash
-cm pull repo mlcommons@ck
+cm pull repo mlcommons@cm4mlops
 ```
 
 You can now test that CM experiment automation is available as follows:

automation/experiment/module.py

@@ -1,3 +1,8 @@
+# Universal experiment automation to support universal benchmarking 
+# and optimization of apps and systems
+#
+# Written by Grigori Fursin
+
 import os
 import itertools
 import copy

automation/script/README-extra.md

@@ -118,6 +118,16 @@ as well as CM debug info as follows:
 cmr "detect os" -j -v
 ```
 
+You can turn on silent mode using CM cfg automation:
+```bash
+cm set cfg --key.script.silent
+```
+or
+```bash
+cm set cfg default --key.script.silent
+```
+
+
 ## Understanding CM scripts
 
 CM scripts are treated as standard CM artifacts with the associated CM automation ["script"](https://github.com/mlcommons/ck/tree/master/cm-mlops/automation/script),

automation/script/_cm.json

@@ -9,6 +9,7 @@
   "desc": "Making native scripts more portable, interoperable and deterministic",
   "developers": "Arjun Suresh and Grigori Fursin",
   "actions_with_help":["run", "docker"],
+  "prototype": true,
   "sort": 1000,
   "tags": [
     "automation"