CM/CMX v3.4.0: added pyproject.toml to install CM and deprecated setu…

…p.py (#1349) ## V3.4.0 - added pyproject.toml to install CM and deprecated setup.py: #1345
mlcommons · Nov 10, 2024 · 584ccb6 · 584ccb6
2 parents 212d825 + d0de459
commit 584ccb6
Show file tree

Hide file tree

Showing 14 changed files with 137 additions and 31 deletions.
diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md
@@ -14,11 +14,13 @@ Modify the project in your own fork and issue a pull request once you want other
 to take a look at what you have done and discuss the proposed changes. 
 Ensure that cla-bot and other checks pass for your Pull requests.
 
-Collective Knowledge (CK) and Collective Mind (CM) were created by [Grigori Fursin](https://cKnowledge.org/gfursin),
-sponsored by cKnowledge.org and cTuning.org, and donated to MLCommons to benefit everyone. 
-Since then, this open-source technology (CM, CM4MLOps, CM4ABTF, CM4Research, etc)
-is being developed as a community effort thanks to all our volunteers, collaborators and contributors
-listed here in alphabetical order:
+Collective Knowledge (CK) and Collective Mind (CM)
+were created by [Grigori Fursin](https://arxiv.org/abs/2406.16791),
+sponsored by cKnowledge.org and cTuning.org, and donated to MLCommons 
+to benefit everyone. Since then, this open-source automation technology
+(CM/CMX, CM4MLOps, CM4ABTF, CM4Research, etc) is being extended 
+as a community effort thanks to all our volunteers, collaborators 
+and contributors listed here in alphabetical order:
 
 * Resmi Arjun
 * Omar Benjelloun (Google)
@@ -62,7 +64,7 @@ listed here in alphabetical order:
 * Arun Tejusve Raghunath Rajan (Cruise)
 * Vijay Janapa Reddi (Harvard University)
 * Andrew Reusch (OctoML)
-* Anandhu S (Kerala Technical University)
+* Anandhu Sooraj (Kerala Technical University)
 * Sergey Serebryakov (HPE)
 * Warren Schultz (Principled Technologies)
 * Amrutha Sheleenderan (Kerala Technical University)

diff --git a/README.md b/README.md
@@ -35,8 +35,9 @@ CK consists of several sub-projects:
     to run automotive benchmark across different models, data sets, software and hardware from different vendors.
 
 * [CMX (the next generation of CM)](cm/docs/cmx) - we are developing the next generation of CM 
-  to make it simpler and more flexible based on user feedback. Please follow 
-  this project [here](https://github.com/orgs/mlcommons/projects/46).
+    to make it simpler and more flexible based on user feedback. Please follow 
+    this project [here]( https://github.com/orgs/mlcommons/projects/46 ).
+
 
 * [Collective Knowledge Playground](https://access.cKnowledge.org) - a unified platform
   to list CM scripts similar to PYPI, aggregate AI/ML Systems benchmarking results in a reproducible format with CM workflows, 
@@ -56,6 +57,11 @@ CK consists of several sub-projects:
 * Copyright (c) 2021-2024 MLCommons
 * Copyright (c) 2014-2021 cTuning foundation
 
+### Maintainers
+
+* CM/CMX/CM4Research: [Grigori Fursin](https://cKnowledge.org/gfursin)
+* CM4MLOps: [Arjun Suresh](https://github.com/arjunsuresh) and [Anandhu Sooraj](https://github.com/anandhu-eng)
+
 ### Motivation and long-term vision
 
 You can learn more about the motivation behind these projects from the following articles and presentations:
@@ -79,8 +85,8 @@ You can learn more about the motivation behind these projects from the following
 
 ### Acknowledgments
 
-Collective Knowledge (CK) and Collective Mind (CM) were created by [Grigori Fursin](https://cKnowledge.org/gfursin),
+The Collective Mind automation framework (CM) was created by [Grigori Fursin](https://cKnowledge.org/gfursin),
 sponsored by cKnowledge.org and cTuning.org, and donated to MLCommons to benefit everyone. 
-Since then, this open-source technology (CM, CM4MLOps, CM4MLPerf, CM4ABTF, CM4Research, etc)
-is being developed as a community effort thanks to all our
+This open-source technology (CM, CM4MLOps, CM4MLPerf, CM4ABTF, CM4Research, etc)
+is being developed as a community effort thanks to all our fantastic
 [volunteers, collaborators and contributors](https://github.com/mlcommons/ck/blob/master/CONTRIBUTING.md)! 
diff --git a/cm/CHANGES.md b/cm/CHANGES.md
@@ -1,3 +1,9 @@
+## V3.4.1
+   - reduced Python min version in pyproject.toml to 3.7 for backwards compatibility
+
+## V3.4.0
+   - added pyproject.toml to install CM and deprecated setup.py
+
 ## V3.3.4
    - added utils.path2: 
      add quotes if spaces in path

diff --git a/cm/README.md b/cm/README.md
@@ -44,12 +44,16 @@ from Nvidia, Intel, AMD, Google, Qualcomm, Amazon and other vendors:
   and simple JSON/YAML descriptions instead of inventing new workflow languages;
 * must have the same interface to run all automations natively, in a cloud or inside containers.
 
+### Maintainers
+
+* [Grigori Fursin](https://cKnowledge.org/gfursin)
+
 ### Resources
 
-* CM v2.x (stable version 2022-cur): [installation on Linux, Windows, MacOS](https://access.cknowledge.org/playground/?action=install) ; 
+* CM v2.x (2022-cur) (stable): [installation on Linux, Windows, MacOS](https://access.cknowledge.org/playground/?action=install) ; 
   [docs](https://docs.mlcommons.org/ck) ; [popular commands](https://github.com/mlcommons/ck/tree/master/cm/docs/demos/some-cm-commands.md) ; 
   [getting started guide](https://github.com/mlcommons/ck/blob/master/docs/getting-started.md)
-* CM v3.x (prototype 2024-cur): [docs](https://github.com/mlcommons/ck/tree/master/cm/docs/cmx)
+* CM v3.x aka CMX (2024-cur) (stable): [docs](https://github.com/orgs/mlcommons/projects/46)
 * MLPerf inference benchmark automated via CM
   * [Run MLPerf for submissions](https://docs.mlcommons.org/inference)
   * [Run MLPerf at the Student Cluster Competition'24](https://docs.mlcommons.org/inference/benchmarks/text_to_image/reproducibility/scc24)
@@ -61,21 +65,21 @@ from Nvidia, Intel, AMD, Google, Qualcomm, Amazon and other vendors:
 
 [Apache 2.0](LICENSE.md)
 
-### Citing CM and CM4MLOps
+### Citing CM project
 
-If you found CM useful, please cite this article: 
+If you found CM automations useful, please cite this article: 
 [ [ArXiv](https://arxiv.org/abs/2406.16791) ], [ [BibTex](https://github.com/mlcommons/ck/blob/master/citation.bib) ].
 
-You can learn more about the motivation behind these projects from the following articles and presentations:
+You can learn more about the motivation behind these projects from the following presentations:
 
-* "Enabling more efficient and cost-effective AI/ML systems with Collective Mind, virtualized MLOps, MLPerf, Collective Knowledge Playground and reproducible optimization tournaments": [ [ArXiv](https://arxiv.org/abs/2406.16791) ] 
+* "Enabling more efficient and cost-effective AI/ML systems with Collective Mind, virtualized MLOps, MLPerf, Collective Knowledge Playground and reproducible optimization tournaments": [ [ArXiv](https://arxiv.org/abs/2406.16791) ]
 * ACM REP'23 keynote about the MLCommons CM automation framework: [ [slides](https://doi.org/10.5281/zenodo.8105339) ] 
 * ACM TechTalk'21 about Collective Knowledge project: [ [YouTube](https://www.youtube.com/watch?v=7zpeIVwICa4) ] [ [slides](https://learning.acm.org/binaries/content/assets/leaning-center/webinar-slides/2021/grigorifursin_techtalk_slides.pdf) ]
 
 ### Acknowledgments
 
-The Collective Mind framework (CM) was created by [Grigori Fursin](https://cKnowledge.org/gfursin),
+The Collective Mind automation framework (CM) was created by [Grigori Fursin](https://cKnowledge.org/gfursin),
 sponsored by cKnowledge.org and cTuning.org, and donated to MLCommons to benefit everyone. 
-Since then, this open-source technology (CM, CM4MLOps, CM4MLPerf, CM4ABTF, CM4Research, etc)
-is being developed as a community effort thanks to all our
+This open-source technology (CM, CM4MLOps, CM4MLPerf, CM4ABTF, CM4Research, etc)
+is being developed as a community effort thanks to all our fantastic
 [volunteers, collaborators and contributors](https://github.com/mlcommons/ck/blob/master/CONTRIBUTING.md)! 
diff --git a/cm/cmind/__init__.py b/cm/cmind/__init__.py
@@ -2,7 +2,7 @@
 #
 # Written by Grigori Fursin
 
-__version__ = "3.3.4"
+__version__ = "3.4.1"
 
 from cmind.core import access
 from cmind.core import x

diff --git a/cm/dev/cmx/README.md b/cm/dev/cmx/README.md
diff --git a/cm/docs/cmx/README.md b/cm/docs/cmx/README.md
@@ -1,6 +1,9 @@
 # Collective Mind v3 aka CMX
 
-We prototype the next generation of CM.
+CMX is the next generation of the CM automation framework 
+being developed by [Grigori Fursin](https://cKnowledge.org/gfursin) 
+based on extensive feedback from CM users.
+
 
 ## Documentation
 

diff --git a/cm/pyproject.toml b/cm/pyproject.toml
@@ -0,0 +1,94 @@
+# Developer(s): Grigori Fursin
+
+[build-system]
+build-backend = "setuptools.build_meta"
+requires = ["setuptools"]
+
+[project]
+name = "cmind"
+
+authors = [
+    {name = "Grigori Fursin", email = "grigori.fursin@ctuning.org"}
+]
+
+maintainers = [
+    {name = "Grigori Fursin", email = "grigori.fursin@ctuning.org"}
+]
+
+description = "Collective Mind automation framework (CM)"
+
+requires-python = ">=3.7"
+
+dependencies = [
+    "pyyaml", 
+    "requests", 
+    "setuptools",
+    "wheel",
+    "giturlparse"
+]
+
+keywords = [
+ "cmind",
+ "cm",
+ "cmx",
+ "collective mind",
+ "automation",
+ "portability",
+ "reusability",
+ "mlops",
+ "devops",
+ "vmlops",
+ "api",
+ "cli"
+]
+
+license = {text = "Apache 2.0"}
+
+classifiers = [
+    "License :: OSI Approved :: Apache Software License",
+    "Operating System :: OS Independent",
+    "Programming Language :: Python :: 3.8",
+    "Programming Language :: Python :: 3.9",
+    "Programming Language :: Python :: 3.10",
+    "Programming Language :: Python :: 3.11",
+    "Programming Language :: Python :: 3.12",
+]
+
+dynamic = ["readme", "version"]
+
+[tool.setuptools.dynamic]
+readme = {file = "README.md", content-type = "text/markdown"}
+version = {attr = "cmind.__version__"}
+
+[tool.setuptools]
+zip-safe = false
+
+[tool.setuptools.package-data]
+cmind = ["repo/*", 
+         "repo/automation/automation/*",
+         "repo/automation/ckx/*",
+         "repo/automation/core/*",
+         "repo/automation/core/cm_60cb625a46b38610/*",
+         "repo/automation/repo/*"
+        ]
+
+[tool.setuptools.packages.find]
+where = ["."]
+include = ["cmind"]
+exclude = ["docs", "docs.*", "tests", "tests.*"]
+
+
+[project.urls]
+Homepage = "https://cKnowledge.org"
+Repository = "https://github.com/mlcommons/ck/tree/master/cm"
+Issues = "https://github.com/mlcommons/ck/issues"
+Changelog = "https://github.com/mlcommons/ck/blob/master/cm/CHANGES.md"
+
+[project.scripts]
+cmind = "cmind.cli:run"
+cm = "cmind.cli:run"
+cmx = "cmind.cli:runx"
+cmr = "cmind.cli:run_script"
+cmrd = "cmind.cli:docker_script"
+cmg = "cmind.cli:gui_script"
+cme = "cmind.cli:run_experiment"
diff --git a/cm/setup.py → cm/setup_.py b/cm/setup.py → cm/setup_.py
diff --git a/cm/setup_install.bat b/cm/setup_install.bat
diff --git a/dev/common-paper/README.md b/dev/common-paper/README.md
diff --git a/dev/reproducibility/README.md b/dev/reproducibility/README.md
diff --git a/dev/meetings/20240731.md → docs/meetings/20240731.md b/dev/meetings/20240731.md → docs/meetings/20240731.md
diff --git a/dev/meetings/20240808.md → docs/meetings/20240808.md b/dev/meetings/20240808.md → docs/meetings/20240808.md