Note that this repository is outdated: we are now using the next generation of the MLCommons CK workflow automation meta-framework (Collective Mind aka CM) developed by the open working group. Feel free to join this community effort to learn how to modularize ML Systems and automate their benchmarking, optimization and deployment in the real world!

Fighting the software and hardware chaos

All CK components for AI and ML are now collected in one repository!

This project is hosted by the cTuning foundation (non-profit R&D organization).

Linux & MacOS: Windows:

This is a Collective Knowledge repository with the functionality to support portable, customizable, reusable and automated workflows. It lets users automatically detect the target platform with already installed software, data and models required for a given workflow using CK software detection plugins and install missing packages in a unified way. Multiple versions of code, data and models can now co-exist in a user or system environment similar to Python virtualenv.

Further info:

• Open CK platform to publish and download stable CK components
• Documentation about portable CK workflows
• Reproducible program workflows
• Related CK publications

Author

• Grigori Fursin

Contributors

• See the list of contributors

Shared CK modules with actions

• apk
• artifact
• crowdnode
• device
• env
• log
• machine
• me
• misc
• os
• package
• platform
• platform.cpu
• platform.dsp
• platform.gpu
• platform.init
• platform.nn
• platform.npu
• platform.os
• result
• script
• soft
• xml

Installation

First install the CK framework as described here.

Then install this CK repository as follows:

 $ ck pull repo:ck-env

 $ ck list soft
 $ ck list package

Usage

You can detect and register in the CK all the instances of GCC and LLVM as follows:

 $ ck detect soft:compiler.gcc
 $ ck detect soft:compiler.llvm

You can now see multiple versions of the detected software registered in the CK as follows:

 $ ck show env

You can then compile and run unified CK benchmarks shared by the community using any of the above compiler instances (GCC, LLVM, ICC ...) and their versions simply as follows:

 $ ck pull repo:ck-autotuning
 $ ck pull repo:ctuning-programs

 $ ck compile program:cbench-automotive-susan --speed
 $ ck run program:cbench-automotive-susan

If you have Android NDK and SDK installed, CK can automatically detect it together with compiler versions (GCC, LLVM), register them and let you compile and run benchmarks on Android simply via:

 $ ck compile program:cbench-automotive-susan --speed --target_os=android21-arm-v7a
 $ ck run program:cbench-automotive-susan --target_os=android21-arm-v7a

You can find further details about our customizable and cross-platform package/environment manager here.

Questions and comments

Please feel free to get in touch with the CK community if you have any questions, suggestions and comments!