Repositories list

• modelbench

  Public
  Run safety benchmarks against AI models and view detailed reports showing how well they performed.

  Python

  •

  Apache License 2.0

  •11•62•144•4•Updated Nov 21, 2024Nov 21, 2024

• datasets-knowledge-graphs

  Public
  Apache License 2.0

  •0•0•0•1•Updated Nov 21, 2024Nov 21, 2024

• cm4mlops

  Public
  A collection of portable, reusable and cross-platform automation recipes (CM scripts) to make it easier to build and benchmark AI systems across diverse models, data sets, software and hardware

  workflowdevopsmachine-learning+ 6automationcross-platformreusableworkflow-automationmlopsmlperf

  Python

  •

  Apache License 2.0

  •20•13•76•3•Updated Nov 21, 2024Nov 21, 2024

• GaNDLF

  Public
  A generalizable application framework for segmentation, regression, and classification using PyTorch

  machine-learningframeworkdeep-learning+ 10regressionpytorchmedical-imagingclassificationsegmentationdata-augmentationbiomedical-image-processing+ 3

  Python

  •

  Apache License 2.0

  •79•165•16•1•Updated Nov 21, 2024Nov 21, 2024

• cm4mlperf-inference

  Public
  Apache License 2.0

  •35•1•3•2•Updated Nov 21, 2024Nov 21, 2024

• inference

  Public
  Reference implementations of MLPerf™ inference benchmarks

  benchmarkmachine-learning

  Python

  •

  Apache License 2.0

  •536•1.2k•165•16•Updated Nov 21, 2024Nov 21, 2024

• algorithmic-efficiency

  Public
  MLCommons Algorithmic Efficiency is a benchmark and competition measuring neural network training speedups due to algorithmic improvements in both training algorithms and models.

  Python

  •

  Apache License 2.0

  •69•334•33•7•Updated Nov 21, 2024Nov 21, 2024

• training_policies

  Public
  Issues related to MLPerf™ training policies, including rules and suggested changes

  Python

  •

  Apache License 2.0

  •66•93•109•3•Updated Nov 21, 2024Nov 21, 2024

• logging

  Public
  MLPerf™ logging library

  Python

  •

  Apache License 2.0

  •46•30•40•3•Updated Nov 21, 2024Nov 21, 2024

• dynabench

  Public
  Python

  •

  MIT License

  •16•19•7•5•Updated Nov 19, 2024Nov 19, 2024

• policies

  Public
  General policies for MLPerf™ including submission rules, coding standards, etc.

  Python

  •

  Apache License 2.0

  •54•28•35•38•Updated Nov 19, 2024Nov 19, 2024

• mobile_app_open

  Public
  Mobile App Open

  C++

  •

  Apache License 2.0

  •23•47•29•3•Updated Nov 19, 2024Nov 19, 2024

• chakra

  Public
  Repository for MLCommons Chakra schema and tools

  Python

  •

  Apache License 2.0

  •38•67•22•9•Updated Nov 16, 2024Nov 16, 2024

• submissions_algorithms

  Public
  Apache License 2.0

  •1•1•1•1•Updated Nov 13, 2024Nov 13, 2024

• training_results_v4.1

  Public
  This repository contains the results and code for the MLPerf™ Training v4.1 benchmark.

  Python

  •2•2•1•0•Updated Nov 13, 2024Nov 13, 2024

• power-dev

  Public
  Dev repo for power measurement for the MLPerf™ benchmarks

  Python

  •

  Apache License 2.0

  •24•18•9•3•Updated Nov 12, 2024Nov 12, 2024

• croissant

  Public
  Croissant is a high-level format for machine learning datasets that brings together four rich layers.

  machine-learningschema-orgjson-ld+ 1datasets

  Python

  •

  Apache License 2.0

  •41•450•115•16•Updated Nov 12, 2024Nov 12, 2024

• ck

  Public
  Collective Knowledge (CK, CM, CM4MLOps and CMX) is an educational project to learn how to run AI, ML and other emerging workloads in the most efficient and cost-effective way across diverse models, data sets, software and hardware.

  educationautomationoptimization+ 15best-practicescollaborationcmmodularityworkflowsportabilityreusability+ 8

  Python

  •

  Apache License 2.0

  •114•609•10•0•Updated Nov 11, 2024Nov 11, 2024

• cla-bot

  Public
  TypeScript

  •

  Apache License 2.0

  •2•0•0•0•Updated Nov 7, 2024Nov 7, 2024

• inference_results_v4.0

  Public
  This repository contains the results and code for the MLPerf™ Inference v4.0 benchmark.

  Apache License 2.0

  •17•10•5•5•Updated Nov 7, 2024Nov 7, 2024

• GaNDLF-Synth

  Public
  Extension for GaNDLF [gandlf.org] to enable synthesis.

  medical-imagingsynthesis

  Python

  •

  Apache License 2.0

  •2•4•6•0•Updated Nov 7, 2024Nov 7, 2024

• inference_policies

  Public
  Issues related to MLPerf™ Inference policies, including rules and suggested changes

  Apache License 2.0

  •52•57•18•3•Updated Nov 5, 2024Nov 5, 2024

• template

  Public template
  Apache License 2.0

  •0•0•0•0•Updated Nov 1, 2024Nov 1, 2024

• tiny

  Public
  MLPerf™ Tiny is an ML benchmark suite for extremely low-power systems such as microcontrollers

  C

  •

  Apache License 2.0

  •88•359•13•4•Updated Oct 28, 2024Oct 28, 2024

• training

  Public
  Reference implementations of MLPerf™ training benchmarks

  benchmarkmachine-learning

  Python

  •

  Apache License 2.0

  •560•1.6k•19•42•Updated Oct 17, 2024Oct 17, 2024

• inference_results_v4.1

  Public
  This repository contains the results and code for the MLPerf™ Inference v4.1 benchmark.

  benchmarkmachine-learning

  Python

  •

  Apache License 2.0

  •8•2•4•1•Updated Oct 16, 2024Oct 16, 2024

• mobile_results_v4.1

  Public
  This repository contains the results and code for the MLPerf™ Mobile Inference v4.1 benchmark.

  Apache License 2.0

  •1•0•0•0•Updated Oct 16, 2024Oct 16, 2024

• inference_results_v3.1

  Public
  This repository contains the results and code for the MLPerf™ Inference v3.1 benchmark.

  Apache License 2.0

  •13•11•9•2•Updated Oct 16, 2024Oct 16, 2024

• mobile_models

  Public
  MLPerf™ Mobile models

  Apache License 2.0

  •10•24•4•0•Updated Oct 16, 2024Oct 16, 2024

• mobile_open

  Public
  MLPerf Mobile benchmarks

  Python

  •

  Apache License 2.0

  •2•10•6•2•Updated Oct 16, 2024Oct 16, 2024