Release 0.2.0 (#60)

* Extended unit tests to classifier and fixed pooling (#17) * Extended unit tests to classifier and fixed pooling * Changed trigger of doctest workflow * Fixing issue #18 * fixed linters * Add pre-commit hooks * Doctest only on PRs * Fixed network conversion from GPU Also tested on Windows machine. * Create python_versions.yml * Update and rename python_versions.yml to tests.yml * Update export.yaml * CI fix (#21) * Create pre-commit.yaml * remove code.yaml * fixing pre-commit * Doctest with pytest * change trigger * change trigger * Delete LICENSE * checkpoint from filesystem (#20) * checkpoint from filesystem * fixed deps * Update README.md * Update LICENSE * Updating LICENSE --------- Co-authored-by: fpaissan <me@francescopaissan.it> Co-authored-by: Francesco Paissan <46992226+fpaissan@users.noreply.github.com> * Create LICENSE (#22) * Update README.md (#23) * new min python version to 3.8 * 🐛 extra_requirements now have a version - fixed CI (#24) * 🐛 extra_requirements now have a version * fixed linter errors * testing actions * fixed linter * removing tf_probability * fixed tf prob version --------- Co-authored-by: fpaissan <me@francescopaissan.it> * Documentation upgrade - guide for contribution (#25) * add contribution guide to docs * documentation with contribution guide * cosmetic * bump version 0.0.4 -> 0.0.5 * Bump requests from 2.28.2 to 2.31.0 (#27) Bumps [requests](https://github.com/psf/requests) from 2.28.2 to 2.31.0. - [Release notes](https://github.com/psf/requests/releases) - [Changelog](https://github.com/psf/requests/blob/main/HISTORY.md) - [Commits](psf/requests@v2.28.2...v2.31.0) --- updated-dependencies: - dependency-name: requests dependency-type: direct:production ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * fix pypi release * Update README.md (#29) * Patch for faster GPU inference (#35) * Patch for faster GPU inference * remove unused zeropad def --------- Co-authored-by: fpaissan <me@francescopaissan.it> * initial commit * add eval loop * add acceleration * modules as dict * add checkpointer * minor * load best checkpoint * restore epoch, optimizer, lr sched * fix logging on multi-gpu * minor fixes * working on single gpu * fix checkpointer + multi-gpu * fp16 might not be ok yet * load_modules and unwrap_model * fixed convert and export * cosmetic on export * add argparse * add metrics -- check something is off with acc * its print strange * fixed checkpointer viz * fix checkpointers and metrics * cosmetic * linters * add credits * fix requirements * fix unittest * remove recipes * remove unused files * remove unused fuctions from networks * fix tests * hot fix * onnx conversion without convert * fix requirements * add default class config and temp folder for debug mode * add doc for class Metric * finish doc MicroMind * update docs * linters fix * new initial page * bump version 0.0.5 -> 0.1.0 * final touches and bumpver * Create .readthedocs.yaml (#42) * Create .readthedocs.yaml * Create requirements.txt * Update .readthedocs.yaml * Update README.md (#43) * Update README.md * Update index.rst * Update README.md * Update index.rst * bump version 0.1.0 -> 0.1.1 * Update core.py - fix test set loading * fix dataloader prepare * Fix checkpointing issues (#48) * fix log hanging * fix checkpointer loading * fix linters * remove breakpoint * remove breakpoint (#49) * remove opt step from test * add metrics in test * Adding image classification recipe (#57) * YOLODataset * it's training, but not really * sth wrong with the loss * remove files after saving better checkpoint * same as last time * fix dict magic * fix dict magic * trying exp train * fix yolo training * yolo ok, but need to fix checkpoints * first refactor * metrics problem * add mAP metric * fix map metric * fix map * working map * refactor and batch/single_image management * metrics code refactor in torch * yolo inference optim * config file management * refactor * major refactor * Starting merge of fp_yolopatch_1 into refactor_yolo (#51) * fix load_state + fix pbar train_loss * remove mAP computation * minor * coco training of yolov8l * increase number of workers * remove breakpoint (#49) * added working mAP * added working mAP * phinet object det - not converging * fix Metric reduction with drop_last * fixed train/val split * little train optim * Fix and improvements to the Metric class (#53) * fix metrics and some polishing * minor bug fixes * cosmetic * fix inference script * bug fix on bbox rescaling * minor * starting training - franz * fix scale boxes * modules parametrization * minor refactor * Fixed Checkpointer logic and uncoupled it from the MicroMind (#54) * separate checkpointer * add status dict * add new checkpointer logic and hooks for save_state * minor to fix importing * polishing training code * ordered imports * moved loss in another file * add recover state to checkpointer * removed breakpoint * fix linters * fix loguru style * fix pytest * moved load_params * update reqs * fix linters * adding some docstrings * add docstrings to prepare_data * Added UserWarning * Delete micromind/co * fix debug option * cosmetic * add credits * fix linters * fix data location * fix data location * fix inference script * supports optional scheduler * minor * started YOLO training * switched to cos scheduler * Experiment configuration system (#55) * conf from python * fixed paths * handles configuration override * refactor + cosmetic * fix linters + remove old parsing * started training with timm * distributed working * works on cifar10 with mixup * now selecting bce_loss * linters * Add credis and some docstrings * cosmetic * fix config * add README, remove inference * fix command line script * removing obj det stuff * fix linters * Add inference script * linters * fix readme * add inference time * add inference time to viz * remove cifar10 * fix linters --------- Co-authored-by: Matteo Beltrami <beltramimatteo01@gmail.com> * Update inference.py * Official YOLO implementation for micromind (#56) * YOLODataset * it's training, but not really * sth wrong with the loss * remove files after saving better checkpoint * same as last time * fix dict magic * fix dict magic * trying exp train * fix yolo training * yolo ok, but need to fix checkpoints * first refactor * metrics problem * add mAP metric * fix map metric * fix map * working map * refactor and batch/single_image management * metrics code refactor in torch * yolo inference optim * config file management * refactor * major refactor * Starting merge of fp_yolopatch_1 into refactor_yolo (#51) * fix load_state + fix pbar train_loss * remove mAP computation * minor * coco training of yolov8l * increase number of workers * remove breakpoint (#49) * added working mAP * added working mAP * phinet object det - not converging * fix Metric reduction with drop_last * fixed train/val split * little train optim * Fix and improvements to the Metric class (#53) * fix metrics and some polishing * minor bug fixes * cosmetic * fix inference script * bug fix on bbox rescaling * minor * starting training - franz * fix scale boxes * modules parametrization * minor refactor * Fixed Checkpointer logic and uncoupled it from the MicroMind (#54) * separate checkpointer * add status dict * add new checkpointer logic and hooks for save_state * minor to fix importing * polishing training code * ordered imports * moved loss in another file * add recover state to checkpointer * removed breakpoint * fix linters * fix loguru style * fix pytest * moved load_params * update reqs * fix linters * adding some docstrings * add docstrings to prepare_data * Added UserWarning * Delete micromind/co * fix debug option * cosmetic * add credits * fix linters * fix data location * fix data location * fix inference script * supports optional scheduler * minor * started YOLO training * switched to cos scheduler * Experiment configuration system (#55) * conf from python * fixed paths * handles configuration override * refactor + cosmetic * fix linters + remove old parsing * update yaml config * passing iou_threshold to average precision function * initial version of xinet * needs fixing for exportability * add xinet train config * add xinet train config * add xinet in classification recipe * xinet training running * add xinet tests to CI * Update README.md * cosmetic * removed main for xinet's file * removed object detection file * fix linters * fix linters * fix credits * add coco training configuration files * add object detection README * Add object detection files to XiNet as well (#59) * fix linters * fix credits * add coco training configuration files * replace micromind.utils.yolo * update inference script * linters * add hparams to checkpointer * fix validation metrics * export in inference script * update changelog * add tinyCLAP folder * fix linters * better docs * fix docs, pls check * fix doc * fix linters --------- Co-authored-by: Matteo Beltrami <beltramimatteo01@gmail.com> * bump version 0.1.1 -> 0.2.0 --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: Matteo Beltrami <71525176+matteobeltrami@users.noreply.github.com> Co-authored-by: SebastianCavada <sebastian.cavada.lab@gmail.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Matteo Tremonti <102596472+Tremo8@users.noreply.github.com> Co-authored-by: Matteo Beltrami <beltramimatteo01@gmail.com>
micromind-toolkit · Nov 28, 2023 · 78dd4bd · 78dd4bd
1 parent d3f9119
commit 78dd4bd
Show file tree

Hide file tree

Showing 35 changed files with 4,440 additions and 423 deletions.
diff --git a/CHANGELOG b/CHANGELOG
@@ -1,7 +1,9 @@
-# Release 0.0.5
+# Release 0.2.0
 
-1. Improved unit tests;
-2. Extended CI to multiple python versions (including 3.8 compatibility);
-3. Updated LICENSE to Apache 2.0;
-4. Minor fixes to image classification recipes;
-5. Added guide to contribution in the documentation;
+1. Added metrics;
+2. Improved checkpointing logic;
+3. Added XiNet (https://shorturl.at/mtHT0);
+4. dded image classification recipe;
+5. Added object detection recipe;
+6. Added object detection recipe;
+7. New parse configuration that exploits python files for config;
diff --git a/docs/source/micromind.networks.rst b/docs/source/micromind.networks.rst
@@ -11,3 +11,21 @@ micromind.networks.phinet module
    :members:
    :undoc-members:
    :show-inheritance:
+
+
+micromind.networks.xinet module
+-------------------------------
+
+.. automodule:: micromind.networks.xinet
+   :members:
+   :undoc-members:
+   :show-inheritance:
+
+
+micromind.networks.yolo module
+------------------------------
+
+.. automodule:: micromind.networks.yolo
+   :members:
+   :undoc-members:
+   :show-inheritance:
diff --git a/docs/source/micromind.utils.rst b/docs/source/micromind.utils.rst
@@ -20,10 +20,10 @@ micromind.utils.helpers module
    :undoc-members:
    :show-inheritance:
 
-micromind.utils.parse module
-----------------------------
+micromind.utils.yolo module
+---------------------------
 
-.. automodule:: micromind.utils.parse
+.. automodule:: micromind.utils.yolo
    :members:
    :undoc-members:
    :show-inheritance:
diff --git a/examples/mind.py b/examples/mind.py
diff --git a/micromind/__init__.py b/micromind/__init__.py
@@ -1,17 +1,4 @@
-from .core import MicroMind, Metric, Stage
+from .core import Metric, MicroMind, Stage
 
 # Package version
-__version__ = "0.1.1"
-
-
-"""datasets_info is a dictionary that contains information about the attributes
-of the datasets.
-This dictionary is used in networks.py inside the from_pretrained class method
-in order to examine the inputs and initialize the PhiNet or, in case of
-mismatching between dataset and Nclasses, raise an AssertionError."""
-datasets_info = {
-    "CIFAR-100": {"Nclasses": 100, "NChannels": 3, "ext": ".pth.tar"},
-    "CIFAR-10": {"Nclasses": 10, "NChannels": 3, "ext": ".pth.tar"},
-    "ImageNet-1k": {"Nclasses": 1000, "NChannels": 3, "ext": ".pth.tar"},
-    "MNIST": {"Nclasses": 10, "NChannels": 1, "ext": ".pth.tar"},
-}
+__version__ = "0.2.0"
diff --git a/micromind/convert.py b/micromind/convert.py
@@ -6,23 +6,45 @@
     - Francesco Paissan, 2023
     - Alberto Ancilotto, 2023
 """
+import os
 from pathlib import Path
-from loguru import logger
 from typing import Union
-import torch.nn as nn
+
 import torch
-import os
+import torch.nn as nn
+
+from .utils.helpers import get_logger
+import micromind as mm
+
+logger = get_logger()
 
 
 @torch.no_grad()
 def convert_to_onnx(
-    net: nn.Module,
+    net: Union[nn.Module, mm.MicroMind],
     save_path: Union[Path, str] = "model.onnx",
     simplify: bool = False,
     replace_forward: bool = False,
 ):
     """Converts nn.Module to onnx and saves it to save_path.
-    Optionally simplifies it."""
+    Optionally simplifies it. This function is internally used from `mm.MicroMind`.
+
+    Arguments
+    ---------
+    net : Union[nn.Module, mm.MicroMind]
+        PyTorch module to be exported.
+    save_path : Union[Path, str]
+        Output path for the ONNX model.
+    simplify : bool
+        `True` if you want to simplify the model. Defaults to False.
+    replace_forward : bool
+        Used if you want to replace the forward method. It is need if you are calling
+        this function on a `mm.MicroMind`. Defaults to False.
+
+    Returns
+    -------
+    The path of the ONNX model. : Path
+    """
     save_path = Path(save_path)
     os.makedirs(save_path.parent, exist_ok=True)
     x = torch.zeros([1] + list(net.input_shape))
@@ -61,18 +83,35 @@ def convert_to_onnx(
 
 @torch.no_grad()
 def convert_to_openvino(
-    net: nn.Module, save_path: Path, replace_forward: bool = False
+    net: Union[nn.Module, mm.MicroMind], save_path: Path, replace_forward: bool = False
 ) -> str:
-    """Converts nn.Module to OpenVINO."""
+    """Converts model to OpenVINO. Uses ONNX in the process and converts networks
+    from channel-first to channel-last (for optimized inference).
+
+    Arguments
+    ---------
+    net : nn.Module
+        PyTorch module to be exported.
+    save_path : Union[Path, str]
+        Output path for the OpenVINO model.
+    replace_forward : bool
+        Used if you want to replace the forward method. It is need if you are calling
+        this function on a `mm.MicroMind`. Defaults to False.
+
+    Returns
+    -------
+    The path of the XML model. : str
+
+    """
     try:
         import os
 
         os.environ["TF_CPP_MIN_LOG_LEVEL"] = "3"
         import sys
         from pathlib import Path
-        from loguru import logger
 
         import onnx
+        from loguru import logger
         from onnx_tf.backend import prepare
         from openvino.tools.mo import main as mo_main
 
@@ -121,23 +160,38 @@ def convert_to_openvino(
 
 @torch.no_grad()
 def convert_to_tflite(
-    net: nn.Module,
-    save_path: Path,
+    net: Union[nn.Module, mm.MicroMind],
+    save_path: Union[Path, str],
     batch_quant: torch.Tensor = None,
     replace_forward: bool = False,
 ) -> None:
-    """Converts nn.Module to tf_lite, optionally quantizes it."""
+    """Converts nn.Module to tf_lite, optionally quantizes it.
+
+    Arguments
+    ---------
+    net : nn.Module
+        PyTorch module to be exported.
+    save_path : Union[Path, str]
+        Output path for the OpenVINO model.
+    batch_quant : torch.Tensor
+        Optional batch for quantization. When passed, it is used to create the
+        statistics of the quantized activations.
+    replace_forward : bool
+        Used if you want to replace the forward method. It is need if you are calling
+        this function on a `mm.MicroMind`. Defaults to False.
+
+    """
     try:
         import os
 
         os.environ["TF_CPP_MIN_LOG_LEVEL"] = "3"
         import shutil
         import sys
         from pathlib import Path
-        from loguru import logger
 
         import numpy as np
         import tensorflow as tf
+        from loguru import logger
 
     except Exception as e:
         print(str(e))