From 6e92a1b5aa6e506d4ff0c621de4a5e48bffecf59 Mon Sep 17 00:00:00 2001 From: cgerum Date: Thu, 14 Sep 2023 14:24:11 +0000 Subject: [PATCH] =?UTF-8?q?Deploying=20to=20gh-pages=20from=20@=20ekut-es/?= =?UTF-8?q?hannah@f77e13d334d69c7ed26ec092d6e47ee10d1e152b=20=F0=9F=9A=80?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- index.html | 2 +- publications/index.html | 3 +++ search/search_index.json | 2 +- sitemap.xml.gz | Bin 127 -> 127 bytes 4 files changed, 5 insertions(+), 2 deletions(-) diff --git a/index.html b/index.html index 9a193738..0e7d72f5 100644 --- a/index.html +++ b/index.html @@ -331,5 +331,5 @@

Automatic Mirroring

diff --git a/publications/index.html b/publications/index.html index eac9af24..f73fd637 100644 --- a/publications/index.html +++ b/publications/index.html @@ -147,6 +147,9 @@ limitations under the License. -->

Related Publications

+

Precise localization within the GI tract by combining classification of CNNs and time-series analysis of HMMs. +by Julia Werner, Christoph Gerum, Moritz Reiber, Jörg Nick, Oliver Bringmann +In 2023 14th International Workshop on Machine Learning in Medical Imaging (MLMI 2023)

Hardware Accelerator and Neural Network Co-Optimization for Ultra-Low-Power Audio Processing Devices by Christoph Gerum, Adrian Frischknecht, Paul Palomero Bernardo, Tobias Hald, Konstantin Lübeck, and Oliver Bringmann In 2022 25th Euromicro Conference on Digital System Design (DSD), pages 1-8, 2022. diff --git a/search/search_index.json b/search/search_index.json index 35a122c4..c78b2f55 100644 --- a/search/search_index.json +++ b/search/search_index.json @@ -1 +1 @@ -{"config":{"indexing":"full","lang":["en"],"min_search_length":3,"prebuild_index":false,"separator":"[\\s\\-]+"},"docs":[{"location":"","text":"HANNAH - Hardware Accelerator and Neural network searcH Getting Started Installing dependencies Dependencies and virtual environments are managed using poetry . python (>=3.8 <3.11) and development headers libsndfile and development headers libsox and development headers a blas implementation and development headers git-lfs for management of checkpoints Ubuntu 20.04+ Install dependencies: sudo apt update sudo apt -y install python3-dev libblas-dev liblapack-dev libsndfile1-dev libsox-dev git-lfs Centos / RHEL / Scientific Linux: 7+ Install dependencies: sudo yum install portaudio-devel libsndfile1-devel libsox-devel -y Install a python 3.9 or python 3.10 version using pyenv . Install poetry curl -sSL https://install.python-poetry.org/ | python3 For alternative installation methods see: https://python-poetry.org/docs/#installation Caution : this usually installs poetry to ~/.local/bin if this folder is not in your path you might need to run poetry as: ~/.local/bin/poetry Software installation In the root directory of the project run: git submodule update --init --recursive poetry install -E vision This creates a virtual environment under ~/.cache/pypoetry/virtualenvs. The environment can be activated using: poetry shell Optional Dependencies We support installation of optional dependencies using poetry's -E commandline flag We currently have the following optional dependencies: Vision-Models Vision models require additional dependencies, these can be installed using: poetry install -E vision These dependencies include kornia and albumentations for image augmentations and timm (torch image models) for baseline neural network models. Onnx-Runtime Backend poetry install -E onnxrt-backend Backend support for running models on onnx-runtime. Tflite-Backend poetry install -E onnx-tf Onnx based conversion of trained models to tensorflow/tensorflow-lite for external inference backends. DVC based experiment management (experimental) poetry install -E dvc This installs dvc based model, data and experiment management. DVC support is highly experimental and subject to change. Installation Tips 1.) venv location poetry installs the dependencies to a virtual environment in ~/.cache/pypoetry/virtualenvs You can change the location of this directory using: poetry config virtualenvs.path Or move it to a subdirectory of the project directory using: poetry config virtualenvs.in-project true Installing the datasets Datasets are downloaded automatically to the datasets data folder by default this is a subfolder of the dataset's data folder. For the VAD Dataset the following Flag is needed to Download/Override the Dataset dataset.override=True Training - Keyword Spotting Training is invoked by hannah-train If available the first GPU of the system will be used by default. Selecting another GPU is possible using the argument trainer. gpus=[number] e.g. for GPU 2 use: hannah-train trainer.gpus=[2] Trained models are saved under trained_models// . Training - VAD Training of VAD is invoked by hannah-train dataset=vad model.n_labels=2 Training of VAD_Extended is invoked by hannah-train dataset=vad_extended model.n_labels=2 VAD dataset variants Selection of other Voice Dataset use dataset.variants=\"[UWNU, de, en, it, fr, es]\" hannah-train dataset=vad model.n_labels=2 dataset.variants=\"[UWNU, de, en, it, fr, es]\" Selection of other Noise Datasets use dataset.noise_dataset=\"[TUT]\" hannah-train dataset=vad model.n_labels=2 dataset.noise_dataset=\"[TUT]\" Selection of dataset Split use dataset.data_split=\"vad_balanced\" hannah-train dataset=vad model.n_labels=2 dataset.data_split=\"vad_balanced\" Create Vad_small Dataset hannah-train dataset=vad model.n_labels=2 dataset.variants=\"[UWNU]\" dataset.noise_dataset=\"[TUT]\" dataset.data_split=\"vad_balanced\" Create VAD_big Dataset hannah-train dataset=vad model.n_labels=2 dataset.variants=\"[UWNU, en, de, fr, es, it]\" dataset.noise_dataset=\"[TUT, FSD50K]\" dataset.data_split=\"vad_balanced\" Training - PAMAP2 Training of PAMAP2 human activity detection dataset is invoked by: hannah-train -cn config_activity Training - Emergency Siren Dataset Training of emergency siren detection dataset is invoked by: hannah-train -cn config_siren_detection Parallel Launchers To launch multiple optimizations in parallel you can use a hydra launcher. Joblib launcher is installed by default: hannah-train --multirun hydra/sweeper=optuna hydra/launcher=joblib optimizer.lr='interval(0.0001,0.1)' optimizer.weight_decay='interval(0, 0.1)' hydra.launcher.n_jobs=5 Launches optimizer hyperparameter optimization with 5 parallel jobs. Early stopping To stop training early when a validation metric does not improve, you can use lightning's early stopping callback: hannah-train early_stopping=default Showing graphical results All experiments are logged to tensorboard: To visualize the results use: tensorboard --logdir trained_models or a subdirectory of trained models if only one experiment or model is of interest. Pre commit hooks This project uses precommit hooks for auto formatting and static code analysis. To enable precommit hooks run the following command in a poetry shell . pre-commit install Try to follow pep8 naming conventions and the rest of pep8 to the best of your abilities. Automatic Mirroring This project automatically mirrors its main branch and all branches prefixed with pub/ to its public github repository. These branches are configured as protected branches by default.","title":"Home"},{"location":"#hannah-hardware-accelerator-and-neural-network-search","text":"","title":"HANNAH - Hardware Accelerator and Neural network searcH"},{"location":"#getting-started","text":"","title":"Getting Started"},{"location":"#installing-dependencies","text":"Dependencies and virtual environments are managed using poetry . python (>=3.8 <3.11) and development headers libsndfile and development headers libsox and development headers a blas implementation and development headers git-lfs for management of checkpoints","title":"Installing dependencies"},{"location":"#ubuntu-2004","text":"Install dependencies: sudo apt update sudo apt -y install python3-dev libblas-dev liblapack-dev libsndfile1-dev libsox-dev git-lfs","title":"Ubuntu 20.04+"},{"location":"#centos-rhel-scientific-linux-7","text":"Install dependencies: sudo yum install portaudio-devel libsndfile1-devel libsox-devel -y Install a python 3.9 or python 3.10 version using pyenv .","title":"Centos / RHEL / Scientific Linux: 7+"},{"location":"#install-poetry","text":"curl -sSL https://install.python-poetry.org/ | python3 For alternative installation methods see: https://python-poetry.org/docs/#installation Caution : this usually installs poetry to ~/.local/bin if this folder is not in your path you might need to run poetry as: ~/.local/bin/poetry","title":"Install poetry"},{"location":"#software-installation","text":"In the root directory of the project run: git submodule update --init --recursive poetry install -E vision This creates a virtual environment under ~/.cache/pypoetry/virtualenvs. The environment can be activated using: poetry shell","title":"Software installation"},{"location":"#optional-dependencies","text":"We support installation of optional dependencies using poetry's -E commandline flag We currently have the following optional dependencies:","title":"Optional Dependencies"},{"location":"#vision-models","text":"Vision models require additional dependencies, these can be installed using: poetry install -E vision These dependencies include kornia and albumentations for image augmentations and timm (torch image models) for baseline neural network models.","title":"Vision-Models"},{"location":"#onnx-runtime-backend","text":"poetry install -E onnxrt-backend Backend support for running models on onnx-runtime.","title":"Onnx-Runtime Backend"},{"location":"#tflite-backend","text":"poetry install -E onnx-tf Onnx based conversion of trained models to tensorflow/tensorflow-lite for external inference backends.","title":"Tflite-Backend"},{"location":"#dvc-based-experiment-management-experimental","text":"poetry install -E dvc This installs dvc based model, data and experiment management. DVC support is highly experimental and subject to change.","title":"DVC based experiment management (experimental)"},{"location":"#installation-tips","text":"1.) venv location poetry installs the dependencies to a virtual environment in ~/.cache/pypoetry/virtualenvs You can change the location of this directory using: poetry config virtualenvs.path Or move it to a subdirectory of the project directory using: poetry config virtualenvs.in-project true","title":"Installation Tips"},{"location":"#installing-the-datasets","text":"Datasets are downloaded automatically to the datasets data folder by default this is a subfolder of the dataset's data folder. For the VAD Dataset the following Flag is needed to Download/Override the Dataset dataset.override=True","title":"Installing the datasets"},{"location":"#training-keyword-spotting","text":"Training is invoked by hannah-train If available the first GPU of the system will be used by default. Selecting another GPU is possible using the argument trainer. gpus=[number] e.g. for GPU 2 use: hannah-train trainer.gpus=[2] Trained models are saved under trained_models// .","title":"Training - Keyword Spotting"},{"location":"#training-vad","text":"Training of VAD is invoked by hannah-train dataset=vad model.n_labels=2 Training of VAD_Extended is invoked by hannah-train dataset=vad_extended model.n_labels=2","title":"Training - VAD"},{"location":"#vad-dataset-variants","text":"Selection of other Voice Dataset use dataset.variants=\"[UWNU, de, en, it, fr, es]\" hannah-train dataset=vad model.n_labels=2 dataset.variants=\"[UWNU, de, en, it, fr, es]\" Selection of other Noise Datasets use dataset.noise_dataset=\"[TUT]\" hannah-train dataset=vad model.n_labels=2 dataset.noise_dataset=\"[TUT]\" Selection of dataset Split use dataset.data_split=\"vad_balanced\" hannah-train dataset=vad model.n_labels=2 dataset.data_split=\"vad_balanced\" Create Vad_small Dataset hannah-train dataset=vad model.n_labels=2 dataset.variants=\"[UWNU]\" dataset.noise_dataset=\"[TUT]\" dataset.data_split=\"vad_balanced\" Create VAD_big Dataset hannah-train dataset=vad model.n_labels=2 dataset.variants=\"[UWNU, en, de, fr, es, it]\" dataset.noise_dataset=\"[TUT, FSD50K]\" dataset.data_split=\"vad_balanced\"","title":"VAD dataset variants"},{"location":"#training-pamap2","text":"Training of PAMAP2 human activity detection dataset is invoked by: hannah-train -cn config_activity","title":"Training - PAMAP2"},{"location":"#training-emergency-siren-dataset","text":"Training of emergency siren detection dataset is invoked by: hannah-train -cn config_siren_detection","title":"Training - Emergency Siren Dataset"},{"location":"#parallel-launchers","text":"To launch multiple optimizations in parallel you can use a hydra launcher. Joblib launcher is installed by default: hannah-train --multirun hydra/sweeper=optuna hydra/launcher=joblib optimizer.lr='interval(0.0001,0.1)' optimizer.weight_decay='interval(0, 0.1)' hydra.launcher.n_jobs=5 Launches optimizer hyperparameter optimization with 5 parallel jobs.","title":"Parallel Launchers"},{"location":"#early-stopping","text":"To stop training early when a validation metric does not improve, you can use lightning's early stopping callback: hannah-train early_stopping=default","title":"Early stopping"},{"location":"#showing-graphical-results","text":"All experiments are logged to tensorboard: To visualize the results use: tensorboard --logdir trained_models or a subdirectory of trained models if only one experiment or model is of interest.","title":"Showing graphical results"},{"location":"#pre-commit-hooks","text":"This project uses precommit hooks for auto formatting and static code analysis. To enable precommit hooks run the following command in a poetry shell . pre-commit install Try to follow pep8 naming conventions and the rest of pep8 to the best of your abilities.","title":"Pre commit hooks"},{"location":"#automatic-mirroring","text":"This project automatically mirrors its main branch and all branches prefixed with pub/ to its public github repository. These branches are configured as protected branches by default.","title":"Automatic Mirroring"},{"location":"fallacies/","text":"Fallacies and Pitfalls Changing OneCycleLR learningrate By default the OneCycleLR learning rate scheduler will ignore the setting for \u00f2ptimizer.lr and instead will use the setting scheduler.max_lr to calculate each steps learning rate. To avoid problems when sweeping the the learning rate, in hannah the default setting for scheduler.max_lr is using the same as optimizer.lr but this can be overridden in project specific configuration files. No SecretStorage or DBUS running During poetry install the system will try to unlock secret storage / keyring. If this fails you will see a Error Message like: [org.freedesktop.DBus.Error.UnknownMethod] ('Object does not exist on path \"/org/freedesktop/secrets/collection/login\"') You can disable this error message by one of the following means: Install and Configure a local DBus and Keyring support. This might work. Setting the default backend for keyring: export PYTHON_KEYRING_BACKEND=keyring.backends.null.Keyring Disabling keyring for the whole system: keyring --disable -b keyring.backends.SecretService.Keyring","title":"Fallacies and Pitfalls"},{"location":"fallacies/#fallacies-and-pitfalls","text":"","title":"Fallacies and Pitfalls"},{"location":"fallacies/#changing-onecyclelr-learningrate","text":"By default the OneCycleLR learning rate scheduler will ignore the setting for \u00f2ptimizer.lr and instead will use the setting scheduler.max_lr to calculate each steps learning rate. To avoid problems when sweeping the the learning rate, in hannah the default setting for scheduler.max_lr is using the same as optimizer.lr but this can be overridden in project specific configuration files.","title":"Changing OneCycleLR learningrate"},{"location":"fallacies/#no-secretstorage-or-dbus-running","text":"During poetry install the system will try to unlock secret storage / keyring. If this fails you will see a Error Message like: [org.freedesktop.DBus.Error.UnknownMethod] ('Object does not exist on path \"/org/freedesktop/secrets/collection/login\"') You can disable this error message by one of the following means: Install and Configure a local DBus and Keyring support. This might work. Setting the default backend for keyring: export PYTHON_KEYRING_BACKEND=keyring.backends.null.Keyring Disabling keyring for the whole system: keyring --disable -b keyring.backends.SecretService.Keyring","title":"No SecretStorage or DBUS running"},{"location":"publications/","text":"Related Publications Hardware Accelerator and Neural Network Co-Optimization for Ultra-Low-Power Audio Processing Devices by Christoph Gerum, Adrian Frischknecht, Paul Palomero Bernardo, Tobias Hald, Konstantin L\u00fcbeck, and Oliver Bringmann In 2022 25th Euromicro Conference on Digital System Design (DSD), pages 1-8, 2022. Keywords: Machine Learning, Neural Networks, AutoML, Neural Architecture Search Behavior of Keyword Spotting Networks Under Noisy Conditions by Anwesh Mohanty, Adrian Frischknecht, Christoph Gerum, and Oliver Bringmann In International Conference on Artificial Neural Networks (ICANN), 2021. Keywords: Keyword spotting, High noise conditions, Adaptive batch normalization, Sinc convolution network, Temporal convolution ResNet. Dynamic Range and Complexity Optimization of Mixed-Signal Machine Learning Systems by Naci Pekcokguler, Dominique Morche, Adrian Frischknecht, Christoph Gerum, Andreas Burg, and Catherine Dehollain In 2021 IEEE International Symposium on Circuits and Systems (ISCAS), pages 1-5, 2021. Keywords: keyword spotting, analog feature extraction, machine learning classi\ufb01er, dynamic range reduction, system complexity optimization Automated HW/SW Co-Design for Edge AI: State, Challenges and Steps Ahead by Oliver Bringmann, Wolfgang Ecker, Ingo Feldner, Adrian Frischknecht, Christoph Gerum, Timo H\u00e4m\u00e4l\u00e4inen, Muhammad Abdullah Hanif, Michael J. Klaiber, Daniel Mueller-Gritschneder, Paul Palomero Bernardo, Sebastian Prebeck, and Muhammad Shafique In Proceedings of the 2021 International Conference on Hardware/Software Codesign and System Synthesis, pages 11\u201320. Association for Computing Machinery, 2021. UltraTrail: A Configurable Ultra-Low Power TC-ResNet AI Accelerator for Efficient Keyword Spotting by Paul Palomero Bernardo, Christoph Gerum, Adrian Frischknecht, Konstantin L\u00fcbeck, and Oliver Bringmann In 2020 IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems (TCAD), pages 1-12, 2020.","title":"Publications"},{"location":"publications/#related-publications","text":"Hardware Accelerator and Neural Network Co-Optimization for Ultra-Low-Power Audio Processing Devices by Christoph Gerum, Adrian Frischknecht, Paul Palomero Bernardo, Tobias Hald, Konstantin L\u00fcbeck, and Oliver Bringmann In 2022 25th Euromicro Conference on Digital System Design (DSD), pages 1-8, 2022. Keywords: Machine Learning, Neural Networks, AutoML, Neural Architecture Search Behavior of Keyword Spotting Networks Under Noisy Conditions by Anwesh Mohanty, Adrian Frischknecht, Christoph Gerum, and Oliver Bringmann In International Conference on Artificial Neural Networks (ICANN), 2021. Keywords: Keyword spotting, High noise conditions, Adaptive batch normalization, Sinc convolution network, Temporal convolution ResNet. Dynamic Range and Complexity Optimization of Mixed-Signal Machine Learning Systems by Naci Pekcokguler, Dominique Morche, Adrian Frischknecht, Christoph Gerum, Andreas Burg, and Catherine Dehollain In 2021 IEEE International Symposium on Circuits and Systems (ISCAS), pages 1-5, 2021. Keywords: keyword spotting, analog feature extraction, machine learning classi\ufb01er, dynamic range reduction, system complexity optimization Automated HW/SW Co-Design for Edge AI: State, Challenges and Steps Ahead by Oliver Bringmann, Wolfgang Ecker, Ingo Feldner, Adrian Frischknecht, Christoph Gerum, Timo H\u00e4m\u00e4l\u00e4inen, Muhammad Abdullah Hanif, Michael J. Klaiber, Daniel Mueller-Gritschneder, Paul Palomero Bernardo, Sebastian Prebeck, and Muhammad Shafique In Proceedings of the 2021 International Conference on Hardware/Software Codesign and System Synthesis, pages 11\u201320. Association for Computing Machinery, 2021. UltraTrail: A Configurable Ultra-Low Power TC-ResNet AI Accelerator for Efficient Keyword Spotting by Paul Palomero Bernardo, Christoph Gerum, Adrian Frischknecht, Konstantin L\u00fcbeck, and Oliver Bringmann In 2020 IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems (TCAD), pages 1-12, 2020.","title":"Related Publications"},{"location":"compression/compresseion/knowledge_distillation/","text":"Pruning TBD","title":"Knowledge Distillation"},{"location":"compression/compresseion/knowledge_distillation/#pruning","text":"TBD","title":"Pruning"},{"location":"compression/compression/clustering/","text":"Clustering k-means \\(k\\) -means clustering can be applied to the weights of all layers at the end of a training epoch. Each time, for a defined number of \\(k\\) clusters per layer the centroids are identified per layer. Next, each weight is replaced with the value of the closest cluster center. The zero is not clustered to preserve the amount of zeros induced by pruning. This results in a total of \\(k+1\\) distinct values. Additionally, at the end of the fitting process, clustering is applied. Afterwards, the weights are not further altered. After completing the training process, every value of each layer of the trained neural network should be equal to one cluster center or zero. Clustering (in combination with pruning and SVD-based compression) can be invoked by: hannah-train compression=full The default amount of clusters is 15 If clustering or SVD was applied, the last checkpoint of the model during training was used for validation and testing the neural network, instead of the best checkpoint as previously used. Fixing the amount of epochs with which the final model is trained enables better comparability of the models for different compression parameters and ensures that the compression is sufficiently integrated into the training process. In particular the choice of a final model with high accuracy, which has not yet been compressed in the training process (via SVD or clustering) is avoided by ensuring that the last checkpoint is used. Consequently, fixing the amount of training epochs for the final model illuminates the effect that the compression during the training has on the model. For applying clustering to quantized models, the batch norm fusion must be considered. This is not yet implemented.","title":"Clustering"},{"location":"compression/compression/clustering/#clustering","text":"","title":"Clustering"},{"location":"compression/compression/clustering/#k-means","text":"\\(k\\) -means clustering can be applied to the weights of all layers at the end of a training epoch. Each time, for a defined number of \\(k\\) clusters per layer the centroids are identified per layer. Next, each weight is replaced with the value of the closest cluster center. The zero is not clustered to preserve the amount of zeros induced by pruning. This results in a total of \\(k+1\\) distinct values. Additionally, at the end of the fitting process, clustering is applied. Afterwards, the weights are not further altered. After completing the training process, every value of each layer of the trained neural network should be equal to one cluster center or zero. Clustering (in combination with pruning and SVD-based compression) can be invoked by: hannah-train compression=full The default amount of clusters is 15 If clustering or SVD was applied, the last checkpoint of the model during training was used for validation and testing the neural network, instead of the best checkpoint as previously used. Fixing the amount of epochs with which the final model is trained enables better comparability of the models for different compression parameters and ensures that the compression is sufficiently integrated into the training process. In particular the choice of a final model with high accuracy, which has not yet been compressed in the training process (via SVD or clustering) is avoided by ensuring that the last checkpoint is used. Consequently, fixing the amount of training epochs for the final model illuminates the effect that the compression during the training has on the model. For applying clustering to quantized models, the batch norm fusion must be considered. This is not yet implemented.","title":"k-means"},{"location":"compression/compression/decomposition/","text":"Decomposition Singular value decomposition (SVD) In a CNN, the SVD can be applied to a fully connected layer, corresponding to a matrix \\(A\\) , resulting in the composition of two fully connected layers of lower dimension associated to the matrices \\(U'\\) and \\(\\Sigma' V^{T'}\\) . If \\(\\Sigma\\) is truncated to a rank \\(r=k\\) , dimensions change to \\(U'\\in \\mathbb{R}^{m \\times k}\\) , \\(\\Sigma' \\in \\mathbb{R}^{k \\times k}\\) and \\(V^{T'}\\in \\mathbb{R}^{k \\times n}\\) . Therefore, the SVD is applied to all weights of the linear layers, resulting in the decomposition of matrices. Subsequently, the weight matrices of the fully connected layers are compressed with a fixed rank \\(r\\) . Afterwards, the model is restructured, resulting in two smaller fully connected layers instead of one larger fully connected layer. SVD-based compression (in combination with pruning and clustering) can be invoked by: hannah-train compression=full The default rank_compression of SVD is 4 This has been implemented for TC-ResNets, which consist of one linear layer as the last layer of the network. Thus, the resulting network comprises of a sequential layer at the end, including one fully connected layer with \\(\\Sigma'*V^{T'}\\) and one fully connected layer with \\(U'\\) . If those partial matrices are multiplied, they ideally yield a close approximation to the original matrix \\(A\\) . Finally, fine-tuning is performed, by continuing the training process with this restructured, compressed network while the same optimizer and parameters are used as for the first training section. With this approach, the SVD-based compression is easily combined with QAT and pruning. Both methods are also applied to the restructured, compressed model during retraining. The layer's name can be adjusted to add other models.","title":"Tensor Decompositions"},{"location":"compression/compression/decomposition/#decomposition","text":"","title":"Decomposition"},{"location":"compression/compression/decomposition/#singular-value-decomposition-svd","text":"In a CNN, the SVD can be applied to a fully connected layer, corresponding to a matrix \\(A\\) , resulting in the composition of two fully connected layers of lower dimension associated to the matrices \\(U'\\) and \\(\\Sigma' V^{T'}\\) . If \\(\\Sigma\\) is truncated to a rank \\(r=k\\) , dimensions change to \\(U'\\in \\mathbb{R}^{m \\times k}\\) , \\(\\Sigma' \\in \\mathbb{R}^{k \\times k}\\) and \\(V^{T'}\\in \\mathbb{R}^{k \\times n}\\) . Therefore, the SVD is applied to all weights of the linear layers, resulting in the decomposition of matrices. Subsequently, the weight matrices of the fully connected layers are compressed with a fixed rank \\(r\\) . Afterwards, the model is restructured, resulting in two smaller fully connected layers instead of one larger fully connected layer. SVD-based compression (in combination with pruning and clustering) can be invoked by: hannah-train compression=full The default rank_compression of SVD is 4 This has been implemented for TC-ResNets, which consist of one linear layer as the last layer of the network. Thus, the resulting network comprises of a sequential layer at the end, including one fully connected layer with \\(\\Sigma'*V^{T'}\\) and one fully connected layer with \\(U'\\) . If those partial matrices are multiplied, they ideally yield a close approximation to the original matrix \\(A\\) . Finally, fine-tuning is performed, by continuing the training process with this restructured, compressed network while the same optimizer and parameters are used as for the first training section. With this approach, the SVD-based compression is easily combined with QAT and pruning. Both methods are also applied to the restructured, compressed model during retraining. The layer's name can be adjusted to add other models.","title":"Singular value decomposition (SVD)"},{"location":"compression/compression/pruning/","text":"Pruning TBD","title":"Pruning"},{"location":"compression/compression/pruning/#pruning","text":"TBD","title":"Pruning"},{"location":"compression/compression/quantization/","text":"Quantization The training framework provides quantization aware training support for model factory based models. We support the following quantization schemes. Symmetric linear Quantization: Symmetric linear quantization applies the following transformation on a floating point value \\(v\\) : \\(\\mbox{quantized}(v)\\ =\\ \\mbox{clamp}(\\mbox{round}(v\\cdot2^{\\mbox{bits}-1}), 2^{-\\mbox{bits}-1}, 2^{\\mbox{bits}-1}-1)\\) Power of 2 Quantization with Zero: Power of 2 quantization applies the following transformation to a floating point value v: \\( \\mbox{quantized}(v)\\ =\\ \\begin{cases} \\mbox{sign}(v)\\ \\cdot\\ 2^{\\mbox{round}(\\log2(\\mbox{abs}(v)))} & \\text{if}\\ v\\ \\ge\\ 2^{2^{\\text{bits}}\\\\ 0 & \\text{otherwise} \\\\ \\end{cases} \\) Power of 2 Quantization can currently only be enabled for weights. Noisy Quantization As described in https://openreview.net/pdf?id=dV19Yyi1fS3 quantizing only a subset of model weights during training can improve accuracies of quantized networks compared to full quantization aware training. The probability of quantizing a weight can be given as parameter noise_prob in in the qconfig. Unfortunately this introduces an additional hyperparameter in the quantization space, good values for noise prob seem to be in the range of 0.7 to 0.9. Even when using noisy quantization eval and test are always run on fully quantized networks. Configuration Quantization can be configured as the qconfig attribute of factory based models: qconfig: target : speech_recognition.models.factory.qconfig.get_trax_qat_qconfig config: bw_b: 8 # Bitwidth for bias bw_w: 6 # Bitwidth for weights bw_f: 8 # Bitwidth for activations power_of_2: false # Use power of two quantization for weights noise_prob: 0.7 # Probability of quantizing a value during training Aditionally standard pytorch quantization aware training is supported by using a standard pytorch qconfig. qconfig: _target_: torch.quantization.get_default_qat_qconfig backend: fbgemm In this case no quantization noise is supported.","title":"Quantization"},{"location":"compression/compression/quantization/#quantization","text":"The training framework provides quantization aware training support for model factory based models. We support the following quantization schemes. Symmetric linear Quantization: Symmetric linear quantization applies the following transformation on a floating point value \\(v\\) : \\(\\mbox{quantized}(v)\\ =\\ \\mbox{clamp}(\\mbox{round}(v\\cdot2^{\\mbox{bits}-1}), 2^{-\\mbox{bits}-1}, 2^{\\mbox{bits}-1}-1)\\) Power of 2 Quantization with Zero: Power of 2 quantization applies the following transformation to a floating point value v: \\( \\mbox{quantized}(v)\\ =\\ \\begin{cases} \\mbox{sign}(v)\\ \\cdot\\ 2^{\\mbox{round}(\\log2(\\mbox{abs}(v)))} & \\text{if}\\ v\\ \\ge\\ 2^{2^{\\text{bits}}\\\\ 0 & \\text{otherwise} \\\\ \\end{cases} \\) Power of 2 Quantization can currently only be enabled for weights.","title":"Quantization"},{"location":"compression/compression/quantization/#noisy-quantization","text":"As described in https://openreview.net/pdf?id=dV19Yyi1fS3 quantizing only a subset of model weights during training can improve accuracies of quantized networks compared to full quantization aware training. The probability of quantizing a weight can be given as parameter noise_prob in in the qconfig. Unfortunately this introduces an additional hyperparameter in the quantization space, good values for noise prob seem to be in the range of 0.7 to 0.9. Even when using noisy quantization eval and test are always run on fully quantized networks.","title":"Noisy Quantization"},{"location":"compression/compression/quantization/#configuration","text":"Quantization can be configured as the qconfig attribute of factory based models: qconfig: target : speech_recognition.models.factory.qconfig.get_trax_qat_qconfig config: bw_b: 8 # Bitwidth for bias bw_w: 6 # Bitwidth for weights bw_f: 8 # Bitwidth for activations power_of_2: false # Use power of two quantization for weights noise_prob: 0.7 # Probability of quantizing a value during training Aditionally standard pytorch quantization aware training is supported by using a standard pytorch qconfig. qconfig: _target_: torch.quantization.get_default_qat_qconfig backend: fbgemm In this case no quantization noise is supported.","title":"Configuration"},{"location":"configuration/configuration/multi_gpu/","text":"Multi GPU training Hannah supports multi GPU-Training using the lightning distributed APIs: We provide preset trainer configs for distributed data parallel training: hannah-train trainer=ddp trainer.gpus=[0,1] And for sharded training using fairscale: hannah-train trainer=sharded trainer.gpus=[0,1] Sharded training distributes some of the model parameters across multiple GPUs and allows fitting bigger models in the same amount of GPU memory.","title":"Multi-GPU support"},{"location":"configuration/configuration/multi_gpu/#multi-gpu-training","text":"Hannah supports multi GPU-Training using the lightning distributed APIs: We provide preset trainer configs for distributed data parallel training: hannah-train trainer=ddp trainer.gpus=[0,1] And for sharded training using fairscale: hannah-train trainer=sharded trainer.gpus=[0,1] Sharded training distributes some of the model parameters across multiple GPUs and allows fitting bigger models in the same amount of GPU memory.","title":"Multi GPU training"},{"location":"configuration/configuration/overview/","text":"Configuration Configurations are managed by hydra . And follow a structured configuration. The currently used configuration can be shown with: hannah-train -c job The default configuration is located under speech_recognition/conf/ . Configuration Structure There are two main locations for configuration: Framework configuration files are located under hannah/conf configuration files located in this directory should be useful for more than one project. Project specific configuration files are located in configs/ these usually contain a project specific configuration file and one or more configured experiments in the subfolder experiment make use of composition using defaults lists as much as possible Guideline for writing configuration files Consider avoiding project specific main configuration files, and try to configure your project using experiments only. Use composition defaults list composition as much as possible. Configuration Groups We currently have the following configuration groups: backend Configures the app to run a subset of validation and test on configured backend Choices are: null (no backend, default), torchmobile (translates model to backend) checkpoint Choices are: null (no checkpoint), all (default, creates checkpoints for all training epoch) best (checkpoint, best 5 values, default) compression Configuration for model compression. dataset Choices are: kws (For Google Keyword Spotting), snips (For hey snips dataset), vad (For Voice Activity Detection) Common configuration options for datasets are: data_folder base folder for data (actual data will usually be stored in $data_folder/dataset_name) cls speech_recognition.dataset.SpeechCommandsDataset dataset name of the dataset group_speakers_by_id true group the data by user id before splitting if available use_default_split false if the dataset has a predefined split, use it (ignores *_pct, and group by id parameters) test_pct Percentage of dataset used for test dev_pct Percentage of dataset used for validation train_pct Percentage of dataset used for training wanted_words Wanted words for keyword datasets input_length Length of input in number of samples samplingrate Sampling rate of data in Hz clear_download \"Remove downloaded archive after dataset has been extracted variants Variant of dataset to use for multilanguage datasets timeshift_ms \"Timeshift the input data by +- given ms\" extract loudest silence_prob % dataset samples that are silence unknown_prob % dataset samples that are unknown test_snr SNR used for test train_snr_high minimal SNR for training data train_snr_low maximal SNR for test data noise_dataset [\"TUT\", \"FSDKaggle\", \"FSDnoisy\"] Downloads all the specified datasets. Use TUT + one other data_split initial split after downloading the datasets(Possibilities: \"vad\", \"vad_speech\", \"vad_balanced\", \"getrennt\") downsample samplerate DESTRUCTIVE! change the samplerate of the real files to the target samplerate. Use better parameter samplingrate sensor resolution can be adjusted for resizing of images (currently only for vision capsule datasets). variants variants for kws - v1, v2 variants for snips - snips variants for vad - UWNU - Mozilla has the following language options: - en: Englisch - de: Detusch - fr: Franz\u00f6sisch - it: Italienisch - es: Spanisch - kab: Kabylisch - ca: Katalanisch - nl: Niderl\u00e4ndisch - eo: Esperanto - fa: Persisch - eu: Baskisch - rw: Kinyarwanda - ru: Russisch - pt: Portugiesisch - pl: Polnisch features Feature extractor to use, choices are: melspec, mfcc, raw, sinc, spectrogram . All feature extractors apart from sinc (Sinc Convolutions) currently use torchaudio feature extractors and use the same input parameters. Sinc Convolutions have the following parameters: normalizer Choices: null (default), fixedpoint (normalize feature values to [-1.0, 1.0]) Feature Normalizer run between feature extraction and the neural network model. model Neural network to train: choices are gds (Old sinc1 with sinc convolutions removed), lstm, tc-res8, branchy-tc-res8 module The toplevel module implementing the training and eval loops. We curently support the following Tasks: stream_classifier : Classification on multichannel 1D data streams image_classifier : Classification on Image Data optimizer Choices are: adadelta, adam, adamax, adamw, asgd, lbfgs, rmsprop, rprop, sgd, sparse_adam Directly instantiate the corresponding pytorch classes and take the same options. profiler Choices are: null , advanced, simple Run profiling for different phases. scheduler Choices are: null , 1cycle, cosine, cosine_warm, cyclic, exponential, multistep, plateau, step Learning rate scheduler to use for scheduling, default is null. trainer Choices: default, dds Capsules the options to the lightning trainer. Currently it sets the following defaults: Default options are: gpus 1 auto_select_gpus True limit_train_batches 1.0 limit_val_batches 1.0 limit_test_batches 1.0 max_epochs 80 default_root_dir . fast_dev_run false overfit_batches 0.0 benchmark True deterministic True Environment Variables The default configurations interpolate the following environment variables: HANNAH_CACHE_DIR Location of a directory used to cache file loading in some datasets HANNAH_DATA_FOLDER Change default location of dataset folders by default we will use subdirectory datasets of current working directory","title":"Overview"},{"location":"configuration/configuration/overview/#configuration","text":"Configurations are managed by hydra . And follow a structured configuration. The currently used configuration can be shown with: hannah-train -c job The default configuration is located under speech_recognition/conf/ .","title":"Configuration"},{"location":"configuration/configuration/overview/#configuration-structure","text":"There are two main locations for configuration: Framework configuration files are located under hannah/conf configuration files located in this directory should be useful for more than one project. Project specific configuration files are located in configs/ these usually contain a project specific configuration file and one or more configured experiments in the subfolder experiment make use of composition using defaults lists as much as possible","title":"Configuration Structure"},{"location":"configuration/configuration/overview/#guideline-for-writing-configuration-files","text":"Consider avoiding project specific main configuration files, and try to configure your project using experiments only. Use composition defaults list composition as much as possible.","title":"Guideline for writing configuration files"},{"location":"configuration/configuration/overview/#configuration-groups","text":"We currently have the following configuration groups:","title":"Configuration Groups"},{"location":"configuration/configuration/overview/#backend","text":"Configures the app to run a subset of validation and test on configured backend Choices are: null (no backend, default), torchmobile (translates model to backend)","title":"backend"},{"location":"configuration/configuration/overview/#checkpoint","text":"Choices are: null (no checkpoint), all (default, creates checkpoints for all training epoch) best (checkpoint, best 5 values, default)","title":"checkpoint"},{"location":"configuration/configuration/overview/#compression","text":"Configuration for model compression.","title":"compression"},{"location":"configuration/configuration/overview/#dataset","text":"Choices are: kws (For Google Keyword Spotting), snips (For hey snips dataset), vad (For Voice Activity Detection) Common configuration options for datasets are: data_folder base folder for data (actual data will usually be stored in $data_folder/dataset_name) cls speech_recognition.dataset.SpeechCommandsDataset dataset name of the dataset group_speakers_by_id true group the data by user id before splitting if available use_default_split false if the dataset has a predefined split, use it (ignores *_pct, and group by id parameters) test_pct Percentage of dataset used for test dev_pct Percentage of dataset used for validation train_pct Percentage of dataset used for training wanted_words Wanted words for keyword datasets input_length Length of input in number of samples samplingrate Sampling rate of data in Hz clear_download \"Remove downloaded archive after dataset has been extracted variants Variant of dataset to use for multilanguage datasets timeshift_ms \"Timeshift the input data by +- given ms\" extract loudest silence_prob % dataset samples that are silence unknown_prob % dataset samples that are unknown test_snr SNR used for test train_snr_high minimal SNR for training data train_snr_low maximal SNR for test data noise_dataset [\"TUT\", \"FSDKaggle\", \"FSDnoisy\"] Downloads all the specified datasets. Use TUT + one other data_split initial split after downloading the datasets(Possibilities: \"vad\", \"vad_speech\", \"vad_balanced\", \"getrennt\") downsample samplerate DESTRUCTIVE! change the samplerate of the real files to the target samplerate. Use better parameter samplingrate sensor resolution can be adjusted for resizing of images (currently only for vision capsule datasets).","title":"dataset"},{"location":"configuration/configuration/overview/#variants","text":"variants for kws - v1, v2 variants for snips - snips variants for vad - UWNU - Mozilla has the following language options: - en: Englisch - de: Detusch - fr: Franz\u00f6sisch - it: Italienisch - es: Spanisch - kab: Kabylisch - ca: Katalanisch - nl: Niderl\u00e4ndisch - eo: Esperanto - fa: Persisch - eu: Baskisch - rw: Kinyarwanda - ru: Russisch - pt: Portugiesisch - pl: Polnisch","title":"variants"},{"location":"configuration/configuration/overview/#features","text":"Feature extractor to use, choices are: melspec, mfcc, raw, sinc, spectrogram . All feature extractors apart from sinc (Sinc Convolutions) currently use torchaudio feature extractors and use the same input parameters. Sinc Convolutions have the following parameters:","title":"features"},{"location":"configuration/configuration/overview/#normalizer","text":"Choices: null (default), fixedpoint (normalize feature values to [-1.0, 1.0]) Feature Normalizer run between feature extraction and the neural network model.","title":"normalizer"},{"location":"configuration/configuration/overview/#model","text":"Neural network to train: choices are gds (Old sinc1 with sinc convolutions removed), lstm, tc-res8, branchy-tc-res8","title":"model"},{"location":"configuration/configuration/overview/#module","text":"The toplevel module implementing the training and eval loops. We curently support the following Tasks: stream_classifier : Classification on multichannel 1D data streams image_classifier : Classification on Image Data","title":"module"},{"location":"configuration/configuration/overview/#optimizer","text":"Choices are: adadelta, adam, adamax, adamw, asgd, lbfgs, rmsprop, rprop, sgd, sparse_adam Directly instantiate the corresponding pytorch classes and take the same options.","title":"optimizer"},{"location":"configuration/configuration/overview/#profiler","text":"Choices are: null , advanced, simple Run profiling for different phases.","title":"profiler"},{"location":"configuration/configuration/overview/#scheduler","text":"Choices are: null , 1cycle, cosine, cosine_warm, cyclic, exponential, multistep, plateau, step Learning rate scheduler to use for scheduling, default is null.","title":"scheduler"},{"location":"configuration/configuration/overview/#trainer","text":"Choices: default, dds Capsules the options to the lightning trainer. Currently it sets the following defaults: Default options are: gpus 1 auto_select_gpus True limit_train_batches 1.0 limit_val_batches 1.0 limit_test_batches 1.0 max_epochs 80 default_root_dir . fast_dev_run false overfit_batches 0.0 benchmark True deterministic True","title":"trainer"},{"location":"configuration/configuration/overview/#environment-variables","text":"The default configurations interpolate the following environment variables: HANNAH_CACHE_DIR Location of a directory used to cache file loading in some datasets HANNAH_DATA_FOLDER Change default location of dataset folders by default we will use subdirectory datasets of current working directory","title":"Environment Variables"},{"location":"deployment/deployment/torchmobile/","text":"Torchmobile backend A backend that runs the trained neural network through torchmobile on the local cpu. This is an example implementation of backend that allows testing of backend integrations without installation of further packages. It should not be used for serious optimizations, as measurements are much to noisy. Configuration val_batches 1 (number of batches used for validation) test_batches 1 (number of batches used for test) val_frequency 10 (run backend every n validation epochs)","title":"Torch Mobile"},{"location":"deployment/deployment/torchmobile/#torchmobile-backend","text":"A backend that runs the trained neural network through torchmobile on the local cpu. This is an example implementation of backend that allows testing of backend integrations without installation of further packages. It should not be used for serious optimizations, as measurements are much to noisy.","title":"Torchmobile backend"},{"location":"deployment/deployment/torchmobile/#configuration","text":"val_batches 1 (number of batches used for validation) test_batches 1 (number of batches used for test) val_frequency 10 (run backend every n validation epochs)","title":"Configuration"},{"location":"deployment/deployment/trax-Ultratrail/","text":"Ultratrail backend To use the ultratrail backend use Install teda: https://atreus.informatik.uni-tuebingen.de/ties/t-rax/teda Set environment variable TEDA_HOME to path of teda checkout To run it use: hannah-train compress=fp_8_8_8 backend=trax_ut Configuration teda_dir ${env:TEDA_HOME,/local/frischkn/neural_networks/teda} backend_dir \"${backend.teda_dir}/rtl/tc-resnet8-accelerator\" standalone False rtl_simulation False synthesis False postsyn_simulation False power_estimation False num_inferences 1 bw_w 8 bw_b 8 bw_f 8 cols 8 rows 8 period 1000 macro_type \"RTL\" use_acc_statistic_model True use_acc_analytical_model False","title":"T-Rax Ultratrail"},{"location":"deployment/deployment/trax-Ultratrail/#ultratrail-backend","text":"To use the ultratrail backend use Install teda: https://atreus.informatik.uni-tuebingen.de/ties/t-rax/teda Set environment variable TEDA_HOME to path of teda checkout To run it use: hannah-train compress=fp_8_8_8 backend=trax_ut","title":"Ultratrail backend"},{"location":"deployment/deployment/trax-Ultratrail/#configuration","text":"teda_dir ${env:TEDA_HOME,/local/frischkn/neural_networks/teda} backend_dir \"${backend.teda_dir}/rtl/tc-resnet8-accelerator\" standalone False rtl_simulation False synthesis False postsyn_simulation False power_estimation False num_inferences 1 bw_w 8 bw_b 8 bw_f 8 cols 8 rows 8 period 1000 macro_type \"RTL\" use_acc_statistic_model True use_acc_analytical_model False","title":"Configuration"},{"location":"deployment/deployment/tvm/","text":"TVM backend A backend that runs the trained neural network through tvm with remote target support through automate. Configuration val_batches 1 (number of batches used for validation) test_batches 1 (number of batches used for test) val_frequency 10 (run backend every n validation epochs) board Target board configuration tuner optional tuner configuration Board Configuration MicroTVM connected boards or simulators Automate connected Remote Boards Tuner Autotuning is configure using the configuration group backend/tuner . There are 3 predefined tuner configurations: \u00e0utotvm Uses the standard configuration for autotuning on the target board, it is generallly the recommended setting for 8-bit quantized models on CPU like devices. As it allows the targets to use hand defined tuning templates, which generally include tvm tuning templates. auto_scheduler Uses the standard tvm auto_scheduler. This scheduler does not support tensorization and is therefore not recommended on targets that support tensorized instructions. This is mostly the case for tensorized instructions on the target boards. baseline This does not use any tuning but uses the predefined schedules directly. All tuner configurations are parametrizable using the following tuner configurations: All autotuning can be deactivated when setting backend.tune = false Tuner Configuration","title":"TVM"},{"location":"deployment/deployment/tvm/#tvm-backend","text":"A backend that runs the trained neural network through tvm with remote target support through automate.","title":"TVM backend"},{"location":"deployment/deployment/tvm/#configuration","text":"val_batches 1 (number of batches used for validation) test_batches 1 (number of batches used for test) val_frequency 10 (run backend every n validation epochs) board Target board configuration tuner optional tuner configuration","title":"Configuration"},{"location":"deployment/deployment/tvm/#board-configuration","text":"","title":"Board Configuration"},{"location":"deployment/deployment/tvm/#microtvm-connected-boards-or-simulators","text":"","title":"MicroTVM connected boards or simulators"},{"location":"deployment/deployment/tvm/#automate-connected-remote-boards","text":"","title":"Automate connected Remote Boards"},{"location":"deployment/deployment/tvm/#tuner","text":"Autotuning is configure using the configuration group backend/tuner . There are 3 predefined tuner configurations: \u00e0utotvm Uses the standard configuration for autotuning on the target board, it is generallly the recommended setting for 8-bit quantized models on CPU like devices. As it allows the targets to use hand defined tuning templates, which generally include tvm tuning templates. auto_scheduler Uses the standard tvm auto_scheduler. This scheduler does not support tensorization and is therefore not recommended on targets that support tensorized instructions. This is mostly the case for tensorized instructions on the target boards. baseline This does not use any tuning but uses the predefined schedules directly. All tuner configurations are parametrizable using the following tuner configurations: All autotuning can be deactivated when setting backend.tune = false","title":"Tuner"},{"location":"deployment/deployment/tvm/#tuner-configuration","text":"","title":"Tuner Configuration"},{"location":"development/devel/api/","text":"hannah.tools.objectdetection_eval eval_train def eval_train(config, module, test=True) Arguments : config - param module: test - Default value = True) module: eval_steps def eval_steps(config, module, hparams, checkpoint) Arguments : config - param module: hparams - param checkpoint: module: checkpoint: eval_checkpoint def eval_checkpoint(config: DictConfig, checkpoint) Arguments : config - DictConfig: checkpoint: config - DictConfig: config - DictConfig: eval def eval(config: DictConfig) Arguments : config - DictConfig: config - DictConfig: config - DictConfig: main @hydra.main(config_name=\"objectdetection_eval\", config_path=\"../conf\", version_base=\"1.2\") def main(config: DictConfig) Arguments : config - DictConfig: config - DictConfig: config - DictConfig: hannah.tools.train hannah.tools hannah.tools.characterize main @hydra.main(config_name=\"characterize\", config_path=\"../conf\", version_base=\"1.2\") def main(config: DictConfig) Arguments : config - DictConfig: config - DictConfig: config - DictConfig: hannah.tools.eval eval_checkpoint def eval_checkpoint(config: DictConfig, checkpoint) -> None Arguments : config - DictConfig: checkpoint: config - DictConfig: config - DictConfig: eval def eval(config: DictConfig) -> Optional[bool] Arguments : config - DictConfig: config - DictConfig: config - DictConfig: main @hydra.main(config_name=\"eval\", config_path=\"conf\", version_base=\"1.2\") def main(config: DictConfig) Arguments : config - DictConfig: config - DictConfig: config - DictConfig: hannah.trainer hannah.trainer.cross_validation hannah.train hannah.logo hannah.features MFCC Objects class MFCC(torchaudio.transforms.MFCC) A simple wrapper around torchaudio mfcc, but melkwargs are given as direct named arguments instead of a dictionary SincConv Objects class SincConv(nn.Module) Sinc convolution: Arguments : in_channels - No. of input channels(must be 1) out_channels - No. of filters(40) sample_rate - sampling rate, default set at 32000 kernel_size - Filter length(101) LogSpectrogram Objects class LogSpectrogram(torch.nn.Module) Create a spectrogram from a audio signal. Arguments : n_fft int, optional - Size of FFT, creates n_fft // 2 + 1 bins. (Default: 400 ) win_length int or None, optional - Window size. (Default: n_fft ) hop_length int or None, optional - Length of hop between STFT windows. (Default: win_length // 2 ) pad int, optional - Two sided padding of signal. (Default: 0 ) window_fn Callable[..., Tensor], optional - A function to create a window tensor that is applied/multiplied to each frame/window. (Default: torch.hann_window ) power float or None, optional - Exponent for the magnitude spectrogram, (must be > 0) e.g., 1 for energy, 2 for power, etc. If None, then the complex spectrum is returned instead. (Default: 2 ) normalized bool, optional - Whether to normalize by magnitude after stft. (Default: False ) wkwargs dict or None, optional - Arguments for window function. (Default: None ) center bool, optional - whether to pad :attr: waveform on both sides so that the :math: t -th frame is centered at time :math: t \\times \\text{hop\\_length} . Default - True pad_mode string, optional - controls the padding method used when :attr: center is True . Default: \"reflect\" onesided bool, optional - controls whether to return half of results to avoid redundancy Default: True forward def forward(waveform: torch.Tensor) -> torch.Tensor Arguments : waveform Tensor - Tensor of audio of dimension (..., time). Returns : Tensor - Dimension (..., freq, time), where freq is n_fft // 2 + 1 where n_fft is the number of Fourier bins, and time is the number of window hops (n_frame). hannah.normalizer FixedPointNormalizer Objects class FixedPointNormalizer(nn.Module) Simple feature normalizer for fixed point models AdaptiveFixedPointNormalizer Objects class AdaptiveFixedPointNormalizer(nn.Module) Simple feature normalizer for fixed point models hannah.models.honk hannah.models.honk.model truncated_normal def truncated_normal(tensor, std_dev=0.01) Arguments : tensor: - std_dev - (Default value = 0.01) SpeechResModel Objects class SpeechResModel(nn.Module) forward def forward(x) Arguments : x: SpeechModel Objects class SpeechModel(nn.Module) forward def forward(x) Arguments : x: hannah.models.hello.models DSConv2d Objects class DSConv2d(nn.Module) forward def forward(x) Arguments : x: DSCNNSpeechModel Objects class DSCNNSpeechModel(nn.Module) forward def forward(x) Arguments : x: DNNSpeechModel Objects class DNNSpeechModel(nn.Module) forward def forward(x) Arguments : x: hannah.models.hello hannah.models.objectdetection.loss bbox_iou def bbox_iou(box1, box2, x1y1x2y2=True, GIoU=False, DIoU=False, CIoU=False, eps=1e-7) Arguments : box1: box2: - x1y1x2y2 - (Default value = True) - GIoU - (Default value = False) - DIoU - (Default value = False) - CIoU - (Default value = False) - eps - (Default value = 1e-7) is_parallel def is_parallel(model) Arguments : model: BCEBlurWithLogitsLoss Objects class BCEBlurWithLogitsLoss(nn.Module) Arguments : eps - (Default value = 0.1) ) - # https://github.com/ultralytics/yolov3/issues/238#issuecomment-598028441# return positive: negative label smoothing BCE targetsreturn 1.0 - 0.5 * eps: 0.5 * epsclass BCEBlurWithLogitsLoss(nn.Module: ) - # https://github.com/ultralytics/yolov3/issues/238#issuecomment-598028441# return positive: forward def forward(pred, true) Arguments : pred: true: FocalLoss Objects class FocalLoss(nn.Module) forward def forward(pred, true) Arguments : pred: true: QFocalLoss Objects class QFocalLoss(nn.Module) forward def forward(pred, true) Arguments : pred: true: ComputeLoss Objects class ComputeLoss() build_targets def build_targets(p, targets) Arguments : p: targets: hannah.models.objectdetection.models FasterRCNN Objects class FasterRCNN(torch.nn.Module) forward def forward(x, y=None) Arguments : x: - y - (Default value = None) transformOutput def transformOutput(cocoGt, output, x, y) Arguments : cocoGt: output: x: y: UltralyticsYolo Objects class UltralyticsYolo(torch.nn.Module) forward def forward(x, y=None) Arguments : x: - y - (Default value = None) train def train(mode=True) Arguments : mode - (Default value = True) transformOutput def transformOutput(cocoGt, output, x, y) Arguments : cocoGt: output: x: y: hannah.models.objectdetection hannah.models.tc.models create_act def create_act(act, clipping_value) Arguments : act: clipping_value: ApproximateGlobalAveragePooling1D Objects class ApproximateGlobalAveragePooling1D(nn.Module) forward def forward(x) Arguments : x: TCResidualBlock Objects class TCResidualBlock(nn.Module) forward def forward(x) Arguments : x: TCResNetModel Objects class TCResNetModel(nn.Module) forward def forward(x) Arguments : x: ExitWrapperBlock Objects class ExitWrapperBlock(nn.Module) forward def forward(x) Arguments : x: BranchyTCResNetModel Objects class BranchyTCResNetModel(TCResNetModel) on_val def on_val() on_val_end def on_val_end() on_test def on_test() on_test_end def on_test_end() reset_stats def reset_stats() print_stats def print_stats() forward def forward(x) Arguments : x: get_loss_function def get_loss_function() hannah.models.tc hannah.models.convnet.models padding_expression def padding_expression(kernel_size, stride, dilation=1) Symbolically calculate padding such that for a given kernel_size, stride and dilation the padding is such that the output dimension is kept the same(stride=1) or halved(stride=2). Note: If the input dimension is 1 and stride = 2, the calculated padding will result in an output with also dimension 1. Parameters kernel_size : Union[int, Expression] stride : Union[int, Expression] dilation : Union[int, Expression], optional description , by default 1 Returns Expression hannah.models.convnet hannah.models.timm DefaultAnomalyDetector Objects class DefaultAnomalyDetector(nn.Module) forward def forward(x) Arguments : x: DefaultClassifierHead Objects class DefaultClassifierHead(nn.Module) forward def forward(x: torch.Tensor) -> torch.Tensor Arguments : x (torch.Tensor): Returns : Resulting torch.Tensor after applying classification DefaultProjectionHead Objects class DefaultProjectionHead(nn.Module) Default projection head for semi supervised classification learning forward def forward(x: torch.Tensor) -> torch.Tensor Forward function for default Projection Head Arguments : x - Input tensor Returns : output tensor DefaultDecoderHead Objects class DefaultDecoderHead(nn.Module) __init__ def __init__(latent_shape, input_shape) Default Decoder Head for autoencoders using TransposedConv2D Arguments : latent_shape(Tuple) - Shape (CxHxW) of the latent representation of the autoencoder input_shape(Tuple) - Shape (CxHxW) of the reconstructed image forward def forward(x) Arguments : x: TimmModel Objects class TimmModel(nn.Module) forward def forward(x: torch.Tensor) -> torch.Tensor Arguments : x - torch.Tensor: x - torch.Tensor: hannah.models hannah.models.lstm.models LSTMModel Objects class LSTMModel(nn.Module) Simple LSTM model. forward def forward(x) Arguments : x: hannah.models.lstm hannah.models.factory.reduction ReductionBlockAdd Objects class ReductionBlockAdd(nn.Module) Reduction block that sums over its inputs forward def forward(x: Tensor) -> Tensor Arguments : x - Tensor: x - Tensor: ReductionBlockConcat Objects class ReductionBlockConcat(nn.Module) Reduction block that concatenates its inputs forward def forward(x: Tensor) -> Tensor Arguments : x - Tensor: x - Tensor: hannah.models.factory.act DummyActivation Objects class DummyActivation(nn.Identity) Dummy class that instantiated to mark a missing activation. This can be used to mark requantization of activations for convolutional layers without activation functions. Arguments : hannah.models.factory hannah.models.factory.quantized Import from new loacation for backwards compatibility hannah.models.factory.qconfig Import from new loacation for backwards compatibility hannah.models.factory.network ConvNet Objects class ConvNet(nn.Module) forward def forward(x) Arguments : x: hannah.models.factory.factory A neural network model factory It allows us to construct quantized and unquantized versions of the same network, allows to explore implementation alternatives using a common neural network construction interface. NormConfig Objects @dataclass class NormConfig() BNConfig Objects @dataclass class BNConfig(NormConfig) ActConfig Objects @dataclass class ActConfig() ELUConfig Objects @dataclass class ELUConfig(ActConfig) HardtanhConfig Objects @dataclass class HardtanhConfig(ActConfig) MinorBlockConfig Objects @dataclass class MinorBlockConfig() target target Operation parallel execute block in parallel with preceding block out_channels number of output channels kernel_size kernel size of this Operation (if applicable) stride stride for this operation use padding use padding for this operation (padding will always try to keep input dimensions / stride) dilation dilation factor to use for this operation groups number of groups for this operation norm normalization to use (true uses networks default configs) act activation to use (true uses default configs) upsampling Upsampling factor for mbconv layers bias use bias for this operation out_quant use output quantization for this operation MajorBlockConfig Objects @dataclass class MajorBlockConfig() stride Union[None, int, Tuple[int, ...], Tuple[int, ...]] last Indicates wether this block is the last reduction block LinearConfig Objects @dataclass class LinearConfig() norm Union[bool, NormConfig] act Union[bool, ActConfig] NetworkConfig Objects @dataclass class NetworkConfig() NetworkFactory Objects class NetworkFactory() act def act(config: ActConfig) -> nn.Module Arguments : config - ActConfig: config - ActConfig: conv2d def conv2d(input_shape: Tuple[int, ...], out_channels: int, kernel_size: Union[int, Tuple[int, ...]], stride: Union[int, Tuple[int, ...]] = 1, padding: Union[int, Tuple[int, ...], bool] = True, dilation: Union[int, Tuple[int, ...]] = 0, groups: int = 1, norm: Union[BNConfig, bool] = False, act: Union[ActConfig, bool] = False, bias: bool = False) -> None Arguments : input_shape - Tuple[int: int: int]: out_channels - int: kernel_size - Union[int: Tuple[int, ...]]: (Default value = 1) stride - Union[int: padding - Union[int: Tuple[int, ...]: bool] - (Default value = False) dilation - int: (Default value = 0) groups - int: (Default value = 1) norm - Union[BNConfig: act - Union[ActConfig: bias - bool: (Default value = False) input_shape - Tuple[int: out_channels - int: kernel_size - Union[int: stride - Union[int: padding - Union[int: dilation - int: (Default value = 0) groups - int: (Default value = 1) norm - Union[BNConfig: act - Union[ActConfig: bias - bool: (Default value = False) mbconv1d def mbconv1d(input_shape: Tuple[int, ...], out_channels: int, kernel_size: int, dilation: int = 1, stride: int = 1, padding: Union[int, bool] = True, bias=False, upsampling: float = 1.0, groups: int = 1, norm: Union[BNConfig, bool] = False, act: Union[ActConfig, bool] = False) Arguments : input_shape - Tuple[int: int: int]: out_channels - int: kernel_size - int: dilation - int: (Default value = 1) stride - int: (Default value = 1) padding - Union[int: bool] - (Default value = False) bias - (Default value = False) upsampling - float: (Default value = 1.0) groups - int: (Default value = 1) norm - Union[BNConfig: act - Union[ActConfig: input_shape - Tuple[int: out_channels - int: kernel_size - int: dilation - int: (Default value = 1) stride - int: (Default value = 1) padding - Union[int: upsampling - float: (Default value = 1.0) groups - int: (Default value = 1) norm - Union[BNConfig: act - Union[ActConfig: conv1d def conv1d(input_shape: Tuple[int, ...], out_channels: int, kernel_size: int, stride: int = 1, bias: bool = False, padding: Union[int, bool] = True, dilation: int = 1, groups: int = 1, norm: Union[BNConfig, bool] = False, act: Union[ActConfig, bool] = False, out_quant: bool = True) -> Tuple[Tuple[int, ...], nn.Module] Arguments : input_shape - Tuple[int: int: int]: out_channels - int: kernel_size - int: stride - int: (Default value = 1) bias - bool: (Default value = False) padding - Union[int: bool] - (Default value = False) dilation - int: (Default value = 1) groups - int: (Default value = 1) norm - Union[BNConfig: act - Union[ActConfig: out_quant - bool: (Default value = True) input_shape - Tuple[int: out_channels - int: kernel_size - int: stride - int: (Default value = 1) bias - bool: (Default value = False) padding - Union[int: dilation - int: (Default value = 1) groups - int: (Default value = 1) norm - Union[BNConfig: act - Union[ActConfig: out_quant - bool: (Default value = True) minor def minor(input_shape, config: MinorBlockConfig, major_stride=None) Arguments : input_shape: - config - MinorBlockConfig: - major_stride - (Default value = None) - config - MinorBlockConfig: forward def forward(input_shape: Tuple[int, ...], config: MajorBlockConfig) Create a forward neural network block without parallelism If parallel is set to [True, False, True, False] Input: ------->|---> parallel: False ---> parallel: False ---> | --> output Arguments : input_shape - Tuple[int, ...]: config - MajorBlockConfig: input_shape - Tuple[int, ...]: config - MajorBlockConfig: residual def residual(input_shape: Tuple[int, ...], config: MajorBlockConfig) Create a neural network block with with residual parallelism If parallel is set to [True, False, True, False] |---> parallel: True ---> parallel: True ---> | Input: ------->| +---> |---> parallel: False ---> parallel: False ---> | If the major block does change the output dimensions compared to the input and one of the branches does not contain any layers, we infer 1x1 conv of maximum group size (gcd (input_channels, output_channels)) to do the downsampling. Arguments : input_shape - Tuple[int, ...]: config - MajorBlockConfig: input_shape - Tuple[int, ...]: config - MajorBlockConfig: input def input(in_channels: int, config: MajorBlockConfig) Create a neural network block with input parallelism If parallel is set to [True, False, True, False] |---> parallel: True ---> | |---> parallel: True ---> + -----------------> | Input:--------->| +---> |---> parallel: False ---> parallel: False ---> | If there are no parallel branches in the network. The major block is a standard feed forward layer. Arguments : in_channels - int: config - MajorBlockConfig: in_channels - int: config - MajorBlockConfig: full def full(in_channels: int, config: MajorBlockConfig) Create a neural network block with full parallelism If parallel is set to [True, False, True, False] |---> parallel: True ---------------------------------- -| Input:--->| +---> | |--> parallel: False --->| | |---> parallel: False ----> | +--->| |--> parallel: True ---->| If there are no parallel blocks the block is a standard feed forward network. Arguments : in_channels - int: config - MajorBlockConfig: in_channels - int: config - MajorBlockConfig: major def major(input_shape, config: MajorBlockConfig) Arguments : input_shape: - config - MajorBlockConfig: - config - MajorBlockConfig: linear def linear(input_shape, config: LinearConfig) Arguments : input_shape: - config - LinearConfig: - config - LinearConfig: identity def identity() -> Identity network def network(input_shape, labels: int, network_config: Union[ListConfig, DictConfig]) Arguments : input_shape: - labels - int: - network_config - NetworkConfig: - labels - int: - network_config - NetworkConfig: create_cnn def create_cnn(input_shape: Sequence[int], labels: int, name: str, conv: Optional[List[MajorBlockConfig]] = None, linear: Optional[List[LinearConfig]] = None, norm: Optional[NormConfig] = None, act: Optional[ActConfig] = None, qconfig: Any = None, dropout: float = 0.5) Arguments : input_shape - Sequence[int]: labels - int: name - str: conv - Optional[List[MajorBlockConfig]]: (Default value = None) linear - Optional[List[LinearConfig]]: (Default value = None) norm - Optional[NormConfig]: (Default value = None) act - Optional[ActConfig]: (Default value = None) qconfig - Any: (Default value = None) dropout - float: (Default value = 0.5) input_shape - Sequence[int]: labels - int: name - str: conv - Optional[List[MajorBlockConfig]]: (Default value = None) linear - Optional[List[LinearConfig]]: (Default value = None) norm - Optional[NormConfig]: (Default value = None) act - Optional[ActConfig]: (Default value = None) qconfig - Any: (Default value = None) dropout - float: (Default value = 0.5) hannah.models.factory.pooling ApproximateGlobalAveragePooling1D Objects class ApproximateGlobalAveragePooling1D(nn.Module) A global average pooling layer, that divides by the next power of 2 instead of true number of elements forward def forward(x) Arguments : x: ApproximateGlobalAveragePooling2D Objects class ApproximateGlobalAveragePooling2D(nn.Module) A global average pooling layer, that divides by the next power of 2 instead of true number of elements forward def forward(x) Arguments : x: hannah.models.factory.qat Import from new loacation for backwards compatibility hannah.models.factory.rounding Import from new loacation for backwards compatibility hannah.models.resnet.models padding_expression def padding_expression(kernel_size, stride, dilation=1) Symbolically calculate padding such that for a given kernel_size, stride and dilation the padding is such that the output dimension is kept the same(stride=1) or halved(stride=2). Note: If the input dimension is 1 and stride = 2, the calculated padding will result in an output with also dimension 1. Parameters kernel_size : Union[int, Expression] stride : Union[int, Expression] dilation : Union[int, Expression], optional description , by default 1 Returns Expression hannah.models.resnet hannah.models.vad.models BottleneckVad Objects class BottleneckVad(nn.Module) forward def forward(x) Arguments : x: num_flat_features def num_flat_features(x) Arguments : x: SmallVad Objects class SmallVad(nn.Module) forward def forward(x) Arguments : x: num_flat_features def num_flat_features(x) Arguments : x: SimpleVad Objects class SimpleVad(nn.Module) forward def forward(x) Arguments : x: num_flat_features def num_flat_features(x) Arguments : x: BottleneckVadModel Objects class BottleneckVadModel(nn.Module) forward def forward(x) Arguments : x: SimpleVadModel Objects class SimpleVadModel(nn.Module) forward def forward(x) Arguments : x: SmallVadModel Objects class SmallVadModel(nn.Module) forward def forward(x) Arguments : x: hannah.models.vad hannah.models.ofa.submodules.elasticLinear ElasticWidthLinear Objects class ElasticWidthLinear(nn.Linear, _Elastic) forward def forward(input: torch.Tensor) -> torch.Tensor Arguments : input - torch.Tensor: input - torch.Tensor: input - torch.Tensor: input - torch.Tensor: get_basic_module def get_basic_module() ElasticQuantWidthLinear Objects class ElasticQuantWidthLinear(nn.Linear, _Elastic) filtered_weight @property def filtered_weight() filtered_bias @property def filtered_bias() scaled_weight @property def scaled_weight() scaled_bias @property def scaled_bias() forward def forward(input: torch.Tensor) -> torch.Tensor Arguments : input - torch.Tensor: input - torch.Tensor: input - torch.Tensor: input - torch.Tensor: get_basic_module def get_basic_module() from_float @classmethod def from_float(cls, mod) Create a qat module from a float module or qparams_dict Args: mod a float module, either produced by torch.quantization utilities or directly from user Arguments : mod: ElasticPermissiveReLU Objects class ElasticPermissiveReLU(nn.ReLU) forward def forward(x) Arguments : x: assemble_basic_module def assemble_basic_module() hannah.models.ofa.submodules.elasticBase _Elastic Objects class _Elastic() get_basic_module def get_basic_module() -> nn.Module get_out_channel_sizes def get_out_channel_sizes() set_out_channel_filter def set_out_channel_filter(out_channel_filter) Arguments : ) -> nn.Module: # Module: so that Sequentials are possible: like DSCreturn copy.deepcopy(self.get_basic_module(): out_channel_filter: ) -> nn.Module: # Module: like DSCreturn copy.deepcopy(self.get_basic_module(): ) -> nn.Module: # Module: like DSCreturn copy.deepcopy(self.get_basic_module(): ) -> nn.Module: # Module: like DSCreturn copy.deepcopy(self.get_basic_module())set_out_channel_filter(self: set_in_channel_filter def set_in_channel_filter(in_channel_filter) Arguments : in_channel_filter: ElasticBase1d Objects class ElasticBase1d(nn.Conv1d, _Elastic) set_bn_parameter def set_bn_parameter(conv: nn.Conv1d, tmp_bn, num_tracked) Caller for BatchNorm Parameters This unifies the call in the different methods, especially in dsc / not dsc forward And assigns the attributes in tmp_bn to the param conv Arguments : conv - nn.Conv1d: tmp_bn: num_tracked: conv - nn.Conv1d: conv - nn.Conv1d: conv - nn.Conv1d: prepare_dsc_for_validation_model def prepare_dsc_for_validation_model(conv_class: nn.Module, full_kernel, full_bias, in_channels, out_channels, grouping, stride, padding, dilation, qconfig=None, out_quant=None, bn_eps=None, bn_momentum=None, bn_caller: tuple = None) This method creates the necessary validation models for DSC. It creates the validation model as torch.Sequence of standard pytorch convolution models. The structure is analog to the DSC method do_dsc. This method can also handle quantization models. Arguments : conv_class - nn.Module: full_kernel: full_bias: in_channels: out_channels: grouping: stride: padding: dilation: # for quantqconfig: (Default value = None) out_quant - (Default value = None) bn_eps - (Default value = None) bn_momentum - (Default value = None) bn_caller - tuple: (Default value = None) conv_class - nn.Module: bn_caller - tuple: (Default value = None) conv_class - nn.Module: bn_caller - tuple: (Default value = None) conv_class - nn.Module: bn_caller - tuple: (Default value = None) do_dsc def do_dsc(input, full_kernel, full_bias, grouping, stride, padding, dilation, quant_weight=None, quant_bias=None, quant_weight_function: Function = None, quant_bias_function: Function = None) This method will perform the DSC(=Depthwise Separable Convolution). This method can also handle quantized models. DSC is done in two steps: 1. Depthwise Separable: Set Group = In_Channels, Output = k*In_Channels 2. Pointwise Convolution, with Grouping = Grouping-Param und Out_Channel = Out_Channel-Param The Params above are used for quantized models Arguments : input: full_kernel: full_bias: grouping: stride: padding: dilation: - quant_weight - (Default value = None) - quant_bias - (Default value = None) - quant_weight_function - Function: (Default value = None) # for quantizationquant_bias_function: Function: (Default value = None) # for quantization: - quant_weight_function - Function: (Default value = None) # for quantizationquant_bias_function: Function: (Default value = None) - quant_weight_function - Function: (Default value = None) # for quantizationquant_bias_function: Function: (Default value = None) - quant_weight_function - Function: (Default value = None) # for quantizationquant_bias_function: Function: (Default value = None) set_in_and_out_channel def set_in_and_out_channel(kernel, filtered: bool = True) This method uses the kernel for setting the input and outputchannel if dynamic width is activated (channelfilters), the amount of channels is reduced, hence we can't use the initial values (self.(in/out)_channel) of the constructor This method sets the self.(in/out)_channel value to the right amount of channels extracted from the kernel that will be used. if filtered is False, the self.initial_(in/out)_channels will be used. The previous values will be stored in the attribute prev_in_channels and prev_out_channels. Arguments : kernel: - filtered - bool: (Default value = True) - filtered - bool: (Default value = True) - filtered - bool: (Default value = True) - filtered - bool: (Default value = True) reset_in_and_out_channel_to_previous def reset_in_and_out_channel_to_previous() Analog to set_in_and_out_channels: Resets the in and out_channels Arguments : set_kernel_size def set_kernel_size(new_kernel_size) If the requested kernel size is outside of the min/max range, clamp it to the min/max range. If the requested kernel size is not an available kernel size, default to the max kernel size Arguments : new_kernel_size - int): the size of the kernel you want to use reset_kernel_size def reset_kernel_size() step_down_kernel_size def step_down_kernel_size() pick_kernel_index def pick_kernel_index(target_kernel_index: int) Arguments : target_kernel_index - int: target_kernel_index - int: target_kernel_index - int: target_kernel_index - int: get_available_kernel_steps def get_available_kernel_steps() get_full_width_kernel def get_full_width_kernel() It applies the kernel transformations to the kernel until the target kernel index is reached Returns : The found target kernel. Args: Returns: get_kernel def get_kernel() If the input and output channels are not filtered, the full kernel is Arguments : Returns : return: The new kernel and bias. set_dilation_size def set_dilation_size(new_dilation_size) Arguments : new_dilation_size: update_padding def update_padding() reset_dilation_size def reset_dilation_size() step_down_dilation_size def step_down_dilation_size() pick_dilation_index def pick_dilation_index(target_dilation_index: int) Arguments : target_dilation_index - int: target_dilation_index - int: target_dilation_index - int: target_dilation_index - int: get_available_dilation_steps def get_available_dilation_steps() get_available_grouping_steps def get_available_grouping_steps() get_available_dsc_steps def get_available_dsc_steps() get_dilation_size def get_dilation_size() pick_group_index def pick_group_index(target_group_index: int) Arguments : target_group_index - int: target_group_index - int: target_group_index - int: target_group_index - int: pick_dsc_index def pick_dsc_index(target_dsc_index: int) Arguments : target_dsc_index - int: target_dsc_index - int: target_dsc_index - int: target_dsc_index - int: reset_group_size def reset_group_size() reset_dscs def reset_dscs() get_group_size def get_group_size() get_dsc def get_dsc() set_group_size def set_group_size(new_group_size) Arguments : new_group_size: set_dsc def set_dsc(new_dsc) Arguments : new_dsc: step_down_group_size def step_down_group_size() step_down_dsc def step_down_dsc() adjust_weights_for_grouping def adjust_weights_for_grouping(weights, input_divided_by) Arguments : weights: input_divided_by: forward def forward(input: torch.Tensor) -> torch.Tensor Arguments : input - torch.Tensor: input - torch.Tensor: input - torch.Tensor: input - torch.Tensor: extra_repr def extra_repr() hannah.models.ofa.submodules.resblock ResBlockBase Objects class ResBlockBase(nn.Module) forward def forward(x) Arguments : x: get_nested_modules def get_nested_modules() ResBlock1d Objects class ResBlock1d(ResBlockBase) forward def forward(x) Arguments : x: get_input_layer def get_input_layer() get_output_layer def get_output_layer() create_internal_channelhelper def create_internal_channelhelper() hannah.models.ofa.submodules hannah.models.ofa.submodules.elasticBatchnorm ElasticWidthBatchnorm1d Objects class ElasticWidthBatchnorm1d(nn.BatchNorm1d) forward def forward(input: torch.Tensor) -> torch.Tensor if self.track_running_stats: logging.warn( \"ElasticWidthBatchnorm with tracked running stats currently not fully implemented!\" ) num_batches_tracked and exponential averaging are currently not implemented. Arguments : input - torch.Tensor: input - torch.Tensor: input - torch.Tensor: input - torch.Tensor: get_basic_batchnorm1d def get_basic_batchnorm1d() assemble_basic_module def assemble_basic_module() -> nn.BatchNorm1d hannah.models.ofa.submodules.elasticchannelhelper ElasticChannelHelper Objects class ElasticChannelHelper(nn.Module) compute_channel_priorities def compute_channel_priorities() set_channel_filter def set_channel_filter() apply_filter_to_module def apply_filter_to_module(module, is_target: bool) Arguments : module: - is_target - bool: - is_target - bool: - is_target - bool: - is_target - bool: step_down_channels def step_down_channels() set_channel_step def set_channel_step(step: int) Arguments : step - int: step - int: step - int: step - int: reset_channel_step def reset_channel_step() set_primary_target def set_primary_target(target: nn.Module) Arguments : target - nn.Module: target - nn.Module: target - nn.Module: target - nn.Module: is_valid_primary_target def is_valid_primary_target(module: nn.Module) -> bool Arguments : module - nn.Module: module - nn.Module: module - nn.Module: module - nn.Module: is_primary_target def is_primary_target(module: nn.Module) -> bool Arguments : module - nn.Module: module - nn.Module: module - nn.Module: module - nn.Module: add_secondary_targets def add_secondary_targets(target: nn.Module) Arguments : target - nn.Module: target - nn.Module: target - nn.Module: target - nn.Module: add_secondary_target_item def add_secondary_target_item(target: nn.Module) Arguments : target - nn.Module: target - nn.Module: target - nn.Module: target - nn.Module: add_sources def add_sources(source: nn.Module) Arguments : source - nn.Module: source - nn.Module: source - nn.Module: source - nn.Module: add_targets def add_targets(target: nn.Module) Arguments : target - nn.Module: target - nn.Module: target - nn.Module: target - nn.Module: add_source_item def add_source_item(source: nn.Module) Arguments : source - nn.Module: source - nn.Module: source - nn.Module: source - nn.Module: discover_target def discover_target(new_target: nn.Module) Arguments : new_target - nn.Module: new_target - nn.Module: new_target - nn.Module: new_target - nn.Module: get_available_width_steps def get_available_width_steps() hannah.models.ofa.submodules.elastickernelconv ElasticConv1d Objects class ElasticConv1d(ElasticBase1d) forward def forward(input: torch.Tensor) -> torch.Tensor Arguments : input - torch.Tensor: input - torch.Tensor: input - torch.Tensor: input - torch.Tensor: get_basic_module def get_basic_module() -> nn.Module ElasticConvReLu1d Objects class ElasticConvReLu1d(ElasticBase1d) forward def forward(input: torch.Tensor) -> torch.Tensor Arguments : input - torch.Tensor: input - torch.Tensor: input - torch.Tensor: input - torch.Tensor: get_basic_module def get_basic_module() -> nn.Module ElasticConvBn1d Objects class ElasticConvBn1d(ElasticConv1d) forward def forward(input: torch.Tensor) -> torch.Tensor Arguments : input - torch.Tensor: input - torch.Tensor: input - torch.Tensor: input - torch.Tensor: get_basic_module def get_basic_module() -> nn.Module ElasticConvBnReLu1d Objects class ElasticConvBnReLu1d(ElasticConvBn1d) forward def forward(input: torch.Tensor) -> torch.Tensor Arguments : input - torch.Tensor: input - torch.Tensor: input - torch.Tensor: input - torch.Tensor: get_basic_module def get_basic_module() -> nn.Module ConvRelu1d Objects class ConvRelu1d(nn.Conv1d) forward def forward(input: torch.Tensor) -> torch.Tensor Arguments : input - torch.Tensor: input - torch.Tensor: input - torch.Tensor: input - torch.Tensor: ConvBn1d Objects class ConvBn1d(nn.Conv1d) forward def forward(input: torch.Tensor) -> torch.Tensor Arguments : input - torch.Tensor: input - torch.Tensor: input - torch.Tensor: input - torch.Tensor: ConvBnReLu1d Objects class ConvBnReLu1d(ConvBn1d) forward def forward(input: torch.Tensor) -> torch.Tensor Arguments : input - torch.Tensor: input - torch.Tensor: input - torch.Tensor: input - torch.Tensor: hannah.models.ofa.submodules.elasticquantkernelconv QuadDataHelper Objects class QuadDataHelper() Data Container so that _forward and _dsc has the same data. _ElasticConvBnNd Objects class _ElasticConvBnNd(ElasticBase1d, qat._ConvForwardMixin) on_warmup_end def on_warmup_end() reset_running_stats def reset_running_stats() reset_bn_parameters def reset_bn_parameters() reset_parameters def reset_parameters() update_bn_stats def update_bn_stats() freeze_bn_stats def freeze_bn_stats() scale_factor @property def scale_factor() full_scale_factor @property def full_scale_factor() does the same as scale_factor but uses the whole kernel. Used for dsc scaled_weight @property def scaled_weight() get_full_kernel_bias def get_full_kernel_bias() Gets the full kernel and bias. Used for dsc extra_repr def extra_repr() forward def forward(input) Arguments : input: train def train(mode=True) Batchnorm's training behavior is using the self.training flag. Prevent changing it if BN is frozen. This makes sure that calling model.train() on a model with a frozen BN will behave properly. Arguments : mode - (Default value = True) from_float @classmethod def from_float(cls, mod) Create a qat module from a float module or qparams_dict Args: mod a float module, either produced by torch.quantization utilities or directly from user Arguments : mod: ElasticQuantConv1d Objects class ElasticQuantConv1d(ElasticBase1d, qat._ConvForwardMixin) forward def forward(input: torch.Tensor) -> torch.Tensor Arguments : input - torch.Tensor: input - torch.Tensor: input - torch.Tensor: input - torch.Tensor: get_basic_module def get_basic_module() -> nn.Module ElasticQuantConvReLu1d Objects class ElasticQuantConvReLu1d(ElasticBase1d, qat._ConvForwardMixin) forward def forward(input: torch.Tensor) -> torch.Tensor Arguments : input - torch.Tensor: input - torch.Tensor: input - torch.Tensor: input - torch.Tensor: get_basic_module def get_basic_module() -> nn.Module ElasticQuantConvBn1d Objects class ElasticQuantConvBn1d(_ElasticConvBnNd) forward def forward(input: torch.Tensor) -> torch.Tensor Arguments : input - torch.Tensor: input - torch.Tensor: input - torch.Tensor: input - torch.Tensor: get_basic_module def get_basic_module() -> nn.Module ElasticQuantConvBnReLu1d Objects class ElasticQuantConvBnReLu1d(ElasticQuantConvBn1d) forward def forward(input: torch.Tensor) -> torch.Tensor Arguments : input - torch.Tensor: input - torch.Tensor: input - torch.Tensor: input - torch.Tensor: get_basic_module def get_basic_module() -> nn.Module hannah.models.ofa.models create def create(name: str, labels: int, input_shape, conv=[], min_depth: int = 1, norm_before_act=True, skew_sampling_distribution: bool = False, dropout: int = 0.5, validate_on_extracted=True, qconfig=None) -> nn.Module The function creates a ofa Model with the given name, labels, input shape, convolutional layers, and other parameters Arguments : name(str) - The name of the model labels(int) - The number of classes in the dataset input_shape - the shape of the input tensor conv - a list of MajorBlockConfig objects (Default value = []) min_depth(int optional - The minimum depth of the model, defaults to 1 norm_before_act - If True, the normalization is performed before the activation function, defaults to True (optional) skew_sampling_distribution(bool optional - If True, the model will use a skewed sampling distribution to sample the number of minor blocks in each major block, defaults to False dropout(int) - float, default 0.5 validate_on_extracted - If True, the model will be validated on the extracted data, defaults to True (optional) qconfig - the quantization configuration to use (Default value = None) name - str: labels - int: min_depth - int: (Default value = 1) skew_sampling_distribution - bool: (Default value = False) dropout - int: (Default value = 0.5) name - str: labels - int: min_depth - int: (Default value = 1) skew_sampling_distribution - bool: (Default value = False) dropout - int: (Default value = 0.5) name - str: labels - int: min_depth - int: (Default value = 1) skew_sampling_distribution - bool: (Default value = False) dropout - int: (Default value = 0.5) Returns : A model object. create_minor_block_sequence def create_minor_block_sequence(blocks, in_channels, stride=1, norm_before_act=True, qconfig=None) Arguments : blocks: in_channels: - stride - (Default value = 1) - norm_before_act - (Default value = True) - qconfig - (Default value = None) # sources: List[nn.Module]: (Default value = [nn.ModuleList([])]) # sources: List[nn.Module]: (Default value = [nn.ModuleList([])]) # sources: List[nn.Module]: (Default value = [nn.ModuleList([])]) create_minor_block def create_minor_block(block_config, in_channels: int, stride: int = 1, norm_before_act=True, sources: List[nn.ModuleList] = [nn.ModuleList([])], qconfig=None) -> Tuple[nn.Module, int] Arguments : block_config: - in_channels - int: - stride - int: (Default value = 1) - norm_before_act - (Default value = True) - sources - List[nn.ModuleList]: (Default value = [nn.ModuleList([])]) - qconfig - (Default value = None) - in_channels - int: - stride - int: (Default value = 1) - sources - List[nn.ModuleList]: (Default value = [nn.ModuleList([])]) - in_channels - int: - stride - int: (Default value = 1) - sources - List[nn.ModuleList]: (Default value = [nn.ModuleList([])]) create_residual_block_1d def create_residual_block_1d(blocks, in_channels, stride=1, norm_before_act=None, qconfig=None, quant_skip=None) -> ResBlock1d Arguments : blocks: in_channels: - stride - (Default value = 1) - norm_before_act - (Default value = None) - qconfig - (Default value = None) - quant_skip - (Default value = None) # sources: List[nn.ModuleList]: (Default value = [nn.ModuleList([])]) # sources: List[nn.ModuleList]: (Default value = [nn.ModuleList([])]) # sources: List[nn.ModuleList]: (Default value = [nn.ModuleList([])]) OFAModel Objects class OFAModel(nn.Module) extract_conv def extract_conv(conv, general_config, parallel=True, bp=False) Arguments : conv: general_config: - parallel - (Default value = True) - bp - (Default value = False) extract_config def extract_config(conv, general_config, bp=False) set all conv attributes in config Arguments : conv: general_config: - bp - (Default value = False) print_config def print_config(filename) Arguments : filename: forward def forward(x) Arguments : x: perform_sequence_discovery def perform_sequence_discovery() get_post_conv def get_post_conv(post_block) Arguments : post_block: get_pre_conv def get_pre_conv(pre_block) Arguments : pre_block: sample_subnetwork def sample_subnetwork() get_random_step def get_random_step(upper_bound: int) -> int Arguments : upper_bound - int: upper_bound - int: upper_bound - int: get_max_submodel_steps def get_max_submodel_steps() get_submodel def get_submodel(state: dict) Arguments : state - dict: state - dict: state - dict: on_warmup_end def on_warmup_end() set_submodel def set_submodel(state: dict) Arguments : state - dict: state - dict: state - dict: build_validation_model def build_validation_model() reset_validation_model def reset_validation_model() get_validation_model_weight_count def get_validation_model_weight_count() extract_elastic_depth_sequence def extract_elastic_depth_sequence(target_depth, quantized=False, clone_mode=False) Arguments : target_depth: - quantized - (Default value = False) - clone_mode - (Default value = False) extract_module_from_depth_step def extract_module_from_depth_step(depth_step) -> nn.Module Arguments : depth_step: get_elastic_depth_output def get_elastic_depth_output(target_depth=None, quantized=False) Arguments : target_depth - (Default value = None) quantized - (Default value = False) step_down_all_channels def step_down_all_channels() reset_active_depth def reset_active_depth() resume_active_elastic_values def resume_active_elastic_values() update_output_channel_count def update_output_channel_count() get_output_linear_layer def get_output_linear_layer(target_depth) Arguments : target_depth: step_down_all_kernels def step_down_all_kernels() reset_all_kernel_sizes def reset_all_kernel_sizes() reset_all_dilation_sizes def reset_all_dilation_sizes() reset_all_group_sizes def reset_all_group_sizes() reset_all_dsc def reset_all_dsc() step_down_all_dilations def step_down_all_dilations() step_down_all_groups def step_down_all_groups() step_down_all_dsc def step_down_all_dsc() go_to_kernel_step def go_to_kernel_step(step: int) Arguments : step - int: step - int: step - int: resume_kernel_sizes_from_step def resume_kernel_sizes_from_step() reset_all_widths def reset_all_widths() progressive_shrinking_add_kernel def progressive_shrinking_add_kernel() progressive_shrinking_add_dilation def progressive_shrinking_add_dilation() progressive_shrinking_add_depth def progressive_shrinking_add_depth() progressive_shrinking_add_group def progressive_shrinking_add_group() progressive_shrinking_add_dsc def progressive_shrinking_add_dsc() progressive_shrinking_compute_channel_priorities def progressive_shrinking_compute_channel_priorities() progressive_shrinking_add_width def progressive_shrinking_add_width() progressive_shrinking_disable_sampling def progressive_shrinking_disable_sampling() reset_shrinking def reset_shrinking() rebuild_extracted_blocks def rebuild_extracted_blocks(blocks) Arguments : blocks: hannah.models.ofa hannah.models.ofa.type_utils hannah.models.ofa.utilities conv1d_auto_padding def conv1d_auto_padding(conv1d: nn.Conv1d) Arguments : conv1d - nn.Conv1d: conv1d - nn.Conv1d: conv1d - nn.Conv1d: conv1d - nn.Conv1d: conv1d_get_padding def conv1d_get_padding(kernel_size, dilation=1) Arguments : kernel_size: - dilation - (Default value = 1) sub_filter_start_end def sub_filter_start_end(kernel_size, sub_kernel_size) Arguments : kernel_size: sub_kernel_size: flatten_module_list def flatten_module_list(modules: nn.Module) -> nn.Module Arguments : modules - nn.Module: modules - nn.Module: modules - nn.Module: modules - nn.Module: module_list_to_module def module_list_to_module(module_list) Arguments : module_list: call_function_from_deep_nested def call_function_from_deep_nested(input, function, type_selection: type = None) Arguments : input: function: - type_selection - type: (Default value = None) - type_selection - type: (Default value = None) - type_selection - type: (Default value = None) - type_selection - type: (Default value = None) get_instances_from_deep_nested def get_instances_from_deep_nested(input, type_selection: type = None) Arguments : input: - type_selection - type: (Default value = None) - type_selection - type: (Default value = None) - type_selection - type: (Default value = None) - type_selection - type: (Default value = None) filter_primary_module_weights def filter_primary_module_weights(weights, in_channel_filter, out_channel_filter) Arguments : weights: in_channel_filter: out_channel_filter: filter_single_dimensional_weights def filter_single_dimensional_weights(weights, channel_filter) Arguments : weights: channel_filter: make_parameter def make_parameter(t: torch.Tensor) -> nn.Parameter Arguments : t - torch.Tensor: t - torch.Tensor: t - torch.Tensor: t - torch.Tensor: adjust_weight_if_needed def adjust_weight_if_needed(module, kernel=None, groups=None) Adjust the weight if the adjustment is needded. This means, if the kernel does not have the size of (out_channel, in_channel / group, kernel). Arguments : kernel - the kernel that should be checked and adjusted if needed. If None module.weight.data will be used (Default value = None) grouping - value of the conv, if None module.groups will be used module - the conv :throws - RuntimeError if there is no last_grouping_param for comporing current group value to past group value returns (kernel, is adjusted) (adjusted if needed) otherwise throws a RuntimeError groups - (Default value = None) is_weight_adjusting_needed def is_weight_adjusting_needed(weights, input_channels, groups) Checks if a weight adjustment is needed Requirement: weight.shape[1] must be input_channels/groups true: weight adjustment is needed Arguments : weights - the weights that needs to be checked input_channels - Input Channels of the Convolution Module groups - Grouping Param of the Convolution Module get_target_weight def get_target_weight(weights, input_channels, groups) Gives the targeted weight shape (out_channel, in_channel // groups, kernel) Arguments : weights - the weights that needs to be checked input_channels - Input Channels of the Convolution Module groups - Grouping Param of the Convolution Module prepare_kernel_for_depthwise_separable_convolution def prepare_kernel_for_depthwise_separable_convolution(model, kernel, bias, in_channels) Prepares the kernel for depthwise separable convolution (step 1 of DSC). This means setting groups = inchannels and outchannels = k * inchannels. Arguments : model: kernel: bias: in_channels: Returns : kernel, bias) Tuple prepare_kernel_for_pointwise_convolution def prepare_kernel_for_pointwise_convolution(kernel, grouping) Prepares the kernel for pointwise convolution (step 2 of DSC). This means setting the kernel window to 1x1. So a kernel with output_channel, input_channel / groups, kernel will be set to ( , ,1) Arguments : kernel: grouping: adjust_weights_for_grouping def adjust_weights_for_grouping(weights, input_divided_by) Adjusts the Weights for the Forward of the Convulution Shape(outchannels, inchannels / group, kW) weight \u2013 filters of shape (out_channels , in_channels / groups , kW) input_divided_by Arguments : weights: input_divided_by: get_kernel_for_dsc def get_kernel_for_dsc(kernel) Part of DSC (Step 2, pointwise convolution) kernel with output_channel, input_channel / groups, kernel will be set to ( , ,1) Arguments : kernel: get_channel_filter def get_channel_filter(current_channel_size, reduced_target_channel_size, channel_priority_list) Arguments : current_channel_size: reduced_target_channel_size: channel_priority_list: create_channel_filter def create_channel_filter(module: nn.Module, kernel, current_channel, reduced_target_channel_size, is_output_filter: bool = True) Arguments : module - nn.Module: kernel: current_channel: reduced_target_channel_size: is_output_filter - bool: (Default value = True) module - nn.Module: is_output_filter - bool: (Default value = True) module - nn.Module: is_output_filter - bool: (Default value = True) module - nn.Module: is_output_filter - bool: (Default value = True) compute_channel_priorities def compute_channel_priorities(module: nn.Module, kernel, channel_index: int = 0) Arguments : module - nn.Module: kernel: channel_index - int: (Default value = 0) module - nn.Module: channel_index - int: (Default value = 0) module - nn.Module: channel_index - int: (Default value = 0) module - nn.Module: channel_index - int: (Default value = 0) hannah.models.ekut.models conv_bn def conv_bn(inp, oup, stride) Arguments : inp: oup: stride: conv_1x1_bn def conv_1x1_bn(inp, oup) Arguments : inp: oup: InvertedResidual Objects class InvertedResidual(nn.Module) forward def forward(x) Arguments : x: RawSpeechModel Objects class RawSpeechModel(nn.Module) Speech Recognition on RAW Data using Wolfgang Fuhls Networks forward def forward(x) Arguments : x: RawSpeechModelInvertedResidual Objects class RawSpeechModelInvertedResidual(nn.Module) forward def forward(x) Arguments : x: hannah.models.ekut hannah.models.utils next_power_of2 def next_power_of2(x) Arguments : x: hannah.models.wavenet.models Conv Objects class Conv(nn.Module) A convolution with the option to be causal and use xavier initialization forward def forward(signal) Arguments : signal: WaveNet Objects class WaveNet(nn.Module) forward def forward(input_data) Arguments : input_data: hannah.models.wavenet hannah.models._vendor hannah.models._vendor.focalnet Mlp Objects class Mlp(nn.Module) forward def forward(x) Arguments : x: FocalModulation Objects class FocalModulation(nn.Module) forward def forward(x) Arguments : x - input features with shape of (B, H, W, C) extra_repr def extra_repr() -> str flops def flops(N) calculate flops for 1 window with token length of N Arguments : N: FocalNetBlock Objects class FocalNetBlock(nn.Module) Focal Modulation Network Block. Arguments : dim(int) - Number of input channels. input_resolution(tuple[int]) - Input resulotion. mlp_ratio(float) - Ratio of mlp hidden dim to embedding dim. drop(float) - Dropout rate. Default: 0.0 drop_path(float) - Stochastic depth rate. Default: 0.0 act_layer(nn.Module) - Activation layer. Default: nn.GELU norm_layer(nn.Module) - Normalization layer. Default: nn.LayerNorm focal_level(int) - Number of focal levels. focal_window(int) - Focal window size at first focal level use_layerscale(bool) - Whether use layerscale layerscale_value(float) - Initial layerscale value use_postln(bool) - Whether use layernorm after modulation forward def forward(x) Arguments : x: extra_repr def extra_repr() -> str flops def flops() BasicLayer Objects class BasicLayer(nn.Module) A basic Focal Transformer layer for one stage. Arguments : dim(int) - Number of input channels. input_resolution(tuple[int]) - Input resolution. depth(int) - Number of blocks. window_size(int) - Local window size. mlp_ratio(float) - Ratio of mlp hidden dim to embedding dim. qkv_bias(bool) - If True, add a learnable bias to query, key, value. Default: True qk_scale(float | None): Override default qk scale of head_dim ** -0.5 if set. drop(float) - Dropout rate. Default: 0.0 drop_path(float | tuple[float]): Stochastic depth rate. Default: 0.0 norm_layer(nn.Module) - Normalization layer. Default: nn.LayerNorm downsample(nn.Module | None): Downsample layer at the end of the layer. Default: None use_checkpoint(bool) - Whether to use checkpointing to save memory. Default: False. focal_level(int) - Number of focal levels focal_window(int) - Focal window size at first focal level use_layerscale(bool) - Whether use layerscale layerscale_value(float) - Initial layerscale value use_postln(bool) - Whether use layernorm after modulation forward def forward(x, H, W) Arguments : x: H: W: extra_repr def extra_repr() -> str flops def flops() PatchEmbed Objects class PatchEmbed(nn.Module) Image to Patch Embedding Arguments : img_size(int) - Image size. Default: 224. patch_size(int) - Patch token size. Default: 4. in_chans(int) - Number of input image channels. Default: 3. embed_dim(int) - Number of linear projection output channels. Default: 96. norm_layer(nn.Module) - Normalization layer. Default: None forward def forward(x) Arguments : x: flops def flops() FocalNet Objects class FocalNet(nn.Module) Focal Modulation Networks (FocalNets) Arguments : img_size(int | tuple(int): Input image size. Default 224 patch_size(int | tuple(int): Patch size. Default: 4 - in_chans(int) - Number of input image channels. Default: 3 - num_classes(int) - Number of classes for classification head. Default: 1000 - embed_dim(int) - Patch embedding dimension. Default: 96 - depths(tuple(int) - Depth of each Focal Transformer layer. - mlp_ratio(float) - Ratio of mlp hidden dim to embedding dim. Default: 4 - drop_rate(float) - Dropout rate. Default: 0 - drop_path_rate(float) - Stochastic depth rate. Default: 0.1 - norm_layer(nn.Module) - Normalization layer. Default: nn.LayerNorm. - patch_norm(bool) - If True, add normalization after patch embedding. Default: True - use_checkpoint(bool) - Whether to use checkpointing to save memory. Default: False - focal_levels(list) - How many focal levels at all stages. Note that this excludes the finest-grain level. Default: [1, 1, 1, 1] - focal_windows(list) - The focal window size at all stages. Default: [7, 5, 3, 1] - use_conv_embed(bool) - Whether use convolutional embedding. We noted that using convolutional embedding usually improve the performance, but we do not use it by default. Default: False - use_layerscale(bool) - Whether use layerscale proposed in CaiT. Default: False - layerscale_value(float) - Value for layer scale. Default: 1e-4 - use_postln(bool) - Whether use layernorm after modulation (it helps stablize training of large models) no_weight_decay @torch.jit.ignore def no_weight_decay() no_weight_decay_keywords @torch.jit.ignore def no_weight_decay_keywords() forward_features def forward_features(x) Arguments : x: forward def forward(x) Arguments : x: flops def flops() build_transforms def build_transforms(img_size, center_crop=False) Arguments : img_size: - center_crop - (Default value = False) build_transforms4display def build_transforms4display(img_size, center_crop=False) Arguments : img_size: - center_crop - (Default value = False) focalnet_tiny_srf @register_model def focalnet_tiny_srf(pretrained=False, **kwargs) Arguments : pretrained - (Default value = False) **kwargs: focalnet_small_srf @register_model def focalnet_small_srf(pretrained=False, **kwargs) Arguments : pretrained - (Default value = False) **kwargs: focalnet_base_srf @register_model def focalnet_base_srf(pretrained=False, **kwargs) Arguments : pretrained - (Default value = False) **kwargs: focalnet_tiny_lrf @register_model def focalnet_tiny_lrf(pretrained=False, **kwargs) Arguments : pretrained - (Default value = False) **kwargs: focalnet_small_lrf @register_model def focalnet_small_lrf(pretrained=False, **kwargs) Arguments : pretrained - (Default value = False) **kwargs: focalnet_base_lrf @register_model def focalnet_base_lrf(pretrained=False, **kwargs) Arguments : pretrained - (Default value = False) **kwargs: focalnet_tiny_iso_16 @register_model def focalnet_tiny_iso_16(pretrained=False, **kwargs) Arguments : pretrained - (Default value = False) **kwargs: focalnet_small_iso_16 @register_model def focalnet_small_iso_16(pretrained=False, **kwargs) Arguments : pretrained - (Default value = False) **kwargs: focalnet_base_iso_16 @register_model def focalnet_base_iso_16(pretrained=False, **kwargs) Arguments : pretrained - (Default value = False) **kwargs: focalnet_large_fl3 @register_model def focalnet_large_fl3(pretrained=False, **kwargs) Arguments : pretrained - (Default value = False) **kwargs: focalnet_large_fl4 @register_model def focalnet_large_fl4(pretrained=False, **kwargs) Arguments : pretrained - (Default value = False) **kwargs: focalnet_xlarge_fl3 @register_model def focalnet_xlarge_fl3(pretrained=False, **kwargs) Arguments : pretrained - (Default value = False) **kwargs: focalnet_xlarge_fl4 @register_model def focalnet_xlarge_fl4(pretrained=False, **kwargs) Arguments : pretrained - (Default value = False) **kwargs: focalnet_huge_fl3 @register_model def focalnet_huge_fl3(pretrained=False, **kwargs) Arguments : pretrained - (Default value = False) **kwargs: focalnet_huge_fl4 @register_model def focalnet_huge_fl4(pretrained=False, **kwargs) Arguments : pretrained - (Default value = False) **kwargs: hannah.models.sinc.models GDSConv Objects class GDSConv(nn.Module) forward def forward(x) Arguments : x: GDSConvBlock Objects class GDSConvBlock(nn.Module) forward def forward(x) Arguments : x: SincNet Objects class SincNet(nn.Module) forward def forward(x) Arguments : x: hannah.models.sinc hannah.models.capsule_net.models hannah.models.capsule_net hannah.models.capsule_net.operators hannah.models.capsule_net.expressions padding_expression def padding_expression(kernel_size, stride, dilation=1) Symbolically calculate padding such that for a given kernel_size, stride and dilation the padding is such that the output dimension is kept the same(stride=1) or halved(stride=2). Note: If the input dimension is 1 and stride = 2, the calculated padding will result in an output with also dimension 1. Parameters kernel_size : Union[int, Expression] stride : Union[int, Expression] dilation : Union[int, Expression], optional description , by default 1 Returns Expression hannah.models.capsule_net.utils hannah.test_linear_classifier hannah.callbacks.dump_layers TestDumperCallback Objects class TestDumperCallback(Callback) on_test_start def on_test_start(pl_trainer, pl_model) Arguments : pl_trainer: pl_model: hannah.callbacks.backends symbolic_batch_dim def symbolic_batch_dim(model) Arguments : model: InferenceBackendBase Objects class InferenceBackendBase(Callback) Base class to run val and test on a backend inference engine run_batch def run_batch(inputs=None) Arguments : inputs - (Default value = None) prepare def prepare(module) Arguments : module: on_validation_epoch_start def on_validation_epoch_start(trainer, pl_module) Arguments : trainer: pl_module: on_validation_batch_end def on_validation_batch_end(trainer, pl_module, outputs, batch, batch_idx, dataloader_idx) Arguments : trainer: pl_module: outputs: batch: batch_idx: dataloader_idx: on_validation_epoch_end def on_validation_epoch_end(trainer, pl_module) Arguments : trainer: pl_module: on_test_epoch_start def on_test_epoch_start(trainer, pl_module) Arguments : trainer: pl_module: quantize def quantize(pl_module: torch.nn.Module) -> torch.nn.Module Arguments : pl_module - torch.nn.Module to quantize Returns - quantized torch.nn.Module on_test_batch_end def on_test_batch_end(trainer, pl_module, outputs, batch, batch_idx, dataloader_idx) Arguments : trainer: pl_module: outputs: batch: batch_idx: dataloader_idx: export def export() -> None Export the model through the target backend TorchMobileBackend Objects class TorchMobileBackend(InferenceBackendBase) Inference backend for torch mobile prepare def prepare(model) Arguments : model torch.nn.Module - nn.Module to be exported Returns (None) run_batch def run_batch(inputs=None) Arguments : inputs - (Default value = None) OnnxTFBackend Objects class OnnxTFBackend(InferenceBackendBase) Inference Backend for tensorflow prepare def prepare(model) Arguments : model: run_batch def run_batch(inputs) Arguments : inputs: OnnxruntimeBackend Objects class OnnxruntimeBackend(InferenceBackendBase) Inference Backend for tensorflow prepare def prepare(model) Arguments : model: run_batch def run_batch(inputs=None) Arguments : inputs - (Default value = None) Returns hannah.callbacks.pruning PruningAmountScheduler Objects class PruningAmountScheduler() FilteredPruning Objects class FilteredPruning(ModelPruning) setup def setup(trainer: Trainer, pl_module: LightningModule, stage: str) Arguments : trainer: pl_module: filter_parameters_to_prune def filter_parameters_to_prune(parameters_to_prune=None) Filter out unprunable parameters Arguments : parameters_to_prune - (Default value = None) on_test_end def on_test_end(trainer, pl_module) -> None Arguments : trainer: pl_module: hannah.callbacks.optimization HydraOptCallback Objects class HydraOptCallback(Callback) on_test_end def on_test_end(trainer, pl_module) Arguments : trainer: pl_module: on_validation_end def on_validation_end(trainer, pl_module) Arguments : trainer: pl_module: test_result def test_result() val_result def val_result() result def result(dict=False) Arguments : dict - (Default value = False) curves def curves(dict=False) Arguments : dict - (Default value = False) hannah.callbacks.fine_tuning hannah.callbacks hannah.callbacks.clustering clustering def clustering(params, inertia, cluster) Arguments : params: inertia: cluster: kMeans Objects class kMeans(Callback) on_fit_end def on_fit_end(trainer, pl_module) Arguments : trainer: pl_module: on_epoch_end def on_epoch_end(trainer, pl_module) Arguments : trainer: pl_module: hannah.callbacks.summaries walk_model def walk_model(model, dummy_input) Adapted from IntelLabs Distiller Arguments : model: dummy_input: MacSummaryCallback Objects class MacSummaryCallback(Callback) predict def predict(pl_module, input=input) Arguments : pl_module: on_train_start @rank_zero_only def on_train_start(trainer, pl_module) Arguments : trainer: pl_module: on_test_end @rank_zero_only def on_test_end(trainer, pl_module) Arguments : trainer: pl_module: on_validation_epoch_end @rank_zero_only def on_validation_epoch_end(trainer, pl_module) Arguments : trainer: pl_module: estimate def estimate(pl_module, input=None) Generate Summary Metrics for neural network Arguments : pl_module(pytorch_lightning.LightningModule) - pytorch lightning module to summarize Returns : dict[str, float]: Dict of MetricName => Metric Value hannah.callbacks.svd_compress SVD Objects class SVD(Callback) on_epoch_start def on_epoch_start(trainer, pl_module) Arguments : trainer: pl_module: hannah.ssl.hard_labeling HardLabeling Objects class HardLabeling(torch.nn.Module) forward def forward(unlabeled_data: torch.Tensor) -> torch.Tensor Calculate pseudo label loss from unlabeled data. compute_uncertainty def compute_uncertainty(data: torch.Tensor, num_forward_passes: int = 10) -> torch.Tensor Compute Monte Carlo uncertainty using standard deviation. get_dropout_layers def get_dropout_layers() Returns all model layers of class dropout, dropblock. hannah.ssl hannah.conf.optimizer SGDConf Objects @dataclass class SGDConf() lr _RequiredParameter MADGRADConf Objects @dataclass class MADGRADConf() lr _RequiredParameter hannah.conf hannah.conf.nas hannah.conf.scheduler OneCycleLRConf Objects @dataclass class OneCycleLRConf() Config for one cycle lr total steps are configured from module hannah.optim hannah.optim.madgrad MADGRAD Objects class MADGRAD(torch.optim.Optimizer) MADGRAD_: A Momentumized, Adaptive, Dual Averaged Gradient Method for Stochastic Optimization. .. _MADGRAD: https://arxiv.org/abs/2101.11075 MADGRAD is a general purpose optimizer that can be used in place of SGD or Adam may converge faster and generalize better. Currently GPU-only. Typically, the same learning rate schedule that is used for SGD or Adam may be used. The overall learning rate is not comparable to either method and should be determined by a hyper-parameter sweep. MADGRAD requires less weight decay than other methods, often as little as zero. Momentum values used for SGD or Adam's beta1 should work here also. On sparse problems both weight_decay and momentum should be set to 0. Arguments : params (iterable): Iterable of parameters to optimize or dicts defining parameter groups. lr (float): Learning rate (default: 1e-2). momentum (float): Momentum value in the range [0,1) (default: 0.9). weight_decay (float): Weight decay, i.e. a L2 penalty (default: 0). eps (float): Term added to the denominator outside of the root operation to improve numerical stability. (default: 1e-6). step def step(closure: Optional[Callable[[], float]] = None) -> Optional[float] Performs a single optimization step. Arguments : closure callable, optional - A closure that reevaluates the model and returns the loss. hannah.optim.RAdam hannah.nas.dataflow.registry hannah.nas.dataflow.dataflow_utils hannah.nas.dataflow.tensor_expression hannah.nas.dataflow.quantization_type hannah.nas.dataflow.register_ops hannah.nas.dataflow.ops.dropout hannah.nas.dataflow.ops.conv2d hannah.nas.dataflow.ops.linear hannah.nas.dataflow.ops.identity hannah.nas.dataflow.ops.relu hannah.nas.dataflow.ops.sum hannah.nas.dataflow.ops.batch_nom hannah.nas.dataflow.ops.add hannah.nas.dataflow.ops hannah.nas.dataflow.ops.concat hannah.nas.dataflow.ops.pooling hannah.nas.dataflow.analysis.dataflow_analysis hannah.nas.dataflow.tensor hannah.nas.dataflow.tensor_type hannah.nas.dataflow.transformations.graph_tranformer hannah.nas.dataflow.scoping_utils hannah.nas.dataflow hannah.nas.dataflow.repeat hannah.nas.dataflow.axis_type AxisTuple Objects @parametrize class AxisTuple() Used to have the axis dict as a parametrized object hannah.nas.dataflow.data_type hannah.nas.dataflow.optional_op hannah.nas.dataflow.compression_type hannah.nas.dataflow.op_type hannah.nas.dataflow.dataflow_graph DataFlowGraph Objects @parametrize class DataFlowGraph(TensorExpression) link_users def link_users() Link the DFG to its users and the users of the DFG to the DFG collect_users def collect_users(node) Traverse graph starting from node and collect all users (including users from subsequent nodes). If a node_b is NOT in collect_users(node_a), this means that node_b is either BEFORE node_a in the graph OR it is in a parallel branch. Parameters node : type description Returns type description find_first_input def find_first_input(node) Recusively traverses the graph from the given node back to its first input. NOTE: The traversal is via OPERANDS and not OUTPUT, meaning that e.g. weight Tensors that are included in Ops in a DFG are not returned Parameters node : type description Returns type description hannah.nas.performance_prediction.simple BackendPredictor Objects class BackendPredictor() A predictor class that instantiates the model and uses the backends predict function to predict performance metrics MACPredictor Objects class MACPredictor() A predictor class that instantiates the model and calculates abstract metrics GCNPredictor Objects class GCNPredictor() A predictor class that instantiates the model and uses the backends predict function to predict performance metrics hannah.nas.performance_prediction.gcn.predictor Predictor Objects class Predictor() __init__ def __init__(fea_name=\"features\") -> None Parent method for different predictor classes. Parameters fea_name : str, optional internal name for features in the graph, as in graph.ndata[fea_name], by default 'features' train def train(dataloader, learning_rate=1e-3, num_epochs=200, validation_dataloader=None, verbose=1) Train GCN model Parameters dataloader : GraphDataLoader training data learning_rate : [type], optional by default 1e-3 num_epochs : int, optional by default 200 validation_dataloader : [type], optional if given, use this data to print validation loss, by default None verbose : int if validation_dataloader is given, print validation MSE every epoch, by default 1 predict def predict(graph) predict cost of graph Parameters graph : dgl.Graph Returns torch.Tensor predicted cost of given graph. Retrieve float value with .item() GCNPredictor Objects class GCNPredictor(Predictor) __init__ def __init__(input_feature_size, hidden_units=[128], readout=\"mean\", fea_name=\"features\") -> None G(raph)CN based network latency/cost predictor. End-to-end from graph to score. Parameters input_feature_size : [type] length of feature vector of a graph node (graph G with n nodes, each with features of length m, i.e. feature matrix F = n x m) hidden_units : int, list, optional size of hidden layer (layers if list) , by default 128 readout : str, optional readout function that is used to aggregate node features, by default 'mean' fea_name : str, optional internal name for features in the graph, as in graph.ndata[fea_name], by default 'features' train_and_fit def train_and_fit(dataloader, learning_rate=1e-3, num_epochs=200, validation_dataloader=None, verbose=0) Train GCN model Parameters dataloader : GraphDataLoader training data learning_rate : [type], optional by default 1e-3 num_epochs : int, optional by default 200 validation_dataloader : [type], optional if given, use this data to print validation loss, by default None verbose : int if validation_dataloader is given, print validation MSE every epoch, by default 1 predict def predict(graph) predict cost of graph Parameters graph : dgl.Graph Returns torch.Tensor predicted cost of given graph. Retrieve float value with .item() GaussianProcessPredictor Objects class GaussianProcessPredictor(Predictor) __init__ def __init__(input_feature_size, hidden_units=128, embedding_size=10, readout=\"mean\", fea_name=\"features\", kernel=\"default\", alpha=1e-10) -> None Predictor that generates a graph embedding that is used as input for a gaussian process predictor. Parameters input_feature_size : [type] length of feature vector of a graph node (graph G with n nodes, each with features of length m, i.e. feature matrix F = n x m) hidden_units : int, list, optional size of hidden layer (layers if list) , by default 128 embedding_size: int, optional size of output embedding readout : str, optional readout function that is used to aggregate node features, by default 'mean' fea_name : str, optional internal name for features in the graph, as in graph.ndata[fea_name], by default 'features' kernel : str, sklearn.gaussian_process.kernels.Kernel, optional The gaussian process kernel to use. input shoudl be either \"default\", or a sklearn Kernel() object by default RBF() + DotProduct() + WhiteKernel() train_and_fit def train_and_fit(dataloader, learning_rate=1e-3, num_epochs=200, validation_dataloader=None, verbose=1) Train GCN model, generate embeddings for training data and fit the predictor with embeddings. Parameters dataloader : GraphDataLoader training data learning_rate : [type], optional by default 1e-3 num_epochs : int, optional by default 200 validation_dataloader : [type], optional if given, use this data to print validation loss, by default None verbose : int if validation_dataloader is given, print validation MSE every epoch,by default 1 Returns float score of predictor on TRAINING data, see sklearn doc of chosen predictor for more info predict def predict(X, return_std=True) Predict cost/latency of graphs. Parameters X : dgl.DGLGraph, list[DGLGraph], dgl.dataloading.GraphDataLoader Input graph(s) return_std : bool, optional if true, return standard dev. else just mean prediction, by default True Returns array (,array) prediction(s) , (if return_std: standard deviation(s)) XGBPredictor Objects class XGBPredictor(Predictor) __init__ def __init__(input_feature_size, hidden_units=128, embedding_size=10, readout=\"mean\", fea_name=\"features\", xgb_param=\"default\") -> None Predictor that generates a graph embedding that is used as input for a xgb based predictor. Parameters input_feature_size : [type] length of feature vector of a graph node (graph G with n nodes, each with features of length m, i.e. feature matrix F = n x m) hidden_units : int, list, optional size of hidden layer (layers if list) , by default 128 embedding_size: int, optional size of output embedding readout : str, optional readout function that is used to aggregate node features, by default 'mean' fea_name : str, optional internal name for features in the graph, as in graph.ndata[fea_name], by default 'features' xgb_param : str, dict, optional The xgb_parameter to use. See https://xgboost.readthedocs.io/en/latest/parameter.html train_and_fit def train_and_fit(dataloader, learning_rate=1e-3, num_epochs=200, num_round=8000, validation_dataloader=None, verbose=1) Train GCN model, generate embeddings for training data and fit the predictor with embeddings. Parameters dataloader : GraphDataLoader training data learning_rate : [type], optional by default 1e-3 num_epochs : int, optional Training epochs for the GCN embedding network, by default 200 num_round : int, optional training rounds for xgb booster, by default 800 validation_dataloader : [type], optional if given, use this data to print validation loss, by default None verbose : int if validation_dataloader is given, print validation MSE every epoch,by default 1 predict def predict(X) Predict cost/latency of graphs. Parameters X : dgl.DGLGraph, list[DGLGraph], dgl.dataloading.GraphDataLoader Input graph(s) Returns array (,array) prediction(s) , (if return_std: standard deviation(s)) prepare_dataloader def prepare_dataloader(dataset, batch_size=50, train_test_split=1, subset=0, seed=0, validation=False) helper function to construct dataloaders from NASGraphDataset Parameters dataset : NASGraphDataset batch_size : int, optional by default 50 train_test_split : float, optional number between 0 and 1, the proportion of the dataset to be used for training, by default 1 subset : int, optional choose only many samples from the dataset. Set 0 for disabling, i.e. whole dataset. by default 0 seed : int, optional set seed for reproduceability validation : bool, optional also output a validation set e.g. for hyperparam tuning Returns tuple(GraphDataLoader, (GraphDataLoader), GraphDataLoader) training dataloader to be used in CostPredictor.train() and test/validation dataloader if train_test_split > 0, else len(test_dataloader) == 0 hannah.nas.performance_prediction.gcn.model hannah.nas.performance_prediction.features.dataset hannah.nas.performance_prediction.features.graph_conversion hannah.nas.performance_prediction hannah.nas.performance_prediction.examples.gcn_predictor_example hannah.nas.performance_prediction.examples.gcn_model_example hannah.nas.performance_prediction.examples.ri_capsule_performance_predictor hannah.nas.performance_prediction.examples.gaussian_process_predictor_example hannah.nas.performance_prediction.examples.xgb_predictor_example hannah.nas.constraints.constraint_model hannah.nas.constraints.dfg_constraint_model ConstraintModel Objects class ConstraintModel() process_optype def process_optype(op: OpType) Extracts the constraints based on the type of op. New variables are added to self.vars and the constraints are added to the solver. Parameters op : OpType process_tensor def process_tensor(tensor: Tensor) Goes through all axis and extracts the constraints for the respective axis sizes Parameters tensor : Tensor hannah.nas.hardware_description.device hannah.nas.hardware_description.memory_type hannah.nas.hardware_description hannah.nas.hardware_description.description hannah.nas.eval.extract hannah.nas.eval hannah.nas.eval.plot hannah.nas.eval.__main__ hannah.nas.eval.prepare prepare_summary def prepare_summary(data: Dict[str, str], base_dir: str = \".\", force: bool = False) -> pd.DataFrame Prepare a summary of one or multiple nas runs Arguments : data Dict[str, str] - A mapping from a short name for a nas run \"e.g KWS\" to a folder containing nas history file e.g. \"trained_models/nas_kws/conv_net_trax\" base_dir str - base directory paths in data mapping are interpreted relative to base directory force bool - force reconstructing of cached results (\"data.pkl\") hannah.nas.expressions.conditions hannah.nas.expressions.op hannah.nas.expressions.arithmetic hannah.nas.expressions.placeholder hannah.nas.expressions hannah.nas.expressions.logic hannah.nas.expressions.shapes hannah.nas.expressions.metrics hannah.nas.expressions.types hannah.nas.expressions.choice hannah.nas.test.test_symbolic_metrics hannah.nas.test.test_parameters hannah.nas.test.test_op_to_torch_conversion hannah.nas.test.test_conditions hannah.nas.test.test_lazy_torch hannah.nas.test.test_mobilenet hannah.nas.test.test_scoping hannah.nas.test.test_darts_space hannah.nas.test.test_arithmetic test_unimplemeted @pytest.mark.parametrize( \"x,y\", [ (IntScalarParameter(0, 0), 2), (IntScalarParameter(0, 0), IntScalarParameter(0, 0)), (DefaultInt(0), 2), ], ) def test_unimplemeted(x, y) Test that unimplemented methods raise unimplemented errors hannah.nas.test.test_z3 hannah.nas.test.test_graph_transformer hannah.nas.test.test_dfg_removal hannah.nas.test.test_repeat hannah.nas.test.test_parametrize hannah.nas.test.network hannah.nas.test.test_description_ultratrail hannah.nas.test.test_dataflow hannah.nas.test.test_conv2d hannah.nas.test.test_constraint_model hannah.nas.test.test_add hannah.nas.test.test_adjacency hannah.nas.parametrization ScalarParameterState Objects @dataclass class ScalarParameterState(ParameterState) sigma variance of sampling parameter hannah.nas.parameters.parameters hannah.nas.parameters.parametrize hannah.nas.parameters.iterators hannah.nas.parameters hannah.nas.parameters.lazy hannah.nas hannah.nas.core.parametrized hannah.nas.core hannah.nas.core.expression hannah.nas.config Scalar Objects @dataclass class Scalar() Representation of all the options to define a scalar. hannah.nas.__main__ hannah.nas.plotter hannah.nas.search.search hannah.nas.search.model_trainer.simple_model_trainer hannah.nas.search.model_trainer.progressive_shrinking ProgressiveShrinkingModelTrainer Objects class ProgressiveShrinkingModelTrainer() warmup def warmup(model, ofa_model) The function rebuilds the trainer with the warmup epochs, fits the model, validates the model, and then calls the on_warmup_end() function to change some internal variables Arguments : model : the model to be trained ofa_model : the model that we want to train train_elastic_width def train_elastic_width(model, ofa_model) The function trains the model for a number of epochs, then adds a width step, then trains the model for a number of epochs, then adds a width step, and so on Arguments : model : the model to train ofa_model : the model that will be trained train_elastic_depth def train_elastic_depth(model, ofa_model) The function trains the model for a number of epochs, then progressively shrinks the depth of the model, and trains the model for a number of epochs again Arguments : model : the model to train ofa_model : the model to be trained train_elastic_kernel def train_elastic_kernel(model, ofa_model) The function trains the elastic kernels by progressively shrinking the model and training the model for a number of epochs and repeats this process until the number of kernel steps is reached Arguments : model : the model to train ofa_model : the model that will be trained train_elastic_dilation def train_elastic_dilation(model, ofa_model) The function trains the model for a number of epochs, then adds a dilation step, and trains the model for a number of epochs, and repeats this process until the number of dilation steps is reached Arguments : model : the model to be trained ofa_model : the model that will be trained train_elastic_grouping def train_elastic_grouping(model, ofa_model) The function trains the model for a number of epochs, then adds a group step, and trains the model for a number of epochs, and repeats this process until the number of group steps is reached Arguments : model : the model to be trained ofa_model : the model that will be trained train_elastic_dsc def train_elastic_dsc(model, ofa_model) The function trains the model for a number of epochs, then adds a dsc step (turns Depthwise Separable Convolution on and off), and trains the model for a number of epochs, and repeats this process until the number of dsc steps is reached Arguments : model : the model to be trained ofa_model : the model that will be trained eval_elastic_width def eval_elastic_width(method_stack, method_index, lightning_model, model, trainer_path, loginfo_output, metrics_output, metrics_csv) This function steps down the width of the model, and then calls the next method in the stack Arguments : method_stack : a list of methods that will be called in order method_index : The index of the current method in the method stack lightning_model : the lightning model to be trained model : the model to be trained trainer_path : The path to the trainer loginfo_output : This is the string that will be printed to the console metrics_output : a string that will be written to the metrics csv file metrics_csv : a string that contains the metrics for the current model Returns : The metrics_csv is being returned. eval_elastic_kernel def eval_elastic_kernel(method_stack, method_index, lightning_model, model, trainer_path, loginfo_output, metrics_output, metrics_csv) This function steps down the kernel size of the model, and then calls the next method in the stack Arguments : method_stack : The list of methods to be called method_index : The index of the current method in the method stack lightning_model : the lightning model to be trained model : the model to be trained trainer_path : The path to the trainer loginfo_output : This is the string that will be printed to the console metrics_output : This is the string that will be printed to the console metrics_csv : a string that contains the metrics for the current model Returns : The metrics_csv is being returned. eval_elastic_dilation def eval_elastic_dilation(method_stack, method_index, lightning_model, model, trainer_path, loginfo_output, metrics_output, metrics_csv) This function evaluates the model with a different dilation size for each layer Arguments : method_stack : The list of methods to be called method_index : The index of the method in the method stack lightning_model : the lightning model to be trained model : the model to be evaluated trainer_path : The path to the trainer loginfo_output : This is the string that will be printed to the console metrics_output : a string that will be written to the metrics csv file metrics_csv : a string that contains the csv data for the metrics Returns : The metrics_csv is being returned. eval_elastic_depth def eval_elastic_depth(method_stack, method_index, lightning_model, model, trainer_path, loginfo_output, metrics_output, metrics_csv) This function will run the next method in the stack for each depth step, and then return the metrics_csv Arguments : method_stack : The list of methods to be called method_index : The index of the current method in the method stack lightning_model : the lightning model to be trained model : The model to be trained trainer_path : The path to the trainer, which is used to save the model loginfo_output : This is the string that will be printed to the console metrics_output : This is the string that will be printed to the console metrics_csv : This is the CSV file that we're writing to Returns : The metrics_csv is being returned. eval_elastic_grouping def eval_elastic_grouping(method_stack, method_index, lightning_model, model, trainer_path, loginfo_output, metrics_output, metrics_csv) This function evaluates the model with a different group size for each layer Arguments : method_stack : The list of methods to be called method_index : The index of the method in the method stack lightning_model : the lightning model to be trained model : the model to be evaluated trainer_path : The path to the trainer loginfo_output : This is the string that will be printed to the console metrics_output : a string that will be written to the metrics csv file metrics_csv : a string that contains the csv data for the metrics Returns : The metrics_csv is being returned. eval_elastic_dsc def eval_elastic_dsc(method_stack, method_index, lightning_model, model, trainer_path, loginfo_output, metrics_output, metrics_csv) This function evaluates the model with a different dsc for each layer Arguments : method_stack : The list of methods to be called method_index : The index of the method in the method stack lightning_model : the lightning model to be trained model : the model to be evaluated trainer_path : The path to the trainer loginfo_output : This is the string that will be printed to the console metrics_output : a string that will be written to the metrics csv file metrics_csv : a string that contains the csv data for the metrics Returns : The metrics_csv is being returned. eval_single_model def eval_single_model(method_stack, method_index, lightning_model, model, trainer_path, loginfo_output, metrics_output, metrics_csv) This function takes in a model, a trainer, and a bunch of other stuff, evaluates the model and tracks the results in der in the given strings and returns a string of metrics Arguments : method_stack : The list of methods that we're evaluating method_index : The index of the method in the method stack lightning_model : the lightning model that we want to evaluate model : The model to be evaluated trainer_path : the path to the trainer object loginfo_output : This is the string that will be printed to the console when the model is being evaluated metrics_output : This is the string that will be written to the metrics file. It contains the method name, the method index, and the method stack metrics_csv : a string that will be written to a csv file Returns : The metrics_csv is being returned. eval_model def eval_model(lightning_model, model) First the method stack for the evaluation ist build and then it is according to this evaluated Arguments : lightning_model : the lightning model model : the model to be evaluated hannah.nas.search.search_old OFANasTrainer Objects class OFANasTrainer(NASTrainerBase) warmup def warmup(model, ofa_model) The function rebuilds the trainer with the warmup epochs, fits the model, validates the model, and then calls the on_warmup_end() function to change some internal variables Arguments : model : the model to be trained ofa_model : the model that we want to train train_elastic_width def train_elastic_width(model, ofa_model) The function trains the model for a number of epochs, then adds a width step, then trains the model for a number of epochs, then adds a width step, and so on Arguments : model : the model to train ofa_model : the model that will be trained train_elastic_depth def train_elastic_depth(model, ofa_model) The function trains the model for a number of epochs, then progressively shrinks the depth of the model, and trains the model for a number of epochs again Arguments : model : the model to train ofa_model : the model to be trained train_elastic_kernel def train_elastic_kernel(model, ofa_model) The function trains the elastic kernels by progressively shrinking the model and training the model for a number of epochs and repeats this process until the number of kernel steps is reached Arguments : model : the model to train ofa_model : the model that will be trained train_elastic_dilation def train_elastic_dilation(model, ofa_model) The function trains the model for a number of epochs, then adds a dilation step, and trains the model for a number of epochs, and repeats this process until the number of dilation steps is reached Arguments : model : the model to be trained ofa_model : the model that will be trained train_elastic_grouping def train_elastic_grouping(model, ofa_model) The function trains the model for a number of epochs, then adds a group step, and trains the model for a number of epochs, and repeats this process until the number of group steps is reached Arguments : model : the model to be trained ofa_model : the model that will be trained train_elastic_dsc def train_elastic_dsc(model, ofa_model) The function trains the model for a number of epochs, then adds a dsc step (turns Depthwise Separable Convolution on and off), and trains the model for a number of epochs, and repeats this process until the number of dsc steps is reached Arguments : model : the model to be trained ofa_model : the model that will be trained eval_elastic_width def eval_elastic_width(method_stack, method_index, lightning_model, model, trainer_path, loginfo_output, metrics_output, metrics_csv) This function steps down the width of the model, and then calls the next method in the stack Arguments : method_stack : a list of methods that will be called in order method_index : The index of the current method in the method stack lightning_model : the lightning model to be trained model : the model to be trained trainer_path : The path to the trainer loginfo_output : This is the string that will be printed to the console metrics_output : a string that will be written to the metrics csv file metrics_csv : a string that contains the metrics for the current model Returns : The metrics_csv is being returned. eval_elastic_kernel def eval_elastic_kernel(method_stack, method_index, lightning_model, model, trainer_path, loginfo_output, metrics_output, metrics_csv) This function steps down the kernel size of the model, and then calls the next method in the stack Arguments : method_stack : The list of methods to be called method_index : The index of the current method in the method stack lightning_model : the lightning model to be trained model : the model to be trained trainer_path : The path to the trainer loginfo_output : This is the string that will be printed to the console metrics_output : This is the string that will be printed to the console metrics_csv : a string that contains the metrics for the current model Returns : The metrics_csv is being returned. eval_elastic_dilation def eval_elastic_dilation(method_stack, method_index, lightning_model, model, trainer_path, loginfo_output, metrics_output, metrics_csv) This function evaluates the model with a different dilation size for each layer Arguments : method_stack : The list of methods to be called method_index : The index of the method in the method stack lightning_model : the lightning model to be trained model : the model to be evaluated trainer_path : The path to the trainer loginfo_output : This is the string that will be printed to the console metrics_output : a string that will be written to the metrics csv file metrics_csv : a string that contains the csv data for the metrics Returns : The metrics_csv is being returned. eval_elastic_depth def eval_elastic_depth(method_stack, method_index, lightning_model, model, trainer_path, loginfo_output, metrics_output, metrics_csv) This function will run the next method in the stack for each depth step, and then return the metrics_csv Arguments : method_stack : The list of methods to be called method_index : The index of the current method in the method stack lightning_model : the lightning model to be trained model : The model to be trained trainer_path : The path to the trainer, which is used to save the model loginfo_output : This is the string that will be printed to the console metrics_output : This is the string that will be printed to the console metrics_csv : This is the CSV file that we're writing to Returns : The metrics_csv is being returned. eval_elastic_grouping def eval_elastic_grouping(method_stack, method_index, lightning_model, model, trainer_path, loginfo_output, metrics_output, metrics_csv) This function evaluates the model with a different group size for each layer Arguments : method_stack : The list of methods to be called method_index : The index of the method in the method stack lightning_model : the lightning model to be trained model : the model to be evaluated trainer_path : The path to the trainer loginfo_output : This is the string that will be printed to the console metrics_output : a string that will be written to the metrics csv file metrics_csv : a string that contains the csv data for the metrics Returns : The metrics_csv is being returned. eval_elastic_dsc def eval_elastic_dsc(method_stack, method_index, lightning_model, model, trainer_path, loginfo_output, metrics_output, metrics_csv) This function evaluates the model with a different dsc for each layer Arguments : method_stack : The list of methods to be called method_index : The index of the method in the method stack lightning_model : the lightning model to be trained model : the model to be evaluated trainer_path : The path to the trainer loginfo_output : This is the string that will be printed to the console metrics_output : a string that will be written to the metrics csv file metrics_csv : a string that contains the csv data for the metrics Returns : The metrics_csv is being returned. eval_single_model def eval_single_model(method_stack, method_index, lightning_model, model, trainer_path, loginfo_output, metrics_output, metrics_csv) This function takes in a model, a trainer, and a bunch of other stuff, evaluates the model and tracks the results in der in the given strings and returns a string of metrics Arguments : method_stack : The list of methods that we're evaluating method_index : The index of the method in the method stack lightning_model : the lightning model that we want to evaluate model : The model to be evaluated trainer_path : the path to the trainer object loginfo_output : This is the string that will be printed to the console when the model is being evaluated metrics_output : This is the string that will be written to the metrics file. It contains the method name, the method index, and the method stack metrics_csv : a string that will be written to a csv file Returns : The metrics_csv is being returned. eval_model def eval_model(lightning_model, model) First the method stack for the evaluation ist build and then it is according to this evaluated Arguments : lightning_model : the lightning model model : the model to be evaluated hannah.nas.search.weight_sharing hannah.nas.search.sampler.base_sampler Sampler Objects class Sampler(ABC) tell_result def tell_result(parameters, metrics) Tell the result of a task hannah.nas.search.sampler.aging_evolution AgingEvolutionSampler Objects class AgingEvolutionSampler(Sampler) Aging Evolution based multi objective optimization next_parameters def next_parameters() Returns a list of current tasks tell_result def tell_result(parameters, metrics) Tell the result of a task hannah.nas.search.sampler.random_sampler hannah.nas.search.sampler.mutator hannah.nas.search.utils hannah.nas.ops hannah.nas.graph_conversion hannah.nas.spaces.darts.darts_space hannah.nas.spaces.mobilenet.mobilenet hannah.nas.utils is_pareto def is_pareto(costs, maximise=False) Arguments : costs : An (n_points, n_costs) array Returns : A (n_points, ) boolean array, indicating whether each point is Pareto efficient hannah.datasets.vision.ri_capsule Rhode island gastroenterology video capsule endoscopy dataset https://www.nature.com/articles/s41597-022-01726-3 https://github.com/acharoen/Rhode-Island-GI-VCE-Technical-Validation hannah.datasets.vision.fake hannah.datasets.vision.cifar hannah.datasets.vision.kvasir hannah.datasets.vision.kvasir_unlabeled KvasirCapsuleUnlabeled Objects class KvasirCapsuleUnlabeled(AbstractDataset) Dataset representing unlabelled videos hannah.datasets.vision hannah.datasets.vision.dresden_capsule Rhode island gastroenterology video capsule endoscopy dataset https://www.nature.com/articles/s41597-022-01726-3 https://github.com/acharoen/Rhode-Island-GI-VCE-Technical-Validation hannah.datasets.vision.utils hannah.datasets.vision.utils.naneye read_naneye def read_naneye(data_file: Union[str, Path]) Read a naneye raw aimage and decode bayer pattern Arguments : data_file Union[str, Path] - path to the datafile Returns : np.ndarray - uint8 array of decoded image data hannah.datasets.vision.base TorchvisionDatasetBase Objects class TorchvisionDatasetBase(VisionDatasetBase) Wrapper around torchvision classification datasets ImageDatasetBase Objects class ImageDatasetBase(VisionDatasetBase) __init__ def __init__(X, y, classes, bbox=None, transform=None) Initialize vision dataset Arguments : X List[str] - List of paths to image files y List[str] - Class id of corresponding image classes List[str] - List of class names, names are ordered by numeric class id bbox Dict[str] - Dict with filename as keys, bbox coordinates as numpy arrays transform Callable[image,image], optional - Optional transformation/augmentation of input images. Defaults to None. hannah.datasets.emergency EmergencySirenDataset Objects class EmergencySirenDataset(AbstractDataset) Emergency Dataset hannah.datasets.Downsample hannah.datasets.collate vision_collate_fn def vision_collate_fn(batch) Function that takes in a batch of data and puts the elements within the batch into a tensor with an additional outer dimension - batch size. The exact output type can be a :class: torch.Tensor , a Sequence of :class: torch.Tensor , a Collection of :class: torch.Tensor , or left unchanged, depending on the input type. This is used as the default function for collation for vision tasks batch_size or batch_sampler is defined in :class: ~torch.utils.data.DataLoader . Here is the general input type (based on the type of the element within the batch) to output type mapping: :class: torch.Tensor -> :class: torch.Tensor (with an added outer dimension batch size) NumPy Arrays -> :class: torch.Tensor float -> :class: torch.Tensor int -> :class: torch.Tensor str -> str (unchanged) bytes -> bytes (unchanged) Mapping[K, V_i] -> Mapping[K, vision_collate([V_1, V_2, ...])] NamedTuple[V1_i, V2_i, ...] -> NamedTuple[vision_collate([V1_1, V1_2, ...]), vision_collate([V2_1, V2_2, ...]), ...] Sequence[V1_i, V2_i, ...] -> Sequence[vision_collate([V1_1, V1_2, ...]), vision_collate([V2_1, V2_2, ...]), ...] Arguments : batch - a single batch to be collated Examples : Example with a batch of int s: vision_collate([0, 1, 2, 3]) tensor([0, 1, 2, 3]) Example with a batch of str s: vision_collate(['a', 'b', 'c']) ['a', 'b', 'c'] Example with Map inside the batch: vision_collate([{'A': 0, 'B': 1}, {'A': 100, 'B': 100}]) - {'A' - tensor([ 0, 100]), 'B': tensor([ 1, 100])} Example with NamedTuple inside the batch: Point = namedtuple('Point', ['x', 'y']) vision_collate([Point(0, 0), Point(1, 1)]) Point(x=tensor([0, 1]), y=tensor([0, 1])) Example with Tuple inside the batch: vision_collate([(0, 1), (2, 3)]) [tensor([0, 2]), tensor([1, 3])] Example with List inside the batch: vision_collate([[0, 1], [2, 3]]) [tensor([0, 2]), tensor([1, 3])] ctc_collate_fn def ctc_collate_fn(data) Creates mini-batch tensors from the list of tuples (src_seq, trg_seq). We should build a custom collate_fn rather than using default collate_fn, because merging sequences (including padding) is not supported in default. Sequences are padded to the maximum length of mini-batch sequences (dynamic padding). Arguments : data - list of tuple (src_seq, src_length, trg_seq, trg_length). src_seq: torch tensor of shape (x,?); variable length. src length: torch tenso of shape 1x1 trg_seq: torch tensor of shape (?); variable length. trg_length: torch_tensor of shape (1x1) Returns - tuple of four torch tensors src_seqs - torch tensor of shape (batch_size, x, padded_length). src_lengths - torch_tensor of shape (batch_size); valid length for each padded source sequence. trg_seqs - torch tensor of shape (batch_size, x, padded_length). trg_lengths - torch tensor of shape (batch_size); valid length for each padded target sequence. hannah.datasets.speech SpeechDataset Objects class SpeechDataset(AbstractDataset) Base Class for speech datasets preprocess def preprocess(example, silence=False, label=0) Run preprocessing and feature extraction SpeechCommandsDataset Objects class SpeechCommandsDataset(SpeechDataset) This class implements reading and preprocessing of speech commands like dataset SpeechHotwordDataset Objects class SpeechHotwordDataset(SpeechDataset) Dataset Class for Hotword dataset e.g. Hey Snips! splits @classmethod def splits(cls, config) Splits the dataset in training, devlopment and test set and returns the three sets as List VadDataset Objects class VadDataset(SpeechDataset) splits @classmethod def splits(cls, config) Splits the dataset in training, devlopment and test set and returns the three sets as List hannah.datasets.activity Data3D Objects class Data3D() 3D-Data PAMPAP2_IMUData Objects class PAMPAP2_IMUData() A IMU set defined by temperature (\u00b0C) 3D-acceleration data (ms -2 ), scale: \u00b116g, resolution: 13-bit 3D-acceleration data (ms -2 ), scale: \u00b16g, resolution: 13-bit 3D-gyroscope data (rad/s) 3D-magnetometer data (\u03bcT) orientation (invalid in this data collection) PAMAP2_DataPoint Objects class PAMAP2_DataPoint() A temporal datapoint in the dataset PAMAP2_DataChunk Objects class PAMAP2_DataChunk() A DataChunk is a item of the pytorch dataset PAMAP2_Dataset Objects class PAMAP2_Dataset(AbstractDataset) Class for the PAMAP2 activity dataset https://archive.ics.uci.edu/ml/datasets/pamap2+physical+activity+monitoring hannah.datasets.directional DirectionalDataset Objects class DirectionalDataset(AbstractDataset) Directional Dataset hannah.datasets.DatasetSplit hannah.datasets hannah.datasets.physio AtrialFibrillationDataset Objects class AtrialFibrillationDataset(PhysioDataset) Atrial Fibrillation Database (https://physionet.org/content/afdb/1.0.0/) hannah.datasets.NoiseDataset hannah.datasets.fake1d hannah.datasets.utils.cache hannah.datasets.utils hannah.datasets.utils.md5 hannah.datasets.base DatasetType Objects class DatasetType(Enum) The type of a dataset partition e.g. train, dev, test AbstractDataset Objects class AbstractDataset(Dataset, ABC) prepare @abstractclassmethod def prepare(cls, config: Dict[str, Any]) -> None Prepare the dataset. This method is run at the beginning of the dataset training. If possible it should download the dataset from its original source, if it is available for public download. Arguments : config Dict[Any] - The dataset configuration splits @abstractclassmethod def splits( cls, config: Dict[str, Any] ) -> Tuple[\"AbstractDataset\", \"AbstractDataset\", \"AbstractDataset\"] Returns the test, validation and train split according to the Dataset config Arguments : config [type] - [description] class_names @abstractproperty def class_names() -> List[str] Returns the names of the classes in the classification dataset class_counts @abstractproperty def class_counts() -> Optional[Dict[int, int]] Returns the number of items in each class of the dataset If this is not applicable to a dataset type e.g. ASR, Semantic Segementation, it may return None __getitem__ @abstractmethod def __getitem__(index) -> List[torch.Tensor] Returns a torch.Tensor for the data item at the corresponding index The length of the list depends on the dataset item to use Arguments : index int - the index of the data item __len__ @abstractmethod def __len__() -> int Returns number of samples in dataset size def size() -> List[int] Returns dimension of output without batch dimension std @property def std() -> Optional[Tuple[int, ...]] Returns channel-wise standard deviation for dataset if applicable mean @property def mean() -> Optional[Tuple[int, ...]] Returns channel-wise means for dataset if applicable resolution @property def resolution() -> Optional[Tuple[int, ...]] Returns resolution for dataset if applicable weights @property def weights() -> Optional[List[float]] Class weights for weighted sampling hannah.datasets.Kitti Kitti Objects class Kitti(AbstractDataset) splits @classmethod def splits(cls, config) Splits the dataset in training, devlopment and test set and returns the three sets as List hannah.nn.quantized hannah.nn.qat Implementations of torch.nn.intrinsics qat with an optional quantize bias parameter. Qconfigs can support an optional bias quantization funciton which should be returned by qconfig.bias() else biases will be quantized with qconfig.activation() _ConvBnNd Objects class _ConvBnNd(nn.modules.conv._ConvNd, _ConvForwardMixin) train def train(mode: bool = True) -> Any Batchnorm's training behavior is using the self.training flag. Prevent changing it if BN is frozen. This makes sure that calling model.train() on a model with a frozen BN will behave properly. from_float @classmethod def from_float(cls, mod) Create a qat module from a float module or qparams_dict Args: mod a float module, either produced by torch.quantization utilities or directly from user ConvBn1d Objects class ConvBn1d(_ConvBnNd) A ConvBn1d module is a module fused from Conv1d and BatchNorm1d, attached with FakeQuantize modules for weight, used in quantization aware training. We combined the interface of :class: torch.nn.Conv1d and :class: torch.nn.BatchNorm1d . Similar to :class: torch.nn.Conv1d , with FakeQuantize modules initialized to default. Attributes : freeze_bn: - weight_fake_quant - fake quant module for weight ConvBnReLU1d Objects class ConvBnReLU1d(ConvBn1d) A ConvBnReLU1d module is a module fused from Conv1d, BatchNorm1d and ReLU, attached with FakeQuantize modules for weight, used in quantization aware training. We combined the interface of :class: torch.nn.Conv1d and :class: torch.nn.BatchNorm1d and :class: torch.nn.ReLU . Similar to torch.nn.Conv1d , with FakeQuantize modules initialized to default. Attributes : weight_fake_quant - fake quant module for weight ConvBn2d Objects class ConvBn2d(_ConvBnNd) A ConvBn2d module is a module fused from Conv2d and BatchNorm2d, attached with FakeQuantize modules for weight, used in quantization aware training. We combined the interface of :class: torch.nn.Conv2d and :class: torch.nn.BatchNorm2d . Similar to :class: torch.nn.Conv2d , with FakeQuantize modules initialized to default. Attributes : freeze_bn: - weight_fake_quant - fake quant module for weight ConvBnReLU2d Objects class ConvBnReLU2d(ConvBn2d) A ConvBnReLU2d module is a module fused from Conv2d, BatchNorm2d and ReLU, attached with FakeQuantize modules for weight, used in quantization aware training. We combined the interface of :class: torch.nn.Conv2d and :class: torch.nn.BatchNorm2d and :class: torch.nn.ReLU . Similar to torch.nn.Conv2d , with FakeQuantize modules initialized to default. Attributes : weight_fake_quant - fake quant module for weight ConvReLU2d Objects class ConvReLU2d(nn.Conv2d, _ConvForwardMixin) A ConvReLU2d module is a fused module of Conv2d and ReLU, attached with FakeQuantize modules for weight for quantization aware training. We combined the interface of :class: ~torch.nn.Conv2d and :class: ~torch.nn.BatchNorm2d . Attributes : weight_fake_quant - fake quant module for weight ConvReLU1d Objects class ConvReLU1d(nn.Conv1d, _ConvForwardMixin) A ConvReLU1d module is fused module of Conv1d and ReLU, attached with FakeQuantize modules for quantization aware training Conv1d Objects class Conv1d(nn.Conv1d, _ConvForwardMixin) A Conv1d module is a Conv1d module , attached with FakeQuantize modules for weight for quantization aware training. Attributes : weight_fake_quant - fake quant module for weight bias_fake_quant - fake quant module for bias activation_post_process - fake_quant_module for activations Conv2d Objects class Conv2d(nn.Conv2d, _ConvForwardMixin) A Conv2d module is a Conv2d module , attached with FakeQuantize modules for weight for quantization aware training. Attributes : weight_fake_quant - fake quant module for weight bias_fake_quant - fake quant module for bias activation_post_process - fake_quant_module for activations Linear Objects class Linear(nn.Linear) A linear module attached with FakeQuantize modules for weight, used for quantization aware training. We adopt the same interface as torch.nn.Linear , please see https://pytorch.org/docs/stable/nn.html#torch.nn.Linear for documentation. Similar to torch.nn.Linear , with FakeQuantize modules initialized to default. Attributes : weight - fake quant module for weight from_float @classmethod def from_float(cls, mod) Create a qat module from a float module or qparams_dict Args: mod a float module, either produced by torch.quantization utilities or directly from user LinearReLU Objects class LinearReLU(nn.Linear) A linear module attached with FakeQuantize modules and ReLU for weight, used for quantization aware training. We adopt the same interface as torch.nn.Linear , please see https://pytorch.org/docs/stable/nn.html#torch.nn.Linear for documentation. Similar to torch.nn.Linear , with FakeQuantize modules initialized to default. Attributes : weight - fake quant module for weight from_float @classmethod def from_float(cls, mod) Create a qat module from a float module or qparams_dict Args: mod a float module, either produced by torch.quantization utilities or directly from user Identity Objects class Identity(nn.Identity) A identity module attached with FakeQuantize modules for weight, used for quantization aware training. We adopt the same interface as torch.nn.Identity , please see https://pytorch.org/docs/stable/nn.html#torch.nn.Identity for documentation. Similar to torch.nn.Identity , with FakeQuantize modules initialized to default. from_float @classmethod def from_float(cls, mod) Create a qat module from a float module or qparams_dict Args: mod a float module, either produced by torch.quantization utilities or directly from user hannah.quantization.callback hannah.quantization.qconfig hannah.quantization.rounding round_downward def round_downward(x: Tensor) -> Tensor Round to nearest upward round_upward def round_upward(x: Tensor) -> Tensor Round to nearest downward round_odd def round_odd(x: Tensor) -> Tensor Round to nearest odd round_even def round_even(x: Tensor) -> Tensor Round to nearest even round_zero def round_zero(x: Tensor) -> Tensor Round towards zero round_infinity def round_infinity(x: Tensor) -> Tensor Round toward infinity truncate_up def truncate_up(x: Tensor) -> Tensor Always round up to next integer truncate_down def truncate_down(x: Tensor) -> Tensor Always round down to next integer truncate_infinity def truncate_infinity(x: Tensor) -> Tensor Always round to next integer in direction infinity truncate_zero def truncate_zero(x: Tensor) -> Tensor Always round to next integer in direction of Zero round_stochastic def round_stochastic(x: Tensor) -> Tensor Round stochastically hannah.modules.vision.anomaly_score class AnomalyScore(CatMetric): def init (self, percentile, nan_strategy=\"warn\", kwargs): super(). init (nan_strategy=nan_strategy, kwargs) self.percentile = percentile def compute(self): anomaly_score = None train_losses = super().compute() if train_losses: normalized_train_errors = torch.stack(train_losses) / ( torch.max(torch.stack(train_losses), dim=0).values ) anomaly_score = np.percentile( normalized_train_errors.cpu().numpy(), self.percentile ) return anomaly_score hannah.modules.vision.anomaly_detection AnomalyDetectionModule Objects class AnomalyDetectionModule(VisionBaseModule) on_test_end def on_test_end() wd_dir = os.getcwd() score, largest_train_error = self.compute_anomaly_score() train_errors = self.normalized_train_errors plt.hist(train_errors.detach().cpu().numpy(), bins=100) plt.axvline(score, linestyle=\"dashed\") plt.title(\"Normalized train reconstruction errors\") plt.savefig(wd_dir + \"/normalized_train_errors.png\") test = ( torch.tensor(self.test_losses, device=self.device) / torch.max(torch.stack(self.train_losses), dim=0).values ) plt.hist(test.detach().cpu().numpy(), bins=100) plt.title(\"Normalized test reconstruction errors\") plt.savefig(wd_dir + \"/normalized_test_errors.png\") print(\"Anomaly score\", score) print( \"Largest train error\", torch.max(torch.stack(self.train_losses), dim=0).values, ) hannah.modules.vision.loss hannah.modules.vision hannah.modules.vision.image_classifier hannah.modules.vision.base hannah.modules.classifier hannah.modules.object_detection hannah.modules.config_utils dump_config def dump_config(output_dir, config) Dumps the configuration to json format Creates file config.json in output_dir Parameters output_dir : str Output directory config : dict Configuration to dump save_model def save_model(output_dir, model) Creates serialization of the model for later inference, evaluation Creates the following files: model.pt: Serialized version of network parameters in pytorch model.json: Serialized version of network parameters in json format model.onnx: full model including paramters in onnx format Parameters output_dir : str Directory to put serialized models model : LightningModule Model to serialize hannah.modules hannah.modules.angle_classifier hannah.modules.metrics Error Objects class Error() Computes Error = 1 - Accuracy_ .. math:: \\text{Error} = 1 - \\frac{1}{N}\\sum_i^N 1(y_i = \\hat{y}_i) Where :math: y is a tensor of target values, and :math: \\hat{y} is a tensor of predictions. This module is a simple wrapper to get the task specific versions of this metric, which is done by setting the task argument to either 'binary' , 'multiclass' or multilabel . See the documentation of :mod: BinaryError , :mod: MulticlassError and :mod: MultilabelError for the specific details of each argument influence and examples. plot_confusion_matrix def plot_confusion_matrix(cf, group_names=None, categories=\"auto\", count=True, percent=True, cbar=True, xyticks=True, xyplotlabels=True, sum_stats=True, figsize=None, cmap=\"Blues\", title=None) This function will make a pretty plot of an sklearn Confusion Matrix cm using a Seaborn heatmap visualization. Arguments cf: confusion matrix to be passed in group_names: List of strings that represent the labels row by row to be shown in each square. categories: List of strings containing the categories to be displayed on the x,y axis. Default is 'auto' count: If True, show the raw number in the confusion matrix. Default is True. normalize: If True, show the proportions for each category. Default is True. cbar: If True, show the color bar. The cbar values are based off the values in the confusion matrix. Default is True. xyticks: If True, show x and y ticks. Default is True. xyplotlabels: If True, show 'True Label' and 'Predicted Label' on the figure. Default is True. sum_stats: If True, display summary statistics below the figure. Default is True. figsize: Tuple representing the figure size. Default will be the matplotlib rcParams value. cmap: Colormap of the values displayed from matplotlib.pyplot.cm. Default is 'Blues' See http://matplotlib.org/examples/color/colormaps_reference.html title: Title for the heatmap. Default is None. hannah.modules.base ClassifierModule Objects class ClassifierModule(LightningModule, ABC) total_training_steps def total_training_steps() -> int Total training steps inferred from datamodule and devices. hannah.modules.augmentation.augmentation hannah.modules.augmentation hannah.modules.augmentation.batch_augmentation BatchAugmentationPipeline Objects class BatchAugmentationPipeline(nn.Module) __init__ def __init__(transforms={}) Augmentation pipeline especially for self supervised learning Arguments : replica int - number of replicated different random augmentations transforms dict - configuration of transforms forward @torch.no_grad() def forward(x) -> torch.Tensor Perform Augmentations Arguments : x torch.Tensor - a torch.Tensor representing the augementation pipeline Returns : Tuple[torch.Tensor, torch.Tensor]; Batch augmented with replica different random augmentations hannah.modules.augmentation.transforms.registry hannah.modules.augmentation.transforms.kornia_transforms hannah.modules.augmentation.transforms hannah.modules.augmentation.bordersearch hannah.utils.imports lazy_import def lazy_import(module_name, callback=None) Returns a proxy module object that will lazily import the given module the first time it is used. Example usage:: Lazy version of import tensorflow as tf tf = lazy_import(\"tensorflow\") Other commands Now the module is loaded tf. version Arguments : module_name - the fully-qualified module name to import callback None - a callback function to call before importing the module Returns : a proxy module object that will be lazily imported when first used LazyModule Objects class LazyModule(types.ModuleType) Proxy module that lazily imports the underlying module the first time it is actually used. Arguments : module_name - the fully-qualified module name to import callback None - a callback function to call before importing the module hannah.utils hannah.utils.utils log_execution_env_state def log_execution_env_state() -> None Log information about the execution environment. File 'config_path' will be copied to directory 'logdir'. A common use-case is passing the path to a (compression) schedule YAML file. Storing a copy of the schedule file, with the experiment logs, is useful in order to reproduce experiments. Arguments : config_path - path to config file, used only when logdir is set logdir - log directory git_root - the path to the .git root directory extract_from_download_cache def extract_from_download_cache(filename, url, cached_files, target_cache, target_folder, target_test_folder=\"\", clear_download=False, no_exist_check=False) -> None extracts given file from cache or donwloads first from url Arguments : filename str - name of the file to download or extract url str - possible url to download the file cached_files (list(str)): cached files in download cache target_cache str - path to the folder to cache file if download necessary target_folder str - path where to extract file target_test_folder str, optional - folder to check if data are already there clear_download bool - clear download after usage no_exist_check bool - disables the check if folder exists fullname def fullname(o) -> Any Get the full classname of an object including surrounding packages/modules/namespaces set_deterministic @contextmanager def set_deterministic(mode, warn_only=False) A contextmanager to set deterministic algorithms hannah.visualization","title":"API Documentation"},{"location":"development/devel/api/#hannahtoolsobjectdetection_eval","text":"","title":"hannah.tools.objectdetection_eval"},{"location":"development/devel/api/#eval_train","text":"def eval_train(config, module, test=True) Arguments : config - param module: test - Default value = True) module:","title":"eval_train"},{"location":"development/devel/api/#eval_steps","text":"def eval_steps(config, module, hparams, checkpoint) Arguments : config - param module: hparams - param checkpoint: module: checkpoint:","title":"eval_steps"},{"location":"development/devel/api/#eval_checkpoint","text":"def eval_checkpoint(config: DictConfig, checkpoint) Arguments : config - DictConfig: checkpoint: config - DictConfig: config - DictConfig:","title":"eval_checkpoint"},{"location":"development/devel/api/#eval","text":"def eval(config: DictConfig) Arguments : config - DictConfig: config - DictConfig: config - DictConfig:","title":"eval"},{"location":"development/devel/api/#main","text":"@hydra.main(config_name=\"objectdetection_eval\", config_path=\"../conf\", version_base=\"1.2\") def main(config: DictConfig) Arguments : config - DictConfig: config - DictConfig: config - DictConfig:","title":"main"},{"location":"development/devel/api/#hannahtoolstrain","text":"","title":"hannah.tools.train"},{"location":"development/devel/api/#hannahtools","text":"","title":"hannah.tools"},{"location":"development/devel/api/#hannahtoolscharacterize","text":"","title":"hannah.tools.characterize"},{"location":"development/devel/api/#main_1","text":"@hydra.main(config_name=\"characterize\", config_path=\"../conf\", version_base=\"1.2\") def main(config: DictConfig) Arguments : config - DictConfig: config - DictConfig: config - DictConfig:","title":"main"},{"location":"development/devel/api/#hannahtoolseval","text":"","title":"hannah.tools.eval"},{"location":"development/devel/api/#eval_checkpoint_1","text":"def eval_checkpoint(config: DictConfig, checkpoint) -> None Arguments : config - DictConfig: checkpoint: config - DictConfig: config - DictConfig:","title":"eval_checkpoint"},{"location":"development/devel/api/#eval_1","text":"def eval(config: DictConfig) -> Optional[bool] Arguments : config - DictConfig: config - DictConfig: config - DictConfig:","title":"eval"},{"location":"development/devel/api/#main_2","text":"@hydra.main(config_name=\"eval\", config_path=\"conf\", version_base=\"1.2\") def main(config: DictConfig) Arguments : config - DictConfig: config - DictConfig: config - DictConfig:","title":"main"},{"location":"development/devel/api/#hannahtrainer","text":"","title":"hannah.trainer"},{"location":"development/devel/api/#hannahtrainercross_validation","text":"","title":"hannah.trainer.cross_validation"},{"location":"development/devel/api/#hannahtrain","text":"","title":"hannah.train"},{"location":"development/devel/api/#hannahlogo","text":"","title":"hannah.logo"},{"location":"development/devel/api/#hannahfeatures","text":"","title":"hannah.features"},{"location":"development/devel/api/#mfcc-objects","text":"class MFCC(torchaudio.transforms.MFCC) A simple wrapper around torchaudio mfcc, but melkwargs are given as direct named arguments instead of a dictionary","title":"MFCC Objects"},{"location":"development/devel/api/#sincconv-objects","text":"class SincConv(nn.Module) Sinc convolution: Arguments : in_channels - No. of input channels(must be 1) out_channels - No. of filters(40) sample_rate - sampling rate, default set at 32000 kernel_size - Filter length(101)","title":"SincConv Objects"},{"location":"development/devel/api/#logspectrogram-objects","text":"class LogSpectrogram(torch.nn.Module) Create a spectrogram from a audio signal. Arguments : n_fft int, optional - Size of FFT, creates n_fft // 2 + 1 bins. (Default: 400 ) win_length int or None, optional - Window size. (Default: n_fft ) hop_length int or None, optional - Length of hop between STFT windows. (Default: win_length // 2 ) pad int, optional - Two sided padding of signal. (Default: 0 ) window_fn Callable[..., Tensor], optional - A function to create a window tensor that is applied/multiplied to each frame/window. (Default: torch.hann_window ) power float or None, optional - Exponent for the magnitude spectrogram, (must be > 0) e.g., 1 for energy, 2 for power, etc. If None, then the complex spectrum is returned instead. (Default: 2 ) normalized bool, optional - Whether to normalize by magnitude after stft. (Default: False ) wkwargs dict or None, optional - Arguments for window function. (Default: None ) center bool, optional - whether to pad :attr: waveform on both sides so that the :math: t -th frame is centered at time :math: t \\times \\text{hop\\_length} . Default - True pad_mode string, optional - controls the padding method used when :attr: center is True . Default: \"reflect\" onesided bool, optional - controls whether to return half of results to avoid redundancy Default: True","title":"LogSpectrogram Objects"},{"location":"development/devel/api/#forward","text":"def forward(waveform: torch.Tensor) -> torch.Tensor Arguments : waveform Tensor - Tensor of audio of dimension (..., time). Returns : Tensor - Dimension (..., freq, time), where freq is n_fft // 2 + 1 where n_fft is the number of Fourier bins, and time is the number of window hops (n_frame).","title":"forward"},{"location":"development/devel/api/#hannahnormalizer","text":"","title":"hannah.normalizer"},{"location":"development/devel/api/#fixedpointnormalizer-objects","text":"class FixedPointNormalizer(nn.Module) Simple feature normalizer for fixed point models","title":"FixedPointNormalizer Objects"},{"location":"development/devel/api/#adaptivefixedpointnormalizer-objects","text":"class AdaptiveFixedPointNormalizer(nn.Module) Simple feature normalizer for fixed point models","title":"AdaptiveFixedPointNormalizer Objects"},{"location":"development/devel/api/#hannahmodelshonk","text":"","title":"hannah.models.honk"},{"location":"development/devel/api/#hannahmodelshonkmodel","text":"","title":"hannah.models.honk.model"},{"location":"development/devel/api/#truncated_normal","text":"def truncated_normal(tensor, std_dev=0.01) Arguments : tensor: - std_dev - (Default value = 0.01)","title":"truncated_normal"},{"location":"development/devel/api/#speechresmodel-objects","text":"class SpeechResModel(nn.Module)","title":"SpeechResModel Objects"},{"location":"development/devel/api/#forward_1","text":"def forward(x) Arguments : x:","title":"forward"},{"location":"development/devel/api/#speechmodel-objects","text":"class SpeechModel(nn.Module)","title":"SpeechModel Objects"},{"location":"development/devel/api/#forward_2","text":"def forward(x) Arguments : x:","title":"forward"},{"location":"development/devel/api/#hannahmodelshellomodels","text":"","title":"hannah.models.hello.models"},{"location":"development/devel/api/#dsconv2d-objects","text":"class DSConv2d(nn.Module)","title":"DSConv2d Objects"},{"location":"development/devel/api/#forward_3","text":"def forward(x) Arguments : x:","title":"forward"},{"location":"development/devel/api/#dscnnspeechmodel-objects","text":"class DSCNNSpeechModel(nn.Module)","title":"DSCNNSpeechModel Objects"},{"location":"development/devel/api/#forward_4","text":"def forward(x) Arguments : x:","title":"forward"},{"location":"development/devel/api/#dnnspeechmodel-objects","text":"class DNNSpeechModel(nn.Module)","title":"DNNSpeechModel Objects"},{"location":"development/devel/api/#forward_5","text":"def forward(x) Arguments : x:","title":"forward"},{"location":"development/devel/api/#hannahmodelshello","text":"","title":"hannah.models.hello"},{"location":"development/devel/api/#hannahmodelsobjectdetectionloss","text":"","title":"hannah.models.objectdetection.loss"},{"location":"development/devel/api/#bbox_iou","text":"def bbox_iou(box1, box2, x1y1x2y2=True, GIoU=False, DIoU=False, CIoU=False, eps=1e-7) Arguments : box1: box2: - x1y1x2y2 - (Default value = True) - GIoU - (Default value = False) - DIoU - (Default value = False) - CIoU - (Default value = False) - eps - (Default value = 1e-7)","title":"bbox_iou"},{"location":"development/devel/api/#is_parallel","text":"def is_parallel(model) Arguments : model:","title":"is_parallel"},{"location":"development/devel/api/#bceblurwithlogitsloss-objects","text":"class BCEBlurWithLogitsLoss(nn.Module) Arguments : eps - (Default value = 0.1) ) - # https://github.com/ultralytics/yolov3/issues/238#issuecomment-598028441# return positive: negative label smoothing BCE targetsreturn 1.0 - 0.5 * eps: 0.5 * epsclass BCEBlurWithLogitsLoss(nn.Module: ) - # https://github.com/ultralytics/yolov3/issues/238#issuecomment-598028441# return positive:","title":"BCEBlurWithLogitsLoss Objects"},{"location":"development/devel/api/#forward_6","text":"def forward(pred, true) Arguments : pred: true:","title":"forward"},{"location":"development/devel/api/#focalloss-objects","text":"class FocalLoss(nn.Module)","title":"FocalLoss Objects"},{"location":"development/devel/api/#forward_7","text":"def forward(pred, true) Arguments : pred: true:","title":"forward"},{"location":"development/devel/api/#qfocalloss-objects","text":"class QFocalLoss(nn.Module)","title":"QFocalLoss Objects"},{"location":"development/devel/api/#forward_8","text":"def forward(pred, true) Arguments : pred: true:","title":"forward"},{"location":"development/devel/api/#computeloss-objects","text":"class ComputeLoss()","title":"ComputeLoss Objects"},{"location":"development/devel/api/#build_targets","text":"def build_targets(p, targets) Arguments : p: targets:","title":"build_targets"},{"location":"development/devel/api/#hannahmodelsobjectdetectionmodels","text":"","title":"hannah.models.objectdetection.models"},{"location":"development/devel/api/#fasterrcnn-objects","text":"class FasterRCNN(torch.nn.Module)","title":"FasterRCNN Objects"},{"location":"development/devel/api/#forward_9","text":"def forward(x, y=None) Arguments : x: - y - (Default value = None)","title":"forward"},{"location":"development/devel/api/#transformoutput","text":"def transformOutput(cocoGt, output, x, y) Arguments : cocoGt: output: x: y:","title":"transformOutput"},{"location":"development/devel/api/#ultralyticsyolo-objects","text":"class UltralyticsYolo(torch.nn.Module)","title":"UltralyticsYolo Objects"},{"location":"development/devel/api/#forward_10","text":"def forward(x, y=None) Arguments : x: - y - (Default value = None)","title":"forward"},{"location":"development/devel/api/#train","text":"def train(mode=True) Arguments : mode - (Default value = True)","title":"train"},{"location":"development/devel/api/#transformoutput_1","text":"def transformOutput(cocoGt, output, x, y) Arguments : cocoGt: output: x: y:","title":"transformOutput"},{"location":"development/devel/api/#hannahmodelsobjectdetection","text":"","title":"hannah.models.objectdetection"},{"location":"development/devel/api/#hannahmodelstcmodels","text":"","title":"hannah.models.tc.models"},{"location":"development/devel/api/#create_act","text":"def create_act(act, clipping_value) Arguments : act: clipping_value:","title":"create_act"},{"location":"development/devel/api/#approximateglobalaveragepooling1d-objects","text":"class ApproximateGlobalAveragePooling1D(nn.Module)","title":"ApproximateGlobalAveragePooling1D Objects"},{"location":"development/devel/api/#forward_11","text":"def forward(x) Arguments : x:","title":"forward"},{"location":"development/devel/api/#tcresidualblock-objects","text":"class TCResidualBlock(nn.Module)","title":"TCResidualBlock Objects"},{"location":"development/devel/api/#forward_12","text":"def forward(x) Arguments : x:","title":"forward"},{"location":"development/devel/api/#tcresnetmodel-objects","text":"class TCResNetModel(nn.Module)","title":"TCResNetModel Objects"},{"location":"development/devel/api/#forward_13","text":"def forward(x) Arguments : x:","title":"forward"},{"location":"development/devel/api/#exitwrapperblock-objects","text":"class ExitWrapperBlock(nn.Module)","title":"ExitWrapperBlock Objects"},{"location":"development/devel/api/#forward_14","text":"def forward(x) Arguments : x:","title":"forward"},{"location":"development/devel/api/#branchytcresnetmodel-objects","text":"class BranchyTCResNetModel(TCResNetModel)","title":"BranchyTCResNetModel Objects"},{"location":"development/devel/api/#on_val","text":"def on_val()","title":"on_val"},{"location":"development/devel/api/#on_val_end","text":"def on_val_end()","title":"on_val_end"},{"location":"development/devel/api/#on_test","text":"def on_test()","title":"on_test"},{"location":"development/devel/api/#on_test_end","text":"def on_test_end()","title":"on_test_end"},{"location":"development/devel/api/#reset_stats","text":"def reset_stats()","title":"reset_stats"},{"location":"development/devel/api/#print_stats","text":"def print_stats()","title":"print_stats"},{"location":"development/devel/api/#forward_15","text":"def forward(x) Arguments : x:","title":"forward"},{"location":"development/devel/api/#get_loss_function","text":"def get_loss_function()","title":"get_loss_function"},{"location":"development/devel/api/#hannahmodelstc","text":"","title":"hannah.models.tc"},{"location":"development/devel/api/#hannahmodelsconvnetmodels","text":"","title":"hannah.models.convnet.models"},{"location":"development/devel/api/#padding_expression","text":"def padding_expression(kernel_size, stride, dilation=1) Symbolically calculate padding such that for a given kernel_size, stride and dilation the padding is such that the output dimension is kept the same(stride=1) or halved(stride=2). Note: If the input dimension is 1 and stride = 2, the calculated padding will result in an output with also dimension 1.","title":"padding_expression"},{"location":"development/devel/api/#parameters","text":"kernel_size : Union[int, Expression] stride : Union[int, Expression] dilation : Union[int, Expression], optional description , by default 1","title":"Parameters"},{"location":"development/devel/api/#returns","text":"Expression","title":"Returns"},{"location":"development/devel/api/#hannahmodelsconvnet","text":"","title":"hannah.models.convnet"},{"location":"development/devel/api/#hannahmodelstimm","text":"","title":"hannah.models.timm"},{"location":"development/devel/api/#defaultanomalydetector-objects","text":"class DefaultAnomalyDetector(nn.Module)","title":"DefaultAnomalyDetector Objects"},{"location":"development/devel/api/#forward_16","text":"def forward(x) Arguments : x:","title":"forward"},{"location":"development/devel/api/#defaultclassifierhead-objects","text":"class DefaultClassifierHead(nn.Module)","title":"DefaultClassifierHead Objects"},{"location":"development/devel/api/#forward_17","text":"def forward(x: torch.Tensor) -> torch.Tensor Arguments : x (torch.Tensor): Returns : Resulting torch.Tensor after applying classification","title":"forward"},{"location":"development/devel/api/#defaultprojectionhead-objects","text":"class DefaultProjectionHead(nn.Module) Default projection head for semi supervised classification learning","title":"DefaultProjectionHead Objects"},{"location":"development/devel/api/#forward_18","text":"def forward(x: torch.Tensor) -> torch.Tensor Forward function for default Projection Head Arguments : x - Input tensor Returns : output tensor","title":"forward"},{"location":"development/devel/api/#defaultdecoderhead-objects","text":"class DefaultDecoderHead(nn.Module)","title":"DefaultDecoderHead Objects"},{"location":"development/devel/api/#__init__","text":"def __init__(latent_shape, input_shape) Default Decoder Head for autoencoders using TransposedConv2D Arguments : latent_shape(Tuple) - Shape (CxHxW) of the latent representation of the autoencoder input_shape(Tuple) - Shape (CxHxW) of the reconstructed image","title":"__init__"},{"location":"development/devel/api/#forward_19","text":"def forward(x) Arguments : x:","title":"forward"},{"location":"development/devel/api/#timmmodel-objects","text":"class TimmModel(nn.Module)","title":"TimmModel Objects"},{"location":"development/devel/api/#forward_20","text":"def forward(x: torch.Tensor) -> torch.Tensor Arguments : x - torch.Tensor: x - torch.Tensor:","title":"forward"},{"location":"development/devel/api/#hannahmodels","text":"","title":"hannah.models"},{"location":"development/devel/api/#hannahmodelslstmmodels","text":"","title":"hannah.models.lstm.models"},{"location":"development/devel/api/#lstmmodel-objects","text":"class LSTMModel(nn.Module) Simple LSTM model.","title":"LSTMModel Objects"},{"location":"development/devel/api/#forward_21","text":"def forward(x) Arguments : x:","title":"forward"},{"location":"development/devel/api/#hannahmodelslstm","text":"","title":"hannah.models.lstm"},{"location":"development/devel/api/#hannahmodelsfactoryreduction","text":"","title":"hannah.models.factory.reduction"},{"location":"development/devel/api/#reductionblockadd-objects","text":"class ReductionBlockAdd(nn.Module) Reduction block that sums over its inputs","title":"ReductionBlockAdd Objects"},{"location":"development/devel/api/#forward_22","text":"def forward(x: Tensor) -> Tensor Arguments : x - Tensor: x - Tensor:","title":"forward"},{"location":"development/devel/api/#reductionblockconcat-objects","text":"class ReductionBlockConcat(nn.Module) Reduction block that concatenates its inputs","title":"ReductionBlockConcat Objects"},{"location":"development/devel/api/#forward_23","text":"def forward(x: Tensor) -> Tensor Arguments : x - Tensor: x - Tensor:","title":"forward"},{"location":"development/devel/api/#hannahmodelsfactoryact","text":"","title":"hannah.models.factory.act"},{"location":"development/devel/api/#dummyactivation-objects","text":"class DummyActivation(nn.Identity) Dummy class that instantiated to mark a missing activation. This can be used to mark requantization of activations for convolutional layers without activation functions. Arguments :","title":"DummyActivation Objects"},{"location":"development/devel/api/#hannahmodelsfactory","text":"","title":"hannah.models.factory"},{"location":"development/devel/api/#hannahmodelsfactoryquantized","text":"Import from new loacation for backwards compatibility","title":"hannah.models.factory.quantized"},{"location":"development/devel/api/#hannahmodelsfactoryqconfig","text":"Import from new loacation for backwards compatibility","title":"hannah.models.factory.qconfig"},{"location":"development/devel/api/#hannahmodelsfactorynetwork","text":"","title":"hannah.models.factory.network"},{"location":"development/devel/api/#convnet-objects","text":"class ConvNet(nn.Module)","title":"ConvNet Objects"},{"location":"development/devel/api/#forward_24","text":"def forward(x) Arguments : x:","title":"forward"},{"location":"development/devel/api/#hannahmodelsfactoryfactory","text":"A neural network model factory It allows us to construct quantized and unquantized versions of the same network, allows to explore implementation alternatives using a common neural network construction interface.","title":"hannah.models.factory.factory"},{"location":"development/devel/api/#normconfig-objects","text":"@dataclass class NormConfig()","title":"NormConfig Objects"},{"location":"development/devel/api/#bnconfig-objects","text":"@dataclass class BNConfig(NormConfig)","title":"BNConfig Objects"},{"location":"development/devel/api/#actconfig-objects","text":"@dataclass class ActConfig()","title":"ActConfig Objects"},{"location":"development/devel/api/#eluconfig-objects","text":"@dataclass class ELUConfig(ActConfig)","title":"ELUConfig Objects"},{"location":"development/devel/api/#hardtanhconfig-objects","text":"@dataclass class HardtanhConfig(ActConfig)","title":"HardtanhConfig Objects"},{"location":"development/devel/api/#minorblockconfig-objects","text":"@dataclass class MinorBlockConfig()","title":"MinorBlockConfig Objects"},{"location":"development/devel/api/#target","text":"target Operation","title":"target"},{"location":"development/devel/api/#parallel","text":"execute block in parallel with preceding block","title":"parallel"},{"location":"development/devel/api/#out_channels","text":"number of output channels","title":"out_channels"},{"location":"development/devel/api/#kernel_size","text":"kernel size of this Operation (if applicable)","title":"kernel_size"},{"location":"development/devel/api/#stride","text":"stride for this operation use","title":"stride"},{"location":"development/devel/api/#padding","text":"use padding for this operation (padding will always try to keep input dimensions / stride)","title":"padding"},{"location":"development/devel/api/#dilation","text":"dilation factor to use for this operation","title":"dilation"},{"location":"development/devel/api/#groups","text":"number of groups for this operation","title":"groups"},{"location":"development/devel/api/#norm","text":"normalization to use (true uses networks default configs)","title":"norm"},{"location":"development/devel/api/#act","text":"activation to use (true uses default configs)","title":"act"},{"location":"development/devel/api/#upsampling","text":"Upsampling factor for mbconv layers","title":"upsampling"},{"location":"development/devel/api/#bias","text":"use bias for this operation","title":"bias"},{"location":"development/devel/api/#out_quant","text":"use output quantization for this operation","title":"out_quant"},{"location":"development/devel/api/#majorblockconfig-objects","text":"@dataclass class MajorBlockConfig()","title":"MajorBlockConfig Objects"},{"location":"development/devel/api/#stride_1","text":"Union[None, int, Tuple[int, ...], Tuple[int, ...]]","title":"stride"},{"location":"development/devel/api/#last","text":"Indicates wether this block is the last reduction block","title":"last"},{"location":"development/devel/api/#linearconfig-objects","text":"@dataclass class LinearConfig()","title":"LinearConfig Objects"},{"location":"development/devel/api/#norm_1","text":"Union[bool, NormConfig]","title":"norm"},{"location":"development/devel/api/#act_1","text":"Union[bool, ActConfig]","title":"act"},{"location":"development/devel/api/#networkconfig-objects","text":"@dataclass class NetworkConfig()","title":"NetworkConfig Objects"},{"location":"development/devel/api/#networkfactory-objects","text":"class NetworkFactory()","title":"NetworkFactory Objects"},{"location":"development/devel/api/#act_2","text":"def act(config: ActConfig) -> nn.Module Arguments : config - ActConfig: config - ActConfig:","title":"act"},{"location":"development/devel/api/#conv2d","text":"def conv2d(input_shape: Tuple[int, ...], out_channels: int, kernel_size: Union[int, Tuple[int, ...]], stride: Union[int, Tuple[int, ...]] = 1, padding: Union[int, Tuple[int, ...], bool] = True, dilation: Union[int, Tuple[int, ...]] = 0, groups: int = 1, norm: Union[BNConfig, bool] = False, act: Union[ActConfig, bool] = False, bias: bool = False) -> None Arguments : input_shape - Tuple[int: int: int]: out_channels - int: kernel_size - Union[int: Tuple[int, ...]]: (Default value = 1) stride - Union[int: padding - Union[int: Tuple[int, ...]: bool] - (Default value = False) dilation - int: (Default value = 0) groups - int: (Default value = 1) norm - Union[BNConfig: act - Union[ActConfig: bias - bool: (Default value = False) input_shape - Tuple[int: out_channels - int: kernel_size - Union[int: stride - Union[int: padding - Union[int: dilation - int: (Default value = 0) groups - int: (Default value = 1) norm - Union[BNConfig: act - Union[ActConfig: bias - bool: (Default value = False)","title":"conv2d"},{"location":"development/devel/api/#mbconv1d","text":"def mbconv1d(input_shape: Tuple[int, ...], out_channels: int, kernel_size: int, dilation: int = 1, stride: int = 1, padding: Union[int, bool] = True, bias=False, upsampling: float = 1.0, groups: int = 1, norm: Union[BNConfig, bool] = False, act: Union[ActConfig, bool] = False) Arguments : input_shape - Tuple[int: int: int]: out_channels - int: kernel_size - int: dilation - int: (Default value = 1) stride - int: (Default value = 1) padding - Union[int: bool] - (Default value = False) bias - (Default value = False) upsampling - float: (Default value = 1.0) groups - int: (Default value = 1) norm - Union[BNConfig: act - Union[ActConfig: input_shape - Tuple[int: out_channels - int: kernel_size - int: dilation - int: (Default value = 1) stride - int: (Default value = 1) padding - Union[int: upsampling - float: (Default value = 1.0) groups - int: (Default value = 1) norm - Union[BNConfig: act - Union[ActConfig:","title":"mbconv1d"},{"location":"development/devel/api/#conv1d","text":"def conv1d(input_shape: Tuple[int, ...], out_channels: int, kernel_size: int, stride: int = 1, bias: bool = False, padding: Union[int, bool] = True, dilation: int = 1, groups: int = 1, norm: Union[BNConfig, bool] = False, act: Union[ActConfig, bool] = False, out_quant: bool = True) -> Tuple[Tuple[int, ...], nn.Module] Arguments : input_shape - Tuple[int: int: int]: out_channels - int: kernel_size - int: stride - int: (Default value = 1) bias - bool: (Default value = False) padding - Union[int: bool] - (Default value = False) dilation - int: (Default value = 1) groups - int: (Default value = 1) norm - Union[BNConfig: act - Union[ActConfig: out_quant - bool: (Default value = True) input_shape - Tuple[int: out_channels - int: kernel_size - int: stride - int: (Default value = 1) bias - bool: (Default value = False) padding - Union[int: dilation - int: (Default value = 1) groups - int: (Default value = 1) norm - Union[BNConfig: act - Union[ActConfig: out_quant - bool: (Default value = True)","title":"conv1d"},{"location":"development/devel/api/#minor","text":"def minor(input_shape, config: MinorBlockConfig, major_stride=None) Arguments : input_shape: - config - MinorBlockConfig: - major_stride - (Default value = None) - config - MinorBlockConfig:","title":"minor"},{"location":"development/devel/api/#forward_25","text":"def forward(input_shape: Tuple[int, ...], config: MajorBlockConfig) Create a forward neural network block without parallelism If parallel is set to [True, False, True, False] Input: ------->|---> parallel: False ---> parallel: False ---> | --> output Arguments : input_shape - Tuple[int, ...]: config - MajorBlockConfig: input_shape - Tuple[int, ...]: config - MajorBlockConfig:","title":"forward"},{"location":"development/devel/api/#residual","text":"def residual(input_shape: Tuple[int, ...], config: MajorBlockConfig) Create a neural network block with with residual parallelism If parallel is set to [True, False, True, False] |---> parallel: True ---> parallel: True ---> | Input: ------->| +---> |---> parallel: False ---> parallel: False ---> | If the major block does change the output dimensions compared to the input and one of the branches does not contain any layers, we infer 1x1 conv of maximum group size (gcd (input_channels, output_channels)) to do the downsampling. Arguments : input_shape - Tuple[int, ...]: config - MajorBlockConfig: input_shape - Tuple[int, ...]: config - MajorBlockConfig:","title":"residual"},{"location":"development/devel/api/#input","text":"def input(in_channels: int, config: MajorBlockConfig) Create a neural network block with input parallelism If parallel is set to [True, False, True, False] |---> parallel: True ---> | |---> parallel: True ---> + -----------------> | Input:--------->| +---> |---> parallel: False ---> parallel: False ---> | If there are no parallel branches in the network. The major block is a standard feed forward layer. Arguments : in_channels - int: config - MajorBlockConfig: in_channels - int: config - MajorBlockConfig:","title":"input"},{"location":"development/devel/api/#full","text":"def full(in_channels: int, config: MajorBlockConfig) Create a neural network block with full parallelism If parallel is set to [True, False, True, False] |---> parallel: True ---------------------------------- -| Input:--->| +---> | |--> parallel: False --->| | |---> parallel: False ----> | +--->| |--> parallel: True ---->| If there are no parallel blocks the block is a standard feed forward network. Arguments : in_channels - int: config - MajorBlockConfig: in_channels - int: config - MajorBlockConfig:","title":"full"},{"location":"development/devel/api/#major","text":"def major(input_shape, config: MajorBlockConfig) Arguments : input_shape: - config - MajorBlockConfig: - config - MajorBlockConfig:","title":"major"},{"location":"development/devel/api/#linear","text":"def linear(input_shape, config: LinearConfig) Arguments : input_shape: - config - LinearConfig: - config - LinearConfig:","title":"linear"},{"location":"development/devel/api/#identity","text":"def identity() -> Identity","title":"identity"},{"location":"development/devel/api/#network","text":"def network(input_shape, labels: int, network_config: Union[ListConfig, DictConfig]) Arguments : input_shape: - labels - int: - network_config - NetworkConfig: - labels - int: - network_config - NetworkConfig:","title":"network"},{"location":"development/devel/api/#create_cnn","text":"def create_cnn(input_shape: Sequence[int], labels: int, name: str, conv: Optional[List[MajorBlockConfig]] = None, linear: Optional[List[LinearConfig]] = None, norm: Optional[NormConfig] = None, act: Optional[ActConfig] = None, qconfig: Any = None, dropout: float = 0.5) Arguments : input_shape - Sequence[int]: labels - int: name - str: conv - Optional[List[MajorBlockConfig]]: (Default value = None) linear - Optional[List[LinearConfig]]: (Default value = None) norm - Optional[NormConfig]: (Default value = None) act - Optional[ActConfig]: (Default value = None) qconfig - Any: (Default value = None) dropout - float: (Default value = 0.5) input_shape - Sequence[int]: labels - int: name - str: conv - Optional[List[MajorBlockConfig]]: (Default value = None) linear - Optional[List[LinearConfig]]: (Default value = None) norm - Optional[NormConfig]: (Default value = None) act - Optional[ActConfig]: (Default value = None) qconfig - Any: (Default value = None) dropout - float: (Default value = 0.5)","title":"create_cnn"},{"location":"development/devel/api/#hannahmodelsfactorypooling","text":"","title":"hannah.models.factory.pooling"},{"location":"development/devel/api/#approximateglobalaveragepooling1d-objects_1","text":"class ApproximateGlobalAveragePooling1D(nn.Module) A global average pooling layer, that divides by the next power of 2 instead of true number of elements","title":"ApproximateGlobalAveragePooling1D Objects"},{"location":"development/devel/api/#forward_26","text":"def forward(x) Arguments : x:","title":"forward"},{"location":"development/devel/api/#approximateglobalaveragepooling2d-objects","text":"class ApproximateGlobalAveragePooling2D(nn.Module) A global average pooling layer, that divides by the next power of 2 instead of true number of elements","title":"ApproximateGlobalAveragePooling2D Objects"},{"location":"development/devel/api/#forward_27","text":"def forward(x) Arguments : x:","title":"forward"},{"location":"development/devel/api/#hannahmodelsfactoryqat","text":"Import from new loacation for backwards compatibility","title":"hannah.models.factory.qat"},{"location":"development/devel/api/#hannahmodelsfactoryrounding","text":"Import from new loacation for backwards compatibility","title":"hannah.models.factory.rounding"},{"location":"development/devel/api/#hannahmodelsresnetmodels","text":"","title":"hannah.models.resnet.models"},{"location":"development/devel/api/#padding_expression_1","text":"def padding_expression(kernel_size, stride, dilation=1) Symbolically calculate padding such that for a given kernel_size, stride and dilation the padding is such that the output dimension is kept the same(stride=1) or halved(stride=2). Note: If the input dimension is 1 and stride = 2, the calculated padding will result in an output with also dimension 1.","title":"padding_expression"},{"location":"development/devel/api/#parameters_1","text":"kernel_size : Union[int, Expression] stride : Union[int, Expression] dilation : Union[int, Expression], optional description , by default 1","title":"Parameters"},{"location":"development/devel/api/#returns_1","text":"Expression","title":"Returns"},{"location":"development/devel/api/#hannahmodelsresnet","text":"","title":"hannah.models.resnet"},{"location":"development/devel/api/#hannahmodelsvadmodels","text":"","title":"hannah.models.vad.models"},{"location":"development/devel/api/#bottleneckvad-objects","text":"class BottleneckVad(nn.Module)","title":"BottleneckVad Objects"},{"location":"development/devel/api/#forward_28","text":"def forward(x) Arguments : x:","title":"forward"},{"location":"development/devel/api/#num_flat_features","text":"def num_flat_features(x) Arguments : x:","title":"num_flat_features"},{"location":"development/devel/api/#smallvad-objects","text":"class SmallVad(nn.Module)","title":"SmallVad Objects"},{"location":"development/devel/api/#forward_29","text":"def forward(x) Arguments : x:","title":"forward"},{"location":"development/devel/api/#num_flat_features_1","text":"def num_flat_features(x) Arguments : x:","title":"num_flat_features"},{"location":"development/devel/api/#simplevad-objects","text":"class SimpleVad(nn.Module)","title":"SimpleVad Objects"},{"location":"development/devel/api/#forward_30","text":"def forward(x) Arguments : x:","title":"forward"},{"location":"development/devel/api/#num_flat_features_2","text":"def num_flat_features(x) Arguments : x:","title":"num_flat_features"},{"location":"development/devel/api/#bottleneckvadmodel-objects","text":"class BottleneckVadModel(nn.Module)","title":"BottleneckVadModel Objects"},{"location":"development/devel/api/#forward_31","text":"def forward(x) Arguments : x:","title":"forward"},{"location":"development/devel/api/#simplevadmodel-objects","text":"class SimpleVadModel(nn.Module)","title":"SimpleVadModel Objects"},{"location":"development/devel/api/#forward_32","text":"def forward(x) Arguments : x:","title":"forward"},{"location":"development/devel/api/#smallvadmodel-objects","text":"class SmallVadModel(nn.Module)","title":"SmallVadModel Objects"},{"location":"development/devel/api/#forward_33","text":"def forward(x) Arguments : x:","title":"forward"},{"location":"development/devel/api/#hannahmodelsvad","text":"","title":"hannah.models.vad"},{"location":"development/devel/api/#hannahmodelsofasubmoduleselasticlinear","text":"","title":"hannah.models.ofa.submodules.elasticLinear"},{"location":"development/devel/api/#elasticwidthlinear-objects","text":"class ElasticWidthLinear(nn.Linear, _Elastic)","title":"ElasticWidthLinear Objects"},{"location":"development/devel/api/#forward_34","text":"def forward(input: torch.Tensor) -> torch.Tensor Arguments : input - torch.Tensor: input - torch.Tensor: input - torch.Tensor: input - torch.Tensor:","title":"forward"},{"location":"development/devel/api/#get_basic_module","text":"def get_basic_module()","title":"get_basic_module"},{"location":"development/devel/api/#elasticquantwidthlinear-objects","text":"class ElasticQuantWidthLinear(nn.Linear, _Elastic)","title":"ElasticQuantWidthLinear Objects"},{"location":"development/devel/api/#filtered_weight","text":"@property def filtered_weight()","title":"filtered_weight"},{"location":"development/devel/api/#filtered_bias","text":"@property def filtered_bias()","title":"filtered_bias"},{"location":"development/devel/api/#scaled_weight","text":"@property def scaled_weight()","title":"scaled_weight"},{"location":"development/devel/api/#scaled_bias","text":"@property def scaled_bias()","title":"scaled_bias"},{"location":"development/devel/api/#forward_35","text":"def forward(input: torch.Tensor) -> torch.Tensor Arguments : input - torch.Tensor: input - torch.Tensor: input - torch.Tensor: input - torch.Tensor:","title":"forward"},{"location":"development/devel/api/#get_basic_module_1","text":"def get_basic_module()","title":"get_basic_module"},{"location":"development/devel/api/#from_float","text":"@classmethod def from_float(cls, mod) Create a qat module from a float module or qparams_dict Args: mod a float module, either produced by torch.quantization utilities or directly from user Arguments : mod:","title":"from_float"},{"location":"development/devel/api/#elasticpermissiverelu-objects","text":"class ElasticPermissiveReLU(nn.ReLU)","title":"ElasticPermissiveReLU Objects"},{"location":"development/devel/api/#forward_36","text":"def forward(x) Arguments : x:","title":"forward"},{"location":"development/devel/api/#assemble_basic_module","text":"def assemble_basic_module()","title":"assemble_basic_module"},{"location":"development/devel/api/#hannahmodelsofasubmoduleselasticbase","text":"","title":"hannah.models.ofa.submodules.elasticBase"},{"location":"development/devel/api/#_elastic-objects","text":"class _Elastic()","title":"_Elastic Objects"},{"location":"development/devel/api/#get_basic_module_2","text":"def get_basic_module() -> nn.Module","title":"get_basic_module"},{"location":"development/devel/api/#get_out_channel_sizes","text":"def get_out_channel_sizes()","title":"get_out_channel_sizes"},{"location":"development/devel/api/#set_out_channel_filter","text":"def set_out_channel_filter(out_channel_filter) Arguments : ) -> nn.Module: # Module: so that Sequentials are possible: like DSCreturn copy.deepcopy(self.get_basic_module(): out_channel_filter: ) -> nn.Module: # Module: like DSCreturn copy.deepcopy(self.get_basic_module(): ) -> nn.Module: # Module: like DSCreturn copy.deepcopy(self.get_basic_module(): ) -> nn.Module: # Module: like DSCreturn copy.deepcopy(self.get_basic_module())set_out_channel_filter(self:","title":"set_out_channel_filter"},{"location":"development/devel/api/#set_in_channel_filter","text":"def set_in_channel_filter(in_channel_filter) Arguments : in_channel_filter:","title":"set_in_channel_filter"},{"location":"development/devel/api/#elasticbase1d-objects","text":"class ElasticBase1d(nn.Conv1d, _Elastic)","title":"ElasticBase1d Objects"},{"location":"development/devel/api/#set_bn_parameter","text":"def set_bn_parameter(conv: nn.Conv1d, tmp_bn, num_tracked) Caller for BatchNorm Parameters This unifies the call in the different methods, especially in dsc / not dsc forward And assigns the attributes in tmp_bn to the param conv Arguments : conv - nn.Conv1d: tmp_bn: num_tracked: conv - nn.Conv1d: conv - nn.Conv1d: conv - nn.Conv1d:","title":"set_bn_parameter"},{"location":"development/devel/api/#prepare_dsc_for_validation_model","text":"def prepare_dsc_for_validation_model(conv_class: nn.Module, full_kernel, full_bias, in_channels, out_channels, grouping, stride, padding, dilation, qconfig=None, out_quant=None, bn_eps=None, bn_momentum=None, bn_caller: tuple = None) This method creates the necessary validation models for DSC. It creates the validation model as torch.Sequence of standard pytorch convolution models. The structure is analog to the DSC method do_dsc. This method can also handle quantization models. Arguments : conv_class - nn.Module: full_kernel: full_bias: in_channels: out_channels: grouping: stride: padding: dilation: # for quantqconfig: (Default value = None) out_quant - (Default value = None) bn_eps - (Default value = None) bn_momentum - (Default value = None) bn_caller - tuple: (Default value = None) conv_class - nn.Module: bn_caller - tuple: (Default value = None) conv_class - nn.Module: bn_caller - tuple: (Default value = None) conv_class - nn.Module: bn_caller - tuple: (Default value = None)","title":"prepare_dsc_for_validation_model"},{"location":"development/devel/api/#do_dsc","text":"def do_dsc(input, full_kernel, full_bias, grouping, stride, padding, dilation, quant_weight=None, quant_bias=None, quant_weight_function: Function = None, quant_bias_function: Function = None) This method will perform the DSC(=Depthwise Separable Convolution). This method can also handle quantized models. DSC is done in two steps: 1. Depthwise Separable: Set Group = In_Channels, Output = k*In_Channels 2. Pointwise Convolution, with Grouping = Grouping-Param und Out_Channel = Out_Channel-Param The Params above are used for quantized models Arguments : input: full_kernel: full_bias: grouping: stride: padding: dilation: - quant_weight - (Default value = None) - quant_bias - (Default value = None) - quant_weight_function - Function: (Default value = None) # for quantizationquant_bias_function: Function: (Default value = None) # for quantization: - quant_weight_function - Function: (Default value = None) # for quantizationquant_bias_function: Function: (Default value = None) - quant_weight_function - Function: (Default value = None) # for quantizationquant_bias_function: Function: (Default value = None) - quant_weight_function - Function: (Default value = None) # for quantizationquant_bias_function: Function: (Default value = None)","title":"do_dsc"},{"location":"development/devel/api/#set_in_and_out_channel","text":"def set_in_and_out_channel(kernel, filtered: bool = True) This method uses the kernel for setting the input and outputchannel if dynamic width is activated (channelfilters), the amount of channels is reduced, hence we can't use the initial values (self.(in/out)_channel) of the constructor This method sets the self.(in/out)_channel value to the right amount of channels extracted from the kernel that will be used. if filtered is False, the self.initial_(in/out)_channels will be used. The previous values will be stored in the attribute prev_in_channels and prev_out_channels. Arguments : kernel: - filtered - bool: (Default value = True) - filtered - bool: (Default value = True) - filtered - bool: (Default value = True) - filtered - bool: (Default value = True)","title":"set_in_and_out_channel"},{"location":"development/devel/api/#reset_in_and_out_channel_to_previous","text":"def reset_in_and_out_channel_to_previous() Analog to set_in_and_out_channels: Resets the in and out_channels Arguments :","title":"reset_in_and_out_channel_to_previous"},{"location":"development/devel/api/#set_kernel_size","text":"def set_kernel_size(new_kernel_size) If the requested kernel size is outside of the min/max range, clamp it to the min/max range. If the requested kernel size is not an available kernel size, default to the max kernel size Arguments : new_kernel_size - int): the size of the kernel you want to use","title":"set_kernel_size"},{"location":"development/devel/api/#reset_kernel_size","text":"def reset_kernel_size()","title":"reset_kernel_size"},{"location":"development/devel/api/#step_down_kernel_size","text":"def step_down_kernel_size()","title":"step_down_kernel_size"},{"location":"development/devel/api/#pick_kernel_index","text":"def pick_kernel_index(target_kernel_index: int) Arguments : target_kernel_index - int: target_kernel_index - int: target_kernel_index - int: target_kernel_index - int:","title":"pick_kernel_index"},{"location":"development/devel/api/#get_available_kernel_steps","text":"def get_available_kernel_steps()","title":"get_available_kernel_steps"},{"location":"development/devel/api/#get_full_width_kernel","text":"def get_full_width_kernel() It applies the kernel transformations to the kernel until the target kernel index is reached Returns : The found target kernel. Args: Returns:","title":"get_full_width_kernel"},{"location":"development/devel/api/#get_kernel","text":"def get_kernel() If the input and output channels are not filtered, the full kernel is Arguments : Returns : return: The new kernel and bias.","title":"get_kernel"},{"location":"development/devel/api/#set_dilation_size","text":"def set_dilation_size(new_dilation_size) Arguments : new_dilation_size:","title":"set_dilation_size"},{"location":"development/devel/api/#update_padding","text":"def update_padding()","title":"update_padding"},{"location":"development/devel/api/#reset_dilation_size","text":"def reset_dilation_size()","title":"reset_dilation_size"},{"location":"development/devel/api/#step_down_dilation_size","text":"def step_down_dilation_size()","title":"step_down_dilation_size"},{"location":"development/devel/api/#pick_dilation_index","text":"def pick_dilation_index(target_dilation_index: int) Arguments : target_dilation_index - int: target_dilation_index - int: target_dilation_index - int: target_dilation_index - int:","title":"pick_dilation_index"},{"location":"development/devel/api/#get_available_dilation_steps","text":"def get_available_dilation_steps()","title":"get_available_dilation_steps"},{"location":"development/devel/api/#get_available_grouping_steps","text":"def get_available_grouping_steps()","title":"get_available_grouping_steps"},{"location":"development/devel/api/#get_available_dsc_steps","text":"def get_available_dsc_steps()","title":"get_available_dsc_steps"},{"location":"development/devel/api/#get_dilation_size","text":"def get_dilation_size()","title":"get_dilation_size"},{"location":"development/devel/api/#pick_group_index","text":"def pick_group_index(target_group_index: int) Arguments : target_group_index - int: target_group_index - int: target_group_index - int: target_group_index - int:","title":"pick_group_index"},{"location":"development/devel/api/#pick_dsc_index","text":"def pick_dsc_index(target_dsc_index: int) Arguments : target_dsc_index - int: target_dsc_index - int: target_dsc_index - int: target_dsc_index - int:","title":"pick_dsc_index"},{"location":"development/devel/api/#reset_group_size","text":"def reset_group_size()","title":"reset_group_size"},{"location":"development/devel/api/#reset_dscs","text":"def reset_dscs()","title":"reset_dscs"},{"location":"development/devel/api/#get_group_size","text":"def get_group_size()","title":"get_group_size"},{"location":"development/devel/api/#get_dsc","text":"def get_dsc()","title":"get_dsc"},{"location":"development/devel/api/#set_group_size","text":"def set_group_size(new_group_size) Arguments : new_group_size:","title":"set_group_size"},{"location":"development/devel/api/#set_dsc","text":"def set_dsc(new_dsc) Arguments : new_dsc:","title":"set_dsc"},{"location":"development/devel/api/#step_down_group_size","text":"def step_down_group_size()","title":"step_down_group_size"},{"location":"development/devel/api/#step_down_dsc","text":"def step_down_dsc()","title":"step_down_dsc"},{"location":"development/devel/api/#adjust_weights_for_grouping","text":"def adjust_weights_for_grouping(weights, input_divided_by) Arguments : weights: input_divided_by:","title":"adjust_weights_for_grouping"},{"location":"development/devel/api/#forward_37","text":"def forward(input: torch.Tensor) -> torch.Tensor Arguments : input - torch.Tensor: input - torch.Tensor: input - torch.Tensor: input - torch.Tensor:","title":"forward"},{"location":"development/devel/api/#extra_repr","text":"def extra_repr()","title":"extra_repr"},{"location":"development/devel/api/#hannahmodelsofasubmodulesresblock","text":"","title":"hannah.models.ofa.submodules.resblock"},{"location":"development/devel/api/#resblockbase-objects","text":"class ResBlockBase(nn.Module)","title":"ResBlockBase Objects"},{"location":"development/devel/api/#forward_38","text":"def forward(x) Arguments : x:","title":"forward"},{"location":"development/devel/api/#get_nested_modules","text":"def get_nested_modules()","title":"get_nested_modules"},{"location":"development/devel/api/#resblock1d-objects","text":"class ResBlock1d(ResBlockBase)","title":"ResBlock1d Objects"},{"location":"development/devel/api/#forward_39","text":"def forward(x) Arguments : x:","title":"forward"},{"location":"development/devel/api/#get_input_layer","text":"def get_input_layer()","title":"get_input_layer"},{"location":"development/devel/api/#get_output_layer","text":"def get_output_layer()","title":"get_output_layer"},{"location":"development/devel/api/#create_internal_channelhelper","text":"def create_internal_channelhelper()","title":"create_internal_channelhelper"},{"location":"development/devel/api/#hannahmodelsofasubmodules","text":"","title":"hannah.models.ofa.submodules"},{"location":"development/devel/api/#hannahmodelsofasubmoduleselasticbatchnorm","text":"","title":"hannah.models.ofa.submodules.elasticBatchnorm"},{"location":"development/devel/api/#elasticwidthbatchnorm1d-objects","text":"class ElasticWidthBatchnorm1d(nn.BatchNorm1d)","title":"ElasticWidthBatchnorm1d Objects"},{"location":"development/devel/api/#forward_40","text":"def forward(input: torch.Tensor) -> torch.Tensor if self.track_running_stats: logging.warn( \"ElasticWidthBatchnorm with tracked running stats currently not fully implemented!\" )","title":"forward"},{"location":"development/devel/api/#num_batches_tracked-and-exponential-averaging-are-currently-not-implemented","text":"Arguments : input - torch.Tensor: input - torch.Tensor: input - torch.Tensor: input - torch.Tensor:","title":"num_batches_tracked and exponential averaging are currently not implemented."},{"location":"development/devel/api/#get_basic_batchnorm1d","text":"def get_basic_batchnorm1d()","title":"get_basic_batchnorm1d"},{"location":"development/devel/api/#assemble_basic_module_1","text":"def assemble_basic_module() -> nn.BatchNorm1d","title":"assemble_basic_module"},{"location":"development/devel/api/#hannahmodelsofasubmoduleselasticchannelhelper","text":"","title":"hannah.models.ofa.submodules.elasticchannelhelper"},{"location":"development/devel/api/#elasticchannelhelper-objects","text":"class ElasticChannelHelper(nn.Module)","title":"ElasticChannelHelper Objects"},{"location":"development/devel/api/#compute_channel_priorities","text":"def compute_channel_priorities()","title":"compute_channel_priorities"},{"location":"development/devel/api/#set_channel_filter","text":"def set_channel_filter()","title":"set_channel_filter"},{"location":"development/devel/api/#apply_filter_to_module","text":"def apply_filter_to_module(module, is_target: bool) Arguments : module: - is_target - bool: - is_target - bool: - is_target - bool: - is_target - bool:","title":"apply_filter_to_module"},{"location":"development/devel/api/#step_down_channels","text":"def step_down_channels()","title":"step_down_channels"},{"location":"development/devel/api/#set_channel_step","text":"def set_channel_step(step: int) Arguments : step - int: step - int: step - int: step - int:","title":"set_channel_step"},{"location":"development/devel/api/#reset_channel_step","text":"def reset_channel_step()","title":"reset_channel_step"},{"location":"development/devel/api/#set_primary_target","text":"def set_primary_target(target: nn.Module) Arguments : target - nn.Module: target - nn.Module: target - nn.Module: target - nn.Module:","title":"set_primary_target"},{"location":"development/devel/api/#is_valid_primary_target","text":"def is_valid_primary_target(module: nn.Module) -> bool Arguments : module - nn.Module: module - nn.Module: module - nn.Module: module - nn.Module:","title":"is_valid_primary_target"},{"location":"development/devel/api/#is_primary_target","text":"def is_primary_target(module: nn.Module) -> bool Arguments : module - nn.Module: module - nn.Module: module - nn.Module: module - nn.Module:","title":"is_primary_target"},{"location":"development/devel/api/#add_secondary_targets","text":"def add_secondary_targets(target: nn.Module) Arguments : target - nn.Module: target - nn.Module: target - nn.Module: target - nn.Module:","title":"add_secondary_targets"},{"location":"development/devel/api/#add_secondary_target_item","text":"def add_secondary_target_item(target: nn.Module) Arguments : target - nn.Module: target - nn.Module: target - nn.Module: target - nn.Module:","title":"add_secondary_target_item"},{"location":"development/devel/api/#add_sources","text":"def add_sources(source: nn.Module) Arguments : source - nn.Module: source - nn.Module: source - nn.Module: source - nn.Module:","title":"add_sources"},{"location":"development/devel/api/#add_targets","text":"def add_targets(target: nn.Module) Arguments : target - nn.Module: target - nn.Module: target - nn.Module: target - nn.Module:","title":"add_targets"},{"location":"development/devel/api/#add_source_item","text":"def add_source_item(source: nn.Module) Arguments : source - nn.Module: source - nn.Module: source - nn.Module: source - nn.Module:","title":"add_source_item"},{"location":"development/devel/api/#discover_target","text":"def discover_target(new_target: nn.Module) Arguments : new_target - nn.Module: new_target - nn.Module: new_target - nn.Module: new_target - nn.Module:","title":"discover_target"},{"location":"development/devel/api/#get_available_width_steps","text":"def get_available_width_steps()","title":"get_available_width_steps"},{"location":"development/devel/api/#hannahmodelsofasubmoduleselastickernelconv","text":"","title":"hannah.models.ofa.submodules.elastickernelconv"},{"location":"development/devel/api/#elasticconv1d-objects","text":"class ElasticConv1d(ElasticBase1d)","title":"ElasticConv1d Objects"},{"location":"development/devel/api/#forward_41","text":"def forward(input: torch.Tensor) -> torch.Tensor Arguments : input - torch.Tensor: input - torch.Tensor: input - torch.Tensor: input - torch.Tensor:","title":"forward"},{"location":"development/devel/api/#get_basic_module_3","text":"def get_basic_module() -> nn.Module","title":"get_basic_module"},{"location":"development/devel/api/#elasticconvrelu1d-objects","text":"class ElasticConvReLu1d(ElasticBase1d)","title":"ElasticConvReLu1d Objects"},{"location":"development/devel/api/#forward_42","text":"def forward(input: torch.Tensor) -> torch.Tensor Arguments : input - torch.Tensor: input - torch.Tensor: input - torch.Tensor: input - torch.Tensor:","title":"forward"},{"location":"development/devel/api/#get_basic_module_4","text":"def get_basic_module() -> nn.Module","title":"get_basic_module"},{"location":"development/devel/api/#elasticconvbn1d-objects","text":"class ElasticConvBn1d(ElasticConv1d)","title":"ElasticConvBn1d Objects"},{"location":"development/devel/api/#forward_43","text":"def forward(input: torch.Tensor) -> torch.Tensor Arguments : input - torch.Tensor: input - torch.Tensor: input - torch.Tensor: input - torch.Tensor:","title":"forward"},{"location":"development/devel/api/#get_basic_module_5","text":"def get_basic_module() -> nn.Module","title":"get_basic_module"},{"location":"development/devel/api/#elasticconvbnrelu1d-objects","text":"class ElasticConvBnReLu1d(ElasticConvBn1d)","title":"ElasticConvBnReLu1d Objects"},{"location":"development/devel/api/#forward_44","text":"def forward(input: torch.Tensor) -> torch.Tensor Arguments : input - torch.Tensor: input - torch.Tensor: input - torch.Tensor: input - torch.Tensor:","title":"forward"},{"location":"development/devel/api/#get_basic_module_6","text":"def get_basic_module() -> nn.Module","title":"get_basic_module"},{"location":"development/devel/api/#convrelu1d-objects","text":"class ConvRelu1d(nn.Conv1d)","title":"ConvRelu1d Objects"},{"location":"development/devel/api/#forward_45","text":"def forward(input: torch.Tensor) -> torch.Tensor Arguments : input - torch.Tensor: input - torch.Tensor: input - torch.Tensor: input - torch.Tensor:","title":"forward"},{"location":"development/devel/api/#convbn1d-objects","text":"class ConvBn1d(nn.Conv1d)","title":"ConvBn1d Objects"},{"location":"development/devel/api/#forward_46","text":"def forward(input: torch.Tensor) -> torch.Tensor Arguments : input - torch.Tensor: input - torch.Tensor: input - torch.Tensor: input - torch.Tensor:","title":"forward"},{"location":"development/devel/api/#convbnrelu1d-objects","text":"class ConvBnReLu1d(ConvBn1d)","title":"ConvBnReLu1d Objects"},{"location":"development/devel/api/#forward_47","text":"def forward(input: torch.Tensor) -> torch.Tensor Arguments : input - torch.Tensor: input - torch.Tensor: input - torch.Tensor: input - torch.Tensor:","title":"forward"},{"location":"development/devel/api/#hannahmodelsofasubmoduleselasticquantkernelconv","text":"","title":"hannah.models.ofa.submodules.elasticquantkernelconv"},{"location":"development/devel/api/#quaddatahelper-objects","text":"class QuadDataHelper() Data Container so that _forward and _dsc has the same data.","title":"QuadDataHelper Objects"},{"location":"development/devel/api/#_elasticconvbnnd-objects","text":"class _ElasticConvBnNd(ElasticBase1d, qat._ConvForwardMixin)","title":"_ElasticConvBnNd Objects"},{"location":"development/devel/api/#on_warmup_end","text":"def on_warmup_end()","title":"on_warmup_end"},{"location":"development/devel/api/#reset_running_stats","text":"def reset_running_stats()","title":"reset_running_stats"},{"location":"development/devel/api/#reset_bn_parameters","text":"def reset_bn_parameters()","title":"reset_bn_parameters"},{"location":"development/devel/api/#reset_parameters","text":"def reset_parameters()","title":"reset_parameters"},{"location":"development/devel/api/#update_bn_stats","text":"def update_bn_stats()","title":"update_bn_stats"},{"location":"development/devel/api/#freeze_bn_stats","text":"def freeze_bn_stats()","title":"freeze_bn_stats"},{"location":"development/devel/api/#scale_factor","text":"@property def scale_factor()","title":"scale_factor"},{"location":"development/devel/api/#full_scale_factor","text":"@property def full_scale_factor() does the same as scale_factor but uses the whole kernel. Used for dsc","title":"full_scale_factor"},{"location":"development/devel/api/#scaled_weight_1","text":"@property def scaled_weight()","title":"scaled_weight"},{"location":"development/devel/api/#get_full_kernel_bias","text":"def get_full_kernel_bias() Gets the full kernel and bias. Used for dsc","title":"get_full_kernel_bias"},{"location":"development/devel/api/#extra_repr_1","text":"def extra_repr()","title":"extra_repr"},{"location":"development/devel/api/#forward_48","text":"def forward(input) Arguments : input:","title":"forward"},{"location":"development/devel/api/#train_1","text":"def train(mode=True) Batchnorm's training behavior is using the self.training flag. Prevent changing it if BN is frozen. This makes sure that calling model.train() on a model with a frozen BN will behave properly. Arguments : mode - (Default value = True)","title":"train"},{"location":"development/devel/api/#from_float_1","text":"@classmethod def from_float(cls, mod) Create a qat module from a float module or qparams_dict Args: mod a float module, either produced by torch.quantization utilities or directly from user Arguments : mod:","title":"from_float"},{"location":"development/devel/api/#elasticquantconv1d-objects","text":"class ElasticQuantConv1d(ElasticBase1d, qat._ConvForwardMixin)","title":"ElasticQuantConv1d Objects"},{"location":"development/devel/api/#forward_49","text":"def forward(input: torch.Tensor) -> torch.Tensor Arguments : input - torch.Tensor: input - torch.Tensor: input - torch.Tensor: input - torch.Tensor:","title":"forward"},{"location":"development/devel/api/#get_basic_module_7","text":"def get_basic_module() -> nn.Module","title":"get_basic_module"},{"location":"development/devel/api/#elasticquantconvrelu1d-objects","text":"class ElasticQuantConvReLu1d(ElasticBase1d, qat._ConvForwardMixin)","title":"ElasticQuantConvReLu1d Objects"},{"location":"development/devel/api/#forward_50","text":"def forward(input: torch.Tensor) -> torch.Tensor Arguments : input - torch.Tensor: input - torch.Tensor: input - torch.Tensor: input - torch.Tensor:","title":"forward"},{"location":"development/devel/api/#get_basic_module_8","text":"def get_basic_module() -> nn.Module","title":"get_basic_module"},{"location":"development/devel/api/#elasticquantconvbn1d-objects","text":"class ElasticQuantConvBn1d(_ElasticConvBnNd)","title":"ElasticQuantConvBn1d Objects"},{"location":"development/devel/api/#forward_51","text":"def forward(input: torch.Tensor) -> torch.Tensor Arguments : input - torch.Tensor: input - torch.Tensor: input - torch.Tensor: input - torch.Tensor:","title":"forward"},{"location":"development/devel/api/#get_basic_module_9","text":"def get_basic_module() -> nn.Module","title":"get_basic_module"},{"location":"development/devel/api/#elasticquantconvbnrelu1d-objects","text":"class ElasticQuantConvBnReLu1d(ElasticQuantConvBn1d)","title":"ElasticQuantConvBnReLu1d Objects"},{"location":"development/devel/api/#forward_52","text":"def forward(input: torch.Tensor) -> torch.Tensor Arguments : input - torch.Tensor: input - torch.Tensor: input - torch.Tensor: input - torch.Tensor:","title":"forward"},{"location":"development/devel/api/#get_basic_module_10","text":"def get_basic_module() -> nn.Module","title":"get_basic_module"},{"location":"development/devel/api/#hannahmodelsofamodels","text":"","title":"hannah.models.ofa.models"},{"location":"development/devel/api/#create","text":"def create(name: str, labels: int, input_shape, conv=[], min_depth: int = 1, norm_before_act=True, skew_sampling_distribution: bool = False, dropout: int = 0.5, validate_on_extracted=True, qconfig=None) -> nn.Module The function creates a ofa Model with the given name, labels, input shape, convolutional layers, and other parameters Arguments : name(str) - The name of the model labels(int) - The number of classes in the dataset input_shape - the shape of the input tensor conv - a list of MajorBlockConfig objects (Default value = []) min_depth(int optional - The minimum depth of the model, defaults to 1 norm_before_act - If True, the normalization is performed before the activation function, defaults to True (optional) skew_sampling_distribution(bool optional - If True, the model will use a skewed sampling distribution to sample the number of minor blocks in each major block, defaults to False dropout(int) - float, default 0.5 validate_on_extracted - If True, the model will be validated on the extracted data, defaults to True (optional) qconfig - the quantization configuration to use (Default value = None) name - str: labels - int: min_depth - int: (Default value = 1) skew_sampling_distribution - bool: (Default value = False) dropout - int: (Default value = 0.5) name - str: labels - int: min_depth - int: (Default value = 1) skew_sampling_distribution - bool: (Default value = False) dropout - int: (Default value = 0.5) name - str: labels - int: min_depth - int: (Default value = 1) skew_sampling_distribution - bool: (Default value = False) dropout - int: (Default value = 0.5) Returns : A model object.","title":"create"},{"location":"development/devel/api/#create_minor_block_sequence","text":"def create_minor_block_sequence(blocks, in_channels, stride=1, norm_before_act=True, qconfig=None) Arguments : blocks: in_channels: - stride - (Default value = 1) - norm_before_act - (Default value = True) - qconfig - (Default value = None) # sources: List[nn.Module]: (Default value = [nn.ModuleList([])]) # sources: List[nn.Module]: (Default value = [nn.ModuleList([])]) # sources: List[nn.Module]: (Default value = [nn.ModuleList([])])","title":"create_minor_block_sequence"},{"location":"development/devel/api/#create_minor_block","text":"def create_minor_block(block_config, in_channels: int, stride: int = 1, norm_before_act=True, sources: List[nn.ModuleList] = [nn.ModuleList([])], qconfig=None) -> Tuple[nn.Module, int] Arguments : block_config: - in_channels - int: - stride - int: (Default value = 1) - norm_before_act - (Default value = True) - sources - List[nn.ModuleList]: (Default value = [nn.ModuleList([])]) - qconfig - (Default value = None) - in_channels - int: - stride - int: (Default value = 1) - sources - List[nn.ModuleList]: (Default value = [nn.ModuleList([])]) - in_channels - int: - stride - int: (Default value = 1) - sources - List[nn.ModuleList]: (Default value = [nn.ModuleList([])])","title":"create_minor_block"},{"location":"development/devel/api/#create_residual_block_1d","text":"def create_residual_block_1d(blocks, in_channels, stride=1, norm_before_act=None, qconfig=None, quant_skip=None) -> ResBlock1d Arguments : blocks: in_channels: - stride - (Default value = 1) - norm_before_act - (Default value = None) - qconfig - (Default value = None) - quant_skip - (Default value = None) # sources: List[nn.ModuleList]: (Default value = [nn.ModuleList([])]) # sources: List[nn.ModuleList]: (Default value = [nn.ModuleList([])]) # sources: List[nn.ModuleList]: (Default value = [nn.ModuleList([])])","title":"create_residual_block_1d"},{"location":"development/devel/api/#ofamodel-objects","text":"class OFAModel(nn.Module)","title":"OFAModel Objects"},{"location":"development/devel/api/#extract_conv","text":"def extract_conv(conv, general_config, parallel=True, bp=False) Arguments : conv: general_config: - parallel - (Default value = True) - bp - (Default value = False)","title":"extract_conv"},{"location":"development/devel/api/#extract_config","text":"def extract_config(conv, general_config, bp=False) set all conv attributes in config Arguments : conv: general_config: - bp - (Default value = False)","title":"extract_config"},{"location":"development/devel/api/#print_config","text":"def print_config(filename) Arguments : filename:","title":"print_config"},{"location":"development/devel/api/#forward_53","text":"def forward(x) Arguments : x:","title":"forward"},{"location":"development/devel/api/#perform_sequence_discovery","text":"def perform_sequence_discovery()","title":"perform_sequence_discovery"},{"location":"development/devel/api/#get_post_conv","text":"def get_post_conv(post_block) Arguments : post_block:","title":"get_post_conv"},{"location":"development/devel/api/#get_pre_conv","text":"def get_pre_conv(pre_block) Arguments : pre_block:","title":"get_pre_conv"},{"location":"development/devel/api/#sample_subnetwork","text":"def sample_subnetwork()","title":"sample_subnetwork"},{"location":"development/devel/api/#get_random_step","text":"def get_random_step(upper_bound: int) -> int Arguments : upper_bound - int: upper_bound - int: upper_bound - int:","title":"get_random_step"},{"location":"development/devel/api/#get_max_submodel_steps","text":"def get_max_submodel_steps()","title":"get_max_submodel_steps"},{"location":"development/devel/api/#get_submodel","text":"def get_submodel(state: dict) Arguments : state - dict: state - dict: state - dict:","title":"get_submodel"},{"location":"development/devel/api/#on_warmup_end_1","text":"def on_warmup_end()","title":"on_warmup_end"},{"location":"development/devel/api/#set_submodel","text":"def set_submodel(state: dict) Arguments : state - dict: state - dict: state - dict:","title":"set_submodel"},{"location":"development/devel/api/#build_validation_model","text":"def build_validation_model()","title":"build_validation_model"},{"location":"development/devel/api/#reset_validation_model","text":"def reset_validation_model()","title":"reset_validation_model"},{"location":"development/devel/api/#get_validation_model_weight_count","text":"def get_validation_model_weight_count()","title":"get_validation_model_weight_count"},{"location":"development/devel/api/#extract_elastic_depth_sequence","text":"def extract_elastic_depth_sequence(target_depth, quantized=False, clone_mode=False) Arguments : target_depth: - quantized - (Default value = False) - clone_mode - (Default value = False)","title":"extract_elastic_depth_sequence"},{"location":"development/devel/api/#extract_module_from_depth_step","text":"def extract_module_from_depth_step(depth_step) -> nn.Module Arguments : depth_step:","title":"extract_module_from_depth_step"},{"location":"development/devel/api/#get_elastic_depth_output","text":"def get_elastic_depth_output(target_depth=None, quantized=False) Arguments : target_depth - (Default value = None) quantized - (Default value = False)","title":"get_elastic_depth_output"},{"location":"development/devel/api/#step_down_all_channels","text":"def step_down_all_channels()","title":"step_down_all_channels"},{"location":"development/devel/api/#reset_active_depth","text":"def reset_active_depth()","title":"reset_active_depth"},{"location":"development/devel/api/#resume_active_elastic_values","text":"def resume_active_elastic_values()","title":"resume_active_elastic_values"},{"location":"development/devel/api/#update_output_channel_count","text":"def update_output_channel_count()","title":"update_output_channel_count"},{"location":"development/devel/api/#get_output_linear_layer","text":"def get_output_linear_layer(target_depth) Arguments : target_depth:","title":"get_output_linear_layer"},{"location":"development/devel/api/#step_down_all_kernels","text":"def step_down_all_kernels()","title":"step_down_all_kernels"},{"location":"development/devel/api/#reset_all_kernel_sizes","text":"def reset_all_kernel_sizes()","title":"reset_all_kernel_sizes"},{"location":"development/devel/api/#reset_all_dilation_sizes","text":"def reset_all_dilation_sizes()","title":"reset_all_dilation_sizes"},{"location":"development/devel/api/#reset_all_group_sizes","text":"def reset_all_group_sizes()","title":"reset_all_group_sizes"},{"location":"development/devel/api/#reset_all_dsc","text":"def reset_all_dsc()","title":"reset_all_dsc"},{"location":"development/devel/api/#step_down_all_dilations","text":"def step_down_all_dilations()","title":"step_down_all_dilations"},{"location":"development/devel/api/#step_down_all_groups","text":"def step_down_all_groups()","title":"step_down_all_groups"},{"location":"development/devel/api/#step_down_all_dsc","text":"def step_down_all_dsc()","title":"step_down_all_dsc"},{"location":"development/devel/api/#go_to_kernel_step","text":"def go_to_kernel_step(step: int) Arguments : step - int: step - int: step - int:","title":"go_to_kernel_step"},{"location":"development/devel/api/#resume_kernel_sizes_from_step","text":"def resume_kernel_sizes_from_step()","title":"resume_kernel_sizes_from_step"},{"location":"development/devel/api/#reset_all_widths","text":"def reset_all_widths()","title":"reset_all_widths"},{"location":"development/devel/api/#progressive_shrinking_add_kernel","text":"def progressive_shrinking_add_kernel()","title":"progressive_shrinking_add_kernel"},{"location":"development/devel/api/#progressive_shrinking_add_dilation","text":"def progressive_shrinking_add_dilation()","title":"progressive_shrinking_add_dilation"},{"location":"development/devel/api/#progressive_shrinking_add_depth","text":"def progressive_shrinking_add_depth()","title":"progressive_shrinking_add_depth"},{"location":"development/devel/api/#progressive_shrinking_add_group","text":"def progressive_shrinking_add_group()","title":"progressive_shrinking_add_group"},{"location":"development/devel/api/#progressive_shrinking_add_dsc","text":"def progressive_shrinking_add_dsc()","title":"progressive_shrinking_add_dsc"},{"location":"development/devel/api/#progressive_shrinking_compute_channel_priorities","text":"def progressive_shrinking_compute_channel_priorities()","title":"progressive_shrinking_compute_channel_priorities"},{"location":"development/devel/api/#progressive_shrinking_add_width","text":"def progressive_shrinking_add_width()","title":"progressive_shrinking_add_width"},{"location":"development/devel/api/#progressive_shrinking_disable_sampling","text":"def progressive_shrinking_disable_sampling()","title":"progressive_shrinking_disable_sampling"},{"location":"development/devel/api/#reset_shrinking","text":"def reset_shrinking()","title":"reset_shrinking"},{"location":"development/devel/api/#rebuild_extracted_blocks","text":"def rebuild_extracted_blocks(blocks) Arguments : blocks:","title":"rebuild_extracted_blocks"},{"location":"development/devel/api/#hannahmodelsofa","text":"","title":"hannah.models.ofa"},{"location":"development/devel/api/#hannahmodelsofatype_utils","text":"","title":"hannah.models.ofa.type_utils"},{"location":"development/devel/api/#hannahmodelsofautilities","text":"","title":"hannah.models.ofa.utilities"},{"location":"development/devel/api/#conv1d_auto_padding","text":"def conv1d_auto_padding(conv1d: nn.Conv1d) Arguments : conv1d - nn.Conv1d: conv1d - nn.Conv1d: conv1d - nn.Conv1d: conv1d - nn.Conv1d:","title":"conv1d_auto_padding"},{"location":"development/devel/api/#conv1d_get_padding","text":"def conv1d_get_padding(kernel_size, dilation=1) Arguments : kernel_size: - dilation - (Default value = 1)","title":"conv1d_get_padding"},{"location":"development/devel/api/#sub_filter_start_end","text":"def sub_filter_start_end(kernel_size, sub_kernel_size) Arguments : kernel_size: sub_kernel_size:","title":"sub_filter_start_end"},{"location":"development/devel/api/#flatten_module_list","text":"def flatten_module_list(modules: nn.Module) -> nn.Module Arguments : modules - nn.Module: modules - nn.Module: modules - nn.Module: modules - nn.Module:","title":"flatten_module_list"},{"location":"development/devel/api/#module_list_to_module","text":"def module_list_to_module(module_list) Arguments : module_list:","title":"module_list_to_module"},{"location":"development/devel/api/#call_function_from_deep_nested","text":"def call_function_from_deep_nested(input, function, type_selection: type = None) Arguments : input: function: - type_selection - type: (Default value = None) - type_selection - type: (Default value = None) - type_selection - type: (Default value = None) - type_selection - type: (Default value = None)","title":"call_function_from_deep_nested"},{"location":"development/devel/api/#get_instances_from_deep_nested","text":"def get_instances_from_deep_nested(input, type_selection: type = None) Arguments : input: - type_selection - type: (Default value = None) - type_selection - type: (Default value = None) - type_selection - type: (Default value = None) - type_selection - type: (Default value = None)","title":"get_instances_from_deep_nested"},{"location":"development/devel/api/#filter_primary_module_weights","text":"def filter_primary_module_weights(weights, in_channel_filter, out_channel_filter) Arguments : weights: in_channel_filter: out_channel_filter:","title":"filter_primary_module_weights"},{"location":"development/devel/api/#filter_single_dimensional_weights","text":"def filter_single_dimensional_weights(weights, channel_filter) Arguments : weights: channel_filter:","title":"filter_single_dimensional_weights"},{"location":"development/devel/api/#make_parameter","text":"def make_parameter(t: torch.Tensor) -> nn.Parameter Arguments : t - torch.Tensor: t - torch.Tensor: t - torch.Tensor: t - torch.Tensor:","title":"make_parameter"},{"location":"development/devel/api/#adjust_weight_if_needed","text":"def adjust_weight_if_needed(module, kernel=None, groups=None) Adjust the weight if the adjustment is needded. This means, if the kernel does not have the size of (out_channel, in_channel / group, kernel). Arguments : kernel - the kernel that should be checked and adjusted if needed. If None module.weight.data will be used (Default value = None) grouping - value of the conv, if None module.groups will be used module - the conv :throws - RuntimeError if there is no last_grouping_param for comporing current group value to past group value returns (kernel, is adjusted) (adjusted if needed) otherwise throws a RuntimeError groups - (Default value = None)","title":"adjust_weight_if_needed"},{"location":"development/devel/api/#is_weight_adjusting_needed","text":"def is_weight_adjusting_needed(weights, input_channels, groups) Checks if a weight adjustment is needed Requirement: weight.shape[1] must be input_channels/groups true: weight adjustment is needed Arguments : weights - the weights that needs to be checked input_channels - Input Channels of the Convolution Module groups - Grouping Param of the Convolution Module","title":"is_weight_adjusting_needed"},{"location":"development/devel/api/#get_target_weight","text":"def get_target_weight(weights, input_channels, groups) Gives the targeted weight shape (out_channel, in_channel // groups, kernel) Arguments : weights - the weights that needs to be checked input_channels - Input Channels of the Convolution Module groups - Grouping Param of the Convolution Module","title":"get_target_weight"},{"location":"development/devel/api/#prepare_kernel_for_depthwise_separable_convolution","text":"def prepare_kernel_for_depthwise_separable_convolution(model, kernel, bias, in_channels) Prepares the kernel for depthwise separable convolution (step 1 of DSC). This means setting groups = inchannels and outchannels = k * inchannels. Arguments : model: kernel: bias: in_channels: Returns : kernel, bias) Tuple","title":"prepare_kernel_for_depthwise_separable_convolution"},{"location":"development/devel/api/#prepare_kernel_for_pointwise_convolution","text":"def prepare_kernel_for_pointwise_convolution(kernel, grouping) Prepares the kernel for pointwise convolution (step 2 of DSC). This means setting the kernel window to 1x1. So a kernel with output_channel, input_channel / groups, kernel will be set to ( , ,1) Arguments : kernel: grouping:","title":"prepare_kernel_for_pointwise_convolution"},{"location":"development/devel/api/#adjust_weights_for_grouping_1","text":"def adjust_weights_for_grouping(weights, input_divided_by) Adjusts the Weights for the Forward of the Convulution Shape(outchannels, inchannels / group, kW) weight \u2013 filters of shape (out_channels , in_channels / groups , kW) input_divided_by Arguments : weights: input_divided_by:","title":"adjust_weights_for_grouping"},{"location":"development/devel/api/#get_kernel_for_dsc","text":"def get_kernel_for_dsc(kernel) Part of DSC (Step 2, pointwise convolution) kernel with output_channel, input_channel / groups, kernel will be set to ( , ,1) Arguments : kernel:","title":"get_kernel_for_dsc"},{"location":"development/devel/api/#get_channel_filter","text":"def get_channel_filter(current_channel_size, reduced_target_channel_size, channel_priority_list) Arguments : current_channel_size: reduced_target_channel_size: channel_priority_list:","title":"get_channel_filter"},{"location":"development/devel/api/#create_channel_filter","text":"def create_channel_filter(module: nn.Module, kernel, current_channel, reduced_target_channel_size, is_output_filter: bool = True) Arguments : module - nn.Module: kernel: current_channel: reduced_target_channel_size: is_output_filter - bool: (Default value = True) module - nn.Module: is_output_filter - bool: (Default value = True) module - nn.Module: is_output_filter - bool: (Default value = True) module - nn.Module: is_output_filter - bool: (Default value = True)","title":"create_channel_filter"},{"location":"development/devel/api/#compute_channel_priorities_1","text":"def compute_channel_priorities(module: nn.Module, kernel, channel_index: int = 0) Arguments : module - nn.Module: kernel: channel_index - int: (Default value = 0) module - nn.Module: channel_index - int: (Default value = 0) module - nn.Module: channel_index - int: (Default value = 0) module - nn.Module: channel_index - int: (Default value = 0)","title":"compute_channel_priorities"},{"location":"development/devel/api/#hannahmodelsekutmodels","text":"","title":"hannah.models.ekut.models"},{"location":"development/devel/api/#conv_bn","text":"def conv_bn(inp, oup, stride) Arguments : inp: oup: stride:","title":"conv_bn"},{"location":"development/devel/api/#conv_1x1_bn","text":"def conv_1x1_bn(inp, oup) Arguments : inp: oup:","title":"conv_1x1_bn"},{"location":"development/devel/api/#invertedresidual-objects","text":"class InvertedResidual(nn.Module)","title":"InvertedResidual Objects"},{"location":"development/devel/api/#forward_54","text":"def forward(x) Arguments : x:","title":"forward"},{"location":"development/devel/api/#rawspeechmodel-objects","text":"class RawSpeechModel(nn.Module) Speech Recognition on RAW Data using Wolfgang Fuhls Networks","title":"RawSpeechModel Objects"},{"location":"development/devel/api/#forward_55","text":"def forward(x) Arguments : x:","title":"forward"},{"location":"development/devel/api/#rawspeechmodelinvertedresidual-objects","text":"class RawSpeechModelInvertedResidual(nn.Module)","title":"RawSpeechModelInvertedResidual Objects"},{"location":"development/devel/api/#forward_56","text":"def forward(x) Arguments : x:","title":"forward"},{"location":"development/devel/api/#hannahmodelsekut","text":"","title":"hannah.models.ekut"},{"location":"development/devel/api/#hannahmodelsutils","text":"","title":"hannah.models.utils"},{"location":"development/devel/api/#next_power_of2","text":"def next_power_of2(x) Arguments : x:","title":"next_power_of2"},{"location":"development/devel/api/#hannahmodelswavenetmodels","text":"","title":"hannah.models.wavenet.models"},{"location":"development/devel/api/#conv-objects","text":"class Conv(nn.Module) A convolution with the option to be causal and use xavier initialization","title":"Conv Objects"},{"location":"development/devel/api/#forward_57","text":"def forward(signal) Arguments : signal:","title":"forward"},{"location":"development/devel/api/#wavenet-objects","text":"class WaveNet(nn.Module)","title":"WaveNet Objects"},{"location":"development/devel/api/#forward_58","text":"def forward(input_data) Arguments : input_data:","title":"forward"},{"location":"development/devel/api/#hannahmodelswavenet","text":"","title":"hannah.models.wavenet"},{"location":"development/devel/api/#hannahmodels_vendor","text":"","title":"hannah.models._vendor"},{"location":"development/devel/api/#hannahmodels_vendorfocalnet","text":"","title":"hannah.models._vendor.focalnet"},{"location":"development/devel/api/#mlp-objects","text":"class Mlp(nn.Module)","title":"Mlp Objects"},{"location":"development/devel/api/#forward_59","text":"def forward(x) Arguments : x:","title":"forward"},{"location":"development/devel/api/#focalmodulation-objects","text":"class FocalModulation(nn.Module)","title":"FocalModulation Objects"},{"location":"development/devel/api/#forward_60","text":"def forward(x) Arguments : x - input features with shape of (B, H, W, C)","title":"forward"},{"location":"development/devel/api/#extra_repr_2","text":"def extra_repr() -> str","title":"extra_repr"},{"location":"development/devel/api/#flops","text":"def flops(N) calculate flops for 1 window with token length of N Arguments : N:","title":"flops"},{"location":"development/devel/api/#focalnetblock-objects","text":"class FocalNetBlock(nn.Module) Focal Modulation Network Block. Arguments : dim(int) - Number of input channels. input_resolution(tuple[int]) - Input resulotion. mlp_ratio(float) - Ratio of mlp hidden dim to embedding dim. drop(float) - Dropout rate. Default: 0.0 drop_path(float) - Stochastic depth rate. Default: 0.0 act_layer(nn.Module) - Activation layer. Default: nn.GELU norm_layer(nn.Module) - Normalization layer. Default: nn.LayerNorm focal_level(int) - Number of focal levels. focal_window(int) - Focal window size at first focal level use_layerscale(bool) - Whether use layerscale layerscale_value(float) - Initial layerscale value use_postln(bool) - Whether use layernorm after modulation","title":"FocalNetBlock Objects"},{"location":"development/devel/api/#forward_61","text":"def forward(x) Arguments : x:","title":"forward"},{"location":"development/devel/api/#extra_repr_3","text":"def extra_repr() -> str","title":"extra_repr"},{"location":"development/devel/api/#flops_1","text":"def flops()","title":"flops"},{"location":"development/devel/api/#basiclayer-objects","text":"class BasicLayer(nn.Module) A basic Focal Transformer layer for one stage. Arguments : dim(int) - Number of input channels. input_resolution(tuple[int]) - Input resolution. depth(int) - Number of blocks. window_size(int) - Local window size. mlp_ratio(float) - Ratio of mlp hidden dim to embedding dim. qkv_bias(bool) - If True, add a learnable bias to query, key, value. Default: True qk_scale(float | None): Override default qk scale of head_dim ** -0.5 if set. drop(float) - Dropout rate. Default: 0.0 drop_path(float | tuple[float]): Stochastic depth rate. Default: 0.0 norm_layer(nn.Module) - Normalization layer. Default: nn.LayerNorm downsample(nn.Module | None): Downsample layer at the end of the layer. Default: None use_checkpoint(bool) - Whether to use checkpointing to save memory. Default: False. focal_level(int) - Number of focal levels focal_window(int) - Focal window size at first focal level use_layerscale(bool) - Whether use layerscale layerscale_value(float) - Initial layerscale value use_postln(bool) - Whether use layernorm after modulation","title":"BasicLayer Objects"},{"location":"development/devel/api/#forward_62","text":"def forward(x, H, W) Arguments : x: H: W:","title":"forward"},{"location":"development/devel/api/#extra_repr_4","text":"def extra_repr() -> str","title":"extra_repr"},{"location":"development/devel/api/#flops_2","text":"def flops()","title":"flops"},{"location":"development/devel/api/#patchembed-objects","text":"class PatchEmbed(nn.Module) Image to Patch Embedding Arguments : img_size(int) - Image size. Default: 224. patch_size(int) - Patch token size. Default: 4. in_chans(int) - Number of input image channels. Default: 3. embed_dim(int) - Number of linear projection output channels. Default: 96. norm_layer(nn.Module) - Normalization layer. Default: None","title":"PatchEmbed Objects"},{"location":"development/devel/api/#forward_63","text":"def forward(x) Arguments : x:","title":"forward"},{"location":"development/devel/api/#flops_3","text":"def flops()","title":"flops"},{"location":"development/devel/api/#focalnet-objects","text":"class FocalNet(nn.Module) Focal Modulation Networks (FocalNets) Arguments : img_size(int | tuple(int): Input image size. Default 224 patch_size(int | tuple(int): Patch size. Default: 4 - in_chans(int) - Number of input image channels. Default: 3 - num_classes(int) - Number of classes for classification head. Default: 1000 - embed_dim(int) - Patch embedding dimension. Default: 96 - depths(tuple(int) - Depth of each Focal Transformer layer. - mlp_ratio(float) - Ratio of mlp hidden dim to embedding dim. Default: 4 - drop_rate(float) - Dropout rate. Default: 0 - drop_path_rate(float) - Stochastic depth rate. Default: 0.1 - norm_layer(nn.Module) - Normalization layer. Default: nn.LayerNorm. - patch_norm(bool) - If True, add normalization after patch embedding. Default: True - use_checkpoint(bool) - Whether to use checkpointing to save memory. Default: False - focal_levels(list) - How many focal levels at all stages. Note that this excludes the finest-grain level. Default: [1, 1, 1, 1] - focal_windows(list) - The focal window size at all stages. Default: [7, 5, 3, 1] - use_conv_embed(bool) - Whether use convolutional embedding. We noted that using convolutional embedding usually improve the performance, but we do not use it by default. Default: False - use_layerscale(bool) - Whether use layerscale proposed in CaiT. Default: False - layerscale_value(float) - Value for layer scale. Default: 1e-4 - use_postln(bool) - Whether use layernorm after modulation (it helps stablize training of large models)","title":"FocalNet Objects"},{"location":"development/devel/api/#no_weight_decay","text":"@torch.jit.ignore def no_weight_decay()","title":"no_weight_decay"},{"location":"development/devel/api/#no_weight_decay_keywords","text":"@torch.jit.ignore def no_weight_decay_keywords()","title":"no_weight_decay_keywords"},{"location":"development/devel/api/#forward_features","text":"def forward_features(x) Arguments : x:","title":"forward_features"},{"location":"development/devel/api/#forward_64","text":"def forward(x) Arguments : x:","title":"forward"},{"location":"development/devel/api/#flops_4","text":"def flops()","title":"flops"},{"location":"development/devel/api/#build_transforms","text":"def build_transforms(img_size, center_crop=False) Arguments : img_size: - center_crop - (Default value = False)","title":"build_transforms"},{"location":"development/devel/api/#build_transforms4display","text":"def build_transforms4display(img_size, center_crop=False) Arguments : img_size: - center_crop - (Default value = False)","title":"build_transforms4display"},{"location":"development/devel/api/#focalnet_tiny_srf","text":"@register_model def focalnet_tiny_srf(pretrained=False, **kwargs) Arguments : pretrained - (Default value = False) **kwargs:","title":"focalnet_tiny_srf"},{"location":"development/devel/api/#focalnet_small_srf","text":"@register_model def focalnet_small_srf(pretrained=False, **kwargs) Arguments : pretrained - (Default value = False) **kwargs:","title":"focalnet_small_srf"},{"location":"development/devel/api/#focalnet_base_srf","text":"@register_model def focalnet_base_srf(pretrained=False, **kwargs) Arguments : pretrained - (Default value = False) **kwargs:","title":"focalnet_base_srf"},{"location":"development/devel/api/#focalnet_tiny_lrf","text":"@register_model def focalnet_tiny_lrf(pretrained=False, **kwargs) Arguments : pretrained - (Default value = False) **kwargs:","title":"focalnet_tiny_lrf"},{"location":"development/devel/api/#focalnet_small_lrf","text":"@register_model def focalnet_small_lrf(pretrained=False, **kwargs) Arguments : pretrained - (Default value = False) **kwargs:","title":"focalnet_small_lrf"},{"location":"development/devel/api/#focalnet_base_lrf","text":"@register_model def focalnet_base_lrf(pretrained=False, **kwargs) Arguments : pretrained - (Default value = False) **kwargs:","title":"focalnet_base_lrf"},{"location":"development/devel/api/#focalnet_tiny_iso_16","text":"@register_model def focalnet_tiny_iso_16(pretrained=False, **kwargs) Arguments : pretrained - (Default value = False) **kwargs:","title":"focalnet_tiny_iso_16"},{"location":"development/devel/api/#focalnet_small_iso_16","text":"@register_model def focalnet_small_iso_16(pretrained=False, **kwargs) Arguments : pretrained - (Default value = False) **kwargs:","title":"focalnet_small_iso_16"},{"location":"development/devel/api/#focalnet_base_iso_16","text":"@register_model def focalnet_base_iso_16(pretrained=False, **kwargs) Arguments : pretrained - (Default value = False) **kwargs:","title":"focalnet_base_iso_16"},{"location":"development/devel/api/#focalnet_large_fl3","text":"@register_model def focalnet_large_fl3(pretrained=False, **kwargs) Arguments : pretrained - (Default value = False) **kwargs:","title":"focalnet_large_fl3"},{"location":"development/devel/api/#focalnet_large_fl4","text":"@register_model def focalnet_large_fl4(pretrained=False, **kwargs) Arguments : pretrained - (Default value = False) **kwargs:","title":"focalnet_large_fl4"},{"location":"development/devel/api/#focalnet_xlarge_fl3","text":"@register_model def focalnet_xlarge_fl3(pretrained=False, **kwargs) Arguments : pretrained - (Default value = False) **kwargs:","title":"focalnet_xlarge_fl3"},{"location":"development/devel/api/#focalnet_xlarge_fl4","text":"@register_model def focalnet_xlarge_fl4(pretrained=False, **kwargs) Arguments : pretrained - (Default value = False) **kwargs:","title":"focalnet_xlarge_fl4"},{"location":"development/devel/api/#focalnet_huge_fl3","text":"@register_model def focalnet_huge_fl3(pretrained=False, **kwargs) Arguments : pretrained - (Default value = False) **kwargs:","title":"focalnet_huge_fl3"},{"location":"development/devel/api/#focalnet_huge_fl4","text":"@register_model def focalnet_huge_fl4(pretrained=False, **kwargs) Arguments : pretrained - (Default value = False) **kwargs:","title":"focalnet_huge_fl4"},{"location":"development/devel/api/#hannahmodelssincmodels","text":"","title":"hannah.models.sinc.models"},{"location":"development/devel/api/#gdsconv-objects","text":"class GDSConv(nn.Module)","title":"GDSConv Objects"},{"location":"development/devel/api/#forward_65","text":"def forward(x) Arguments : x:","title":"forward"},{"location":"development/devel/api/#gdsconvblock-objects","text":"class GDSConvBlock(nn.Module)","title":"GDSConvBlock Objects"},{"location":"development/devel/api/#forward_66","text":"def forward(x) Arguments : x:","title":"forward"},{"location":"development/devel/api/#sincnet-objects","text":"class SincNet(nn.Module)","title":"SincNet Objects"},{"location":"development/devel/api/#forward_67","text":"def forward(x) Arguments : x:","title":"forward"},{"location":"development/devel/api/#hannahmodelssinc","text":"","title":"hannah.models.sinc"},{"location":"development/devel/api/#hannahmodelscapsule_netmodels","text":"","title":"hannah.models.capsule_net.models"},{"location":"development/devel/api/#hannahmodelscapsule_net","text":"","title":"hannah.models.capsule_net"},{"location":"development/devel/api/#hannahmodelscapsule_netoperators","text":"","title":"hannah.models.capsule_net.operators"},{"location":"development/devel/api/#hannahmodelscapsule_netexpressions","text":"","title":"hannah.models.capsule_net.expressions"},{"location":"development/devel/api/#padding_expression_2","text":"def padding_expression(kernel_size, stride, dilation=1) Symbolically calculate padding such that for a given kernel_size, stride and dilation the padding is such that the output dimension is kept the same(stride=1) or halved(stride=2). Note: If the input dimension is 1 and stride = 2, the calculated padding will result in an output with also dimension 1.","title":"padding_expression"},{"location":"development/devel/api/#parameters_2","text":"kernel_size : Union[int, Expression] stride : Union[int, Expression] dilation : Union[int, Expression], optional description , by default 1","title":"Parameters"},{"location":"development/devel/api/#returns_2","text":"Expression","title":"Returns"},{"location":"development/devel/api/#hannahmodelscapsule_netutils","text":"","title":"hannah.models.capsule_net.utils"},{"location":"development/devel/api/#hannahtest_linear_classifier","text":"","title":"hannah.test_linear_classifier"},{"location":"development/devel/api/#hannahcallbacksdump_layers","text":"","title":"hannah.callbacks.dump_layers"},{"location":"development/devel/api/#testdumpercallback-objects","text":"class TestDumperCallback(Callback)","title":"TestDumperCallback Objects"},{"location":"development/devel/api/#on_test_start","text":"def on_test_start(pl_trainer, pl_model) Arguments : pl_trainer: pl_model:","title":"on_test_start"},{"location":"development/devel/api/#hannahcallbacksbackends","text":"","title":"hannah.callbacks.backends"},{"location":"development/devel/api/#symbolic_batch_dim","text":"def symbolic_batch_dim(model) Arguments : model:","title":"symbolic_batch_dim"},{"location":"development/devel/api/#inferencebackendbase-objects","text":"class InferenceBackendBase(Callback) Base class to run val and test on a backend inference engine","title":"InferenceBackendBase Objects"},{"location":"development/devel/api/#run_batch","text":"def run_batch(inputs=None) Arguments : inputs - (Default value = None)","title":"run_batch"},{"location":"development/devel/api/#prepare","text":"def prepare(module) Arguments : module:","title":"prepare"},{"location":"development/devel/api/#on_validation_epoch_start","text":"def on_validation_epoch_start(trainer, pl_module) Arguments : trainer: pl_module:","title":"on_validation_epoch_start"},{"location":"development/devel/api/#on_validation_batch_end","text":"def on_validation_batch_end(trainer, pl_module, outputs, batch, batch_idx, dataloader_idx) Arguments : trainer: pl_module: outputs: batch: batch_idx: dataloader_idx:","title":"on_validation_batch_end"},{"location":"development/devel/api/#on_validation_epoch_end","text":"def on_validation_epoch_end(trainer, pl_module) Arguments : trainer: pl_module:","title":"on_validation_epoch_end"},{"location":"development/devel/api/#on_test_epoch_start","text":"def on_test_epoch_start(trainer, pl_module) Arguments : trainer: pl_module:","title":"on_test_epoch_start"},{"location":"development/devel/api/#quantize","text":"def quantize(pl_module: torch.nn.Module) -> torch.nn.Module Arguments : pl_module - torch.nn.Module to quantize Returns - quantized torch.nn.Module","title":"quantize"},{"location":"development/devel/api/#on_test_batch_end","text":"def on_test_batch_end(trainer, pl_module, outputs, batch, batch_idx, dataloader_idx) Arguments : trainer: pl_module: outputs: batch: batch_idx: dataloader_idx:","title":"on_test_batch_end"},{"location":"development/devel/api/#export","text":"def export() -> None Export the model through the target backend","title":"export"},{"location":"development/devel/api/#torchmobilebackend-objects","text":"class TorchMobileBackend(InferenceBackendBase) Inference backend for torch mobile","title":"TorchMobileBackend Objects"},{"location":"development/devel/api/#prepare_1","text":"def prepare(model) Arguments : model torch.nn.Module - nn.Module to be exported Returns (None)","title":"prepare"},{"location":"development/devel/api/#run_batch_1","text":"def run_batch(inputs=None) Arguments : inputs - (Default value = None)","title":"run_batch"},{"location":"development/devel/api/#onnxtfbackend-objects","text":"class OnnxTFBackend(InferenceBackendBase) Inference Backend for tensorflow","title":"OnnxTFBackend Objects"},{"location":"development/devel/api/#prepare_2","text":"def prepare(model) Arguments : model:","title":"prepare"},{"location":"development/devel/api/#run_batch_2","text":"def run_batch(inputs) Arguments : inputs:","title":"run_batch"},{"location":"development/devel/api/#onnxruntimebackend-objects","text":"class OnnxruntimeBackend(InferenceBackendBase) Inference Backend for tensorflow","title":"OnnxruntimeBackend Objects"},{"location":"development/devel/api/#prepare_3","text":"def prepare(model) Arguments : model:","title":"prepare"},{"location":"development/devel/api/#run_batch_3","text":"def run_batch(inputs=None) Arguments : inputs - (Default value = None) Returns","title":"run_batch"},{"location":"development/devel/api/#hannahcallbackspruning","text":"","title":"hannah.callbacks.pruning"},{"location":"development/devel/api/#pruningamountscheduler-objects","text":"class PruningAmountScheduler()","title":"PruningAmountScheduler Objects"},{"location":"development/devel/api/#filteredpruning-objects","text":"class FilteredPruning(ModelPruning)","title":"FilteredPruning Objects"},{"location":"development/devel/api/#setup","text":"def setup(trainer: Trainer, pl_module: LightningModule, stage: str) Arguments : trainer: pl_module:","title":"setup"},{"location":"development/devel/api/#filter_parameters_to_prune","text":"def filter_parameters_to_prune(parameters_to_prune=None) Filter out unprunable parameters Arguments : parameters_to_prune - (Default value = None)","title":"filter_parameters_to_prune"},{"location":"development/devel/api/#on_test_end_1","text":"def on_test_end(trainer, pl_module) -> None Arguments : trainer: pl_module:","title":"on_test_end"},{"location":"development/devel/api/#hannahcallbacksoptimization","text":"","title":"hannah.callbacks.optimization"},{"location":"development/devel/api/#hydraoptcallback-objects","text":"class HydraOptCallback(Callback)","title":"HydraOptCallback Objects"},{"location":"development/devel/api/#on_test_end_2","text":"def on_test_end(trainer, pl_module) Arguments : trainer: pl_module:","title":"on_test_end"},{"location":"development/devel/api/#on_validation_end","text":"def on_validation_end(trainer, pl_module) Arguments : trainer: pl_module:","title":"on_validation_end"},{"location":"development/devel/api/#test_result","text":"def test_result()","title":"test_result"},{"location":"development/devel/api/#val_result","text":"def val_result()","title":"val_result"},{"location":"development/devel/api/#result","text":"def result(dict=False) Arguments : dict - (Default value = False)","title":"result"},{"location":"development/devel/api/#curves","text":"def curves(dict=False) Arguments : dict - (Default value = False)","title":"curves"},{"location":"development/devel/api/#hannahcallbacksfine_tuning","text":"","title":"hannah.callbacks.fine_tuning"},{"location":"development/devel/api/#hannahcallbacks","text":"","title":"hannah.callbacks"},{"location":"development/devel/api/#hannahcallbacksclustering","text":"","title":"hannah.callbacks.clustering"},{"location":"development/devel/api/#clustering","text":"def clustering(params, inertia, cluster) Arguments : params: inertia: cluster:","title":"clustering"},{"location":"development/devel/api/#kmeans-objects","text":"class kMeans(Callback)","title":"kMeans Objects"},{"location":"development/devel/api/#on_fit_end","text":"def on_fit_end(trainer, pl_module) Arguments : trainer: pl_module:","title":"on_fit_end"},{"location":"development/devel/api/#on_epoch_end","text":"def on_epoch_end(trainer, pl_module) Arguments : trainer: pl_module:","title":"on_epoch_end"},{"location":"development/devel/api/#hannahcallbackssummaries","text":"","title":"hannah.callbacks.summaries"},{"location":"development/devel/api/#walk_model","text":"def walk_model(model, dummy_input) Adapted from IntelLabs Distiller Arguments : model: dummy_input:","title":"walk_model"},{"location":"development/devel/api/#macsummarycallback-objects","text":"class MacSummaryCallback(Callback)","title":"MacSummaryCallback Objects"},{"location":"development/devel/api/#predict","text":"def predict(pl_module, input=input) Arguments : pl_module:","title":"predict"},{"location":"development/devel/api/#on_train_start","text":"@rank_zero_only def on_train_start(trainer, pl_module) Arguments : trainer: pl_module:","title":"on_train_start"},{"location":"development/devel/api/#on_test_end_3","text":"@rank_zero_only def on_test_end(trainer, pl_module) Arguments : trainer: pl_module:","title":"on_test_end"},{"location":"development/devel/api/#on_validation_epoch_end_1","text":"@rank_zero_only def on_validation_epoch_end(trainer, pl_module) Arguments : trainer: pl_module:","title":"on_validation_epoch_end"},{"location":"development/devel/api/#estimate","text":"def estimate(pl_module, input=None) Generate Summary Metrics for neural network Arguments : pl_module(pytorch_lightning.LightningModule) - pytorch lightning module to summarize Returns : dict[str, float]: Dict of MetricName => Metric Value","title":"estimate"},{"location":"development/devel/api/#hannahcallbackssvd_compress","text":"","title":"hannah.callbacks.svd_compress"},{"location":"development/devel/api/#svd-objects","text":"class SVD(Callback)","title":"SVD Objects"},{"location":"development/devel/api/#on_epoch_start","text":"def on_epoch_start(trainer, pl_module) Arguments : trainer: pl_module:","title":"on_epoch_start"},{"location":"development/devel/api/#hannahsslhard_labeling","text":"","title":"hannah.ssl.hard_labeling"},{"location":"development/devel/api/#hardlabeling-objects","text":"class HardLabeling(torch.nn.Module)","title":"HardLabeling Objects"},{"location":"development/devel/api/#forward_68","text":"def forward(unlabeled_data: torch.Tensor) -> torch.Tensor Calculate pseudo label loss from unlabeled data.","title":"forward"},{"location":"development/devel/api/#compute_uncertainty","text":"def compute_uncertainty(data: torch.Tensor, num_forward_passes: int = 10) -> torch.Tensor Compute Monte Carlo uncertainty using standard deviation.","title":"compute_uncertainty"},{"location":"development/devel/api/#get_dropout_layers","text":"def get_dropout_layers() Returns all model layers of class dropout, dropblock.","title":"get_dropout_layers"},{"location":"development/devel/api/#hannahssl","text":"","title":"hannah.ssl"},{"location":"development/devel/api/#hannahconfoptimizer","text":"","title":"hannah.conf.optimizer"},{"location":"development/devel/api/#sgdconf-objects","text":"@dataclass class SGDConf()","title":"SGDConf Objects"},{"location":"development/devel/api/#lr","text":"_RequiredParameter","title":"lr"},{"location":"development/devel/api/#madgradconf-objects","text":"@dataclass class MADGRADConf()","title":"MADGRADConf Objects"},{"location":"development/devel/api/#lr_1","text":"_RequiredParameter","title":"lr"},{"location":"development/devel/api/#hannahconf","text":"","title":"hannah.conf"},{"location":"development/devel/api/#hannahconfnas","text":"","title":"hannah.conf.nas"},{"location":"development/devel/api/#hannahconfscheduler","text":"","title":"hannah.conf.scheduler"},{"location":"development/devel/api/#onecyclelrconf-objects","text":"@dataclass class OneCycleLRConf() Config for one cycle lr total steps are configured from module","title":"OneCycleLRConf Objects"},{"location":"development/devel/api/#hannahoptim","text":"","title":"hannah.optim"},{"location":"development/devel/api/#hannahoptimmadgrad","text":"","title":"hannah.optim.madgrad"},{"location":"development/devel/api/#madgrad-objects","text":"class MADGRAD(torch.optim.Optimizer) MADGRAD_: A Momentumized, Adaptive, Dual Averaged Gradient Method for Stochastic Optimization. .. _MADGRAD: https://arxiv.org/abs/2101.11075 MADGRAD is a general purpose optimizer that can be used in place of SGD or Adam may converge faster and generalize better. Currently GPU-only. Typically, the same learning rate schedule that is used for SGD or Adam may be used. The overall learning rate is not comparable to either method and should be determined by a hyper-parameter sweep. MADGRAD requires less weight decay than other methods, often as little as zero. Momentum values used for SGD or Adam's beta1 should work here also. On sparse problems both weight_decay and momentum should be set to 0. Arguments : params (iterable): Iterable of parameters to optimize or dicts defining parameter groups. lr (float): Learning rate (default: 1e-2). momentum (float): Momentum value in the range [0,1) (default: 0.9). weight_decay (float): Weight decay, i.e. a L2 penalty (default: 0). eps (float): Term added to the denominator outside of the root operation to improve numerical stability. (default: 1e-6).","title":"MADGRAD Objects"},{"location":"development/devel/api/#step","text":"def step(closure: Optional[Callable[[], float]] = None) -> Optional[float] Performs a single optimization step. Arguments : closure callable, optional - A closure that reevaluates the model and returns the loss.","title":"step"},{"location":"development/devel/api/#hannahoptimradam","text":"","title":"hannah.optim.RAdam"},{"location":"development/devel/api/#hannahnasdataflowregistry","text":"","title":"hannah.nas.dataflow.registry"},{"location":"development/devel/api/#hannahnasdataflowdataflow_utils","text":"","title":"hannah.nas.dataflow.dataflow_utils"},{"location":"development/devel/api/#hannahnasdataflowtensor_expression","text":"","title":"hannah.nas.dataflow.tensor_expression"},{"location":"development/devel/api/#hannahnasdataflowquantization_type","text":"","title":"hannah.nas.dataflow.quantization_type"},{"location":"development/devel/api/#hannahnasdataflowregister_ops","text":"","title":"hannah.nas.dataflow.register_ops"},{"location":"development/devel/api/#hannahnasdataflowopsdropout","text":"","title":"hannah.nas.dataflow.ops.dropout"},{"location":"development/devel/api/#hannahnasdataflowopsconv2d","text":"","title":"hannah.nas.dataflow.ops.conv2d"},{"location":"development/devel/api/#hannahnasdataflowopslinear","text":"","title":"hannah.nas.dataflow.ops.linear"},{"location":"development/devel/api/#hannahnasdataflowopsidentity","text":"","title":"hannah.nas.dataflow.ops.identity"},{"location":"development/devel/api/#hannahnasdataflowopsrelu","text":"","title":"hannah.nas.dataflow.ops.relu"},{"location":"development/devel/api/#hannahnasdataflowopssum","text":"","title":"hannah.nas.dataflow.ops.sum"},{"location":"development/devel/api/#hannahnasdataflowopsbatch_nom","text":"","title":"hannah.nas.dataflow.ops.batch_nom"},{"location":"development/devel/api/#hannahnasdataflowopsadd","text":"","title":"hannah.nas.dataflow.ops.add"},{"location":"development/devel/api/#hannahnasdataflowops","text":"","title":"hannah.nas.dataflow.ops"},{"location":"development/devel/api/#hannahnasdataflowopsconcat","text":"","title":"hannah.nas.dataflow.ops.concat"},{"location":"development/devel/api/#hannahnasdataflowopspooling","text":"","title":"hannah.nas.dataflow.ops.pooling"},{"location":"development/devel/api/#hannahnasdataflowanalysisdataflow_analysis","text":"","title":"hannah.nas.dataflow.analysis.dataflow_analysis"},{"location":"development/devel/api/#hannahnasdataflowtensor","text":"","title":"hannah.nas.dataflow.tensor"},{"location":"development/devel/api/#hannahnasdataflowtensor_type","text":"","title":"hannah.nas.dataflow.tensor_type"},{"location":"development/devel/api/#hannahnasdataflowtransformationsgraph_tranformer","text":"","title":"hannah.nas.dataflow.transformations.graph_tranformer"},{"location":"development/devel/api/#hannahnasdataflowscoping_utils","text":"","title":"hannah.nas.dataflow.scoping_utils"},{"location":"development/devel/api/#hannahnasdataflow","text":"","title":"hannah.nas.dataflow"},{"location":"development/devel/api/#hannahnasdataflowrepeat","text":"","title":"hannah.nas.dataflow.repeat"},{"location":"development/devel/api/#hannahnasdataflowaxis_type","text":"","title":"hannah.nas.dataflow.axis_type"},{"location":"development/devel/api/#axistuple-objects","text":"@parametrize class AxisTuple() Used to have the axis dict as a parametrized object","title":"AxisTuple Objects"},{"location":"development/devel/api/#hannahnasdataflowdata_type","text":"","title":"hannah.nas.dataflow.data_type"},{"location":"development/devel/api/#hannahnasdataflowoptional_op","text":"","title":"hannah.nas.dataflow.optional_op"},{"location":"development/devel/api/#hannahnasdataflowcompression_type","text":"","title":"hannah.nas.dataflow.compression_type"},{"location":"development/devel/api/#hannahnasdataflowop_type","text":"","title":"hannah.nas.dataflow.op_type"},{"location":"development/devel/api/#hannahnasdataflowdataflow_graph","text":"","title":"hannah.nas.dataflow.dataflow_graph"},{"location":"development/devel/api/#dataflowgraph-objects","text":"@parametrize class DataFlowGraph(TensorExpression)","title":"DataFlowGraph Objects"},{"location":"development/devel/api/#link_users","text":"def link_users() Link the DFG to its users and the users of the DFG to the DFG","title":"link_users"},{"location":"development/devel/api/#collect_users","text":"def collect_users(node) Traverse graph starting from node and collect all users (including users from subsequent nodes). If a node_b is NOT in collect_users(node_a), this means that node_b is either BEFORE node_a in the graph OR it is in a parallel branch.","title":"collect_users"},{"location":"development/devel/api/#parameters_3","text":"node : type description","title":"Parameters"},{"location":"development/devel/api/#returns_3","text":"type description","title":"Returns"},{"location":"development/devel/api/#find_first_input","text":"def find_first_input(node) Recusively traverses the graph from the given node back to its first input. NOTE: The traversal is via OPERANDS and not OUTPUT, meaning that e.g. weight Tensors that are included in Ops in a DFG are not returned","title":"find_first_input"},{"location":"development/devel/api/#parameters_4","text":"node : type description","title":"Parameters"},{"location":"development/devel/api/#returns_4","text":"type description","title":"Returns"},{"location":"development/devel/api/#hannahnasperformance_predictionsimple","text":"","title":"hannah.nas.performance_prediction.simple"},{"location":"development/devel/api/#backendpredictor-objects","text":"class BackendPredictor() A predictor class that instantiates the model and uses the backends predict function to predict performance metrics","title":"BackendPredictor Objects"},{"location":"development/devel/api/#macpredictor-objects","text":"class MACPredictor() A predictor class that instantiates the model and calculates abstract metrics","title":"MACPredictor Objects"},{"location":"development/devel/api/#gcnpredictor-objects","text":"class GCNPredictor() A predictor class that instantiates the model and uses the backends predict function to predict performance metrics","title":"GCNPredictor Objects"},{"location":"development/devel/api/#hannahnasperformance_predictiongcnpredictor","text":"","title":"hannah.nas.performance_prediction.gcn.predictor"},{"location":"development/devel/api/#predictor-objects","text":"class Predictor()","title":"Predictor Objects"},{"location":"development/devel/api/#__init___1","text":"def __init__(fea_name=\"features\") -> None Parent method for different predictor classes.","title":"__init__"},{"location":"development/devel/api/#parameters_5","text":"fea_name : str, optional internal name for features in the graph, as in graph.ndata[fea_name], by default 'features'","title":"Parameters"},{"location":"development/devel/api/#train_2","text":"def train(dataloader, learning_rate=1e-3, num_epochs=200, validation_dataloader=None, verbose=1) Train GCN model","title":"train"},{"location":"development/devel/api/#parameters_6","text":"dataloader : GraphDataLoader training data learning_rate : [type], optional by default 1e-3 num_epochs : int, optional by default 200 validation_dataloader : [type], optional if given, use this data to print validation loss, by default None verbose : int if validation_dataloader is given, print validation MSE every epoch, by default 1","title":"Parameters"},{"location":"development/devel/api/#predict_1","text":"def predict(graph) predict cost of graph","title":"predict"},{"location":"development/devel/api/#parameters_7","text":"graph : dgl.Graph","title":"Parameters"},{"location":"development/devel/api/#returns_5","text":"torch.Tensor predicted cost of given graph. Retrieve float value with .item()","title":"Returns"},{"location":"development/devel/api/#gcnpredictor-objects_1","text":"class GCNPredictor(Predictor)","title":"GCNPredictor Objects"},{"location":"development/devel/api/#__init___2","text":"def __init__(input_feature_size, hidden_units=[128], readout=\"mean\", fea_name=\"features\") -> None G(raph)CN based network latency/cost predictor. End-to-end from graph to score.","title":"__init__"},{"location":"development/devel/api/#parameters_8","text":"input_feature_size : [type] length of feature vector of a graph node (graph G with n nodes, each with features of length m, i.e. feature matrix F = n x m) hidden_units : int, list, optional size of hidden layer (layers if list) , by default 128 readout : str, optional readout function that is used to aggregate node features, by default 'mean' fea_name : str, optional internal name for features in the graph, as in graph.ndata[fea_name], by default 'features'","title":"Parameters"},{"location":"development/devel/api/#train_and_fit","text":"def train_and_fit(dataloader, learning_rate=1e-3, num_epochs=200, validation_dataloader=None, verbose=0) Train GCN model","title":"train_and_fit"},{"location":"development/devel/api/#parameters_9","text":"dataloader : GraphDataLoader training data learning_rate : [type], optional by default 1e-3 num_epochs : int, optional by default 200 validation_dataloader : [type], optional if given, use this data to print validation loss, by default None verbose : int if validation_dataloader is given, print validation MSE every epoch, by default 1","title":"Parameters"},{"location":"development/devel/api/#predict_2","text":"def predict(graph) predict cost of graph","title":"predict"},{"location":"development/devel/api/#parameters_10","text":"graph : dgl.Graph","title":"Parameters"},{"location":"development/devel/api/#returns_6","text":"torch.Tensor predicted cost of given graph. Retrieve float value with .item()","title":"Returns"},{"location":"development/devel/api/#gaussianprocesspredictor-objects","text":"class GaussianProcessPredictor(Predictor)","title":"GaussianProcessPredictor Objects"},{"location":"development/devel/api/#__init___3","text":"def __init__(input_feature_size, hidden_units=128, embedding_size=10, readout=\"mean\", fea_name=\"features\", kernel=\"default\", alpha=1e-10) -> None Predictor that generates a graph embedding that is used as input for a gaussian process predictor.","title":"__init__"},{"location":"development/devel/api/#parameters_11","text":"input_feature_size : [type] length of feature vector of a graph node (graph G with n nodes, each with features of length m, i.e. feature matrix F = n x m) hidden_units : int, list, optional size of hidden layer (layers if list) , by default 128 embedding_size: int, optional size of output embedding readout : str, optional readout function that is used to aggregate node features, by default 'mean' fea_name : str, optional internal name for features in the graph, as in graph.ndata[fea_name], by default 'features' kernel : str, sklearn.gaussian_process.kernels.Kernel, optional The gaussian process kernel to use. input shoudl be either \"default\", or a sklearn Kernel() object by default RBF() + DotProduct() + WhiteKernel()","title":"Parameters"},{"location":"development/devel/api/#train_and_fit_1","text":"def train_and_fit(dataloader, learning_rate=1e-3, num_epochs=200, validation_dataloader=None, verbose=1) Train GCN model, generate embeddings for training data and fit the predictor with embeddings.","title":"train_and_fit"},{"location":"development/devel/api/#parameters_12","text":"dataloader : GraphDataLoader training data learning_rate : [type], optional by default 1e-3 num_epochs : int, optional by default 200 validation_dataloader : [type], optional if given, use this data to print validation loss, by default None verbose : int if validation_dataloader is given, print validation MSE every epoch,by default 1","title":"Parameters"},{"location":"development/devel/api/#returns_7","text":"float score of predictor on TRAINING data, see sklearn doc of chosen predictor for more info","title":"Returns"},{"location":"development/devel/api/#predict_3","text":"def predict(X, return_std=True) Predict cost/latency of graphs.","title":"predict"},{"location":"development/devel/api/#parameters_13","text":"X : dgl.DGLGraph, list[DGLGraph], dgl.dataloading.GraphDataLoader Input graph(s) return_std : bool, optional if true, return standard dev. else just mean prediction, by default True","title":"Parameters"},{"location":"development/devel/api/#returns_8","text":"array (,array) prediction(s) , (if return_std: standard deviation(s))","title":"Returns"},{"location":"development/devel/api/#xgbpredictor-objects","text":"class XGBPredictor(Predictor)","title":"XGBPredictor Objects"},{"location":"development/devel/api/#__init___4","text":"def __init__(input_feature_size, hidden_units=128, embedding_size=10, readout=\"mean\", fea_name=\"features\", xgb_param=\"default\") -> None Predictor that generates a graph embedding that is used as input for a xgb based predictor.","title":"__init__"},{"location":"development/devel/api/#parameters_14","text":"input_feature_size : [type] length of feature vector of a graph node (graph G with n nodes, each with features of length m, i.e. feature matrix F = n x m) hidden_units : int, list, optional size of hidden layer (layers if list) , by default 128 embedding_size: int, optional size of output embedding readout : str, optional readout function that is used to aggregate node features, by default 'mean' fea_name : str, optional internal name for features in the graph, as in graph.ndata[fea_name], by default 'features' xgb_param : str, dict, optional The xgb_parameter to use. See https://xgboost.readthedocs.io/en/latest/parameter.html","title":"Parameters"},{"location":"development/devel/api/#train_and_fit_2","text":"def train_and_fit(dataloader, learning_rate=1e-3, num_epochs=200, num_round=8000, validation_dataloader=None, verbose=1) Train GCN model, generate embeddings for training data and fit the predictor with embeddings.","title":"train_and_fit"},{"location":"development/devel/api/#parameters_15","text":"dataloader : GraphDataLoader training data learning_rate : [type], optional by default 1e-3 num_epochs : int, optional Training epochs for the GCN embedding network, by default 200 num_round : int, optional training rounds for xgb booster, by default 800 validation_dataloader : [type], optional if given, use this data to print validation loss, by default None verbose : int if validation_dataloader is given, print validation MSE every epoch,by default 1","title":"Parameters"},{"location":"development/devel/api/#predict_4","text":"def predict(X) Predict cost/latency of graphs.","title":"predict"},{"location":"development/devel/api/#parameters_16","text":"X : dgl.DGLGraph, list[DGLGraph], dgl.dataloading.GraphDataLoader Input graph(s) Returns array (,array) prediction(s) , (if return_std: standard deviation(s))","title":"Parameters"},{"location":"development/devel/api/#prepare_dataloader","text":"def prepare_dataloader(dataset, batch_size=50, train_test_split=1, subset=0, seed=0, validation=False) helper function to construct dataloaders from NASGraphDataset","title":"prepare_dataloader"},{"location":"development/devel/api/#parameters_17","text":"dataset : NASGraphDataset batch_size : int, optional by default 50 train_test_split : float, optional number between 0 and 1, the proportion of the dataset to be used for training, by default 1 subset : int, optional choose only many samples from the dataset. Set 0 for disabling, i.e. whole dataset. by default 0 seed : int, optional set seed for reproduceability validation : bool, optional also output a validation set e.g. for hyperparam tuning","title":"Parameters"},{"location":"development/devel/api/#returns_9","text":"tuple(GraphDataLoader, (GraphDataLoader), GraphDataLoader) training dataloader to be used in CostPredictor.train() and test/validation dataloader if train_test_split > 0, else len(test_dataloader) == 0","title":"Returns"},{"location":"development/devel/api/#hannahnasperformance_predictiongcnmodel","text":"","title":"hannah.nas.performance_prediction.gcn.model"},{"location":"development/devel/api/#hannahnasperformance_predictionfeaturesdataset","text":"","title":"hannah.nas.performance_prediction.features.dataset"},{"location":"development/devel/api/#hannahnasperformance_predictionfeaturesgraph_conversion","text":"","title":"hannah.nas.performance_prediction.features.graph_conversion"},{"location":"development/devel/api/#hannahnasperformance_prediction","text":"","title":"hannah.nas.performance_prediction"},{"location":"development/devel/api/#hannahnasperformance_predictionexamplesgcn_predictor_example","text":"","title":"hannah.nas.performance_prediction.examples.gcn_predictor_example"},{"location":"development/devel/api/#hannahnasperformance_predictionexamplesgcn_model_example","text":"","title":"hannah.nas.performance_prediction.examples.gcn_model_example"},{"location":"development/devel/api/#hannahnasperformance_predictionexamplesri_capsule_performance_predictor","text":"","title":"hannah.nas.performance_prediction.examples.ri_capsule_performance_predictor"},{"location":"development/devel/api/#hannahnasperformance_predictionexamplesgaussian_process_predictor_example","text":"","title":"hannah.nas.performance_prediction.examples.gaussian_process_predictor_example"},{"location":"development/devel/api/#hannahnasperformance_predictionexamplesxgb_predictor_example","text":"","title":"hannah.nas.performance_prediction.examples.xgb_predictor_example"},{"location":"development/devel/api/#hannahnasconstraintsconstraint_model","text":"","title":"hannah.nas.constraints.constraint_model"},{"location":"development/devel/api/#hannahnasconstraintsdfg_constraint_model","text":"","title":"hannah.nas.constraints.dfg_constraint_model"},{"location":"development/devel/api/#constraintmodel-objects","text":"class ConstraintModel()","title":"ConstraintModel Objects"},{"location":"development/devel/api/#process_optype","text":"def process_optype(op: OpType) Extracts the constraints based on the type of op. New variables are added to self.vars and the constraints are added to the solver.","title":"process_optype"},{"location":"development/devel/api/#parameters_18","text":"op : OpType","title":"Parameters"},{"location":"development/devel/api/#process_tensor","text":"def process_tensor(tensor: Tensor) Goes through all axis and extracts the constraints for the respective axis sizes","title":"process_tensor"},{"location":"development/devel/api/#parameters_19","text":"tensor : Tensor","title":"Parameters"},{"location":"development/devel/api/#hannahnashardware_descriptiondevice","text":"","title":"hannah.nas.hardware_description.device"},{"location":"development/devel/api/#hannahnashardware_descriptionmemory_type","text":"","title":"hannah.nas.hardware_description.memory_type"},{"location":"development/devel/api/#hannahnashardware_description","text":"","title":"hannah.nas.hardware_description"},{"location":"development/devel/api/#hannahnashardware_descriptiondescription","text":"","title":"hannah.nas.hardware_description.description"},{"location":"development/devel/api/#hannahnasevalextract","text":"","title":"hannah.nas.eval.extract"},{"location":"development/devel/api/#hannahnaseval","text":"","title":"hannah.nas.eval"},{"location":"development/devel/api/#hannahnasevalplot","text":"","title":"hannah.nas.eval.plot"},{"location":"development/devel/api/#hannahnaseval__main__","text":"","title":"hannah.nas.eval.__main__"},{"location":"development/devel/api/#hannahnasevalprepare","text":"","title":"hannah.nas.eval.prepare"},{"location":"development/devel/api/#prepare_summary","text":"def prepare_summary(data: Dict[str, str], base_dir: str = \".\", force: bool = False) -> pd.DataFrame Prepare a summary of one or multiple nas runs Arguments : data Dict[str, str] - A mapping from a short name for a nas run \"e.g KWS\" to a folder containing nas history file e.g. \"trained_models/nas_kws/conv_net_trax\" base_dir str - base directory paths in data mapping are interpreted relative to base directory force bool - force reconstructing of cached results (\"data.pkl\")","title":"prepare_summary"},{"location":"development/devel/api/#hannahnasexpressionsconditions","text":"","title":"hannah.nas.expressions.conditions"},{"location":"development/devel/api/#hannahnasexpressionsop","text":"","title":"hannah.nas.expressions.op"},{"location":"development/devel/api/#hannahnasexpressionsarithmetic","text":"","title":"hannah.nas.expressions.arithmetic"},{"location":"development/devel/api/#hannahnasexpressionsplaceholder","text":"","title":"hannah.nas.expressions.placeholder"},{"location":"development/devel/api/#hannahnasexpressions","text":"","title":"hannah.nas.expressions"},{"location":"development/devel/api/#hannahnasexpressionslogic","text":"","title":"hannah.nas.expressions.logic"},{"location":"development/devel/api/#hannahnasexpressionsshapes","text":"","title":"hannah.nas.expressions.shapes"},{"location":"development/devel/api/#hannahnasexpressionsmetrics","text":"","title":"hannah.nas.expressions.metrics"},{"location":"development/devel/api/#hannahnasexpressionstypes","text":"","title":"hannah.nas.expressions.types"},{"location":"development/devel/api/#hannahnasexpressionschoice","text":"","title":"hannah.nas.expressions.choice"},{"location":"development/devel/api/#hannahnastesttest_symbolic_metrics","text":"","title":"hannah.nas.test.test_symbolic_metrics"},{"location":"development/devel/api/#hannahnastesttest_parameters","text":"","title":"hannah.nas.test.test_parameters"},{"location":"development/devel/api/#hannahnastesttest_op_to_torch_conversion","text":"","title":"hannah.nas.test.test_op_to_torch_conversion"},{"location":"development/devel/api/#hannahnastesttest_conditions","text":"","title":"hannah.nas.test.test_conditions"},{"location":"development/devel/api/#hannahnastesttest_lazy_torch","text":"","title":"hannah.nas.test.test_lazy_torch"},{"location":"development/devel/api/#hannahnastesttest_mobilenet","text":"","title":"hannah.nas.test.test_mobilenet"},{"location":"development/devel/api/#hannahnastesttest_scoping","text":"","title":"hannah.nas.test.test_scoping"},{"location":"development/devel/api/#hannahnastesttest_darts_space","text":"","title":"hannah.nas.test.test_darts_space"},{"location":"development/devel/api/#hannahnastesttest_arithmetic","text":"","title":"hannah.nas.test.test_arithmetic"},{"location":"development/devel/api/#test_unimplemeted","text":"@pytest.mark.parametrize( \"x,y\", [ (IntScalarParameter(0, 0), 2), (IntScalarParameter(0, 0), IntScalarParameter(0, 0)), (DefaultInt(0), 2), ], ) def test_unimplemeted(x, y) Test that unimplemented methods raise unimplemented errors","title":"test_unimplemeted"},{"location":"development/devel/api/#hannahnastesttest_z3","text":"","title":"hannah.nas.test.test_z3"},{"location":"development/devel/api/#hannahnastesttest_graph_transformer","text":"","title":"hannah.nas.test.test_graph_transformer"},{"location":"development/devel/api/#hannahnastesttest_dfg_removal","text":"","title":"hannah.nas.test.test_dfg_removal"},{"location":"development/devel/api/#hannahnastesttest_repeat","text":"","title":"hannah.nas.test.test_repeat"},{"location":"development/devel/api/#hannahnastesttest_parametrize","text":"","title":"hannah.nas.test.test_parametrize"},{"location":"development/devel/api/#hannahnastestnetwork","text":"","title":"hannah.nas.test.network"},{"location":"development/devel/api/#hannahnastesttest_description_ultratrail","text":"","title":"hannah.nas.test.test_description_ultratrail"},{"location":"development/devel/api/#hannahnastesttest_dataflow","text":"","title":"hannah.nas.test.test_dataflow"},{"location":"development/devel/api/#hannahnastesttest_conv2d","text":"","title":"hannah.nas.test.test_conv2d"},{"location":"development/devel/api/#hannahnastesttest_constraint_model","text":"","title":"hannah.nas.test.test_constraint_model"},{"location":"development/devel/api/#hannahnastesttest_add","text":"","title":"hannah.nas.test.test_add"},{"location":"development/devel/api/#hannahnastesttest_adjacency","text":"","title":"hannah.nas.test.test_adjacency"},{"location":"development/devel/api/#hannahnasparametrization","text":"","title":"hannah.nas.parametrization"},{"location":"development/devel/api/#scalarparameterstate-objects","text":"@dataclass class ScalarParameterState(ParameterState)","title":"ScalarParameterState Objects"},{"location":"development/devel/api/#sigma","text":"variance of sampling parameter","title":"sigma"},{"location":"development/devel/api/#hannahnasparametersparameters","text":"","title":"hannah.nas.parameters.parameters"},{"location":"development/devel/api/#hannahnasparametersparametrize","text":"","title":"hannah.nas.parameters.parametrize"},{"location":"development/devel/api/#hannahnasparametersiterators","text":"","title":"hannah.nas.parameters.iterators"},{"location":"development/devel/api/#hannahnasparameters","text":"","title":"hannah.nas.parameters"},{"location":"development/devel/api/#hannahnasparameterslazy","text":"","title":"hannah.nas.parameters.lazy"},{"location":"development/devel/api/#hannahnas","text":"","title":"hannah.nas"},{"location":"development/devel/api/#hannahnascoreparametrized","text":"","title":"hannah.nas.core.parametrized"},{"location":"development/devel/api/#hannahnascore","text":"","title":"hannah.nas.core"},{"location":"development/devel/api/#hannahnascoreexpression","text":"","title":"hannah.nas.core.expression"},{"location":"development/devel/api/#hannahnasconfig","text":"","title":"hannah.nas.config"},{"location":"development/devel/api/#scalar-objects","text":"@dataclass class Scalar() Representation of all the options to define a scalar.","title":"Scalar Objects"},{"location":"development/devel/api/#hannahnas__main__","text":"","title":"hannah.nas.__main__"},{"location":"development/devel/api/#hannahnasplotter","text":"","title":"hannah.nas.plotter"},{"location":"development/devel/api/#hannahnassearchsearch","text":"","title":"hannah.nas.search.search"},{"location":"development/devel/api/#hannahnassearchmodel_trainersimple_model_trainer","text":"","title":"hannah.nas.search.model_trainer.simple_model_trainer"},{"location":"development/devel/api/#hannahnassearchmodel_trainerprogressive_shrinking","text":"","title":"hannah.nas.search.model_trainer.progressive_shrinking"},{"location":"development/devel/api/#progressiveshrinkingmodeltrainer-objects","text":"class ProgressiveShrinkingModelTrainer()","title":"ProgressiveShrinkingModelTrainer Objects"},{"location":"development/devel/api/#warmup","text":"def warmup(model, ofa_model) The function rebuilds the trainer with the warmup epochs, fits the model, validates the model, and then calls the on_warmup_end() function to change some internal variables Arguments : model : the model to be trained ofa_model : the model that we want to train","title":"warmup"},{"location":"development/devel/api/#train_elastic_width","text":"def train_elastic_width(model, ofa_model) The function trains the model for a number of epochs, then adds a width step, then trains the model for a number of epochs, then adds a width step, and so on Arguments : model : the model to train ofa_model : the model that will be trained","title":"train_elastic_width"},{"location":"development/devel/api/#train_elastic_depth","text":"def train_elastic_depth(model, ofa_model) The function trains the model for a number of epochs, then progressively shrinks the depth of the model, and trains the model for a number of epochs again Arguments : model : the model to train ofa_model : the model to be trained","title":"train_elastic_depth"},{"location":"development/devel/api/#train_elastic_kernel","text":"def train_elastic_kernel(model, ofa_model) The function trains the elastic kernels by progressively shrinking the model and training the model for a number of epochs and repeats this process until the number of kernel steps is reached Arguments : model : the model to train ofa_model : the model that will be trained","title":"train_elastic_kernel"},{"location":"development/devel/api/#train_elastic_dilation","text":"def train_elastic_dilation(model, ofa_model) The function trains the model for a number of epochs, then adds a dilation step, and trains the model for a number of epochs, and repeats this process until the number of dilation steps is reached Arguments : model : the model to be trained ofa_model : the model that will be trained","title":"train_elastic_dilation"},{"location":"development/devel/api/#train_elastic_grouping","text":"def train_elastic_grouping(model, ofa_model) The function trains the model for a number of epochs, then adds a group step, and trains the model for a number of epochs, and repeats this process until the number of group steps is reached Arguments : model : the model to be trained ofa_model : the model that will be trained","title":"train_elastic_grouping"},{"location":"development/devel/api/#train_elastic_dsc","text":"def train_elastic_dsc(model, ofa_model) The function trains the model for a number of epochs, then adds a dsc step (turns Depthwise Separable Convolution on and off), and trains the model for a number of epochs, and repeats this process until the number of dsc steps is reached Arguments : model : the model to be trained ofa_model : the model that will be trained","title":"train_elastic_dsc"},{"location":"development/devel/api/#eval_elastic_width","text":"def eval_elastic_width(method_stack, method_index, lightning_model, model, trainer_path, loginfo_output, metrics_output, metrics_csv) This function steps down the width of the model, and then calls the next method in the stack Arguments : method_stack : a list of methods that will be called in order method_index : The index of the current method in the method stack lightning_model : the lightning model to be trained model : the model to be trained trainer_path : The path to the trainer loginfo_output : This is the string that will be printed to the console metrics_output : a string that will be written to the metrics csv file metrics_csv : a string that contains the metrics for the current model Returns : The metrics_csv is being returned.","title":"eval_elastic_width"},{"location":"development/devel/api/#eval_elastic_kernel","text":"def eval_elastic_kernel(method_stack, method_index, lightning_model, model, trainer_path, loginfo_output, metrics_output, metrics_csv) This function steps down the kernel size of the model, and then calls the next method in the stack Arguments : method_stack : The list of methods to be called method_index : The index of the current method in the method stack lightning_model : the lightning model to be trained model : the model to be trained trainer_path : The path to the trainer loginfo_output : This is the string that will be printed to the console metrics_output : This is the string that will be printed to the console metrics_csv : a string that contains the metrics for the current model Returns : The metrics_csv is being returned.","title":"eval_elastic_kernel"},{"location":"development/devel/api/#eval_elastic_dilation","text":"def eval_elastic_dilation(method_stack, method_index, lightning_model, model, trainer_path, loginfo_output, metrics_output, metrics_csv) This function evaluates the model with a different dilation size for each layer Arguments : method_stack : The list of methods to be called method_index : The index of the method in the method stack lightning_model : the lightning model to be trained model : the model to be evaluated trainer_path : The path to the trainer loginfo_output : This is the string that will be printed to the console metrics_output : a string that will be written to the metrics csv file metrics_csv : a string that contains the csv data for the metrics Returns : The metrics_csv is being returned.","title":"eval_elastic_dilation"},{"location":"development/devel/api/#eval_elastic_depth","text":"def eval_elastic_depth(method_stack, method_index, lightning_model, model, trainer_path, loginfo_output, metrics_output, metrics_csv) This function will run the next method in the stack for each depth step, and then return the metrics_csv Arguments : method_stack : The list of methods to be called method_index : The index of the current method in the method stack lightning_model : the lightning model to be trained model : The model to be trained trainer_path : The path to the trainer, which is used to save the model loginfo_output : This is the string that will be printed to the console metrics_output : This is the string that will be printed to the console metrics_csv : This is the CSV file that we're writing to Returns : The metrics_csv is being returned.","title":"eval_elastic_depth"},{"location":"development/devel/api/#eval_elastic_grouping","text":"def eval_elastic_grouping(method_stack, method_index, lightning_model, model, trainer_path, loginfo_output, metrics_output, metrics_csv) This function evaluates the model with a different group size for each layer Arguments : method_stack : The list of methods to be called method_index : The index of the method in the method stack lightning_model : the lightning model to be trained model : the model to be evaluated trainer_path : The path to the trainer loginfo_output : This is the string that will be printed to the console metrics_output : a string that will be written to the metrics csv file metrics_csv : a string that contains the csv data for the metrics Returns : The metrics_csv is being returned.","title":"eval_elastic_grouping"},{"location":"development/devel/api/#eval_elastic_dsc","text":"def eval_elastic_dsc(method_stack, method_index, lightning_model, model, trainer_path, loginfo_output, metrics_output, metrics_csv) This function evaluates the model with a different dsc for each layer Arguments : method_stack : The list of methods to be called method_index : The index of the method in the method stack lightning_model : the lightning model to be trained model : the model to be evaluated trainer_path : The path to the trainer loginfo_output : This is the string that will be printed to the console metrics_output : a string that will be written to the metrics csv file metrics_csv : a string that contains the csv data for the metrics Returns : The metrics_csv is being returned.","title":"eval_elastic_dsc"},{"location":"development/devel/api/#eval_single_model","text":"def eval_single_model(method_stack, method_index, lightning_model, model, trainer_path, loginfo_output, metrics_output, metrics_csv) This function takes in a model, a trainer, and a bunch of other stuff, evaluates the model and tracks the results in der in the given strings and returns a string of metrics Arguments : method_stack : The list of methods that we're evaluating method_index : The index of the method in the method stack lightning_model : the lightning model that we want to evaluate model : The model to be evaluated trainer_path : the path to the trainer object loginfo_output : This is the string that will be printed to the console when the model is being evaluated metrics_output : This is the string that will be written to the metrics file. It contains the method name, the method index, and the method stack metrics_csv : a string that will be written to a csv file Returns : The metrics_csv is being returned.","title":"eval_single_model"},{"location":"development/devel/api/#eval_model","text":"def eval_model(lightning_model, model) First the method stack for the evaluation ist build and then it is according to this evaluated Arguments : lightning_model : the lightning model model : the model to be evaluated","title":"eval_model"},{"location":"development/devel/api/#hannahnassearchsearch_old","text":"","title":"hannah.nas.search.search_old"},{"location":"development/devel/api/#ofanastrainer-objects","text":"class OFANasTrainer(NASTrainerBase)","title":"OFANasTrainer Objects"},{"location":"development/devel/api/#warmup_1","text":"def warmup(model, ofa_model) The function rebuilds the trainer with the warmup epochs, fits the model, validates the model, and then calls the on_warmup_end() function to change some internal variables Arguments : model : the model to be trained ofa_model : the model that we want to train","title":"warmup"},{"location":"development/devel/api/#train_elastic_width_1","text":"def train_elastic_width(model, ofa_model) The function trains the model for a number of epochs, then adds a width step, then trains the model for a number of epochs, then adds a width step, and so on Arguments : model : the model to train ofa_model : the model that will be trained","title":"train_elastic_width"},{"location":"development/devel/api/#train_elastic_depth_1","text":"def train_elastic_depth(model, ofa_model) The function trains the model for a number of epochs, then progressively shrinks the depth of the model, and trains the model for a number of epochs again Arguments : model : the model to train ofa_model : the model to be trained","title":"train_elastic_depth"},{"location":"development/devel/api/#train_elastic_kernel_1","text":"def train_elastic_kernel(model, ofa_model) The function trains the elastic kernels by progressively shrinking the model and training the model for a number of epochs and repeats this process until the number of kernel steps is reached Arguments : model : the model to train ofa_model : the model that will be trained","title":"train_elastic_kernel"},{"location":"development/devel/api/#train_elastic_dilation_1","text":"def train_elastic_dilation(model, ofa_model) The function trains the model for a number of epochs, then adds a dilation step, and trains the model for a number of epochs, and repeats this process until the number of dilation steps is reached Arguments : model : the model to be trained ofa_model : the model that will be trained","title":"train_elastic_dilation"},{"location":"development/devel/api/#train_elastic_grouping_1","text":"def train_elastic_grouping(model, ofa_model) The function trains the model for a number of epochs, then adds a group step, and trains the model for a number of epochs, and repeats this process until the number of group steps is reached Arguments : model : the model to be trained ofa_model : the model that will be trained","title":"train_elastic_grouping"},{"location":"development/devel/api/#train_elastic_dsc_1","text":"def train_elastic_dsc(model, ofa_model) The function trains the model for a number of epochs, then adds a dsc step (turns Depthwise Separable Convolution on and off), and trains the model for a number of epochs, and repeats this process until the number of dsc steps is reached Arguments : model : the model to be trained ofa_model : the model that will be trained","title":"train_elastic_dsc"},{"location":"development/devel/api/#eval_elastic_width_1","text":"def eval_elastic_width(method_stack, method_index, lightning_model, model, trainer_path, loginfo_output, metrics_output, metrics_csv) This function steps down the width of the model, and then calls the next method in the stack Arguments : method_stack : a list of methods that will be called in order method_index : The index of the current method in the method stack lightning_model : the lightning model to be trained model : the model to be trained trainer_path : The path to the trainer loginfo_output : This is the string that will be printed to the console metrics_output : a string that will be written to the metrics csv file metrics_csv : a string that contains the metrics for the current model Returns : The metrics_csv is being returned.","title":"eval_elastic_width"},{"location":"development/devel/api/#eval_elastic_kernel_1","text":"def eval_elastic_kernel(method_stack, method_index, lightning_model, model, trainer_path, loginfo_output, metrics_output, metrics_csv) This function steps down the kernel size of the model, and then calls the next method in the stack Arguments : method_stack : The list of methods to be called method_index : The index of the current method in the method stack lightning_model : the lightning model to be trained model : the model to be trained trainer_path : The path to the trainer loginfo_output : This is the string that will be printed to the console metrics_output : This is the string that will be printed to the console metrics_csv : a string that contains the metrics for the current model Returns : The metrics_csv is being returned.","title":"eval_elastic_kernel"},{"location":"development/devel/api/#eval_elastic_dilation_1","text":"def eval_elastic_dilation(method_stack, method_index, lightning_model, model, trainer_path, loginfo_output, metrics_output, metrics_csv) This function evaluates the model with a different dilation size for each layer Arguments : method_stack : The list of methods to be called method_index : The index of the method in the method stack lightning_model : the lightning model to be trained model : the model to be evaluated trainer_path : The path to the trainer loginfo_output : This is the string that will be printed to the console metrics_output : a string that will be written to the metrics csv file metrics_csv : a string that contains the csv data for the metrics Returns : The metrics_csv is being returned.","title":"eval_elastic_dilation"},{"location":"development/devel/api/#eval_elastic_depth_1","text":"def eval_elastic_depth(method_stack, method_index, lightning_model, model, trainer_path, loginfo_output, metrics_output, metrics_csv) This function will run the next method in the stack for each depth step, and then return the metrics_csv Arguments : method_stack : The list of methods to be called method_index : The index of the current method in the method stack lightning_model : the lightning model to be trained model : The model to be trained trainer_path : The path to the trainer, which is used to save the model loginfo_output : This is the string that will be printed to the console metrics_output : This is the string that will be printed to the console metrics_csv : This is the CSV file that we're writing to Returns : The metrics_csv is being returned.","title":"eval_elastic_depth"},{"location":"development/devel/api/#eval_elastic_grouping_1","text":"def eval_elastic_grouping(method_stack, method_index, lightning_model, model, trainer_path, loginfo_output, metrics_output, metrics_csv) This function evaluates the model with a different group size for each layer Arguments : method_stack : The list of methods to be called method_index : The index of the method in the method stack lightning_model : the lightning model to be trained model : the model to be evaluated trainer_path : The path to the trainer loginfo_output : This is the string that will be printed to the console metrics_output : a string that will be written to the metrics csv file metrics_csv : a string that contains the csv data for the metrics Returns : The metrics_csv is being returned.","title":"eval_elastic_grouping"},{"location":"development/devel/api/#eval_elastic_dsc_1","text":"def eval_elastic_dsc(method_stack, method_index, lightning_model, model, trainer_path, loginfo_output, metrics_output, metrics_csv) This function evaluates the model with a different dsc for each layer Arguments : method_stack : The list of methods to be called method_index : The index of the method in the method stack lightning_model : the lightning model to be trained model : the model to be evaluated trainer_path : The path to the trainer loginfo_output : This is the string that will be printed to the console metrics_output : a string that will be written to the metrics csv file metrics_csv : a string that contains the csv data for the metrics Returns : The metrics_csv is being returned.","title":"eval_elastic_dsc"},{"location":"development/devel/api/#eval_single_model_1","text":"def eval_single_model(method_stack, method_index, lightning_model, model, trainer_path, loginfo_output, metrics_output, metrics_csv) This function takes in a model, a trainer, and a bunch of other stuff, evaluates the model and tracks the results in der in the given strings and returns a string of metrics Arguments : method_stack : The list of methods that we're evaluating method_index : The index of the method in the method stack lightning_model : the lightning model that we want to evaluate model : The model to be evaluated trainer_path : the path to the trainer object loginfo_output : This is the string that will be printed to the console when the model is being evaluated metrics_output : This is the string that will be written to the metrics file. It contains the method name, the method index, and the method stack metrics_csv : a string that will be written to a csv file Returns : The metrics_csv is being returned.","title":"eval_single_model"},{"location":"development/devel/api/#eval_model_1","text":"def eval_model(lightning_model, model) First the method stack for the evaluation ist build and then it is according to this evaluated Arguments : lightning_model : the lightning model model : the model to be evaluated","title":"eval_model"},{"location":"development/devel/api/#hannahnassearchweight_sharing","text":"","title":"hannah.nas.search.weight_sharing"},{"location":"development/devel/api/#hannahnassearchsamplerbase_sampler","text":"","title":"hannah.nas.search.sampler.base_sampler"},{"location":"development/devel/api/#sampler-objects","text":"class Sampler(ABC)","title":"Sampler Objects"},{"location":"development/devel/api/#tell_result","text":"def tell_result(parameters, metrics) Tell the result of a task","title":"tell_result"},{"location":"development/devel/api/#hannahnassearchsampleraging_evolution","text":"","title":"hannah.nas.search.sampler.aging_evolution"},{"location":"development/devel/api/#agingevolutionsampler-objects","text":"class AgingEvolutionSampler(Sampler) Aging Evolution based multi objective optimization","title":"AgingEvolutionSampler Objects"},{"location":"development/devel/api/#next_parameters","text":"def next_parameters() Returns a list of current tasks","title":"next_parameters"},{"location":"development/devel/api/#tell_result_1","text":"def tell_result(parameters, metrics) Tell the result of a task","title":"tell_result"},{"location":"development/devel/api/#hannahnassearchsamplerrandom_sampler","text":"","title":"hannah.nas.search.sampler.random_sampler"},{"location":"development/devel/api/#hannahnassearchsamplermutator","text":"","title":"hannah.nas.search.sampler.mutator"},{"location":"development/devel/api/#hannahnassearchutils","text":"","title":"hannah.nas.search.utils"},{"location":"development/devel/api/#hannahnasops","text":"","title":"hannah.nas.ops"},{"location":"development/devel/api/#hannahnasgraph_conversion","text":"","title":"hannah.nas.graph_conversion"},{"location":"development/devel/api/#hannahnasspacesdartsdarts_space","text":"","title":"hannah.nas.spaces.darts.darts_space"},{"location":"development/devel/api/#hannahnasspacesmobilenetmobilenet","text":"","title":"hannah.nas.spaces.mobilenet.mobilenet"},{"location":"development/devel/api/#hannahnasutils","text":"","title":"hannah.nas.utils"},{"location":"development/devel/api/#is_pareto","text":"def is_pareto(costs, maximise=False) Arguments : costs : An (n_points, n_costs) array Returns : A (n_points, ) boolean array, indicating whether each point is Pareto efficient","title":"is_pareto"},{"location":"development/devel/api/#hannahdatasetsvisionri_capsule","text":"Rhode island gastroenterology video capsule endoscopy dataset https://www.nature.com/articles/s41597-022-01726-3 https://github.com/acharoen/Rhode-Island-GI-VCE-Technical-Validation","title":"hannah.datasets.vision.ri_capsule"},{"location":"development/devel/api/#hannahdatasetsvisionfake","text":"","title":"hannah.datasets.vision.fake"},{"location":"development/devel/api/#hannahdatasetsvisioncifar","text":"","title":"hannah.datasets.vision.cifar"},{"location":"development/devel/api/#hannahdatasetsvisionkvasir","text":"","title":"hannah.datasets.vision.kvasir"},{"location":"development/devel/api/#hannahdatasetsvisionkvasir_unlabeled","text":"","title":"hannah.datasets.vision.kvasir_unlabeled"},{"location":"development/devel/api/#kvasircapsuleunlabeled-objects","text":"class KvasirCapsuleUnlabeled(AbstractDataset) Dataset representing unlabelled videos","title":"KvasirCapsuleUnlabeled Objects"},{"location":"development/devel/api/#hannahdatasetsvision","text":"","title":"hannah.datasets.vision"},{"location":"development/devel/api/#hannahdatasetsvisiondresden_capsule","text":"Rhode island gastroenterology video capsule endoscopy dataset https://www.nature.com/articles/s41597-022-01726-3 https://github.com/acharoen/Rhode-Island-GI-VCE-Technical-Validation","title":"hannah.datasets.vision.dresden_capsule"},{"location":"development/devel/api/#hannahdatasetsvisionutils","text":"","title":"hannah.datasets.vision.utils"},{"location":"development/devel/api/#hannahdatasetsvisionutilsnaneye","text":"","title":"hannah.datasets.vision.utils.naneye"},{"location":"development/devel/api/#read_naneye","text":"def read_naneye(data_file: Union[str, Path]) Read a naneye raw aimage and decode bayer pattern Arguments : data_file Union[str, Path] - path to the datafile Returns : np.ndarray - uint8 array of decoded image data","title":"read_naneye"},{"location":"development/devel/api/#hannahdatasetsvisionbase","text":"","title":"hannah.datasets.vision.base"},{"location":"development/devel/api/#torchvisiondatasetbase-objects","text":"class TorchvisionDatasetBase(VisionDatasetBase) Wrapper around torchvision classification datasets","title":"TorchvisionDatasetBase Objects"},{"location":"development/devel/api/#imagedatasetbase-objects","text":"class ImageDatasetBase(VisionDatasetBase)","title":"ImageDatasetBase Objects"},{"location":"development/devel/api/#__init___5","text":"def __init__(X, y, classes, bbox=None, transform=None) Initialize vision dataset Arguments : X List[str] - List of paths to image files y List[str] - Class id of corresponding image classes List[str] - List of class names, names are ordered by numeric class id bbox Dict[str] - Dict with filename as keys, bbox coordinates as numpy arrays transform Callable[image,image], optional - Optional transformation/augmentation of input images. Defaults to None.","title":"__init__"},{"location":"development/devel/api/#hannahdatasetsemergency","text":"","title":"hannah.datasets.emergency"},{"location":"development/devel/api/#emergencysirendataset-objects","text":"class EmergencySirenDataset(AbstractDataset) Emergency Dataset","title":"EmergencySirenDataset Objects"},{"location":"development/devel/api/#hannahdatasetsdownsample","text":"","title":"hannah.datasets.Downsample"},{"location":"development/devel/api/#hannahdatasetscollate","text":"","title":"hannah.datasets.collate"},{"location":"development/devel/api/#vision_collate_fn","text":"def vision_collate_fn(batch) Function that takes in a batch of data and puts the elements within the batch into a tensor with an additional outer dimension - batch size. The exact output type can be a :class: torch.Tensor , a Sequence of :class: torch.Tensor , a Collection of :class: torch.Tensor , or left unchanged, depending on the input type. This is used as the default function for collation for vision tasks batch_size or batch_sampler is defined in :class: ~torch.utils.data.DataLoader . Here is the general input type (based on the type of the element within the batch) to output type mapping: :class: torch.Tensor -> :class: torch.Tensor (with an added outer dimension batch size) NumPy Arrays -> :class: torch.Tensor float -> :class: torch.Tensor int -> :class: torch.Tensor str -> str (unchanged) bytes -> bytes (unchanged) Mapping[K, V_i] -> Mapping[K, vision_collate([V_1, V_2, ...])] NamedTuple[V1_i, V2_i, ...] -> NamedTuple[vision_collate([V1_1, V1_2, ...]), vision_collate([V2_1, V2_2, ...]), ...] Sequence[V1_i, V2_i, ...] -> Sequence[vision_collate([V1_1, V1_2, ...]), vision_collate([V2_1, V2_2, ...]), ...] Arguments : batch - a single batch to be collated Examples :","title":"vision_collate_fn"},{"location":"development/devel/api/#example-with-a-batch-of-ints","text":"vision_collate([0, 1, 2, 3]) tensor([0, 1, 2, 3])","title":"Example with a batch of ints:"},{"location":"development/devel/api/#example-with-a-batch-of-strs","text":"vision_collate(['a', 'b', 'c']) ['a', 'b', 'c']","title":"Example with a batch of strs:"},{"location":"development/devel/api/#example-with-map-inside-the-batch","text":"vision_collate([{'A': 0, 'B': 1}, {'A': 100, 'B': 100}]) - {'A' - tensor([ 0, 100]), 'B': tensor([ 1, 100])}","title":"Example with Map inside the batch:"},{"location":"development/devel/api/#example-with-namedtuple-inside-the-batch","text":"Point = namedtuple('Point', ['x', 'y']) vision_collate([Point(0, 0), Point(1, 1)]) Point(x=tensor([0, 1]), y=tensor([0, 1]))","title":"Example with NamedTuple inside the batch:"},{"location":"development/devel/api/#example-with-tuple-inside-the-batch","text":"vision_collate([(0, 1), (2, 3)]) [tensor([0, 2]), tensor([1, 3])]","title":"Example with Tuple inside the batch:"},{"location":"development/devel/api/#example-with-list-inside-the-batch","text":"vision_collate([[0, 1], [2, 3]]) [tensor([0, 2]), tensor([1, 3])]","title":"Example with List inside the batch:"},{"location":"development/devel/api/#ctc_collate_fn","text":"def ctc_collate_fn(data) Creates mini-batch tensors from the list of tuples (src_seq, trg_seq). We should build a custom collate_fn rather than using default collate_fn, because merging sequences (including padding) is not supported in default. Sequences are padded to the maximum length of mini-batch sequences (dynamic padding). Arguments : data - list of tuple (src_seq, src_length, trg_seq, trg_length). src_seq: torch tensor of shape (x,?); variable length. src length: torch tenso of shape 1x1 trg_seq: torch tensor of shape (?); variable length. trg_length: torch_tensor of shape (1x1) Returns - tuple of four torch tensors src_seqs - torch tensor of shape (batch_size, x, padded_length). src_lengths - torch_tensor of shape (batch_size); valid length for each padded source sequence. trg_seqs - torch tensor of shape (batch_size, x, padded_length). trg_lengths - torch tensor of shape (batch_size); valid length for each padded target sequence.","title":"ctc_collate_fn"},{"location":"development/devel/api/#hannahdatasetsspeech","text":"","title":"hannah.datasets.speech"},{"location":"development/devel/api/#speechdataset-objects","text":"class SpeechDataset(AbstractDataset) Base Class for speech datasets","title":"SpeechDataset Objects"},{"location":"development/devel/api/#preprocess","text":"def preprocess(example, silence=False, label=0) Run preprocessing and feature extraction","title":"preprocess"},{"location":"development/devel/api/#speechcommandsdataset-objects","text":"class SpeechCommandsDataset(SpeechDataset) This class implements reading and preprocessing of speech commands like dataset","title":"SpeechCommandsDataset Objects"},{"location":"development/devel/api/#speechhotworddataset-objects","text":"class SpeechHotwordDataset(SpeechDataset) Dataset Class for Hotword dataset e.g. Hey Snips!","title":"SpeechHotwordDataset Objects"},{"location":"development/devel/api/#splits","text":"@classmethod def splits(cls, config) Splits the dataset in training, devlopment and test set and returns the three sets as List","title":"splits"},{"location":"development/devel/api/#vaddataset-objects","text":"class VadDataset(SpeechDataset)","title":"VadDataset Objects"},{"location":"development/devel/api/#splits_1","text":"@classmethod def splits(cls, config) Splits the dataset in training, devlopment and test set and returns the three sets as List","title":"splits"},{"location":"development/devel/api/#hannahdatasetsactivity","text":"","title":"hannah.datasets.activity"},{"location":"development/devel/api/#data3d-objects","text":"class Data3D() 3D-Data","title":"Data3D Objects"},{"location":"development/devel/api/#pampap2_imudata-objects","text":"class PAMPAP2_IMUData() A IMU set defined by temperature (\u00b0C) 3D-acceleration data (ms -2 ), scale: \u00b116g, resolution: 13-bit 3D-acceleration data (ms -2 ), scale: \u00b16g, resolution: 13-bit 3D-gyroscope data (rad/s) 3D-magnetometer data (\u03bcT) orientation (invalid in this data collection)","title":"PAMPAP2_IMUData Objects"},{"location":"development/devel/api/#pamap2_datapoint-objects","text":"class PAMAP2_DataPoint() A temporal datapoint in the dataset","title":"PAMAP2_DataPoint Objects"},{"location":"development/devel/api/#pamap2_datachunk-objects","text":"class PAMAP2_DataChunk() A DataChunk is a item of the pytorch dataset","title":"PAMAP2_DataChunk Objects"},{"location":"development/devel/api/#pamap2_dataset-objects","text":"class PAMAP2_Dataset(AbstractDataset) Class for the PAMAP2 activity dataset https://archive.ics.uci.edu/ml/datasets/pamap2+physical+activity+monitoring","title":"PAMAP2_Dataset Objects"},{"location":"development/devel/api/#hannahdatasetsdirectional","text":"","title":"hannah.datasets.directional"},{"location":"development/devel/api/#directionaldataset-objects","text":"class DirectionalDataset(AbstractDataset) Directional Dataset","title":"DirectionalDataset Objects"},{"location":"development/devel/api/#hannahdatasetsdatasetsplit","text":"","title":"hannah.datasets.DatasetSplit"},{"location":"development/devel/api/#hannahdatasets","text":"","title":"hannah.datasets"},{"location":"development/devel/api/#hannahdatasetsphysio","text":"","title":"hannah.datasets.physio"},{"location":"development/devel/api/#atrialfibrillationdataset-objects","text":"class AtrialFibrillationDataset(PhysioDataset) Atrial Fibrillation Database (https://physionet.org/content/afdb/1.0.0/)","title":"AtrialFibrillationDataset Objects"},{"location":"development/devel/api/#hannahdatasetsnoisedataset","text":"","title":"hannah.datasets.NoiseDataset"},{"location":"development/devel/api/#hannahdatasetsfake1d","text":"","title":"hannah.datasets.fake1d"},{"location":"development/devel/api/#hannahdatasetsutilscache","text":"","title":"hannah.datasets.utils.cache"},{"location":"development/devel/api/#hannahdatasetsutils","text":"","title":"hannah.datasets.utils"},{"location":"development/devel/api/#hannahdatasetsutilsmd5","text":"","title":"hannah.datasets.utils.md5"},{"location":"development/devel/api/#hannahdatasetsbase","text":"","title":"hannah.datasets.base"},{"location":"development/devel/api/#datasettype-objects","text":"class DatasetType(Enum) The type of a dataset partition e.g. train, dev, test","title":"DatasetType Objects"},{"location":"development/devel/api/#abstractdataset-objects","text":"class AbstractDataset(Dataset, ABC)","title":"AbstractDataset Objects"},{"location":"development/devel/api/#prepare_4","text":"@abstractclassmethod def prepare(cls, config: Dict[str, Any]) -> None Prepare the dataset. This method is run at the beginning of the dataset training. If possible it should download the dataset from its original source, if it is available for public download. Arguments : config Dict[Any] - The dataset configuration","title":"prepare"},{"location":"development/devel/api/#splits_2","text":"@abstractclassmethod def splits( cls, config: Dict[str, Any] ) -> Tuple[\"AbstractDataset\", \"AbstractDataset\", \"AbstractDataset\"] Returns the test, validation and train split according to the Dataset config Arguments : config [type] - [description]","title":"splits"},{"location":"development/devel/api/#class_names","text":"@abstractproperty def class_names() -> List[str] Returns the names of the classes in the classification dataset","title":"class_names"},{"location":"development/devel/api/#class_counts","text":"@abstractproperty def class_counts() -> Optional[Dict[int, int]] Returns the number of items in each class of the dataset If this is not applicable to a dataset type e.g. ASR, Semantic Segementation, it may return None","title":"class_counts"},{"location":"development/devel/api/#__getitem__","text":"@abstractmethod def __getitem__(index) -> List[torch.Tensor] Returns a torch.Tensor for the data item at the corresponding index The length of the list depends on the dataset item to use Arguments : index int - the index of the data item","title":"__getitem__"},{"location":"development/devel/api/#__len__","text":"@abstractmethod def __len__() -> int Returns number of samples in dataset","title":"__len__"},{"location":"development/devel/api/#size","text":"def size() -> List[int] Returns dimension of output without batch dimension","title":"size"},{"location":"development/devel/api/#std","text":"@property def std() -> Optional[Tuple[int, ...]] Returns channel-wise standard deviation for dataset if applicable","title":"std"},{"location":"development/devel/api/#mean","text":"@property def mean() -> Optional[Tuple[int, ...]] Returns channel-wise means for dataset if applicable","title":"mean"},{"location":"development/devel/api/#resolution","text":"@property def resolution() -> Optional[Tuple[int, ...]] Returns resolution for dataset if applicable","title":"resolution"},{"location":"development/devel/api/#weights","text":"@property def weights() -> Optional[List[float]] Class weights for weighted sampling","title":"weights"},{"location":"development/devel/api/#hannahdatasetskitti","text":"","title":"hannah.datasets.Kitti"},{"location":"development/devel/api/#kitti-objects","text":"class Kitti(AbstractDataset)","title":"Kitti Objects"},{"location":"development/devel/api/#splits_3","text":"@classmethod def splits(cls, config) Splits the dataset in training, devlopment and test set and returns the three sets as List","title":"splits"},{"location":"development/devel/api/#hannahnnquantized","text":"","title":"hannah.nn.quantized"},{"location":"development/devel/api/#hannahnnqat","text":"Implementations of torch.nn.intrinsics qat with an optional quantize bias parameter. Qconfigs can support an optional bias quantization funciton which should be returned by qconfig.bias() else biases will be quantized with qconfig.activation()","title":"hannah.nn.qat"},{"location":"development/devel/api/#_convbnnd-objects","text":"class _ConvBnNd(nn.modules.conv._ConvNd, _ConvForwardMixin)","title":"_ConvBnNd Objects"},{"location":"development/devel/api/#train_3","text":"def train(mode: bool = True) -> Any Batchnorm's training behavior is using the self.training flag. Prevent changing it if BN is frozen. This makes sure that calling model.train() on a model with a frozen BN will behave properly.","title":"train"},{"location":"development/devel/api/#from_float_2","text":"@classmethod def from_float(cls, mod) Create a qat module from a float module or qparams_dict Args: mod a float module, either produced by torch.quantization utilities or directly from user","title":"from_float"},{"location":"development/devel/api/#convbn1d-objects_1","text":"class ConvBn1d(_ConvBnNd) A ConvBn1d module is a module fused from Conv1d and BatchNorm1d, attached with FakeQuantize modules for weight, used in quantization aware training. We combined the interface of :class: torch.nn.Conv1d and :class: torch.nn.BatchNorm1d . Similar to :class: torch.nn.Conv1d , with FakeQuantize modules initialized to default. Attributes : freeze_bn: - weight_fake_quant - fake quant module for weight","title":"ConvBn1d Objects"},{"location":"development/devel/api/#convbnrelu1d-objects_1","text":"class ConvBnReLU1d(ConvBn1d) A ConvBnReLU1d module is a module fused from Conv1d, BatchNorm1d and ReLU, attached with FakeQuantize modules for weight, used in quantization aware training. We combined the interface of :class: torch.nn.Conv1d and :class: torch.nn.BatchNorm1d and :class: torch.nn.ReLU . Similar to torch.nn.Conv1d , with FakeQuantize modules initialized to default. Attributes : weight_fake_quant - fake quant module for weight","title":"ConvBnReLU1d Objects"},{"location":"development/devel/api/#convbn2d-objects","text":"class ConvBn2d(_ConvBnNd) A ConvBn2d module is a module fused from Conv2d and BatchNorm2d, attached with FakeQuantize modules for weight, used in quantization aware training. We combined the interface of :class: torch.nn.Conv2d and :class: torch.nn.BatchNorm2d . Similar to :class: torch.nn.Conv2d , with FakeQuantize modules initialized to default. Attributes : freeze_bn: - weight_fake_quant - fake quant module for weight","title":"ConvBn2d Objects"},{"location":"development/devel/api/#convbnrelu2d-objects","text":"class ConvBnReLU2d(ConvBn2d) A ConvBnReLU2d module is a module fused from Conv2d, BatchNorm2d and ReLU, attached with FakeQuantize modules for weight, used in quantization aware training. We combined the interface of :class: torch.nn.Conv2d and :class: torch.nn.BatchNorm2d and :class: torch.nn.ReLU . Similar to torch.nn.Conv2d , with FakeQuantize modules initialized to default. Attributes : weight_fake_quant - fake quant module for weight","title":"ConvBnReLU2d Objects"},{"location":"development/devel/api/#convrelu2d-objects","text":"class ConvReLU2d(nn.Conv2d, _ConvForwardMixin) A ConvReLU2d module is a fused module of Conv2d and ReLU, attached with FakeQuantize modules for weight for quantization aware training. We combined the interface of :class: ~torch.nn.Conv2d and :class: ~torch.nn.BatchNorm2d . Attributes : weight_fake_quant - fake quant module for weight","title":"ConvReLU2d Objects"},{"location":"development/devel/api/#convrelu1d-objects_1","text":"class ConvReLU1d(nn.Conv1d, _ConvForwardMixin) A ConvReLU1d module is fused module of Conv1d and ReLU, attached with FakeQuantize modules for quantization aware training","title":"ConvReLU1d Objects"},{"location":"development/devel/api/#conv1d-objects","text":"class Conv1d(nn.Conv1d, _ConvForwardMixin) A Conv1d module is a Conv1d module , attached with FakeQuantize modules for weight for quantization aware training. Attributes : weight_fake_quant - fake quant module for weight bias_fake_quant - fake quant module for bias activation_post_process - fake_quant_module for activations","title":"Conv1d Objects"},{"location":"development/devel/api/#conv2d-objects","text":"class Conv2d(nn.Conv2d, _ConvForwardMixin) A Conv2d module is a Conv2d module , attached with FakeQuantize modules for weight for quantization aware training. Attributes : weight_fake_quant - fake quant module for weight bias_fake_quant - fake quant module for bias activation_post_process - fake_quant_module for activations","title":"Conv2d Objects"},{"location":"development/devel/api/#linear-objects","text":"class Linear(nn.Linear) A linear module attached with FakeQuantize modules for weight, used for quantization aware training. We adopt the same interface as torch.nn.Linear , please see https://pytorch.org/docs/stable/nn.html#torch.nn.Linear for documentation. Similar to torch.nn.Linear , with FakeQuantize modules initialized to default. Attributes : weight - fake quant module for weight","title":"Linear Objects"},{"location":"development/devel/api/#from_float_3","text":"@classmethod def from_float(cls, mod) Create a qat module from a float module or qparams_dict Args: mod a float module, either produced by torch.quantization utilities or directly from user","title":"from_float"},{"location":"development/devel/api/#linearrelu-objects","text":"class LinearReLU(nn.Linear) A linear module attached with FakeQuantize modules and ReLU for weight, used for quantization aware training. We adopt the same interface as torch.nn.Linear , please see https://pytorch.org/docs/stable/nn.html#torch.nn.Linear for documentation. Similar to torch.nn.Linear , with FakeQuantize modules initialized to default. Attributes : weight - fake quant module for weight","title":"LinearReLU Objects"},{"location":"development/devel/api/#from_float_4","text":"@classmethod def from_float(cls, mod) Create a qat module from a float module or qparams_dict Args: mod a float module, either produced by torch.quantization utilities or directly from user","title":"from_float"},{"location":"development/devel/api/#identity-objects","text":"class Identity(nn.Identity) A identity module attached with FakeQuantize modules for weight, used for quantization aware training. We adopt the same interface as torch.nn.Identity , please see https://pytorch.org/docs/stable/nn.html#torch.nn.Identity for documentation. Similar to torch.nn.Identity , with FakeQuantize modules initialized to default.","title":"Identity Objects"},{"location":"development/devel/api/#from_float_5","text":"@classmethod def from_float(cls, mod) Create a qat module from a float module or qparams_dict Args: mod a float module, either produced by torch.quantization utilities or directly from user","title":"from_float"},{"location":"development/devel/api/#hannahquantizationcallback","text":"","title":"hannah.quantization.callback"},{"location":"development/devel/api/#hannahquantizationqconfig","text":"","title":"hannah.quantization.qconfig"},{"location":"development/devel/api/#hannahquantizationrounding","text":"","title":"hannah.quantization.rounding"},{"location":"development/devel/api/#round_downward","text":"def round_downward(x: Tensor) -> Tensor Round to nearest upward","title":"round_downward"},{"location":"development/devel/api/#round_upward","text":"def round_upward(x: Tensor) -> Tensor Round to nearest downward","title":"round_upward"},{"location":"development/devel/api/#round_odd","text":"def round_odd(x: Tensor) -> Tensor Round to nearest odd","title":"round_odd"},{"location":"development/devel/api/#round_even","text":"def round_even(x: Tensor) -> Tensor Round to nearest even","title":"round_even"},{"location":"development/devel/api/#round_zero","text":"def round_zero(x: Tensor) -> Tensor Round towards zero","title":"round_zero"},{"location":"development/devel/api/#round_infinity","text":"def round_infinity(x: Tensor) -> Tensor Round toward infinity","title":"round_infinity"},{"location":"development/devel/api/#truncate_up","text":"def truncate_up(x: Tensor) -> Tensor Always round up to next integer","title":"truncate_up"},{"location":"development/devel/api/#truncate_down","text":"def truncate_down(x: Tensor) -> Tensor Always round down to next integer","title":"truncate_down"},{"location":"development/devel/api/#truncate_infinity","text":"def truncate_infinity(x: Tensor) -> Tensor Always round to next integer in direction infinity","title":"truncate_infinity"},{"location":"development/devel/api/#truncate_zero","text":"def truncate_zero(x: Tensor) -> Tensor Always round to next integer in direction of Zero","title":"truncate_zero"},{"location":"development/devel/api/#round_stochastic","text":"def round_stochastic(x: Tensor) -> Tensor Round stochastically","title":"round_stochastic"},{"location":"development/devel/api/#hannahmodulesvisionanomaly_score","text":"class AnomalyScore(CatMetric): def init (self, percentile, nan_strategy=\"warn\", kwargs): super(). init (nan_strategy=nan_strategy, kwargs) self.percentile = percentile def compute(self): anomaly_score = None train_losses = super().compute() if train_losses: normalized_train_errors = torch.stack(train_losses) / ( torch.max(torch.stack(train_losses), dim=0).values ) anomaly_score = np.percentile( normalized_train_errors.cpu().numpy(), self.percentile ) return anomaly_score","title":"hannah.modules.vision.anomaly_score"},{"location":"development/devel/api/#hannahmodulesvisionanomaly_detection","text":"","title":"hannah.modules.vision.anomaly_detection"},{"location":"development/devel/api/#anomalydetectionmodule-objects","text":"class AnomalyDetectionModule(VisionBaseModule)","title":"AnomalyDetectionModule Objects"},{"location":"development/devel/api/#on_test_end_4","text":"def on_test_end() wd_dir = os.getcwd() score, largest_train_error = self.compute_anomaly_score() train_errors = self.normalized_train_errors plt.hist(train_errors.detach().cpu().numpy(), bins=100) plt.axvline(score, linestyle=\"dashed\") plt.title(\"Normalized train reconstruction errors\") plt.savefig(wd_dir + \"/normalized_train_errors.png\") test = ( torch.tensor(self.test_losses, device=self.device) / torch.max(torch.stack(self.train_losses), dim=0).values ) plt.hist(test.detach().cpu().numpy(), bins=100) plt.title(\"Normalized test reconstruction errors\") plt.savefig(wd_dir + \"/normalized_test_errors.png\") print(\"Anomaly score\", score) print( \"Largest train error\", torch.max(torch.stack(self.train_losses), dim=0).values, )","title":"on_test_end"},{"location":"development/devel/api/#hannahmodulesvisionloss","text":"","title":"hannah.modules.vision.loss"},{"location":"development/devel/api/#hannahmodulesvision","text":"","title":"hannah.modules.vision"},{"location":"development/devel/api/#hannahmodulesvisionimage_classifier","text":"","title":"hannah.modules.vision.image_classifier"},{"location":"development/devel/api/#hannahmodulesvisionbase","text":"","title":"hannah.modules.vision.base"},{"location":"development/devel/api/#hannahmodulesclassifier","text":"","title":"hannah.modules.classifier"},{"location":"development/devel/api/#hannahmodulesobject_detection","text":"","title":"hannah.modules.object_detection"},{"location":"development/devel/api/#hannahmodulesconfig_utils","text":"","title":"hannah.modules.config_utils"},{"location":"development/devel/api/#dump_config","text":"def dump_config(output_dir, config) Dumps the configuration to json format Creates file config.json in output_dir","title":"dump_config"},{"location":"development/devel/api/#parameters_20","text":"output_dir : str Output directory config : dict Configuration to dump","title":"Parameters"},{"location":"development/devel/api/#save_model","text":"def save_model(output_dir, model) Creates serialization of the model for later inference, evaluation Creates the following files: model.pt: Serialized version of network parameters in pytorch model.json: Serialized version of network parameters in json format model.onnx: full model including paramters in onnx format","title":"save_model"},{"location":"development/devel/api/#parameters_21","text":"output_dir : str Directory to put serialized models model : LightningModule Model to serialize","title":"Parameters"},{"location":"development/devel/api/#hannahmodules","text":"","title":"hannah.modules"},{"location":"development/devel/api/#hannahmodulesangle_classifier","text":"","title":"hannah.modules.angle_classifier"},{"location":"development/devel/api/#hannahmodulesmetrics","text":"","title":"hannah.modules.metrics"},{"location":"development/devel/api/#error-objects","text":"class Error() Computes Error = 1 - Accuracy_ .. math:: \\text{Error} = 1 - \\frac{1}{N}\\sum_i^N 1(y_i = \\hat{y}_i) Where :math: y is a tensor of target values, and :math: \\hat{y} is a tensor of predictions. This module is a simple wrapper to get the task specific versions of this metric, which is done by setting the task argument to either 'binary' , 'multiclass' or multilabel . See the documentation of :mod: BinaryError , :mod: MulticlassError and :mod: MultilabelError for the specific details of each argument influence and examples.","title":"Error Objects"},{"location":"development/devel/api/#plot_confusion_matrix","text":"def plot_confusion_matrix(cf, group_names=None, categories=\"auto\", count=True, percent=True, cbar=True, xyticks=True, xyplotlabels=True, sum_stats=True, figsize=None, cmap=\"Blues\", title=None) This function will make a pretty plot of an sklearn Confusion Matrix cm using a Seaborn heatmap visualization.","title":"plot_confusion_matrix"},{"location":"development/devel/api/#arguments","text":"cf: confusion matrix to be passed in group_names: List of strings that represent the labels row by row to be shown in each square. categories: List of strings containing the categories to be displayed on the x,y axis. Default is 'auto' count: If True, show the raw number in the confusion matrix. Default is True. normalize: If True, show the proportions for each category. Default is True. cbar: If True, show the color bar. The cbar values are based off the values in the confusion matrix. Default is True. xyticks: If True, show x and y ticks. Default is True. xyplotlabels: If True, show 'True Label' and 'Predicted Label' on the figure. Default is True. sum_stats: If True, display summary statistics below the figure. Default is True. figsize: Tuple representing the figure size. Default will be the matplotlib rcParams value. cmap: Colormap of the values displayed from matplotlib.pyplot.cm. Default is 'Blues' See http://matplotlib.org/examples/color/colormaps_reference.html title: Title for the heatmap. Default is None.","title":"Arguments"},{"location":"development/devel/api/#hannahmodulesbase","text":"","title":"hannah.modules.base"},{"location":"development/devel/api/#classifiermodule-objects","text":"class ClassifierModule(LightningModule, ABC)","title":"ClassifierModule Objects"},{"location":"development/devel/api/#total_training_steps","text":"def total_training_steps() -> int Total training steps inferred from datamodule and devices.","title":"total_training_steps"},{"location":"development/devel/api/#hannahmodulesaugmentationaugmentation","text":"","title":"hannah.modules.augmentation.augmentation"},{"location":"development/devel/api/#hannahmodulesaugmentation","text":"","title":"hannah.modules.augmentation"},{"location":"development/devel/api/#hannahmodulesaugmentationbatch_augmentation","text":"","title":"hannah.modules.augmentation.batch_augmentation"},{"location":"development/devel/api/#batchaugmentationpipeline-objects","text":"class BatchAugmentationPipeline(nn.Module)","title":"BatchAugmentationPipeline Objects"},{"location":"development/devel/api/#__init___6","text":"def __init__(transforms={}) Augmentation pipeline especially for self supervised learning Arguments : replica int - number of replicated different random augmentations transforms dict - configuration of transforms","title":"__init__"},{"location":"development/devel/api/#forward_69","text":"@torch.no_grad() def forward(x) -> torch.Tensor Perform Augmentations Arguments : x torch.Tensor - a torch.Tensor representing the augementation pipeline Returns : Tuple[torch.Tensor, torch.Tensor]; Batch augmented with replica different random augmentations","title":"forward"},{"location":"development/devel/api/#hannahmodulesaugmentationtransformsregistry","text":"","title":"hannah.modules.augmentation.transforms.registry"},{"location":"development/devel/api/#hannahmodulesaugmentationtransformskornia_transforms","text":"","title":"hannah.modules.augmentation.transforms.kornia_transforms"},{"location":"development/devel/api/#hannahmodulesaugmentationtransforms","text":"","title":"hannah.modules.augmentation.transforms"},{"location":"development/devel/api/#hannahmodulesaugmentationbordersearch","text":"","title":"hannah.modules.augmentation.bordersearch"},{"location":"development/devel/api/#hannahutilsimports","text":"","title":"hannah.utils.imports"},{"location":"development/devel/api/#lazy_import","text":"def lazy_import(module_name, callback=None) Returns a proxy module object that will lazily import the given module the first time it is used. Example usage::","title":"lazy_import"},{"location":"development/devel/api/#lazy-version-of-import-tensorflow-as-tf","text":"tf = lazy_import(\"tensorflow\")","title":"Lazy version of import tensorflow as tf"},{"location":"development/devel/api/#other-commands","text":"","title":"Other commands"},{"location":"development/devel/api/#now-the-module-is-loaded","text":"tf. version Arguments : module_name - the fully-qualified module name to import callback None - a callback function to call before importing the module Returns : a proxy module object that will be lazily imported when first used","title":"Now the module is loaded"},{"location":"development/devel/api/#lazymodule-objects","text":"class LazyModule(types.ModuleType) Proxy module that lazily imports the underlying module the first time it is actually used. Arguments : module_name - the fully-qualified module name to import callback None - a callback function to call before importing the module","title":"LazyModule Objects"},{"location":"development/devel/api/#hannahutils","text":"","title":"hannah.utils"},{"location":"development/devel/api/#hannahutilsutils","text":"","title":"hannah.utils.utils"},{"location":"development/devel/api/#log_execution_env_state","text":"def log_execution_env_state() -> None Log information about the execution environment. File 'config_path' will be copied to directory 'logdir'. A common use-case is passing the path to a (compression) schedule YAML file. Storing a copy of the schedule file, with the experiment logs, is useful in order to reproduce experiments. Arguments : config_path - path to config file, used only when logdir is set logdir - log directory git_root - the path to the .git root directory","title":"log_execution_env_state"},{"location":"development/devel/api/#extract_from_download_cache","text":"def extract_from_download_cache(filename, url, cached_files, target_cache, target_folder, target_test_folder=\"\", clear_download=False, no_exist_check=False) -> None extracts given file from cache or donwloads first from url Arguments : filename str - name of the file to download or extract url str - possible url to download the file cached_files (list(str)): cached files in download cache target_cache str - path to the folder to cache file if download necessary target_folder str - path where to extract file target_test_folder str, optional - folder to check if data are already there clear_download bool - clear download after usage no_exist_check bool - disables the check if folder exists","title":"extract_from_download_cache"},{"location":"development/devel/api/#fullname","text":"def fullname(o) -> Any Get the full classname of an object including surrounding packages/modules/namespaces","title":"fullname"},{"location":"development/devel/api/#set_deterministic","text":"@contextmanager def set_deterministic(mode, warn_only=False) A contextmanager to set deterministic algorithms","title":"set_deterministic"},{"location":"development/devel/api/#hannahvisualization","text":"","title":"hannah.visualization"},{"location":"development/devel/debugging/","text":"Debugging Hannah supports several debugging tools. Trainer Debugging flags: Most of the lightning trainer debug flags are usable through the hydra configs using trainer.flag name The most important ones are: trainer.fast_dev_run=True This provides a kind of unit test for the model by running one single batch of training, validation and test on the model. trainer.overfit_batches=0.01 This tests if the model can overfit a small percentage (0.01) of the dataset. A good model should be able reach almost 100 percent accuracy very quickly. trainer.limit_*_batches This limits the amount of batches used for training. e.g.: hannah-train trainer.limit_train_batches=0.1 trainer.limit_val_batches=0.1 trainer.limit_test_batches=0.1 Limits the training to 10 percent of available batches for train, validation and test.","title":"Debugging"},{"location":"development/devel/debugging/#debugging","text":"Hannah supports several debugging tools.","title":"Debugging"},{"location":"development/devel/debugging/#trainer-debugging-flags","text":"Most of the lightning trainer debug flags are usable through the hydra configs using trainer.flag name The most important ones are:","title":"Trainer Debugging flags:"},{"location":"development/devel/debugging/#trainerfast_dev_runtrue","text":"This provides a kind of unit test for the model by running one single batch of training, validation and test on the model.","title":"trainer.fast_dev_run=True"},{"location":"development/devel/debugging/#traineroverfit_batches001","text":"This tests if the model can overfit a small percentage (0.01) of the dataset. A good model should be able reach almost 100 percent accuracy very quickly.","title":"trainer.overfit_batches=0.01"},{"location":"development/devel/debugging/#trainerlimit__batches","text":"This limits the amount of batches used for training. e.g.: hannah-train trainer.limit_train_batches=0.1 trainer.limit_val_batches=0.1 trainer.limit_test_batches=0.1 Limits the training to 10 percent of available batches for train, validation and test.","title":"trainer.limit_*_batches"},{"location":"development/devel/overview/","text":"Development Guidelines The main development repository is available at: https://es-git.cs.uni-tuebingen.de/es/ai/hannah/hannah Code Style This project uses pre commit hooks for auto formatting and static code analysis. To enable pre commit hooks run the following command in a poetry shell . pre-commit install Naming Conventions Try to follow (pep8)[https://pep8.org/#naming-conventions] and the rest of pep8 to the best of your abilities. The short summary of pep8 is: Use all lowercase names with underscores for methods/functions/modules/packages. Use CamelCase for Classes and Type Variables Use all upperscore with underscores for constants Logging / Console Messages Try to avoid print for doing outputs whenever possible. Use python module (logging)[https://docs.python.org/3.8/howto/logging.html] instead. Default logglevel for informational messages should be logging.info() . If you want to be able to have more finegrained control, use a dedicated logger for your module e.g.: logger = logging.getLogger('datasets.speech') ... logger.info() logger.debug() Testing / CI Try to provide unit tests for new features, unit tests should be written using pytest . Unit test should be designed to run pretty fast and are run for each push. Tests marked with `@pytest.mark.integration are only triggered when --integration is given on the pytest commandline. These tests are only run on pushs to the main branch. Resolving merge conflicts in poetry.lock If you have changed poetry.lock this can result in merge conflicts. The easiest way to resolve them is: git checkout --theirs poetry.lock poetry lock --no-update Try to avoid running poetry update on feature branches. Branching We apply the following code branch naming convention. main : main development branch should be as stable as possible f/* : feature branches, development branches (will be squash merged after review, and successful passing of merge request pipeline) fix/* bug fix branches, branches containing little self contained bug fixes for existing features (feel free to merge without code review, if you can reasonably assume that they are not breaking anything that is not already broken) pub/* : publication branches, branches containing the version corresponding to a specific, full branch name should contain abbreviation of publication and publication year. Publication branches and main branch are automatically mirrored to the public github repository: https://github.com/ekut-es/hannah","title":"Overview"},{"location":"development/devel/overview/#development-guidelines","text":"The main development repository is available at: https://es-git.cs.uni-tuebingen.de/es/ai/hannah/hannah","title":"Development Guidelines"},{"location":"development/devel/overview/#code-style","text":"This project uses pre commit hooks for auto formatting and static code analysis. To enable pre commit hooks run the following command in a poetry shell . pre-commit install","title":"Code Style"},{"location":"development/devel/overview/#naming-conventions","text":"Try to follow (pep8)[https://pep8.org/#naming-conventions] and the rest of pep8 to the best of your abilities. The short summary of pep8 is: Use all lowercase names with underscores for methods/functions/modules/packages. Use CamelCase for Classes and Type Variables Use all upperscore with underscores for constants","title":"Naming Conventions"},{"location":"development/devel/overview/#logging-console-messages","text":"Try to avoid print for doing outputs whenever possible. Use python module (logging)[https://docs.python.org/3.8/howto/logging.html] instead. Default logglevel for informational messages should be logging.info() . If you want to be able to have more finegrained control, use a dedicated logger for your module e.g.: logger = logging.getLogger('datasets.speech') ... logger.info() logger.debug()","title":"Logging / Console Messages"},{"location":"development/devel/overview/#testing-ci","text":"Try to provide unit tests for new features, unit tests should be written using pytest . Unit test should be designed to run pretty fast and are run for each push. Tests marked with `@pytest.mark.integration are only triggered when --integration is given on the pytest commandline. These tests are only run on pushs to the main branch.","title":"Testing / CI"},{"location":"development/devel/overview/#resolving-merge-conflicts-in-poetrylock","text":"If you have changed poetry.lock this can result in merge conflicts. The easiest way to resolve them is: git checkout --theirs poetry.lock poetry lock --no-update Try to avoid running poetry update on feature branches.","title":"Resolving merge conflicts in poetry.lock"},{"location":"development/devel/overview/#branching","text":"We apply the following code branch naming convention. main : main development branch should be as stable as possible f/* : feature branches, development branches (will be squash merged after review, and successful passing of merge request pipeline) fix/* bug fix branches, branches containing little self contained bug fixes for existing features (feel free to merge without code review, if you can reasonably assume that they are not breaking anything that is not already broken) pub/* : publication branches, branches containing the version corresponding to a specific, full branch name should contain abbreviation of publication and publication year. Publication branches and main branch are automatically mirrored to the public github repository: https://github.com/ekut-es/hannah","title":"Branching"},{"location":"development/devel/profiling/","text":"Profiling Tools To debug training inefficiencies we support the following profiling tools: pytorch autograd profiler: hannah-train profiler=simple trainer.max_epochs=1 profiler=pytorch By default profiles are logged to the trainings output directory and can be visualized using tensorboard. For the example run use: tensorboard --logdir trained_models/test/tc-res8 Simple Profiler: hannah-train profiler=simple trainer.max_epochs=1 profiler=simple Generates a summary of the durations of the individual training steps: Action | Mean duration (s) |Num calls | Total time (s) | Percentage % | ------------------------------------------------------------------------------------------------------------------------------------ Total | - |_ | 29.42 | 100 % | ------------------------------------------------------------------------------------------------------------------------------------ run_training_epoch | 21.692 |1 | 21.692 | 73.733 | run_training_batch | 0.030893 |288 | 8.8971 | 30.241 | optimizer_step_and_closure_0 | 0.030348 |288 | 8.7402 | 29.708 | get_train_batch | 0.022253 |288 | 6.4087 | 21.783 | evaluation_step_and_end | 0.021021 |277 | 5.8228 | 19.792 | validation_step | 0.020913 |277 | 5.7929 | 19.69 | training_step_and_backward | 0.019199 |288 | 5.5292 | 18.794 | model_forward | 0.010312 |288 | 2.9697 | 10.094 | training_step | 0.010076 |288 | 2.9017 | 9.8631 | model_backward | 0.0076062 |288 | 2.1906 | 7.4458 | on_validation_epoch_end | 0.091247 |2 | 0.18249 | 0.6203 | on_train_batch_end | 0.0003854 |288 | 0.111 | 0.37728 | on_train_start | 0.09102 |1 | 0.09102 | 0.30938 | on_train_epoch_end | 0.055033 |1 | 0.055033 | 0.18706 | cache_result | 1.6399e-05 |2290 | 0.037554 | 0.12765 | on_validation_batch_end | 9.3069e-05 |277 | 0.02578 | 0.087628 | on_validation_end | 0.008392 |2 | 0.016784 | 0.05705 | on_train_end | 0.0075968 |1 | 0.0075968 | 0.025822 | on_batch_start | 2.4213e-05 |288 | 0.0069733 | 0.023702 | on_after_backward | 1.5198e-05 |288 | 0.004377 | 0.014877 | on_before_zero_grad | 1.4302e-05 |288 | 0.0041189 | 0.014 | on_train_batch_start | 1.264e-05 |288 | 0.0036403 | 0.012374 | on_batch_end | 1.1853e-05 |288 | 0.0034136 | 0.011603 | training_step_end | 9.123e-06 |288 | 0.0026274 | 0.0089307 | on_validation_batch_start | 8.5754e-06 |277 | 0.0023754 | 0.008074 | validation_step_end | 7.5117e-06 |277 | 0.0020807 | 0.0070725 | on_validation_start | 0.00019774 |2 | 0.00039547 | 0.0013442 | on_epoch_start | 0.00021634 |1 | 0.00021634 | 0.00073533 | on_epoch_end | 1.4326e-05 |3 | 4.2978e-05 | 0.00014608 | on_validation_epoch_start | 1.0761e-05 |2 | 2.1523e-05 | 7.3157e-05 | on_fit_start | 2.0443e-05 |1 | 2.0443e-05 | 6.9486e-05 | on_train_epoch_start | 1.2379e-05 |1 | 1.2379e-05 | 4.2077e-05 | on_before_accelerator_backend_setup | 1.1862e-05 |1 | 1.1862e-05 | 4.0319e-05 | Advanced Profiler hannah-train profiler=simple trainer.max_epochs=1 profiler=advanced Creates python level performance summaries for the individual Training steps: Profile stats for: optimizer_step_and_closure_0 8064 function calls in 8.897 seconds Ordered by: cumulative time List reduced from 25 to 20 due to restriction <20> ncalls tottime percall cumtime percall filename:lineno(function) 288 0.001 0.000 8.897 0.031 accelerator.py:265(optimizer_step) 288 0.000 0.000 8.896 0.031 accelerator.py:282(run_optimizer_step) 288 0.000 0.000 8.896 0.031 training_type_plugin.py:159(optimizer_step) 288 0.000 0.000 8.896 0.031 grad_mode.py:12(decorate_context) 288 0.000 0.000 8.894 0.031 adamw.py:54(step) 288 0.000 0.000 8.893 0.031 training_loop.py:648(train_step_and_backward_closure) 288 0.000 0.000 8.893 0.031 training_loop.py:737(training_step_and_backward) 288 0.000 0.000 8.892 0.031 contextlib.py:108(__enter__) 288 0.000 0.000 8.892 0.031 {built-in method builtins.next} 288 0.000 0.000 8.892 0.031 profilers.py:62(profile) 288 0.000 0.000 8.892 0.031 profilers.py:250(start) 288 8.892 0.031 8.892 0.031 {method 'enable' of '_lsprof.Profiler' objects} 288 0.000 0.000 0.001 0.000 grad_mode.py:65(__enter__) 288 0.000 0.000 0.000 0.000 accelerator.py:104(lightning_module) 288 0.000 0.000 0.000 0.000 contextlib.py:238(helper) 288 0.000 0.000 0.000 0.000 training_type_plugin.py:91(lightning_module) 288 0.000 0.000 0.000 0.000 grad_mode.py:104(__enter__) 288 0.000 0.000 0.000 0.000 contextlib.py:82(__init__) 576 0.000 0.000 0.000 0.000 {built-in method torch._C.is_grad_enabled} 288 0.000 0.000 0.000 0.000 base.py:82(unwrap_lightning_module)","title":"Profiling"},{"location":"development/devel/profiling/#profiling-tools","text":"To debug training inefficiencies we support the following profiling tools:","title":"Profiling Tools"},{"location":"development/devel/profiling/#pytorch-autograd-profiler","text":"hannah-train profiler=simple trainer.max_epochs=1 profiler=pytorch By default profiles are logged to the trainings output directory and can be visualized using tensorboard. For the example run use: tensorboard --logdir trained_models/test/tc-res8","title":"pytorch autograd profiler:"},{"location":"development/devel/profiling/#simple-profiler","text":"hannah-train profiler=simple trainer.max_epochs=1 profiler=simple Generates a summary of the durations of the individual training steps: Action | Mean duration (s) |Num calls | Total time (s) | Percentage % | ------------------------------------------------------------------------------------------------------------------------------------ Total | - |_ | 29.42 | 100 % | ------------------------------------------------------------------------------------------------------------------------------------ run_training_epoch | 21.692 |1 | 21.692 | 73.733 | run_training_batch | 0.030893 |288 | 8.8971 | 30.241 | optimizer_step_and_closure_0 | 0.030348 |288 | 8.7402 | 29.708 | get_train_batch | 0.022253 |288 | 6.4087 | 21.783 | evaluation_step_and_end | 0.021021 |277 | 5.8228 | 19.792 | validation_step | 0.020913 |277 | 5.7929 | 19.69 | training_step_and_backward | 0.019199 |288 | 5.5292 | 18.794 | model_forward | 0.010312 |288 | 2.9697 | 10.094 | training_step | 0.010076 |288 | 2.9017 | 9.8631 | model_backward | 0.0076062 |288 | 2.1906 | 7.4458 | on_validation_epoch_end | 0.091247 |2 | 0.18249 | 0.6203 | on_train_batch_end | 0.0003854 |288 | 0.111 | 0.37728 | on_train_start | 0.09102 |1 | 0.09102 | 0.30938 | on_train_epoch_end | 0.055033 |1 | 0.055033 | 0.18706 | cache_result | 1.6399e-05 |2290 | 0.037554 | 0.12765 | on_validation_batch_end | 9.3069e-05 |277 | 0.02578 | 0.087628 | on_validation_end | 0.008392 |2 | 0.016784 | 0.05705 | on_train_end | 0.0075968 |1 | 0.0075968 | 0.025822 | on_batch_start | 2.4213e-05 |288 | 0.0069733 | 0.023702 | on_after_backward | 1.5198e-05 |288 | 0.004377 | 0.014877 | on_before_zero_grad | 1.4302e-05 |288 | 0.0041189 | 0.014 | on_train_batch_start | 1.264e-05 |288 | 0.0036403 | 0.012374 | on_batch_end | 1.1853e-05 |288 | 0.0034136 | 0.011603 | training_step_end | 9.123e-06 |288 | 0.0026274 | 0.0089307 | on_validation_batch_start | 8.5754e-06 |277 | 0.0023754 | 0.008074 | validation_step_end | 7.5117e-06 |277 | 0.0020807 | 0.0070725 | on_validation_start | 0.00019774 |2 | 0.00039547 | 0.0013442 | on_epoch_start | 0.00021634 |1 | 0.00021634 | 0.00073533 | on_epoch_end | 1.4326e-05 |3 | 4.2978e-05 | 0.00014608 | on_validation_epoch_start | 1.0761e-05 |2 | 2.1523e-05 | 7.3157e-05 | on_fit_start | 2.0443e-05 |1 | 2.0443e-05 | 6.9486e-05 | on_train_epoch_start | 1.2379e-05 |1 | 1.2379e-05 | 4.2077e-05 | on_before_accelerator_backend_setup | 1.1862e-05 |1 | 1.1862e-05 | 4.0319e-05 |","title":"Simple Profiler:"},{"location":"development/devel/profiling/#advanced-profiler","text":"hannah-train profiler=simple trainer.max_epochs=1 profiler=advanced Creates python level performance summaries for the individual Training steps: Profile stats for: optimizer_step_and_closure_0 8064 function calls in 8.897 seconds Ordered by: cumulative time List reduced from 25 to 20 due to restriction <20> ncalls tottime percall cumtime percall filename:lineno(function) 288 0.001 0.000 8.897 0.031 accelerator.py:265(optimizer_step) 288 0.000 0.000 8.896 0.031 accelerator.py:282(run_optimizer_step) 288 0.000 0.000 8.896 0.031 training_type_plugin.py:159(optimizer_step) 288 0.000 0.000 8.896 0.031 grad_mode.py:12(decorate_context) 288 0.000 0.000 8.894 0.031 adamw.py:54(step) 288 0.000 0.000 8.893 0.031 training_loop.py:648(train_step_and_backward_closure) 288 0.000 0.000 8.893 0.031 training_loop.py:737(training_step_and_backward) 288 0.000 0.000 8.892 0.031 contextlib.py:108(__enter__) 288 0.000 0.000 8.892 0.031 {built-in method builtins.next} 288 0.000 0.000 8.892 0.031 profilers.py:62(profile) 288 0.000 0.000 8.892 0.031 profilers.py:250(start) 288 8.892 0.031 8.892 0.031 {method 'enable' of '_lsprof.Profiler' objects} 288 0.000 0.000 0.001 0.000 grad_mode.py:65(__enter__) 288 0.000 0.000 0.000 0.000 accelerator.py:104(lightning_module) 288 0.000 0.000 0.000 0.000 contextlib.py:238(helper) 288 0.000 0.000 0.000 0.000 training_type_plugin.py:91(lightning_module) 288 0.000 0.000 0.000 0.000 grad_mode.py:104(__enter__) 288 0.000 0.000 0.000 0.000 contextlib.py:82(__init__) 576 0.000 0.000 0.000 0.000 {built-in method torch._C.is_grad_enabled} 288 0.000 0.000 0.000 0.000 base.py:82(unwrap_lightning_module)","title":"Advanced Profiler"},{"location":"evaluation/eval/","text":"Evaluation mode Evaluation of trained models can be run using hannah-eval checkpints=[]","title":"Evaluation"},{"location":"evaluation/eval/#evaluation-mode","text":"Evaluation of trained models can be run using hannah-eval checkpints=[]","title":"Evaluation mode"},{"location":"optimization/optimization/hyperparameter/","text":"Hyperparameter Search The hydra based configuration management allows using multiple hydra sweeper plugins for hyperparameter optimization. The optuna sweeper is installed by default. Sweeper plugins support specifying the hyperparameter search space on the commandline. For example the following command optimizes the learning rate and weight decay parameters for the default optimizer ( adamw ): hannah-train scheduler.max_lr='interval(0.00001, 0.001)' optimizer.weight_decay='interval(0.00001, 0.001)' hydra/sweeper=optuna hydra/launcher=joblib -m Parametrization can be given on the commandline as well as using configuration files for a detailed documentation have a look at basic override syntax and extended override syntax . Optuna Options The optuna hyperparameter optimizer has the following options. hannah-train hydra/sweeper=optuna --cfg hydra -p hydra.sweeper # @package hydra.sweeper sampler: _target_: optuna.samplers.TPESampler seed: null consider_prior: true prior_weight: 1.0 consider_magic_clip: true consider_endpoints: false n_startup_trials: 10 n_ei_candidates: 24 multivariate: false warn_independent_sampling: true _target_: hydra_plugins.hydra_optuna_sweeper.optuna_sweeper.OptunaSweeper direction: minimize storage: null study_name: null n_trials: 20 n_jobs: 2 search_space: {} Try especially increasing the number of trials to run n_trials , and if the sweeper options are dependent on each other try enabling multivariate sampling e.g. sampler.multivariate . State Persistence If you want to save the optimization results for later analyis you can save them to a relational or redis database by setting the study_name and storage options. e.g.: hannah-train experiment_id=\"test_optuna_resume\" hydra/sweeper=optuna hydra.sweeper.study_name='${experiment_id}' hydra.sweeper.storage='sqlite:///${experiment_id}.sqlite' trainer.max_epochs=15 scheduler.max_lr='interval(0.0001, 0.1)' module.num_workers=4 -m The tuning results can then be visualized using the tools from optuna.visualize : A simple example would be: import optuna storage = \"sqlite:///test_optuna_resume.sqlite\" study_name = \"test_optuna_resume\" def main(): study = optuna.load_study(study_name, storage) fig = optuna.visualization.plot_optimization_history(study) fig.write_image(\"history.png\") fig.write_image(\"history.pdf\") if __name__ == '__main__': main() For a slightly extended visualization script have a look at scripts/optuna_visualize_results.py or use optuna-dashboard for a web based visualization. Resuming Trials An additional benefit of exporting the study state to database is that trials can be somewhat resumed at a later time by just restarting the trial with the same parameters. Although this has a view problems: - The hydra sweeper will start it's job numbering from scratch Running on ml-cloud It is possible to use the sqlite backend to store trials on ml-cloud for an example look at scripts/ml_cloud_hyperopt.sh but sqlite might be somewhat unstabble when running multiple workers.","title":"Hyperparameter Optimization"},{"location":"optimization/optimization/hyperparameter/#hyperparameter-search","text":"The hydra based configuration management allows using multiple hydra sweeper plugins for hyperparameter optimization. The optuna sweeper is installed by default. Sweeper plugins support specifying the hyperparameter search space on the commandline. For example the following command optimizes the learning rate and weight decay parameters for the default optimizer ( adamw ): hannah-train scheduler.max_lr='interval(0.00001, 0.001)' optimizer.weight_decay='interval(0.00001, 0.001)' hydra/sweeper=optuna hydra/launcher=joblib -m Parametrization can be given on the commandline as well as using configuration files for a detailed documentation have a look at basic override syntax and extended override syntax .","title":"Hyperparameter Search"},{"location":"optimization/optimization/hyperparameter/#optuna-options","text":"The optuna hyperparameter optimizer has the following options. hannah-train hydra/sweeper=optuna --cfg hydra -p hydra.sweeper # @package hydra.sweeper sampler: _target_: optuna.samplers.TPESampler seed: null consider_prior: true prior_weight: 1.0 consider_magic_clip: true consider_endpoints: false n_startup_trials: 10 n_ei_candidates: 24 multivariate: false warn_independent_sampling: true _target_: hydra_plugins.hydra_optuna_sweeper.optuna_sweeper.OptunaSweeper direction: minimize storage: null study_name: null n_trials: 20 n_jobs: 2 search_space: {} Try especially increasing the number of trials to run n_trials , and if the sweeper options are dependent on each other try enabling multivariate sampling e.g. sampler.multivariate .","title":"Optuna Options"},{"location":"optimization/optimization/hyperparameter/#state-persistence","text":"If you want to save the optimization results for later analyis you can save them to a relational or redis database by setting the study_name and storage options. e.g.: hannah-train experiment_id=\"test_optuna_resume\" hydra/sweeper=optuna hydra.sweeper.study_name='${experiment_id}' hydra.sweeper.storage='sqlite:///${experiment_id}.sqlite' trainer.max_epochs=15 scheduler.max_lr='interval(0.0001, 0.1)' module.num_workers=4 -m The tuning results can then be visualized using the tools from optuna.visualize : A simple example would be: import optuna storage = \"sqlite:///test_optuna_resume.sqlite\" study_name = \"test_optuna_resume\" def main(): study = optuna.load_study(study_name, storage) fig = optuna.visualization.plot_optimization_history(study) fig.write_image(\"history.png\") fig.write_image(\"history.pdf\") if __name__ == '__main__': main() For a slightly extended visualization script have a look at scripts/optuna_visualize_results.py or use optuna-dashboard for a web based visualization.","title":"State Persistence"},{"location":"optimization/optimization/hyperparameter/#resuming-trials","text":"An additional benefit of exporting the study state to database is that trials can be somewhat resumed at a later time by just restarting the trial with the same parameters. Although this has a view problems: - The hydra sweeper will start it's job numbering from scratch","title":"Resuming Trials"},{"location":"optimization/optimization/hyperparameter/#running-on-ml-cloud","text":"It is possible to use the sqlite backend to store trials on ml-cloud for an example look at scripts/ml_cloud_hyperopt.sh but sqlite might be somewhat unstabble when running multiple workers.","title":"Running on ml-cloud"},{"location":"optimization/optimization/nas/","text":":warning: Note: We are currently in the process of restructuring the Neural Architecture Search, in particular the way search spaces are constructed. Some things might not work as usual. Neural architecture search In contrast to hyperparameter optimization neural architecture search, explores new neural network hyperparameters. A aging evolution based neural architecture search has been implemented as a hydra plugin: hannah-train --config-name config_unas To launch multiple configuration jobs in parallel use joblib launcher: hannah-train --config-name config_unas hydra/launcher=joblib Parametrization for neural architecture search need to be given as YAML configuration files at the moment. For an example see: speech_recognition/conf/config_unas.yaml Parametrization The Parametrization contains the following elements: Choice Parameter Choice Parameters select options from a list of parameters. They are configured as a list of options in the parameters. Example: conv_size: [1,3,5,7,9,11] Choice List Parameters Choice List Parameters represent a variable length list of Choices. They are configured with the follwing parameters: min Minimum length of list max Maximum length of list+1 choices List of Choices Example: min: 4 max: 10 choices: - _target_: \"torch.nn.Conv2d\" size: 3 - _target : \"torch.nn.MaxPool2d\" size: 7 Warning : Mutations for intervall parameters currently always sample randomly from the range of values Intervall Parameters Intervall Parameters represent a Scalar Value from an intervall of Values They are configure with the following parameters: lower lower bound of intervall [lower, upper[ upper upper bound of intervall [lower, upper[ int set to true to generate integers log set to true to generate log scaled distribution Subset Parameter Subset Parameters select a subset of a list of choices. They are configured using the following parameters: choices List of choices to sample from size size of the subset to generate Partition Parameter Partition parameters split the list of choices into a predefined number of partitions. They are configured using the following parameters: choices List of choices to partition partition Number of partitions to generate","title":"Neural Architecture Search"},{"location":"optimization/optimization/nas/#neural-architecture-search","text":"In contrast to hyperparameter optimization neural architecture search, explores new neural network hyperparameters. A aging evolution based neural architecture search has been implemented as a hydra plugin: hannah-train --config-name config_unas To launch multiple configuration jobs in parallel use joblib launcher: hannah-train --config-name config_unas hydra/launcher=joblib Parametrization for neural architecture search need to be given as YAML configuration files at the moment. For an example see: speech_recognition/conf/config_unas.yaml","title":"Neural architecture search"},{"location":"optimization/optimization/nas/#parametrization","text":"The Parametrization contains the following elements:","title":"Parametrization"},{"location":"optimization/optimization/nas/#choice-parameter","text":"Choice Parameters select options from a list of parameters. They are configured as a list of options in the parameters. Example: conv_size: [1,3,5,7,9,11]","title":"Choice Parameter"},{"location":"optimization/optimization/nas/#choice-list-parameters","text":"Choice List Parameters represent a variable length list of Choices. They are configured with the follwing parameters: min Minimum length of list max Maximum length of list+1 choices List of Choices Example: min: 4 max: 10 choices: - _target_: \"torch.nn.Conv2d\" size: 3 - _target : \"torch.nn.MaxPool2d\" size: 7 Warning : Mutations for intervall parameters currently always sample randomly from the range of values","title":"Choice List Parameters"},{"location":"optimization/optimization/nas/#intervall-parameters","text":"Intervall Parameters represent a Scalar Value from an intervall of Values They are configure with the following parameters: lower lower bound of intervall [lower, upper[ upper upper bound of intervall [lower, upper[ int set to true to generate integers log set to true to generate log scaled distribution","title":"Intervall Parameters"},{"location":"optimization/optimization/nas/#subset-parameter","text":"Subset Parameters select a subset of a list of choices. They are configured using the following parameters: choices List of choices to sample from size size of the subset to generate","title":"Subset Parameter"},{"location":"optimization/optimization/nas/#partition-parameter","text":"Partition parameters split the list of choices into a predefined number of partitions. They are configured using the following parameters: choices List of choices to partition partition Number of partitions to generate","title":"Partition Parameter"}]} \ No newline at end of file +{"config":{"indexing":"full","lang":["en"],"min_search_length":3,"prebuild_index":false,"separator":"[\\s\\-]+"},"docs":[{"location":"","text":"HANNAH - Hardware Accelerator and Neural network searcH Getting Started Installing dependencies Dependencies and virtual environments are managed using poetry . python (>=3.8 <3.11) and development headers libsndfile and development headers libsox and development headers a blas implementation and development headers git-lfs for management of checkpoints Ubuntu 20.04+ Install dependencies: sudo apt update sudo apt -y install python3-dev libblas-dev liblapack-dev libsndfile1-dev libsox-dev git-lfs Centos / RHEL / Scientific Linux: 7+ Install dependencies: sudo yum install portaudio-devel libsndfile1-devel libsox-devel -y Install a python 3.9 or python 3.10 version using pyenv . Install poetry curl -sSL https://install.python-poetry.org/ | python3 For alternative installation methods see: https://python-poetry.org/docs/#installation Caution : this usually installs poetry to ~/.local/bin if this folder is not in your path you might need to run poetry as: ~/.local/bin/poetry Software installation In the root directory of the project run: git submodule update --init --recursive poetry install -E vision This creates a virtual environment under ~/.cache/pypoetry/virtualenvs. The environment can be activated using: poetry shell Optional Dependencies We support installation of optional dependencies using poetry's -E commandline flag We currently have the following optional dependencies: Vision-Models Vision models require additional dependencies, these can be installed using: poetry install -E vision These dependencies include kornia and albumentations for image augmentations and timm (torch image models) for baseline neural network models. Onnx-Runtime Backend poetry install -E onnxrt-backend Backend support for running models on onnx-runtime. Tflite-Backend poetry install -E onnx-tf Onnx based conversion of trained models to tensorflow/tensorflow-lite for external inference backends. DVC based experiment management (experimental) poetry install -E dvc This installs dvc based model, data and experiment management. DVC support is highly experimental and subject to change. Installation Tips 1.) venv location poetry installs the dependencies to a virtual environment in ~/.cache/pypoetry/virtualenvs You can change the location of this directory using: poetry config virtualenvs.path Or move it to a subdirectory of the project directory using: poetry config virtualenvs.in-project true Installing the datasets Datasets are downloaded automatically to the datasets data folder by default this is a subfolder of the dataset's data folder. For the VAD Dataset the following Flag is needed to Download/Override the Dataset dataset.override=True Training - Keyword Spotting Training is invoked by hannah-train If available the first GPU of the system will be used by default. Selecting another GPU is possible using the argument trainer. gpus=[number] e.g. for GPU 2 use: hannah-train trainer.gpus=[2] Trained models are saved under trained_models// . Training - VAD Training of VAD is invoked by hannah-train dataset=vad model.n_labels=2 Training of VAD_Extended is invoked by hannah-train dataset=vad_extended model.n_labels=2 VAD dataset variants Selection of other Voice Dataset use dataset.variants=\"[UWNU, de, en, it, fr, es]\" hannah-train dataset=vad model.n_labels=2 dataset.variants=\"[UWNU, de, en, it, fr, es]\" Selection of other Noise Datasets use dataset.noise_dataset=\"[TUT]\" hannah-train dataset=vad model.n_labels=2 dataset.noise_dataset=\"[TUT]\" Selection of dataset Split use dataset.data_split=\"vad_balanced\" hannah-train dataset=vad model.n_labels=2 dataset.data_split=\"vad_balanced\" Create Vad_small Dataset hannah-train dataset=vad model.n_labels=2 dataset.variants=\"[UWNU]\" dataset.noise_dataset=\"[TUT]\" dataset.data_split=\"vad_balanced\" Create VAD_big Dataset hannah-train dataset=vad model.n_labels=2 dataset.variants=\"[UWNU, en, de, fr, es, it]\" dataset.noise_dataset=\"[TUT, FSD50K]\" dataset.data_split=\"vad_balanced\" Training - PAMAP2 Training of PAMAP2 human activity detection dataset is invoked by: hannah-train -cn config_activity Training - Emergency Siren Dataset Training of emergency siren detection dataset is invoked by: hannah-train -cn config_siren_detection Parallel Launchers To launch multiple optimizations in parallel you can use a hydra launcher. Joblib launcher is installed by default: hannah-train --multirun hydra/sweeper=optuna hydra/launcher=joblib optimizer.lr='interval(0.0001,0.1)' optimizer.weight_decay='interval(0, 0.1)' hydra.launcher.n_jobs=5 Launches optimizer hyperparameter optimization with 5 parallel jobs. Early stopping To stop training early when a validation metric does not improve, you can use lightning's early stopping callback: hannah-train early_stopping=default Showing graphical results All experiments are logged to tensorboard: To visualize the results use: tensorboard --logdir trained_models or a subdirectory of trained models if only one experiment or model is of interest. Pre commit hooks This project uses precommit hooks for auto formatting and static code analysis. To enable precommit hooks run the following command in a poetry shell . pre-commit install Try to follow pep8 naming conventions and the rest of pep8 to the best of your abilities. Automatic Mirroring This project automatically mirrors its main branch and all branches prefixed with pub/ to its public github repository. These branches are configured as protected branches by default.","title":"Home"},{"location":"#hannah-hardware-accelerator-and-neural-network-search","text":"","title":"HANNAH - Hardware Accelerator and Neural network searcH"},{"location":"#getting-started","text":"","title":"Getting Started"},{"location":"#installing-dependencies","text":"Dependencies and virtual environments are managed using poetry . python (>=3.8 <3.11) and development headers libsndfile and development headers libsox and development headers a blas implementation and development headers git-lfs for management of checkpoints","title":"Installing dependencies"},{"location":"#ubuntu-2004","text":"Install dependencies: sudo apt update sudo apt -y install python3-dev libblas-dev liblapack-dev libsndfile1-dev libsox-dev git-lfs","title":"Ubuntu 20.04+"},{"location":"#centos-rhel-scientific-linux-7","text":"Install dependencies: sudo yum install portaudio-devel libsndfile1-devel libsox-devel -y Install a python 3.9 or python 3.10 version using pyenv .","title":"Centos / RHEL / Scientific Linux: 7+"},{"location":"#install-poetry","text":"curl -sSL https://install.python-poetry.org/ | python3 For alternative installation methods see: https://python-poetry.org/docs/#installation Caution : this usually installs poetry to ~/.local/bin if this folder is not in your path you might need to run poetry as: ~/.local/bin/poetry","title":"Install poetry"},{"location":"#software-installation","text":"In the root directory of the project run: git submodule update --init --recursive poetry install -E vision This creates a virtual environment under ~/.cache/pypoetry/virtualenvs. The environment can be activated using: poetry shell","title":"Software installation"},{"location":"#optional-dependencies","text":"We support installation of optional dependencies using poetry's -E commandline flag We currently have the following optional dependencies:","title":"Optional Dependencies"},{"location":"#vision-models","text":"Vision models require additional dependencies, these can be installed using: poetry install -E vision These dependencies include kornia and albumentations for image augmentations and timm (torch image models) for baseline neural network models.","title":"Vision-Models"},{"location":"#onnx-runtime-backend","text":"poetry install -E onnxrt-backend Backend support for running models on onnx-runtime.","title":"Onnx-Runtime Backend"},{"location":"#tflite-backend","text":"poetry install -E onnx-tf Onnx based conversion of trained models to tensorflow/tensorflow-lite for external inference backends.","title":"Tflite-Backend"},{"location":"#dvc-based-experiment-management-experimental","text":"poetry install -E dvc This installs dvc based model, data and experiment management. DVC support is highly experimental and subject to change.","title":"DVC based experiment management (experimental)"},{"location":"#installation-tips","text":"1.) venv location poetry installs the dependencies to a virtual environment in ~/.cache/pypoetry/virtualenvs You can change the location of this directory using: poetry config virtualenvs.path Or move it to a subdirectory of the project directory using: poetry config virtualenvs.in-project true","title":"Installation Tips"},{"location":"#installing-the-datasets","text":"Datasets are downloaded automatically to the datasets data folder by default this is a subfolder of the dataset's data folder. For the VAD Dataset the following Flag is needed to Download/Override the Dataset dataset.override=True","title":"Installing the datasets"},{"location":"#training-keyword-spotting","text":"Training is invoked by hannah-train If available the first GPU of the system will be used by default. Selecting another GPU is possible using the argument trainer. gpus=[number] e.g. for GPU 2 use: hannah-train trainer.gpus=[2] Trained models are saved under trained_models// .","title":"Training - Keyword Spotting"},{"location":"#training-vad","text":"Training of VAD is invoked by hannah-train dataset=vad model.n_labels=2 Training of VAD_Extended is invoked by hannah-train dataset=vad_extended model.n_labels=2","title":"Training - VAD"},{"location":"#vad-dataset-variants","text":"Selection of other Voice Dataset use dataset.variants=\"[UWNU, de, en, it, fr, es]\" hannah-train dataset=vad model.n_labels=2 dataset.variants=\"[UWNU, de, en, it, fr, es]\" Selection of other Noise Datasets use dataset.noise_dataset=\"[TUT]\" hannah-train dataset=vad model.n_labels=2 dataset.noise_dataset=\"[TUT]\" Selection of dataset Split use dataset.data_split=\"vad_balanced\" hannah-train dataset=vad model.n_labels=2 dataset.data_split=\"vad_balanced\" Create Vad_small Dataset hannah-train dataset=vad model.n_labels=2 dataset.variants=\"[UWNU]\" dataset.noise_dataset=\"[TUT]\" dataset.data_split=\"vad_balanced\" Create VAD_big Dataset hannah-train dataset=vad model.n_labels=2 dataset.variants=\"[UWNU, en, de, fr, es, it]\" dataset.noise_dataset=\"[TUT, FSD50K]\" dataset.data_split=\"vad_balanced\"","title":"VAD dataset variants"},{"location":"#training-pamap2","text":"Training of PAMAP2 human activity detection dataset is invoked by: hannah-train -cn config_activity","title":"Training - PAMAP2"},{"location":"#training-emergency-siren-dataset","text":"Training of emergency siren detection dataset is invoked by: hannah-train -cn config_siren_detection","title":"Training - Emergency Siren Dataset"},{"location":"#parallel-launchers","text":"To launch multiple optimizations in parallel you can use a hydra launcher. Joblib launcher is installed by default: hannah-train --multirun hydra/sweeper=optuna hydra/launcher=joblib optimizer.lr='interval(0.0001,0.1)' optimizer.weight_decay='interval(0, 0.1)' hydra.launcher.n_jobs=5 Launches optimizer hyperparameter optimization with 5 parallel jobs.","title":"Parallel Launchers"},{"location":"#early-stopping","text":"To stop training early when a validation metric does not improve, you can use lightning's early stopping callback: hannah-train early_stopping=default","title":"Early stopping"},{"location":"#showing-graphical-results","text":"All experiments are logged to tensorboard: To visualize the results use: tensorboard --logdir trained_models or a subdirectory of trained models if only one experiment or model is of interest.","title":"Showing graphical results"},{"location":"#pre-commit-hooks","text":"This project uses precommit hooks for auto formatting and static code analysis. To enable precommit hooks run the following command in a poetry shell . pre-commit install Try to follow pep8 naming conventions and the rest of pep8 to the best of your abilities.","title":"Pre commit hooks"},{"location":"#automatic-mirroring","text":"This project automatically mirrors its main branch and all branches prefixed with pub/ to its public github repository. These branches are configured as protected branches by default.","title":"Automatic Mirroring"},{"location":"fallacies/","text":"Fallacies and Pitfalls Changing OneCycleLR learningrate By default the OneCycleLR learning rate scheduler will ignore the setting for \u00f2ptimizer.lr and instead will use the setting scheduler.max_lr to calculate each steps learning rate. To avoid problems when sweeping the the learning rate, in hannah the default setting for scheduler.max_lr is using the same as optimizer.lr but this can be overridden in project specific configuration files. No SecretStorage or DBUS running During poetry install the system will try to unlock secret storage / keyring. If this fails you will see a Error Message like: [org.freedesktop.DBus.Error.UnknownMethod] ('Object does not exist on path \"/org/freedesktop/secrets/collection/login\"') You can disable this error message by one of the following means: Install and Configure a local DBus and Keyring support. This might work. Setting the default backend for keyring: export PYTHON_KEYRING_BACKEND=keyring.backends.null.Keyring Disabling keyring for the whole system: keyring --disable -b keyring.backends.SecretService.Keyring","title":"Fallacies and Pitfalls"},{"location":"fallacies/#fallacies-and-pitfalls","text":"","title":"Fallacies and Pitfalls"},{"location":"fallacies/#changing-onecyclelr-learningrate","text":"By default the OneCycleLR learning rate scheduler will ignore the setting for \u00f2ptimizer.lr and instead will use the setting scheduler.max_lr to calculate each steps learning rate. To avoid problems when sweeping the the learning rate, in hannah the default setting for scheduler.max_lr is using the same as optimizer.lr but this can be overridden in project specific configuration files.","title":"Changing OneCycleLR learningrate"},{"location":"fallacies/#no-secretstorage-or-dbus-running","text":"During poetry install the system will try to unlock secret storage / keyring. If this fails you will see a Error Message like: [org.freedesktop.DBus.Error.UnknownMethod] ('Object does not exist on path \"/org/freedesktop/secrets/collection/login\"') You can disable this error message by one of the following means: Install and Configure a local DBus and Keyring support. This might work. Setting the default backend for keyring: export PYTHON_KEYRING_BACKEND=keyring.backends.null.Keyring Disabling keyring for the whole system: keyring --disable -b keyring.backends.SecretService.Keyring","title":"No SecretStorage or DBUS running"},{"location":"publications/","text":"Related Publications Precise localization within the GI tract by combining classification of CNNs and time-series analysis of HMMs. by Julia Werner, Christoph Gerum, Moritz Reiber, J\u00f6rg Nick, Oliver Bringmann In 2023 14th International Workshop on Machine Learning in Medical Imaging (MLMI 2023) Hardware Accelerator and Neural Network Co-Optimization for Ultra-Low-Power Audio Processing Devices by Christoph Gerum, Adrian Frischknecht, Paul Palomero Bernardo, Tobias Hald, Konstantin L\u00fcbeck, and Oliver Bringmann In 2022 25th Euromicro Conference on Digital System Design (DSD), pages 1-8, 2022. Keywords: Machine Learning, Neural Networks, AutoML, Neural Architecture Search Behavior of Keyword Spotting Networks Under Noisy Conditions by Anwesh Mohanty, Adrian Frischknecht, Christoph Gerum, and Oliver Bringmann In International Conference on Artificial Neural Networks (ICANN), 2021. Keywords: Keyword spotting, High noise conditions, Adaptive batch normalization, Sinc convolution network, Temporal convolution ResNet. Dynamic Range and Complexity Optimization of Mixed-Signal Machine Learning Systems by Naci Pekcokguler, Dominique Morche, Adrian Frischknecht, Christoph Gerum, Andreas Burg, and Catherine Dehollain In 2021 IEEE International Symposium on Circuits and Systems (ISCAS), pages 1-5, 2021. Keywords: keyword spotting, analog feature extraction, machine learning classi\ufb01er, dynamic range reduction, system complexity optimization Automated HW/SW Co-Design for Edge AI: State, Challenges and Steps Ahead by Oliver Bringmann, Wolfgang Ecker, Ingo Feldner, Adrian Frischknecht, Christoph Gerum, Timo H\u00e4m\u00e4l\u00e4inen, Muhammad Abdullah Hanif, Michael J. Klaiber, Daniel Mueller-Gritschneder, Paul Palomero Bernardo, Sebastian Prebeck, and Muhammad Shafique In Proceedings of the 2021 International Conference on Hardware/Software Codesign and System Synthesis, pages 11\u201320. Association for Computing Machinery, 2021. UltraTrail: A Configurable Ultra-Low Power TC-ResNet AI Accelerator for Efficient Keyword Spotting by Paul Palomero Bernardo, Christoph Gerum, Adrian Frischknecht, Konstantin L\u00fcbeck, and Oliver Bringmann In 2020 IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems (TCAD), pages 1-12, 2020.","title":"Publications"},{"location":"publications/#related-publications","text":"Precise localization within the GI tract by combining classification of CNNs and time-series analysis of HMMs. by Julia Werner, Christoph Gerum, Moritz Reiber, J\u00f6rg Nick, Oliver Bringmann In 2023 14th International Workshop on Machine Learning in Medical Imaging (MLMI 2023) Hardware Accelerator and Neural Network Co-Optimization for Ultra-Low-Power Audio Processing Devices by Christoph Gerum, Adrian Frischknecht, Paul Palomero Bernardo, Tobias Hald, Konstantin L\u00fcbeck, and Oliver Bringmann In 2022 25th Euromicro Conference on Digital System Design (DSD), pages 1-8, 2022. Keywords: Machine Learning, Neural Networks, AutoML, Neural Architecture Search Behavior of Keyword Spotting Networks Under Noisy Conditions by Anwesh Mohanty, Adrian Frischknecht, Christoph Gerum, and Oliver Bringmann In International Conference on Artificial Neural Networks (ICANN), 2021. Keywords: Keyword spotting, High noise conditions, Adaptive batch normalization, Sinc convolution network, Temporal convolution ResNet. Dynamic Range and Complexity Optimization of Mixed-Signal Machine Learning Systems by Naci Pekcokguler, Dominique Morche, Adrian Frischknecht, Christoph Gerum, Andreas Burg, and Catherine Dehollain In 2021 IEEE International Symposium on Circuits and Systems (ISCAS), pages 1-5, 2021. Keywords: keyword spotting, analog feature extraction, machine learning classi\ufb01er, dynamic range reduction, system complexity optimization Automated HW/SW Co-Design for Edge AI: State, Challenges and Steps Ahead by Oliver Bringmann, Wolfgang Ecker, Ingo Feldner, Adrian Frischknecht, Christoph Gerum, Timo H\u00e4m\u00e4l\u00e4inen, Muhammad Abdullah Hanif, Michael J. Klaiber, Daniel Mueller-Gritschneder, Paul Palomero Bernardo, Sebastian Prebeck, and Muhammad Shafique In Proceedings of the 2021 International Conference on Hardware/Software Codesign and System Synthesis, pages 11\u201320. Association for Computing Machinery, 2021. UltraTrail: A Configurable Ultra-Low Power TC-ResNet AI Accelerator for Efficient Keyword Spotting by Paul Palomero Bernardo, Christoph Gerum, Adrian Frischknecht, Konstantin L\u00fcbeck, and Oliver Bringmann In 2020 IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems (TCAD), pages 1-12, 2020.","title":"Related Publications"},{"location":"compression/compresseion/knowledge_distillation/","text":"Pruning TBD","title":"Knowledge Distillation"},{"location":"compression/compresseion/knowledge_distillation/#pruning","text":"TBD","title":"Pruning"},{"location":"compression/compression/clustering/","text":"Clustering k-means \\(k\\) -means clustering can be applied to the weights of all layers at the end of a training epoch. Each time, for a defined number of \\(k\\) clusters per layer the centroids are identified per layer. Next, each weight is replaced with the value of the closest cluster center. The zero is not clustered to preserve the amount of zeros induced by pruning. This results in a total of \\(k+1\\) distinct values. Additionally, at the end of the fitting process, clustering is applied. Afterwards, the weights are not further altered. After completing the training process, every value of each layer of the trained neural network should be equal to one cluster center or zero. Clustering (in combination with pruning and SVD-based compression) can be invoked by: hannah-train compression=full The default amount of clusters is 15 If clustering or SVD was applied, the last checkpoint of the model during training was used for validation and testing the neural network, instead of the best checkpoint as previously used. Fixing the amount of epochs with which the final model is trained enables better comparability of the models for different compression parameters and ensures that the compression is sufficiently integrated into the training process. In particular the choice of a final model with high accuracy, which has not yet been compressed in the training process (via SVD or clustering) is avoided by ensuring that the last checkpoint is used. Consequently, fixing the amount of training epochs for the final model illuminates the effect that the compression during the training has on the model. For applying clustering to quantized models, the batch norm fusion must be considered. This is not yet implemented.","title":"Clustering"},{"location":"compression/compression/clustering/#clustering","text":"","title":"Clustering"},{"location":"compression/compression/clustering/#k-means","text":"\\(k\\) -means clustering can be applied to the weights of all layers at the end of a training epoch. Each time, for a defined number of \\(k\\) clusters per layer the centroids are identified per layer. Next, each weight is replaced with the value of the closest cluster center. The zero is not clustered to preserve the amount of zeros induced by pruning. This results in a total of \\(k+1\\) distinct values. Additionally, at the end of the fitting process, clustering is applied. Afterwards, the weights are not further altered. After completing the training process, every value of each layer of the trained neural network should be equal to one cluster center or zero. Clustering (in combination with pruning and SVD-based compression) can be invoked by: hannah-train compression=full The default amount of clusters is 15 If clustering or SVD was applied, the last checkpoint of the model during training was used for validation and testing the neural network, instead of the best checkpoint as previously used. Fixing the amount of epochs with which the final model is trained enables better comparability of the models for different compression parameters and ensures that the compression is sufficiently integrated into the training process. In particular the choice of a final model with high accuracy, which has not yet been compressed in the training process (via SVD or clustering) is avoided by ensuring that the last checkpoint is used. Consequently, fixing the amount of training epochs for the final model illuminates the effect that the compression during the training has on the model. For applying clustering to quantized models, the batch norm fusion must be considered. This is not yet implemented.","title":"k-means"},{"location":"compression/compression/decomposition/","text":"Decomposition Singular value decomposition (SVD) In a CNN, the SVD can be applied to a fully connected layer, corresponding to a matrix \\(A\\) , resulting in the composition of two fully connected layers of lower dimension associated to the matrices \\(U'\\) and \\(\\Sigma' V^{T'}\\) . If \\(\\Sigma\\) is truncated to a rank \\(r=k\\) , dimensions change to \\(U'\\in \\mathbb{R}^{m \\times k}\\) , \\(\\Sigma' \\in \\mathbb{R}^{k \\times k}\\) and \\(V^{T'}\\in \\mathbb{R}^{k \\times n}\\) . Therefore, the SVD is applied to all weights of the linear layers, resulting in the decomposition of matrices. Subsequently, the weight matrices of the fully connected layers are compressed with a fixed rank \\(r\\) . Afterwards, the model is restructured, resulting in two smaller fully connected layers instead of one larger fully connected layer. SVD-based compression (in combination with pruning and clustering) can be invoked by: hannah-train compression=full The default rank_compression of SVD is 4 This has been implemented for TC-ResNets, which consist of one linear layer as the last layer of the network. Thus, the resulting network comprises of a sequential layer at the end, including one fully connected layer with \\(\\Sigma'*V^{T'}\\) and one fully connected layer with \\(U'\\) . If those partial matrices are multiplied, they ideally yield a close approximation to the original matrix \\(A\\) . Finally, fine-tuning is performed, by continuing the training process with this restructured, compressed network while the same optimizer and parameters are used as for the first training section. With this approach, the SVD-based compression is easily combined with QAT and pruning. Both methods are also applied to the restructured, compressed model during retraining. The layer's name can be adjusted to add other models.","title":"Tensor Decompositions"},{"location":"compression/compression/decomposition/#decomposition","text":"","title":"Decomposition"},{"location":"compression/compression/decomposition/#singular-value-decomposition-svd","text":"In a CNN, the SVD can be applied to a fully connected layer, corresponding to a matrix \\(A\\) , resulting in the composition of two fully connected layers of lower dimension associated to the matrices \\(U'\\) and \\(\\Sigma' V^{T'}\\) . If \\(\\Sigma\\) is truncated to a rank \\(r=k\\) , dimensions change to \\(U'\\in \\mathbb{R}^{m \\times k}\\) , \\(\\Sigma' \\in \\mathbb{R}^{k \\times k}\\) and \\(V^{T'}\\in \\mathbb{R}^{k \\times n}\\) . Therefore, the SVD is applied to all weights of the linear layers, resulting in the decomposition of matrices. Subsequently, the weight matrices of the fully connected layers are compressed with a fixed rank \\(r\\) . Afterwards, the model is restructured, resulting in two smaller fully connected layers instead of one larger fully connected layer. SVD-based compression (in combination with pruning and clustering) can be invoked by: hannah-train compression=full The default rank_compression of SVD is 4 This has been implemented for TC-ResNets, which consist of one linear layer as the last layer of the network. Thus, the resulting network comprises of a sequential layer at the end, including one fully connected layer with \\(\\Sigma'*V^{T'}\\) and one fully connected layer with \\(U'\\) . If those partial matrices are multiplied, they ideally yield a close approximation to the original matrix \\(A\\) . Finally, fine-tuning is performed, by continuing the training process with this restructured, compressed network while the same optimizer and parameters are used as for the first training section. With this approach, the SVD-based compression is easily combined with QAT and pruning. Both methods are also applied to the restructured, compressed model during retraining. The layer's name can be adjusted to add other models.","title":"Singular value decomposition (SVD)"},{"location":"compression/compression/pruning/","text":"Pruning TBD","title":"Pruning"},{"location":"compression/compression/pruning/#pruning","text":"TBD","title":"Pruning"},{"location":"compression/compression/quantization/","text":"Quantization The training framework provides quantization aware training support for model factory based models. We support the following quantization schemes. Symmetric linear Quantization: Symmetric linear quantization applies the following transformation on a floating point value \\(v\\) : \\(\\mbox{quantized}(v)\\ =\\ \\mbox{clamp}(\\mbox{round}(v\\cdot2^{\\mbox{bits}-1}), 2^{-\\mbox{bits}-1}, 2^{\\mbox{bits}-1}-1)\\) Power of 2 Quantization with Zero: Power of 2 quantization applies the following transformation to a floating point value v: \\( \\mbox{quantized}(v)\\ =\\ \\begin{cases} \\mbox{sign}(v)\\ \\cdot\\ 2^{\\mbox{round}(\\log2(\\mbox{abs}(v)))} & \\text{if}\\ v\\ \\ge\\ 2^{2^{\\text{bits}}\\\\ 0 & \\text{otherwise} \\\\ \\end{cases} \\) Power of 2 Quantization can currently only be enabled for weights. Noisy Quantization As described in https://openreview.net/pdf?id=dV19Yyi1fS3 quantizing only a subset of model weights during training can improve accuracies of quantized networks compared to full quantization aware training. The probability of quantizing a weight can be given as parameter noise_prob in in the qconfig. Unfortunately this introduces an additional hyperparameter in the quantization space, good values for noise prob seem to be in the range of 0.7 to 0.9. Even when using noisy quantization eval and test are always run on fully quantized networks. Configuration Quantization can be configured as the qconfig attribute of factory based models: qconfig: target : speech_recognition.models.factory.qconfig.get_trax_qat_qconfig config: bw_b: 8 # Bitwidth for bias bw_w: 6 # Bitwidth for weights bw_f: 8 # Bitwidth for activations power_of_2: false # Use power of two quantization for weights noise_prob: 0.7 # Probability of quantizing a value during training Aditionally standard pytorch quantization aware training is supported by using a standard pytorch qconfig. qconfig: _target_: torch.quantization.get_default_qat_qconfig backend: fbgemm In this case no quantization noise is supported.","title":"Quantization"},{"location":"compression/compression/quantization/#quantization","text":"The training framework provides quantization aware training support for model factory based models. We support the following quantization schemes. Symmetric linear Quantization: Symmetric linear quantization applies the following transformation on a floating point value \\(v\\) : \\(\\mbox{quantized}(v)\\ =\\ \\mbox{clamp}(\\mbox{round}(v\\cdot2^{\\mbox{bits}-1}), 2^{-\\mbox{bits}-1}, 2^{\\mbox{bits}-1}-1)\\) Power of 2 Quantization with Zero: Power of 2 quantization applies the following transformation to a floating point value v: \\( \\mbox{quantized}(v)\\ =\\ \\begin{cases} \\mbox{sign}(v)\\ \\cdot\\ 2^{\\mbox{round}(\\log2(\\mbox{abs}(v)))} & \\text{if}\\ v\\ \\ge\\ 2^{2^{\\text{bits}}\\\\ 0 & \\text{otherwise} \\\\ \\end{cases} \\) Power of 2 Quantization can currently only be enabled for weights.","title":"Quantization"},{"location":"compression/compression/quantization/#noisy-quantization","text":"As described in https://openreview.net/pdf?id=dV19Yyi1fS3 quantizing only a subset of model weights during training can improve accuracies of quantized networks compared to full quantization aware training. The probability of quantizing a weight can be given as parameter noise_prob in in the qconfig. Unfortunately this introduces an additional hyperparameter in the quantization space, good values for noise prob seem to be in the range of 0.7 to 0.9. Even when using noisy quantization eval and test are always run on fully quantized networks.","title":"Noisy Quantization"},{"location":"compression/compression/quantization/#configuration","text":"Quantization can be configured as the qconfig attribute of factory based models: qconfig: target : speech_recognition.models.factory.qconfig.get_trax_qat_qconfig config: bw_b: 8 # Bitwidth for bias bw_w: 6 # Bitwidth for weights bw_f: 8 # Bitwidth for activations power_of_2: false # Use power of two quantization for weights noise_prob: 0.7 # Probability of quantizing a value during training Aditionally standard pytorch quantization aware training is supported by using a standard pytorch qconfig. qconfig: _target_: torch.quantization.get_default_qat_qconfig backend: fbgemm In this case no quantization noise is supported.","title":"Configuration"},{"location":"configuration/configuration/multi_gpu/","text":"Multi GPU training Hannah supports multi GPU-Training using the lightning distributed APIs: We provide preset trainer configs for distributed data parallel training: hannah-train trainer=ddp trainer.gpus=[0,1] And for sharded training using fairscale: hannah-train trainer=sharded trainer.gpus=[0,1] Sharded training distributes some of the model parameters across multiple GPUs and allows fitting bigger models in the same amount of GPU memory.","title":"Multi-GPU support"},{"location":"configuration/configuration/multi_gpu/#multi-gpu-training","text":"Hannah supports multi GPU-Training using the lightning distributed APIs: We provide preset trainer configs for distributed data parallel training: hannah-train trainer=ddp trainer.gpus=[0,1] And for sharded training using fairscale: hannah-train trainer=sharded trainer.gpus=[0,1] Sharded training distributes some of the model parameters across multiple GPUs and allows fitting bigger models in the same amount of GPU memory.","title":"Multi GPU training"},{"location":"configuration/configuration/overview/","text":"Configuration Configurations are managed by hydra . And follow a structured configuration. The currently used configuration can be shown with: hannah-train -c job The default configuration is located under speech_recognition/conf/ . Configuration Structure There are two main locations for configuration: Framework configuration files are located under hannah/conf configuration files located in this directory should be useful for more than one project. Project specific configuration files are located in configs/ these usually contain a project specific configuration file and one or more configured experiments in the subfolder experiment make use of composition using defaults lists as much as possible Guideline for writing configuration files Consider avoiding project specific main configuration files, and try to configure your project using experiments only. Use composition defaults list composition as much as possible. Configuration Groups We currently have the following configuration groups: backend Configures the app to run a subset of validation and test on configured backend Choices are: null (no backend, default), torchmobile (translates model to backend) checkpoint Choices are: null (no checkpoint), all (default, creates checkpoints for all training epoch) best (checkpoint, best 5 values, default) compression Configuration for model compression. dataset Choices are: kws (For Google Keyword Spotting), snips (For hey snips dataset), vad (For Voice Activity Detection) Common configuration options for datasets are: data_folder base folder for data (actual data will usually be stored in $data_folder/dataset_name) cls speech_recognition.dataset.SpeechCommandsDataset dataset name of the dataset group_speakers_by_id true group the data by user id before splitting if available use_default_split false if the dataset has a predefined split, use it (ignores *_pct, and group by id parameters) test_pct Percentage of dataset used for test dev_pct Percentage of dataset used for validation train_pct Percentage of dataset used for training wanted_words Wanted words for keyword datasets input_length Length of input in number of samples samplingrate Sampling rate of data in Hz clear_download \"Remove downloaded archive after dataset has been extracted variants Variant of dataset to use for multilanguage datasets timeshift_ms \"Timeshift the input data by +- given ms\" extract loudest silence_prob % dataset samples that are silence unknown_prob % dataset samples that are unknown test_snr SNR used for test train_snr_high minimal SNR for training data train_snr_low maximal SNR for test data noise_dataset [\"TUT\", \"FSDKaggle\", \"FSDnoisy\"] Downloads all the specified datasets. Use TUT + one other data_split initial split after downloading the datasets(Possibilities: \"vad\", \"vad_speech\", \"vad_balanced\", \"getrennt\") downsample samplerate DESTRUCTIVE! change the samplerate of the real files to the target samplerate. Use better parameter samplingrate sensor resolution can be adjusted for resizing of images (currently only for vision capsule datasets). variants variants for kws - v1, v2 variants for snips - snips variants for vad - UWNU - Mozilla has the following language options: - en: Englisch - de: Detusch - fr: Franz\u00f6sisch - it: Italienisch - es: Spanisch - kab: Kabylisch - ca: Katalanisch - nl: Niderl\u00e4ndisch - eo: Esperanto - fa: Persisch - eu: Baskisch - rw: Kinyarwanda - ru: Russisch - pt: Portugiesisch - pl: Polnisch features Feature extractor to use, choices are: melspec, mfcc, raw, sinc, spectrogram . All feature extractors apart from sinc (Sinc Convolutions) currently use torchaudio feature extractors and use the same input parameters. Sinc Convolutions have the following parameters: normalizer Choices: null (default), fixedpoint (normalize feature values to [-1.0, 1.0]) Feature Normalizer run between feature extraction and the neural network model. model Neural network to train: choices are gds (Old sinc1 with sinc convolutions removed), lstm, tc-res8, branchy-tc-res8 module The toplevel module implementing the training and eval loops. We curently support the following Tasks: stream_classifier : Classification on multichannel 1D data streams image_classifier : Classification on Image Data optimizer Choices are: adadelta, adam, adamax, adamw, asgd, lbfgs, rmsprop, rprop, sgd, sparse_adam Directly instantiate the corresponding pytorch classes and take the same options. profiler Choices are: null , advanced, simple Run profiling for different phases. scheduler Choices are: null , 1cycle, cosine, cosine_warm, cyclic, exponential, multistep, plateau, step Learning rate scheduler to use for scheduling, default is null. trainer Choices: default, dds Capsules the options to the lightning trainer. Currently it sets the following defaults: Default options are: gpus 1 auto_select_gpus True limit_train_batches 1.0 limit_val_batches 1.0 limit_test_batches 1.0 max_epochs 80 default_root_dir . fast_dev_run false overfit_batches 0.0 benchmark True deterministic True Environment Variables The default configurations interpolate the following environment variables: HANNAH_CACHE_DIR Location of a directory used to cache file loading in some datasets HANNAH_DATA_FOLDER Change default location of dataset folders by default we will use subdirectory datasets of current working directory","title":"Overview"},{"location":"configuration/configuration/overview/#configuration","text":"Configurations are managed by hydra . And follow a structured configuration. The currently used configuration can be shown with: hannah-train -c job The default configuration is located under speech_recognition/conf/ .","title":"Configuration"},{"location":"configuration/configuration/overview/#configuration-structure","text":"There are two main locations for configuration: Framework configuration files are located under hannah/conf configuration files located in this directory should be useful for more than one project. Project specific configuration files are located in configs/ these usually contain a project specific configuration file and one or more configured experiments in the subfolder experiment make use of composition using defaults lists as much as possible","title":"Configuration Structure"},{"location":"configuration/configuration/overview/#guideline-for-writing-configuration-files","text":"Consider avoiding project specific main configuration files, and try to configure your project using experiments only. Use composition defaults list composition as much as possible.","title":"Guideline for writing configuration files"},{"location":"configuration/configuration/overview/#configuration-groups","text":"We currently have the following configuration groups:","title":"Configuration Groups"},{"location":"configuration/configuration/overview/#backend","text":"Configures the app to run a subset of validation and test on configured backend Choices are: null (no backend, default), torchmobile (translates model to backend)","title":"backend"},{"location":"configuration/configuration/overview/#checkpoint","text":"Choices are: null (no checkpoint), all (default, creates checkpoints for all training epoch) best (checkpoint, best 5 values, default)","title":"checkpoint"},{"location":"configuration/configuration/overview/#compression","text":"Configuration for model compression.","title":"compression"},{"location":"configuration/configuration/overview/#dataset","text":"Choices are: kws (For Google Keyword Spotting), snips (For hey snips dataset), vad (For Voice Activity Detection) Common configuration options for datasets are: data_folder base folder for data (actual data will usually be stored in $data_folder/dataset_name) cls speech_recognition.dataset.SpeechCommandsDataset dataset name of the dataset group_speakers_by_id true group the data by user id before splitting if available use_default_split false if the dataset has a predefined split, use it (ignores *_pct, and group by id parameters) test_pct Percentage of dataset used for test dev_pct Percentage of dataset used for validation train_pct Percentage of dataset used for training wanted_words Wanted words for keyword datasets input_length Length of input in number of samples samplingrate Sampling rate of data in Hz clear_download \"Remove downloaded archive after dataset has been extracted variants Variant of dataset to use for multilanguage datasets timeshift_ms \"Timeshift the input data by +- given ms\" extract loudest silence_prob % dataset samples that are silence unknown_prob % dataset samples that are unknown test_snr SNR used for test train_snr_high minimal SNR for training data train_snr_low maximal SNR for test data noise_dataset [\"TUT\", \"FSDKaggle\", \"FSDnoisy\"] Downloads all the specified datasets. Use TUT + one other data_split initial split after downloading the datasets(Possibilities: \"vad\", \"vad_speech\", \"vad_balanced\", \"getrennt\") downsample samplerate DESTRUCTIVE! change the samplerate of the real files to the target samplerate. Use better parameter samplingrate sensor resolution can be adjusted for resizing of images (currently only for vision capsule datasets).","title":"dataset"},{"location":"configuration/configuration/overview/#variants","text":"variants for kws - v1, v2 variants for snips - snips variants for vad - UWNU - Mozilla has the following language options: - en: Englisch - de: Detusch - fr: Franz\u00f6sisch - it: Italienisch - es: Spanisch - kab: Kabylisch - ca: Katalanisch - nl: Niderl\u00e4ndisch - eo: Esperanto - fa: Persisch - eu: Baskisch - rw: Kinyarwanda - ru: Russisch - pt: Portugiesisch - pl: Polnisch","title":"variants"},{"location":"configuration/configuration/overview/#features","text":"Feature extractor to use, choices are: melspec, mfcc, raw, sinc, spectrogram . All feature extractors apart from sinc (Sinc Convolutions) currently use torchaudio feature extractors and use the same input parameters. Sinc Convolutions have the following parameters:","title":"features"},{"location":"configuration/configuration/overview/#normalizer","text":"Choices: null (default), fixedpoint (normalize feature values to [-1.0, 1.0]) Feature Normalizer run between feature extraction and the neural network model.","title":"normalizer"},{"location":"configuration/configuration/overview/#model","text":"Neural network to train: choices are gds (Old sinc1 with sinc convolutions removed), lstm, tc-res8, branchy-tc-res8","title":"model"},{"location":"configuration/configuration/overview/#module","text":"The toplevel module implementing the training and eval loops. We curently support the following Tasks: stream_classifier : Classification on multichannel 1D data streams image_classifier : Classification on Image Data","title":"module"},{"location":"configuration/configuration/overview/#optimizer","text":"Choices are: adadelta, adam, adamax, adamw, asgd, lbfgs, rmsprop, rprop, sgd, sparse_adam Directly instantiate the corresponding pytorch classes and take the same options.","title":"optimizer"},{"location":"configuration/configuration/overview/#profiler","text":"Choices are: null , advanced, simple Run profiling for different phases.","title":"profiler"},{"location":"configuration/configuration/overview/#scheduler","text":"Choices are: null , 1cycle, cosine, cosine_warm, cyclic, exponential, multistep, plateau, step Learning rate scheduler to use for scheduling, default is null.","title":"scheduler"},{"location":"configuration/configuration/overview/#trainer","text":"Choices: default, dds Capsules the options to the lightning trainer. Currently it sets the following defaults: Default options are: gpus 1 auto_select_gpus True limit_train_batches 1.0 limit_val_batches 1.0 limit_test_batches 1.0 max_epochs 80 default_root_dir . fast_dev_run false overfit_batches 0.0 benchmark True deterministic True","title":"trainer"},{"location":"configuration/configuration/overview/#environment-variables","text":"The default configurations interpolate the following environment variables: HANNAH_CACHE_DIR Location of a directory used to cache file loading in some datasets HANNAH_DATA_FOLDER Change default location of dataset folders by default we will use subdirectory datasets of current working directory","title":"Environment Variables"},{"location":"deployment/deployment/torchmobile/","text":"Torchmobile backend A backend that runs the trained neural network through torchmobile on the local cpu. This is an example implementation of backend that allows testing of backend integrations without installation of further packages. It should not be used for serious optimizations, as measurements are much to noisy. Configuration val_batches 1 (number of batches used for validation) test_batches 1 (number of batches used for test) val_frequency 10 (run backend every n validation epochs)","title":"Torch Mobile"},{"location":"deployment/deployment/torchmobile/#torchmobile-backend","text":"A backend that runs the trained neural network through torchmobile on the local cpu. This is an example implementation of backend that allows testing of backend integrations without installation of further packages. It should not be used for serious optimizations, as measurements are much to noisy.","title":"Torchmobile backend"},{"location":"deployment/deployment/torchmobile/#configuration","text":"val_batches 1 (number of batches used for validation) test_batches 1 (number of batches used for test) val_frequency 10 (run backend every n validation epochs)","title":"Configuration"},{"location":"deployment/deployment/trax-Ultratrail/","text":"Ultratrail backend To use the ultratrail backend use Install teda: https://atreus.informatik.uni-tuebingen.de/ties/t-rax/teda Set environment variable TEDA_HOME to path of teda checkout To run it use: hannah-train compress=fp_8_8_8 backend=trax_ut Configuration teda_dir ${env:TEDA_HOME,/local/frischkn/neural_networks/teda} backend_dir \"${backend.teda_dir}/rtl/tc-resnet8-accelerator\" standalone False rtl_simulation False synthesis False postsyn_simulation False power_estimation False num_inferences 1 bw_w 8 bw_b 8 bw_f 8 cols 8 rows 8 period 1000 macro_type \"RTL\" use_acc_statistic_model True use_acc_analytical_model False","title":"T-Rax Ultratrail"},{"location":"deployment/deployment/trax-Ultratrail/#ultratrail-backend","text":"To use the ultratrail backend use Install teda: https://atreus.informatik.uni-tuebingen.de/ties/t-rax/teda Set environment variable TEDA_HOME to path of teda checkout To run it use: hannah-train compress=fp_8_8_8 backend=trax_ut","title":"Ultratrail backend"},{"location":"deployment/deployment/trax-Ultratrail/#configuration","text":"teda_dir ${env:TEDA_HOME,/local/frischkn/neural_networks/teda} backend_dir \"${backend.teda_dir}/rtl/tc-resnet8-accelerator\" standalone False rtl_simulation False synthesis False postsyn_simulation False power_estimation False num_inferences 1 bw_w 8 bw_b 8 bw_f 8 cols 8 rows 8 period 1000 macro_type \"RTL\" use_acc_statistic_model True use_acc_analytical_model False","title":"Configuration"},{"location":"deployment/deployment/tvm/","text":"TVM backend A backend that runs the trained neural network through tvm with remote target support through automate. Configuration val_batches 1 (number of batches used for validation) test_batches 1 (number of batches used for test) val_frequency 10 (run backend every n validation epochs) board Target board configuration tuner optional tuner configuration Board Configuration MicroTVM connected boards or simulators Automate connected Remote Boards Tuner Autotuning is configure using the configuration group backend/tuner . There are 3 predefined tuner configurations: \u00e0utotvm Uses the standard configuration for autotuning on the target board, it is generallly the recommended setting for 8-bit quantized models on CPU like devices. As it allows the targets to use hand defined tuning templates, which generally include tvm tuning templates. auto_scheduler Uses the standard tvm auto_scheduler. This scheduler does not support tensorization and is therefore not recommended on targets that support tensorized instructions. This is mostly the case for tensorized instructions on the target boards. baseline This does not use any tuning but uses the predefined schedules directly. All tuner configurations are parametrizable using the following tuner configurations: All autotuning can be deactivated when setting backend.tune = false Tuner Configuration","title":"TVM"},{"location":"deployment/deployment/tvm/#tvm-backend","text":"A backend that runs the trained neural network through tvm with remote target support through automate.","title":"TVM backend"},{"location":"deployment/deployment/tvm/#configuration","text":"val_batches 1 (number of batches used for validation) test_batches 1 (number of batches used for test) val_frequency 10 (run backend every n validation epochs) board Target board configuration tuner optional tuner configuration","title":"Configuration"},{"location":"deployment/deployment/tvm/#board-configuration","text":"","title":"Board Configuration"},{"location":"deployment/deployment/tvm/#microtvm-connected-boards-or-simulators","text":"","title":"MicroTVM connected boards or simulators"},{"location":"deployment/deployment/tvm/#automate-connected-remote-boards","text":"","title":"Automate connected Remote Boards"},{"location":"deployment/deployment/tvm/#tuner","text":"Autotuning is configure using the configuration group backend/tuner . There are 3 predefined tuner configurations: \u00e0utotvm Uses the standard configuration for autotuning on the target board, it is generallly the recommended setting for 8-bit quantized models on CPU like devices. As it allows the targets to use hand defined tuning templates, which generally include tvm tuning templates. auto_scheduler Uses the standard tvm auto_scheduler. This scheduler does not support tensorization and is therefore not recommended on targets that support tensorized instructions. This is mostly the case for tensorized instructions on the target boards. baseline This does not use any tuning but uses the predefined schedules directly. All tuner configurations are parametrizable using the following tuner configurations: All autotuning can be deactivated when setting backend.tune = false","title":"Tuner"},{"location":"deployment/deployment/tvm/#tuner-configuration","text":"","title":"Tuner Configuration"},{"location":"development/devel/api/","text":"hannah.tools.objectdetection_eval eval_train def eval_train(config, module, test=True) Arguments : config - param module: test - Default value = True) module: eval_steps def eval_steps(config, module, hparams, checkpoint) Arguments : config - param module: hparams - param checkpoint: module: checkpoint: eval_checkpoint def eval_checkpoint(config: DictConfig, checkpoint) Arguments : config - DictConfig: checkpoint: config - DictConfig: config - DictConfig: eval def eval(config: DictConfig) Arguments : config - DictConfig: config - DictConfig: config - DictConfig: main @hydra.main(config_name=\"objectdetection_eval\", config_path=\"../conf\", version_base=\"1.2\") def main(config: DictConfig) Arguments : config - DictConfig: config - DictConfig: config - DictConfig: hannah.tools.train hannah.tools hannah.tools.characterize main @hydra.main(config_name=\"characterize\", config_path=\"../conf\", version_base=\"1.2\") def main(config: DictConfig) Arguments : config - DictConfig: config - DictConfig: config - DictConfig: hannah.tools.eval eval_checkpoint def eval_checkpoint(config: DictConfig, checkpoint) -> None Arguments : config - DictConfig: checkpoint: config - DictConfig: config - DictConfig: eval def eval(config: DictConfig) -> Optional[bool] Arguments : config - DictConfig: config - DictConfig: config - DictConfig: main @hydra.main(config_name=\"eval\", config_path=\"conf\", version_base=\"1.2\") def main(config: DictConfig) Arguments : config - DictConfig: config - DictConfig: config - DictConfig: hannah.trainer hannah.trainer.cross_validation hannah.train hannah.logo hannah.features MFCC Objects class MFCC(torchaudio.transforms.MFCC) A simple wrapper around torchaudio mfcc, but melkwargs are given as direct named arguments instead of a dictionary SincConv Objects class SincConv(nn.Module) Sinc convolution: Arguments : in_channels - No. of input channels(must be 1) out_channels - No. of filters(40) sample_rate - sampling rate, default set at 32000 kernel_size - Filter length(101) LogSpectrogram Objects class LogSpectrogram(torch.nn.Module) Create a spectrogram from a audio signal. Arguments : n_fft int, optional - Size of FFT, creates n_fft // 2 + 1 bins. (Default: 400 ) win_length int or None, optional - Window size. (Default: n_fft ) hop_length int or None, optional - Length of hop between STFT windows. (Default: win_length // 2 ) pad int, optional - Two sided padding of signal. (Default: 0 ) window_fn Callable[..., Tensor], optional - A function to create a window tensor that is applied/multiplied to each frame/window. (Default: torch.hann_window ) power float or None, optional - Exponent for the magnitude spectrogram, (must be > 0) e.g., 1 for energy, 2 for power, etc. If None, then the complex spectrum is returned instead. (Default: 2 ) normalized bool, optional - Whether to normalize by magnitude after stft. (Default: False ) wkwargs dict or None, optional - Arguments for window function. (Default: None ) center bool, optional - whether to pad :attr: waveform on both sides so that the :math: t -th frame is centered at time :math: t \\times \\text{hop\\_length} . Default - True pad_mode string, optional - controls the padding method used when :attr: center is True . Default: \"reflect\" onesided bool, optional - controls whether to return half of results to avoid redundancy Default: True forward def forward(waveform: torch.Tensor) -> torch.Tensor Arguments : waveform Tensor - Tensor of audio of dimension (..., time). Returns : Tensor - Dimension (..., freq, time), where freq is n_fft // 2 + 1 where n_fft is the number of Fourier bins, and time is the number of window hops (n_frame). hannah.normalizer FixedPointNormalizer Objects class FixedPointNormalizer(nn.Module) Simple feature normalizer for fixed point models AdaptiveFixedPointNormalizer Objects class AdaptiveFixedPointNormalizer(nn.Module) Simple feature normalizer for fixed point models hannah.models.honk hannah.models.honk.model truncated_normal def truncated_normal(tensor, std_dev=0.01) Arguments : tensor: - std_dev - (Default value = 0.01) SpeechResModel Objects class SpeechResModel(nn.Module) forward def forward(x) Arguments : x: SpeechModel Objects class SpeechModel(nn.Module) forward def forward(x) Arguments : x: hannah.models.hello.models DSConv2d Objects class DSConv2d(nn.Module) forward def forward(x) Arguments : x: DSCNNSpeechModel Objects class DSCNNSpeechModel(nn.Module) forward def forward(x) Arguments : x: DNNSpeechModel Objects class DNNSpeechModel(nn.Module) forward def forward(x) Arguments : x: hannah.models.hello hannah.models.objectdetection.loss bbox_iou def bbox_iou(box1, box2, x1y1x2y2=True, GIoU=False, DIoU=False, CIoU=False, eps=1e-7) Arguments : box1: box2: - x1y1x2y2 - (Default value = True) - GIoU - (Default value = False) - DIoU - (Default value = False) - CIoU - (Default value = False) - eps - (Default value = 1e-7) is_parallel def is_parallel(model) Arguments : model: BCEBlurWithLogitsLoss Objects class BCEBlurWithLogitsLoss(nn.Module) Arguments : eps - (Default value = 0.1) ) - # https://github.com/ultralytics/yolov3/issues/238#issuecomment-598028441# return positive: negative label smoothing BCE targetsreturn 1.0 - 0.5 * eps: 0.5 * epsclass BCEBlurWithLogitsLoss(nn.Module: ) - # https://github.com/ultralytics/yolov3/issues/238#issuecomment-598028441# return positive: forward def forward(pred, true) Arguments : pred: true: FocalLoss Objects class FocalLoss(nn.Module) forward def forward(pred, true) Arguments : pred: true: QFocalLoss Objects class QFocalLoss(nn.Module) forward def forward(pred, true) Arguments : pred: true: ComputeLoss Objects class ComputeLoss() build_targets def build_targets(p, targets) Arguments : p: targets: hannah.models.objectdetection.models FasterRCNN Objects class FasterRCNN(torch.nn.Module) forward def forward(x, y=None) Arguments : x: - y - (Default value = None) transformOutput def transformOutput(cocoGt, output, x, y) Arguments : cocoGt: output: x: y: UltralyticsYolo Objects class UltralyticsYolo(torch.nn.Module) forward def forward(x, y=None) Arguments : x: - y - (Default value = None) train def train(mode=True) Arguments : mode - (Default value = True) transformOutput def transformOutput(cocoGt, output, x, y) Arguments : cocoGt: output: x: y: hannah.models.objectdetection hannah.models.tc.models create_act def create_act(act, clipping_value) Arguments : act: clipping_value: ApproximateGlobalAveragePooling1D Objects class ApproximateGlobalAveragePooling1D(nn.Module) forward def forward(x) Arguments : x: TCResidualBlock Objects class TCResidualBlock(nn.Module) forward def forward(x) Arguments : x: TCResNetModel Objects class TCResNetModel(nn.Module) forward def forward(x) Arguments : x: ExitWrapperBlock Objects class ExitWrapperBlock(nn.Module) forward def forward(x) Arguments : x: BranchyTCResNetModel Objects class BranchyTCResNetModel(TCResNetModel) on_val def on_val() on_val_end def on_val_end() on_test def on_test() on_test_end def on_test_end() reset_stats def reset_stats() print_stats def print_stats() forward def forward(x) Arguments : x: get_loss_function def get_loss_function() hannah.models.tc hannah.models.convnet.models padding_expression def padding_expression(kernel_size, stride, dilation=1) Symbolically calculate padding such that for a given kernel_size, stride and dilation the padding is such that the output dimension is kept the same(stride=1) or halved(stride=2). Note: If the input dimension is 1 and stride = 2, the calculated padding will result in an output with also dimension 1. Parameters kernel_size : Union[int, Expression] stride : Union[int, Expression] dilation : Union[int, Expression], optional description , by default 1 Returns Expression hannah.models.convnet hannah.models.timm DefaultAnomalyDetector Objects class DefaultAnomalyDetector(nn.Module) forward def forward(x) Arguments : x: DefaultClassifierHead Objects class DefaultClassifierHead(nn.Module) forward def forward(x: torch.Tensor) -> torch.Tensor Arguments : x (torch.Tensor): Returns : Resulting torch.Tensor after applying classification DefaultProjectionHead Objects class DefaultProjectionHead(nn.Module) Default projection head for semi supervised classification learning forward def forward(x: torch.Tensor) -> torch.Tensor Forward function for default Projection Head Arguments : x - Input tensor Returns : output tensor DefaultDecoderHead Objects class DefaultDecoderHead(nn.Module) __init__ def __init__(latent_shape, input_shape) Default Decoder Head for autoencoders using TransposedConv2D Arguments : latent_shape(Tuple) - Shape (CxHxW) of the latent representation of the autoencoder input_shape(Tuple) - Shape (CxHxW) of the reconstructed image forward def forward(x) Arguments : x: TimmModel Objects class TimmModel(nn.Module) forward def forward(x: torch.Tensor) -> torch.Tensor Arguments : x - torch.Tensor: x - torch.Tensor: hannah.models hannah.models.lstm.models LSTMModel Objects class LSTMModel(nn.Module) Simple LSTM model. forward def forward(x) Arguments : x: hannah.models.lstm hannah.models.factory.reduction ReductionBlockAdd Objects class ReductionBlockAdd(nn.Module) Reduction block that sums over its inputs forward def forward(x: Tensor) -> Tensor Arguments : x - Tensor: x - Tensor: ReductionBlockConcat Objects class ReductionBlockConcat(nn.Module) Reduction block that concatenates its inputs forward def forward(x: Tensor) -> Tensor Arguments : x - Tensor: x - Tensor: hannah.models.factory.act DummyActivation Objects class DummyActivation(nn.Identity) Dummy class that instantiated to mark a missing activation. This can be used to mark requantization of activations for convolutional layers without activation functions. Arguments : hannah.models.factory hannah.models.factory.quantized Import from new loacation for backwards compatibility hannah.models.factory.qconfig Import from new loacation for backwards compatibility hannah.models.factory.network ConvNet Objects class ConvNet(nn.Module) forward def forward(x) Arguments : x: hannah.models.factory.factory A neural network model factory It allows us to construct quantized and unquantized versions of the same network, allows to explore implementation alternatives using a common neural network construction interface. NormConfig Objects @dataclass class NormConfig() BNConfig Objects @dataclass class BNConfig(NormConfig) ActConfig Objects @dataclass class ActConfig() ELUConfig Objects @dataclass class ELUConfig(ActConfig) HardtanhConfig Objects @dataclass class HardtanhConfig(ActConfig) MinorBlockConfig Objects @dataclass class MinorBlockConfig() target target Operation parallel execute block in parallel with preceding block out_channels number of output channels kernel_size kernel size of this Operation (if applicable) stride stride for this operation use padding use padding for this operation (padding will always try to keep input dimensions / stride) dilation dilation factor to use for this operation groups number of groups for this operation norm normalization to use (true uses networks default configs) act activation to use (true uses default configs) upsampling Upsampling factor for mbconv layers bias use bias for this operation out_quant use output quantization for this operation MajorBlockConfig Objects @dataclass class MajorBlockConfig() stride Union[None, int, Tuple[int, ...], Tuple[int, ...]] last Indicates wether this block is the last reduction block LinearConfig Objects @dataclass class LinearConfig() norm Union[bool, NormConfig] act Union[bool, ActConfig] NetworkConfig Objects @dataclass class NetworkConfig() NetworkFactory Objects class NetworkFactory() act def act(config: ActConfig) -> nn.Module Arguments : config - ActConfig: config - ActConfig: conv2d def conv2d(input_shape: Tuple[int, ...], out_channels: int, kernel_size: Union[int, Tuple[int, ...]], stride: Union[int, Tuple[int, ...]] = 1, padding: Union[int, Tuple[int, ...], bool] = True, dilation: Union[int, Tuple[int, ...]] = 0, groups: int = 1, norm: Union[BNConfig, bool] = False, act: Union[ActConfig, bool] = False, bias: bool = False) -> None Arguments : input_shape - Tuple[int: int: int]: out_channels - int: kernel_size - Union[int: Tuple[int, ...]]: (Default value = 1) stride - Union[int: padding - Union[int: Tuple[int, ...]: bool] - (Default value = False) dilation - int: (Default value = 0) groups - int: (Default value = 1) norm - Union[BNConfig: act - Union[ActConfig: bias - bool: (Default value = False) input_shape - Tuple[int: out_channels - int: kernel_size - Union[int: stride - Union[int: padding - Union[int: dilation - int: (Default value = 0) groups - int: (Default value = 1) norm - Union[BNConfig: act - Union[ActConfig: bias - bool: (Default value = False) mbconv1d def mbconv1d(input_shape: Tuple[int, ...], out_channels: int, kernel_size: int, dilation: int = 1, stride: int = 1, padding: Union[int, bool] = True, bias=False, upsampling: float = 1.0, groups: int = 1, norm: Union[BNConfig, bool] = False, act: Union[ActConfig, bool] = False) Arguments : input_shape - Tuple[int: int: int]: out_channels - int: kernel_size - int: dilation - int: (Default value = 1) stride - int: (Default value = 1) padding - Union[int: bool] - (Default value = False) bias - (Default value = False) upsampling - float: (Default value = 1.0) groups - int: (Default value = 1) norm - Union[BNConfig: act - Union[ActConfig: input_shape - Tuple[int: out_channels - int: kernel_size - int: dilation - int: (Default value = 1) stride - int: (Default value = 1) padding - Union[int: upsampling - float: (Default value = 1.0) groups - int: (Default value = 1) norm - Union[BNConfig: act - Union[ActConfig: conv1d def conv1d(input_shape: Tuple[int, ...], out_channels: int, kernel_size: int, stride: int = 1, bias: bool = False, padding: Union[int, bool] = True, dilation: int = 1, groups: int = 1, norm: Union[BNConfig, bool] = False, act: Union[ActConfig, bool] = False, out_quant: bool = True) -> Tuple[Tuple[int, ...], nn.Module] Arguments : input_shape - Tuple[int: int: int]: out_channels - int: kernel_size - int: stride - int: (Default value = 1) bias - bool: (Default value = False) padding - Union[int: bool] - (Default value = False) dilation - int: (Default value = 1) groups - int: (Default value = 1) norm - Union[BNConfig: act - Union[ActConfig: out_quant - bool: (Default value = True) input_shape - Tuple[int: out_channels - int: kernel_size - int: stride - int: (Default value = 1) bias - bool: (Default value = False) padding - Union[int: dilation - int: (Default value = 1) groups - int: (Default value = 1) norm - Union[BNConfig: act - Union[ActConfig: out_quant - bool: (Default value = True) minor def minor(input_shape, config: MinorBlockConfig, major_stride=None) Arguments : input_shape: - config - MinorBlockConfig: - major_stride - (Default value = None) - config - MinorBlockConfig: forward def forward(input_shape: Tuple[int, ...], config: MajorBlockConfig) Create a forward neural network block without parallelism If parallel is set to [True, False, True, False] Input: ------->|---> parallel: False ---> parallel: False ---> | --> output Arguments : input_shape - Tuple[int, ...]: config - MajorBlockConfig: input_shape - Tuple[int, ...]: config - MajorBlockConfig: residual def residual(input_shape: Tuple[int, ...], config: MajorBlockConfig) Create a neural network block with with residual parallelism If parallel is set to [True, False, True, False] |---> parallel: True ---> parallel: True ---> | Input: ------->| +---> |---> parallel: False ---> parallel: False ---> | If the major block does change the output dimensions compared to the input and one of the branches does not contain any layers, we infer 1x1 conv of maximum group size (gcd (input_channels, output_channels)) to do the downsampling. Arguments : input_shape - Tuple[int, ...]: config - MajorBlockConfig: input_shape - Tuple[int, ...]: config - MajorBlockConfig: input def input(in_channels: int, config: MajorBlockConfig) Create a neural network block with input parallelism If parallel is set to [True, False, True, False] |---> parallel: True ---> | |---> parallel: True ---> + -----------------> | Input:--------->| +---> |---> parallel: False ---> parallel: False ---> | If there are no parallel branches in the network. The major block is a standard feed forward layer. Arguments : in_channels - int: config - MajorBlockConfig: in_channels - int: config - MajorBlockConfig: full def full(in_channels: int, config: MajorBlockConfig) Create a neural network block with full parallelism If parallel is set to [True, False, True, False] |---> parallel: True ---------------------------------- -| Input:--->| +---> | |--> parallel: False --->| | |---> parallel: False ----> | +--->| |--> parallel: True ---->| If there are no parallel blocks the block is a standard feed forward network. Arguments : in_channels - int: config - MajorBlockConfig: in_channels - int: config - MajorBlockConfig: major def major(input_shape, config: MajorBlockConfig) Arguments : input_shape: - config - MajorBlockConfig: - config - MajorBlockConfig: linear def linear(input_shape, config: LinearConfig) Arguments : input_shape: - config - LinearConfig: - config - LinearConfig: identity def identity() -> Identity network def network(input_shape, labels: int, network_config: Union[ListConfig, DictConfig]) Arguments : input_shape: - labels - int: - network_config - NetworkConfig: - labels - int: - network_config - NetworkConfig: create_cnn def create_cnn(input_shape: Sequence[int], labels: int, name: str, conv: Optional[List[MajorBlockConfig]] = None, linear: Optional[List[LinearConfig]] = None, norm: Optional[NormConfig] = None, act: Optional[ActConfig] = None, qconfig: Any = None, dropout: float = 0.5) Arguments : input_shape - Sequence[int]: labels - int: name - str: conv - Optional[List[MajorBlockConfig]]: (Default value = None) linear - Optional[List[LinearConfig]]: (Default value = None) norm - Optional[NormConfig]: (Default value = None) act - Optional[ActConfig]: (Default value = None) qconfig - Any: (Default value = None) dropout - float: (Default value = 0.5) input_shape - Sequence[int]: labels - int: name - str: conv - Optional[List[MajorBlockConfig]]: (Default value = None) linear - Optional[List[LinearConfig]]: (Default value = None) norm - Optional[NormConfig]: (Default value = None) act - Optional[ActConfig]: (Default value = None) qconfig - Any: (Default value = None) dropout - float: (Default value = 0.5) hannah.models.factory.pooling ApproximateGlobalAveragePooling1D Objects class ApproximateGlobalAveragePooling1D(nn.Module) A global average pooling layer, that divides by the next power of 2 instead of true number of elements forward def forward(x) Arguments : x: ApproximateGlobalAveragePooling2D Objects class ApproximateGlobalAveragePooling2D(nn.Module) A global average pooling layer, that divides by the next power of 2 instead of true number of elements forward def forward(x) Arguments : x: hannah.models.factory.qat Import from new loacation for backwards compatibility hannah.models.factory.rounding Import from new loacation for backwards compatibility hannah.models.resnet.models padding_expression def padding_expression(kernel_size, stride, dilation=1) Symbolically calculate padding such that for a given kernel_size, stride and dilation the padding is such that the output dimension is kept the same(stride=1) or halved(stride=2). Note: If the input dimension is 1 and stride = 2, the calculated padding will result in an output with also dimension 1. Parameters kernel_size : Union[int, Expression] stride : Union[int, Expression] dilation : Union[int, Expression], optional description , by default 1 Returns Expression hannah.models.resnet hannah.models.vad.models BottleneckVad Objects class BottleneckVad(nn.Module) forward def forward(x) Arguments : x: num_flat_features def num_flat_features(x) Arguments : x: SmallVad Objects class SmallVad(nn.Module) forward def forward(x) Arguments : x: num_flat_features def num_flat_features(x) Arguments : x: SimpleVad Objects class SimpleVad(nn.Module) forward def forward(x) Arguments : x: num_flat_features def num_flat_features(x) Arguments : x: BottleneckVadModel Objects class BottleneckVadModel(nn.Module) forward def forward(x) Arguments : x: SimpleVadModel Objects class SimpleVadModel(nn.Module) forward def forward(x) Arguments : x: SmallVadModel Objects class SmallVadModel(nn.Module) forward def forward(x) Arguments : x: hannah.models.vad hannah.models.ofa.submodules.elasticLinear ElasticWidthLinear Objects class ElasticWidthLinear(nn.Linear, _Elastic) forward def forward(input: torch.Tensor) -> torch.Tensor Arguments : input - torch.Tensor: input - torch.Tensor: input - torch.Tensor: input - torch.Tensor: get_basic_module def get_basic_module() ElasticQuantWidthLinear Objects class ElasticQuantWidthLinear(nn.Linear, _Elastic) filtered_weight @property def filtered_weight() filtered_bias @property def filtered_bias() scaled_weight @property def scaled_weight() scaled_bias @property def scaled_bias() forward def forward(input: torch.Tensor) -> torch.Tensor Arguments : input - torch.Tensor: input - torch.Tensor: input - torch.Tensor: input - torch.Tensor: get_basic_module def get_basic_module() from_float @classmethod def from_float(cls, mod) Create a qat module from a float module or qparams_dict Args: mod a float module, either produced by torch.quantization utilities or directly from user Arguments : mod: ElasticPermissiveReLU Objects class ElasticPermissiveReLU(nn.ReLU) forward def forward(x) Arguments : x: assemble_basic_module def assemble_basic_module() hannah.models.ofa.submodules.elasticBase _Elastic Objects class _Elastic() get_basic_module def get_basic_module() -> nn.Module get_out_channel_sizes def get_out_channel_sizes() set_out_channel_filter def set_out_channel_filter(out_channel_filter) Arguments : ) -> nn.Module: # Module: so that Sequentials are possible: like DSCreturn copy.deepcopy(self.get_basic_module(): out_channel_filter: ) -> nn.Module: # Module: like DSCreturn copy.deepcopy(self.get_basic_module(): ) -> nn.Module: # Module: like DSCreturn copy.deepcopy(self.get_basic_module(): ) -> nn.Module: # Module: like DSCreturn copy.deepcopy(self.get_basic_module())set_out_channel_filter(self: set_in_channel_filter def set_in_channel_filter(in_channel_filter) Arguments : in_channel_filter: ElasticBase1d Objects class ElasticBase1d(nn.Conv1d, _Elastic) set_bn_parameter def set_bn_parameter(conv: nn.Conv1d, tmp_bn, num_tracked) Caller for BatchNorm Parameters This unifies the call in the different methods, especially in dsc / not dsc forward And assigns the attributes in tmp_bn to the param conv Arguments : conv - nn.Conv1d: tmp_bn: num_tracked: conv - nn.Conv1d: conv - nn.Conv1d: conv - nn.Conv1d: prepare_dsc_for_validation_model def prepare_dsc_for_validation_model(conv_class: nn.Module, full_kernel, full_bias, in_channels, out_channels, grouping, stride, padding, dilation, qconfig=None, out_quant=None, bn_eps=None, bn_momentum=None, bn_caller: tuple = None) This method creates the necessary validation models for DSC. It creates the validation model as torch.Sequence of standard pytorch convolution models. The structure is analog to the DSC method do_dsc. This method can also handle quantization models. Arguments : conv_class - nn.Module: full_kernel: full_bias: in_channels: out_channels: grouping: stride: padding: dilation: # for quantqconfig: (Default value = None) out_quant - (Default value = None) bn_eps - (Default value = None) bn_momentum - (Default value = None) bn_caller - tuple: (Default value = None) conv_class - nn.Module: bn_caller - tuple: (Default value = None) conv_class - nn.Module: bn_caller - tuple: (Default value = None) conv_class - nn.Module: bn_caller - tuple: (Default value = None) do_dsc def do_dsc(input, full_kernel, full_bias, grouping, stride, padding, dilation, quant_weight=None, quant_bias=None, quant_weight_function: Function = None, quant_bias_function: Function = None) This method will perform the DSC(=Depthwise Separable Convolution). This method can also handle quantized models. DSC is done in two steps: 1. Depthwise Separable: Set Group = In_Channels, Output = k*In_Channels 2. Pointwise Convolution, with Grouping = Grouping-Param und Out_Channel = Out_Channel-Param The Params above are used for quantized models Arguments : input: full_kernel: full_bias: grouping: stride: padding: dilation: - quant_weight - (Default value = None) - quant_bias - (Default value = None) - quant_weight_function - Function: (Default value = None) # for quantizationquant_bias_function: Function: (Default value = None) # for quantization: - quant_weight_function - Function: (Default value = None) # for quantizationquant_bias_function: Function: (Default value = None) - quant_weight_function - Function: (Default value = None) # for quantizationquant_bias_function: Function: (Default value = None) - quant_weight_function - Function: (Default value = None) # for quantizationquant_bias_function: Function: (Default value = None) set_in_and_out_channel def set_in_and_out_channel(kernel, filtered: bool = True) This method uses the kernel for setting the input and outputchannel if dynamic width is activated (channelfilters), the amount of channels is reduced, hence we can't use the initial values (self.(in/out)_channel) of the constructor This method sets the self.(in/out)_channel value to the right amount of channels extracted from the kernel that will be used. if filtered is False, the self.initial_(in/out)_channels will be used. The previous values will be stored in the attribute prev_in_channels and prev_out_channels. Arguments : kernel: - filtered - bool: (Default value = True) - filtered - bool: (Default value = True) - filtered - bool: (Default value = True) - filtered - bool: (Default value = True) reset_in_and_out_channel_to_previous def reset_in_and_out_channel_to_previous() Analog to set_in_and_out_channels: Resets the in and out_channels Arguments : set_kernel_size def set_kernel_size(new_kernel_size) If the requested kernel size is outside of the min/max range, clamp it to the min/max range. If the requested kernel size is not an available kernel size, default to the max kernel size Arguments : new_kernel_size - int): the size of the kernel you want to use reset_kernel_size def reset_kernel_size() step_down_kernel_size def step_down_kernel_size() pick_kernel_index def pick_kernel_index(target_kernel_index: int) Arguments : target_kernel_index - int: target_kernel_index - int: target_kernel_index - int: target_kernel_index - int: get_available_kernel_steps def get_available_kernel_steps() get_full_width_kernel def get_full_width_kernel() It applies the kernel transformations to the kernel until the target kernel index is reached Returns : The found target kernel. Args: Returns: get_kernel def get_kernel() If the input and output channels are not filtered, the full kernel is Arguments : Returns : return: The new kernel and bias. set_dilation_size def set_dilation_size(new_dilation_size) Arguments : new_dilation_size: update_padding def update_padding() reset_dilation_size def reset_dilation_size() step_down_dilation_size def step_down_dilation_size() pick_dilation_index def pick_dilation_index(target_dilation_index: int) Arguments : target_dilation_index - int: target_dilation_index - int: target_dilation_index - int: target_dilation_index - int: get_available_dilation_steps def get_available_dilation_steps() get_available_grouping_steps def get_available_grouping_steps() get_available_dsc_steps def get_available_dsc_steps() get_dilation_size def get_dilation_size() pick_group_index def pick_group_index(target_group_index: int) Arguments : target_group_index - int: target_group_index - int: target_group_index - int: target_group_index - int: pick_dsc_index def pick_dsc_index(target_dsc_index: int) Arguments : target_dsc_index - int: target_dsc_index - int: target_dsc_index - int: target_dsc_index - int: reset_group_size def reset_group_size() reset_dscs def reset_dscs() get_group_size def get_group_size() get_dsc def get_dsc() set_group_size def set_group_size(new_group_size) Arguments : new_group_size: set_dsc def set_dsc(new_dsc) Arguments : new_dsc: step_down_group_size def step_down_group_size() step_down_dsc def step_down_dsc() adjust_weights_for_grouping def adjust_weights_for_grouping(weights, input_divided_by) Arguments : weights: input_divided_by: forward def forward(input: torch.Tensor) -> torch.Tensor Arguments : input - torch.Tensor: input - torch.Tensor: input - torch.Tensor: input - torch.Tensor: extra_repr def extra_repr() hannah.models.ofa.submodules.resblock ResBlockBase Objects class ResBlockBase(nn.Module) forward def forward(x) Arguments : x: get_nested_modules def get_nested_modules() ResBlock1d Objects class ResBlock1d(ResBlockBase) forward def forward(x) Arguments : x: get_input_layer def get_input_layer() get_output_layer def get_output_layer() create_internal_channelhelper def create_internal_channelhelper() hannah.models.ofa.submodules hannah.models.ofa.submodules.elasticBatchnorm ElasticWidthBatchnorm1d Objects class ElasticWidthBatchnorm1d(nn.BatchNorm1d) forward def forward(input: torch.Tensor) -> torch.Tensor if self.track_running_stats: logging.warn( \"ElasticWidthBatchnorm with tracked running stats currently not fully implemented!\" ) num_batches_tracked and exponential averaging are currently not implemented. Arguments : input - torch.Tensor: input - torch.Tensor: input - torch.Tensor: input - torch.Tensor: get_basic_batchnorm1d def get_basic_batchnorm1d() assemble_basic_module def assemble_basic_module() -> nn.BatchNorm1d hannah.models.ofa.submodules.elasticchannelhelper ElasticChannelHelper Objects class ElasticChannelHelper(nn.Module) compute_channel_priorities def compute_channel_priorities() set_channel_filter def set_channel_filter() apply_filter_to_module def apply_filter_to_module(module, is_target: bool) Arguments : module: - is_target - bool: - is_target - bool: - is_target - bool: - is_target - bool: step_down_channels def step_down_channels() set_channel_step def set_channel_step(step: int) Arguments : step - int: step - int: step - int: step - int: reset_channel_step def reset_channel_step() set_primary_target def set_primary_target(target: nn.Module) Arguments : target - nn.Module: target - nn.Module: target - nn.Module: target - nn.Module: is_valid_primary_target def is_valid_primary_target(module: nn.Module) -> bool Arguments : module - nn.Module: module - nn.Module: module - nn.Module: module - nn.Module: is_primary_target def is_primary_target(module: nn.Module) -> bool Arguments : module - nn.Module: module - nn.Module: module - nn.Module: module - nn.Module: add_secondary_targets def add_secondary_targets(target: nn.Module) Arguments : target - nn.Module: target - nn.Module: target - nn.Module: target - nn.Module: add_secondary_target_item def add_secondary_target_item(target: nn.Module) Arguments : target - nn.Module: target - nn.Module: target - nn.Module: target - nn.Module: add_sources def add_sources(source: nn.Module) Arguments : source - nn.Module: source - nn.Module: source - nn.Module: source - nn.Module: add_targets def add_targets(target: nn.Module) Arguments : target - nn.Module: target - nn.Module: target - nn.Module: target - nn.Module: add_source_item def add_source_item(source: nn.Module) Arguments : source - nn.Module: source - nn.Module: source - nn.Module: source - nn.Module: discover_target def discover_target(new_target: nn.Module) Arguments : new_target - nn.Module: new_target - nn.Module: new_target - nn.Module: new_target - nn.Module: get_available_width_steps def get_available_width_steps() hannah.models.ofa.submodules.elastickernelconv ElasticConv1d Objects class ElasticConv1d(ElasticBase1d) forward def forward(input: torch.Tensor) -> torch.Tensor Arguments : input - torch.Tensor: input - torch.Tensor: input - torch.Tensor: input - torch.Tensor: get_basic_module def get_basic_module() -> nn.Module ElasticConvReLu1d Objects class ElasticConvReLu1d(ElasticBase1d) forward def forward(input: torch.Tensor) -> torch.Tensor Arguments : input - torch.Tensor: input - torch.Tensor: input - torch.Tensor: input - torch.Tensor: get_basic_module def get_basic_module() -> nn.Module ElasticConvBn1d Objects class ElasticConvBn1d(ElasticConv1d) forward def forward(input: torch.Tensor) -> torch.Tensor Arguments : input - torch.Tensor: input - torch.Tensor: input - torch.Tensor: input - torch.Tensor: get_basic_module def get_basic_module() -> nn.Module ElasticConvBnReLu1d Objects class ElasticConvBnReLu1d(ElasticConvBn1d) forward def forward(input: torch.Tensor) -> torch.Tensor Arguments : input - torch.Tensor: input - torch.Tensor: input - torch.Tensor: input - torch.Tensor: get_basic_module def get_basic_module() -> nn.Module ConvRelu1d Objects class ConvRelu1d(nn.Conv1d) forward def forward(input: torch.Tensor) -> torch.Tensor Arguments : input - torch.Tensor: input - torch.Tensor: input - torch.Tensor: input - torch.Tensor: ConvBn1d Objects class ConvBn1d(nn.Conv1d) forward def forward(input: torch.Tensor) -> torch.Tensor Arguments : input - torch.Tensor: input - torch.Tensor: input - torch.Tensor: input - torch.Tensor: ConvBnReLu1d Objects class ConvBnReLu1d(ConvBn1d) forward def forward(input: torch.Tensor) -> torch.Tensor Arguments : input - torch.Tensor: input - torch.Tensor: input - torch.Tensor: input - torch.Tensor: hannah.models.ofa.submodules.elasticquantkernelconv QuadDataHelper Objects class QuadDataHelper() Data Container so that _forward and _dsc has the same data. _ElasticConvBnNd Objects class _ElasticConvBnNd(ElasticBase1d, qat._ConvForwardMixin) on_warmup_end def on_warmup_end() reset_running_stats def reset_running_stats() reset_bn_parameters def reset_bn_parameters() reset_parameters def reset_parameters() update_bn_stats def update_bn_stats() freeze_bn_stats def freeze_bn_stats() scale_factor @property def scale_factor() full_scale_factor @property def full_scale_factor() does the same as scale_factor but uses the whole kernel. Used for dsc scaled_weight @property def scaled_weight() get_full_kernel_bias def get_full_kernel_bias() Gets the full kernel and bias. Used for dsc extra_repr def extra_repr() forward def forward(input) Arguments : input: train def train(mode=True) Batchnorm's training behavior is using the self.training flag. Prevent changing it if BN is frozen. This makes sure that calling model.train() on a model with a frozen BN will behave properly. Arguments : mode - (Default value = True) from_float @classmethod def from_float(cls, mod) Create a qat module from a float module or qparams_dict Args: mod a float module, either produced by torch.quantization utilities or directly from user Arguments : mod: ElasticQuantConv1d Objects class ElasticQuantConv1d(ElasticBase1d, qat._ConvForwardMixin) forward def forward(input: torch.Tensor) -> torch.Tensor Arguments : input - torch.Tensor: input - torch.Tensor: input - torch.Tensor: input - torch.Tensor: get_basic_module def get_basic_module() -> nn.Module ElasticQuantConvReLu1d Objects class ElasticQuantConvReLu1d(ElasticBase1d, qat._ConvForwardMixin) forward def forward(input: torch.Tensor) -> torch.Tensor Arguments : input - torch.Tensor: input - torch.Tensor: input - torch.Tensor: input - torch.Tensor: get_basic_module def get_basic_module() -> nn.Module ElasticQuantConvBn1d Objects class ElasticQuantConvBn1d(_ElasticConvBnNd) forward def forward(input: torch.Tensor) -> torch.Tensor Arguments : input - torch.Tensor: input - torch.Tensor: input - torch.Tensor: input - torch.Tensor: get_basic_module def get_basic_module() -> nn.Module ElasticQuantConvBnReLu1d Objects class ElasticQuantConvBnReLu1d(ElasticQuantConvBn1d) forward def forward(input: torch.Tensor) -> torch.Tensor Arguments : input - torch.Tensor: input - torch.Tensor: input - torch.Tensor: input - torch.Tensor: get_basic_module def get_basic_module() -> nn.Module hannah.models.ofa.models create def create(name: str, labels: int, input_shape, conv=[], min_depth: int = 1, norm_before_act=True, skew_sampling_distribution: bool = False, dropout: int = 0.5, validate_on_extracted=True, qconfig=None) -> nn.Module The function creates a ofa Model with the given name, labels, input shape, convolutional layers, and other parameters Arguments : name(str) - The name of the model labels(int) - The number of classes in the dataset input_shape - the shape of the input tensor conv - a list of MajorBlockConfig objects (Default value = []) min_depth(int optional - The minimum depth of the model, defaults to 1 norm_before_act - If True, the normalization is performed before the activation function, defaults to True (optional) skew_sampling_distribution(bool optional - If True, the model will use a skewed sampling distribution to sample the number of minor blocks in each major block, defaults to False dropout(int) - float, default 0.5 validate_on_extracted - If True, the model will be validated on the extracted data, defaults to True (optional) qconfig - the quantization configuration to use (Default value = None) name - str: labels - int: min_depth - int: (Default value = 1) skew_sampling_distribution - bool: (Default value = False) dropout - int: (Default value = 0.5) name - str: labels - int: min_depth - int: (Default value = 1) skew_sampling_distribution - bool: (Default value = False) dropout - int: (Default value = 0.5) name - str: labels - int: min_depth - int: (Default value = 1) skew_sampling_distribution - bool: (Default value = False) dropout - int: (Default value = 0.5) Returns : A model object. create_minor_block_sequence def create_minor_block_sequence(blocks, in_channels, stride=1, norm_before_act=True, qconfig=None) Arguments : blocks: in_channels: - stride - (Default value = 1) - norm_before_act - (Default value = True) - qconfig - (Default value = None) # sources: List[nn.Module]: (Default value = [nn.ModuleList([])]) # sources: List[nn.Module]: (Default value = [nn.ModuleList([])]) # sources: List[nn.Module]: (Default value = [nn.ModuleList([])]) create_minor_block def create_minor_block(block_config, in_channels: int, stride: int = 1, norm_before_act=True, sources: List[nn.ModuleList] = [nn.ModuleList([])], qconfig=None) -> Tuple[nn.Module, int] Arguments : block_config: - in_channels - int: - stride - int: (Default value = 1) - norm_before_act - (Default value = True) - sources - List[nn.ModuleList]: (Default value = [nn.ModuleList([])]) - qconfig - (Default value = None) - in_channels - int: - stride - int: (Default value = 1) - sources - List[nn.ModuleList]: (Default value = [nn.ModuleList([])]) - in_channels - int: - stride - int: (Default value = 1) - sources - List[nn.ModuleList]: (Default value = [nn.ModuleList([])]) create_residual_block_1d def create_residual_block_1d(blocks, in_channels, stride=1, norm_before_act=None, qconfig=None, quant_skip=None) -> ResBlock1d Arguments : blocks: in_channels: - stride - (Default value = 1) - norm_before_act - (Default value = None) - qconfig - (Default value = None) - quant_skip - (Default value = None) # sources: List[nn.ModuleList]: (Default value = [nn.ModuleList([])]) # sources: List[nn.ModuleList]: (Default value = [nn.ModuleList([])]) # sources: List[nn.ModuleList]: (Default value = [nn.ModuleList([])]) OFAModel Objects class OFAModel(nn.Module) extract_conv def extract_conv(conv, general_config, parallel=True, bp=False) Arguments : conv: general_config: - parallel - (Default value = True) - bp - (Default value = False) extract_config def extract_config(conv, general_config, bp=False) set all conv attributes in config Arguments : conv: general_config: - bp - (Default value = False) print_config def print_config(filename) Arguments : filename: forward def forward(x) Arguments : x: perform_sequence_discovery def perform_sequence_discovery() get_post_conv def get_post_conv(post_block) Arguments : post_block: get_pre_conv def get_pre_conv(pre_block) Arguments : pre_block: sample_subnetwork def sample_subnetwork() get_random_step def get_random_step(upper_bound: int) -> int Arguments : upper_bound - int: upper_bound - int: upper_bound - int: get_max_submodel_steps def get_max_submodel_steps() get_submodel def get_submodel(state: dict) Arguments : state - dict: state - dict: state - dict: on_warmup_end def on_warmup_end() set_submodel def set_submodel(state: dict) Arguments : state - dict: state - dict: state - dict: build_validation_model def build_validation_model() reset_validation_model def reset_validation_model() get_validation_model_weight_count def get_validation_model_weight_count() extract_elastic_depth_sequence def extract_elastic_depth_sequence(target_depth, quantized=False, clone_mode=False) Arguments : target_depth: - quantized - (Default value = False) - clone_mode - (Default value = False) extract_module_from_depth_step def extract_module_from_depth_step(depth_step) -> nn.Module Arguments : depth_step: get_elastic_depth_output def get_elastic_depth_output(target_depth=None, quantized=False) Arguments : target_depth - (Default value = None) quantized - (Default value = False) step_down_all_channels def step_down_all_channels() reset_active_depth def reset_active_depth() resume_active_elastic_values def resume_active_elastic_values() update_output_channel_count def update_output_channel_count() get_output_linear_layer def get_output_linear_layer(target_depth) Arguments : target_depth: step_down_all_kernels def step_down_all_kernels() reset_all_kernel_sizes def reset_all_kernel_sizes() reset_all_dilation_sizes def reset_all_dilation_sizes() reset_all_group_sizes def reset_all_group_sizes() reset_all_dsc def reset_all_dsc() step_down_all_dilations def step_down_all_dilations() step_down_all_groups def step_down_all_groups() step_down_all_dsc def step_down_all_dsc() go_to_kernel_step def go_to_kernel_step(step: int) Arguments : step - int: step - int: step - int: resume_kernel_sizes_from_step def resume_kernel_sizes_from_step() reset_all_widths def reset_all_widths() progressive_shrinking_add_kernel def progressive_shrinking_add_kernel() progressive_shrinking_add_dilation def progressive_shrinking_add_dilation() progressive_shrinking_add_depth def progressive_shrinking_add_depth() progressive_shrinking_add_group def progressive_shrinking_add_group() progressive_shrinking_add_dsc def progressive_shrinking_add_dsc() progressive_shrinking_compute_channel_priorities def progressive_shrinking_compute_channel_priorities() progressive_shrinking_add_width def progressive_shrinking_add_width() progressive_shrinking_disable_sampling def progressive_shrinking_disable_sampling() reset_shrinking def reset_shrinking() rebuild_extracted_blocks def rebuild_extracted_blocks(blocks) Arguments : blocks: hannah.models.ofa hannah.models.ofa.type_utils hannah.models.ofa.utilities conv1d_auto_padding def conv1d_auto_padding(conv1d: nn.Conv1d) Arguments : conv1d - nn.Conv1d: conv1d - nn.Conv1d: conv1d - nn.Conv1d: conv1d - nn.Conv1d: conv1d_get_padding def conv1d_get_padding(kernel_size, dilation=1) Arguments : kernel_size: - dilation - (Default value = 1) sub_filter_start_end def sub_filter_start_end(kernel_size, sub_kernel_size) Arguments : kernel_size: sub_kernel_size: flatten_module_list def flatten_module_list(modules: nn.Module) -> nn.Module Arguments : modules - nn.Module: modules - nn.Module: modules - nn.Module: modules - nn.Module: module_list_to_module def module_list_to_module(module_list) Arguments : module_list: call_function_from_deep_nested def call_function_from_deep_nested(input, function, type_selection: type = None) Arguments : input: function: - type_selection - type: (Default value = None) - type_selection - type: (Default value = None) - type_selection - type: (Default value = None) - type_selection - type: (Default value = None) get_instances_from_deep_nested def get_instances_from_deep_nested(input, type_selection: type = None) Arguments : input: - type_selection - type: (Default value = None) - type_selection - type: (Default value = None) - type_selection - type: (Default value = None) - type_selection - type: (Default value = None) filter_primary_module_weights def filter_primary_module_weights(weights, in_channel_filter, out_channel_filter) Arguments : weights: in_channel_filter: out_channel_filter: filter_single_dimensional_weights def filter_single_dimensional_weights(weights, channel_filter) Arguments : weights: channel_filter: make_parameter def make_parameter(t: torch.Tensor) -> nn.Parameter Arguments : t - torch.Tensor: t - torch.Tensor: t - torch.Tensor: t - torch.Tensor: adjust_weight_if_needed def adjust_weight_if_needed(module, kernel=None, groups=None) Adjust the weight if the adjustment is needded. This means, if the kernel does not have the size of (out_channel, in_channel / group, kernel). Arguments : kernel - the kernel that should be checked and adjusted if needed. If None module.weight.data will be used (Default value = None) grouping - value of the conv, if None module.groups will be used module - the conv :throws - RuntimeError if there is no last_grouping_param for comporing current group value to past group value returns (kernel, is adjusted) (adjusted if needed) otherwise throws a RuntimeError groups - (Default value = None) is_weight_adjusting_needed def is_weight_adjusting_needed(weights, input_channels, groups) Checks if a weight adjustment is needed Requirement: weight.shape[1] must be input_channels/groups true: weight adjustment is needed Arguments : weights - the weights that needs to be checked input_channels - Input Channels of the Convolution Module groups - Grouping Param of the Convolution Module get_target_weight def get_target_weight(weights, input_channels, groups) Gives the targeted weight shape (out_channel, in_channel // groups, kernel) Arguments : weights - the weights that needs to be checked input_channels - Input Channels of the Convolution Module groups - Grouping Param of the Convolution Module prepare_kernel_for_depthwise_separable_convolution def prepare_kernel_for_depthwise_separable_convolution(model, kernel, bias, in_channels) Prepares the kernel for depthwise separable convolution (step 1 of DSC). This means setting groups = inchannels and outchannels = k * inchannels. Arguments : model: kernel: bias: in_channels: Returns : kernel, bias) Tuple prepare_kernel_for_pointwise_convolution def prepare_kernel_for_pointwise_convolution(kernel, grouping) Prepares the kernel for pointwise convolution (step 2 of DSC). This means setting the kernel window to 1x1. So a kernel with output_channel, input_channel / groups, kernel will be set to ( , ,1) Arguments : kernel: grouping: adjust_weights_for_grouping def adjust_weights_for_grouping(weights, input_divided_by) Adjusts the Weights for the Forward of the Convulution Shape(outchannels, inchannels / group, kW) weight \u2013 filters of shape (out_channels , in_channels / groups , kW) input_divided_by Arguments : weights: input_divided_by: get_kernel_for_dsc def get_kernel_for_dsc(kernel) Part of DSC (Step 2, pointwise convolution) kernel with output_channel, input_channel / groups, kernel will be set to ( , ,1) Arguments : kernel: get_channel_filter def get_channel_filter(current_channel_size, reduced_target_channel_size, channel_priority_list) Arguments : current_channel_size: reduced_target_channel_size: channel_priority_list: create_channel_filter def create_channel_filter(module: nn.Module, kernel, current_channel, reduced_target_channel_size, is_output_filter: bool = True) Arguments : module - nn.Module: kernel: current_channel: reduced_target_channel_size: is_output_filter - bool: (Default value = True) module - nn.Module: is_output_filter - bool: (Default value = True) module - nn.Module: is_output_filter - bool: (Default value = True) module - nn.Module: is_output_filter - bool: (Default value = True) compute_channel_priorities def compute_channel_priorities(module: nn.Module, kernel, channel_index: int = 0) Arguments : module - nn.Module: kernel: channel_index - int: (Default value = 0) module - nn.Module: channel_index - int: (Default value = 0) module - nn.Module: channel_index - int: (Default value = 0) module - nn.Module: channel_index - int: (Default value = 0) hannah.models.ekut.models conv_bn def conv_bn(inp, oup, stride) Arguments : inp: oup: stride: conv_1x1_bn def conv_1x1_bn(inp, oup) Arguments : inp: oup: InvertedResidual Objects class InvertedResidual(nn.Module) forward def forward(x) Arguments : x: RawSpeechModel Objects class RawSpeechModel(nn.Module) Speech Recognition on RAW Data using Wolfgang Fuhls Networks forward def forward(x) Arguments : x: RawSpeechModelInvertedResidual Objects class RawSpeechModelInvertedResidual(nn.Module) forward def forward(x) Arguments : x: hannah.models.ekut hannah.models.utils next_power_of2 def next_power_of2(x) Arguments : x: hannah.models.wavenet.models Conv Objects class Conv(nn.Module) A convolution with the option to be causal and use xavier initialization forward def forward(signal) Arguments : signal: WaveNet Objects class WaveNet(nn.Module) forward def forward(input_data) Arguments : input_data: hannah.models.wavenet hannah.models._vendor hannah.models._vendor.focalnet Mlp Objects class Mlp(nn.Module) forward def forward(x) Arguments : x: FocalModulation Objects class FocalModulation(nn.Module) forward def forward(x) Arguments : x - input features with shape of (B, H, W, C) extra_repr def extra_repr() -> str flops def flops(N) calculate flops for 1 window with token length of N Arguments : N: FocalNetBlock Objects class FocalNetBlock(nn.Module) Focal Modulation Network Block. Arguments : dim(int) - Number of input channels. input_resolution(tuple[int]) - Input resulotion. mlp_ratio(float) - Ratio of mlp hidden dim to embedding dim. drop(float) - Dropout rate. Default: 0.0 drop_path(float) - Stochastic depth rate. Default: 0.0 act_layer(nn.Module) - Activation layer. Default: nn.GELU norm_layer(nn.Module) - Normalization layer. Default: nn.LayerNorm focal_level(int) - Number of focal levels. focal_window(int) - Focal window size at first focal level use_layerscale(bool) - Whether use layerscale layerscale_value(float) - Initial layerscale value use_postln(bool) - Whether use layernorm after modulation forward def forward(x) Arguments : x: extra_repr def extra_repr() -> str flops def flops() BasicLayer Objects class BasicLayer(nn.Module) A basic Focal Transformer layer for one stage. Arguments : dim(int) - Number of input channels. input_resolution(tuple[int]) - Input resolution. depth(int) - Number of blocks. window_size(int) - Local window size. mlp_ratio(float) - Ratio of mlp hidden dim to embedding dim. qkv_bias(bool) - If True, add a learnable bias to query, key, value. Default: True qk_scale(float | None): Override default qk scale of head_dim ** -0.5 if set. drop(float) - Dropout rate. Default: 0.0 drop_path(float | tuple[float]): Stochastic depth rate. Default: 0.0 norm_layer(nn.Module) - Normalization layer. Default: nn.LayerNorm downsample(nn.Module | None): Downsample layer at the end of the layer. Default: None use_checkpoint(bool) - Whether to use checkpointing to save memory. Default: False. focal_level(int) - Number of focal levels focal_window(int) - Focal window size at first focal level use_layerscale(bool) - Whether use layerscale layerscale_value(float) - Initial layerscale value use_postln(bool) - Whether use layernorm after modulation forward def forward(x, H, W) Arguments : x: H: W: extra_repr def extra_repr() -> str flops def flops() PatchEmbed Objects class PatchEmbed(nn.Module) Image to Patch Embedding Arguments : img_size(int) - Image size. Default: 224. patch_size(int) - Patch token size. Default: 4. in_chans(int) - Number of input image channels. Default: 3. embed_dim(int) - Number of linear projection output channels. Default: 96. norm_layer(nn.Module) - Normalization layer. Default: None forward def forward(x) Arguments : x: flops def flops() FocalNet Objects class FocalNet(nn.Module) Focal Modulation Networks (FocalNets) Arguments : img_size(int | tuple(int): Input image size. Default 224 patch_size(int | tuple(int): Patch size. Default: 4 - in_chans(int) - Number of input image channels. Default: 3 - num_classes(int) - Number of classes for classification head. Default: 1000 - embed_dim(int) - Patch embedding dimension. Default: 96 - depths(tuple(int) - Depth of each Focal Transformer layer. - mlp_ratio(float) - Ratio of mlp hidden dim to embedding dim. Default: 4 - drop_rate(float) - Dropout rate. Default: 0 - drop_path_rate(float) - Stochastic depth rate. Default: 0.1 - norm_layer(nn.Module) - Normalization layer. Default: nn.LayerNorm. - patch_norm(bool) - If True, add normalization after patch embedding. Default: True - use_checkpoint(bool) - Whether to use checkpointing to save memory. Default: False - focal_levels(list) - How many focal levels at all stages. Note that this excludes the finest-grain level. Default: [1, 1, 1, 1] - focal_windows(list) - The focal window size at all stages. Default: [7, 5, 3, 1] - use_conv_embed(bool) - Whether use convolutional embedding. We noted that using convolutional embedding usually improve the performance, but we do not use it by default. Default: False - use_layerscale(bool) - Whether use layerscale proposed in CaiT. Default: False - layerscale_value(float) - Value for layer scale. Default: 1e-4 - use_postln(bool) - Whether use layernorm after modulation (it helps stablize training of large models) no_weight_decay @torch.jit.ignore def no_weight_decay() no_weight_decay_keywords @torch.jit.ignore def no_weight_decay_keywords() forward_features def forward_features(x) Arguments : x: forward def forward(x) Arguments : x: flops def flops() build_transforms def build_transforms(img_size, center_crop=False) Arguments : img_size: - center_crop - (Default value = False) build_transforms4display def build_transforms4display(img_size, center_crop=False) Arguments : img_size: - center_crop - (Default value = False) focalnet_tiny_srf @register_model def focalnet_tiny_srf(pretrained=False, **kwargs) Arguments : pretrained - (Default value = False) **kwargs: focalnet_small_srf @register_model def focalnet_small_srf(pretrained=False, **kwargs) Arguments : pretrained - (Default value = False) **kwargs: focalnet_base_srf @register_model def focalnet_base_srf(pretrained=False, **kwargs) Arguments : pretrained - (Default value = False) **kwargs: focalnet_tiny_lrf @register_model def focalnet_tiny_lrf(pretrained=False, **kwargs) Arguments : pretrained - (Default value = False) **kwargs: focalnet_small_lrf @register_model def focalnet_small_lrf(pretrained=False, **kwargs) Arguments : pretrained - (Default value = False) **kwargs: focalnet_base_lrf @register_model def focalnet_base_lrf(pretrained=False, **kwargs) Arguments : pretrained - (Default value = False) **kwargs: focalnet_tiny_iso_16 @register_model def focalnet_tiny_iso_16(pretrained=False, **kwargs) Arguments : pretrained - (Default value = False) **kwargs: focalnet_small_iso_16 @register_model def focalnet_small_iso_16(pretrained=False, **kwargs) Arguments : pretrained - (Default value = False) **kwargs: focalnet_base_iso_16 @register_model def focalnet_base_iso_16(pretrained=False, **kwargs) Arguments : pretrained - (Default value = False) **kwargs: focalnet_large_fl3 @register_model def focalnet_large_fl3(pretrained=False, **kwargs) Arguments : pretrained - (Default value = False) **kwargs: focalnet_large_fl4 @register_model def focalnet_large_fl4(pretrained=False, **kwargs) Arguments : pretrained - (Default value = False) **kwargs: focalnet_xlarge_fl3 @register_model def focalnet_xlarge_fl3(pretrained=False, **kwargs) Arguments : pretrained - (Default value = False) **kwargs: focalnet_xlarge_fl4 @register_model def focalnet_xlarge_fl4(pretrained=False, **kwargs) Arguments : pretrained - (Default value = False) **kwargs: focalnet_huge_fl3 @register_model def focalnet_huge_fl3(pretrained=False, **kwargs) Arguments : pretrained - (Default value = False) **kwargs: focalnet_huge_fl4 @register_model def focalnet_huge_fl4(pretrained=False, **kwargs) Arguments : pretrained - (Default value = False) **kwargs: hannah.models.sinc.models GDSConv Objects class GDSConv(nn.Module) forward def forward(x) Arguments : x: GDSConvBlock Objects class GDSConvBlock(nn.Module) forward def forward(x) Arguments : x: SincNet Objects class SincNet(nn.Module) forward def forward(x) Arguments : x: hannah.models.sinc hannah.models.capsule_net.models hannah.models.capsule_net hannah.models.capsule_net.operators hannah.models.capsule_net.expressions padding_expression def padding_expression(kernel_size, stride, dilation=1) Symbolically calculate padding such that for a given kernel_size, stride and dilation the padding is such that the output dimension is kept the same(stride=1) or halved(stride=2). Note: If the input dimension is 1 and stride = 2, the calculated padding will result in an output with also dimension 1. Parameters kernel_size : Union[int, Expression] stride : Union[int, Expression] dilation : Union[int, Expression], optional description , by default 1 Returns Expression hannah.models.capsule_net.utils hannah.test_linear_classifier hannah.callbacks.dump_layers TestDumperCallback Objects class TestDumperCallback(Callback) on_test_start def on_test_start(pl_trainer, pl_model) Arguments : pl_trainer: pl_model: hannah.callbacks.backends symbolic_batch_dim def symbolic_batch_dim(model) Arguments : model: InferenceBackendBase Objects class InferenceBackendBase(Callback) Base class to run val and test on a backend inference engine run_batch def run_batch(inputs=None) Arguments : inputs - (Default value = None) prepare def prepare(module) Arguments : module: on_validation_epoch_start def on_validation_epoch_start(trainer, pl_module) Arguments : trainer: pl_module: on_validation_batch_end def on_validation_batch_end(trainer, pl_module, outputs, batch, batch_idx, dataloader_idx) Arguments : trainer: pl_module: outputs: batch: batch_idx: dataloader_idx: on_validation_epoch_end def on_validation_epoch_end(trainer, pl_module) Arguments : trainer: pl_module: on_test_epoch_start def on_test_epoch_start(trainer, pl_module) Arguments : trainer: pl_module: quantize def quantize(pl_module: torch.nn.Module) -> torch.nn.Module Arguments : pl_module - torch.nn.Module to quantize Returns - quantized torch.nn.Module on_test_batch_end def on_test_batch_end(trainer, pl_module, outputs, batch, batch_idx, dataloader_idx) Arguments : trainer: pl_module: outputs: batch: batch_idx: dataloader_idx: export def export() -> None Export the model through the target backend TorchMobileBackend Objects class TorchMobileBackend(InferenceBackendBase) Inference backend for torch mobile prepare def prepare(model) Arguments : model torch.nn.Module - nn.Module to be exported Returns (None) run_batch def run_batch(inputs=None) Arguments : inputs - (Default value = None) OnnxTFBackend Objects class OnnxTFBackend(InferenceBackendBase) Inference Backend for tensorflow prepare def prepare(model) Arguments : model: run_batch def run_batch(inputs) Arguments : inputs: OnnxruntimeBackend Objects class OnnxruntimeBackend(InferenceBackendBase) Inference Backend for tensorflow prepare def prepare(model) Arguments : model: run_batch def run_batch(inputs=None) Arguments : inputs - (Default value = None) Returns hannah.callbacks.pruning PruningAmountScheduler Objects class PruningAmountScheduler() FilteredPruning Objects class FilteredPruning(ModelPruning) setup def setup(trainer: Trainer, pl_module: LightningModule, stage: str) Arguments : trainer: pl_module: filter_parameters_to_prune def filter_parameters_to_prune(parameters_to_prune=None) Filter out unprunable parameters Arguments : parameters_to_prune - (Default value = None) on_test_end def on_test_end(trainer, pl_module) -> None Arguments : trainer: pl_module: hannah.callbacks.optimization HydraOptCallback Objects class HydraOptCallback(Callback) on_test_end def on_test_end(trainer, pl_module) Arguments : trainer: pl_module: on_validation_end def on_validation_end(trainer, pl_module) Arguments : trainer: pl_module: test_result def test_result() val_result def val_result() result def result(dict=False) Arguments : dict - (Default value = False) curves def curves(dict=False) Arguments : dict - (Default value = False) hannah.callbacks.fine_tuning hannah.callbacks hannah.callbacks.clustering clustering def clustering(params, inertia, cluster) Arguments : params: inertia: cluster: kMeans Objects class kMeans(Callback) on_fit_end def on_fit_end(trainer, pl_module) Arguments : trainer: pl_module: on_epoch_end def on_epoch_end(trainer, pl_module) Arguments : trainer: pl_module: hannah.callbacks.summaries walk_model def walk_model(model, dummy_input) Adapted from IntelLabs Distiller Arguments : model: dummy_input: MacSummaryCallback Objects class MacSummaryCallback(Callback) predict def predict(pl_module, input=input) Arguments : pl_module: on_train_start @rank_zero_only def on_train_start(trainer, pl_module) Arguments : trainer: pl_module: on_test_end @rank_zero_only def on_test_end(trainer, pl_module) Arguments : trainer: pl_module: on_validation_epoch_end @rank_zero_only def on_validation_epoch_end(trainer, pl_module) Arguments : trainer: pl_module: estimate def estimate(pl_module, input=None) Generate Summary Metrics for neural network Arguments : pl_module(pytorch_lightning.LightningModule) - pytorch lightning module to summarize Returns : dict[str, float]: Dict of MetricName => Metric Value hannah.callbacks.svd_compress SVD Objects class SVD(Callback) on_epoch_start def on_epoch_start(trainer, pl_module) Arguments : trainer: pl_module: hannah.ssl.hard_labeling HardLabeling Objects class HardLabeling(torch.nn.Module) forward def forward(unlabeled_data: torch.Tensor) -> torch.Tensor Calculate pseudo label loss from unlabeled data. compute_uncertainty def compute_uncertainty(data: torch.Tensor, num_forward_passes: int = 10) -> torch.Tensor Compute Monte Carlo uncertainty using standard deviation. get_dropout_layers def get_dropout_layers() Returns all model layers of class dropout, dropblock. hannah.ssl hannah.conf.optimizer SGDConf Objects @dataclass class SGDConf() lr _RequiredParameter MADGRADConf Objects @dataclass class MADGRADConf() lr _RequiredParameter hannah.conf hannah.conf.nas hannah.conf.scheduler OneCycleLRConf Objects @dataclass class OneCycleLRConf() Config for one cycle lr total steps are configured from module hannah.optim hannah.optim.madgrad MADGRAD Objects class MADGRAD(torch.optim.Optimizer) MADGRAD_: A Momentumized, Adaptive, Dual Averaged Gradient Method for Stochastic Optimization. .. _MADGRAD: https://arxiv.org/abs/2101.11075 MADGRAD is a general purpose optimizer that can be used in place of SGD or Adam may converge faster and generalize better. Currently GPU-only. Typically, the same learning rate schedule that is used for SGD or Adam may be used. The overall learning rate is not comparable to either method and should be determined by a hyper-parameter sweep. MADGRAD requires less weight decay than other methods, often as little as zero. Momentum values used for SGD or Adam's beta1 should work here also. On sparse problems both weight_decay and momentum should be set to 0. Arguments : params (iterable): Iterable of parameters to optimize or dicts defining parameter groups. lr (float): Learning rate (default: 1e-2). momentum (float): Momentum value in the range [0,1) (default: 0.9). weight_decay (float): Weight decay, i.e. a L2 penalty (default: 0). eps (float): Term added to the denominator outside of the root operation to improve numerical stability. (default: 1e-6). step def step(closure: Optional[Callable[[], float]] = None) -> Optional[float] Performs a single optimization step. Arguments : closure callable, optional - A closure that reevaluates the model and returns the loss. hannah.optim.RAdam hannah.nas.dataflow.registry hannah.nas.dataflow.dataflow_utils hannah.nas.dataflow.tensor_expression hannah.nas.dataflow.quantization_type hannah.nas.dataflow.register_ops hannah.nas.dataflow.ops.dropout hannah.nas.dataflow.ops.conv2d hannah.nas.dataflow.ops.linear hannah.nas.dataflow.ops.identity hannah.nas.dataflow.ops.relu hannah.nas.dataflow.ops.sum hannah.nas.dataflow.ops.batch_nom hannah.nas.dataflow.ops.add hannah.nas.dataflow.ops hannah.nas.dataflow.ops.concat hannah.nas.dataflow.ops.pooling hannah.nas.dataflow.analysis.dataflow_analysis hannah.nas.dataflow.tensor hannah.nas.dataflow.tensor_type hannah.nas.dataflow.transformations.graph_tranformer hannah.nas.dataflow.scoping_utils hannah.nas.dataflow hannah.nas.dataflow.repeat hannah.nas.dataflow.axis_type AxisTuple Objects @parametrize class AxisTuple() Used to have the axis dict as a parametrized object hannah.nas.dataflow.data_type hannah.nas.dataflow.optional_op hannah.nas.dataflow.compression_type hannah.nas.dataflow.op_type hannah.nas.dataflow.dataflow_graph DataFlowGraph Objects @parametrize class DataFlowGraph(TensorExpression) link_users def link_users() Link the DFG to its users and the users of the DFG to the DFG collect_users def collect_users(node) Traverse graph starting from node and collect all users (including users from subsequent nodes). If a node_b is NOT in collect_users(node_a), this means that node_b is either BEFORE node_a in the graph OR it is in a parallel branch. Parameters node : type description Returns type description find_first_input def find_first_input(node) Recusively traverses the graph from the given node back to its first input. NOTE: The traversal is via OPERANDS and not OUTPUT, meaning that e.g. weight Tensors that are included in Ops in a DFG are not returned Parameters node : type description Returns type description hannah.nas.performance_prediction.simple BackendPredictor Objects class BackendPredictor() A predictor class that instantiates the model and uses the backends predict function to predict performance metrics MACPredictor Objects class MACPredictor() A predictor class that instantiates the model and calculates abstract metrics GCNPredictor Objects class GCNPredictor() A predictor class that instantiates the model and uses the backends predict function to predict performance metrics hannah.nas.performance_prediction.gcn.predictor Predictor Objects class Predictor() __init__ def __init__(fea_name=\"features\") -> None Parent method for different predictor classes. Parameters fea_name : str, optional internal name for features in the graph, as in graph.ndata[fea_name], by default 'features' train def train(dataloader, learning_rate=1e-3, num_epochs=200, validation_dataloader=None, verbose=1) Train GCN model Parameters dataloader : GraphDataLoader training data learning_rate : [type], optional by default 1e-3 num_epochs : int, optional by default 200 validation_dataloader : [type], optional if given, use this data to print validation loss, by default None verbose : int if validation_dataloader is given, print validation MSE every epoch, by default 1 predict def predict(graph) predict cost of graph Parameters graph : dgl.Graph Returns torch.Tensor predicted cost of given graph. Retrieve float value with .item() GCNPredictor Objects class GCNPredictor(Predictor) __init__ def __init__(input_feature_size, hidden_units=[128], readout=\"mean\", fea_name=\"features\") -> None G(raph)CN based network latency/cost predictor. End-to-end from graph to score. Parameters input_feature_size : [type] length of feature vector of a graph node (graph G with n nodes, each with features of length m, i.e. feature matrix F = n x m) hidden_units : int, list, optional size of hidden layer (layers if list) , by default 128 readout : str, optional readout function that is used to aggregate node features, by default 'mean' fea_name : str, optional internal name for features in the graph, as in graph.ndata[fea_name], by default 'features' train_and_fit def train_and_fit(dataloader, learning_rate=1e-3, num_epochs=200, validation_dataloader=None, verbose=0) Train GCN model Parameters dataloader : GraphDataLoader training data learning_rate : [type], optional by default 1e-3 num_epochs : int, optional by default 200 validation_dataloader : [type], optional if given, use this data to print validation loss, by default None verbose : int if validation_dataloader is given, print validation MSE every epoch, by default 1 predict def predict(graph) predict cost of graph Parameters graph : dgl.Graph Returns torch.Tensor predicted cost of given graph. Retrieve float value with .item() GaussianProcessPredictor Objects class GaussianProcessPredictor(Predictor) __init__ def __init__(input_feature_size, hidden_units=128, embedding_size=10, readout=\"mean\", fea_name=\"features\", kernel=\"default\", alpha=1e-10) -> None Predictor that generates a graph embedding that is used as input for a gaussian process predictor. Parameters input_feature_size : [type] length of feature vector of a graph node (graph G with n nodes, each with features of length m, i.e. feature matrix F = n x m) hidden_units : int, list, optional size of hidden layer (layers if list) , by default 128 embedding_size: int, optional size of output embedding readout : str, optional readout function that is used to aggregate node features, by default 'mean' fea_name : str, optional internal name for features in the graph, as in graph.ndata[fea_name], by default 'features' kernel : str, sklearn.gaussian_process.kernels.Kernel, optional The gaussian process kernel to use. input shoudl be either \"default\", or a sklearn Kernel() object by default RBF() + DotProduct() + WhiteKernel() train_and_fit def train_and_fit(dataloader, learning_rate=1e-3, num_epochs=200, validation_dataloader=None, verbose=1) Train GCN model, generate embeddings for training data and fit the predictor with embeddings. Parameters dataloader : GraphDataLoader training data learning_rate : [type], optional by default 1e-3 num_epochs : int, optional by default 200 validation_dataloader : [type], optional if given, use this data to print validation loss, by default None verbose : int if validation_dataloader is given, print validation MSE every epoch,by default 1 Returns float score of predictor on TRAINING data, see sklearn doc of chosen predictor for more info predict def predict(X, return_std=True) Predict cost/latency of graphs. Parameters X : dgl.DGLGraph, list[DGLGraph], dgl.dataloading.GraphDataLoader Input graph(s) return_std : bool, optional if true, return standard dev. else just mean prediction, by default True Returns array (,array) prediction(s) , (if return_std: standard deviation(s)) XGBPredictor Objects class XGBPredictor(Predictor) __init__ def __init__(input_feature_size, hidden_units=128, embedding_size=10, readout=\"mean\", fea_name=\"features\", xgb_param=\"default\") -> None Predictor that generates a graph embedding that is used as input for a xgb based predictor. Parameters input_feature_size : [type] length of feature vector of a graph node (graph G with n nodes, each with features of length m, i.e. feature matrix F = n x m) hidden_units : int, list, optional size of hidden layer (layers if list) , by default 128 embedding_size: int, optional size of output embedding readout : str, optional readout function that is used to aggregate node features, by default 'mean' fea_name : str, optional internal name for features in the graph, as in graph.ndata[fea_name], by default 'features' xgb_param : str, dict, optional The xgb_parameter to use. See https://xgboost.readthedocs.io/en/latest/parameter.html train_and_fit def train_and_fit(dataloader, learning_rate=1e-3, num_epochs=200, num_round=8000, validation_dataloader=None, verbose=1) Train GCN model, generate embeddings for training data and fit the predictor with embeddings. Parameters dataloader : GraphDataLoader training data learning_rate : [type], optional by default 1e-3 num_epochs : int, optional Training epochs for the GCN embedding network, by default 200 num_round : int, optional training rounds for xgb booster, by default 800 validation_dataloader : [type], optional if given, use this data to print validation loss, by default None verbose : int if validation_dataloader is given, print validation MSE every epoch,by default 1 predict def predict(X) Predict cost/latency of graphs. Parameters X : dgl.DGLGraph, list[DGLGraph], dgl.dataloading.GraphDataLoader Input graph(s) Returns array (,array) prediction(s) , (if return_std: standard deviation(s)) prepare_dataloader def prepare_dataloader(dataset, batch_size=50, train_test_split=1, subset=0, seed=0, validation=False) helper function to construct dataloaders from NASGraphDataset Parameters dataset : NASGraphDataset batch_size : int, optional by default 50 train_test_split : float, optional number between 0 and 1, the proportion of the dataset to be used for training, by default 1 subset : int, optional choose only many samples from the dataset. Set 0 for disabling, i.e. whole dataset. by default 0 seed : int, optional set seed for reproduceability validation : bool, optional also output a validation set e.g. for hyperparam tuning Returns tuple(GraphDataLoader, (GraphDataLoader), GraphDataLoader) training dataloader to be used in CostPredictor.train() and test/validation dataloader if train_test_split > 0, else len(test_dataloader) == 0 hannah.nas.performance_prediction.gcn.model hannah.nas.performance_prediction.features.dataset hannah.nas.performance_prediction.features.graph_conversion hannah.nas.performance_prediction hannah.nas.performance_prediction.examples.gcn_predictor_example hannah.nas.performance_prediction.examples.gcn_model_example hannah.nas.performance_prediction.examples.ri_capsule_performance_predictor hannah.nas.performance_prediction.examples.gaussian_process_predictor_example hannah.nas.performance_prediction.examples.xgb_predictor_example hannah.nas.constraints.constraint_model hannah.nas.constraints.dfg_constraint_model ConstraintModel Objects class ConstraintModel() process_optype def process_optype(op: OpType) Extracts the constraints based on the type of op. New variables are added to self.vars and the constraints are added to the solver. Parameters op : OpType process_tensor def process_tensor(tensor: Tensor) Goes through all axis and extracts the constraints for the respective axis sizes Parameters tensor : Tensor hannah.nas.hardware_description.device hannah.nas.hardware_description.memory_type hannah.nas.hardware_description hannah.nas.hardware_description.description hannah.nas.eval.extract hannah.nas.eval hannah.nas.eval.plot hannah.nas.eval.__main__ hannah.nas.eval.prepare prepare_summary def prepare_summary(data: Dict[str, str], base_dir: str = \".\", force: bool = False) -> pd.DataFrame Prepare a summary of one or multiple nas runs Arguments : data Dict[str, str] - A mapping from a short name for a nas run \"e.g KWS\" to a folder containing nas history file e.g. \"trained_models/nas_kws/conv_net_trax\" base_dir str - base directory paths in data mapping are interpreted relative to base directory force bool - force reconstructing of cached results (\"data.pkl\") hannah.nas.expressions.conditions hannah.nas.expressions.op hannah.nas.expressions.arithmetic hannah.nas.expressions.placeholder hannah.nas.expressions hannah.nas.expressions.logic hannah.nas.expressions.shapes hannah.nas.expressions.metrics hannah.nas.expressions.types hannah.nas.expressions.choice hannah.nas.test.test_symbolic_metrics hannah.nas.test.test_parameters hannah.nas.test.test_op_to_torch_conversion hannah.nas.test.test_conditions hannah.nas.test.test_lazy_torch hannah.nas.test.test_mobilenet hannah.nas.test.test_scoping hannah.nas.test.test_darts_space hannah.nas.test.test_arithmetic test_unimplemeted @pytest.mark.parametrize( \"x,y\", [ (IntScalarParameter(0, 0), 2), (IntScalarParameter(0, 0), IntScalarParameter(0, 0)), (DefaultInt(0), 2), ], ) def test_unimplemeted(x, y) Test that unimplemented methods raise unimplemented errors hannah.nas.test.test_z3 hannah.nas.test.test_graph_transformer hannah.nas.test.test_dfg_removal hannah.nas.test.test_repeat hannah.nas.test.test_parametrize hannah.nas.test.network hannah.nas.test.test_description_ultratrail hannah.nas.test.test_dataflow hannah.nas.test.test_conv2d hannah.nas.test.test_constraint_model hannah.nas.test.test_add hannah.nas.test.test_adjacency hannah.nas.parametrization ScalarParameterState Objects @dataclass class ScalarParameterState(ParameterState) sigma variance of sampling parameter hannah.nas.parameters.parameters hannah.nas.parameters.parametrize hannah.nas.parameters.iterators hannah.nas.parameters hannah.nas.parameters.lazy hannah.nas hannah.nas.core.parametrized hannah.nas.core hannah.nas.core.expression hannah.nas.config Scalar Objects @dataclass class Scalar() Representation of all the options to define a scalar. hannah.nas.__main__ hannah.nas.plotter hannah.nas.search.search hannah.nas.search.model_trainer.simple_model_trainer hannah.nas.search.model_trainer.progressive_shrinking ProgressiveShrinkingModelTrainer Objects class ProgressiveShrinkingModelTrainer() warmup def warmup(model, ofa_model) The function rebuilds the trainer with the warmup epochs, fits the model, validates the model, and then calls the on_warmup_end() function to change some internal variables Arguments : model : the model to be trained ofa_model : the model that we want to train train_elastic_width def train_elastic_width(model, ofa_model) The function trains the model for a number of epochs, then adds a width step, then trains the model for a number of epochs, then adds a width step, and so on Arguments : model : the model to train ofa_model : the model that will be trained train_elastic_depth def train_elastic_depth(model, ofa_model) The function trains the model for a number of epochs, then progressively shrinks the depth of the model, and trains the model for a number of epochs again Arguments : model : the model to train ofa_model : the model to be trained train_elastic_kernel def train_elastic_kernel(model, ofa_model) The function trains the elastic kernels by progressively shrinking the model and training the model for a number of epochs and repeats this process until the number of kernel steps is reached Arguments : model : the model to train ofa_model : the model that will be trained train_elastic_dilation def train_elastic_dilation(model, ofa_model) The function trains the model for a number of epochs, then adds a dilation step, and trains the model for a number of epochs, and repeats this process until the number of dilation steps is reached Arguments : model : the model to be trained ofa_model : the model that will be trained train_elastic_grouping def train_elastic_grouping(model, ofa_model) The function trains the model for a number of epochs, then adds a group step, and trains the model for a number of epochs, and repeats this process until the number of group steps is reached Arguments : model : the model to be trained ofa_model : the model that will be trained train_elastic_dsc def train_elastic_dsc(model, ofa_model) The function trains the model for a number of epochs, then adds a dsc step (turns Depthwise Separable Convolution on and off), and trains the model for a number of epochs, and repeats this process until the number of dsc steps is reached Arguments : model : the model to be trained ofa_model : the model that will be trained eval_elastic_width def eval_elastic_width(method_stack, method_index, lightning_model, model, trainer_path, loginfo_output, metrics_output, metrics_csv) This function steps down the width of the model, and then calls the next method in the stack Arguments : method_stack : a list of methods that will be called in order method_index : The index of the current method in the method stack lightning_model : the lightning model to be trained model : the model to be trained trainer_path : The path to the trainer loginfo_output : This is the string that will be printed to the console metrics_output : a string that will be written to the metrics csv file metrics_csv : a string that contains the metrics for the current model Returns : The metrics_csv is being returned. eval_elastic_kernel def eval_elastic_kernel(method_stack, method_index, lightning_model, model, trainer_path, loginfo_output, metrics_output, metrics_csv) This function steps down the kernel size of the model, and then calls the next method in the stack Arguments : method_stack : The list of methods to be called method_index : The index of the current method in the method stack lightning_model : the lightning model to be trained model : the model to be trained trainer_path : The path to the trainer loginfo_output : This is the string that will be printed to the console metrics_output : This is the string that will be printed to the console metrics_csv : a string that contains the metrics for the current model Returns : The metrics_csv is being returned. eval_elastic_dilation def eval_elastic_dilation(method_stack, method_index, lightning_model, model, trainer_path, loginfo_output, metrics_output, metrics_csv) This function evaluates the model with a different dilation size for each layer Arguments : method_stack : The list of methods to be called method_index : The index of the method in the method stack lightning_model : the lightning model to be trained model : the model to be evaluated trainer_path : The path to the trainer loginfo_output : This is the string that will be printed to the console metrics_output : a string that will be written to the metrics csv file metrics_csv : a string that contains the csv data for the metrics Returns : The metrics_csv is being returned. eval_elastic_depth def eval_elastic_depth(method_stack, method_index, lightning_model, model, trainer_path, loginfo_output, metrics_output, metrics_csv) This function will run the next method in the stack for each depth step, and then return the metrics_csv Arguments : method_stack : The list of methods to be called method_index : The index of the current method in the method stack lightning_model : the lightning model to be trained model : The model to be trained trainer_path : The path to the trainer, which is used to save the model loginfo_output : This is the string that will be printed to the console metrics_output : This is the string that will be printed to the console metrics_csv : This is the CSV file that we're writing to Returns : The metrics_csv is being returned. eval_elastic_grouping def eval_elastic_grouping(method_stack, method_index, lightning_model, model, trainer_path, loginfo_output, metrics_output, metrics_csv) This function evaluates the model with a different group size for each layer Arguments : method_stack : The list of methods to be called method_index : The index of the method in the method stack lightning_model : the lightning model to be trained model : the model to be evaluated trainer_path : The path to the trainer loginfo_output : This is the string that will be printed to the console metrics_output : a string that will be written to the metrics csv file metrics_csv : a string that contains the csv data for the metrics Returns : The metrics_csv is being returned. eval_elastic_dsc def eval_elastic_dsc(method_stack, method_index, lightning_model, model, trainer_path, loginfo_output, metrics_output, metrics_csv) This function evaluates the model with a different dsc for each layer Arguments : method_stack : The list of methods to be called method_index : The index of the method in the method stack lightning_model : the lightning model to be trained model : the model to be evaluated trainer_path : The path to the trainer loginfo_output : This is the string that will be printed to the console metrics_output : a string that will be written to the metrics csv file metrics_csv : a string that contains the csv data for the metrics Returns : The metrics_csv is being returned. eval_single_model def eval_single_model(method_stack, method_index, lightning_model, model, trainer_path, loginfo_output, metrics_output, metrics_csv) This function takes in a model, a trainer, and a bunch of other stuff, evaluates the model and tracks the results in der in the given strings and returns a string of metrics Arguments : method_stack : The list of methods that we're evaluating method_index : The index of the method in the method stack lightning_model : the lightning model that we want to evaluate model : The model to be evaluated trainer_path : the path to the trainer object loginfo_output : This is the string that will be printed to the console when the model is being evaluated metrics_output : This is the string that will be written to the metrics file. It contains the method name, the method index, and the method stack metrics_csv : a string that will be written to a csv file Returns : The metrics_csv is being returned. eval_model def eval_model(lightning_model, model) First the method stack for the evaluation ist build and then it is according to this evaluated Arguments : lightning_model : the lightning model model : the model to be evaluated hannah.nas.search.search_old OFANasTrainer Objects class OFANasTrainer(NASTrainerBase) warmup def warmup(model, ofa_model) The function rebuilds the trainer with the warmup epochs, fits the model, validates the model, and then calls the on_warmup_end() function to change some internal variables Arguments : model : the model to be trained ofa_model : the model that we want to train train_elastic_width def train_elastic_width(model, ofa_model) The function trains the model for a number of epochs, then adds a width step, then trains the model for a number of epochs, then adds a width step, and so on Arguments : model : the model to train ofa_model : the model that will be trained train_elastic_depth def train_elastic_depth(model, ofa_model) The function trains the model for a number of epochs, then progressively shrinks the depth of the model, and trains the model for a number of epochs again Arguments : model : the model to train ofa_model : the model to be trained train_elastic_kernel def train_elastic_kernel(model, ofa_model) The function trains the elastic kernels by progressively shrinking the model and training the model for a number of epochs and repeats this process until the number of kernel steps is reached Arguments : model : the model to train ofa_model : the model that will be trained train_elastic_dilation def train_elastic_dilation(model, ofa_model) The function trains the model for a number of epochs, then adds a dilation step, and trains the model for a number of epochs, and repeats this process until the number of dilation steps is reached Arguments : model : the model to be trained ofa_model : the model that will be trained train_elastic_grouping def train_elastic_grouping(model, ofa_model) The function trains the model for a number of epochs, then adds a group step, and trains the model for a number of epochs, and repeats this process until the number of group steps is reached Arguments : model : the model to be trained ofa_model : the model that will be trained train_elastic_dsc def train_elastic_dsc(model, ofa_model) The function trains the model for a number of epochs, then adds a dsc step (turns Depthwise Separable Convolution on and off), and trains the model for a number of epochs, and repeats this process until the number of dsc steps is reached Arguments : model : the model to be trained ofa_model : the model that will be trained eval_elastic_width def eval_elastic_width(method_stack, method_index, lightning_model, model, trainer_path, loginfo_output, metrics_output, metrics_csv) This function steps down the width of the model, and then calls the next method in the stack Arguments : method_stack : a list of methods that will be called in order method_index : The index of the current method in the method stack lightning_model : the lightning model to be trained model : the model to be trained trainer_path : The path to the trainer loginfo_output : This is the string that will be printed to the console metrics_output : a string that will be written to the metrics csv file metrics_csv : a string that contains the metrics for the current model Returns : The metrics_csv is being returned. eval_elastic_kernel def eval_elastic_kernel(method_stack, method_index, lightning_model, model, trainer_path, loginfo_output, metrics_output, metrics_csv) This function steps down the kernel size of the model, and then calls the next method in the stack Arguments : method_stack : The list of methods to be called method_index : The index of the current method in the method stack lightning_model : the lightning model to be trained model : the model to be trained trainer_path : The path to the trainer loginfo_output : This is the string that will be printed to the console metrics_output : This is the string that will be printed to the console metrics_csv : a string that contains the metrics for the current model Returns : The metrics_csv is being returned. eval_elastic_dilation def eval_elastic_dilation(method_stack, method_index, lightning_model, model, trainer_path, loginfo_output, metrics_output, metrics_csv) This function evaluates the model with a different dilation size for each layer Arguments : method_stack : The list of methods to be called method_index : The index of the method in the method stack lightning_model : the lightning model to be trained model : the model to be evaluated trainer_path : The path to the trainer loginfo_output : This is the string that will be printed to the console metrics_output : a string that will be written to the metrics csv file metrics_csv : a string that contains the csv data for the metrics Returns : The metrics_csv is being returned. eval_elastic_depth def eval_elastic_depth(method_stack, method_index, lightning_model, model, trainer_path, loginfo_output, metrics_output, metrics_csv) This function will run the next method in the stack for each depth step, and then return the metrics_csv Arguments : method_stack : The list of methods to be called method_index : The index of the current method in the method stack lightning_model : the lightning model to be trained model : The model to be trained trainer_path : The path to the trainer, which is used to save the model loginfo_output : This is the string that will be printed to the console metrics_output : This is the string that will be printed to the console metrics_csv : This is the CSV file that we're writing to Returns : The metrics_csv is being returned. eval_elastic_grouping def eval_elastic_grouping(method_stack, method_index, lightning_model, model, trainer_path, loginfo_output, metrics_output, metrics_csv) This function evaluates the model with a different group size for each layer Arguments : method_stack : The list of methods to be called method_index : The index of the method in the method stack lightning_model : the lightning model to be trained model : the model to be evaluated trainer_path : The path to the trainer loginfo_output : This is the string that will be printed to the console metrics_output : a string that will be written to the metrics csv file metrics_csv : a string that contains the csv data for the metrics Returns : The metrics_csv is being returned. eval_elastic_dsc def eval_elastic_dsc(method_stack, method_index, lightning_model, model, trainer_path, loginfo_output, metrics_output, metrics_csv) This function evaluates the model with a different dsc for each layer Arguments : method_stack : The list of methods to be called method_index : The index of the method in the method stack lightning_model : the lightning model to be trained model : the model to be evaluated trainer_path : The path to the trainer loginfo_output : This is the string that will be printed to the console metrics_output : a string that will be written to the metrics csv file metrics_csv : a string that contains the csv data for the metrics Returns : The metrics_csv is being returned. eval_single_model def eval_single_model(method_stack, method_index, lightning_model, model, trainer_path, loginfo_output, metrics_output, metrics_csv) This function takes in a model, a trainer, and a bunch of other stuff, evaluates the model and tracks the results in der in the given strings and returns a string of metrics Arguments : method_stack : The list of methods that we're evaluating method_index : The index of the method in the method stack lightning_model : the lightning model that we want to evaluate model : The model to be evaluated trainer_path : the path to the trainer object loginfo_output : This is the string that will be printed to the console when the model is being evaluated metrics_output : This is the string that will be written to the metrics file. It contains the method name, the method index, and the method stack metrics_csv : a string that will be written to a csv file Returns : The metrics_csv is being returned. eval_model def eval_model(lightning_model, model) First the method stack for the evaluation ist build and then it is according to this evaluated Arguments : lightning_model : the lightning model model : the model to be evaluated hannah.nas.search.weight_sharing hannah.nas.search.sampler.base_sampler Sampler Objects class Sampler(ABC) tell_result def tell_result(parameters, metrics) Tell the result of a task hannah.nas.search.sampler.aging_evolution AgingEvolutionSampler Objects class AgingEvolutionSampler(Sampler) Aging Evolution based multi objective optimization next_parameters def next_parameters() Returns a list of current tasks tell_result def tell_result(parameters, metrics) Tell the result of a task hannah.nas.search.sampler.random_sampler hannah.nas.search.sampler.mutator hannah.nas.search.utils hannah.nas.ops hannah.nas.graph_conversion hannah.nas.spaces.darts.darts_space hannah.nas.spaces.mobilenet.mobilenet hannah.nas.utils is_pareto def is_pareto(costs, maximise=False) Arguments : costs : An (n_points, n_costs) array Returns : A (n_points, ) boolean array, indicating whether each point is Pareto efficient hannah.datasets.vision.ri_capsule Rhode island gastroenterology video capsule endoscopy dataset https://www.nature.com/articles/s41597-022-01726-3 https://github.com/acharoen/Rhode-Island-GI-VCE-Technical-Validation hannah.datasets.vision.fake hannah.datasets.vision.cifar hannah.datasets.vision.kvasir hannah.datasets.vision.kvasir_unlabeled KvasirCapsuleUnlabeled Objects class KvasirCapsuleUnlabeled(AbstractDataset) Dataset representing unlabelled videos hannah.datasets.vision hannah.datasets.vision.dresden_capsule Rhode island gastroenterology video capsule endoscopy dataset https://www.nature.com/articles/s41597-022-01726-3 https://github.com/acharoen/Rhode-Island-GI-VCE-Technical-Validation hannah.datasets.vision.utils hannah.datasets.vision.utils.naneye read_naneye def read_naneye(data_file: Union[str, Path]) Read a naneye raw aimage and decode bayer pattern Arguments : data_file Union[str, Path] - path to the datafile Returns : np.ndarray - uint8 array of decoded image data hannah.datasets.vision.base TorchvisionDatasetBase Objects class TorchvisionDatasetBase(VisionDatasetBase) Wrapper around torchvision classification datasets ImageDatasetBase Objects class ImageDatasetBase(VisionDatasetBase) __init__ def __init__(X, y, classes, bbox=None, transform=None) Initialize vision dataset Arguments : X List[str] - List of paths to image files y List[str] - Class id of corresponding image classes List[str] - List of class names, names are ordered by numeric class id bbox Dict[str] - Dict with filename as keys, bbox coordinates as numpy arrays transform Callable[image,image], optional - Optional transformation/augmentation of input images. Defaults to None. hannah.datasets.emergency EmergencySirenDataset Objects class EmergencySirenDataset(AbstractDataset) Emergency Dataset hannah.datasets.Downsample hannah.datasets.collate vision_collate_fn def vision_collate_fn(batch) Function that takes in a batch of data and puts the elements within the batch into a tensor with an additional outer dimension - batch size. The exact output type can be a :class: torch.Tensor , a Sequence of :class: torch.Tensor , a Collection of :class: torch.Tensor , or left unchanged, depending on the input type. This is used as the default function for collation for vision tasks batch_size or batch_sampler is defined in :class: ~torch.utils.data.DataLoader . Here is the general input type (based on the type of the element within the batch) to output type mapping: :class: torch.Tensor -> :class: torch.Tensor (with an added outer dimension batch size) NumPy Arrays -> :class: torch.Tensor float -> :class: torch.Tensor int -> :class: torch.Tensor str -> str (unchanged) bytes -> bytes (unchanged) Mapping[K, V_i] -> Mapping[K, vision_collate([V_1, V_2, ...])] NamedTuple[V1_i, V2_i, ...] -> NamedTuple[vision_collate([V1_1, V1_2, ...]), vision_collate([V2_1, V2_2, ...]), ...] Sequence[V1_i, V2_i, ...] -> Sequence[vision_collate([V1_1, V1_2, ...]), vision_collate([V2_1, V2_2, ...]), ...] Arguments : batch - a single batch to be collated Examples : Example with a batch of int s: vision_collate([0, 1, 2, 3]) tensor([0, 1, 2, 3]) Example with a batch of str s: vision_collate(['a', 'b', 'c']) ['a', 'b', 'c'] Example with Map inside the batch: vision_collate([{'A': 0, 'B': 1}, {'A': 100, 'B': 100}]) - {'A' - tensor([ 0, 100]), 'B': tensor([ 1, 100])} Example with NamedTuple inside the batch: Point = namedtuple('Point', ['x', 'y']) vision_collate([Point(0, 0), Point(1, 1)]) Point(x=tensor([0, 1]), y=tensor([0, 1])) Example with Tuple inside the batch: vision_collate([(0, 1), (2, 3)]) [tensor([0, 2]), tensor([1, 3])] Example with List inside the batch: vision_collate([[0, 1], [2, 3]]) [tensor([0, 2]), tensor([1, 3])] ctc_collate_fn def ctc_collate_fn(data) Creates mini-batch tensors from the list of tuples (src_seq, trg_seq). We should build a custom collate_fn rather than using default collate_fn, because merging sequences (including padding) is not supported in default. Sequences are padded to the maximum length of mini-batch sequences (dynamic padding). Arguments : data - list of tuple (src_seq, src_length, trg_seq, trg_length). src_seq: torch tensor of shape (x,?); variable length. src length: torch tenso of shape 1x1 trg_seq: torch tensor of shape (?); variable length. trg_length: torch_tensor of shape (1x1) Returns - tuple of four torch tensors src_seqs - torch tensor of shape (batch_size, x, padded_length). src_lengths - torch_tensor of shape (batch_size); valid length for each padded source sequence. trg_seqs - torch tensor of shape (batch_size, x, padded_length). trg_lengths - torch tensor of shape (batch_size); valid length for each padded target sequence. hannah.datasets.speech SpeechDataset Objects class SpeechDataset(AbstractDataset) Base Class for speech datasets preprocess def preprocess(example, silence=False, label=0) Run preprocessing and feature extraction SpeechCommandsDataset Objects class SpeechCommandsDataset(SpeechDataset) This class implements reading and preprocessing of speech commands like dataset SpeechHotwordDataset Objects class SpeechHotwordDataset(SpeechDataset) Dataset Class for Hotword dataset e.g. Hey Snips! splits @classmethod def splits(cls, config) Splits the dataset in training, devlopment and test set and returns the three sets as List VadDataset Objects class VadDataset(SpeechDataset) splits @classmethod def splits(cls, config) Splits the dataset in training, devlopment and test set and returns the three sets as List hannah.datasets.activity Data3D Objects class Data3D() 3D-Data PAMPAP2_IMUData Objects class PAMPAP2_IMUData() A IMU set defined by temperature (\u00b0C) 3D-acceleration data (ms -2 ), scale: \u00b116g, resolution: 13-bit 3D-acceleration data (ms -2 ), scale: \u00b16g, resolution: 13-bit 3D-gyroscope data (rad/s) 3D-magnetometer data (\u03bcT) orientation (invalid in this data collection) PAMAP2_DataPoint Objects class PAMAP2_DataPoint() A temporal datapoint in the dataset PAMAP2_DataChunk Objects class PAMAP2_DataChunk() A DataChunk is a item of the pytorch dataset PAMAP2_Dataset Objects class PAMAP2_Dataset(AbstractDataset) Class for the PAMAP2 activity dataset https://archive.ics.uci.edu/ml/datasets/pamap2+physical+activity+monitoring hannah.datasets.directional DirectionalDataset Objects class DirectionalDataset(AbstractDataset) Directional Dataset hannah.datasets.DatasetSplit hannah.datasets hannah.datasets.physio AtrialFibrillationDataset Objects class AtrialFibrillationDataset(PhysioDataset) Atrial Fibrillation Database (https://physionet.org/content/afdb/1.0.0/) hannah.datasets.NoiseDataset hannah.datasets.fake1d hannah.datasets.utils.cache hannah.datasets.utils hannah.datasets.utils.md5 hannah.datasets.base DatasetType Objects class DatasetType(Enum) The type of a dataset partition e.g. train, dev, test AbstractDataset Objects class AbstractDataset(Dataset, ABC) prepare @abstractclassmethod def prepare(cls, config: Dict[str, Any]) -> None Prepare the dataset. This method is run at the beginning of the dataset training. If possible it should download the dataset from its original source, if it is available for public download. Arguments : config Dict[Any] - The dataset configuration splits @abstractclassmethod def splits( cls, config: Dict[str, Any] ) -> Tuple[\"AbstractDataset\", \"AbstractDataset\", \"AbstractDataset\"] Returns the test, validation and train split according to the Dataset config Arguments : config [type] - [description] class_names @abstractproperty def class_names() -> List[str] Returns the names of the classes in the classification dataset class_counts @abstractproperty def class_counts() -> Optional[Dict[int, int]] Returns the number of items in each class of the dataset If this is not applicable to a dataset type e.g. ASR, Semantic Segementation, it may return None __getitem__ @abstractmethod def __getitem__(index) -> List[torch.Tensor] Returns a torch.Tensor for the data item at the corresponding index The length of the list depends on the dataset item to use Arguments : index int - the index of the data item __len__ @abstractmethod def __len__() -> int Returns number of samples in dataset size def size() -> List[int] Returns dimension of output without batch dimension std @property def std() -> Optional[Tuple[int, ...]] Returns channel-wise standard deviation for dataset if applicable mean @property def mean() -> Optional[Tuple[int, ...]] Returns channel-wise means for dataset if applicable resolution @property def resolution() -> Optional[Tuple[int, ...]] Returns resolution for dataset if applicable weights @property def weights() -> Optional[List[float]] Class weights for weighted sampling hannah.datasets.Kitti Kitti Objects class Kitti(AbstractDataset) splits @classmethod def splits(cls, config) Splits the dataset in training, devlopment and test set and returns the three sets as List hannah.nn.quantized hannah.nn.qat Implementations of torch.nn.intrinsics qat with an optional quantize bias parameter. Qconfigs can support an optional bias quantization funciton which should be returned by qconfig.bias() else biases will be quantized with qconfig.activation() _ConvBnNd Objects class _ConvBnNd(nn.modules.conv._ConvNd, _ConvForwardMixin) train def train(mode: bool = True) -> Any Batchnorm's training behavior is using the self.training flag. Prevent changing it if BN is frozen. This makes sure that calling model.train() on a model with a frozen BN will behave properly. from_float @classmethod def from_float(cls, mod) Create a qat module from a float module or qparams_dict Args: mod a float module, either produced by torch.quantization utilities or directly from user ConvBn1d Objects class ConvBn1d(_ConvBnNd) A ConvBn1d module is a module fused from Conv1d and BatchNorm1d, attached with FakeQuantize modules for weight, used in quantization aware training. We combined the interface of :class: torch.nn.Conv1d and :class: torch.nn.BatchNorm1d . Similar to :class: torch.nn.Conv1d , with FakeQuantize modules initialized to default. Attributes : freeze_bn: - weight_fake_quant - fake quant module for weight ConvBnReLU1d Objects class ConvBnReLU1d(ConvBn1d) A ConvBnReLU1d module is a module fused from Conv1d, BatchNorm1d and ReLU, attached with FakeQuantize modules for weight, used in quantization aware training. We combined the interface of :class: torch.nn.Conv1d and :class: torch.nn.BatchNorm1d and :class: torch.nn.ReLU . Similar to torch.nn.Conv1d , with FakeQuantize modules initialized to default. Attributes : weight_fake_quant - fake quant module for weight ConvBn2d Objects class ConvBn2d(_ConvBnNd) A ConvBn2d module is a module fused from Conv2d and BatchNorm2d, attached with FakeQuantize modules for weight, used in quantization aware training. We combined the interface of :class: torch.nn.Conv2d and :class: torch.nn.BatchNorm2d . Similar to :class: torch.nn.Conv2d , with FakeQuantize modules initialized to default. Attributes : freeze_bn: - weight_fake_quant - fake quant module for weight ConvBnReLU2d Objects class ConvBnReLU2d(ConvBn2d) A ConvBnReLU2d module is a module fused from Conv2d, BatchNorm2d and ReLU, attached with FakeQuantize modules for weight, used in quantization aware training. We combined the interface of :class: torch.nn.Conv2d and :class: torch.nn.BatchNorm2d and :class: torch.nn.ReLU . Similar to torch.nn.Conv2d , with FakeQuantize modules initialized to default. Attributes : weight_fake_quant - fake quant module for weight ConvReLU2d Objects class ConvReLU2d(nn.Conv2d, _ConvForwardMixin) A ConvReLU2d module is a fused module of Conv2d and ReLU, attached with FakeQuantize modules for weight for quantization aware training. We combined the interface of :class: ~torch.nn.Conv2d and :class: ~torch.nn.BatchNorm2d . Attributes : weight_fake_quant - fake quant module for weight ConvReLU1d Objects class ConvReLU1d(nn.Conv1d, _ConvForwardMixin) A ConvReLU1d module is fused module of Conv1d and ReLU, attached with FakeQuantize modules for quantization aware training Conv1d Objects class Conv1d(nn.Conv1d, _ConvForwardMixin) A Conv1d module is a Conv1d module , attached with FakeQuantize modules for weight for quantization aware training. Attributes : weight_fake_quant - fake quant module for weight bias_fake_quant - fake quant module for bias activation_post_process - fake_quant_module for activations Conv2d Objects class Conv2d(nn.Conv2d, _ConvForwardMixin) A Conv2d module is a Conv2d module , attached with FakeQuantize modules for weight for quantization aware training. Attributes : weight_fake_quant - fake quant module for weight bias_fake_quant - fake quant module for bias activation_post_process - fake_quant_module for activations Linear Objects class Linear(nn.Linear) A linear module attached with FakeQuantize modules for weight, used for quantization aware training. We adopt the same interface as torch.nn.Linear , please see https://pytorch.org/docs/stable/nn.html#torch.nn.Linear for documentation. Similar to torch.nn.Linear , with FakeQuantize modules initialized to default. Attributes : weight - fake quant module for weight from_float @classmethod def from_float(cls, mod) Create a qat module from a float module or qparams_dict Args: mod a float module, either produced by torch.quantization utilities or directly from user LinearReLU Objects class LinearReLU(nn.Linear) A linear module attached with FakeQuantize modules and ReLU for weight, used for quantization aware training. We adopt the same interface as torch.nn.Linear , please see https://pytorch.org/docs/stable/nn.html#torch.nn.Linear for documentation. Similar to torch.nn.Linear , with FakeQuantize modules initialized to default. Attributes : weight - fake quant module for weight from_float @classmethod def from_float(cls, mod) Create a qat module from a float module or qparams_dict Args: mod a float module, either produced by torch.quantization utilities or directly from user Identity Objects class Identity(nn.Identity) A identity module attached with FakeQuantize modules for weight, used for quantization aware training. We adopt the same interface as torch.nn.Identity , please see https://pytorch.org/docs/stable/nn.html#torch.nn.Identity for documentation. Similar to torch.nn.Identity , with FakeQuantize modules initialized to default. from_float @classmethod def from_float(cls, mod) Create a qat module from a float module or qparams_dict Args: mod a float module, either produced by torch.quantization utilities or directly from user hannah.quantization.callback hannah.quantization.qconfig hannah.quantization.rounding round_downward def round_downward(x: Tensor) -> Tensor Round to nearest upward round_upward def round_upward(x: Tensor) -> Tensor Round to nearest downward round_odd def round_odd(x: Tensor) -> Tensor Round to nearest odd round_even def round_even(x: Tensor) -> Tensor Round to nearest even round_zero def round_zero(x: Tensor) -> Tensor Round towards zero round_infinity def round_infinity(x: Tensor) -> Tensor Round toward infinity truncate_up def truncate_up(x: Tensor) -> Tensor Always round up to next integer truncate_down def truncate_down(x: Tensor) -> Tensor Always round down to next integer truncate_infinity def truncate_infinity(x: Tensor) -> Tensor Always round to next integer in direction infinity truncate_zero def truncate_zero(x: Tensor) -> Tensor Always round to next integer in direction of Zero round_stochastic def round_stochastic(x: Tensor) -> Tensor Round stochastically hannah.modules.vision.anomaly_score class AnomalyScore(CatMetric): def init (self, percentile, nan_strategy=\"warn\", kwargs): super(). init (nan_strategy=nan_strategy, kwargs) self.percentile = percentile def compute(self): anomaly_score = None train_losses = super().compute() if train_losses: normalized_train_errors = torch.stack(train_losses) / ( torch.max(torch.stack(train_losses), dim=0).values ) anomaly_score = np.percentile( normalized_train_errors.cpu().numpy(), self.percentile ) return anomaly_score hannah.modules.vision.anomaly_detection AnomalyDetectionModule Objects class AnomalyDetectionModule(VisionBaseModule) on_test_end def on_test_end() wd_dir = os.getcwd() score, largest_train_error = self.compute_anomaly_score() train_errors = self.normalized_train_errors plt.hist(train_errors.detach().cpu().numpy(), bins=100) plt.axvline(score, linestyle=\"dashed\") plt.title(\"Normalized train reconstruction errors\") plt.savefig(wd_dir + \"/normalized_train_errors.png\") test = ( torch.tensor(self.test_losses, device=self.device) / torch.max(torch.stack(self.train_losses), dim=0).values ) plt.hist(test.detach().cpu().numpy(), bins=100) plt.title(\"Normalized test reconstruction errors\") plt.savefig(wd_dir + \"/normalized_test_errors.png\") print(\"Anomaly score\", score) print( \"Largest train error\", torch.max(torch.stack(self.train_losses), dim=0).values, ) hannah.modules.vision.loss hannah.modules.vision hannah.modules.vision.image_classifier hannah.modules.vision.base hannah.modules.classifier hannah.modules.object_detection hannah.modules.config_utils dump_config def dump_config(output_dir, config) Dumps the configuration to json format Creates file config.json in output_dir Parameters output_dir : str Output directory config : dict Configuration to dump save_model def save_model(output_dir, model) Creates serialization of the model for later inference, evaluation Creates the following files: model.pt: Serialized version of network parameters in pytorch model.json: Serialized version of network parameters in json format model.onnx: full model including paramters in onnx format Parameters output_dir : str Directory to put serialized models model : LightningModule Model to serialize hannah.modules hannah.modules.angle_classifier hannah.modules.metrics Error Objects class Error() Computes Error = 1 - Accuracy_ .. math:: \\text{Error} = 1 - \\frac{1}{N}\\sum_i^N 1(y_i = \\hat{y}_i) Where :math: y is a tensor of target values, and :math: \\hat{y} is a tensor of predictions. This module is a simple wrapper to get the task specific versions of this metric, which is done by setting the task argument to either 'binary' , 'multiclass' or multilabel . See the documentation of :mod: BinaryError , :mod: MulticlassError and :mod: MultilabelError for the specific details of each argument influence and examples. plot_confusion_matrix def plot_confusion_matrix(cf, group_names=None, categories=\"auto\", count=True, percent=True, cbar=True, xyticks=True, xyplotlabels=True, sum_stats=True, figsize=None, cmap=\"Blues\", title=None) This function will make a pretty plot of an sklearn Confusion Matrix cm using a Seaborn heatmap visualization. Arguments cf: confusion matrix to be passed in group_names: List of strings that represent the labels row by row to be shown in each square. categories: List of strings containing the categories to be displayed on the x,y axis. Default is 'auto' count: If True, show the raw number in the confusion matrix. Default is True. normalize: If True, show the proportions for each category. Default is True. cbar: If True, show the color bar. The cbar values are based off the values in the confusion matrix. Default is True. xyticks: If True, show x and y ticks. Default is True. xyplotlabels: If True, show 'True Label' and 'Predicted Label' on the figure. Default is True. sum_stats: If True, display summary statistics below the figure. Default is True. figsize: Tuple representing the figure size. Default will be the matplotlib rcParams value. cmap: Colormap of the values displayed from matplotlib.pyplot.cm. Default is 'Blues' See http://matplotlib.org/examples/color/colormaps_reference.html title: Title for the heatmap. Default is None. hannah.modules.base ClassifierModule Objects class ClassifierModule(LightningModule, ABC) total_training_steps def total_training_steps() -> int Total training steps inferred from datamodule and devices. hannah.modules.augmentation.augmentation hannah.modules.augmentation hannah.modules.augmentation.batch_augmentation BatchAugmentationPipeline Objects class BatchAugmentationPipeline(nn.Module) __init__ def __init__(transforms={}) Augmentation pipeline especially for self supervised learning Arguments : replica int - number of replicated different random augmentations transforms dict - configuration of transforms forward @torch.no_grad() def forward(x) -> torch.Tensor Perform Augmentations Arguments : x torch.Tensor - a torch.Tensor representing the augementation pipeline Returns : Tuple[torch.Tensor, torch.Tensor]; Batch augmented with replica different random augmentations hannah.modules.augmentation.transforms.registry hannah.modules.augmentation.transforms.kornia_transforms hannah.modules.augmentation.transforms hannah.modules.augmentation.bordersearch hannah.utils.imports lazy_import def lazy_import(module_name, callback=None) Returns a proxy module object that will lazily import the given module the first time it is used. Example usage:: Lazy version of import tensorflow as tf tf = lazy_import(\"tensorflow\") Other commands Now the module is loaded tf. version Arguments : module_name - the fully-qualified module name to import callback None - a callback function to call before importing the module Returns : a proxy module object that will be lazily imported when first used LazyModule Objects class LazyModule(types.ModuleType) Proxy module that lazily imports the underlying module the first time it is actually used. Arguments : module_name - the fully-qualified module name to import callback None - a callback function to call before importing the module hannah.utils hannah.utils.utils log_execution_env_state def log_execution_env_state() -> None Log information about the execution environment. File 'config_path' will be copied to directory 'logdir'. A common use-case is passing the path to a (compression) schedule YAML file. Storing a copy of the schedule file, with the experiment logs, is useful in order to reproduce experiments. Arguments : config_path - path to config file, used only when logdir is set logdir - log directory git_root - the path to the .git root directory extract_from_download_cache def extract_from_download_cache(filename, url, cached_files, target_cache, target_folder, target_test_folder=\"\", clear_download=False, no_exist_check=False) -> None extracts given file from cache or donwloads first from url Arguments : filename str - name of the file to download or extract url str - possible url to download the file cached_files (list(str)): cached files in download cache target_cache str - path to the folder to cache file if download necessary target_folder str - path where to extract file target_test_folder str, optional - folder to check if data are already there clear_download bool - clear download after usage no_exist_check bool - disables the check if folder exists fullname def fullname(o) -> Any Get the full classname of an object including surrounding packages/modules/namespaces set_deterministic @contextmanager def set_deterministic(mode, warn_only=False) A contextmanager to set deterministic algorithms hannah.visualization","title":"API Documentation"},{"location":"development/devel/api/#hannahtoolsobjectdetection_eval","text":"","title":"hannah.tools.objectdetection_eval"},{"location":"development/devel/api/#eval_train","text":"def eval_train(config, module, test=True) Arguments : config - param module: test - Default value = True) module:","title":"eval_train"},{"location":"development/devel/api/#eval_steps","text":"def eval_steps(config, module, hparams, checkpoint) Arguments : config - param module: hparams - param checkpoint: module: checkpoint:","title":"eval_steps"},{"location":"development/devel/api/#eval_checkpoint","text":"def eval_checkpoint(config: DictConfig, checkpoint) Arguments : config - DictConfig: checkpoint: config - DictConfig: config - DictConfig:","title":"eval_checkpoint"},{"location":"development/devel/api/#eval","text":"def eval(config: DictConfig) Arguments : config - DictConfig: config - DictConfig: config - DictConfig:","title":"eval"},{"location":"development/devel/api/#main","text":"@hydra.main(config_name=\"objectdetection_eval\", config_path=\"../conf\", version_base=\"1.2\") def main(config: DictConfig) Arguments : config - DictConfig: config - DictConfig: config - DictConfig:","title":"main"},{"location":"development/devel/api/#hannahtoolstrain","text":"","title":"hannah.tools.train"},{"location":"development/devel/api/#hannahtools","text":"","title":"hannah.tools"},{"location":"development/devel/api/#hannahtoolscharacterize","text":"","title":"hannah.tools.characterize"},{"location":"development/devel/api/#main_1","text":"@hydra.main(config_name=\"characterize\", config_path=\"../conf\", version_base=\"1.2\") def main(config: DictConfig) Arguments : config - DictConfig: config - DictConfig: config - DictConfig:","title":"main"},{"location":"development/devel/api/#hannahtoolseval","text":"","title":"hannah.tools.eval"},{"location":"development/devel/api/#eval_checkpoint_1","text":"def eval_checkpoint(config: DictConfig, checkpoint) -> None Arguments : config - DictConfig: checkpoint: config - DictConfig: config - DictConfig:","title":"eval_checkpoint"},{"location":"development/devel/api/#eval_1","text":"def eval(config: DictConfig) -> Optional[bool] Arguments : config - DictConfig: config - DictConfig: config - DictConfig:","title":"eval"},{"location":"development/devel/api/#main_2","text":"@hydra.main(config_name=\"eval\", config_path=\"conf\", version_base=\"1.2\") def main(config: DictConfig) Arguments : config - DictConfig: config - DictConfig: config - DictConfig:","title":"main"},{"location":"development/devel/api/#hannahtrainer","text":"","title":"hannah.trainer"},{"location":"development/devel/api/#hannahtrainercross_validation","text":"","title":"hannah.trainer.cross_validation"},{"location":"development/devel/api/#hannahtrain","text":"","title":"hannah.train"},{"location":"development/devel/api/#hannahlogo","text":"","title":"hannah.logo"},{"location":"development/devel/api/#hannahfeatures","text":"","title":"hannah.features"},{"location":"development/devel/api/#mfcc-objects","text":"class MFCC(torchaudio.transforms.MFCC) A simple wrapper around torchaudio mfcc, but melkwargs are given as direct named arguments instead of a dictionary","title":"MFCC Objects"},{"location":"development/devel/api/#sincconv-objects","text":"class SincConv(nn.Module) Sinc convolution: Arguments : in_channels - No. of input channels(must be 1) out_channels - No. of filters(40) sample_rate - sampling rate, default set at 32000 kernel_size - Filter length(101)","title":"SincConv Objects"},{"location":"development/devel/api/#logspectrogram-objects","text":"class LogSpectrogram(torch.nn.Module) Create a spectrogram from a audio signal. Arguments : n_fft int, optional - Size of FFT, creates n_fft // 2 + 1 bins. (Default: 400 ) win_length int or None, optional - Window size. (Default: n_fft ) hop_length int or None, optional - Length of hop between STFT windows. (Default: win_length // 2 ) pad int, optional - Two sided padding of signal. (Default: 0 ) window_fn Callable[..., Tensor], optional - A function to create a window tensor that is applied/multiplied to each frame/window. (Default: torch.hann_window ) power float or None, optional - Exponent for the magnitude spectrogram, (must be > 0) e.g., 1 for energy, 2 for power, etc. If None, then the complex spectrum is returned instead. (Default: 2 ) normalized bool, optional - Whether to normalize by magnitude after stft. (Default: False ) wkwargs dict or None, optional - Arguments for window function. (Default: None ) center bool, optional - whether to pad :attr: waveform on both sides so that the :math: t -th frame is centered at time :math: t \\times \\text{hop\\_length} . Default - True pad_mode string, optional - controls the padding method used when :attr: center is True . Default: \"reflect\" onesided bool, optional - controls whether to return half of results to avoid redundancy Default: True","title":"LogSpectrogram Objects"},{"location":"development/devel/api/#forward","text":"def forward(waveform: torch.Tensor) -> torch.Tensor Arguments : waveform Tensor - Tensor of audio of dimension (..., time). Returns : Tensor - Dimension (..., freq, time), where freq is n_fft // 2 + 1 where n_fft is the number of Fourier bins, and time is the number of window hops (n_frame).","title":"forward"},{"location":"development/devel/api/#hannahnormalizer","text":"","title":"hannah.normalizer"},{"location":"development/devel/api/#fixedpointnormalizer-objects","text":"class FixedPointNormalizer(nn.Module) Simple feature normalizer for fixed point models","title":"FixedPointNormalizer Objects"},{"location":"development/devel/api/#adaptivefixedpointnormalizer-objects","text":"class AdaptiveFixedPointNormalizer(nn.Module) Simple feature normalizer for fixed point models","title":"AdaptiveFixedPointNormalizer Objects"},{"location":"development/devel/api/#hannahmodelshonk","text":"","title":"hannah.models.honk"},{"location":"development/devel/api/#hannahmodelshonkmodel","text":"","title":"hannah.models.honk.model"},{"location":"development/devel/api/#truncated_normal","text":"def truncated_normal(tensor, std_dev=0.01) Arguments : tensor: - std_dev - (Default value = 0.01)","title":"truncated_normal"},{"location":"development/devel/api/#speechresmodel-objects","text":"class SpeechResModel(nn.Module)","title":"SpeechResModel Objects"},{"location":"development/devel/api/#forward_1","text":"def forward(x) Arguments : x:","title":"forward"},{"location":"development/devel/api/#speechmodel-objects","text":"class SpeechModel(nn.Module)","title":"SpeechModel Objects"},{"location":"development/devel/api/#forward_2","text":"def forward(x) Arguments : x:","title":"forward"},{"location":"development/devel/api/#hannahmodelshellomodels","text":"","title":"hannah.models.hello.models"},{"location":"development/devel/api/#dsconv2d-objects","text":"class DSConv2d(nn.Module)","title":"DSConv2d Objects"},{"location":"development/devel/api/#forward_3","text":"def forward(x) Arguments : x:","title":"forward"},{"location":"development/devel/api/#dscnnspeechmodel-objects","text":"class DSCNNSpeechModel(nn.Module)","title":"DSCNNSpeechModel Objects"},{"location":"development/devel/api/#forward_4","text":"def forward(x) Arguments : x:","title":"forward"},{"location":"development/devel/api/#dnnspeechmodel-objects","text":"class DNNSpeechModel(nn.Module)","title":"DNNSpeechModel Objects"},{"location":"development/devel/api/#forward_5","text":"def forward(x) Arguments : x:","title":"forward"},{"location":"development/devel/api/#hannahmodelshello","text":"","title":"hannah.models.hello"},{"location":"development/devel/api/#hannahmodelsobjectdetectionloss","text":"","title":"hannah.models.objectdetection.loss"},{"location":"development/devel/api/#bbox_iou","text":"def bbox_iou(box1, box2, x1y1x2y2=True, GIoU=False, DIoU=False, CIoU=False, eps=1e-7) Arguments : box1: box2: - x1y1x2y2 - (Default value = True) - GIoU - (Default value = False) - DIoU - (Default value = False) - CIoU - (Default value = False) - eps - (Default value = 1e-7)","title":"bbox_iou"},{"location":"development/devel/api/#is_parallel","text":"def is_parallel(model) Arguments : model:","title":"is_parallel"},{"location":"development/devel/api/#bceblurwithlogitsloss-objects","text":"class BCEBlurWithLogitsLoss(nn.Module) Arguments : eps - (Default value = 0.1) ) - # https://github.com/ultralytics/yolov3/issues/238#issuecomment-598028441# return positive: negative label smoothing BCE targetsreturn 1.0 - 0.5 * eps: 0.5 * epsclass BCEBlurWithLogitsLoss(nn.Module: ) - # https://github.com/ultralytics/yolov3/issues/238#issuecomment-598028441# return positive:","title":"BCEBlurWithLogitsLoss Objects"},{"location":"development/devel/api/#forward_6","text":"def forward(pred, true) Arguments : pred: true:","title":"forward"},{"location":"development/devel/api/#focalloss-objects","text":"class FocalLoss(nn.Module)","title":"FocalLoss Objects"},{"location":"development/devel/api/#forward_7","text":"def forward(pred, true) Arguments : pred: true:","title":"forward"},{"location":"development/devel/api/#qfocalloss-objects","text":"class QFocalLoss(nn.Module)","title":"QFocalLoss Objects"},{"location":"development/devel/api/#forward_8","text":"def forward(pred, true) Arguments : pred: true:","title":"forward"},{"location":"development/devel/api/#computeloss-objects","text":"class ComputeLoss()","title":"ComputeLoss Objects"},{"location":"development/devel/api/#build_targets","text":"def build_targets(p, targets) Arguments : p: targets:","title":"build_targets"},{"location":"development/devel/api/#hannahmodelsobjectdetectionmodels","text":"","title":"hannah.models.objectdetection.models"},{"location":"development/devel/api/#fasterrcnn-objects","text":"class FasterRCNN(torch.nn.Module)","title":"FasterRCNN Objects"},{"location":"development/devel/api/#forward_9","text":"def forward(x, y=None) Arguments : x: - y - (Default value = None)","title":"forward"},{"location":"development/devel/api/#transformoutput","text":"def transformOutput(cocoGt, output, x, y) Arguments : cocoGt: output: x: y:","title":"transformOutput"},{"location":"development/devel/api/#ultralyticsyolo-objects","text":"class UltralyticsYolo(torch.nn.Module)","title":"UltralyticsYolo Objects"},{"location":"development/devel/api/#forward_10","text":"def forward(x, y=None) Arguments : x: - y - (Default value = None)","title":"forward"},{"location":"development/devel/api/#train","text":"def train(mode=True) Arguments : mode - (Default value = True)","title":"train"},{"location":"development/devel/api/#transformoutput_1","text":"def transformOutput(cocoGt, output, x, y) Arguments : cocoGt: output: x: y:","title":"transformOutput"},{"location":"development/devel/api/#hannahmodelsobjectdetection","text":"","title":"hannah.models.objectdetection"},{"location":"development/devel/api/#hannahmodelstcmodels","text":"","title":"hannah.models.tc.models"},{"location":"development/devel/api/#create_act","text":"def create_act(act, clipping_value) Arguments : act: clipping_value:","title":"create_act"},{"location":"development/devel/api/#approximateglobalaveragepooling1d-objects","text":"class ApproximateGlobalAveragePooling1D(nn.Module)","title":"ApproximateGlobalAveragePooling1D Objects"},{"location":"development/devel/api/#forward_11","text":"def forward(x) Arguments : x:","title":"forward"},{"location":"development/devel/api/#tcresidualblock-objects","text":"class TCResidualBlock(nn.Module)","title":"TCResidualBlock Objects"},{"location":"development/devel/api/#forward_12","text":"def forward(x) Arguments : x:","title":"forward"},{"location":"development/devel/api/#tcresnetmodel-objects","text":"class TCResNetModel(nn.Module)","title":"TCResNetModel Objects"},{"location":"development/devel/api/#forward_13","text":"def forward(x) Arguments : x:","title":"forward"},{"location":"development/devel/api/#exitwrapperblock-objects","text":"class ExitWrapperBlock(nn.Module)","title":"ExitWrapperBlock Objects"},{"location":"development/devel/api/#forward_14","text":"def forward(x) Arguments : x:","title":"forward"},{"location":"development/devel/api/#branchytcresnetmodel-objects","text":"class BranchyTCResNetModel(TCResNetModel)","title":"BranchyTCResNetModel Objects"},{"location":"development/devel/api/#on_val","text":"def on_val()","title":"on_val"},{"location":"development/devel/api/#on_val_end","text":"def on_val_end()","title":"on_val_end"},{"location":"development/devel/api/#on_test","text":"def on_test()","title":"on_test"},{"location":"development/devel/api/#on_test_end","text":"def on_test_end()","title":"on_test_end"},{"location":"development/devel/api/#reset_stats","text":"def reset_stats()","title":"reset_stats"},{"location":"development/devel/api/#print_stats","text":"def print_stats()","title":"print_stats"},{"location":"development/devel/api/#forward_15","text":"def forward(x) Arguments : x:","title":"forward"},{"location":"development/devel/api/#get_loss_function","text":"def get_loss_function()","title":"get_loss_function"},{"location":"development/devel/api/#hannahmodelstc","text":"","title":"hannah.models.tc"},{"location":"development/devel/api/#hannahmodelsconvnetmodels","text":"","title":"hannah.models.convnet.models"},{"location":"development/devel/api/#padding_expression","text":"def padding_expression(kernel_size, stride, dilation=1) Symbolically calculate padding such that for a given kernel_size, stride and dilation the padding is such that the output dimension is kept the same(stride=1) or halved(stride=2). Note: If the input dimension is 1 and stride = 2, the calculated padding will result in an output with also dimension 1.","title":"padding_expression"},{"location":"development/devel/api/#parameters","text":"kernel_size : Union[int, Expression] stride : Union[int, Expression] dilation : Union[int, Expression], optional description , by default 1","title":"Parameters"},{"location":"development/devel/api/#returns","text":"Expression","title":"Returns"},{"location":"development/devel/api/#hannahmodelsconvnet","text":"","title":"hannah.models.convnet"},{"location":"development/devel/api/#hannahmodelstimm","text":"","title":"hannah.models.timm"},{"location":"development/devel/api/#defaultanomalydetector-objects","text":"class DefaultAnomalyDetector(nn.Module)","title":"DefaultAnomalyDetector Objects"},{"location":"development/devel/api/#forward_16","text":"def forward(x) Arguments : x:","title":"forward"},{"location":"development/devel/api/#defaultclassifierhead-objects","text":"class DefaultClassifierHead(nn.Module)","title":"DefaultClassifierHead Objects"},{"location":"development/devel/api/#forward_17","text":"def forward(x: torch.Tensor) -> torch.Tensor Arguments : x (torch.Tensor): Returns : Resulting torch.Tensor after applying classification","title":"forward"},{"location":"development/devel/api/#defaultprojectionhead-objects","text":"class DefaultProjectionHead(nn.Module) Default projection head for semi supervised classification learning","title":"DefaultProjectionHead Objects"},{"location":"development/devel/api/#forward_18","text":"def forward(x: torch.Tensor) -> torch.Tensor Forward function for default Projection Head Arguments : x - Input tensor Returns : output tensor","title":"forward"},{"location":"development/devel/api/#defaultdecoderhead-objects","text":"class DefaultDecoderHead(nn.Module)","title":"DefaultDecoderHead Objects"},{"location":"development/devel/api/#__init__","text":"def __init__(latent_shape, input_shape) Default Decoder Head for autoencoders using TransposedConv2D Arguments : latent_shape(Tuple) - Shape (CxHxW) of the latent representation of the autoencoder input_shape(Tuple) - Shape (CxHxW) of the reconstructed image","title":"__init__"},{"location":"development/devel/api/#forward_19","text":"def forward(x) Arguments : x:","title":"forward"},{"location":"development/devel/api/#timmmodel-objects","text":"class TimmModel(nn.Module)","title":"TimmModel Objects"},{"location":"development/devel/api/#forward_20","text":"def forward(x: torch.Tensor) -> torch.Tensor Arguments : x - torch.Tensor: x - torch.Tensor:","title":"forward"},{"location":"development/devel/api/#hannahmodels","text":"","title":"hannah.models"},{"location":"development/devel/api/#hannahmodelslstmmodels","text":"","title":"hannah.models.lstm.models"},{"location":"development/devel/api/#lstmmodel-objects","text":"class LSTMModel(nn.Module) Simple LSTM model.","title":"LSTMModel Objects"},{"location":"development/devel/api/#forward_21","text":"def forward(x) Arguments : x:","title":"forward"},{"location":"development/devel/api/#hannahmodelslstm","text":"","title":"hannah.models.lstm"},{"location":"development/devel/api/#hannahmodelsfactoryreduction","text":"","title":"hannah.models.factory.reduction"},{"location":"development/devel/api/#reductionblockadd-objects","text":"class ReductionBlockAdd(nn.Module) Reduction block that sums over its inputs","title":"ReductionBlockAdd Objects"},{"location":"development/devel/api/#forward_22","text":"def forward(x: Tensor) -> Tensor Arguments : x - Tensor: x - Tensor:","title":"forward"},{"location":"development/devel/api/#reductionblockconcat-objects","text":"class ReductionBlockConcat(nn.Module) Reduction block that concatenates its inputs","title":"ReductionBlockConcat Objects"},{"location":"development/devel/api/#forward_23","text":"def forward(x: Tensor) -> Tensor Arguments : x - Tensor: x - Tensor:","title":"forward"},{"location":"development/devel/api/#hannahmodelsfactoryact","text":"","title":"hannah.models.factory.act"},{"location":"development/devel/api/#dummyactivation-objects","text":"class DummyActivation(nn.Identity) Dummy class that instantiated to mark a missing activation. This can be used to mark requantization of activations for convolutional layers without activation functions. Arguments :","title":"DummyActivation Objects"},{"location":"development/devel/api/#hannahmodelsfactory","text":"","title":"hannah.models.factory"},{"location":"development/devel/api/#hannahmodelsfactoryquantized","text":"Import from new loacation for backwards compatibility","title":"hannah.models.factory.quantized"},{"location":"development/devel/api/#hannahmodelsfactoryqconfig","text":"Import from new loacation for backwards compatibility","title":"hannah.models.factory.qconfig"},{"location":"development/devel/api/#hannahmodelsfactorynetwork","text":"","title":"hannah.models.factory.network"},{"location":"development/devel/api/#convnet-objects","text":"class ConvNet(nn.Module)","title":"ConvNet Objects"},{"location":"development/devel/api/#forward_24","text":"def forward(x) Arguments : x:","title":"forward"},{"location":"development/devel/api/#hannahmodelsfactoryfactory","text":"A neural network model factory It allows us to construct quantized and unquantized versions of the same network, allows to explore implementation alternatives using a common neural network construction interface.","title":"hannah.models.factory.factory"},{"location":"development/devel/api/#normconfig-objects","text":"@dataclass class NormConfig()","title":"NormConfig Objects"},{"location":"development/devel/api/#bnconfig-objects","text":"@dataclass class BNConfig(NormConfig)","title":"BNConfig Objects"},{"location":"development/devel/api/#actconfig-objects","text":"@dataclass class ActConfig()","title":"ActConfig Objects"},{"location":"development/devel/api/#eluconfig-objects","text":"@dataclass class ELUConfig(ActConfig)","title":"ELUConfig Objects"},{"location":"development/devel/api/#hardtanhconfig-objects","text":"@dataclass class HardtanhConfig(ActConfig)","title":"HardtanhConfig Objects"},{"location":"development/devel/api/#minorblockconfig-objects","text":"@dataclass class MinorBlockConfig()","title":"MinorBlockConfig Objects"},{"location":"development/devel/api/#target","text":"target Operation","title":"target"},{"location":"development/devel/api/#parallel","text":"execute block in parallel with preceding block","title":"parallel"},{"location":"development/devel/api/#out_channels","text":"number of output channels","title":"out_channels"},{"location":"development/devel/api/#kernel_size","text":"kernel size of this Operation (if applicable)","title":"kernel_size"},{"location":"development/devel/api/#stride","text":"stride for this operation use","title":"stride"},{"location":"development/devel/api/#padding","text":"use padding for this operation (padding will always try to keep input dimensions / stride)","title":"padding"},{"location":"development/devel/api/#dilation","text":"dilation factor to use for this operation","title":"dilation"},{"location":"development/devel/api/#groups","text":"number of groups for this operation","title":"groups"},{"location":"development/devel/api/#norm","text":"normalization to use (true uses networks default configs)","title":"norm"},{"location":"development/devel/api/#act","text":"activation to use (true uses default configs)","title":"act"},{"location":"development/devel/api/#upsampling","text":"Upsampling factor for mbconv layers","title":"upsampling"},{"location":"development/devel/api/#bias","text":"use bias for this operation","title":"bias"},{"location":"development/devel/api/#out_quant","text":"use output quantization for this operation","title":"out_quant"},{"location":"development/devel/api/#majorblockconfig-objects","text":"@dataclass class MajorBlockConfig()","title":"MajorBlockConfig Objects"},{"location":"development/devel/api/#stride_1","text":"Union[None, int, Tuple[int, ...], Tuple[int, ...]]","title":"stride"},{"location":"development/devel/api/#last","text":"Indicates wether this block is the last reduction block","title":"last"},{"location":"development/devel/api/#linearconfig-objects","text":"@dataclass class LinearConfig()","title":"LinearConfig Objects"},{"location":"development/devel/api/#norm_1","text":"Union[bool, NormConfig]","title":"norm"},{"location":"development/devel/api/#act_1","text":"Union[bool, ActConfig]","title":"act"},{"location":"development/devel/api/#networkconfig-objects","text":"@dataclass class NetworkConfig()","title":"NetworkConfig Objects"},{"location":"development/devel/api/#networkfactory-objects","text":"class NetworkFactory()","title":"NetworkFactory Objects"},{"location":"development/devel/api/#act_2","text":"def act(config: ActConfig) -> nn.Module Arguments : config - ActConfig: config - ActConfig:","title":"act"},{"location":"development/devel/api/#conv2d","text":"def conv2d(input_shape: Tuple[int, ...], out_channels: int, kernel_size: Union[int, Tuple[int, ...]], stride: Union[int, Tuple[int, ...]] = 1, padding: Union[int, Tuple[int, ...], bool] = True, dilation: Union[int, Tuple[int, ...]] = 0, groups: int = 1, norm: Union[BNConfig, bool] = False, act: Union[ActConfig, bool] = False, bias: bool = False) -> None Arguments : input_shape - Tuple[int: int: int]: out_channels - int: kernel_size - Union[int: Tuple[int, ...]]: (Default value = 1) stride - Union[int: padding - Union[int: Tuple[int, ...]: bool] - (Default value = False) dilation - int: (Default value = 0) groups - int: (Default value = 1) norm - Union[BNConfig: act - Union[ActConfig: bias - bool: (Default value = False) input_shape - Tuple[int: out_channels - int: kernel_size - Union[int: stride - Union[int: padding - Union[int: dilation - int: (Default value = 0) groups - int: (Default value = 1) norm - Union[BNConfig: act - Union[ActConfig: bias - bool: (Default value = False)","title":"conv2d"},{"location":"development/devel/api/#mbconv1d","text":"def mbconv1d(input_shape: Tuple[int, ...], out_channels: int, kernel_size: int, dilation: int = 1, stride: int = 1, padding: Union[int, bool] = True, bias=False, upsampling: float = 1.0, groups: int = 1, norm: Union[BNConfig, bool] = False, act: Union[ActConfig, bool] = False) Arguments : input_shape - Tuple[int: int: int]: out_channels - int: kernel_size - int: dilation - int: (Default value = 1) stride - int: (Default value = 1) padding - Union[int: bool] - (Default value = False) bias - (Default value = False) upsampling - float: (Default value = 1.0) groups - int: (Default value = 1) norm - Union[BNConfig: act - Union[ActConfig: input_shape - Tuple[int: out_channels - int: kernel_size - int: dilation - int: (Default value = 1) stride - int: (Default value = 1) padding - Union[int: upsampling - float: (Default value = 1.0) groups - int: (Default value = 1) norm - Union[BNConfig: act - Union[ActConfig:","title":"mbconv1d"},{"location":"development/devel/api/#conv1d","text":"def conv1d(input_shape: Tuple[int, ...], out_channels: int, kernel_size: int, stride: int = 1, bias: bool = False, padding: Union[int, bool] = True, dilation: int = 1, groups: int = 1, norm: Union[BNConfig, bool] = False, act: Union[ActConfig, bool] = False, out_quant: bool = True) -> Tuple[Tuple[int, ...], nn.Module] Arguments : input_shape - Tuple[int: int: int]: out_channels - int: kernel_size - int: stride - int: (Default value = 1) bias - bool: (Default value = False) padding - Union[int: bool] - (Default value = False) dilation - int: (Default value = 1) groups - int: (Default value = 1) norm - Union[BNConfig: act - Union[ActConfig: out_quant - bool: (Default value = True) input_shape - Tuple[int: out_channels - int: kernel_size - int: stride - int: (Default value = 1) bias - bool: (Default value = False) padding - Union[int: dilation - int: (Default value = 1) groups - int: (Default value = 1) norm - Union[BNConfig: act - Union[ActConfig: out_quant - bool: (Default value = True)","title":"conv1d"},{"location":"development/devel/api/#minor","text":"def minor(input_shape, config: MinorBlockConfig, major_stride=None) Arguments : input_shape: - config - MinorBlockConfig: - major_stride - (Default value = None) - config - MinorBlockConfig:","title":"minor"},{"location":"development/devel/api/#forward_25","text":"def forward(input_shape: Tuple[int, ...], config: MajorBlockConfig) Create a forward neural network block without parallelism If parallel is set to [True, False, True, False] Input: ------->|---> parallel: False ---> parallel: False ---> | --> output Arguments : input_shape - Tuple[int, ...]: config - MajorBlockConfig: input_shape - Tuple[int, ...]: config - MajorBlockConfig:","title":"forward"},{"location":"development/devel/api/#residual","text":"def residual(input_shape: Tuple[int, ...], config: MajorBlockConfig) Create a neural network block with with residual parallelism If parallel is set to [True, False, True, False] |---> parallel: True ---> parallel: True ---> | Input: ------->| +---> |---> parallel: False ---> parallel: False ---> | If the major block does change the output dimensions compared to the input and one of the branches does not contain any layers, we infer 1x1 conv of maximum group size (gcd (input_channels, output_channels)) to do the downsampling. Arguments : input_shape - Tuple[int, ...]: config - MajorBlockConfig: input_shape - Tuple[int, ...]: config - MajorBlockConfig:","title":"residual"},{"location":"development/devel/api/#input","text":"def input(in_channels: int, config: MajorBlockConfig) Create a neural network block with input parallelism If parallel is set to [True, False, True, False] |---> parallel: True ---> | |---> parallel: True ---> + -----------------> | Input:--------->| +---> |---> parallel: False ---> parallel: False ---> | If there are no parallel branches in the network. The major block is a standard feed forward layer. Arguments : in_channels - int: config - MajorBlockConfig: in_channels - int: config - MajorBlockConfig:","title":"input"},{"location":"development/devel/api/#full","text":"def full(in_channels: int, config: MajorBlockConfig) Create a neural network block with full parallelism If parallel is set to [True, False, True, False] |---> parallel: True ---------------------------------- -| Input:--->| +---> | |--> parallel: False --->| | |---> parallel: False ----> | +--->| |--> parallel: True ---->| If there are no parallel blocks the block is a standard feed forward network. Arguments : in_channels - int: config - MajorBlockConfig: in_channels - int: config - MajorBlockConfig:","title":"full"},{"location":"development/devel/api/#major","text":"def major(input_shape, config: MajorBlockConfig) Arguments : input_shape: - config - MajorBlockConfig: - config - MajorBlockConfig:","title":"major"},{"location":"development/devel/api/#linear","text":"def linear(input_shape, config: LinearConfig) Arguments : input_shape: - config - LinearConfig: - config - LinearConfig:","title":"linear"},{"location":"development/devel/api/#identity","text":"def identity() -> Identity","title":"identity"},{"location":"development/devel/api/#network","text":"def network(input_shape, labels: int, network_config: Union[ListConfig, DictConfig]) Arguments : input_shape: - labels - int: - network_config - NetworkConfig: - labels - int: - network_config - NetworkConfig:","title":"network"},{"location":"development/devel/api/#create_cnn","text":"def create_cnn(input_shape: Sequence[int], labels: int, name: str, conv: Optional[List[MajorBlockConfig]] = None, linear: Optional[List[LinearConfig]] = None, norm: Optional[NormConfig] = None, act: Optional[ActConfig] = None, qconfig: Any = None, dropout: float = 0.5) Arguments : input_shape - Sequence[int]: labels - int: name - str: conv - Optional[List[MajorBlockConfig]]: (Default value = None) linear - Optional[List[LinearConfig]]: (Default value = None) norm - Optional[NormConfig]: (Default value = None) act - Optional[ActConfig]: (Default value = None) qconfig - Any: (Default value = None) dropout - float: (Default value = 0.5) input_shape - Sequence[int]: labels - int: name - str: conv - Optional[List[MajorBlockConfig]]: (Default value = None) linear - Optional[List[LinearConfig]]: (Default value = None) norm - Optional[NormConfig]: (Default value = None) act - Optional[ActConfig]: (Default value = None) qconfig - Any: (Default value = None) dropout - float: (Default value = 0.5)","title":"create_cnn"},{"location":"development/devel/api/#hannahmodelsfactorypooling","text":"","title":"hannah.models.factory.pooling"},{"location":"development/devel/api/#approximateglobalaveragepooling1d-objects_1","text":"class ApproximateGlobalAveragePooling1D(nn.Module) A global average pooling layer, that divides by the next power of 2 instead of true number of elements","title":"ApproximateGlobalAveragePooling1D Objects"},{"location":"development/devel/api/#forward_26","text":"def forward(x) Arguments : x:","title":"forward"},{"location":"development/devel/api/#approximateglobalaveragepooling2d-objects","text":"class ApproximateGlobalAveragePooling2D(nn.Module) A global average pooling layer, that divides by the next power of 2 instead of true number of elements","title":"ApproximateGlobalAveragePooling2D Objects"},{"location":"development/devel/api/#forward_27","text":"def forward(x) Arguments : x:","title":"forward"},{"location":"development/devel/api/#hannahmodelsfactoryqat","text":"Import from new loacation for backwards compatibility","title":"hannah.models.factory.qat"},{"location":"development/devel/api/#hannahmodelsfactoryrounding","text":"Import from new loacation for backwards compatibility","title":"hannah.models.factory.rounding"},{"location":"development/devel/api/#hannahmodelsresnetmodels","text":"","title":"hannah.models.resnet.models"},{"location":"development/devel/api/#padding_expression_1","text":"def padding_expression(kernel_size, stride, dilation=1) Symbolically calculate padding such that for a given kernel_size, stride and dilation the padding is such that the output dimension is kept the same(stride=1) or halved(stride=2). Note: If the input dimension is 1 and stride = 2, the calculated padding will result in an output with also dimension 1.","title":"padding_expression"},{"location":"development/devel/api/#parameters_1","text":"kernel_size : Union[int, Expression] stride : Union[int, Expression] dilation : Union[int, Expression], optional description , by default 1","title":"Parameters"},{"location":"development/devel/api/#returns_1","text":"Expression","title":"Returns"},{"location":"development/devel/api/#hannahmodelsresnet","text":"","title":"hannah.models.resnet"},{"location":"development/devel/api/#hannahmodelsvadmodels","text":"","title":"hannah.models.vad.models"},{"location":"development/devel/api/#bottleneckvad-objects","text":"class BottleneckVad(nn.Module)","title":"BottleneckVad Objects"},{"location":"development/devel/api/#forward_28","text":"def forward(x) Arguments : x:","title":"forward"},{"location":"development/devel/api/#num_flat_features","text":"def num_flat_features(x) Arguments : x:","title":"num_flat_features"},{"location":"development/devel/api/#smallvad-objects","text":"class SmallVad(nn.Module)","title":"SmallVad Objects"},{"location":"development/devel/api/#forward_29","text":"def forward(x) Arguments : x:","title":"forward"},{"location":"development/devel/api/#num_flat_features_1","text":"def num_flat_features(x) Arguments : x:","title":"num_flat_features"},{"location":"development/devel/api/#simplevad-objects","text":"class SimpleVad(nn.Module)","title":"SimpleVad Objects"},{"location":"development/devel/api/#forward_30","text":"def forward(x) Arguments : x:","title":"forward"},{"location":"development/devel/api/#num_flat_features_2","text":"def num_flat_features(x) Arguments : x:","title":"num_flat_features"},{"location":"development/devel/api/#bottleneckvadmodel-objects","text":"class BottleneckVadModel(nn.Module)","title":"BottleneckVadModel Objects"},{"location":"development/devel/api/#forward_31","text":"def forward(x) Arguments : x:","title":"forward"},{"location":"development/devel/api/#simplevadmodel-objects","text":"class SimpleVadModel(nn.Module)","title":"SimpleVadModel Objects"},{"location":"development/devel/api/#forward_32","text":"def forward(x) Arguments : x:","title":"forward"},{"location":"development/devel/api/#smallvadmodel-objects","text":"class SmallVadModel(nn.Module)","title":"SmallVadModel Objects"},{"location":"development/devel/api/#forward_33","text":"def forward(x) Arguments : x:","title":"forward"},{"location":"development/devel/api/#hannahmodelsvad","text":"","title":"hannah.models.vad"},{"location":"development/devel/api/#hannahmodelsofasubmoduleselasticlinear","text":"","title":"hannah.models.ofa.submodules.elasticLinear"},{"location":"development/devel/api/#elasticwidthlinear-objects","text":"class ElasticWidthLinear(nn.Linear, _Elastic)","title":"ElasticWidthLinear Objects"},{"location":"development/devel/api/#forward_34","text":"def forward(input: torch.Tensor) -> torch.Tensor Arguments : input - torch.Tensor: input - torch.Tensor: input - torch.Tensor: input - torch.Tensor:","title":"forward"},{"location":"development/devel/api/#get_basic_module","text":"def get_basic_module()","title":"get_basic_module"},{"location":"development/devel/api/#elasticquantwidthlinear-objects","text":"class ElasticQuantWidthLinear(nn.Linear, _Elastic)","title":"ElasticQuantWidthLinear Objects"},{"location":"development/devel/api/#filtered_weight","text":"@property def filtered_weight()","title":"filtered_weight"},{"location":"development/devel/api/#filtered_bias","text":"@property def filtered_bias()","title":"filtered_bias"},{"location":"development/devel/api/#scaled_weight","text":"@property def scaled_weight()","title":"scaled_weight"},{"location":"development/devel/api/#scaled_bias","text":"@property def scaled_bias()","title":"scaled_bias"},{"location":"development/devel/api/#forward_35","text":"def forward(input: torch.Tensor) -> torch.Tensor Arguments : input - torch.Tensor: input - torch.Tensor: input - torch.Tensor: input - torch.Tensor:","title":"forward"},{"location":"development/devel/api/#get_basic_module_1","text":"def get_basic_module()","title":"get_basic_module"},{"location":"development/devel/api/#from_float","text":"@classmethod def from_float(cls, mod) Create a qat module from a float module or qparams_dict Args: mod a float module, either produced by torch.quantization utilities or directly from user Arguments : mod:","title":"from_float"},{"location":"development/devel/api/#elasticpermissiverelu-objects","text":"class ElasticPermissiveReLU(nn.ReLU)","title":"ElasticPermissiveReLU Objects"},{"location":"development/devel/api/#forward_36","text":"def forward(x) Arguments : x:","title":"forward"},{"location":"development/devel/api/#assemble_basic_module","text":"def assemble_basic_module()","title":"assemble_basic_module"},{"location":"development/devel/api/#hannahmodelsofasubmoduleselasticbase","text":"","title":"hannah.models.ofa.submodules.elasticBase"},{"location":"development/devel/api/#_elastic-objects","text":"class _Elastic()","title":"_Elastic Objects"},{"location":"development/devel/api/#get_basic_module_2","text":"def get_basic_module() -> nn.Module","title":"get_basic_module"},{"location":"development/devel/api/#get_out_channel_sizes","text":"def get_out_channel_sizes()","title":"get_out_channel_sizes"},{"location":"development/devel/api/#set_out_channel_filter","text":"def set_out_channel_filter(out_channel_filter) Arguments : ) -> nn.Module: # Module: so that Sequentials are possible: like DSCreturn copy.deepcopy(self.get_basic_module(): out_channel_filter: ) -> nn.Module: # Module: like DSCreturn copy.deepcopy(self.get_basic_module(): ) -> nn.Module: # Module: like DSCreturn copy.deepcopy(self.get_basic_module(): ) -> nn.Module: # Module: like DSCreturn copy.deepcopy(self.get_basic_module())set_out_channel_filter(self:","title":"set_out_channel_filter"},{"location":"development/devel/api/#set_in_channel_filter","text":"def set_in_channel_filter(in_channel_filter) Arguments : in_channel_filter:","title":"set_in_channel_filter"},{"location":"development/devel/api/#elasticbase1d-objects","text":"class ElasticBase1d(nn.Conv1d, _Elastic)","title":"ElasticBase1d Objects"},{"location":"development/devel/api/#set_bn_parameter","text":"def set_bn_parameter(conv: nn.Conv1d, tmp_bn, num_tracked) Caller for BatchNorm Parameters This unifies the call in the different methods, especially in dsc / not dsc forward And assigns the attributes in tmp_bn to the param conv Arguments : conv - nn.Conv1d: tmp_bn: num_tracked: conv - nn.Conv1d: conv - nn.Conv1d: conv - nn.Conv1d:","title":"set_bn_parameter"},{"location":"development/devel/api/#prepare_dsc_for_validation_model","text":"def prepare_dsc_for_validation_model(conv_class: nn.Module, full_kernel, full_bias, in_channels, out_channels, grouping, stride, padding, dilation, qconfig=None, out_quant=None, bn_eps=None, bn_momentum=None, bn_caller: tuple = None) This method creates the necessary validation models for DSC. It creates the validation model as torch.Sequence of standard pytorch convolution models. The structure is analog to the DSC method do_dsc. This method can also handle quantization models. Arguments : conv_class - nn.Module: full_kernel: full_bias: in_channels: out_channels: grouping: stride: padding: dilation: # for quantqconfig: (Default value = None) out_quant - (Default value = None) bn_eps - (Default value = None) bn_momentum - (Default value = None) bn_caller - tuple: (Default value = None) conv_class - nn.Module: bn_caller - tuple: (Default value = None) conv_class - nn.Module: bn_caller - tuple: (Default value = None) conv_class - nn.Module: bn_caller - tuple: (Default value = None)","title":"prepare_dsc_for_validation_model"},{"location":"development/devel/api/#do_dsc","text":"def do_dsc(input, full_kernel, full_bias, grouping, stride, padding, dilation, quant_weight=None, quant_bias=None, quant_weight_function: Function = None, quant_bias_function: Function = None) This method will perform the DSC(=Depthwise Separable Convolution). This method can also handle quantized models. DSC is done in two steps: 1. Depthwise Separable: Set Group = In_Channels, Output = k*In_Channels 2. Pointwise Convolution, with Grouping = Grouping-Param und Out_Channel = Out_Channel-Param The Params above are used for quantized models Arguments : input: full_kernel: full_bias: grouping: stride: padding: dilation: - quant_weight - (Default value = None) - quant_bias - (Default value = None) - quant_weight_function - Function: (Default value = None) # for quantizationquant_bias_function: Function: (Default value = None) # for quantization: - quant_weight_function - Function: (Default value = None) # for quantizationquant_bias_function: Function: (Default value = None) - quant_weight_function - Function: (Default value = None) # for quantizationquant_bias_function: Function: (Default value = None) - quant_weight_function - Function: (Default value = None) # for quantizationquant_bias_function: Function: (Default value = None)","title":"do_dsc"},{"location":"development/devel/api/#set_in_and_out_channel","text":"def set_in_and_out_channel(kernel, filtered: bool = True) This method uses the kernel for setting the input and outputchannel if dynamic width is activated (channelfilters), the amount of channels is reduced, hence we can't use the initial values (self.(in/out)_channel) of the constructor This method sets the self.(in/out)_channel value to the right amount of channels extracted from the kernel that will be used. if filtered is False, the self.initial_(in/out)_channels will be used. The previous values will be stored in the attribute prev_in_channels and prev_out_channels. Arguments : kernel: - filtered - bool: (Default value = True) - filtered - bool: (Default value = True) - filtered - bool: (Default value = True) - filtered - bool: (Default value = True)","title":"set_in_and_out_channel"},{"location":"development/devel/api/#reset_in_and_out_channel_to_previous","text":"def reset_in_and_out_channel_to_previous() Analog to set_in_and_out_channels: Resets the in and out_channels Arguments :","title":"reset_in_and_out_channel_to_previous"},{"location":"development/devel/api/#set_kernel_size","text":"def set_kernel_size(new_kernel_size) If the requested kernel size is outside of the min/max range, clamp it to the min/max range. If the requested kernel size is not an available kernel size, default to the max kernel size Arguments : new_kernel_size - int): the size of the kernel you want to use","title":"set_kernel_size"},{"location":"development/devel/api/#reset_kernel_size","text":"def reset_kernel_size()","title":"reset_kernel_size"},{"location":"development/devel/api/#step_down_kernel_size","text":"def step_down_kernel_size()","title":"step_down_kernel_size"},{"location":"development/devel/api/#pick_kernel_index","text":"def pick_kernel_index(target_kernel_index: int) Arguments : target_kernel_index - int: target_kernel_index - int: target_kernel_index - int: target_kernel_index - int:","title":"pick_kernel_index"},{"location":"development/devel/api/#get_available_kernel_steps","text":"def get_available_kernel_steps()","title":"get_available_kernel_steps"},{"location":"development/devel/api/#get_full_width_kernel","text":"def get_full_width_kernel() It applies the kernel transformations to the kernel until the target kernel index is reached Returns : The found target kernel. Args: Returns:","title":"get_full_width_kernel"},{"location":"development/devel/api/#get_kernel","text":"def get_kernel() If the input and output channels are not filtered, the full kernel is Arguments : Returns : return: The new kernel and bias.","title":"get_kernel"},{"location":"development/devel/api/#set_dilation_size","text":"def set_dilation_size(new_dilation_size) Arguments : new_dilation_size:","title":"set_dilation_size"},{"location":"development/devel/api/#update_padding","text":"def update_padding()","title":"update_padding"},{"location":"development/devel/api/#reset_dilation_size","text":"def reset_dilation_size()","title":"reset_dilation_size"},{"location":"development/devel/api/#step_down_dilation_size","text":"def step_down_dilation_size()","title":"step_down_dilation_size"},{"location":"development/devel/api/#pick_dilation_index","text":"def pick_dilation_index(target_dilation_index: int) Arguments : target_dilation_index - int: target_dilation_index - int: target_dilation_index - int: target_dilation_index - int:","title":"pick_dilation_index"},{"location":"development/devel/api/#get_available_dilation_steps","text":"def get_available_dilation_steps()","title":"get_available_dilation_steps"},{"location":"development/devel/api/#get_available_grouping_steps","text":"def get_available_grouping_steps()","title":"get_available_grouping_steps"},{"location":"development/devel/api/#get_available_dsc_steps","text":"def get_available_dsc_steps()","title":"get_available_dsc_steps"},{"location":"development/devel/api/#get_dilation_size","text":"def get_dilation_size()","title":"get_dilation_size"},{"location":"development/devel/api/#pick_group_index","text":"def pick_group_index(target_group_index: int) Arguments : target_group_index - int: target_group_index - int: target_group_index - int: target_group_index - int:","title":"pick_group_index"},{"location":"development/devel/api/#pick_dsc_index","text":"def pick_dsc_index(target_dsc_index: int) Arguments : target_dsc_index - int: target_dsc_index - int: target_dsc_index - int: target_dsc_index - int:","title":"pick_dsc_index"},{"location":"development/devel/api/#reset_group_size","text":"def reset_group_size()","title":"reset_group_size"},{"location":"development/devel/api/#reset_dscs","text":"def reset_dscs()","title":"reset_dscs"},{"location":"development/devel/api/#get_group_size","text":"def get_group_size()","title":"get_group_size"},{"location":"development/devel/api/#get_dsc","text":"def get_dsc()","title":"get_dsc"},{"location":"development/devel/api/#set_group_size","text":"def set_group_size(new_group_size) Arguments : new_group_size:","title":"set_group_size"},{"location":"development/devel/api/#set_dsc","text":"def set_dsc(new_dsc) Arguments : new_dsc:","title":"set_dsc"},{"location":"development/devel/api/#step_down_group_size","text":"def step_down_group_size()","title":"step_down_group_size"},{"location":"development/devel/api/#step_down_dsc","text":"def step_down_dsc()","title":"step_down_dsc"},{"location":"development/devel/api/#adjust_weights_for_grouping","text":"def adjust_weights_for_grouping(weights, input_divided_by) Arguments : weights: input_divided_by:","title":"adjust_weights_for_grouping"},{"location":"development/devel/api/#forward_37","text":"def forward(input: torch.Tensor) -> torch.Tensor Arguments : input - torch.Tensor: input - torch.Tensor: input - torch.Tensor: input - torch.Tensor:","title":"forward"},{"location":"development/devel/api/#extra_repr","text":"def extra_repr()","title":"extra_repr"},{"location":"development/devel/api/#hannahmodelsofasubmodulesresblock","text":"","title":"hannah.models.ofa.submodules.resblock"},{"location":"development/devel/api/#resblockbase-objects","text":"class ResBlockBase(nn.Module)","title":"ResBlockBase Objects"},{"location":"development/devel/api/#forward_38","text":"def forward(x) Arguments : x:","title":"forward"},{"location":"development/devel/api/#get_nested_modules","text":"def get_nested_modules()","title":"get_nested_modules"},{"location":"development/devel/api/#resblock1d-objects","text":"class ResBlock1d(ResBlockBase)","title":"ResBlock1d Objects"},{"location":"development/devel/api/#forward_39","text":"def forward(x) Arguments : x:","title":"forward"},{"location":"development/devel/api/#get_input_layer","text":"def get_input_layer()","title":"get_input_layer"},{"location":"development/devel/api/#get_output_layer","text":"def get_output_layer()","title":"get_output_layer"},{"location":"development/devel/api/#create_internal_channelhelper","text":"def create_internal_channelhelper()","title":"create_internal_channelhelper"},{"location":"development/devel/api/#hannahmodelsofasubmodules","text":"","title":"hannah.models.ofa.submodules"},{"location":"development/devel/api/#hannahmodelsofasubmoduleselasticbatchnorm","text":"","title":"hannah.models.ofa.submodules.elasticBatchnorm"},{"location":"development/devel/api/#elasticwidthbatchnorm1d-objects","text":"class ElasticWidthBatchnorm1d(nn.BatchNorm1d)","title":"ElasticWidthBatchnorm1d Objects"},{"location":"development/devel/api/#forward_40","text":"def forward(input: torch.Tensor) -> torch.Tensor if self.track_running_stats: logging.warn( \"ElasticWidthBatchnorm with tracked running stats currently not fully implemented!\" )","title":"forward"},{"location":"development/devel/api/#num_batches_tracked-and-exponential-averaging-are-currently-not-implemented","text":"Arguments : input - torch.Tensor: input - torch.Tensor: input - torch.Tensor: input - torch.Tensor:","title":"num_batches_tracked and exponential averaging are currently not implemented."},{"location":"development/devel/api/#get_basic_batchnorm1d","text":"def get_basic_batchnorm1d()","title":"get_basic_batchnorm1d"},{"location":"development/devel/api/#assemble_basic_module_1","text":"def assemble_basic_module() -> nn.BatchNorm1d","title":"assemble_basic_module"},{"location":"development/devel/api/#hannahmodelsofasubmoduleselasticchannelhelper","text":"","title":"hannah.models.ofa.submodules.elasticchannelhelper"},{"location":"development/devel/api/#elasticchannelhelper-objects","text":"class ElasticChannelHelper(nn.Module)","title":"ElasticChannelHelper Objects"},{"location":"development/devel/api/#compute_channel_priorities","text":"def compute_channel_priorities()","title":"compute_channel_priorities"},{"location":"development/devel/api/#set_channel_filter","text":"def set_channel_filter()","title":"set_channel_filter"},{"location":"development/devel/api/#apply_filter_to_module","text":"def apply_filter_to_module(module, is_target: bool) Arguments : module: - is_target - bool: - is_target - bool: - is_target - bool: - is_target - bool:","title":"apply_filter_to_module"},{"location":"development/devel/api/#step_down_channels","text":"def step_down_channels()","title":"step_down_channels"},{"location":"development/devel/api/#set_channel_step","text":"def set_channel_step(step: int) Arguments : step - int: step - int: step - int: step - int:","title":"set_channel_step"},{"location":"development/devel/api/#reset_channel_step","text":"def reset_channel_step()","title":"reset_channel_step"},{"location":"development/devel/api/#set_primary_target","text":"def set_primary_target(target: nn.Module) Arguments : target - nn.Module: target - nn.Module: target - nn.Module: target - nn.Module:","title":"set_primary_target"},{"location":"development/devel/api/#is_valid_primary_target","text":"def is_valid_primary_target(module: nn.Module) -> bool Arguments : module - nn.Module: module - nn.Module: module - nn.Module: module - nn.Module:","title":"is_valid_primary_target"},{"location":"development/devel/api/#is_primary_target","text":"def is_primary_target(module: nn.Module) -> bool Arguments : module - nn.Module: module - nn.Module: module - nn.Module: module - nn.Module:","title":"is_primary_target"},{"location":"development/devel/api/#add_secondary_targets","text":"def add_secondary_targets(target: nn.Module) Arguments : target - nn.Module: target - nn.Module: target - nn.Module: target - nn.Module:","title":"add_secondary_targets"},{"location":"development/devel/api/#add_secondary_target_item","text":"def add_secondary_target_item(target: nn.Module) Arguments : target - nn.Module: target - nn.Module: target - nn.Module: target - nn.Module:","title":"add_secondary_target_item"},{"location":"development/devel/api/#add_sources","text":"def add_sources(source: nn.Module) Arguments : source - nn.Module: source - nn.Module: source - nn.Module: source - nn.Module:","title":"add_sources"},{"location":"development/devel/api/#add_targets","text":"def add_targets(target: nn.Module) Arguments : target - nn.Module: target - nn.Module: target - nn.Module: target - nn.Module:","title":"add_targets"},{"location":"development/devel/api/#add_source_item","text":"def add_source_item(source: nn.Module) Arguments : source - nn.Module: source - nn.Module: source - nn.Module: source - nn.Module:","title":"add_source_item"},{"location":"development/devel/api/#discover_target","text":"def discover_target(new_target: nn.Module) Arguments : new_target - nn.Module: new_target - nn.Module: new_target - nn.Module: new_target - nn.Module:","title":"discover_target"},{"location":"development/devel/api/#get_available_width_steps","text":"def get_available_width_steps()","title":"get_available_width_steps"},{"location":"development/devel/api/#hannahmodelsofasubmoduleselastickernelconv","text":"","title":"hannah.models.ofa.submodules.elastickernelconv"},{"location":"development/devel/api/#elasticconv1d-objects","text":"class ElasticConv1d(ElasticBase1d)","title":"ElasticConv1d Objects"},{"location":"development/devel/api/#forward_41","text":"def forward(input: torch.Tensor) -> torch.Tensor Arguments : input - torch.Tensor: input - torch.Tensor: input - torch.Tensor: input - torch.Tensor:","title":"forward"},{"location":"development/devel/api/#get_basic_module_3","text":"def get_basic_module() -> nn.Module","title":"get_basic_module"},{"location":"development/devel/api/#elasticconvrelu1d-objects","text":"class ElasticConvReLu1d(ElasticBase1d)","title":"ElasticConvReLu1d Objects"},{"location":"development/devel/api/#forward_42","text":"def forward(input: torch.Tensor) -> torch.Tensor Arguments : input - torch.Tensor: input - torch.Tensor: input - torch.Tensor: input - torch.Tensor:","title":"forward"},{"location":"development/devel/api/#get_basic_module_4","text":"def get_basic_module() -> nn.Module","title":"get_basic_module"},{"location":"development/devel/api/#elasticconvbn1d-objects","text":"class ElasticConvBn1d(ElasticConv1d)","title":"ElasticConvBn1d Objects"},{"location":"development/devel/api/#forward_43","text":"def forward(input: torch.Tensor) -> torch.Tensor Arguments : input - torch.Tensor: input - torch.Tensor: input - torch.Tensor: input - torch.Tensor:","title":"forward"},{"location":"development/devel/api/#get_basic_module_5","text":"def get_basic_module() -> nn.Module","title":"get_basic_module"},{"location":"development/devel/api/#elasticconvbnrelu1d-objects","text":"class ElasticConvBnReLu1d(ElasticConvBn1d)","title":"ElasticConvBnReLu1d Objects"},{"location":"development/devel/api/#forward_44","text":"def forward(input: torch.Tensor) -> torch.Tensor Arguments : input - torch.Tensor: input - torch.Tensor: input - torch.Tensor: input - torch.Tensor:","title":"forward"},{"location":"development/devel/api/#get_basic_module_6","text":"def get_basic_module() -> nn.Module","title":"get_basic_module"},{"location":"development/devel/api/#convrelu1d-objects","text":"class ConvRelu1d(nn.Conv1d)","title":"ConvRelu1d Objects"},{"location":"development/devel/api/#forward_45","text":"def forward(input: torch.Tensor) -> torch.Tensor Arguments : input - torch.Tensor: input - torch.Tensor: input - torch.Tensor: input - torch.Tensor:","title":"forward"},{"location":"development/devel/api/#convbn1d-objects","text":"class ConvBn1d(nn.Conv1d)","title":"ConvBn1d Objects"},{"location":"development/devel/api/#forward_46","text":"def forward(input: torch.Tensor) -> torch.Tensor Arguments : input - torch.Tensor: input - torch.Tensor: input - torch.Tensor: input - torch.Tensor:","title":"forward"},{"location":"development/devel/api/#convbnrelu1d-objects","text":"class ConvBnReLu1d(ConvBn1d)","title":"ConvBnReLu1d Objects"},{"location":"development/devel/api/#forward_47","text":"def forward(input: torch.Tensor) -> torch.Tensor Arguments : input - torch.Tensor: input - torch.Tensor: input - torch.Tensor: input - torch.Tensor:","title":"forward"},{"location":"development/devel/api/#hannahmodelsofasubmoduleselasticquantkernelconv","text":"","title":"hannah.models.ofa.submodules.elasticquantkernelconv"},{"location":"development/devel/api/#quaddatahelper-objects","text":"class QuadDataHelper() Data Container so that _forward and _dsc has the same data.","title":"QuadDataHelper Objects"},{"location":"development/devel/api/#_elasticconvbnnd-objects","text":"class _ElasticConvBnNd(ElasticBase1d, qat._ConvForwardMixin)","title":"_ElasticConvBnNd Objects"},{"location":"development/devel/api/#on_warmup_end","text":"def on_warmup_end()","title":"on_warmup_end"},{"location":"development/devel/api/#reset_running_stats","text":"def reset_running_stats()","title":"reset_running_stats"},{"location":"development/devel/api/#reset_bn_parameters","text":"def reset_bn_parameters()","title":"reset_bn_parameters"},{"location":"development/devel/api/#reset_parameters","text":"def reset_parameters()","title":"reset_parameters"},{"location":"development/devel/api/#update_bn_stats","text":"def update_bn_stats()","title":"update_bn_stats"},{"location":"development/devel/api/#freeze_bn_stats","text":"def freeze_bn_stats()","title":"freeze_bn_stats"},{"location":"development/devel/api/#scale_factor","text":"@property def scale_factor()","title":"scale_factor"},{"location":"development/devel/api/#full_scale_factor","text":"@property def full_scale_factor() does the same as scale_factor but uses the whole kernel. Used for dsc","title":"full_scale_factor"},{"location":"development/devel/api/#scaled_weight_1","text":"@property def scaled_weight()","title":"scaled_weight"},{"location":"development/devel/api/#get_full_kernel_bias","text":"def get_full_kernel_bias() Gets the full kernel and bias. Used for dsc","title":"get_full_kernel_bias"},{"location":"development/devel/api/#extra_repr_1","text":"def extra_repr()","title":"extra_repr"},{"location":"development/devel/api/#forward_48","text":"def forward(input) Arguments : input:","title":"forward"},{"location":"development/devel/api/#train_1","text":"def train(mode=True) Batchnorm's training behavior is using the self.training flag. Prevent changing it if BN is frozen. This makes sure that calling model.train() on a model with a frozen BN will behave properly. Arguments : mode - (Default value = True)","title":"train"},{"location":"development/devel/api/#from_float_1","text":"@classmethod def from_float(cls, mod) Create a qat module from a float module or qparams_dict Args: mod a float module, either produced by torch.quantization utilities or directly from user Arguments : mod:","title":"from_float"},{"location":"development/devel/api/#elasticquantconv1d-objects","text":"class ElasticQuantConv1d(ElasticBase1d, qat._ConvForwardMixin)","title":"ElasticQuantConv1d Objects"},{"location":"development/devel/api/#forward_49","text":"def forward(input: torch.Tensor) -> torch.Tensor Arguments : input - torch.Tensor: input - torch.Tensor: input - torch.Tensor: input - torch.Tensor:","title":"forward"},{"location":"development/devel/api/#get_basic_module_7","text":"def get_basic_module() -> nn.Module","title":"get_basic_module"},{"location":"development/devel/api/#elasticquantconvrelu1d-objects","text":"class ElasticQuantConvReLu1d(ElasticBase1d, qat._ConvForwardMixin)","title":"ElasticQuantConvReLu1d Objects"},{"location":"development/devel/api/#forward_50","text":"def forward(input: torch.Tensor) -> torch.Tensor Arguments : input - torch.Tensor: input - torch.Tensor: input - torch.Tensor: input - torch.Tensor:","title":"forward"},{"location":"development/devel/api/#get_basic_module_8","text":"def get_basic_module() -> nn.Module","title":"get_basic_module"},{"location":"development/devel/api/#elasticquantconvbn1d-objects","text":"class ElasticQuantConvBn1d(_ElasticConvBnNd)","title":"ElasticQuantConvBn1d Objects"},{"location":"development/devel/api/#forward_51","text":"def forward(input: torch.Tensor) -> torch.Tensor Arguments : input - torch.Tensor: input - torch.Tensor: input - torch.Tensor: input - torch.Tensor:","title":"forward"},{"location":"development/devel/api/#get_basic_module_9","text":"def get_basic_module() -> nn.Module","title":"get_basic_module"},{"location":"development/devel/api/#elasticquantconvbnrelu1d-objects","text":"class ElasticQuantConvBnReLu1d(ElasticQuantConvBn1d)","title":"ElasticQuantConvBnReLu1d Objects"},{"location":"development/devel/api/#forward_52","text":"def forward(input: torch.Tensor) -> torch.Tensor Arguments : input - torch.Tensor: input - torch.Tensor: input - torch.Tensor: input - torch.Tensor:","title":"forward"},{"location":"development/devel/api/#get_basic_module_10","text":"def get_basic_module() -> nn.Module","title":"get_basic_module"},{"location":"development/devel/api/#hannahmodelsofamodels","text":"","title":"hannah.models.ofa.models"},{"location":"development/devel/api/#create","text":"def create(name: str, labels: int, input_shape, conv=[], min_depth: int = 1, norm_before_act=True, skew_sampling_distribution: bool = False, dropout: int = 0.5, validate_on_extracted=True, qconfig=None) -> nn.Module The function creates a ofa Model with the given name, labels, input shape, convolutional layers, and other parameters Arguments : name(str) - The name of the model labels(int) - The number of classes in the dataset input_shape - the shape of the input tensor conv - a list of MajorBlockConfig objects (Default value = []) min_depth(int optional - The minimum depth of the model, defaults to 1 norm_before_act - If True, the normalization is performed before the activation function, defaults to True (optional) skew_sampling_distribution(bool optional - If True, the model will use a skewed sampling distribution to sample the number of minor blocks in each major block, defaults to False dropout(int) - float, default 0.5 validate_on_extracted - If True, the model will be validated on the extracted data, defaults to True (optional) qconfig - the quantization configuration to use (Default value = None) name - str: labels - int: min_depth - int: (Default value = 1) skew_sampling_distribution - bool: (Default value = False) dropout - int: (Default value = 0.5) name - str: labels - int: min_depth - int: (Default value = 1) skew_sampling_distribution - bool: (Default value = False) dropout - int: (Default value = 0.5) name - str: labels - int: min_depth - int: (Default value = 1) skew_sampling_distribution - bool: (Default value = False) dropout - int: (Default value = 0.5) Returns : A model object.","title":"create"},{"location":"development/devel/api/#create_minor_block_sequence","text":"def create_minor_block_sequence(blocks, in_channels, stride=1, norm_before_act=True, qconfig=None) Arguments : blocks: in_channels: - stride - (Default value = 1) - norm_before_act - (Default value = True) - qconfig - (Default value = None) # sources: List[nn.Module]: (Default value = [nn.ModuleList([])]) # sources: List[nn.Module]: (Default value = [nn.ModuleList([])]) # sources: List[nn.Module]: (Default value = [nn.ModuleList([])])","title":"create_minor_block_sequence"},{"location":"development/devel/api/#create_minor_block","text":"def create_minor_block(block_config, in_channels: int, stride: int = 1, norm_before_act=True, sources: List[nn.ModuleList] = [nn.ModuleList([])], qconfig=None) -> Tuple[nn.Module, int] Arguments : block_config: - in_channels - int: - stride - int: (Default value = 1) - norm_before_act - (Default value = True) - sources - List[nn.ModuleList]: (Default value = [nn.ModuleList([])]) - qconfig - (Default value = None) - in_channels - int: - stride - int: (Default value = 1) - sources - List[nn.ModuleList]: (Default value = [nn.ModuleList([])]) - in_channels - int: - stride - int: (Default value = 1) - sources - List[nn.ModuleList]: (Default value = [nn.ModuleList([])])","title":"create_minor_block"},{"location":"development/devel/api/#create_residual_block_1d","text":"def create_residual_block_1d(blocks, in_channels, stride=1, norm_before_act=None, qconfig=None, quant_skip=None) -> ResBlock1d Arguments : blocks: in_channels: - stride - (Default value = 1) - norm_before_act - (Default value = None) - qconfig - (Default value = None) - quant_skip - (Default value = None) # sources: List[nn.ModuleList]: (Default value = [nn.ModuleList([])]) # sources: List[nn.ModuleList]: (Default value = [nn.ModuleList([])]) # sources: List[nn.ModuleList]: (Default value = [nn.ModuleList([])])","title":"create_residual_block_1d"},{"location":"development/devel/api/#ofamodel-objects","text":"class OFAModel(nn.Module)","title":"OFAModel Objects"},{"location":"development/devel/api/#extract_conv","text":"def extract_conv(conv, general_config, parallel=True, bp=False) Arguments : conv: general_config: - parallel - (Default value = True) - bp - (Default value = False)","title":"extract_conv"},{"location":"development/devel/api/#extract_config","text":"def extract_config(conv, general_config, bp=False) set all conv attributes in config Arguments : conv: general_config: - bp - (Default value = False)","title":"extract_config"},{"location":"development/devel/api/#print_config","text":"def print_config(filename) Arguments : filename:","title":"print_config"},{"location":"development/devel/api/#forward_53","text":"def forward(x) Arguments : x:","title":"forward"},{"location":"development/devel/api/#perform_sequence_discovery","text":"def perform_sequence_discovery()","title":"perform_sequence_discovery"},{"location":"development/devel/api/#get_post_conv","text":"def get_post_conv(post_block) Arguments : post_block:","title":"get_post_conv"},{"location":"development/devel/api/#get_pre_conv","text":"def get_pre_conv(pre_block) Arguments : pre_block:","title":"get_pre_conv"},{"location":"development/devel/api/#sample_subnetwork","text":"def sample_subnetwork()","title":"sample_subnetwork"},{"location":"development/devel/api/#get_random_step","text":"def get_random_step(upper_bound: int) -> int Arguments : upper_bound - int: upper_bound - int: upper_bound - int:","title":"get_random_step"},{"location":"development/devel/api/#get_max_submodel_steps","text":"def get_max_submodel_steps()","title":"get_max_submodel_steps"},{"location":"development/devel/api/#get_submodel","text":"def get_submodel(state: dict) Arguments : state - dict: state - dict: state - dict:","title":"get_submodel"},{"location":"development/devel/api/#on_warmup_end_1","text":"def on_warmup_end()","title":"on_warmup_end"},{"location":"development/devel/api/#set_submodel","text":"def set_submodel(state: dict) Arguments : state - dict: state - dict: state - dict:","title":"set_submodel"},{"location":"development/devel/api/#build_validation_model","text":"def build_validation_model()","title":"build_validation_model"},{"location":"development/devel/api/#reset_validation_model","text":"def reset_validation_model()","title":"reset_validation_model"},{"location":"development/devel/api/#get_validation_model_weight_count","text":"def get_validation_model_weight_count()","title":"get_validation_model_weight_count"},{"location":"development/devel/api/#extract_elastic_depth_sequence","text":"def extract_elastic_depth_sequence(target_depth, quantized=False, clone_mode=False) Arguments : target_depth: - quantized - (Default value = False) - clone_mode - (Default value = False)","title":"extract_elastic_depth_sequence"},{"location":"development/devel/api/#extract_module_from_depth_step","text":"def extract_module_from_depth_step(depth_step) -> nn.Module Arguments : depth_step:","title":"extract_module_from_depth_step"},{"location":"development/devel/api/#get_elastic_depth_output","text":"def get_elastic_depth_output(target_depth=None, quantized=False) Arguments : target_depth - (Default value = None) quantized - (Default value = False)","title":"get_elastic_depth_output"},{"location":"development/devel/api/#step_down_all_channels","text":"def step_down_all_channels()","title":"step_down_all_channels"},{"location":"development/devel/api/#reset_active_depth","text":"def reset_active_depth()","title":"reset_active_depth"},{"location":"development/devel/api/#resume_active_elastic_values","text":"def resume_active_elastic_values()","title":"resume_active_elastic_values"},{"location":"development/devel/api/#update_output_channel_count","text":"def update_output_channel_count()","title":"update_output_channel_count"},{"location":"development/devel/api/#get_output_linear_layer","text":"def get_output_linear_layer(target_depth) Arguments : target_depth:","title":"get_output_linear_layer"},{"location":"development/devel/api/#step_down_all_kernels","text":"def step_down_all_kernels()","title":"step_down_all_kernels"},{"location":"development/devel/api/#reset_all_kernel_sizes","text":"def reset_all_kernel_sizes()","title":"reset_all_kernel_sizes"},{"location":"development/devel/api/#reset_all_dilation_sizes","text":"def reset_all_dilation_sizes()","title":"reset_all_dilation_sizes"},{"location":"development/devel/api/#reset_all_group_sizes","text":"def reset_all_group_sizes()","title":"reset_all_group_sizes"},{"location":"development/devel/api/#reset_all_dsc","text":"def reset_all_dsc()","title":"reset_all_dsc"},{"location":"development/devel/api/#step_down_all_dilations","text":"def step_down_all_dilations()","title":"step_down_all_dilations"},{"location":"development/devel/api/#step_down_all_groups","text":"def step_down_all_groups()","title":"step_down_all_groups"},{"location":"development/devel/api/#step_down_all_dsc","text":"def step_down_all_dsc()","title":"step_down_all_dsc"},{"location":"development/devel/api/#go_to_kernel_step","text":"def go_to_kernel_step(step: int) Arguments : step - int: step - int: step - int:","title":"go_to_kernel_step"},{"location":"development/devel/api/#resume_kernel_sizes_from_step","text":"def resume_kernel_sizes_from_step()","title":"resume_kernel_sizes_from_step"},{"location":"development/devel/api/#reset_all_widths","text":"def reset_all_widths()","title":"reset_all_widths"},{"location":"development/devel/api/#progressive_shrinking_add_kernel","text":"def progressive_shrinking_add_kernel()","title":"progressive_shrinking_add_kernel"},{"location":"development/devel/api/#progressive_shrinking_add_dilation","text":"def progressive_shrinking_add_dilation()","title":"progressive_shrinking_add_dilation"},{"location":"development/devel/api/#progressive_shrinking_add_depth","text":"def progressive_shrinking_add_depth()","title":"progressive_shrinking_add_depth"},{"location":"development/devel/api/#progressive_shrinking_add_group","text":"def progressive_shrinking_add_group()","title":"progressive_shrinking_add_group"},{"location":"development/devel/api/#progressive_shrinking_add_dsc","text":"def progressive_shrinking_add_dsc()","title":"progressive_shrinking_add_dsc"},{"location":"development/devel/api/#progressive_shrinking_compute_channel_priorities","text":"def progressive_shrinking_compute_channel_priorities()","title":"progressive_shrinking_compute_channel_priorities"},{"location":"development/devel/api/#progressive_shrinking_add_width","text":"def progressive_shrinking_add_width()","title":"progressive_shrinking_add_width"},{"location":"development/devel/api/#progressive_shrinking_disable_sampling","text":"def progressive_shrinking_disable_sampling()","title":"progressive_shrinking_disable_sampling"},{"location":"development/devel/api/#reset_shrinking","text":"def reset_shrinking()","title":"reset_shrinking"},{"location":"development/devel/api/#rebuild_extracted_blocks","text":"def rebuild_extracted_blocks(blocks) Arguments : blocks:","title":"rebuild_extracted_blocks"},{"location":"development/devel/api/#hannahmodelsofa","text":"","title":"hannah.models.ofa"},{"location":"development/devel/api/#hannahmodelsofatype_utils","text":"","title":"hannah.models.ofa.type_utils"},{"location":"development/devel/api/#hannahmodelsofautilities","text":"","title":"hannah.models.ofa.utilities"},{"location":"development/devel/api/#conv1d_auto_padding","text":"def conv1d_auto_padding(conv1d: nn.Conv1d) Arguments : conv1d - nn.Conv1d: conv1d - nn.Conv1d: conv1d - nn.Conv1d: conv1d - nn.Conv1d:","title":"conv1d_auto_padding"},{"location":"development/devel/api/#conv1d_get_padding","text":"def conv1d_get_padding(kernel_size, dilation=1) Arguments : kernel_size: - dilation - (Default value = 1)","title":"conv1d_get_padding"},{"location":"development/devel/api/#sub_filter_start_end","text":"def sub_filter_start_end(kernel_size, sub_kernel_size) Arguments : kernel_size: sub_kernel_size:","title":"sub_filter_start_end"},{"location":"development/devel/api/#flatten_module_list","text":"def flatten_module_list(modules: nn.Module) -> nn.Module Arguments : modules - nn.Module: modules - nn.Module: modules - nn.Module: modules - nn.Module:","title":"flatten_module_list"},{"location":"development/devel/api/#module_list_to_module","text":"def module_list_to_module(module_list) Arguments : module_list:","title":"module_list_to_module"},{"location":"development/devel/api/#call_function_from_deep_nested","text":"def call_function_from_deep_nested(input, function, type_selection: type = None) Arguments : input: function: - type_selection - type: (Default value = None) - type_selection - type: (Default value = None) - type_selection - type: (Default value = None) - type_selection - type: (Default value = None)","title":"call_function_from_deep_nested"},{"location":"development/devel/api/#get_instances_from_deep_nested","text":"def get_instances_from_deep_nested(input, type_selection: type = None) Arguments : input: - type_selection - type: (Default value = None) - type_selection - type: (Default value = None) - type_selection - type: (Default value = None) - type_selection - type: (Default value = None)","title":"get_instances_from_deep_nested"},{"location":"development/devel/api/#filter_primary_module_weights","text":"def filter_primary_module_weights(weights, in_channel_filter, out_channel_filter) Arguments : weights: in_channel_filter: out_channel_filter:","title":"filter_primary_module_weights"},{"location":"development/devel/api/#filter_single_dimensional_weights","text":"def filter_single_dimensional_weights(weights, channel_filter) Arguments : weights: channel_filter:","title":"filter_single_dimensional_weights"},{"location":"development/devel/api/#make_parameter","text":"def make_parameter(t: torch.Tensor) -> nn.Parameter Arguments : t - torch.Tensor: t - torch.Tensor: t - torch.Tensor: t - torch.Tensor:","title":"make_parameter"},{"location":"development/devel/api/#adjust_weight_if_needed","text":"def adjust_weight_if_needed(module, kernel=None, groups=None) Adjust the weight if the adjustment is needded. This means, if the kernel does not have the size of (out_channel, in_channel / group, kernel). Arguments : kernel - the kernel that should be checked and adjusted if needed. If None module.weight.data will be used (Default value = None) grouping - value of the conv, if None module.groups will be used module - the conv :throws - RuntimeError if there is no last_grouping_param for comporing current group value to past group value returns (kernel, is adjusted) (adjusted if needed) otherwise throws a RuntimeError groups - (Default value = None)","title":"adjust_weight_if_needed"},{"location":"development/devel/api/#is_weight_adjusting_needed","text":"def is_weight_adjusting_needed(weights, input_channels, groups) Checks if a weight adjustment is needed Requirement: weight.shape[1] must be input_channels/groups true: weight adjustment is needed Arguments : weights - the weights that needs to be checked input_channels - Input Channels of the Convolution Module groups - Grouping Param of the Convolution Module","title":"is_weight_adjusting_needed"},{"location":"development/devel/api/#get_target_weight","text":"def get_target_weight(weights, input_channels, groups) Gives the targeted weight shape (out_channel, in_channel // groups, kernel) Arguments : weights - the weights that needs to be checked input_channels - Input Channels of the Convolution Module groups - Grouping Param of the Convolution Module","title":"get_target_weight"},{"location":"development/devel/api/#prepare_kernel_for_depthwise_separable_convolution","text":"def prepare_kernel_for_depthwise_separable_convolution(model, kernel, bias, in_channels) Prepares the kernel for depthwise separable convolution (step 1 of DSC). This means setting groups = inchannels and outchannels = k * inchannels. Arguments : model: kernel: bias: in_channels: Returns : kernel, bias) Tuple","title":"prepare_kernel_for_depthwise_separable_convolution"},{"location":"development/devel/api/#prepare_kernel_for_pointwise_convolution","text":"def prepare_kernel_for_pointwise_convolution(kernel, grouping) Prepares the kernel for pointwise convolution (step 2 of DSC). This means setting the kernel window to 1x1. So a kernel with output_channel, input_channel / groups, kernel will be set to ( , ,1) Arguments : kernel: grouping:","title":"prepare_kernel_for_pointwise_convolution"},{"location":"development/devel/api/#adjust_weights_for_grouping_1","text":"def adjust_weights_for_grouping(weights, input_divided_by) Adjusts the Weights for the Forward of the Convulution Shape(outchannels, inchannels / group, kW) weight \u2013 filters of shape (out_channels , in_channels / groups , kW) input_divided_by Arguments : weights: input_divided_by:","title":"adjust_weights_for_grouping"},{"location":"development/devel/api/#get_kernel_for_dsc","text":"def get_kernel_for_dsc(kernel) Part of DSC (Step 2, pointwise convolution) kernel with output_channel, input_channel / groups, kernel will be set to ( , ,1) Arguments : kernel:","title":"get_kernel_for_dsc"},{"location":"development/devel/api/#get_channel_filter","text":"def get_channel_filter(current_channel_size, reduced_target_channel_size, channel_priority_list) Arguments : current_channel_size: reduced_target_channel_size: channel_priority_list:","title":"get_channel_filter"},{"location":"development/devel/api/#create_channel_filter","text":"def create_channel_filter(module: nn.Module, kernel, current_channel, reduced_target_channel_size, is_output_filter: bool = True) Arguments : module - nn.Module: kernel: current_channel: reduced_target_channel_size: is_output_filter - bool: (Default value = True) module - nn.Module: is_output_filter - bool: (Default value = True) module - nn.Module: is_output_filter - bool: (Default value = True) module - nn.Module: is_output_filter - bool: (Default value = True)","title":"create_channel_filter"},{"location":"development/devel/api/#compute_channel_priorities_1","text":"def compute_channel_priorities(module: nn.Module, kernel, channel_index: int = 0) Arguments : module - nn.Module: kernel: channel_index - int: (Default value = 0) module - nn.Module: channel_index - int: (Default value = 0) module - nn.Module: channel_index - int: (Default value = 0) module - nn.Module: channel_index - int: (Default value = 0)","title":"compute_channel_priorities"},{"location":"development/devel/api/#hannahmodelsekutmodels","text":"","title":"hannah.models.ekut.models"},{"location":"development/devel/api/#conv_bn","text":"def conv_bn(inp, oup, stride) Arguments : inp: oup: stride:","title":"conv_bn"},{"location":"development/devel/api/#conv_1x1_bn","text":"def conv_1x1_bn(inp, oup) Arguments : inp: oup:","title":"conv_1x1_bn"},{"location":"development/devel/api/#invertedresidual-objects","text":"class InvertedResidual(nn.Module)","title":"InvertedResidual Objects"},{"location":"development/devel/api/#forward_54","text":"def forward(x) Arguments : x:","title":"forward"},{"location":"development/devel/api/#rawspeechmodel-objects","text":"class RawSpeechModel(nn.Module) Speech Recognition on RAW Data using Wolfgang Fuhls Networks","title":"RawSpeechModel Objects"},{"location":"development/devel/api/#forward_55","text":"def forward(x) Arguments : x:","title":"forward"},{"location":"development/devel/api/#rawspeechmodelinvertedresidual-objects","text":"class RawSpeechModelInvertedResidual(nn.Module)","title":"RawSpeechModelInvertedResidual Objects"},{"location":"development/devel/api/#forward_56","text":"def forward(x) Arguments : x:","title":"forward"},{"location":"development/devel/api/#hannahmodelsekut","text":"","title":"hannah.models.ekut"},{"location":"development/devel/api/#hannahmodelsutils","text":"","title":"hannah.models.utils"},{"location":"development/devel/api/#next_power_of2","text":"def next_power_of2(x) Arguments : x:","title":"next_power_of2"},{"location":"development/devel/api/#hannahmodelswavenetmodels","text":"","title":"hannah.models.wavenet.models"},{"location":"development/devel/api/#conv-objects","text":"class Conv(nn.Module) A convolution with the option to be causal and use xavier initialization","title":"Conv Objects"},{"location":"development/devel/api/#forward_57","text":"def forward(signal) Arguments : signal:","title":"forward"},{"location":"development/devel/api/#wavenet-objects","text":"class WaveNet(nn.Module)","title":"WaveNet Objects"},{"location":"development/devel/api/#forward_58","text":"def forward(input_data) Arguments : input_data:","title":"forward"},{"location":"development/devel/api/#hannahmodelswavenet","text":"","title":"hannah.models.wavenet"},{"location":"development/devel/api/#hannahmodels_vendor","text":"","title":"hannah.models._vendor"},{"location":"development/devel/api/#hannahmodels_vendorfocalnet","text":"","title":"hannah.models._vendor.focalnet"},{"location":"development/devel/api/#mlp-objects","text":"class Mlp(nn.Module)","title":"Mlp Objects"},{"location":"development/devel/api/#forward_59","text":"def forward(x) Arguments : x:","title":"forward"},{"location":"development/devel/api/#focalmodulation-objects","text":"class FocalModulation(nn.Module)","title":"FocalModulation Objects"},{"location":"development/devel/api/#forward_60","text":"def forward(x) Arguments : x - input features with shape of (B, H, W, C)","title":"forward"},{"location":"development/devel/api/#extra_repr_2","text":"def extra_repr() -> str","title":"extra_repr"},{"location":"development/devel/api/#flops","text":"def flops(N) calculate flops for 1 window with token length of N Arguments : N:","title":"flops"},{"location":"development/devel/api/#focalnetblock-objects","text":"class FocalNetBlock(nn.Module) Focal Modulation Network Block. Arguments : dim(int) - Number of input channels. input_resolution(tuple[int]) - Input resulotion. mlp_ratio(float) - Ratio of mlp hidden dim to embedding dim. drop(float) - Dropout rate. Default: 0.0 drop_path(float) - Stochastic depth rate. Default: 0.0 act_layer(nn.Module) - Activation layer. Default: nn.GELU norm_layer(nn.Module) - Normalization layer. Default: nn.LayerNorm focal_level(int) - Number of focal levels. focal_window(int) - Focal window size at first focal level use_layerscale(bool) - Whether use layerscale layerscale_value(float) - Initial layerscale value use_postln(bool) - Whether use layernorm after modulation","title":"FocalNetBlock Objects"},{"location":"development/devel/api/#forward_61","text":"def forward(x) Arguments : x:","title":"forward"},{"location":"development/devel/api/#extra_repr_3","text":"def extra_repr() -> str","title":"extra_repr"},{"location":"development/devel/api/#flops_1","text":"def flops()","title":"flops"},{"location":"development/devel/api/#basiclayer-objects","text":"class BasicLayer(nn.Module) A basic Focal Transformer layer for one stage. Arguments : dim(int) - Number of input channels. input_resolution(tuple[int]) - Input resolution. depth(int) - Number of blocks. window_size(int) - Local window size. mlp_ratio(float) - Ratio of mlp hidden dim to embedding dim. qkv_bias(bool) - If True, add a learnable bias to query, key, value. Default: True qk_scale(float | None): Override default qk scale of head_dim ** -0.5 if set. drop(float) - Dropout rate. Default: 0.0 drop_path(float | tuple[float]): Stochastic depth rate. Default: 0.0 norm_layer(nn.Module) - Normalization layer. Default: nn.LayerNorm downsample(nn.Module | None): Downsample layer at the end of the layer. Default: None use_checkpoint(bool) - Whether to use checkpointing to save memory. Default: False. focal_level(int) - Number of focal levels focal_window(int) - Focal window size at first focal level use_layerscale(bool) - Whether use layerscale layerscale_value(float) - Initial layerscale value use_postln(bool) - Whether use layernorm after modulation","title":"BasicLayer Objects"},{"location":"development/devel/api/#forward_62","text":"def forward(x, H, W) Arguments : x: H: W:","title":"forward"},{"location":"development/devel/api/#extra_repr_4","text":"def extra_repr() -> str","title":"extra_repr"},{"location":"development/devel/api/#flops_2","text":"def flops()","title":"flops"},{"location":"development/devel/api/#patchembed-objects","text":"class PatchEmbed(nn.Module) Image to Patch Embedding Arguments : img_size(int) - Image size. Default: 224. patch_size(int) - Patch token size. Default: 4. in_chans(int) - Number of input image channels. Default: 3. embed_dim(int) - Number of linear projection output channels. Default: 96. norm_layer(nn.Module) - Normalization layer. Default: None","title":"PatchEmbed Objects"},{"location":"development/devel/api/#forward_63","text":"def forward(x) Arguments : x:","title":"forward"},{"location":"development/devel/api/#flops_3","text":"def flops()","title":"flops"},{"location":"development/devel/api/#focalnet-objects","text":"class FocalNet(nn.Module) Focal Modulation Networks (FocalNets) Arguments : img_size(int | tuple(int): Input image size. Default 224 patch_size(int | tuple(int): Patch size. Default: 4 - in_chans(int) - Number of input image channels. Default: 3 - num_classes(int) - Number of classes for classification head. Default: 1000 - embed_dim(int) - Patch embedding dimension. Default: 96 - depths(tuple(int) - Depth of each Focal Transformer layer. - mlp_ratio(float) - Ratio of mlp hidden dim to embedding dim. Default: 4 - drop_rate(float) - Dropout rate. Default: 0 - drop_path_rate(float) - Stochastic depth rate. Default: 0.1 - norm_layer(nn.Module) - Normalization layer. Default: nn.LayerNorm. - patch_norm(bool) - If True, add normalization after patch embedding. Default: True - use_checkpoint(bool) - Whether to use checkpointing to save memory. Default: False - focal_levels(list) - How many focal levels at all stages. Note that this excludes the finest-grain level. Default: [1, 1, 1, 1] - focal_windows(list) - The focal window size at all stages. Default: [7, 5, 3, 1] - use_conv_embed(bool) - Whether use convolutional embedding. We noted that using convolutional embedding usually improve the performance, but we do not use it by default. Default: False - use_layerscale(bool) - Whether use layerscale proposed in CaiT. Default: False - layerscale_value(float) - Value for layer scale. Default: 1e-4 - use_postln(bool) - Whether use layernorm after modulation (it helps stablize training of large models)","title":"FocalNet Objects"},{"location":"development/devel/api/#no_weight_decay","text":"@torch.jit.ignore def no_weight_decay()","title":"no_weight_decay"},{"location":"development/devel/api/#no_weight_decay_keywords","text":"@torch.jit.ignore def no_weight_decay_keywords()","title":"no_weight_decay_keywords"},{"location":"development/devel/api/#forward_features","text":"def forward_features(x) Arguments : x:","title":"forward_features"},{"location":"development/devel/api/#forward_64","text":"def forward(x) Arguments : x:","title":"forward"},{"location":"development/devel/api/#flops_4","text":"def flops()","title":"flops"},{"location":"development/devel/api/#build_transforms","text":"def build_transforms(img_size, center_crop=False) Arguments : img_size: - center_crop - (Default value = False)","title":"build_transforms"},{"location":"development/devel/api/#build_transforms4display","text":"def build_transforms4display(img_size, center_crop=False) Arguments : img_size: - center_crop - (Default value = False)","title":"build_transforms4display"},{"location":"development/devel/api/#focalnet_tiny_srf","text":"@register_model def focalnet_tiny_srf(pretrained=False, **kwargs) Arguments : pretrained - (Default value = False) **kwargs:","title":"focalnet_tiny_srf"},{"location":"development/devel/api/#focalnet_small_srf","text":"@register_model def focalnet_small_srf(pretrained=False, **kwargs) Arguments : pretrained - (Default value = False) **kwargs:","title":"focalnet_small_srf"},{"location":"development/devel/api/#focalnet_base_srf","text":"@register_model def focalnet_base_srf(pretrained=False, **kwargs) Arguments : pretrained - (Default value = False) **kwargs:","title":"focalnet_base_srf"},{"location":"development/devel/api/#focalnet_tiny_lrf","text":"@register_model def focalnet_tiny_lrf(pretrained=False, **kwargs) Arguments : pretrained - (Default value = False) **kwargs:","title":"focalnet_tiny_lrf"},{"location":"development/devel/api/#focalnet_small_lrf","text":"@register_model def focalnet_small_lrf(pretrained=False, **kwargs) Arguments : pretrained - (Default value = False) **kwargs:","title":"focalnet_small_lrf"},{"location":"development/devel/api/#focalnet_base_lrf","text":"@register_model def focalnet_base_lrf(pretrained=False, **kwargs) Arguments : pretrained - (Default value = False) **kwargs:","title":"focalnet_base_lrf"},{"location":"development/devel/api/#focalnet_tiny_iso_16","text":"@register_model def focalnet_tiny_iso_16(pretrained=False, **kwargs) Arguments : pretrained - (Default value = False) **kwargs:","title":"focalnet_tiny_iso_16"},{"location":"development/devel/api/#focalnet_small_iso_16","text":"@register_model def focalnet_small_iso_16(pretrained=False, **kwargs) Arguments : pretrained - (Default value = False) **kwargs:","title":"focalnet_small_iso_16"},{"location":"development/devel/api/#focalnet_base_iso_16","text":"@register_model def focalnet_base_iso_16(pretrained=False, **kwargs) Arguments : pretrained - (Default value = False) **kwargs:","title":"focalnet_base_iso_16"},{"location":"development/devel/api/#focalnet_large_fl3","text":"@register_model def focalnet_large_fl3(pretrained=False, **kwargs) Arguments : pretrained - (Default value = False) **kwargs:","title":"focalnet_large_fl3"},{"location":"development/devel/api/#focalnet_large_fl4","text":"@register_model def focalnet_large_fl4(pretrained=False, **kwargs) Arguments : pretrained - (Default value = False) **kwargs:","title":"focalnet_large_fl4"},{"location":"development/devel/api/#focalnet_xlarge_fl3","text":"@register_model def focalnet_xlarge_fl3(pretrained=False, **kwargs) Arguments : pretrained - (Default value = False) **kwargs:","title":"focalnet_xlarge_fl3"},{"location":"development/devel/api/#focalnet_xlarge_fl4","text":"@register_model def focalnet_xlarge_fl4(pretrained=False, **kwargs) Arguments : pretrained - (Default value = False) **kwargs:","title":"focalnet_xlarge_fl4"},{"location":"development/devel/api/#focalnet_huge_fl3","text":"@register_model def focalnet_huge_fl3(pretrained=False, **kwargs) Arguments : pretrained - (Default value = False) **kwargs:","title":"focalnet_huge_fl3"},{"location":"development/devel/api/#focalnet_huge_fl4","text":"@register_model def focalnet_huge_fl4(pretrained=False, **kwargs) Arguments : pretrained - (Default value = False) **kwargs:","title":"focalnet_huge_fl4"},{"location":"development/devel/api/#hannahmodelssincmodels","text":"","title":"hannah.models.sinc.models"},{"location":"development/devel/api/#gdsconv-objects","text":"class GDSConv(nn.Module)","title":"GDSConv Objects"},{"location":"development/devel/api/#forward_65","text":"def forward(x) Arguments : x:","title":"forward"},{"location":"development/devel/api/#gdsconvblock-objects","text":"class GDSConvBlock(nn.Module)","title":"GDSConvBlock Objects"},{"location":"development/devel/api/#forward_66","text":"def forward(x) Arguments : x:","title":"forward"},{"location":"development/devel/api/#sincnet-objects","text":"class SincNet(nn.Module)","title":"SincNet Objects"},{"location":"development/devel/api/#forward_67","text":"def forward(x) Arguments : x:","title":"forward"},{"location":"development/devel/api/#hannahmodelssinc","text":"","title":"hannah.models.sinc"},{"location":"development/devel/api/#hannahmodelscapsule_netmodels","text":"","title":"hannah.models.capsule_net.models"},{"location":"development/devel/api/#hannahmodelscapsule_net","text":"","title":"hannah.models.capsule_net"},{"location":"development/devel/api/#hannahmodelscapsule_netoperators","text":"","title":"hannah.models.capsule_net.operators"},{"location":"development/devel/api/#hannahmodelscapsule_netexpressions","text":"","title":"hannah.models.capsule_net.expressions"},{"location":"development/devel/api/#padding_expression_2","text":"def padding_expression(kernel_size, stride, dilation=1) Symbolically calculate padding such that for a given kernel_size, stride and dilation the padding is such that the output dimension is kept the same(stride=1) or halved(stride=2). Note: If the input dimension is 1 and stride = 2, the calculated padding will result in an output with also dimension 1.","title":"padding_expression"},{"location":"development/devel/api/#parameters_2","text":"kernel_size : Union[int, Expression] stride : Union[int, Expression] dilation : Union[int, Expression], optional description , by default 1","title":"Parameters"},{"location":"development/devel/api/#returns_2","text":"Expression","title":"Returns"},{"location":"development/devel/api/#hannahmodelscapsule_netutils","text":"","title":"hannah.models.capsule_net.utils"},{"location":"development/devel/api/#hannahtest_linear_classifier","text":"","title":"hannah.test_linear_classifier"},{"location":"development/devel/api/#hannahcallbacksdump_layers","text":"","title":"hannah.callbacks.dump_layers"},{"location":"development/devel/api/#testdumpercallback-objects","text":"class TestDumperCallback(Callback)","title":"TestDumperCallback Objects"},{"location":"development/devel/api/#on_test_start","text":"def on_test_start(pl_trainer, pl_model) Arguments : pl_trainer: pl_model:","title":"on_test_start"},{"location":"development/devel/api/#hannahcallbacksbackends","text":"","title":"hannah.callbacks.backends"},{"location":"development/devel/api/#symbolic_batch_dim","text":"def symbolic_batch_dim(model) Arguments : model:","title":"symbolic_batch_dim"},{"location":"development/devel/api/#inferencebackendbase-objects","text":"class InferenceBackendBase(Callback) Base class to run val and test on a backend inference engine","title":"InferenceBackendBase Objects"},{"location":"development/devel/api/#run_batch","text":"def run_batch(inputs=None) Arguments : inputs - (Default value = None)","title":"run_batch"},{"location":"development/devel/api/#prepare","text":"def prepare(module) Arguments : module:","title":"prepare"},{"location":"development/devel/api/#on_validation_epoch_start","text":"def on_validation_epoch_start(trainer, pl_module) Arguments : trainer: pl_module:","title":"on_validation_epoch_start"},{"location":"development/devel/api/#on_validation_batch_end","text":"def on_validation_batch_end(trainer, pl_module, outputs, batch, batch_idx, dataloader_idx) Arguments : trainer: pl_module: outputs: batch: batch_idx: dataloader_idx:","title":"on_validation_batch_end"},{"location":"development/devel/api/#on_validation_epoch_end","text":"def on_validation_epoch_end(trainer, pl_module) Arguments : trainer: pl_module:","title":"on_validation_epoch_end"},{"location":"development/devel/api/#on_test_epoch_start","text":"def on_test_epoch_start(trainer, pl_module) Arguments : trainer: pl_module:","title":"on_test_epoch_start"},{"location":"development/devel/api/#quantize","text":"def quantize(pl_module: torch.nn.Module) -> torch.nn.Module Arguments : pl_module - torch.nn.Module to quantize Returns - quantized torch.nn.Module","title":"quantize"},{"location":"development/devel/api/#on_test_batch_end","text":"def on_test_batch_end(trainer, pl_module, outputs, batch, batch_idx, dataloader_idx) Arguments : trainer: pl_module: outputs: batch: batch_idx: dataloader_idx:","title":"on_test_batch_end"},{"location":"development/devel/api/#export","text":"def export() -> None Export the model through the target backend","title":"export"},{"location":"development/devel/api/#torchmobilebackend-objects","text":"class TorchMobileBackend(InferenceBackendBase) Inference backend for torch mobile","title":"TorchMobileBackend Objects"},{"location":"development/devel/api/#prepare_1","text":"def prepare(model) Arguments : model torch.nn.Module - nn.Module to be exported Returns (None)","title":"prepare"},{"location":"development/devel/api/#run_batch_1","text":"def run_batch(inputs=None) Arguments : inputs - (Default value = None)","title":"run_batch"},{"location":"development/devel/api/#onnxtfbackend-objects","text":"class OnnxTFBackend(InferenceBackendBase) Inference Backend for tensorflow","title":"OnnxTFBackend Objects"},{"location":"development/devel/api/#prepare_2","text":"def prepare(model) Arguments : model:","title":"prepare"},{"location":"development/devel/api/#run_batch_2","text":"def run_batch(inputs) Arguments : inputs:","title":"run_batch"},{"location":"development/devel/api/#onnxruntimebackend-objects","text":"class OnnxruntimeBackend(InferenceBackendBase) Inference Backend for tensorflow","title":"OnnxruntimeBackend Objects"},{"location":"development/devel/api/#prepare_3","text":"def prepare(model) Arguments : model:","title":"prepare"},{"location":"development/devel/api/#run_batch_3","text":"def run_batch(inputs=None) Arguments : inputs - (Default value = None) Returns","title":"run_batch"},{"location":"development/devel/api/#hannahcallbackspruning","text":"","title":"hannah.callbacks.pruning"},{"location":"development/devel/api/#pruningamountscheduler-objects","text":"class PruningAmountScheduler()","title":"PruningAmountScheduler Objects"},{"location":"development/devel/api/#filteredpruning-objects","text":"class FilteredPruning(ModelPruning)","title":"FilteredPruning Objects"},{"location":"development/devel/api/#setup","text":"def setup(trainer: Trainer, pl_module: LightningModule, stage: str) Arguments : trainer: pl_module:","title":"setup"},{"location":"development/devel/api/#filter_parameters_to_prune","text":"def filter_parameters_to_prune(parameters_to_prune=None) Filter out unprunable parameters Arguments : parameters_to_prune - (Default value = None)","title":"filter_parameters_to_prune"},{"location":"development/devel/api/#on_test_end_1","text":"def on_test_end(trainer, pl_module) -> None Arguments : trainer: pl_module:","title":"on_test_end"},{"location":"development/devel/api/#hannahcallbacksoptimization","text":"","title":"hannah.callbacks.optimization"},{"location":"development/devel/api/#hydraoptcallback-objects","text":"class HydraOptCallback(Callback)","title":"HydraOptCallback Objects"},{"location":"development/devel/api/#on_test_end_2","text":"def on_test_end(trainer, pl_module) Arguments : trainer: pl_module:","title":"on_test_end"},{"location":"development/devel/api/#on_validation_end","text":"def on_validation_end(trainer, pl_module) Arguments : trainer: pl_module:","title":"on_validation_end"},{"location":"development/devel/api/#test_result","text":"def test_result()","title":"test_result"},{"location":"development/devel/api/#val_result","text":"def val_result()","title":"val_result"},{"location":"development/devel/api/#result","text":"def result(dict=False) Arguments : dict - (Default value = False)","title":"result"},{"location":"development/devel/api/#curves","text":"def curves(dict=False) Arguments : dict - (Default value = False)","title":"curves"},{"location":"development/devel/api/#hannahcallbacksfine_tuning","text":"","title":"hannah.callbacks.fine_tuning"},{"location":"development/devel/api/#hannahcallbacks","text":"","title":"hannah.callbacks"},{"location":"development/devel/api/#hannahcallbacksclustering","text":"","title":"hannah.callbacks.clustering"},{"location":"development/devel/api/#clustering","text":"def clustering(params, inertia, cluster) Arguments : params: inertia: cluster:","title":"clustering"},{"location":"development/devel/api/#kmeans-objects","text":"class kMeans(Callback)","title":"kMeans Objects"},{"location":"development/devel/api/#on_fit_end","text":"def on_fit_end(trainer, pl_module) Arguments : trainer: pl_module:","title":"on_fit_end"},{"location":"development/devel/api/#on_epoch_end","text":"def on_epoch_end(trainer, pl_module) Arguments : trainer: pl_module:","title":"on_epoch_end"},{"location":"development/devel/api/#hannahcallbackssummaries","text":"","title":"hannah.callbacks.summaries"},{"location":"development/devel/api/#walk_model","text":"def walk_model(model, dummy_input) Adapted from IntelLabs Distiller Arguments : model: dummy_input:","title":"walk_model"},{"location":"development/devel/api/#macsummarycallback-objects","text":"class MacSummaryCallback(Callback)","title":"MacSummaryCallback Objects"},{"location":"development/devel/api/#predict","text":"def predict(pl_module, input=input) Arguments : pl_module:","title":"predict"},{"location":"development/devel/api/#on_train_start","text":"@rank_zero_only def on_train_start(trainer, pl_module) Arguments : trainer: pl_module:","title":"on_train_start"},{"location":"development/devel/api/#on_test_end_3","text":"@rank_zero_only def on_test_end(trainer, pl_module) Arguments : trainer: pl_module:","title":"on_test_end"},{"location":"development/devel/api/#on_validation_epoch_end_1","text":"@rank_zero_only def on_validation_epoch_end(trainer, pl_module) Arguments : trainer: pl_module:","title":"on_validation_epoch_end"},{"location":"development/devel/api/#estimate","text":"def estimate(pl_module, input=None) Generate Summary Metrics for neural network Arguments : pl_module(pytorch_lightning.LightningModule) - pytorch lightning module to summarize Returns : dict[str, float]: Dict of MetricName => Metric Value","title":"estimate"},{"location":"development/devel/api/#hannahcallbackssvd_compress","text":"","title":"hannah.callbacks.svd_compress"},{"location":"development/devel/api/#svd-objects","text":"class SVD(Callback)","title":"SVD Objects"},{"location":"development/devel/api/#on_epoch_start","text":"def on_epoch_start(trainer, pl_module) Arguments : trainer: pl_module:","title":"on_epoch_start"},{"location":"development/devel/api/#hannahsslhard_labeling","text":"","title":"hannah.ssl.hard_labeling"},{"location":"development/devel/api/#hardlabeling-objects","text":"class HardLabeling(torch.nn.Module)","title":"HardLabeling Objects"},{"location":"development/devel/api/#forward_68","text":"def forward(unlabeled_data: torch.Tensor) -> torch.Tensor Calculate pseudo label loss from unlabeled data.","title":"forward"},{"location":"development/devel/api/#compute_uncertainty","text":"def compute_uncertainty(data: torch.Tensor, num_forward_passes: int = 10) -> torch.Tensor Compute Monte Carlo uncertainty using standard deviation.","title":"compute_uncertainty"},{"location":"development/devel/api/#get_dropout_layers","text":"def get_dropout_layers() Returns all model layers of class dropout, dropblock.","title":"get_dropout_layers"},{"location":"development/devel/api/#hannahssl","text":"","title":"hannah.ssl"},{"location":"development/devel/api/#hannahconfoptimizer","text":"","title":"hannah.conf.optimizer"},{"location":"development/devel/api/#sgdconf-objects","text":"@dataclass class SGDConf()","title":"SGDConf Objects"},{"location":"development/devel/api/#lr","text":"_RequiredParameter","title":"lr"},{"location":"development/devel/api/#madgradconf-objects","text":"@dataclass class MADGRADConf()","title":"MADGRADConf Objects"},{"location":"development/devel/api/#lr_1","text":"_RequiredParameter","title":"lr"},{"location":"development/devel/api/#hannahconf","text":"","title":"hannah.conf"},{"location":"development/devel/api/#hannahconfnas","text":"","title":"hannah.conf.nas"},{"location":"development/devel/api/#hannahconfscheduler","text":"","title":"hannah.conf.scheduler"},{"location":"development/devel/api/#onecyclelrconf-objects","text":"@dataclass class OneCycleLRConf() Config for one cycle lr total steps are configured from module","title":"OneCycleLRConf Objects"},{"location":"development/devel/api/#hannahoptim","text":"","title":"hannah.optim"},{"location":"development/devel/api/#hannahoptimmadgrad","text":"","title":"hannah.optim.madgrad"},{"location":"development/devel/api/#madgrad-objects","text":"class MADGRAD(torch.optim.Optimizer) MADGRAD_: A Momentumized, Adaptive, Dual Averaged Gradient Method for Stochastic Optimization. .. _MADGRAD: https://arxiv.org/abs/2101.11075 MADGRAD is a general purpose optimizer that can be used in place of SGD or Adam may converge faster and generalize better. Currently GPU-only. Typically, the same learning rate schedule that is used for SGD or Adam may be used. The overall learning rate is not comparable to either method and should be determined by a hyper-parameter sweep. MADGRAD requires less weight decay than other methods, often as little as zero. Momentum values used for SGD or Adam's beta1 should work here also. On sparse problems both weight_decay and momentum should be set to 0. Arguments : params (iterable): Iterable of parameters to optimize or dicts defining parameter groups. lr (float): Learning rate (default: 1e-2). momentum (float): Momentum value in the range [0,1) (default: 0.9). weight_decay (float): Weight decay, i.e. a L2 penalty (default: 0). eps (float): Term added to the denominator outside of the root operation to improve numerical stability. (default: 1e-6).","title":"MADGRAD Objects"},{"location":"development/devel/api/#step","text":"def step(closure: Optional[Callable[[], float]] = None) -> Optional[float] Performs a single optimization step. Arguments : closure callable, optional - A closure that reevaluates the model and returns the loss.","title":"step"},{"location":"development/devel/api/#hannahoptimradam","text":"","title":"hannah.optim.RAdam"},{"location":"development/devel/api/#hannahnasdataflowregistry","text":"","title":"hannah.nas.dataflow.registry"},{"location":"development/devel/api/#hannahnasdataflowdataflow_utils","text":"","title":"hannah.nas.dataflow.dataflow_utils"},{"location":"development/devel/api/#hannahnasdataflowtensor_expression","text":"","title":"hannah.nas.dataflow.tensor_expression"},{"location":"development/devel/api/#hannahnasdataflowquantization_type","text":"","title":"hannah.nas.dataflow.quantization_type"},{"location":"development/devel/api/#hannahnasdataflowregister_ops","text":"","title":"hannah.nas.dataflow.register_ops"},{"location":"development/devel/api/#hannahnasdataflowopsdropout","text":"","title":"hannah.nas.dataflow.ops.dropout"},{"location":"development/devel/api/#hannahnasdataflowopsconv2d","text":"","title":"hannah.nas.dataflow.ops.conv2d"},{"location":"development/devel/api/#hannahnasdataflowopslinear","text":"","title":"hannah.nas.dataflow.ops.linear"},{"location":"development/devel/api/#hannahnasdataflowopsidentity","text":"","title":"hannah.nas.dataflow.ops.identity"},{"location":"development/devel/api/#hannahnasdataflowopsrelu","text":"","title":"hannah.nas.dataflow.ops.relu"},{"location":"development/devel/api/#hannahnasdataflowopssum","text":"","title":"hannah.nas.dataflow.ops.sum"},{"location":"development/devel/api/#hannahnasdataflowopsbatch_nom","text":"","title":"hannah.nas.dataflow.ops.batch_nom"},{"location":"development/devel/api/#hannahnasdataflowopsadd","text":"","title":"hannah.nas.dataflow.ops.add"},{"location":"development/devel/api/#hannahnasdataflowops","text":"","title":"hannah.nas.dataflow.ops"},{"location":"development/devel/api/#hannahnasdataflowopsconcat","text":"","title":"hannah.nas.dataflow.ops.concat"},{"location":"development/devel/api/#hannahnasdataflowopspooling","text":"","title":"hannah.nas.dataflow.ops.pooling"},{"location":"development/devel/api/#hannahnasdataflowanalysisdataflow_analysis","text":"","title":"hannah.nas.dataflow.analysis.dataflow_analysis"},{"location":"development/devel/api/#hannahnasdataflowtensor","text":"","title":"hannah.nas.dataflow.tensor"},{"location":"development/devel/api/#hannahnasdataflowtensor_type","text":"","title":"hannah.nas.dataflow.tensor_type"},{"location":"development/devel/api/#hannahnasdataflowtransformationsgraph_tranformer","text":"","title":"hannah.nas.dataflow.transformations.graph_tranformer"},{"location":"development/devel/api/#hannahnasdataflowscoping_utils","text":"","title":"hannah.nas.dataflow.scoping_utils"},{"location":"development/devel/api/#hannahnasdataflow","text":"","title":"hannah.nas.dataflow"},{"location":"development/devel/api/#hannahnasdataflowrepeat","text":"","title":"hannah.nas.dataflow.repeat"},{"location":"development/devel/api/#hannahnasdataflowaxis_type","text":"","title":"hannah.nas.dataflow.axis_type"},{"location":"development/devel/api/#axistuple-objects","text":"@parametrize class AxisTuple() Used to have the axis dict as a parametrized object","title":"AxisTuple Objects"},{"location":"development/devel/api/#hannahnasdataflowdata_type","text":"","title":"hannah.nas.dataflow.data_type"},{"location":"development/devel/api/#hannahnasdataflowoptional_op","text":"","title":"hannah.nas.dataflow.optional_op"},{"location":"development/devel/api/#hannahnasdataflowcompression_type","text":"","title":"hannah.nas.dataflow.compression_type"},{"location":"development/devel/api/#hannahnasdataflowop_type","text":"","title":"hannah.nas.dataflow.op_type"},{"location":"development/devel/api/#hannahnasdataflowdataflow_graph","text":"","title":"hannah.nas.dataflow.dataflow_graph"},{"location":"development/devel/api/#dataflowgraph-objects","text":"@parametrize class DataFlowGraph(TensorExpression)","title":"DataFlowGraph Objects"},{"location":"development/devel/api/#link_users","text":"def link_users() Link the DFG to its users and the users of the DFG to the DFG","title":"link_users"},{"location":"development/devel/api/#collect_users","text":"def collect_users(node) Traverse graph starting from node and collect all users (including users from subsequent nodes). If a node_b is NOT in collect_users(node_a), this means that node_b is either BEFORE node_a in the graph OR it is in a parallel branch.","title":"collect_users"},{"location":"development/devel/api/#parameters_3","text":"node : type description","title":"Parameters"},{"location":"development/devel/api/#returns_3","text":"type description","title":"Returns"},{"location":"development/devel/api/#find_first_input","text":"def find_first_input(node) Recusively traverses the graph from the given node back to its first input. NOTE: The traversal is via OPERANDS and not OUTPUT, meaning that e.g. weight Tensors that are included in Ops in a DFG are not returned","title":"find_first_input"},{"location":"development/devel/api/#parameters_4","text":"node : type description","title":"Parameters"},{"location":"development/devel/api/#returns_4","text":"type description","title":"Returns"},{"location":"development/devel/api/#hannahnasperformance_predictionsimple","text":"","title":"hannah.nas.performance_prediction.simple"},{"location":"development/devel/api/#backendpredictor-objects","text":"class BackendPredictor() A predictor class that instantiates the model and uses the backends predict function to predict performance metrics","title":"BackendPredictor Objects"},{"location":"development/devel/api/#macpredictor-objects","text":"class MACPredictor() A predictor class that instantiates the model and calculates abstract metrics","title":"MACPredictor Objects"},{"location":"development/devel/api/#gcnpredictor-objects","text":"class GCNPredictor() A predictor class that instantiates the model and uses the backends predict function to predict performance metrics","title":"GCNPredictor Objects"},{"location":"development/devel/api/#hannahnasperformance_predictiongcnpredictor","text":"","title":"hannah.nas.performance_prediction.gcn.predictor"},{"location":"development/devel/api/#predictor-objects","text":"class Predictor()","title":"Predictor Objects"},{"location":"development/devel/api/#__init___1","text":"def __init__(fea_name=\"features\") -> None Parent method for different predictor classes.","title":"__init__"},{"location":"development/devel/api/#parameters_5","text":"fea_name : str, optional internal name for features in the graph, as in graph.ndata[fea_name], by default 'features'","title":"Parameters"},{"location":"development/devel/api/#train_2","text":"def train(dataloader, learning_rate=1e-3, num_epochs=200, validation_dataloader=None, verbose=1) Train GCN model","title":"train"},{"location":"development/devel/api/#parameters_6","text":"dataloader : GraphDataLoader training data learning_rate : [type], optional by default 1e-3 num_epochs : int, optional by default 200 validation_dataloader : [type], optional if given, use this data to print validation loss, by default None verbose : int if validation_dataloader is given, print validation MSE every epoch, by default 1","title":"Parameters"},{"location":"development/devel/api/#predict_1","text":"def predict(graph) predict cost of graph","title":"predict"},{"location":"development/devel/api/#parameters_7","text":"graph : dgl.Graph","title":"Parameters"},{"location":"development/devel/api/#returns_5","text":"torch.Tensor predicted cost of given graph. Retrieve float value with .item()","title":"Returns"},{"location":"development/devel/api/#gcnpredictor-objects_1","text":"class GCNPredictor(Predictor)","title":"GCNPredictor Objects"},{"location":"development/devel/api/#__init___2","text":"def __init__(input_feature_size, hidden_units=[128], readout=\"mean\", fea_name=\"features\") -> None G(raph)CN based network latency/cost predictor. End-to-end from graph to score.","title":"__init__"},{"location":"development/devel/api/#parameters_8","text":"input_feature_size : [type] length of feature vector of a graph node (graph G with n nodes, each with features of length m, i.e. feature matrix F = n x m) hidden_units : int, list, optional size of hidden layer (layers if list) , by default 128 readout : str, optional readout function that is used to aggregate node features, by default 'mean' fea_name : str, optional internal name for features in the graph, as in graph.ndata[fea_name], by default 'features'","title":"Parameters"},{"location":"development/devel/api/#train_and_fit","text":"def train_and_fit(dataloader, learning_rate=1e-3, num_epochs=200, validation_dataloader=None, verbose=0) Train GCN model","title":"train_and_fit"},{"location":"development/devel/api/#parameters_9","text":"dataloader : GraphDataLoader training data learning_rate : [type], optional by default 1e-3 num_epochs : int, optional by default 200 validation_dataloader : [type], optional if given, use this data to print validation loss, by default None verbose : int if validation_dataloader is given, print validation MSE every epoch, by default 1","title":"Parameters"},{"location":"development/devel/api/#predict_2","text":"def predict(graph) predict cost of graph","title":"predict"},{"location":"development/devel/api/#parameters_10","text":"graph : dgl.Graph","title":"Parameters"},{"location":"development/devel/api/#returns_6","text":"torch.Tensor predicted cost of given graph. Retrieve float value with .item()","title":"Returns"},{"location":"development/devel/api/#gaussianprocesspredictor-objects","text":"class GaussianProcessPredictor(Predictor)","title":"GaussianProcessPredictor Objects"},{"location":"development/devel/api/#__init___3","text":"def __init__(input_feature_size, hidden_units=128, embedding_size=10, readout=\"mean\", fea_name=\"features\", kernel=\"default\", alpha=1e-10) -> None Predictor that generates a graph embedding that is used as input for a gaussian process predictor.","title":"__init__"},{"location":"development/devel/api/#parameters_11","text":"input_feature_size : [type] length of feature vector of a graph node (graph G with n nodes, each with features of length m, i.e. feature matrix F = n x m) hidden_units : int, list, optional size of hidden layer (layers if list) , by default 128 embedding_size: int, optional size of output embedding readout : str, optional readout function that is used to aggregate node features, by default 'mean' fea_name : str, optional internal name for features in the graph, as in graph.ndata[fea_name], by default 'features' kernel : str, sklearn.gaussian_process.kernels.Kernel, optional The gaussian process kernel to use. input shoudl be either \"default\", or a sklearn Kernel() object by default RBF() + DotProduct() + WhiteKernel()","title":"Parameters"},{"location":"development/devel/api/#train_and_fit_1","text":"def train_and_fit(dataloader, learning_rate=1e-3, num_epochs=200, validation_dataloader=None, verbose=1) Train GCN model, generate embeddings for training data and fit the predictor with embeddings.","title":"train_and_fit"},{"location":"development/devel/api/#parameters_12","text":"dataloader : GraphDataLoader training data learning_rate : [type], optional by default 1e-3 num_epochs : int, optional by default 200 validation_dataloader : [type], optional if given, use this data to print validation loss, by default None verbose : int if validation_dataloader is given, print validation MSE every epoch,by default 1","title":"Parameters"},{"location":"development/devel/api/#returns_7","text":"float score of predictor on TRAINING data, see sklearn doc of chosen predictor for more info","title":"Returns"},{"location":"development/devel/api/#predict_3","text":"def predict(X, return_std=True) Predict cost/latency of graphs.","title":"predict"},{"location":"development/devel/api/#parameters_13","text":"X : dgl.DGLGraph, list[DGLGraph], dgl.dataloading.GraphDataLoader Input graph(s) return_std : bool, optional if true, return standard dev. else just mean prediction, by default True","title":"Parameters"},{"location":"development/devel/api/#returns_8","text":"array (,array) prediction(s) , (if return_std: standard deviation(s))","title":"Returns"},{"location":"development/devel/api/#xgbpredictor-objects","text":"class XGBPredictor(Predictor)","title":"XGBPredictor Objects"},{"location":"development/devel/api/#__init___4","text":"def __init__(input_feature_size, hidden_units=128, embedding_size=10, readout=\"mean\", fea_name=\"features\", xgb_param=\"default\") -> None Predictor that generates a graph embedding that is used as input for a xgb based predictor.","title":"__init__"},{"location":"development/devel/api/#parameters_14","text":"input_feature_size : [type] length of feature vector of a graph node (graph G with n nodes, each with features of length m, i.e. feature matrix F = n x m) hidden_units : int, list, optional size of hidden layer (layers if list) , by default 128 embedding_size: int, optional size of output embedding readout : str, optional readout function that is used to aggregate node features, by default 'mean' fea_name : str, optional internal name for features in the graph, as in graph.ndata[fea_name], by default 'features' xgb_param : str, dict, optional The xgb_parameter to use. See https://xgboost.readthedocs.io/en/latest/parameter.html","title":"Parameters"},{"location":"development/devel/api/#train_and_fit_2","text":"def train_and_fit(dataloader, learning_rate=1e-3, num_epochs=200, num_round=8000, validation_dataloader=None, verbose=1) Train GCN model, generate embeddings for training data and fit the predictor with embeddings.","title":"train_and_fit"},{"location":"development/devel/api/#parameters_15","text":"dataloader : GraphDataLoader training data learning_rate : [type], optional by default 1e-3 num_epochs : int, optional Training epochs for the GCN embedding network, by default 200 num_round : int, optional training rounds for xgb booster, by default 800 validation_dataloader : [type], optional if given, use this data to print validation loss, by default None verbose : int if validation_dataloader is given, print validation MSE every epoch,by default 1","title":"Parameters"},{"location":"development/devel/api/#predict_4","text":"def predict(X) Predict cost/latency of graphs.","title":"predict"},{"location":"development/devel/api/#parameters_16","text":"X : dgl.DGLGraph, list[DGLGraph], dgl.dataloading.GraphDataLoader Input graph(s) Returns array (,array) prediction(s) , (if return_std: standard deviation(s))","title":"Parameters"},{"location":"development/devel/api/#prepare_dataloader","text":"def prepare_dataloader(dataset, batch_size=50, train_test_split=1, subset=0, seed=0, validation=False) helper function to construct dataloaders from NASGraphDataset","title":"prepare_dataloader"},{"location":"development/devel/api/#parameters_17","text":"dataset : NASGraphDataset batch_size : int, optional by default 50 train_test_split : float, optional number between 0 and 1, the proportion of the dataset to be used for training, by default 1 subset : int, optional choose only many samples from the dataset. Set 0 for disabling, i.e. whole dataset. by default 0 seed : int, optional set seed for reproduceability validation : bool, optional also output a validation set e.g. for hyperparam tuning","title":"Parameters"},{"location":"development/devel/api/#returns_9","text":"tuple(GraphDataLoader, (GraphDataLoader), GraphDataLoader) training dataloader to be used in CostPredictor.train() and test/validation dataloader if train_test_split > 0, else len(test_dataloader) == 0","title":"Returns"},{"location":"development/devel/api/#hannahnasperformance_predictiongcnmodel","text":"","title":"hannah.nas.performance_prediction.gcn.model"},{"location":"development/devel/api/#hannahnasperformance_predictionfeaturesdataset","text":"","title":"hannah.nas.performance_prediction.features.dataset"},{"location":"development/devel/api/#hannahnasperformance_predictionfeaturesgraph_conversion","text":"","title":"hannah.nas.performance_prediction.features.graph_conversion"},{"location":"development/devel/api/#hannahnasperformance_prediction","text":"","title":"hannah.nas.performance_prediction"},{"location":"development/devel/api/#hannahnasperformance_predictionexamplesgcn_predictor_example","text":"","title":"hannah.nas.performance_prediction.examples.gcn_predictor_example"},{"location":"development/devel/api/#hannahnasperformance_predictionexamplesgcn_model_example","text":"","title":"hannah.nas.performance_prediction.examples.gcn_model_example"},{"location":"development/devel/api/#hannahnasperformance_predictionexamplesri_capsule_performance_predictor","text":"","title":"hannah.nas.performance_prediction.examples.ri_capsule_performance_predictor"},{"location":"development/devel/api/#hannahnasperformance_predictionexamplesgaussian_process_predictor_example","text":"","title":"hannah.nas.performance_prediction.examples.gaussian_process_predictor_example"},{"location":"development/devel/api/#hannahnasperformance_predictionexamplesxgb_predictor_example","text":"","title":"hannah.nas.performance_prediction.examples.xgb_predictor_example"},{"location":"development/devel/api/#hannahnasconstraintsconstraint_model","text":"","title":"hannah.nas.constraints.constraint_model"},{"location":"development/devel/api/#hannahnasconstraintsdfg_constraint_model","text":"","title":"hannah.nas.constraints.dfg_constraint_model"},{"location":"development/devel/api/#constraintmodel-objects","text":"class ConstraintModel()","title":"ConstraintModel Objects"},{"location":"development/devel/api/#process_optype","text":"def process_optype(op: OpType) Extracts the constraints based on the type of op. New variables are added to self.vars and the constraints are added to the solver.","title":"process_optype"},{"location":"development/devel/api/#parameters_18","text":"op : OpType","title":"Parameters"},{"location":"development/devel/api/#process_tensor","text":"def process_tensor(tensor: Tensor) Goes through all axis and extracts the constraints for the respective axis sizes","title":"process_tensor"},{"location":"development/devel/api/#parameters_19","text":"tensor : Tensor","title":"Parameters"},{"location":"development/devel/api/#hannahnashardware_descriptiondevice","text":"","title":"hannah.nas.hardware_description.device"},{"location":"development/devel/api/#hannahnashardware_descriptionmemory_type","text":"","title":"hannah.nas.hardware_description.memory_type"},{"location":"development/devel/api/#hannahnashardware_description","text":"","title":"hannah.nas.hardware_description"},{"location":"development/devel/api/#hannahnashardware_descriptiondescription","text":"","title":"hannah.nas.hardware_description.description"},{"location":"development/devel/api/#hannahnasevalextract","text":"","title":"hannah.nas.eval.extract"},{"location":"development/devel/api/#hannahnaseval","text":"","title":"hannah.nas.eval"},{"location":"development/devel/api/#hannahnasevalplot","text":"","title":"hannah.nas.eval.plot"},{"location":"development/devel/api/#hannahnaseval__main__","text":"","title":"hannah.nas.eval.__main__"},{"location":"development/devel/api/#hannahnasevalprepare","text":"","title":"hannah.nas.eval.prepare"},{"location":"development/devel/api/#prepare_summary","text":"def prepare_summary(data: Dict[str, str], base_dir: str = \".\", force: bool = False) -> pd.DataFrame Prepare a summary of one or multiple nas runs Arguments : data Dict[str, str] - A mapping from a short name for a nas run \"e.g KWS\" to a folder containing nas history file e.g. \"trained_models/nas_kws/conv_net_trax\" base_dir str - base directory paths in data mapping are interpreted relative to base directory force bool - force reconstructing of cached results (\"data.pkl\")","title":"prepare_summary"},{"location":"development/devel/api/#hannahnasexpressionsconditions","text":"","title":"hannah.nas.expressions.conditions"},{"location":"development/devel/api/#hannahnasexpressionsop","text":"","title":"hannah.nas.expressions.op"},{"location":"development/devel/api/#hannahnasexpressionsarithmetic","text":"","title":"hannah.nas.expressions.arithmetic"},{"location":"development/devel/api/#hannahnasexpressionsplaceholder","text":"","title":"hannah.nas.expressions.placeholder"},{"location":"development/devel/api/#hannahnasexpressions","text":"","title":"hannah.nas.expressions"},{"location":"development/devel/api/#hannahnasexpressionslogic","text":"","title":"hannah.nas.expressions.logic"},{"location":"development/devel/api/#hannahnasexpressionsshapes","text":"","title":"hannah.nas.expressions.shapes"},{"location":"development/devel/api/#hannahnasexpressionsmetrics","text":"","title":"hannah.nas.expressions.metrics"},{"location":"development/devel/api/#hannahnasexpressionstypes","text":"","title":"hannah.nas.expressions.types"},{"location":"development/devel/api/#hannahnasexpressionschoice","text":"","title":"hannah.nas.expressions.choice"},{"location":"development/devel/api/#hannahnastesttest_symbolic_metrics","text":"","title":"hannah.nas.test.test_symbolic_metrics"},{"location":"development/devel/api/#hannahnastesttest_parameters","text":"","title":"hannah.nas.test.test_parameters"},{"location":"development/devel/api/#hannahnastesttest_op_to_torch_conversion","text":"","title":"hannah.nas.test.test_op_to_torch_conversion"},{"location":"development/devel/api/#hannahnastesttest_conditions","text":"","title":"hannah.nas.test.test_conditions"},{"location":"development/devel/api/#hannahnastesttest_lazy_torch","text":"","title":"hannah.nas.test.test_lazy_torch"},{"location":"development/devel/api/#hannahnastesttest_mobilenet","text":"","title":"hannah.nas.test.test_mobilenet"},{"location":"development/devel/api/#hannahnastesttest_scoping","text":"","title":"hannah.nas.test.test_scoping"},{"location":"development/devel/api/#hannahnastesttest_darts_space","text":"","title":"hannah.nas.test.test_darts_space"},{"location":"development/devel/api/#hannahnastesttest_arithmetic","text":"","title":"hannah.nas.test.test_arithmetic"},{"location":"development/devel/api/#test_unimplemeted","text":"@pytest.mark.parametrize( \"x,y\", [ (IntScalarParameter(0, 0), 2), (IntScalarParameter(0, 0), IntScalarParameter(0, 0)), (DefaultInt(0), 2), ], ) def test_unimplemeted(x, y) Test that unimplemented methods raise unimplemented errors","title":"test_unimplemeted"},{"location":"development/devel/api/#hannahnastesttest_z3","text":"","title":"hannah.nas.test.test_z3"},{"location":"development/devel/api/#hannahnastesttest_graph_transformer","text":"","title":"hannah.nas.test.test_graph_transformer"},{"location":"development/devel/api/#hannahnastesttest_dfg_removal","text":"","title":"hannah.nas.test.test_dfg_removal"},{"location":"development/devel/api/#hannahnastesttest_repeat","text":"","title":"hannah.nas.test.test_repeat"},{"location":"development/devel/api/#hannahnastesttest_parametrize","text":"","title":"hannah.nas.test.test_parametrize"},{"location":"development/devel/api/#hannahnastestnetwork","text":"","title":"hannah.nas.test.network"},{"location":"development/devel/api/#hannahnastesttest_description_ultratrail","text":"","title":"hannah.nas.test.test_description_ultratrail"},{"location":"development/devel/api/#hannahnastesttest_dataflow","text":"","title":"hannah.nas.test.test_dataflow"},{"location":"development/devel/api/#hannahnastesttest_conv2d","text":"","title":"hannah.nas.test.test_conv2d"},{"location":"development/devel/api/#hannahnastesttest_constraint_model","text":"","title":"hannah.nas.test.test_constraint_model"},{"location":"development/devel/api/#hannahnastesttest_add","text":"","title":"hannah.nas.test.test_add"},{"location":"development/devel/api/#hannahnastesttest_adjacency","text":"","title":"hannah.nas.test.test_adjacency"},{"location":"development/devel/api/#hannahnasparametrization","text":"","title":"hannah.nas.parametrization"},{"location":"development/devel/api/#scalarparameterstate-objects","text":"@dataclass class ScalarParameterState(ParameterState)","title":"ScalarParameterState Objects"},{"location":"development/devel/api/#sigma","text":"variance of sampling parameter","title":"sigma"},{"location":"development/devel/api/#hannahnasparametersparameters","text":"","title":"hannah.nas.parameters.parameters"},{"location":"development/devel/api/#hannahnasparametersparametrize","text":"","title":"hannah.nas.parameters.parametrize"},{"location":"development/devel/api/#hannahnasparametersiterators","text":"","title":"hannah.nas.parameters.iterators"},{"location":"development/devel/api/#hannahnasparameters","text":"","title":"hannah.nas.parameters"},{"location":"development/devel/api/#hannahnasparameterslazy","text":"","title":"hannah.nas.parameters.lazy"},{"location":"development/devel/api/#hannahnas","text":"","title":"hannah.nas"},{"location":"development/devel/api/#hannahnascoreparametrized","text":"","title":"hannah.nas.core.parametrized"},{"location":"development/devel/api/#hannahnascore","text":"","title":"hannah.nas.core"},{"location":"development/devel/api/#hannahnascoreexpression","text":"","title":"hannah.nas.core.expression"},{"location":"development/devel/api/#hannahnasconfig","text":"","title":"hannah.nas.config"},{"location":"development/devel/api/#scalar-objects","text":"@dataclass class Scalar() Representation of all the options to define a scalar.","title":"Scalar Objects"},{"location":"development/devel/api/#hannahnas__main__","text":"","title":"hannah.nas.__main__"},{"location":"development/devel/api/#hannahnasplotter","text":"","title":"hannah.nas.plotter"},{"location":"development/devel/api/#hannahnassearchsearch","text":"","title":"hannah.nas.search.search"},{"location":"development/devel/api/#hannahnassearchmodel_trainersimple_model_trainer","text":"","title":"hannah.nas.search.model_trainer.simple_model_trainer"},{"location":"development/devel/api/#hannahnassearchmodel_trainerprogressive_shrinking","text":"","title":"hannah.nas.search.model_trainer.progressive_shrinking"},{"location":"development/devel/api/#progressiveshrinkingmodeltrainer-objects","text":"class ProgressiveShrinkingModelTrainer()","title":"ProgressiveShrinkingModelTrainer Objects"},{"location":"development/devel/api/#warmup","text":"def warmup(model, ofa_model) The function rebuilds the trainer with the warmup epochs, fits the model, validates the model, and then calls the on_warmup_end() function to change some internal variables Arguments : model : the model to be trained ofa_model : the model that we want to train","title":"warmup"},{"location":"development/devel/api/#train_elastic_width","text":"def train_elastic_width(model, ofa_model) The function trains the model for a number of epochs, then adds a width step, then trains the model for a number of epochs, then adds a width step, and so on Arguments : model : the model to train ofa_model : the model that will be trained","title":"train_elastic_width"},{"location":"development/devel/api/#train_elastic_depth","text":"def train_elastic_depth(model, ofa_model) The function trains the model for a number of epochs, then progressively shrinks the depth of the model, and trains the model for a number of epochs again Arguments : model : the model to train ofa_model : the model to be trained","title":"train_elastic_depth"},{"location":"development/devel/api/#train_elastic_kernel","text":"def train_elastic_kernel(model, ofa_model) The function trains the elastic kernels by progressively shrinking the model and training the model for a number of epochs and repeats this process until the number of kernel steps is reached Arguments : model : the model to train ofa_model : the model that will be trained","title":"train_elastic_kernel"},{"location":"development/devel/api/#train_elastic_dilation","text":"def train_elastic_dilation(model, ofa_model) The function trains the model for a number of epochs, then adds a dilation step, and trains the model for a number of epochs, and repeats this process until the number of dilation steps is reached Arguments : model : the model to be trained ofa_model : the model that will be trained","title":"train_elastic_dilation"},{"location":"development/devel/api/#train_elastic_grouping","text":"def train_elastic_grouping(model, ofa_model) The function trains the model for a number of epochs, then adds a group step, and trains the model for a number of epochs, and repeats this process until the number of group steps is reached Arguments : model : the model to be trained ofa_model : the model that will be trained","title":"train_elastic_grouping"},{"location":"development/devel/api/#train_elastic_dsc","text":"def train_elastic_dsc(model, ofa_model) The function trains the model for a number of epochs, then adds a dsc step (turns Depthwise Separable Convolution on and off), and trains the model for a number of epochs, and repeats this process until the number of dsc steps is reached Arguments : model : the model to be trained ofa_model : the model that will be trained","title":"train_elastic_dsc"},{"location":"development/devel/api/#eval_elastic_width","text":"def eval_elastic_width(method_stack, method_index, lightning_model, model, trainer_path, loginfo_output, metrics_output, metrics_csv) This function steps down the width of the model, and then calls the next method in the stack Arguments : method_stack : a list of methods that will be called in order method_index : The index of the current method in the method stack lightning_model : the lightning model to be trained model : the model to be trained trainer_path : The path to the trainer loginfo_output : This is the string that will be printed to the console metrics_output : a string that will be written to the metrics csv file metrics_csv : a string that contains the metrics for the current model Returns : The metrics_csv is being returned.","title":"eval_elastic_width"},{"location":"development/devel/api/#eval_elastic_kernel","text":"def eval_elastic_kernel(method_stack, method_index, lightning_model, model, trainer_path, loginfo_output, metrics_output, metrics_csv) This function steps down the kernel size of the model, and then calls the next method in the stack Arguments : method_stack : The list of methods to be called method_index : The index of the current method in the method stack lightning_model : the lightning model to be trained model : the model to be trained trainer_path : The path to the trainer loginfo_output : This is the string that will be printed to the console metrics_output : This is the string that will be printed to the console metrics_csv : a string that contains the metrics for the current model Returns : The metrics_csv is being returned.","title":"eval_elastic_kernel"},{"location":"development/devel/api/#eval_elastic_dilation","text":"def eval_elastic_dilation(method_stack, method_index, lightning_model, model, trainer_path, loginfo_output, metrics_output, metrics_csv) This function evaluates the model with a different dilation size for each layer Arguments : method_stack : The list of methods to be called method_index : The index of the method in the method stack lightning_model : the lightning model to be trained model : the model to be evaluated trainer_path : The path to the trainer loginfo_output : This is the string that will be printed to the console metrics_output : a string that will be written to the metrics csv file metrics_csv : a string that contains the csv data for the metrics Returns : The metrics_csv is being returned.","title":"eval_elastic_dilation"},{"location":"development/devel/api/#eval_elastic_depth","text":"def eval_elastic_depth(method_stack, method_index, lightning_model, model, trainer_path, loginfo_output, metrics_output, metrics_csv) This function will run the next method in the stack for each depth step, and then return the metrics_csv Arguments : method_stack : The list of methods to be called method_index : The index of the current method in the method stack lightning_model : the lightning model to be trained model : The model to be trained trainer_path : The path to the trainer, which is used to save the model loginfo_output : This is the string that will be printed to the console metrics_output : This is the string that will be printed to the console metrics_csv : This is the CSV file that we're writing to Returns : The metrics_csv is being returned.","title":"eval_elastic_depth"},{"location":"development/devel/api/#eval_elastic_grouping","text":"def eval_elastic_grouping(method_stack, method_index, lightning_model, model, trainer_path, loginfo_output, metrics_output, metrics_csv) This function evaluates the model with a different group size for each layer Arguments : method_stack : The list of methods to be called method_index : The index of the method in the method stack lightning_model : the lightning model to be trained model : the model to be evaluated trainer_path : The path to the trainer loginfo_output : This is the string that will be printed to the console metrics_output : a string that will be written to the metrics csv file metrics_csv : a string that contains the csv data for the metrics Returns : The metrics_csv is being returned.","title":"eval_elastic_grouping"},{"location":"development/devel/api/#eval_elastic_dsc","text":"def eval_elastic_dsc(method_stack, method_index, lightning_model, model, trainer_path, loginfo_output, metrics_output, metrics_csv) This function evaluates the model with a different dsc for each layer Arguments : method_stack : The list of methods to be called method_index : The index of the method in the method stack lightning_model : the lightning model to be trained model : the model to be evaluated trainer_path : The path to the trainer loginfo_output : This is the string that will be printed to the console metrics_output : a string that will be written to the metrics csv file metrics_csv : a string that contains the csv data for the metrics Returns : The metrics_csv is being returned.","title":"eval_elastic_dsc"},{"location":"development/devel/api/#eval_single_model","text":"def eval_single_model(method_stack, method_index, lightning_model, model, trainer_path, loginfo_output, metrics_output, metrics_csv) This function takes in a model, a trainer, and a bunch of other stuff, evaluates the model and tracks the results in der in the given strings and returns a string of metrics Arguments : method_stack : The list of methods that we're evaluating method_index : The index of the method in the method stack lightning_model : the lightning model that we want to evaluate model : The model to be evaluated trainer_path : the path to the trainer object loginfo_output : This is the string that will be printed to the console when the model is being evaluated metrics_output : This is the string that will be written to the metrics file. It contains the method name, the method index, and the method stack metrics_csv : a string that will be written to a csv file Returns : The metrics_csv is being returned.","title":"eval_single_model"},{"location":"development/devel/api/#eval_model","text":"def eval_model(lightning_model, model) First the method stack for the evaluation ist build and then it is according to this evaluated Arguments : lightning_model : the lightning model model : the model to be evaluated","title":"eval_model"},{"location":"development/devel/api/#hannahnassearchsearch_old","text":"","title":"hannah.nas.search.search_old"},{"location":"development/devel/api/#ofanastrainer-objects","text":"class OFANasTrainer(NASTrainerBase)","title":"OFANasTrainer Objects"},{"location":"development/devel/api/#warmup_1","text":"def warmup(model, ofa_model) The function rebuilds the trainer with the warmup epochs, fits the model, validates the model, and then calls the on_warmup_end() function to change some internal variables Arguments : model : the model to be trained ofa_model : the model that we want to train","title":"warmup"},{"location":"development/devel/api/#train_elastic_width_1","text":"def train_elastic_width(model, ofa_model) The function trains the model for a number of epochs, then adds a width step, then trains the model for a number of epochs, then adds a width step, and so on Arguments : model : the model to train ofa_model : the model that will be trained","title":"train_elastic_width"},{"location":"development/devel/api/#train_elastic_depth_1","text":"def train_elastic_depth(model, ofa_model) The function trains the model for a number of epochs, then progressively shrinks the depth of the model, and trains the model for a number of epochs again Arguments : model : the model to train ofa_model : the model to be trained","title":"train_elastic_depth"},{"location":"development/devel/api/#train_elastic_kernel_1","text":"def train_elastic_kernel(model, ofa_model) The function trains the elastic kernels by progressively shrinking the model and training the model for a number of epochs and repeats this process until the number of kernel steps is reached Arguments : model : the model to train ofa_model : the model that will be trained","title":"train_elastic_kernel"},{"location":"development/devel/api/#train_elastic_dilation_1","text":"def train_elastic_dilation(model, ofa_model) The function trains the model for a number of epochs, then adds a dilation step, and trains the model for a number of epochs, and repeats this process until the number of dilation steps is reached Arguments : model : the model to be trained ofa_model : the model that will be trained","title":"train_elastic_dilation"},{"location":"development/devel/api/#train_elastic_grouping_1","text":"def train_elastic_grouping(model, ofa_model) The function trains the model for a number of epochs, then adds a group step, and trains the model for a number of epochs, and repeats this process until the number of group steps is reached Arguments : model : the model to be trained ofa_model : the model that will be trained","title":"train_elastic_grouping"},{"location":"development/devel/api/#train_elastic_dsc_1","text":"def train_elastic_dsc(model, ofa_model) The function trains the model for a number of epochs, then adds a dsc step (turns Depthwise Separable Convolution on and off), and trains the model for a number of epochs, and repeats this process until the number of dsc steps is reached Arguments : model : the model to be trained ofa_model : the model that will be trained","title":"train_elastic_dsc"},{"location":"development/devel/api/#eval_elastic_width_1","text":"def eval_elastic_width(method_stack, method_index, lightning_model, model, trainer_path, loginfo_output, metrics_output, metrics_csv) This function steps down the width of the model, and then calls the next method in the stack Arguments : method_stack : a list of methods that will be called in order method_index : The index of the current method in the method stack lightning_model : the lightning model to be trained model : the model to be trained trainer_path : The path to the trainer loginfo_output : This is the string that will be printed to the console metrics_output : a string that will be written to the metrics csv file metrics_csv : a string that contains the metrics for the current model Returns : The metrics_csv is being returned.","title":"eval_elastic_width"},{"location":"development/devel/api/#eval_elastic_kernel_1","text":"def eval_elastic_kernel(method_stack, method_index, lightning_model, model, trainer_path, loginfo_output, metrics_output, metrics_csv) This function steps down the kernel size of the model, and then calls the next method in the stack Arguments : method_stack : The list of methods to be called method_index : The index of the current method in the method stack lightning_model : the lightning model to be trained model : the model to be trained trainer_path : The path to the trainer loginfo_output : This is the string that will be printed to the console metrics_output : This is the string that will be printed to the console metrics_csv : a string that contains the metrics for the current model Returns : The metrics_csv is being returned.","title":"eval_elastic_kernel"},{"location":"development/devel/api/#eval_elastic_dilation_1","text":"def eval_elastic_dilation(method_stack, method_index, lightning_model, model, trainer_path, loginfo_output, metrics_output, metrics_csv) This function evaluates the model with a different dilation size for each layer Arguments : method_stack : The list of methods to be called method_index : The index of the method in the method stack lightning_model : the lightning model to be trained model : the model to be evaluated trainer_path : The path to the trainer loginfo_output : This is the string that will be printed to the console metrics_output : a string that will be written to the metrics csv file metrics_csv : a string that contains the csv data for the metrics Returns : The metrics_csv is being returned.","title":"eval_elastic_dilation"},{"location":"development/devel/api/#eval_elastic_depth_1","text":"def eval_elastic_depth(method_stack, method_index, lightning_model, model, trainer_path, loginfo_output, metrics_output, metrics_csv) This function will run the next method in the stack for each depth step, and then return the metrics_csv Arguments : method_stack : The list of methods to be called method_index : The index of the current method in the method stack lightning_model : the lightning model to be trained model : The model to be trained trainer_path : The path to the trainer, which is used to save the model loginfo_output : This is the string that will be printed to the console metrics_output : This is the string that will be printed to the console metrics_csv : This is the CSV file that we're writing to Returns : The metrics_csv is being returned.","title":"eval_elastic_depth"},{"location":"development/devel/api/#eval_elastic_grouping_1","text":"def eval_elastic_grouping(method_stack, method_index, lightning_model, model, trainer_path, loginfo_output, metrics_output, metrics_csv) This function evaluates the model with a different group size for each layer Arguments : method_stack : The list of methods to be called method_index : The index of the method in the method stack lightning_model : the lightning model to be trained model : the model to be evaluated trainer_path : The path to the trainer loginfo_output : This is the string that will be printed to the console metrics_output : a string that will be written to the metrics csv file metrics_csv : a string that contains the csv data for the metrics Returns : The metrics_csv is being returned.","title":"eval_elastic_grouping"},{"location":"development/devel/api/#eval_elastic_dsc_1","text":"def eval_elastic_dsc(method_stack, method_index, lightning_model, model, trainer_path, loginfo_output, metrics_output, metrics_csv) This function evaluates the model with a different dsc for each layer Arguments : method_stack : The list of methods to be called method_index : The index of the method in the method stack lightning_model : the lightning model to be trained model : the model to be evaluated trainer_path : The path to the trainer loginfo_output : This is the string that will be printed to the console metrics_output : a string that will be written to the metrics csv file metrics_csv : a string that contains the csv data for the metrics Returns : The metrics_csv is being returned.","title":"eval_elastic_dsc"},{"location":"development/devel/api/#eval_single_model_1","text":"def eval_single_model(method_stack, method_index, lightning_model, model, trainer_path, loginfo_output, metrics_output, metrics_csv) This function takes in a model, a trainer, and a bunch of other stuff, evaluates the model and tracks the results in der in the given strings and returns a string of metrics Arguments : method_stack : The list of methods that we're evaluating method_index : The index of the method in the method stack lightning_model : the lightning model that we want to evaluate model : The model to be evaluated trainer_path : the path to the trainer object loginfo_output : This is the string that will be printed to the console when the model is being evaluated metrics_output : This is the string that will be written to the metrics file. It contains the method name, the method index, and the method stack metrics_csv : a string that will be written to a csv file Returns : The metrics_csv is being returned.","title":"eval_single_model"},{"location":"development/devel/api/#eval_model_1","text":"def eval_model(lightning_model, model) First the method stack for the evaluation ist build and then it is according to this evaluated Arguments : lightning_model : the lightning model model : the model to be evaluated","title":"eval_model"},{"location":"development/devel/api/#hannahnassearchweight_sharing","text":"","title":"hannah.nas.search.weight_sharing"},{"location":"development/devel/api/#hannahnassearchsamplerbase_sampler","text":"","title":"hannah.nas.search.sampler.base_sampler"},{"location":"development/devel/api/#sampler-objects","text":"class Sampler(ABC)","title":"Sampler Objects"},{"location":"development/devel/api/#tell_result","text":"def tell_result(parameters, metrics) Tell the result of a task","title":"tell_result"},{"location":"development/devel/api/#hannahnassearchsampleraging_evolution","text":"","title":"hannah.nas.search.sampler.aging_evolution"},{"location":"development/devel/api/#agingevolutionsampler-objects","text":"class AgingEvolutionSampler(Sampler) Aging Evolution based multi objective optimization","title":"AgingEvolutionSampler Objects"},{"location":"development/devel/api/#next_parameters","text":"def next_parameters() Returns a list of current tasks","title":"next_parameters"},{"location":"development/devel/api/#tell_result_1","text":"def tell_result(parameters, metrics) Tell the result of a task","title":"tell_result"},{"location":"development/devel/api/#hannahnassearchsamplerrandom_sampler","text":"","title":"hannah.nas.search.sampler.random_sampler"},{"location":"development/devel/api/#hannahnassearchsamplermutator","text":"","title":"hannah.nas.search.sampler.mutator"},{"location":"development/devel/api/#hannahnassearchutils","text":"","title":"hannah.nas.search.utils"},{"location":"development/devel/api/#hannahnasops","text":"","title":"hannah.nas.ops"},{"location":"development/devel/api/#hannahnasgraph_conversion","text":"","title":"hannah.nas.graph_conversion"},{"location":"development/devel/api/#hannahnasspacesdartsdarts_space","text":"","title":"hannah.nas.spaces.darts.darts_space"},{"location":"development/devel/api/#hannahnasspacesmobilenetmobilenet","text":"","title":"hannah.nas.spaces.mobilenet.mobilenet"},{"location":"development/devel/api/#hannahnasutils","text":"","title":"hannah.nas.utils"},{"location":"development/devel/api/#is_pareto","text":"def is_pareto(costs, maximise=False) Arguments : costs : An (n_points, n_costs) array Returns : A (n_points, ) boolean array, indicating whether each point is Pareto efficient","title":"is_pareto"},{"location":"development/devel/api/#hannahdatasetsvisionri_capsule","text":"Rhode island gastroenterology video capsule endoscopy dataset https://www.nature.com/articles/s41597-022-01726-3 https://github.com/acharoen/Rhode-Island-GI-VCE-Technical-Validation","title":"hannah.datasets.vision.ri_capsule"},{"location":"development/devel/api/#hannahdatasetsvisionfake","text":"","title":"hannah.datasets.vision.fake"},{"location":"development/devel/api/#hannahdatasetsvisioncifar","text":"","title":"hannah.datasets.vision.cifar"},{"location":"development/devel/api/#hannahdatasetsvisionkvasir","text":"","title":"hannah.datasets.vision.kvasir"},{"location":"development/devel/api/#hannahdatasetsvisionkvasir_unlabeled","text":"","title":"hannah.datasets.vision.kvasir_unlabeled"},{"location":"development/devel/api/#kvasircapsuleunlabeled-objects","text":"class KvasirCapsuleUnlabeled(AbstractDataset) Dataset representing unlabelled videos","title":"KvasirCapsuleUnlabeled Objects"},{"location":"development/devel/api/#hannahdatasetsvision","text":"","title":"hannah.datasets.vision"},{"location":"development/devel/api/#hannahdatasetsvisiondresden_capsule","text":"Rhode island gastroenterology video capsule endoscopy dataset https://www.nature.com/articles/s41597-022-01726-3 https://github.com/acharoen/Rhode-Island-GI-VCE-Technical-Validation","title":"hannah.datasets.vision.dresden_capsule"},{"location":"development/devel/api/#hannahdatasetsvisionutils","text":"","title":"hannah.datasets.vision.utils"},{"location":"development/devel/api/#hannahdatasetsvisionutilsnaneye","text":"","title":"hannah.datasets.vision.utils.naneye"},{"location":"development/devel/api/#read_naneye","text":"def read_naneye(data_file: Union[str, Path]) Read a naneye raw aimage and decode bayer pattern Arguments : data_file Union[str, Path] - path to the datafile Returns : np.ndarray - uint8 array of decoded image data","title":"read_naneye"},{"location":"development/devel/api/#hannahdatasetsvisionbase","text":"","title":"hannah.datasets.vision.base"},{"location":"development/devel/api/#torchvisiondatasetbase-objects","text":"class TorchvisionDatasetBase(VisionDatasetBase) Wrapper around torchvision classification datasets","title":"TorchvisionDatasetBase Objects"},{"location":"development/devel/api/#imagedatasetbase-objects","text":"class ImageDatasetBase(VisionDatasetBase)","title":"ImageDatasetBase Objects"},{"location":"development/devel/api/#__init___5","text":"def __init__(X, y, classes, bbox=None, transform=None) Initialize vision dataset Arguments : X List[str] - List of paths to image files y List[str] - Class id of corresponding image classes List[str] - List of class names, names are ordered by numeric class id bbox Dict[str] - Dict with filename as keys, bbox coordinates as numpy arrays transform Callable[image,image], optional - Optional transformation/augmentation of input images. Defaults to None.","title":"__init__"},{"location":"development/devel/api/#hannahdatasetsemergency","text":"","title":"hannah.datasets.emergency"},{"location":"development/devel/api/#emergencysirendataset-objects","text":"class EmergencySirenDataset(AbstractDataset) Emergency Dataset","title":"EmergencySirenDataset Objects"},{"location":"development/devel/api/#hannahdatasetsdownsample","text":"","title":"hannah.datasets.Downsample"},{"location":"development/devel/api/#hannahdatasetscollate","text":"","title":"hannah.datasets.collate"},{"location":"development/devel/api/#vision_collate_fn","text":"def vision_collate_fn(batch) Function that takes in a batch of data and puts the elements within the batch into a tensor with an additional outer dimension - batch size. The exact output type can be a :class: torch.Tensor , a Sequence of :class: torch.Tensor , a Collection of :class: torch.Tensor , or left unchanged, depending on the input type. This is used as the default function for collation for vision tasks batch_size or batch_sampler is defined in :class: ~torch.utils.data.DataLoader . Here is the general input type (based on the type of the element within the batch) to output type mapping: :class: torch.Tensor -> :class: torch.Tensor (with an added outer dimension batch size) NumPy Arrays -> :class: torch.Tensor float -> :class: torch.Tensor int -> :class: torch.Tensor str -> str (unchanged) bytes -> bytes (unchanged) Mapping[K, V_i] -> Mapping[K, vision_collate([V_1, V_2, ...])] NamedTuple[V1_i, V2_i, ...] -> NamedTuple[vision_collate([V1_1, V1_2, ...]), vision_collate([V2_1, V2_2, ...]), ...] Sequence[V1_i, V2_i, ...] -> Sequence[vision_collate([V1_1, V1_2, ...]), vision_collate([V2_1, V2_2, ...]), ...] Arguments : batch - a single batch to be collated Examples :","title":"vision_collate_fn"},{"location":"development/devel/api/#example-with-a-batch-of-ints","text":"vision_collate([0, 1, 2, 3]) tensor([0, 1, 2, 3])","title":"Example with a batch of ints:"},{"location":"development/devel/api/#example-with-a-batch-of-strs","text":"vision_collate(['a', 'b', 'c']) ['a', 'b', 'c']","title":"Example with a batch of strs:"},{"location":"development/devel/api/#example-with-map-inside-the-batch","text":"vision_collate([{'A': 0, 'B': 1}, {'A': 100, 'B': 100}]) - {'A' - tensor([ 0, 100]), 'B': tensor([ 1, 100])}","title":"Example with Map inside the batch:"},{"location":"development/devel/api/#example-with-namedtuple-inside-the-batch","text":"Point = namedtuple('Point', ['x', 'y']) vision_collate([Point(0, 0), Point(1, 1)]) Point(x=tensor([0, 1]), y=tensor([0, 1]))","title":"Example with NamedTuple inside the batch:"},{"location":"development/devel/api/#example-with-tuple-inside-the-batch","text":"vision_collate([(0, 1), (2, 3)]) [tensor([0, 2]), tensor([1, 3])]","title":"Example with Tuple inside the batch:"},{"location":"development/devel/api/#example-with-list-inside-the-batch","text":"vision_collate([[0, 1], [2, 3]]) [tensor([0, 2]), tensor([1, 3])]","title":"Example with List inside the batch:"},{"location":"development/devel/api/#ctc_collate_fn","text":"def ctc_collate_fn(data) Creates mini-batch tensors from the list of tuples (src_seq, trg_seq). We should build a custom collate_fn rather than using default collate_fn, because merging sequences (including padding) is not supported in default. Sequences are padded to the maximum length of mini-batch sequences (dynamic padding). Arguments : data - list of tuple (src_seq, src_length, trg_seq, trg_length). src_seq: torch tensor of shape (x,?); variable length. src length: torch tenso of shape 1x1 trg_seq: torch tensor of shape (?); variable length. trg_length: torch_tensor of shape (1x1) Returns - tuple of four torch tensors src_seqs - torch tensor of shape (batch_size, x, padded_length). src_lengths - torch_tensor of shape (batch_size); valid length for each padded source sequence. trg_seqs - torch tensor of shape (batch_size, x, padded_length). trg_lengths - torch tensor of shape (batch_size); valid length for each padded target sequence.","title":"ctc_collate_fn"},{"location":"development/devel/api/#hannahdatasetsspeech","text":"","title":"hannah.datasets.speech"},{"location":"development/devel/api/#speechdataset-objects","text":"class SpeechDataset(AbstractDataset) Base Class for speech datasets","title":"SpeechDataset Objects"},{"location":"development/devel/api/#preprocess","text":"def preprocess(example, silence=False, label=0) Run preprocessing and feature extraction","title":"preprocess"},{"location":"development/devel/api/#speechcommandsdataset-objects","text":"class SpeechCommandsDataset(SpeechDataset) This class implements reading and preprocessing of speech commands like dataset","title":"SpeechCommandsDataset Objects"},{"location":"development/devel/api/#speechhotworddataset-objects","text":"class SpeechHotwordDataset(SpeechDataset) Dataset Class for Hotword dataset e.g. Hey Snips!","title":"SpeechHotwordDataset Objects"},{"location":"development/devel/api/#splits","text":"@classmethod def splits(cls, config) Splits the dataset in training, devlopment and test set and returns the three sets as List","title":"splits"},{"location":"development/devel/api/#vaddataset-objects","text":"class VadDataset(SpeechDataset)","title":"VadDataset Objects"},{"location":"development/devel/api/#splits_1","text":"@classmethod def splits(cls, config) Splits the dataset in training, devlopment and test set and returns the three sets as List","title":"splits"},{"location":"development/devel/api/#hannahdatasetsactivity","text":"","title":"hannah.datasets.activity"},{"location":"development/devel/api/#data3d-objects","text":"class Data3D() 3D-Data","title":"Data3D Objects"},{"location":"development/devel/api/#pampap2_imudata-objects","text":"class PAMPAP2_IMUData() A IMU set defined by temperature (\u00b0C) 3D-acceleration data (ms -2 ), scale: \u00b116g, resolution: 13-bit 3D-acceleration data (ms -2 ), scale: \u00b16g, resolution: 13-bit 3D-gyroscope data (rad/s) 3D-magnetometer data (\u03bcT) orientation (invalid in this data collection)","title":"PAMPAP2_IMUData Objects"},{"location":"development/devel/api/#pamap2_datapoint-objects","text":"class PAMAP2_DataPoint() A temporal datapoint in the dataset","title":"PAMAP2_DataPoint Objects"},{"location":"development/devel/api/#pamap2_datachunk-objects","text":"class PAMAP2_DataChunk() A DataChunk is a item of the pytorch dataset","title":"PAMAP2_DataChunk Objects"},{"location":"development/devel/api/#pamap2_dataset-objects","text":"class PAMAP2_Dataset(AbstractDataset) Class for the PAMAP2 activity dataset https://archive.ics.uci.edu/ml/datasets/pamap2+physical+activity+monitoring","title":"PAMAP2_Dataset Objects"},{"location":"development/devel/api/#hannahdatasetsdirectional","text":"","title":"hannah.datasets.directional"},{"location":"development/devel/api/#directionaldataset-objects","text":"class DirectionalDataset(AbstractDataset) Directional Dataset","title":"DirectionalDataset Objects"},{"location":"development/devel/api/#hannahdatasetsdatasetsplit","text":"","title":"hannah.datasets.DatasetSplit"},{"location":"development/devel/api/#hannahdatasets","text":"","title":"hannah.datasets"},{"location":"development/devel/api/#hannahdatasetsphysio","text":"","title":"hannah.datasets.physio"},{"location":"development/devel/api/#atrialfibrillationdataset-objects","text":"class AtrialFibrillationDataset(PhysioDataset) Atrial Fibrillation Database (https://physionet.org/content/afdb/1.0.0/)","title":"AtrialFibrillationDataset Objects"},{"location":"development/devel/api/#hannahdatasetsnoisedataset","text":"","title":"hannah.datasets.NoiseDataset"},{"location":"development/devel/api/#hannahdatasetsfake1d","text":"","title":"hannah.datasets.fake1d"},{"location":"development/devel/api/#hannahdatasetsutilscache","text":"","title":"hannah.datasets.utils.cache"},{"location":"development/devel/api/#hannahdatasetsutils","text":"","title":"hannah.datasets.utils"},{"location":"development/devel/api/#hannahdatasetsutilsmd5","text":"","title":"hannah.datasets.utils.md5"},{"location":"development/devel/api/#hannahdatasetsbase","text":"","title":"hannah.datasets.base"},{"location":"development/devel/api/#datasettype-objects","text":"class DatasetType(Enum) The type of a dataset partition e.g. train, dev, test","title":"DatasetType Objects"},{"location":"development/devel/api/#abstractdataset-objects","text":"class AbstractDataset(Dataset, ABC)","title":"AbstractDataset Objects"},{"location":"development/devel/api/#prepare_4","text":"@abstractclassmethod def prepare(cls, config: Dict[str, Any]) -> None Prepare the dataset. This method is run at the beginning of the dataset training. If possible it should download the dataset from its original source, if it is available for public download. Arguments : config Dict[Any] - The dataset configuration","title":"prepare"},{"location":"development/devel/api/#splits_2","text":"@abstractclassmethod def splits( cls, config: Dict[str, Any] ) -> Tuple[\"AbstractDataset\", \"AbstractDataset\", \"AbstractDataset\"] Returns the test, validation and train split according to the Dataset config Arguments : config [type] - [description]","title":"splits"},{"location":"development/devel/api/#class_names","text":"@abstractproperty def class_names() -> List[str] Returns the names of the classes in the classification dataset","title":"class_names"},{"location":"development/devel/api/#class_counts","text":"@abstractproperty def class_counts() -> Optional[Dict[int, int]] Returns the number of items in each class of the dataset If this is not applicable to a dataset type e.g. ASR, Semantic Segementation, it may return None","title":"class_counts"},{"location":"development/devel/api/#__getitem__","text":"@abstractmethod def __getitem__(index) -> List[torch.Tensor] Returns a torch.Tensor for the data item at the corresponding index The length of the list depends on the dataset item to use Arguments : index int - the index of the data item","title":"__getitem__"},{"location":"development/devel/api/#__len__","text":"@abstractmethod def __len__() -> int Returns number of samples in dataset","title":"__len__"},{"location":"development/devel/api/#size","text":"def size() -> List[int] Returns dimension of output without batch dimension","title":"size"},{"location":"development/devel/api/#std","text":"@property def std() -> Optional[Tuple[int, ...]] Returns channel-wise standard deviation for dataset if applicable","title":"std"},{"location":"development/devel/api/#mean","text":"@property def mean() -> Optional[Tuple[int, ...]] Returns channel-wise means for dataset if applicable","title":"mean"},{"location":"development/devel/api/#resolution","text":"@property def resolution() -> Optional[Tuple[int, ...]] Returns resolution for dataset if applicable","title":"resolution"},{"location":"development/devel/api/#weights","text":"@property def weights() -> Optional[List[float]] Class weights for weighted sampling","title":"weights"},{"location":"development/devel/api/#hannahdatasetskitti","text":"","title":"hannah.datasets.Kitti"},{"location":"development/devel/api/#kitti-objects","text":"class Kitti(AbstractDataset)","title":"Kitti Objects"},{"location":"development/devel/api/#splits_3","text":"@classmethod def splits(cls, config) Splits the dataset in training, devlopment and test set and returns the three sets as List","title":"splits"},{"location":"development/devel/api/#hannahnnquantized","text":"","title":"hannah.nn.quantized"},{"location":"development/devel/api/#hannahnnqat","text":"Implementations of torch.nn.intrinsics qat with an optional quantize bias parameter. Qconfigs can support an optional bias quantization funciton which should be returned by qconfig.bias() else biases will be quantized with qconfig.activation()","title":"hannah.nn.qat"},{"location":"development/devel/api/#_convbnnd-objects","text":"class _ConvBnNd(nn.modules.conv._ConvNd, _ConvForwardMixin)","title":"_ConvBnNd Objects"},{"location":"development/devel/api/#train_3","text":"def train(mode: bool = True) -> Any Batchnorm's training behavior is using the self.training flag. Prevent changing it if BN is frozen. This makes sure that calling model.train() on a model with a frozen BN will behave properly.","title":"train"},{"location":"development/devel/api/#from_float_2","text":"@classmethod def from_float(cls, mod) Create a qat module from a float module or qparams_dict Args: mod a float module, either produced by torch.quantization utilities or directly from user","title":"from_float"},{"location":"development/devel/api/#convbn1d-objects_1","text":"class ConvBn1d(_ConvBnNd) A ConvBn1d module is a module fused from Conv1d and BatchNorm1d, attached with FakeQuantize modules for weight, used in quantization aware training. We combined the interface of :class: torch.nn.Conv1d and :class: torch.nn.BatchNorm1d . Similar to :class: torch.nn.Conv1d , with FakeQuantize modules initialized to default. Attributes : freeze_bn: - weight_fake_quant - fake quant module for weight","title":"ConvBn1d Objects"},{"location":"development/devel/api/#convbnrelu1d-objects_1","text":"class ConvBnReLU1d(ConvBn1d) A ConvBnReLU1d module is a module fused from Conv1d, BatchNorm1d and ReLU, attached with FakeQuantize modules for weight, used in quantization aware training. We combined the interface of :class: torch.nn.Conv1d and :class: torch.nn.BatchNorm1d and :class: torch.nn.ReLU . Similar to torch.nn.Conv1d , with FakeQuantize modules initialized to default. Attributes : weight_fake_quant - fake quant module for weight","title":"ConvBnReLU1d Objects"},{"location":"development/devel/api/#convbn2d-objects","text":"class ConvBn2d(_ConvBnNd) A ConvBn2d module is a module fused from Conv2d and BatchNorm2d, attached with FakeQuantize modules for weight, used in quantization aware training. We combined the interface of :class: torch.nn.Conv2d and :class: torch.nn.BatchNorm2d . Similar to :class: torch.nn.Conv2d , with FakeQuantize modules initialized to default. Attributes : freeze_bn: - weight_fake_quant - fake quant module for weight","title":"ConvBn2d Objects"},{"location":"development/devel/api/#convbnrelu2d-objects","text":"class ConvBnReLU2d(ConvBn2d) A ConvBnReLU2d module is a module fused from Conv2d, BatchNorm2d and ReLU, attached with FakeQuantize modules for weight, used in quantization aware training. We combined the interface of :class: torch.nn.Conv2d and :class: torch.nn.BatchNorm2d and :class: torch.nn.ReLU . Similar to torch.nn.Conv2d , with FakeQuantize modules initialized to default. Attributes : weight_fake_quant - fake quant module for weight","title":"ConvBnReLU2d Objects"},{"location":"development/devel/api/#convrelu2d-objects","text":"class ConvReLU2d(nn.Conv2d, _ConvForwardMixin) A ConvReLU2d module is a fused module of Conv2d and ReLU, attached with FakeQuantize modules for weight for quantization aware training. We combined the interface of :class: ~torch.nn.Conv2d and :class: ~torch.nn.BatchNorm2d . Attributes : weight_fake_quant - fake quant module for weight","title":"ConvReLU2d Objects"},{"location":"development/devel/api/#convrelu1d-objects_1","text":"class ConvReLU1d(nn.Conv1d, _ConvForwardMixin) A ConvReLU1d module is fused module of Conv1d and ReLU, attached with FakeQuantize modules for quantization aware training","title":"ConvReLU1d Objects"},{"location":"development/devel/api/#conv1d-objects","text":"class Conv1d(nn.Conv1d, _ConvForwardMixin) A Conv1d module is a Conv1d module , attached with FakeQuantize modules for weight for quantization aware training. Attributes : weight_fake_quant - fake quant module for weight bias_fake_quant - fake quant module for bias activation_post_process - fake_quant_module for activations","title":"Conv1d Objects"},{"location":"development/devel/api/#conv2d-objects","text":"class Conv2d(nn.Conv2d, _ConvForwardMixin) A Conv2d module is a Conv2d module , attached with FakeQuantize modules for weight for quantization aware training. Attributes : weight_fake_quant - fake quant module for weight bias_fake_quant - fake quant module for bias activation_post_process - fake_quant_module for activations","title":"Conv2d Objects"},{"location":"development/devel/api/#linear-objects","text":"class Linear(nn.Linear) A linear module attached with FakeQuantize modules for weight, used for quantization aware training. We adopt the same interface as torch.nn.Linear , please see https://pytorch.org/docs/stable/nn.html#torch.nn.Linear for documentation. Similar to torch.nn.Linear , with FakeQuantize modules initialized to default. Attributes : weight - fake quant module for weight","title":"Linear Objects"},{"location":"development/devel/api/#from_float_3","text":"@classmethod def from_float(cls, mod) Create a qat module from a float module or qparams_dict Args: mod a float module, either produced by torch.quantization utilities or directly from user","title":"from_float"},{"location":"development/devel/api/#linearrelu-objects","text":"class LinearReLU(nn.Linear) A linear module attached with FakeQuantize modules and ReLU for weight, used for quantization aware training. We adopt the same interface as torch.nn.Linear , please see https://pytorch.org/docs/stable/nn.html#torch.nn.Linear for documentation. Similar to torch.nn.Linear , with FakeQuantize modules initialized to default. Attributes : weight - fake quant module for weight","title":"LinearReLU Objects"},{"location":"development/devel/api/#from_float_4","text":"@classmethod def from_float(cls, mod) Create a qat module from a float module or qparams_dict Args: mod a float module, either produced by torch.quantization utilities or directly from user","title":"from_float"},{"location":"development/devel/api/#identity-objects","text":"class Identity(nn.Identity) A identity module attached with FakeQuantize modules for weight, used for quantization aware training. We adopt the same interface as torch.nn.Identity , please see https://pytorch.org/docs/stable/nn.html#torch.nn.Identity for documentation. Similar to torch.nn.Identity , with FakeQuantize modules initialized to default.","title":"Identity Objects"},{"location":"development/devel/api/#from_float_5","text":"@classmethod def from_float(cls, mod) Create a qat module from a float module or qparams_dict Args: mod a float module, either produced by torch.quantization utilities or directly from user","title":"from_float"},{"location":"development/devel/api/#hannahquantizationcallback","text":"","title":"hannah.quantization.callback"},{"location":"development/devel/api/#hannahquantizationqconfig","text":"","title":"hannah.quantization.qconfig"},{"location":"development/devel/api/#hannahquantizationrounding","text":"","title":"hannah.quantization.rounding"},{"location":"development/devel/api/#round_downward","text":"def round_downward(x: Tensor) -> Tensor Round to nearest upward","title":"round_downward"},{"location":"development/devel/api/#round_upward","text":"def round_upward(x: Tensor) -> Tensor Round to nearest downward","title":"round_upward"},{"location":"development/devel/api/#round_odd","text":"def round_odd(x: Tensor) -> Tensor Round to nearest odd","title":"round_odd"},{"location":"development/devel/api/#round_even","text":"def round_even(x: Tensor) -> Tensor Round to nearest even","title":"round_even"},{"location":"development/devel/api/#round_zero","text":"def round_zero(x: Tensor) -> Tensor Round towards zero","title":"round_zero"},{"location":"development/devel/api/#round_infinity","text":"def round_infinity(x: Tensor) -> Tensor Round toward infinity","title":"round_infinity"},{"location":"development/devel/api/#truncate_up","text":"def truncate_up(x: Tensor) -> Tensor Always round up to next integer","title":"truncate_up"},{"location":"development/devel/api/#truncate_down","text":"def truncate_down(x: Tensor) -> Tensor Always round down to next integer","title":"truncate_down"},{"location":"development/devel/api/#truncate_infinity","text":"def truncate_infinity(x: Tensor) -> Tensor Always round to next integer in direction infinity","title":"truncate_infinity"},{"location":"development/devel/api/#truncate_zero","text":"def truncate_zero(x: Tensor) -> Tensor Always round to next integer in direction of Zero","title":"truncate_zero"},{"location":"development/devel/api/#round_stochastic","text":"def round_stochastic(x: Tensor) -> Tensor Round stochastically","title":"round_stochastic"},{"location":"development/devel/api/#hannahmodulesvisionanomaly_score","text":"class AnomalyScore(CatMetric): def init (self, percentile, nan_strategy=\"warn\", kwargs): super(). init (nan_strategy=nan_strategy, kwargs) self.percentile = percentile def compute(self): anomaly_score = None train_losses = super().compute() if train_losses: normalized_train_errors = torch.stack(train_losses) / ( torch.max(torch.stack(train_losses), dim=0).values ) anomaly_score = np.percentile( normalized_train_errors.cpu().numpy(), self.percentile ) return anomaly_score","title":"hannah.modules.vision.anomaly_score"},{"location":"development/devel/api/#hannahmodulesvisionanomaly_detection","text":"","title":"hannah.modules.vision.anomaly_detection"},{"location":"development/devel/api/#anomalydetectionmodule-objects","text":"class AnomalyDetectionModule(VisionBaseModule)","title":"AnomalyDetectionModule Objects"},{"location":"development/devel/api/#on_test_end_4","text":"def on_test_end() wd_dir = os.getcwd() score, largest_train_error = self.compute_anomaly_score() train_errors = self.normalized_train_errors plt.hist(train_errors.detach().cpu().numpy(), bins=100) plt.axvline(score, linestyle=\"dashed\") plt.title(\"Normalized train reconstruction errors\") plt.savefig(wd_dir + \"/normalized_train_errors.png\") test = ( torch.tensor(self.test_losses, device=self.device) / torch.max(torch.stack(self.train_losses), dim=0).values ) plt.hist(test.detach().cpu().numpy(), bins=100) plt.title(\"Normalized test reconstruction errors\") plt.savefig(wd_dir + \"/normalized_test_errors.png\") print(\"Anomaly score\", score) print( \"Largest train error\", torch.max(torch.stack(self.train_losses), dim=0).values, )","title":"on_test_end"},{"location":"development/devel/api/#hannahmodulesvisionloss","text":"","title":"hannah.modules.vision.loss"},{"location":"development/devel/api/#hannahmodulesvision","text":"","title":"hannah.modules.vision"},{"location":"development/devel/api/#hannahmodulesvisionimage_classifier","text":"","title":"hannah.modules.vision.image_classifier"},{"location":"development/devel/api/#hannahmodulesvisionbase","text":"","title":"hannah.modules.vision.base"},{"location":"development/devel/api/#hannahmodulesclassifier","text":"","title":"hannah.modules.classifier"},{"location":"development/devel/api/#hannahmodulesobject_detection","text":"","title":"hannah.modules.object_detection"},{"location":"development/devel/api/#hannahmodulesconfig_utils","text":"","title":"hannah.modules.config_utils"},{"location":"development/devel/api/#dump_config","text":"def dump_config(output_dir, config) Dumps the configuration to json format Creates file config.json in output_dir","title":"dump_config"},{"location":"development/devel/api/#parameters_20","text":"output_dir : str Output directory config : dict Configuration to dump","title":"Parameters"},{"location":"development/devel/api/#save_model","text":"def save_model(output_dir, model) Creates serialization of the model for later inference, evaluation Creates the following files: model.pt: Serialized version of network parameters in pytorch model.json: Serialized version of network parameters in json format model.onnx: full model including paramters in onnx format","title":"save_model"},{"location":"development/devel/api/#parameters_21","text":"output_dir : str Directory to put serialized models model : LightningModule Model to serialize","title":"Parameters"},{"location":"development/devel/api/#hannahmodules","text":"","title":"hannah.modules"},{"location":"development/devel/api/#hannahmodulesangle_classifier","text":"","title":"hannah.modules.angle_classifier"},{"location":"development/devel/api/#hannahmodulesmetrics","text":"","title":"hannah.modules.metrics"},{"location":"development/devel/api/#error-objects","text":"class Error() Computes Error = 1 - Accuracy_ .. math:: \\text{Error} = 1 - \\frac{1}{N}\\sum_i^N 1(y_i = \\hat{y}_i) Where :math: y is a tensor of target values, and :math: \\hat{y} is a tensor of predictions. This module is a simple wrapper to get the task specific versions of this metric, which is done by setting the task argument to either 'binary' , 'multiclass' or multilabel . See the documentation of :mod: BinaryError , :mod: MulticlassError and :mod: MultilabelError for the specific details of each argument influence and examples.","title":"Error Objects"},{"location":"development/devel/api/#plot_confusion_matrix","text":"def plot_confusion_matrix(cf, group_names=None, categories=\"auto\", count=True, percent=True, cbar=True, xyticks=True, xyplotlabels=True, sum_stats=True, figsize=None, cmap=\"Blues\", title=None) This function will make a pretty plot of an sklearn Confusion Matrix cm using a Seaborn heatmap visualization.","title":"plot_confusion_matrix"},{"location":"development/devel/api/#arguments","text":"cf: confusion matrix to be passed in group_names: List of strings that represent the labels row by row to be shown in each square. categories: List of strings containing the categories to be displayed on the x,y axis. Default is 'auto' count: If True, show the raw number in the confusion matrix. Default is True. normalize: If True, show the proportions for each category. Default is True. cbar: If True, show the color bar. The cbar values are based off the values in the confusion matrix. Default is True. xyticks: If True, show x and y ticks. Default is True. xyplotlabels: If True, show 'True Label' and 'Predicted Label' on the figure. Default is True. sum_stats: If True, display summary statistics below the figure. Default is True. figsize: Tuple representing the figure size. Default will be the matplotlib rcParams value. cmap: Colormap of the values displayed from matplotlib.pyplot.cm. Default is 'Blues' See http://matplotlib.org/examples/color/colormaps_reference.html title: Title for the heatmap. Default is None.","title":"Arguments"},{"location":"development/devel/api/#hannahmodulesbase","text":"","title":"hannah.modules.base"},{"location":"development/devel/api/#classifiermodule-objects","text":"class ClassifierModule(LightningModule, ABC)","title":"ClassifierModule Objects"},{"location":"development/devel/api/#total_training_steps","text":"def total_training_steps() -> int Total training steps inferred from datamodule and devices.","title":"total_training_steps"},{"location":"development/devel/api/#hannahmodulesaugmentationaugmentation","text":"","title":"hannah.modules.augmentation.augmentation"},{"location":"development/devel/api/#hannahmodulesaugmentation","text":"","title":"hannah.modules.augmentation"},{"location":"development/devel/api/#hannahmodulesaugmentationbatch_augmentation","text":"","title":"hannah.modules.augmentation.batch_augmentation"},{"location":"development/devel/api/#batchaugmentationpipeline-objects","text":"class BatchAugmentationPipeline(nn.Module)","title":"BatchAugmentationPipeline Objects"},{"location":"development/devel/api/#__init___6","text":"def __init__(transforms={}) Augmentation pipeline especially for self supervised learning Arguments : replica int - number of replicated different random augmentations transforms dict - configuration of transforms","title":"__init__"},{"location":"development/devel/api/#forward_69","text":"@torch.no_grad() def forward(x) -> torch.Tensor Perform Augmentations Arguments : x torch.Tensor - a torch.Tensor representing the augementation pipeline Returns : Tuple[torch.Tensor, torch.Tensor]; Batch augmented with replica different random augmentations","title":"forward"},{"location":"development/devel/api/#hannahmodulesaugmentationtransformsregistry","text":"","title":"hannah.modules.augmentation.transforms.registry"},{"location":"development/devel/api/#hannahmodulesaugmentationtransformskornia_transforms","text":"","title":"hannah.modules.augmentation.transforms.kornia_transforms"},{"location":"development/devel/api/#hannahmodulesaugmentationtransforms","text":"","title":"hannah.modules.augmentation.transforms"},{"location":"development/devel/api/#hannahmodulesaugmentationbordersearch","text":"","title":"hannah.modules.augmentation.bordersearch"},{"location":"development/devel/api/#hannahutilsimports","text":"","title":"hannah.utils.imports"},{"location":"development/devel/api/#lazy_import","text":"def lazy_import(module_name, callback=None) Returns a proxy module object that will lazily import the given module the first time it is used. Example usage::","title":"lazy_import"},{"location":"development/devel/api/#lazy-version-of-import-tensorflow-as-tf","text":"tf = lazy_import(\"tensorflow\")","title":"Lazy version of import tensorflow as tf"},{"location":"development/devel/api/#other-commands","text":"","title":"Other commands"},{"location":"development/devel/api/#now-the-module-is-loaded","text":"tf. version Arguments : module_name - the fully-qualified module name to import callback None - a callback function to call before importing the module Returns : a proxy module object that will be lazily imported when first used","title":"Now the module is loaded"},{"location":"development/devel/api/#lazymodule-objects","text":"class LazyModule(types.ModuleType) Proxy module that lazily imports the underlying module the first time it is actually used. Arguments : module_name - the fully-qualified module name to import callback None - a callback function to call before importing the module","title":"LazyModule Objects"},{"location":"development/devel/api/#hannahutils","text":"","title":"hannah.utils"},{"location":"development/devel/api/#hannahutilsutils","text":"","title":"hannah.utils.utils"},{"location":"development/devel/api/#log_execution_env_state","text":"def log_execution_env_state() -> None Log information about the execution environment. File 'config_path' will be copied to directory 'logdir'. A common use-case is passing the path to a (compression) schedule YAML file. Storing a copy of the schedule file, with the experiment logs, is useful in order to reproduce experiments. Arguments : config_path - path to config file, used only when logdir is set logdir - log directory git_root - the path to the .git root directory","title":"log_execution_env_state"},{"location":"development/devel/api/#extract_from_download_cache","text":"def extract_from_download_cache(filename, url, cached_files, target_cache, target_folder, target_test_folder=\"\", clear_download=False, no_exist_check=False) -> None extracts given file from cache or donwloads first from url Arguments : filename str - name of the file to download or extract url str - possible url to download the file cached_files (list(str)): cached files in download cache target_cache str - path to the folder to cache file if download necessary target_folder str - path where to extract file target_test_folder str, optional - folder to check if data are already there clear_download bool - clear download after usage no_exist_check bool - disables the check if folder exists","title":"extract_from_download_cache"},{"location":"development/devel/api/#fullname","text":"def fullname(o) -> Any Get the full classname of an object including surrounding packages/modules/namespaces","title":"fullname"},{"location":"development/devel/api/#set_deterministic","text":"@contextmanager def set_deterministic(mode, warn_only=False) A contextmanager to set deterministic algorithms","title":"set_deterministic"},{"location":"development/devel/api/#hannahvisualization","text":"","title":"hannah.visualization"},{"location":"development/devel/debugging/","text":"Debugging Hannah supports several debugging tools. Trainer Debugging flags: Most of the lightning trainer debug flags are usable through the hydra configs using trainer.flag name The most important ones are: trainer.fast_dev_run=True This provides a kind of unit test for the model by running one single batch of training, validation and test on the model. trainer.overfit_batches=0.01 This tests if the model can overfit a small percentage (0.01) of the dataset. A good model should be able reach almost 100 percent accuracy very quickly. trainer.limit_*_batches This limits the amount of batches used for training. e.g.: hannah-train trainer.limit_train_batches=0.1 trainer.limit_val_batches=0.1 trainer.limit_test_batches=0.1 Limits the training to 10 percent of available batches for train, validation and test.","title":"Debugging"},{"location":"development/devel/debugging/#debugging","text":"Hannah supports several debugging tools.","title":"Debugging"},{"location":"development/devel/debugging/#trainer-debugging-flags","text":"Most of the lightning trainer debug flags are usable through the hydra configs using trainer.flag name The most important ones are:","title":"Trainer Debugging flags:"},{"location":"development/devel/debugging/#trainerfast_dev_runtrue","text":"This provides a kind of unit test for the model by running one single batch of training, validation and test on the model.","title":"trainer.fast_dev_run=True"},{"location":"development/devel/debugging/#traineroverfit_batches001","text":"This tests if the model can overfit a small percentage (0.01) of the dataset. A good model should be able reach almost 100 percent accuracy very quickly.","title":"trainer.overfit_batches=0.01"},{"location":"development/devel/debugging/#trainerlimit__batches","text":"This limits the amount of batches used for training. e.g.: hannah-train trainer.limit_train_batches=0.1 trainer.limit_val_batches=0.1 trainer.limit_test_batches=0.1 Limits the training to 10 percent of available batches for train, validation and test.","title":"trainer.limit_*_batches"},{"location":"development/devel/overview/","text":"Development Guidelines The main development repository is available at: https://es-git.cs.uni-tuebingen.de/es/ai/hannah/hannah Code Style This project uses pre commit hooks for auto formatting and static code analysis. To enable pre commit hooks run the following command in a poetry shell . pre-commit install Naming Conventions Try to follow (pep8)[https://pep8.org/#naming-conventions] and the rest of pep8 to the best of your abilities. The short summary of pep8 is: Use all lowercase names with underscores for methods/functions/modules/packages. Use CamelCase for Classes and Type Variables Use all upperscore with underscores for constants Logging / Console Messages Try to avoid print for doing outputs whenever possible. Use python module (logging)[https://docs.python.org/3.8/howto/logging.html] instead. Default logglevel for informational messages should be logging.info() . If you want to be able to have more finegrained control, use a dedicated logger for your module e.g.: logger = logging.getLogger('datasets.speech') ... logger.info() logger.debug() Testing / CI Try to provide unit tests for new features, unit tests should be written using pytest . Unit test should be designed to run pretty fast and are run for each push. Tests marked with `@pytest.mark.integration are only triggered when --integration is given on the pytest commandline. These tests are only run on pushs to the main branch. Resolving merge conflicts in poetry.lock If you have changed poetry.lock this can result in merge conflicts. The easiest way to resolve them is: git checkout --theirs poetry.lock poetry lock --no-update Try to avoid running poetry update on feature branches. Branching We apply the following code branch naming convention. main : main development branch should be as stable as possible f/* : feature branches, development branches (will be squash merged after review, and successful passing of merge request pipeline) fix/* bug fix branches, branches containing little self contained bug fixes for existing features (feel free to merge without code review, if you can reasonably assume that they are not breaking anything that is not already broken) pub/* : publication branches, branches containing the version corresponding to a specific, full branch name should contain abbreviation of publication and publication year. Publication branches and main branch are automatically mirrored to the public github repository: https://github.com/ekut-es/hannah","title":"Overview"},{"location":"development/devel/overview/#development-guidelines","text":"The main development repository is available at: https://es-git.cs.uni-tuebingen.de/es/ai/hannah/hannah","title":"Development Guidelines"},{"location":"development/devel/overview/#code-style","text":"This project uses pre commit hooks for auto formatting and static code analysis. To enable pre commit hooks run the following command in a poetry shell . pre-commit install","title":"Code Style"},{"location":"development/devel/overview/#naming-conventions","text":"Try to follow (pep8)[https://pep8.org/#naming-conventions] and the rest of pep8 to the best of your abilities. The short summary of pep8 is: Use all lowercase names with underscores for methods/functions/modules/packages. Use CamelCase for Classes and Type Variables Use all upperscore with underscores for constants","title":"Naming Conventions"},{"location":"development/devel/overview/#logging-console-messages","text":"Try to avoid print for doing outputs whenever possible. Use python module (logging)[https://docs.python.org/3.8/howto/logging.html] instead. Default logglevel for informational messages should be logging.info() . If you want to be able to have more finegrained control, use a dedicated logger for your module e.g.: logger = logging.getLogger('datasets.speech') ... logger.info() logger.debug()","title":"Logging / Console Messages"},{"location":"development/devel/overview/#testing-ci","text":"Try to provide unit tests for new features, unit tests should be written using pytest . Unit test should be designed to run pretty fast and are run for each push. Tests marked with `@pytest.mark.integration are only triggered when --integration is given on the pytest commandline. These tests are only run on pushs to the main branch.","title":"Testing / CI"},{"location":"development/devel/overview/#resolving-merge-conflicts-in-poetrylock","text":"If you have changed poetry.lock this can result in merge conflicts. The easiest way to resolve them is: git checkout --theirs poetry.lock poetry lock --no-update Try to avoid running poetry update on feature branches.","title":"Resolving merge conflicts in poetry.lock"},{"location":"development/devel/overview/#branching","text":"We apply the following code branch naming convention. main : main development branch should be as stable as possible f/* : feature branches, development branches (will be squash merged after review, and successful passing of merge request pipeline) fix/* bug fix branches, branches containing little self contained bug fixes for existing features (feel free to merge without code review, if you can reasonably assume that they are not breaking anything that is not already broken) pub/* : publication branches, branches containing the version corresponding to a specific, full branch name should contain abbreviation of publication and publication year. Publication branches and main branch are automatically mirrored to the public github repository: https://github.com/ekut-es/hannah","title":"Branching"},{"location":"development/devel/profiling/","text":"Profiling Tools To debug training inefficiencies we support the following profiling tools: pytorch autograd profiler: hannah-train profiler=simple trainer.max_epochs=1 profiler=pytorch By default profiles are logged to the trainings output directory and can be visualized using tensorboard. For the example run use: tensorboard --logdir trained_models/test/tc-res8 Simple Profiler: hannah-train profiler=simple trainer.max_epochs=1 profiler=simple Generates a summary of the durations of the individual training steps: Action | Mean duration (s) |Num calls | Total time (s) | Percentage % | ------------------------------------------------------------------------------------------------------------------------------------ Total | - |_ | 29.42 | 100 % | ------------------------------------------------------------------------------------------------------------------------------------ run_training_epoch | 21.692 |1 | 21.692 | 73.733 | run_training_batch | 0.030893 |288 | 8.8971 | 30.241 | optimizer_step_and_closure_0 | 0.030348 |288 | 8.7402 | 29.708 | get_train_batch | 0.022253 |288 | 6.4087 | 21.783 | evaluation_step_and_end | 0.021021 |277 | 5.8228 | 19.792 | validation_step | 0.020913 |277 | 5.7929 | 19.69 | training_step_and_backward | 0.019199 |288 | 5.5292 | 18.794 | model_forward | 0.010312 |288 | 2.9697 | 10.094 | training_step | 0.010076 |288 | 2.9017 | 9.8631 | model_backward | 0.0076062 |288 | 2.1906 | 7.4458 | on_validation_epoch_end | 0.091247 |2 | 0.18249 | 0.6203 | on_train_batch_end | 0.0003854 |288 | 0.111 | 0.37728 | on_train_start | 0.09102 |1 | 0.09102 | 0.30938 | on_train_epoch_end | 0.055033 |1 | 0.055033 | 0.18706 | cache_result | 1.6399e-05 |2290 | 0.037554 | 0.12765 | on_validation_batch_end | 9.3069e-05 |277 | 0.02578 | 0.087628 | on_validation_end | 0.008392 |2 | 0.016784 | 0.05705 | on_train_end | 0.0075968 |1 | 0.0075968 | 0.025822 | on_batch_start | 2.4213e-05 |288 | 0.0069733 | 0.023702 | on_after_backward | 1.5198e-05 |288 | 0.004377 | 0.014877 | on_before_zero_grad | 1.4302e-05 |288 | 0.0041189 | 0.014 | on_train_batch_start | 1.264e-05 |288 | 0.0036403 | 0.012374 | on_batch_end | 1.1853e-05 |288 | 0.0034136 | 0.011603 | training_step_end | 9.123e-06 |288 | 0.0026274 | 0.0089307 | on_validation_batch_start | 8.5754e-06 |277 | 0.0023754 | 0.008074 | validation_step_end | 7.5117e-06 |277 | 0.0020807 | 0.0070725 | on_validation_start | 0.00019774 |2 | 0.00039547 | 0.0013442 | on_epoch_start | 0.00021634 |1 | 0.00021634 | 0.00073533 | on_epoch_end | 1.4326e-05 |3 | 4.2978e-05 | 0.00014608 | on_validation_epoch_start | 1.0761e-05 |2 | 2.1523e-05 | 7.3157e-05 | on_fit_start | 2.0443e-05 |1 | 2.0443e-05 | 6.9486e-05 | on_train_epoch_start | 1.2379e-05 |1 | 1.2379e-05 | 4.2077e-05 | on_before_accelerator_backend_setup | 1.1862e-05 |1 | 1.1862e-05 | 4.0319e-05 | Advanced Profiler hannah-train profiler=simple trainer.max_epochs=1 profiler=advanced Creates python level performance summaries for the individual Training steps: Profile stats for: optimizer_step_and_closure_0 8064 function calls in 8.897 seconds Ordered by: cumulative time List reduced from 25 to 20 due to restriction <20> ncalls tottime percall cumtime percall filename:lineno(function) 288 0.001 0.000 8.897 0.031 accelerator.py:265(optimizer_step) 288 0.000 0.000 8.896 0.031 accelerator.py:282(run_optimizer_step) 288 0.000 0.000 8.896 0.031 training_type_plugin.py:159(optimizer_step) 288 0.000 0.000 8.896 0.031 grad_mode.py:12(decorate_context) 288 0.000 0.000 8.894 0.031 adamw.py:54(step) 288 0.000 0.000 8.893 0.031 training_loop.py:648(train_step_and_backward_closure) 288 0.000 0.000 8.893 0.031 training_loop.py:737(training_step_and_backward) 288 0.000 0.000 8.892 0.031 contextlib.py:108(__enter__) 288 0.000 0.000 8.892 0.031 {built-in method builtins.next} 288 0.000 0.000 8.892 0.031 profilers.py:62(profile) 288 0.000 0.000 8.892 0.031 profilers.py:250(start) 288 8.892 0.031 8.892 0.031 {method 'enable' of '_lsprof.Profiler' objects} 288 0.000 0.000 0.001 0.000 grad_mode.py:65(__enter__) 288 0.000 0.000 0.000 0.000 accelerator.py:104(lightning_module) 288 0.000 0.000 0.000 0.000 contextlib.py:238(helper) 288 0.000 0.000 0.000 0.000 training_type_plugin.py:91(lightning_module) 288 0.000 0.000 0.000 0.000 grad_mode.py:104(__enter__) 288 0.000 0.000 0.000 0.000 contextlib.py:82(__init__) 576 0.000 0.000 0.000 0.000 {built-in method torch._C.is_grad_enabled} 288 0.000 0.000 0.000 0.000 base.py:82(unwrap_lightning_module)","title":"Profiling"},{"location":"development/devel/profiling/#profiling-tools","text":"To debug training inefficiencies we support the following profiling tools:","title":"Profiling Tools"},{"location":"development/devel/profiling/#pytorch-autograd-profiler","text":"hannah-train profiler=simple trainer.max_epochs=1 profiler=pytorch By default profiles are logged to the trainings output directory and can be visualized using tensorboard. For the example run use: tensorboard --logdir trained_models/test/tc-res8","title":"pytorch autograd profiler:"},{"location":"development/devel/profiling/#simple-profiler","text":"hannah-train profiler=simple trainer.max_epochs=1 profiler=simple Generates a summary of the durations of the individual training steps: Action | Mean duration (s) |Num calls | Total time (s) | Percentage % | ------------------------------------------------------------------------------------------------------------------------------------ Total | - |_ | 29.42 | 100 % | ------------------------------------------------------------------------------------------------------------------------------------ run_training_epoch | 21.692 |1 | 21.692 | 73.733 | run_training_batch | 0.030893 |288 | 8.8971 | 30.241 | optimizer_step_and_closure_0 | 0.030348 |288 | 8.7402 | 29.708 | get_train_batch | 0.022253 |288 | 6.4087 | 21.783 | evaluation_step_and_end | 0.021021 |277 | 5.8228 | 19.792 | validation_step | 0.020913 |277 | 5.7929 | 19.69 | training_step_and_backward | 0.019199 |288 | 5.5292 | 18.794 | model_forward | 0.010312 |288 | 2.9697 | 10.094 | training_step | 0.010076 |288 | 2.9017 | 9.8631 | model_backward | 0.0076062 |288 | 2.1906 | 7.4458 | on_validation_epoch_end | 0.091247 |2 | 0.18249 | 0.6203 | on_train_batch_end | 0.0003854 |288 | 0.111 | 0.37728 | on_train_start | 0.09102 |1 | 0.09102 | 0.30938 | on_train_epoch_end | 0.055033 |1 | 0.055033 | 0.18706 | cache_result | 1.6399e-05 |2290 | 0.037554 | 0.12765 | on_validation_batch_end | 9.3069e-05 |277 | 0.02578 | 0.087628 | on_validation_end | 0.008392 |2 | 0.016784 | 0.05705 | on_train_end | 0.0075968 |1 | 0.0075968 | 0.025822 | on_batch_start | 2.4213e-05 |288 | 0.0069733 | 0.023702 | on_after_backward | 1.5198e-05 |288 | 0.004377 | 0.014877 | on_before_zero_grad | 1.4302e-05 |288 | 0.0041189 | 0.014 | on_train_batch_start | 1.264e-05 |288 | 0.0036403 | 0.012374 | on_batch_end | 1.1853e-05 |288 | 0.0034136 | 0.011603 | training_step_end | 9.123e-06 |288 | 0.0026274 | 0.0089307 | on_validation_batch_start | 8.5754e-06 |277 | 0.0023754 | 0.008074 | validation_step_end | 7.5117e-06 |277 | 0.0020807 | 0.0070725 | on_validation_start | 0.00019774 |2 | 0.00039547 | 0.0013442 | on_epoch_start | 0.00021634 |1 | 0.00021634 | 0.00073533 | on_epoch_end | 1.4326e-05 |3 | 4.2978e-05 | 0.00014608 | on_validation_epoch_start | 1.0761e-05 |2 | 2.1523e-05 | 7.3157e-05 | on_fit_start | 2.0443e-05 |1 | 2.0443e-05 | 6.9486e-05 | on_train_epoch_start | 1.2379e-05 |1 | 1.2379e-05 | 4.2077e-05 | on_before_accelerator_backend_setup | 1.1862e-05 |1 | 1.1862e-05 | 4.0319e-05 |","title":"Simple Profiler:"},{"location":"development/devel/profiling/#advanced-profiler","text":"hannah-train profiler=simple trainer.max_epochs=1 profiler=advanced Creates python level performance summaries for the individual Training steps: Profile stats for: optimizer_step_and_closure_0 8064 function calls in 8.897 seconds Ordered by: cumulative time List reduced from 25 to 20 due to restriction <20> ncalls tottime percall cumtime percall filename:lineno(function) 288 0.001 0.000 8.897 0.031 accelerator.py:265(optimizer_step) 288 0.000 0.000 8.896 0.031 accelerator.py:282(run_optimizer_step) 288 0.000 0.000 8.896 0.031 training_type_plugin.py:159(optimizer_step) 288 0.000 0.000 8.896 0.031 grad_mode.py:12(decorate_context) 288 0.000 0.000 8.894 0.031 adamw.py:54(step) 288 0.000 0.000 8.893 0.031 training_loop.py:648(train_step_and_backward_closure) 288 0.000 0.000 8.893 0.031 training_loop.py:737(training_step_and_backward) 288 0.000 0.000 8.892 0.031 contextlib.py:108(__enter__) 288 0.000 0.000 8.892 0.031 {built-in method builtins.next} 288 0.000 0.000 8.892 0.031 profilers.py:62(profile) 288 0.000 0.000 8.892 0.031 profilers.py:250(start) 288 8.892 0.031 8.892 0.031 {method 'enable' of '_lsprof.Profiler' objects} 288 0.000 0.000 0.001 0.000 grad_mode.py:65(__enter__) 288 0.000 0.000 0.000 0.000 accelerator.py:104(lightning_module) 288 0.000 0.000 0.000 0.000 contextlib.py:238(helper) 288 0.000 0.000 0.000 0.000 training_type_plugin.py:91(lightning_module) 288 0.000 0.000 0.000 0.000 grad_mode.py:104(__enter__) 288 0.000 0.000 0.000 0.000 contextlib.py:82(__init__) 576 0.000 0.000 0.000 0.000 {built-in method torch._C.is_grad_enabled} 288 0.000 0.000 0.000 0.000 base.py:82(unwrap_lightning_module)","title":"Advanced Profiler"},{"location":"evaluation/eval/","text":"Evaluation mode Evaluation of trained models can be run using hannah-eval checkpints=[]","title":"Evaluation"},{"location":"evaluation/eval/#evaluation-mode","text":"Evaluation of trained models can be run using hannah-eval checkpints=[]","title":"Evaluation mode"},{"location":"optimization/optimization/hyperparameter/","text":"Hyperparameter Search The hydra based configuration management allows using multiple hydra sweeper plugins for hyperparameter optimization. The optuna sweeper is installed by default. Sweeper plugins support specifying the hyperparameter search space on the commandline. For example the following command optimizes the learning rate and weight decay parameters for the default optimizer ( adamw ): hannah-train scheduler.max_lr='interval(0.00001, 0.001)' optimizer.weight_decay='interval(0.00001, 0.001)' hydra/sweeper=optuna hydra/launcher=joblib -m Parametrization can be given on the commandline as well as using configuration files for a detailed documentation have a look at basic override syntax and extended override syntax . Optuna Options The optuna hyperparameter optimizer has the following options. hannah-train hydra/sweeper=optuna --cfg hydra -p hydra.sweeper # @package hydra.sweeper sampler: _target_: optuna.samplers.TPESampler seed: null consider_prior: true prior_weight: 1.0 consider_magic_clip: true consider_endpoints: false n_startup_trials: 10 n_ei_candidates: 24 multivariate: false warn_independent_sampling: true _target_: hydra_plugins.hydra_optuna_sweeper.optuna_sweeper.OptunaSweeper direction: minimize storage: null study_name: null n_trials: 20 n_jobs: 2 search_space: {} Try especially increasing the number of trials to run n_trials , and if the sweeper options are dependent on each other try enabling multivariate sampling e.g. sampler.multivariate . State Persistence If you want to save the optimization results for later analyis you can save them to a relational or redis database by setting the study_name and storage options. e.g.: hannah-train experiment_id=\"test_optuna_resume\" hydra/sweeper=optuna hydra.sweeper.study_name='${experiment_id}' hydra.sweeper.storage='sqlite:///${experiment_id}.sqlite' trainer.max_epochs=15 scheduler.max_lr='interval(0.0001, 0.1)' module.num_workers=4 -m The tuning results can then be visualized using the tools from optuna.visualize : A simple example would be: import optuna storage = \"sqlite:///test_optuna_resume.sqlite\" study_name = \"test_optuna_resume\" def main(): study = optuna.load_study(study_name, storage) fig = optuna.visualization.plot_optimization_history(study) fig.write_image(\"history.png\") fig.write_image(\"history.pdf\") if __name__ == '__main__': main() For a slightly extended visualization script have a look at scripts/optuna_visualize_results.py or use optuna-dashboard for a web based visualization. Resuming Trials An additional benefit of exporting the study state to database is that trials can be somewhat resumed at a later time by just restarting the trial with the same parameters. Although this has a view problems: - The hydra sweeper will start it's job numbering from scratch Running on ml-cloud It is possible to use the sqlite backend to store trials on ml-cloud for an example look at scripts/ml_cloud_hyperopt.sh but sqlite might be somewhat unstabble when running multiple workers.","title":"Hyperparameter Optimization"},{"location":"optimization/optimization/hyperparameter/#hyperparameter-search","text":"The hydra based configuration management allows using multiple hydra sweeper plugins for hyperparameter optimization. The optuna sweeper is installed by default. Sweeper plugins support specifying the hyperparameter search space on the commandline. For example the following command optimizes the learning rate and weight decay parameters for the default optimizer ( adamw ): hannah-train scheduler.max_lr='interval(0.00001, 0.001)' optimizer.weight_decay='interval(0.00001, 0.001)' hydra/sweeper=optuna hydra/launcher=joblib -m Parametrization can be given on the commandline as well as using configuration files for a detailed documentation have a look at basic override syntax and extended override syntax .","title":"Hyperparameter Search"},{"location":"optimization/optimization/hyperparameter/#optuna-options","text":"The optuna hyperparameter optimizer has the following options. hannah-train hydra/sweeper=optuna --cfg hydra -p hydra.sweeper # @package hydra.sweeper sampler: _target_: optuna.samplers.TPESampler seed: null consider_prior: true prior_weight: 1.0 consider_magic_clip: true consider_endpoints: false n_startup_trials: 10 n_ei_candidates: 24 multivariate: false warn_independent_sampling: true _target_: hydra_plugins.hydra_optuna_sweeper.optuna_sweeper.OptunaSweeper direction: minimize storage: null study_name: null n_trials: 20 n_jobs: 2 search_space: {} Try especially increasing the number of trials to run n_trials , and if the sweeper options are dependent on each other try enabling multivariate sampling e.g. sampler.multivariate .","title":"Optuna Options"},{"location":"optimization/optimization/hyperparameter/#state-persistence","text":"If you want to save the optimization results for later analyis you can save them to a relational or redis database by setting the study_name and storage options. e.g.: hannah-train experiment_id=\"test_optuna_resume\" hydra/sweeper=optuna hydra.sweeper.study_name='${experiment_id}' hydra.sweeper.storage='sqlite:///${experiment_id}.sqlite' trainer.max_epochs=15 scheduler.max_lr='interval(0.0001, 0.1)' module.num_workers=4 -m The tuning results can then be visualized using the tools from optuna.visualize : A simple example would be: import optuna storage = \"sqlite:///test_optuna_resume.sqlite\" study_name = \"test_optuna_resume\" def main(): study = optuna.load_study(study_name, storage) fig = optuna.visualization.plot_optimization_history(study) fig.write_image(\"history.png\") fig.write_image(\"history.pdf\") if __name__ == '__main__': main() For a slightly extended visualization script have a look at scripts/optuna_visualize_results.py or use optuna-dashboard for a web based visualization.","title":"State Persistence"},{"location":"optimization/optimization/hyperparameter/#resuming-trials","text":"An additional benefit of exporting the study state to database is that trials can be somewhat resumed at a later time by just restarting the trial with the same parameters. Although this has a view problems: - The hydra sweeper will start it's job numbering from scratch","title":"Resuming Trials"},{"location":"optimization/optimization/hyperparameter/#running-on-ml-cloud","text":"It is possible to use the sqlite backend to store trials on ml-cloud for an example look at scripts/ml_cloud_hyperopt.sh but sqlite might be somewhat unstabble when running multiple workers.","title":"Running on ml-cloud"},{"location":"optimization/optimization/nas/","text":":warning: Note: We are currently in the process of restructuring the Neural Architecture Search, in particular the way search spaces are constructed. Some things might not work as usual. Neural architecture search In contrast to hyperparameter optimization neural architecture search, explores new neural network hyperparameters. A aging evolution based neural architecture search has been implemented as a hydra plugin: hannah-train --config-name config_unas To launch multiple configuration jobs in parallel use joblib launcher: hannah-train --config-name config_unas hydra/launcher=joblib Parametrization for neural architecture search need to be given as YAML configuration files at the moment. For an example see: speech_recognition/conf/config_unas.yaml Parametrization The Parametrization contains the following elements: Choice Parameter Choice Parameters select options from a list of parameters. They are configured as a list of options in the parameters. Example: conv_size: [1,3,5,7,9,11] Choice List Parameters Choice List Parameters represent a variable length list of Choices. They are configured with the follwing parameters: min Minimum length of list max Maximum length of list+1 choices List of Choices Example: min: 4 max: 10 choices: - _target_: \"torch.nn.Conv2d\" size: 3 - _target : \"torch.nn.MaxPool2d\" size: 7 Warning : Mutations for intervall parameters currently always sample randomly from the range of values Intervall Parameters Intervall Parameters represent a Scalar Value from an intervall of Values They are configure with the following parameters: lower lower bound of intervall [lower, upper[ upper upper bound of intervall [lower, upper[ int set to true to generate integers log set to true to generate log scaled distribution Subset Parameter Subset Parameters select a subset of a list of choices. They are configured using the following parameters: choices List of choices to sample from size size of the subset to generate Partition Parameter Partition parameters split the list of choices into a predefined number of partitions. They are configured using the following parameters: choices List of choices to partition partition Number of partitions to generate","title":"Neural Architecture Search"},{"location":"optimization/optimization/nas/#neural-architecture-search","text":"In contrast to hyperparameter optimization neural architecture search, explores new neural network hyperparameters. A aging evolution based neural architecture search has been implemented as a hydra plugin: hannah-train --config-name config_unas To launch multiple configuration jobs in parallel use joblib launcher: hannah-train --config-name config_unas hydra/launcher=joblib Parametrization for neural architecture search need to be given as YAML configuration files at the moment. For an example see: speech_recognition/conf/config_unas.yaml","title":"Neural architecture search"},{"location":"optimization/optimization/nas/#parametrization","text":"The Parametrization contains the following elements:","title":"Parametrization"},{"location":"optimization/optimization/nas/#choice-parameter","text":"Choice Parameters select options from a list of parameters. They are configured as a list of options in the parameters. Example: conv_size: [1,3,5,7,9,11]","title":"Choice Parameter"},{"location":"optimization/optimization/nas/#choice-list-parameters","text":"Choice List Parameters represent a variable length list of Choices. They are configured with the follwing parameters: min Minimum length of list max Maximum length of list+1 choices List of Choices Example: min: 4 max: 10 choices: - _target_: \"torch.nn.Conv2d\" size: 3 - _target : \"torch.nn.MaxPool2d\" size: 7 Warning : Mutations for intervall parameters currently always sample randomly from the range of values","title":"Choice List Parameters"},{"location":"optimization/optimization/nas/#intervall-parameters","text":"Intervall Parameters represent a Scalar Value from an intervall of Values They are configure with the following parameters: lower lower bound of intervall [lower, upper[ upper upper bound of intervall [lower, upper[ int set to true to generate integers log set to true to generate log scaled distribution","title":"Intervall Parameters"},{"location":"optimization/optimization/nas/#subset-parameter","text":"Subset Parameters select a subset of a list of choices. They are configured using the following parameters: choices List of choices to sample from size size of the subset to generate","title":"Subset Parameter"},{"location":"optimization/optimization/nas/#partition-parameter","text":"Partition parameters split the list of choices into a predefined number of partitions. They are configured using the following parameters: choices List of choices to partition partition Number of partitions to generate","title":"Partition Parameter"}]} \ No newline at end of file diff --git a/sitemap.xml.gz b/sitemap.xml.gz index c0b2df5fc11885d28c470316695f43a640934c6a..7e822abef4b8a1397bb381f63ca2c82269f2cd6f 100644 GIT binary patch delta 13 Ucmb=gXP58h;OG)(p2%JS02xLDbN~PV delta 13 Ucmb=gXP58h;AsBEG?Bdm03C+}PXGV_