Skip to content

Automatic DNN compression tool with various model compression and neural architecture search techniques

License

Notifications You must be signed in to change notification settings

SKKU-ESLAB/Auto-Compression

Repository files navigation

DNAS-Compression

Model compression techniques with differentiable neural architecture search.

Currently, pruning and quantization are supported.

  • Pruning : Channel-level / Group-level
    • Sparsity and group size can be set
  • Quantization : Uniform quantization
    • Bitwidth can be set

References

This project is implemented based on FBNet reproduced version.

Usage

  • Requirement:
    • pytorch 1.6.0
    • tensorboardX 1.8
    • scipy 1.4.0
    • numpy 1.19.4
  1. Choose what type of compression would you run

    • Pruning ( channel / group )
    • Quantization
    • Both pruning and quantization
  2. Edit hyperparmeters in supernet_functions/config_for_supernet.py

    • Usual hyperparameters
      • batch size
      • learning rate
      • epochs
    • Special hyperparameters (pay attention to it!)
      • alpha, beta for adjustment of flops loss
      • w_share_in_train
      • thetas_lr
      • train_thetas_from_the_epoch
  3. Run supernet_main_file.py

    Quick start command:

    python3 supernet_main_file.py --train_or_sample train

About

Automatic DNN compression tool with various model compression and neural architecture search techniques

Topics

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published