Skip to content

ghimiredhikura/Awasome-Pruning

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 
 
 
 
 
 
 

Repository files navigation

Awesome Pruning Awesome

Visitors

Awesome resources in deep neural network pruning. This collection is inspired by he-y/Awesome-Pruning.

[Note: You are welcome to create pool requests and add more interesting papers.]

Section Year of Publication
Conference Publications 2024 2023 2022 2021 2020 2019 2018 2017
Journal Publications 2024 2023 2022 2021 2020
Survey Articles 2020~2023
Other Publications 2022~2023
Pruning Software and Toolbox 2019~2023
Symbol Meaning
U Unstructured or Weight Pruning
S Structured or Filter or Channel Pruning
A Official or Author Implementation
O Unofficial or 3rd Party Implementation

Conference Publications

2024

Venue Title Type Code
ICLR Towards Meta-Pruning via Optimal Transport S PyTorch[A]
ICLR Towards Energy Efficient Spiking Neural Networks: An Unstructured Pruning Framework U PyTorch[A]
ICLR Masks, Signs, And Learning Rate Rewinding S PyTorch[A]
ICLR Scaling Laws for Sparsely-Connected Foundation Models S PyTorch[A]
ICLR Sparse Model Soups: A Recipe for Improved Pruning via Model Averaging S
ICLR Adaptive Sharpness-Aware Pruning for Robust Sparse Networks S
ICLR What Makes a Good Prune? Maximal Unstructured Pruning for Maximal Cosine Similarity U PyTorch[A]
ICLR In defense of parameter sharing for model-compression S/U
ICLR ECoFLaP: Efficient Coarse-to-Fine Layer-Wise Pruning for Vision-Language Models U
ICLR Data-independent Module-aware Pruning for Hierarchical Vision Transformers S PyTorch[A]
ICLR SWAP: Sparse Entropic Wasserstein Regression for Robust Network Pruning S
ICLR Sparse Weight Averaging with Multiple Particles for Iterative Magnitude Pruning U
ICLR Synergistic Patch Pruning for Vision Transformer: Unifying Intra- & Inter-Layer Patch Importance S
ICLR FedP3: Federated Personalized and Privacy-friendly Network Pruning under Model Heterogeneity S
ICLR The Need for Speed: Pruning Transformers with One Recipe S PyTorch[A]
ICLR SAS: Structured Activation Sparsification S PyTorch[A]
CVPR OrthCaps: An Orthogonal CapsNet with Sparse Attention Routing and Pruning S PyTorch[A]
CVPR Zero-TPrune: Zero-Shot Token Pruning through Leveraging of the Attention Graph in Pre-Trained Transformers S PyTorch[A]
CVPR Finding Lottery Tickets in Vision Models via Data-driven Spectral Foresight Pruning S PyTorch[A]
CVPR BilevelPruning: Unified Dynamic and Static Channel Pruning for Convolutional Neural Networks S
CVPR FedMef: Towards Memory-efficient Federated Dynamic Pruning S
CVPR Resource-Efficient Transformer Pruning for Finetuning of Large Models S
CVPR Device-Wise Federated Network Pruning S
CVPR Auto-Train-Once: Controller Network Guided Automatic Network Pruning from Scratch S
CVPR Jointly Training and Pruning CNNs via Learnable Agent Guidance and Alignment S
CVPR Diversity-aware Channel Pruning for StyleGAN Compression S PyTorch[A]
CVPR MADTP: Multimodal Alignment-Guided Dynamic Token Pruning for Accelerating Vision-Language Transformer S PyTorch[A]
AAAI Dynamic Feature Pruning and Consolidation for Occluded Person Re-Identification S
AAAI REPrune: Channel Pruning via Kernel Representative Selection S
AAAI Revisiting Gradient Pruning: A Dual Realization for Defending against Gradient Attacks S
AAAI IRPruneDet: Efficient Infrared Small Target Detection via Wavelet Structure-Regularized Soft Channel Pruning S
AAAI EPSD: Early Pruning with Self-Distillation for Efficient Model Compression S
WACV Pruning from Scratch via Shared Pruning Module and Nuclear norm-based Regularization S PyTorch[A]
WACV Towards Better Structured Pruning Saliency by Reorganizing Convolution S PyTorch[A]
WACV Torque based Structured Pruning for Deep Neural Network S
WACV Revisiting Token Pruning for Object Detection and Instance Segmentation S PyTorch[A]
WACV Token Fusion: Bridging the Gap Between Token Pruning and Token Merging S
WACV PATROL: Privacy-Oriented Pruning for Collaborative Inference Against Model Inversion Attacks S

2023

Venue Title Type Code
NIPS Diff-Pruning: Structural Pruning for Diffusion Models S PyTorch[A]
NIPS LLM-Pruner: On the Structural Pruning of Large Language Models S PyTorch[A]
ICCV Automatic Network Pruning via Hilbert-Schmidt Independence Criterion Lasso under Information Bottleneck Principle S PyTorch[A]
ICCV Unified Data-Free Compression: Pruning and Quantization without Fine-Tuning S PyTorch[A]
ICCV Structural Alignment for Network Pruning through Partial Regularization S
ICCV Differentiable Transportation Pruning S
ICCV Dynamic Token Pruning in Plain Vision Transformers for Semantic Segmentation S PyTorch[A]
ICCV Towards Fairness-aware Adversarial Network Pruning S
ICCV Efficient Joint Optimization of Layer-Adaptive Weight Pruning in Deep Neural Networks S PyTorch[A]
CVPR DepGraph: Towards Any Structural Pruning S PyTorch[A]
CVPR X-Pruner: eXplainable Pruning for Vision Transformers U/S
CVPR Joint Token Pruning and Squeezing Towards More Aggressive Compression of Vision Transformers S PyTorch[A]
CVPR Global Vision Transformer Pruning with Hessian-Aware Saliency S
CVPR CP3: Channel Pruning Plug-in for Point-based Networks S
CVPR Training Debiased Subnetworks With Contrastive Weight Pruning U
CVPR Pruning Parameterization With Bi-Level Optimization for Efficient Semantic Segmentation on the Edge S
CVPR Structural Alignment for Network Pruning through Partial Regularization S PyTorch[A]
ICLR JaxPruner: A concise library for sparsity research U/S PyTorch[A]
ICLR OTOv2: Automatic, Generic, User-Friendly S PyTorch[A]
ICLR How I Learned to Stop Worrying and Love Retraining U PyTorch[A]
ICLR Token Merging: Your ViT But Faster U/S PyTorch[A]
ICLR Revisiting Pruning at Initialization Through the Lens of Ramanujan Graphs U PyTorch[A] (soon...)
ICLR Unmasking the Lottery Ticket Hypothesis: What's Encoded in a Winning Ticket's Mask? U
ICLR NTK-SAP: Improving neural network pruning by aligning training dynamics U
ICLR DFPC: Data flow driven pruning of coupled channels without data S PyTorch[A]
ICLR TVSPrune - Pruning Non-discriminative filters via Total Variation separability of intermediate representations without fine tuning S PyTorch[A]
ICLR Pruning Deep Neural Networks from a Sparsity Perspective U PyTorch[A]
ICLR A Unified Framework of Soft Threshold Pruning U PyTorch[A]
WACV Calibrating Deep Neural Networks Using Explicit Regularisation and Dynamic Data Pruning S
WACV Attend Who Is Weak: Pruning-Assisted Medical Image Localization Under Sophisticated and Implicit Imbalances S
ICASSP WHC: Weighted Hybrid Criterion for Filter Pruning on Convolutional Neural Networks S PyTorch[A]

2022

Venue Title Type Code
CVPR Interspace Pruning: Using Adaptive Filter Representations To Improve Training of Sparse CNNs U
CVPR Revisiting Random Channel Pruning for Neural Network Compression S PyTorch[A] (soon...)
CVPR Fire Together Wire Together: A Dynamic Pruning Approach With Self-Supervised Mask Prediction S PyTorch[A]
CVPR When to Prune? A Policy towards Early Structural Pruning S
CVPR Dreaming to Prune Image Deraining Networks S
ICLR SOSP: Efficiently Capturing Global Correlations by Second-Order Structured Pruning S
ICLR Learning Pruning-Friendly Networks via Frank-Wolfe: One-Shot, Any-Sparsity, And No Retraining U PyTorch[A]
ICLR Revisit Kernel Pruning with Lottery Regulated Grouped Convolutions S PyTorch[A]
ICLR Dual Lottery Ticket Hypothesis U PyTorch[A]
NIPS SAViT: Structure-Aware Vision Transformer Pruning via Collaborative Optimization S PyTorch[A](soon...)
NIPS Structural Pruning via Latency-Saliency Knapsack S PyTorch[A]
ACCV Filter Pruning via Automatic Pruning Rate Search⋆ S
ACCV Network Pruning via Feature Shift Minimization S PyTorch[A]
ACCV Lightweight Alpha Matting Network Using Distillation-Based Channel Pruning S PyTorch[A]
ACCV Adaptive FSP : Adaptive Architecture Search with Filter Shape Pruning S
ECCV Soft Masking for Cost-Constrained Channel Pruning S PyTorch[A]
WACV Hessian-Aware Pruning and Optimal Neural Implant S PyTorch[A]
WACV PPCD-GAN: Progressive Pruning and Class-Aware Distillation for Large-Scale Conditional GANs Compression S
WACV Channel Pruning via Lookahead Search Guided Reinforcement Learning S
WACV EZCrop: Energy-Zoned Channels for Robust Output Pruning S PyTorch[A]
ICIP One-Cycle Pruning: Pruning Convnets With Tight Training Budget U
ICIP RAPID: A Single Stage Pruning Framework U
ICIP The Rise of the Lottery Heroes: Why Zero-Shot Pruning is Hard U
ICIP Truncated Lottery Ticket for Deep Pruning U
ICIP Which Metrics For Network Pruning: Final Accuracy? or Accuracy Drop? S/U
ISMSI Structured Pruning with Automatic Pruning Rate Derivation for Image Processing Neural Networks S

2021

Venue Title Type Code
ICLR Neural Pruning via Growing Regularization S PyTorch[A]
ICLR Network Pruning That Matters: A Case Study on Retraining Variants S PyTorch[A]
ICLR Layer-adaptive Sparsity for the Magnitude-based Pruning U PyTorch[A]
NIPS Only Train Once: A One-Shot Neural Network Training And Pruning Framework S PyTorch[A]
CVPR NPAS: A Compiler-Aware Framework of Unified Network Pruning and Architecture Search for Beyond Real-Time Mobile Acceleration S
CVPR Network Pruning via Performance Maximization S
CVPR Convolutional Neural Network Pruning With Structural Redundancy Reduction* S
CVPR Manifold Regularized Dynamic Network Pruning S PyTorch[A]
CVPR Joint-DetNAS: Upgrade Your Detector With NAS, Pruning and Dynamic Distillation S
ICCV ResRep: Lossless CNN Pruning via Decoupling Remembering and Forgetting S
ICCV Achieving On-Mobile Real-Time Super-Resolution With Neural Architecture and Pruning Search S
ICCV GDP: Stabilized Neural Network Pruning via Gates With Differentiable Polarization* S
WACV Holistic Filter Pruning for Efficient Deep Neural Networks S
ICML Accelerate CNNs from Three Dimensions: A Comprehensive Pruning Framework S
ICML Group Fisher Pruning for Practical Network Compression S PyTorch[A]

2020

Venue Title Type Code
CVPR HRank: Filter Pruning using High-Rank Feature Map S PyTorch[A]
CVPR Towards efficient model compression via learned global ranking S PyTorch[A]
CVPR Learning Filter Pruning Criteria for Deep Convolutional Neural Networks Acceleration S
CVPR Group Sparsity: The Hinge Between Filter Pruning and Decomposition for Network Compression S PyTorch[A]
CVPR APQ: Joint Search for Network Architecture, Pruning and Quantization Policy S PyTorch[A]
ICLR Budgeted Training: Rethinking Deep Neural Network Training Under Resource Constraints U
MLSys Shrinkbench: What is the State of Neural Network Pruning? PyTorch[A]
BMBS Similarity Based Filter Pruning for Efficient Super-Resolution Models S

2019

Venue Title Type Code
CVPR Filter Pruning via Geometric Median for Deep Convolutional Neural Networks Acceleration S PyTorch[A]
CVPR Variational Convolutional Neural Network Pruning S
CVPR Towards Optimal Structured CNN Pruning via Generative Adversarial Learning S PyTorch[A]
CVPR Partial Order Pruning: For Best Speed/Accuracy Trade-Off in Neural Architecture Search S PyTorch[A]
CVPR Importance Estimation for Neural Network Pruning S PyTorch[A]
ICLR The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networks U PyTorch[A]
ICLR SNIP: Single-shot Network Pruning based on Connection Sensitivity U Tensorflow[A]
ICCV MetaPruning: Meta-Learning for Automatic Neural Network Channel Pruning S PyTorch[A]
ICCV Accelerate CNN via Recursive Bayesian Pruning S

2018

Venue Title Type Code
CVPR PackNet: Adding Multiple Tasks to a Single Network by Iterative Pruning S PyTorch[A]
CVPR NISP: Pruning Networks Using Neuron Importance Score Propagation S
ICIP Online Filter Clustering and Pruning for Efficient Convnets S
IJCAI Soft Filter Pruning for Accelerating Deep Convolutional Neural Networks S PyTorch[A]

2017

Venue Title Type Code
CVPR Designing Energy-Efficient Convolutional Neural Networks Using Energy-Aware Pruning S
ICLR Pruning Filters for Efficient ConvNets S PyTorch[O]
ICCV Channel Pruning for Accelerating Very Deep Neural Networks S PyTorch[A]
ICCV ThiNet: A Filter Level Pruning Method for Deep Neural Network Compression S Caffe[A]
ICCV Learning Efficient Convolutional Networks Through Network Slimming S PyTorch[A]

Journal Publications

2024

Journal Title Type Code
Neural Networks Efficient tensor decomposition-based filter pruning S PyTorch[A]
IEEE Trans. NNLS Enhanced Network Compression Through Tensor Decompositions and Pruning S PyTorch[A]
IEEE Transactions on Artificial Intelligence Distilled Gradual Pruning with Pruned Fine-tuning U PyTorch[A]

2023

Journal Title Type Code
IEEE Trans Circuits Syst Video Technol DCFP: Distribution Calibrated Filter Pruning for Lightweight and Accurate Long-tail Semantic Segmentation S
IEEE Internet Things J. SNPF: Sensitiveness Based Network Pruning Framework for Efficient Edge Computing S
IEEE Trans. NNLS Manipulating Identical Filter Redundancy for Efficient Pruning on Deep and Complicated CNN S
IEEE Trans. NNLS Block-Wise Partner Learning for Model Compression S PyTorch[A]
IEEE Trans. NNLS Hierarchical Threshold Pruning Based on Uniform Response Criterion S
IEEE Trans. NNLS CATRO: Channel Pruning via Class-Aware Trace Ratio Optimization S
IEEE Trans. NNLS Adaptive Filter Pruning via Sensitivity Feedback S
Neurocomputing Filter pruning with uniqueness mechanism in the frequency domain for efficient neural networks S
IEEE Trans. PAMI Compact Neural Network via Stacking Hybrid Units S
IEEE Trans. PAMI Performance-aware Approximation of Global Channel Pruning for Multitask CNNs S PyTorch[A]
IEEE Trans. PAMI Adaptive Search-and-Training for Robust and Efficient Network Pruning S
Image Vis. Comput. Loss-aware automatic selection of structured pruning criteria for deep neural network acceleration S PyTorch[A]
Comput. Vis. Image Underst. Feature independent Filter Pruning by Successive Layers analysis S
IEEE Access Differentiable Neural Architecture, Mixed Precision and Accelerator Co-Search S

2022

Journal Title Type Code
IEEE Trans. Image Process. Efficient Layer Compression Without Pruning S
IEEE Trans. PAMI Learning to Explore Distillability and Sparsability: A Joint Framework for Model Compression S
IEEE Trans. PAMI 1xN Pattern for Pruning Convolutional Neural Networks S PyTorch[A]
IEEE Trans. NNLS Filter Pruning by Switching to Neighboring CNNs With Good Attribute S
IEEE Trans. NNLS Model Pruning Enables Efficient Federated Learning on Edge Devices S
IEEE Trans. NNLS DAIS: Automatic Channel Pruning via Differentiable Annealing Indicator Search S
IEEE Trans. NNLS Network Pruning Using Adaptive Exemplar Filters S PyTorch[A]
IEEE Trans. NNLS Carrying Out CNN Channel Pruning in a White Box S PyTorch[A]
IEEE Trans. NNLS Pruning Networks With Cross-Layer Ranking & k-Reciprocal Nearest Filters S PyTorch[A]
IEEE Trans. NNLS Filter Sketch for Network Pruning S PyTorch[A]
Neurocomputing FPFS: Filter-level pruning via distance weight measuring filter similarity S
Neurocomputing RUFP: Reinitializing unimportant filters for soft pruning S
Neural Netw HRel: Filter pruning based on High Relevance between activation maps and class labels S PyTorch[A]*
Comput. Intell. Neurosci. Differentiable Network Pruning via Polarization of Probabilistic Channelwise Soft Masks S
J. Syst. Archit. Optimizing deep neural networks on intelligent edge accelerators via flexible-rate filter pruning S
Appl. Sci. Magnitude and Similarity Based Variable Rate Filter Pruning for Efficient Convolution Neural Networks S PyTorch[A]
Sensors Filter Pruning via Measuring Feature Map Information S
IEEE Access Automated Filter Pruning Based on High-Dimensional Bayesian Optimization S
IEEE Signal Process. Lett. A Low-Complexity Modified ThiNet Algorithm for Pruning Convolutional Neural Networks S

2021

Journal Title Type Code
IEEE Trans. PAMI Discrimination-Aware Network Pruning for Deep Model Compression S PyTorch[A]~

2020

Journal Title Type Code
IEEE Trans. NNLS EDP: An Efficient Decomposition and Pruning Scheme for Convolutional Neural Network Compression S
IEEE Access Filter Pruning Without Damaging Networks Capacity S
Electronics Pruning Convolutional Neural Networks with an Attention Mechanism for Remote Sensing Image Classification S

Survey Articles

Year Venue Title
2023 Artif. Intell. Rev. Deep neural network pruning method based on sensitive layers and reinforcement learning
2023 arVix A Survey on Deep Neural Network Pruning: Taxonomy, Comparison, Analysis, and Recommendations
2023 arVix Structured Pruning for Deep Convolutional Neural Networks: A survey
2022 Electronics A Survey on Efficient Convolutional Neural Networks and Hardware Acceleration
2022 I-SMAC A Survey on Filter Pruning Techniques for Optimization of Deep Neural Networks
2021 JMLR Sparsity in Deep Learning: Pruning and growth for efficient inference and training in neural networks
2021 Neurocomputing Pruning and quantization for deep neural network acceleration: A survey
2020 IEEE Access Methods for Pruning Deep Neural Networks

Other Publications

Year Venue Title Code
2023 arVix Why is the State of Neural Network Pruning so Confusing? On the Fairness, Comparison Setup, and Trainability in Network Pruning PyTorch[A](soon...)
2023 arVix Ten Lessons We Have Learned in the New "Sparseland": A Short Handbook for Sparse Neural Network Researchers
2022 ICML Tutorial -- Sparsity in Deep Learning: Pruning and growth for efficient inference and training

Pruning Software and Toolbox

Year Title Type Code
2023 UPop: Unified and Progressive Pruning for Compressing Vision-Language Transformers S PyTorch[A]
2023 DepGraph: Towards Any Structural Pruning S PyTorch[A]
2023 Torch-Pruning S PyTorch[A]
2023 JaxPruner: JaxPruner: A concise library for sparsity research U/S PyTorch[A]
2022 FasterAI: Prune and Distill your models with FastAI and PyTorch U PyTorch[A]
2022 Simplify: A Python library for optimizing pruned neural networks PyTorch[A]
2021 PyTorchViz [A small package to create visualizations of PyTorch execution graphs] PyTorch[A]
2020 What is the State of Neural Network Pruning? S/U PyTorch[A]
2019 Official PyTorch Pruning Tool S/U PyTorch[A]