Readme.md

⏱️ AutoML方向

入门指南

注:⚡为基础必读,💎为基础选读,💡为进阶阅读

AutoML 与超参数优化

AutoML综述

• ⚡ 📄 AutoML: A Survey of the State-of-the-Art

贝叶斯优化基础

先通过阅读专栏了解贝叶斯优化和高斯过程的基本原理，然后阅读两篇论文了解SMBO的经典方法

• ⚡ 📄 (BO & GP)
• ⚡ 📄 (SMAC) Sequential Model-Based Optimization for General Algorithm Configuration (extended version)
• ⚡ 📄 (TPE) Algorithms for Hyper-Parameter Optimization

黑盒优化系统

通过openbox自己搭建SMBO的基本流程 阅读openbox源代码，了解其内部实现

• ⚡ 🛠️ Openbox

多精度优化

• ⚡ 📄 Hyperband: A Novel Bandit-Based Approach to Hyperparameter Optimization
• ⚡ 📄 BOHB: Robust and Efficient Hyperparameter Optimization at Scale

多精度优化

• 💎 📄 DEHB: Evolutionary Hyperband for Scalable, Robust and Efficient Hyperparameter Optimization
• 💎 📄 Efficient Automatic CASH via Rising Bandits
• 💎 📄 (DyHPO) Supervising the Multi-Fidelity Race of Hyperparameter Configurations

AutoML系统

• ⚡ 📄 Auto-WEKA: combined selection and hyperparameter optimization of classification algorithms
• ⚡ 📄 (auto-sklearn) Efficient and Robust Automated Machine Learning

AutoML系统

通过开源autoML系统自行搭建自动化机器学习的完整流程，并了解其内部实现

• ⚡ 🛠️ auto-sklearn: https://github.com/automl/auto-sklearn
• ⚡ 🛠️ MindWare: https://github.com/thomas-young-2013/mindware

迁移学习综述

• ⚡ 📄 Transfer Learning for Bayesian Optimization: A Survey

迁移学习

• ⚡ 📄 (RGPE) Scalable Meta-Learning for Bayesian Optimization using Ranking-Weighted Gaussian Process Ensembles

• ⚡ 📄 (TAF) Scalable Gaussian process-based transfer surrogates for hyperparameter optimization

• 💎 📄 Learning Hyperparameter Optimization Initializations

• 💎 📄 Hyperparameter Search Space Pruning – A NewComponent for Sequential Model-Based Hyperparameter Optimization

• 💎 📄 (POGOE) Scalable Hyperparameter Optimization with Products of Gaussian Process Experts

• 💎 📄 (TST) Two-Stage Transfer Surrogate Model for Automatic Hyperparameter Optimization

• 💎 📄 (TransBO) TransBO: Hyperparameter Optimization via Two-Phase Transfer Learning

网络结构搜索(NAS)

NAS综述

• ⚡ 📄 Neural Architecture Search: A Survey

NAS基础

• ⚡ 📄 Neural Architecture Search with Reinforcement Learning
• ⚡ 📄 Learning Transferable Architectures for Scalable Image Recognition

One-Shot NAS

• ⚡ 📄 DARTS: Differentiable Architecture Search

One-Shot NAS

• 💎 📄 (DARTS-PT) Rethinking Architecture Selection in Differentiable NAS
• 💎 📄 β-DARTS: Beta-Decay Regularization for Differentiable Architecture Search

Zero-Shot Proxy

• ⚡ 📄 Accelerating Neural Architecture Search via Proxy Data
• ⚡ 📄 Zero-Cost Operation Scoring in Differentiable Architecture Search
• ⚡ 📄 ProxyBO: Accelerating Neural Architecture Search via Bayesian Optimization with Zero-Cost Proxies

模型压缩

模型压缩综述

对模型压缩感兴趣的同学可以在这一部分基础上自己查询相关论文阅读

• 💎 📄 Model Compression for Deep Neural Networks: A Survey

模型剪枝基础

• 💎 📄 Pruning Filters for Efficient ConvNets
• 💎 📄 Learning both weights and connections for efficient neural network

知识蒸馏基础

• 💎 📄 Distilling the Knowledge in a Neural Network
• 💎 📄 Contrastive Representation Distillationhttps://arxiv.org/abs/1910.10699

参数量化基础

• 💎 📄 ZeroQ: A Novel Zero Shot Quantization Framework
• 💎 📄 Post-Training Sparsity-Aware Quantization

大语言模型与AutoML

LLM for AutoML

使用大模型帮助AutoML的流程执行

• 💎 📄 Large Language Models to Enhance Bayesian Optimization
• 💎 📄 Large Language Models for Automated Data Science: Introducing CAAFE for Context-Aware Automated Feature Engineering