New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Scaling SGD Batch Size to 32K for ImageNet Training #15

Open

nocotan opened this issue Jan 4, 2021 · 0 comments

Assignees

Labels

Member

nocotan commented Jan 4, 2021

一言でいうと

大規模バッチ学習のためのLayer-wise Adaptive Rate Scaling (LARS)を提案．

論文リンク

https://digitalassets.lib.berkeley.edu/techreports/ucb/text/EECS-2017-156.pdf

著者/所属機関

Yang You, Igor Gitman, Boris Ginsburg (UC Berkeley)

投稿日付(yyyy/MM/dd)

2017/09/16

概要

新規性・差分

異なるレイヤーで異なる学習率を適用する初の手法．

手法

結果

コメント

nocotan self-assigned this

nocotan added the deep learning label

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment