Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

【PaddlePaddle Hackathon 2】47、在 Paddle 中新增 LARC #40285

Closed
TCChenlong opened this issue Mar 8, 2022 · 2 comments
Closed

【PaddlePaddle Hackathon 2】47、在 Paddle 中新增 LARC #40285

TCChenlong opened this issue Mar 8, 2022 · 2 comments

Comments

@TCChenlong
Copy link
Contributor

TCChenlong commented Mar 8, 2022

(此 ISSUE 为 PaddlePaddle Hackathon 第二期活动的任务 ISSUE,更多详见 【PaddlePaddle Hackathon 第二期】任务总览

【任务说明】

  • 任务标题:在 Paddle 中新增 LARC
  • 技术标签:深度学习框架,C++,优化算法
  • 任务难度:中等
  • 详细描述: LARC 优化器是 OpenSeq2Seq 框架中提出的一种优化器。相对于每层都使用相同学习率的 SGD 优化器,该 LARC 优化器考虑到在某些场景下,较大的学习率带来的大幅度参数更新,会导致训练出现分化。为使得训练更稳定,LARC 优化器引入了 trust 系数用来裁剪每一层的学习率。此任务的目标是在 Paddle 框架中,在已实现的 SGD 算法基础上,结合 OpenSeq2Seq 给出的场景,复现 LARC 优化器相对于 SGD 优化器的优势。调用路径:paddle.optimizer.LARC。

【提交内容】

【参考内容】

【技术要求】

  • 了解 Paddle 动静态图下 Optimize pass过程
  • 熟练掌握 C++ 、Python
  • 熟悉 LARC 优化器算法原理和适用场景

【答疑交流】

  • 如果在开发中对于上述任务有任何问题,欢迎在本 ISSUE 下留言交流。
  • 对于开发中的共性问题,在活动过程中,会定期组织答疑,请大家关注官网&QQ群的通知,及时参与。
@paddle-bot-old
Copy link

paddle-bot-old bot commented Mar 8, 2022

您好,我们已经收到了您的问题,会安排技术人员尽快解答您的问题,请耐心等待。请您再次检查是否提供了清晰的问题描述、复现代码、环境&版本、报错信息等。同时,您也可以通过查看官网API文档常见问题历史IssueAI社区来寻求解答。祝您生活愉快~

Hi! We've received your issue and please be patient to get responded. We will arrange technicians to answer your questions as soon as possible. Please make sure that you have posted enough message to demo your request. You may also check out the APIFAQGithub Issue and AI community to get the answer.Have a nice day!

@paddle-bot
Copy link

paddle-bot bot commented Mar 28, 2023

Since you haven't replied for more than a year, we have closed this issue/pr.
If the problem is not solved or there is a follow-up one, please reopen it at any time and we will continue to follow up.
由于您超过一年未回复,我们将关闭这个issue/pr。
若问题未解决或有后续问题,请随时重新打开,我们会继续跟进。

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants