Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Soft Actor Critic (SAC) Model #627

Merged
merged 43 commits into from
Sep 8, 2021
Merged

Soft Actor Critic (SAC) Model #627

merged 43 commits into from
Sep 8, 2021

Conversation

blahBlahhhJ
Copy link
Contributor

@blahBlahhhJ blahBlahhhJ commented May 1, 2021

What does this PR do?

Update for #596 (issue)
Implementation of SAC model for Reinforcement Learning
pendulum_sac_results

Before submitting

  • Was this discussed/approved via a Github issue? (no need for typos and docs improvements)
  • Did you read the contributor guideline, Pull Request section?
  • Did you make sure your PR does only one thing, instead of bundling different changes together?
  • Did you make sure to update the documentation with your changes?
  • Did you write any new necessary tests? [not needed for typos/docs]
  • Did you verify new and existing tests pass locally with your changes?
  • If you made a notable change (that affects users), did you update the CHANGELOG?

PR review

  • Is this pull request ready for review? (if not, please submit in draft mode)

Anyone in the community is free to review the PR once the tests have passed.
If we didn't discuss your PR in Github issues there's a high chance it will not be merged.

Did you have fun?

Make sure you had fun coding 🙃

@pep8speaks
Copy link

pep8speaks commented May 1, 2021

Hello @blahBlahhhJ! Thanks for updating this PR.

Line 162:37: W503 line break before binary operator

Comment last updated at 2021-08-13 12:42:11 UTC

@github-actions github-actions bot added documentation Improvements or additions to documentation model labels May 1, 2021
@blahBlahhhJ blahBlahhhJ marked this pull request as draft May 1, 2021 03:55
@blahBlahhhJ blahBlahhhJ marked this pull request as ready for review May 1, 2021 04:02
@blahBlahhhJ
Copy link
Contributor Author

@akihironitta
Hi! Another RL pull request! This is for Soft Actor Critic which supports continuous action space and off-policy learning. Could you review and give me some feedbacks? About the PEP8 error, I tried both before and after and they lead to different errors so I don't know what to do.
Thanks in advance!

@codecov
Copy link

codecov bot commented May 5, 2021

Codecov Report

Merging #627 (4486569) into master (4046278) will decrease coverage by 49.26%.
The diff coverage is 0.00%.

❗ Current head 4486569 differs from pull request most recent head 427d5ab. Consider uploading reports for the commit 427d5ab to get more accurate results
Impacted file tree graph

@@             Coverage Diff             @@
##           master     #627       +/-   ##
===========================================
- Coverage   72.82%   23.55%   -49.27%     
===========================================
  Files         121      123        +2     
  Lines        7550     7782      +232     
===========================================
- Hits         5498     1833     -3665     
- Misses       2052     5949     +3897     
Flag Coverage Δ
cpu 23.55% <0.00%> (-49.27%) ⬇️
pytest 23.55% <0.00%> (-49.27%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
pl_bolts/models/rl/__init__.py 0.00% <0.00%> (-100.00%) ⬇️
pl_bolts/models/rl/common/agents.py 0.00% <0.00%> (-100.00%) ⬇️
pl_bolts/models/rl/common/distributions.py 0.00% <0.00%> (ø)
pl_bolts/models/rl/common/networks.py 0.00% <0.00%> (-92.31%) ⬇️
pl_bolts/models/rl/sac_model.py 0.00% <0.00%> (ø)
pl_bolts/models/rl/dueling_dqn_model.py 0.00% <0.00%> (-100.00%) ⬇️
pl_bolts/models/rl/advantage_actor_critic_model.py 0.00% <0.00%> (-97.71%) ⬇️
pl_bolts/models/rl/double_dqn_model.py 0.00% <0.00%> (-95.84%) ⬇️
...l_bolts/models/rl/vanilla_policy_gradient_model.py 0.00% <0.00%> (-93.45%) ⬇️
... and 74 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update fa7d1df...427d5ab. Read the comment docs.

Copy link
Member

@Borda Borda left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

add some more typing if possible

pl_bolts/models/rl/common/distributions.py Outdated Show resolved Hide resolved
pl_bolts/models/rl/sac_model.py Outdated Show resolved Hide resolved
tests/models/rl/unit/test_sac.py Outdated Show resolved Hide resolved
@Borda Borda added the ready label Jun 24, 2021
Copy link
Member

@Borda Borda left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

pls add chlog note

@Borda
Copy link
Member

Borda commented Aug 13, 2021

Thank you @blahBlahhhJ for this nice PR
I have just merged the #598 which brought some collision, could you pls chek them

@mergify mergify bot removed the has conflicts label Aug 13, 2021
@blahBlahhhJ
Copy link
Contributor Author

Thank you @blahBlahhhJ for this nice PR
I have just merged the #598 which brought some collision, could you pls chek them

@Borda Hi, thanks for merging the other PR! I've fixed all the conflict.

@Borda
Copy link
Member

Borda commented Aug 25, 2021

Hi, thanks for merging the other PR! I've fixed all the conflict.

@blahBlahhhJ did you have a chance to check the conflicts?

@blahBlahhhJ
Copy link
Contributor Author

Hi, thanks for merging the other PR! I've fixed all the conflict.

@blahBlahhhJ did you have a chance to check the conflicts?

hi I think there’s no more conflicts. Fixed them days ago.

@mergify mergify bot removed the has conflicts label Sep 7, 2021
@SeanNaren SeanNaren enabled auto-merge (squash) September 7, 2021 22:23
@SeanNaren SeanNaren merged commit f6a7e98 into Lightning-Universe:master Sep 8, 2021
@SeanNaren
Copy link
Contributor

Thanks @blahBlahhhJ for your patience on this one, bolts has been under maintained so apologies on the time taken to get this merged :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
documentation Improvements or additions to documentation model waiting on author
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants