Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore: uniform sac network sizes #145

Merged
merged 3 commits into from
Mar 29, 2023
Merged

Conversation

limbryan
Copy link
Collaborator

@limbryan limbryan commented Mar 17, 2023

This PR is to try to make the Deep RL algorithms more uniform. It allows the SAC, DIAYN and DADS implementation to have separate policy (actor) and critic architectures, as is already the case in the TD3 implementation. This allows more flexibility and is needed sometimes for comparison with QD algorithms which do not necessarily have a critic which needs a bigger size while the actor can be smaller.

This PR changes:

  • the hidden_layer_sizes to policy_hidden_layer_size and critic_hidden_layer_size for SAC, DIAYN and DADS

Checks

  • a clear description of the PR has been added
  • sufficient tests have been written
  • relevant section added to the documentation (N/A)
  • example notebook added to the repo (N/A)
  • clean docstrings and comments have been written (N/A)
  • if any issue/observation has been discovered, a new issue has been opened (N/A)

Future improvements

for DADS and DIAYN there is also the discriminator or dynamics architecture to expose. For now it is set to the critic architecture

@limbryan limbryan changed the base branch from main to develop March 17, 2023 20:33
@codecov-commenter
Copy link

codecov-commenter commented Mar 17, 2023

Codecov Report

Merging #145 (cad8b1f) into develop (6fa19e7) will increase coverage by 0.01%.
The diff coverage is 100.00%.

📣 This organization is not using Codecov’s GitHub App Integration. We recommend you install it so Codecov can continue to function properly for your repositories. Learn more

@@             Coverage Diff             @@
##           develop     #145      +/-   ##
===========================================
+ Coverage    92.28%   92.29%   +0.01%     
===========================================
  Files          116      116              
  Lines         6763     6772       +9     
===========================================
+ Hits          6241     6250       +9     
  Misses         522      522              
Impacted Files Coverage Δ
qdax/baselines/dads.py 97.00% <ø> (ø)
qdax/baselines/diayn.py 93.07% <ø> (ø)
qdax/core/neuroevolution/networks/dads_networks.py 94.02% <ø> (ø)
...dax/core/neuroevolution/networks/diayn_networks.py 100.00% <ø> (ø)
qdax/core/neuroevolution/networks/sac_networks.py 100.00% <ø> (ø)
qdax/baselines/sac.py 94.37% <100.00%> (+0.03%) ⬆️
qdax/baselines/sac_pbt.py 96.49% <100.00%> (+0.03%) ⬆️
tests/baselines_test/dads_smerl_test.py 97.18% <100.00%> (+0.04%) ⬆️
tests/baselines_test/dads_test.py 96.96% <100.00%> (+0.04%) ⬆️
tests/baselines_test/diayn_smerl_test.py 97.01% <100.00%> (+0.04%) ⬆️
... and 4 more

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

Copy link
Collaborator

@felixchalumeau felixchalumeau left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Notebooks might need to be updated as well.

Copy link
Collaborator

@felixchalumeau felixchalumeau left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good 👍

@limbryan limbryan merged commit 6f78a4a into develop Mar 29, 2023
@limbryan limbryan deleted the chore/uniform_sac_network_sizes branch March 30, 2023 01:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants