[gym] Add symmetry surrogate loss in rllib custom ppo #780
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
I'm in favour of changing the name of the option as you suggested. It would be nice if you could update to documentation to explain what it does and give the source of the article introducing this approach. Thank you ! |
|
I've renamed the loss to |
|
If you are motivated, then it would be nice yes ! You could even add mine in addition that is giving more insights regarding the interesting properties that emerges and how to tune the parameters. |
mwulfman
force-pushed
the
add-mirror-surrogate
branch
from
eb6f7e4 to
fccd03c
Compare
|
I have updated the documentation to add the mentioned articles. |
mwulfman
changed the title
Merged
duburcqa
added a commit
that referenced
this pull request
May 24, 2024
* [core] Make 'jiminy::Model' and 'jiminy::Robot' copyable. (#791) * [core] Add stairs terrain (#797) * [core] Define position, velocity and effort limits at motor level. (#798) * [core] Implement velocity limits at motor-level. (#798) * [core] Allow direct access to all motors / sensors attached to a robot. (#798) * [core] Remove spring-damper position bounds dynamics to always use constraints. (#798) * [core] Remove damper velocity bounds dynamics. (#798) * [core] Define motor armature option at motor-side. (#798) * [core] Remove dedicated robot attributes for position, velocity and effort limits. (#798) * [core] Remove inefficient motor / sensor convenience helpers of 'Robot'. (#798) * [core] Log total joint effort instead of confusing motor effort on joint side. (#798) * [core] Enable attaching encoders either on motor or joint side. (#798) * [core] Mechanical reduction is now fully supported. (#798) * [core] Fix adding flexibility frame on root joint of fixed-base robot. (#798) * [core] Move 'enable(Velocity|Effort)Limit' options from AbstractMotor to SimpleMotor. (#798) * [core] Add velocity-torque slope when approaching maximum torque. (#798) * [core] Keep track of motor efforts both on motor and joint sides. (#801) * [core] Effort sensors now measure motor effort before transmission. (#801) * [core|python] Various cleanup. (#790) * [python] Introduce first-class trajectory object. (#790) * [python/log] Fix trajectory and sensor data extraction and replay from log. (#800) * [python/plot] Get around tabbed figure hanging for 'matplotlib>=3.8'. (#795) * [python/viewer] Add missing documentation. (#790) * [python/replay] Make specifying 'robot' optional in most helpers. (#800) * [gym/common] Add generic stacked quantity wrapper. (#783) * [gym/common] Do NOT pass 'truncated' argument to 'compute_reward'. (#783) * [gym/common] More generic quantity batching. (#783) * [gym/common] Pass 'info' in argument of 'has_terminated' for consistency. (#783) * [gym/common] Remove BaseJiminyEnv 'enforce_bounded_spaces' option by lack of use-case. (#783) * [gym/common] Add masked quantity. (#784) * [gym/common] Add average odometry velocity quantity. (#784) * [gym/common] Introduce reward objects. (#784) * [gym/common] Add locomotion rewards. (#784) * [gym/common] Add base reward mixture class to ease pipeline instantiation. (#786) * [gym/common] Add basic survive reward. (#786) * [gym/common] Increase timeout ratio from 10 to 15 to avoid Mac OS CI failure. (#786) * [gym/common] Add compose reward pipeline wrapper. (#787) * [gym/common] Support specifying reward in pipeline config. (#787) * [gym/common] Add simulation vs trajectory quantity evaluation mode. (#791) * [gym/common] Expose trajectory database in quantity manager. (#792) * [gym/common] Expose interpolate mode for trajectories. (#792) * [gym/common] Add support of trajectory reference in reward computation. (#792) * [gym/common] Enable specifying reference trajectories in pipeline config. (#792) * [gym/common] Get around some memory alignment issue with Boost < 1.78. (#794) * [gym/zoo] Update Atlas learning environment. (#800) * [gym/zoo] Update all robotic environments so that random actions do not trigger truncation. (#801) * [gym/rllib] Add symmetry surrogate loss in rllib custom ppo (#780) * [gym/rllib] Fix checkpoint save location. (#788) * [gym/rllib] Update checkpoint_dir so it is dependant on iteration number (#789) * [misc] Update install instructions on windows. (#787)
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR adds the possibility to add the surrogate loss on the mirrored observation-action training data. Naming can be changed if desired as we could name it also "symmetry_surrogate_loss" for instance.