Develop hybrid action staging #4702
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Co-authored-by: Ervin T <ervin@unity3d.com> Co-authored-by: Vincent-Pierre BERGES <vincentpierre@unity3d.com>
Co-authored-by: Ervin T <ervin@unity3d.com> Co-authored-by: Vincent-Pierre BERGES <vincentpierre@unity3d.com>
* Add hybrid action capability flag (#4576) * Change BrainParametersProto to support ActionSpec (#4579) * Assign new BrainParametersProto fields based on capabilities (#4581) * ActionBuffer with hybrid actions for RemotePolicy (#4592) * Barracuda inference for hybrid actions (#4611) * Refactor BarracudaModel loader checks (#4629) * Export separate nodes for continuous/discrete actions (#4655) * Separate continuous/discrete actions in AgentActionProto (#4698) * Force different nodes for new and deprecated action output (#4705)
* clean up docs on actions, code examples, reference to hybrid * undo change from other branch * Update docs/Python-API.md Co-authored-by: andrewcoh <54679309+andrewcoh@users.noreply.github.com> * Update docs/Python-API.md Co-authored-by: andrewcoh <54679309+andrewcoh@users.noreply.github.com> * Update docs/Python-API.md Co-authored-by: andrewcoh <54679309+andrewcoh@users.noreply.github.com> * Update docs/Python-API.md Co-authored-by: andrewcoh <54679309+andrewcoh@users.noreply.github.com> * cleanup Co-authored-by: andrewcoh <54679309+andrewcoh@users.noreply.github.com>
andrewcoh
requested review from
ervteng,
vincentpierre,
chriselion and
dongruoping
chriselion
reviewed
Dec 9, 2020
| has_nan = np.isnan(d) | ||
| if has_nan: | ||
| raise RuntimeError("Continuous NaN action detected.") | ||
| d = np.sum(action.discrete) |
There was a problem hiding this comment.
action.discrete is int32's right? I don't think we need to check them for NaNs
There was a problem hiding this comment.
(OK to do after merging to master)
chriselion
approved these changes
Dec 9, 2020
ervteng
reviewed
Dec 9, 2020
| - Actions can either use `Discrete` or `Continuous` spaces. | ||
| - When using `Discrete` it is possible to assign multiple action branches, and | ||
| to mask certain actions. | ||
| - Agents can either use `Discrete` or `Continuous` actions. |
There was a problem hiding this comment.
Suggested change
| - Agents can either use `Discrete` or `Continuous` actions. | |
| - Agents can use `Discrete` and/or `Continuous` actions. |
ervteng
reviewed
Dec 9, 2020
| action. For example : In a game direction input (no movement, left, right) and | ||
| - `action_spec` is an `ActionSpec` namedtuple that defines the number and types | ||
| of actions for the Agent. | ||
|
|
There was a problem hiding this comment.
Suggested change
| #### ActionSpec |
dongruoping
approved these changes
Dec 9, 2020
I'll update in a subsequent PR. |
10 tasks
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Proposed change(s)
Hybrid Actions staging branch
To merge in before it is ready to merge into master:
ActionTupleActionModelC#TODO:
Merge masterAddressing clipped actions was non-trivial. Please look at how ActionInfo and clipping in the torch policy has changed.Fix GAIL simple RL (after C#)(GAIL tests passed but still using demonstrations with old proto fields)DocsUseful links (Github issues, JIRA tickets, ML-Agents forum threads etc.)
Types of change(s)
Checklist
Other comments