Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix bug in model export with new action output nodes #4705

Merged

Conversation

dongruoping
Copy link
Contributor

Proposed change(s)

Force the new action output and deprecated action output to be different nodes in the graph

Useful links (Github issues, JIRA tickets, ML-Agents forum threads etc.)

The deprecated action_out_deprecated is assigned to be the same as continuous or discrete action output according to action spec.

However the new action output and deprecated action output will point to the same tensor and same node in the graph, causing one of them being missing is the graph since one node can only have one name.

Solving this issue by forcing them to be different nodes in the graph.

Also fixing missing nodes in dynamic_axes.

Types of change(s)

  • Bug fix
  • New feature
  • Code refactor
  • Breaking change
  • Documentation update
  • Other (please describe)

Checklist

  • Added tests that prove my fix is effective or that my feature works
  • Updated the changelog (if applicable)
  • Updated the documentation (if applicable)
  • Updated the migration guide (if applicable)

Other comments

Copy link
Contributor

@vincentpierre vincentpierre left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Left a comment : rename the discrete output node before merging 🚢 🇮🇹

if self.policy.behavior_spec.action_spec.discrete_size > 0:
self.output_names += ["discrete_actions", "discrete_action_output_shape"]
self.dynamic_axes.update({"discrete_actions": {0: "batch"}})
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since the output will actually be discrete_actions_log_probs instead of discrete_actions, we need to rename this output.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this will have to keep the same name for backward compat concern(old C# + new python), and we give the actual discrete action a new name like sampled_discrete_actions

@dongruoping dongruoping merged commit fa3e093 into develop-hybrid-actions-csharp Dec 4, 2020
@delete-merged-branch delete-merged-branch bot deleted the develop-hybrid-fix-export branch December 4, 2020 18:30
dongruoping added a commit that referenced this pull request Dec 4, 2020
* Add hybrid action capability flag (#4576)

* Change BrainParametersProto to support ActionSpec (#4579)

* Assign new BrainParametersProto fields based on capabilities (#4581)

* ActionBuffer with hybrid actions for RemotePolicy (#4592)

* Barracuda inference for hybrid actions (#4611)

* Refactor BarracudaModel loader checks (#4629)

* Export separate nodes for continuous/discrete actions (#4655)

* Separate continuous/discrete actions in AgentActionProto (#4698)

* Force different nodes for new and deprecated action output (#4705)
@github-actions github-actions bot locked as resolved and limited conversation to collaborators Dec 4, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants