Code

C# changes for hybrid action spaces#4587

Code

Merged

dongruoping merged 181 commits intodevelop-hybrid-action-stagingUnity-Technologies/ml-agents:develop-hybrid-action-stagingfrom develop-hybrid-actions-csharpUnity-Technologies/ml-agents:develop-hybrid-actions-csharpCopy head branch name to clipboardDec 4, 2020

+1,719-655

Commits

Commits on Aug 25, 2020

• hybrid bheavior spec

  andrewcoh

  committed
  e692362

  Copy full SHA for e692362

Commits on Sep 15, 2020

• Merge branch 'master' into develop-hybrid-actions

  andrewcoh

  committed
  f0daeaf

  Copy full SHA for f0daeaf

Commits on Sep 21, 2020

• ignore commit

  andrewcoh

  committed
  f976cf0

  Copy full SHA for f976cf0

Commits on Sep 22, 2020

• add action_out to dist

  andrewcoh

  committed
  92affe6

  Copy full SHA for 92affe6

Commits on Sep 23, 2020

• add action_out abstract method and docstring

  andrewcoh

  committed
  13d66ce

  Copy full SHA for 13d66ce

• prepare to merge action_out

  andrewcoh

  committed
  9926300

  Copy full SHA for 9926300

• Merge branch 'develop-actions-out' into develop-hybrid-actions

  andrewcoh

  committed
  ca281f7

  Copy full SHA for ca281f7

• rename to exported model output

  andrewcoh

  committed
  65ebcec

  Copy full SHA for 65ebcec

• Merge branch 'develop-actions-out' into develop-hybrid-actions

  andrewcoh

  committed
  45d33af

  Copy full SHA for 45d33af

• separate tensors for disc/cont

  andrewcoh

  committed
  4e49aad

  Copy full SHA for 4e49aad

Commits on Sep 29, 2020

• ppo extended

  andrewcoh

  committed
  a9fb4ed

  Copy full SHA for a9fb4ed

• Add test env for hybrid actions, clean up BehaviorSpec (#4522)

  

  Ervin T

  authored
  a586077

  Copy full SHA for a586077

• Better hybrid actions test env (#4523)

  Show description for 3c85cb3
  

  Ervin T

  authored
  3c85cb3

  Copy full SHA for 3c85cb3

• fixing errors

  andrewcoh

  committed
  bc2f248

  Copy full SHA for bc2f248

• Merge branch 'develop-hybrid-actions' of https://github.com/Unity-Technologies/ml-agents into develop-hybrid-actions

  andrewcoh

  committed
  12f7140

  Copy full SHA for 12f7140

Commits on Oct 1, 2020

• action models

  andrewcoh

  committed
  702a61b

  Copy full SHA for 702a61b

• commenting action pre continuous

  andrewcoh

  committed
  1800c87

  Copy full SHA for 1800c87

Commits on Oct 3, 2020

• trains successfully

  andrewcoh

  committed
  165cdf1

  Copy full SHA for 165cdf1

Commits on Oct 5, 2020

• action model as a singleton

  andrewcoh

  committed
  9f7cdd5

  Copy full SHA for 9f7cdd5

• removed abstract class

  andrewcoh

  committed
  75cef49

  Copy full SHA for 75cef49

Commits on Oct 6, 2020

• use ModuleList

  andrewcoh

  committed
  851e540

  Copy full SHA for 851e540

• simple rl tests pass

  andrewcoh

  committed
  781bc84

  Copy full SHA for 781bc84

Commits on Oct 7, 2020

• rename ActionModel

  andrewcoh

  committed
  fb8e704

  Copy full SHA for fb8e704

• remove old behaviorspec

  andrewcoh

  committed
  9268080

  Copy full SHA for 9268080

Commits on Oct 8, 2020

• remove comments

  andrewcoh

  committed
  0375166

  Copy full SHA for 0375166

Commits on Oct 9, 2020

• tf tests except gail pass

  andrewcoh

  committed
  f213278

  Copy full SHA for f213278

Commits on Oct 16, 2020

• add hybrid action capability flag

  dongruoping

  committed
  74d34aa

  Copy full SHA for 74d34aa

• Add hybrid action capability flag (#4576)

  dongruoping

  authored
  7bd5497

  Copy full SHA for 7bd5497

Commits on Oct 19, 2020

• Merge branch 'master' into develop-hybrid-actions-singleton

  andrewcoh

  committed
  5f0d71a

  Copy full SHA for 5f0d71a

• update simple rl tests

  andrewcoh

  committed
  a761c54

  Copy full SHA for a761c54

• ActionSpec and ActionBuffer (#4578)

  andrewcoh

  authored
  cd5d408

  Copy full SHA for cd5d408

• add separate hybrid test file

  andrewcoh

  committed
  f40d142

  Copy full SHA for f40d142

• Change BrainParametersProto to support ActionSpec (#4579)

  dongruoping

  authored
  b519e73

  Copy full SHA for b519e73

Commits on Oct 20, 2020

• Assign new BrainParametersProto fields based on capabilities (#4581)

  dongruoping

  authored
  7b74e72

  Copy full SHA for 7b74e72

• add ActionSpec; test_simple_rl torch passes

  andrewcoh

  committed
  ba8bdcf

  Copy full SHA for ba8bdcf

• remove uneccesary type from set_actions

  andrewcoh

  committed
  6c93137

  Copy full SHA for 6c93137

• ignoring Instance of 'AbstractContextManager' has no 'enter_context' member (no-member)

  andrewcoh

  committed
  03f7e47

  Copy full SHA for 03f7e47

• fixing tensorflow tests

  andrewcoh

  committed
  87d2049

  Copy full SHA for 87d2049

• use proper spec in environment.py

  andrewcoh

  committed
  a889e8f

  Copy full SHA for a889e8f

• fix tf bc test

  andrewcoh

  committed
  24be76f

  Copy full SHA for 24be76f

• fix mlagents-envs tests

  andrewcoh

  committed
  12fa45a

  Copy full SHA for 12fa45a

Commits on Oct 21, 2020

• [bug-fix] Fix Gym and some Policy tests for ActionSpec (#4590)

  Show description for ff4f3b8
  

  Ervin T

  authored
  ff4f3b8

  Copy full SHA for ff4f3b8

Commits on Oct 22, 2020

• remove *_action_* from function names

  andrewcoh

  committed
  3e807c6

  Copy full SHA for 3e807c6

• make_fake_trajectory/step take ActionSpec arg

  andrewcoh

  committed
  fed5c20

  Copy full SHA for fed5c20

• remove ActionType

  andrewcoh

  committed
  d119c1a

  Copy full SHA for d119c1a

• remove self.action_spec from policy/bc

  andrewcoh

  committed
  5a37dfe

  Copy full SHA for 5a37dfe

• fix action_spec refs

  andrewcoh

  committed
  1e5e440

  Copy full SHA for 1e5e440

• Add __eq__ and __str__ to ActionSpec

  andrewcoh

  committed
  194505e

  Copy full SHA for 194505e

• add static method to create continuous/discrete

  andrewcoh

  committed
  4baaa7a

  Copy full SHA for 4baaa7a

• ActionBuffer with hybrid actions for RemotePolicy (#4592)

  dongruoping

  authored
  c77724c

  Copy full SHA for c77724c

Commits on Oct 23, 2020

• fix recurrent sac test

  andrewcoh

  committed
  60337b8

  Copy full SHA for 60337b8

• fix yamato

  andrewcoh

  committed
  841110a

  Copy full SHA for 841110a

• resolve conflicts

  andrewcoh

  committed
  ebd50b2

  Copy full SHA for ebd50b2

• Merge branch 'master' into develop-action-spec

  andrewcoh

  authored
  2e4dcf2

  Copy full SHA for 2e4dcf2

• fix entropy_sum after merge

  andrewcoh

  committed
  1337d07

  Copy full SHA for 1337d07

• fix yamato

  andrewcoh

  committed
  c05c40e

  Copy full SHA for c05c40e

• moved type and shape checking into ActionSpec

  andrewcoh

  committed
  1b96170

  Copy full SHA for 1b96170

• removed action_spec.size

  andrewcoh

  committed
  c940d41

  Copy full SHA for c940d41

• fix specs in torch util

  andrewcoh

  committed
  b0d9a48

  Copy full SHA for b0d9a48

• fixed tests/ -> single validate_action func

  andrewcoh

  committed
  d2bb5d0

  Copy full SHA for d2bb5d0

• make is_discrete/is_continuous strict

  andrewcoh

  committed
  ad144c3

  Copy full SHA for ad144c3

• add docstrings

  andrewcoh

  committed
  9090821

  Copy full SHA for 9090821

Commits on Oct 24, 2020

• rename make_x to creat_x/remove redundant properties

  andrewcoh

  committed
  f23e395

  Copy full SHA for f23e395

• make validate action private

  andrewcoh

  committed
  785848e

  Copy full SHA for 785848e

• fix advanced vis encoder simple rl

  andrewcoh

  committed
  9af9ee9

  Copy full SHA for 9af9ee9

• fix recurrent/advanced ppo tests

  andrewcoh

  committed
  600d307

  Copy full SHA for 600d307

• fix recurrent sac

  andrewcoh

  committed
  42bdfce

  Copy full SHA for 42bdfce

• reduce visual advanced steps

  andrewcoh

  committed
  754f5b8

  Copy full SHA for 754f5b8

Commits on Oct 25, 2020

• reduce recurrent step/increase batch size

  andrewcoh

  committed
  a8813fc

  Copy full SHA for a8813fc

• add ActionBuffers and utils

  andrewcoh

  committed
  64091cc

  Copy full SHA for 64091cc

Commits on Oct 26, 2020

• reduce steps_per_update recurrent sac

  andrewcoh

  committed
  a8204bd

  Copy full SHA for a8204bd

• fix AgentExperience typing

  andrewcoh

  committed
  b5ca548

  Copy full SHA for b5ca548

• recurrent sac passes locally but fails on CI for inexplicable reasons

  andrewcoh

  committed
  ed11b10

  Copy full SHA for ed11b10

• increase seq length

  andrewcoh

  committed
  442f29a

  Copy full SHA for 442f29a

• rename create random to random action

  andrewcoh

  committed
  8733ec1

  Copy full SHA for 8733ec1

• rename create empty to empty action

  andrewcoh

  committed
  199d15b

  Copy full SHA for 199d15b

• Merge branch 'develop-action-spec' into develop-action-buffer

  andrewcoh

  committed
  00a824c

  Copy full SHA for 00a824c

• Merge branch 'master' into develop-action-buffer

  andrewcoh

  committed
  b0ed241

  Copy full SHA for b0ed241

Commits on Oct 27, 2020

• action buffer passes continuous

  andrewcoh

  committed
  bfaa249

  Copy full SHA for bfaa249

• discrete runs/cont passes

  andrewcoh

  committed
  d927497

  Copy full SHA for d927497

• debugging discrete

  andrewcoh

  committed
  0d33e1f

  Copy full SHA for 0d33e1f

• 2d discrete passes

  andrewcoh

  committed
  8f06a67

  Copy full SHA for 8f06a67

Commits on Oct 28, 2020

• sac continuous and discrete train

  andrewcoh

  committed
  da1c85a

  Copy full SHA for da1c85a

Commits on Oct 29, 2020

• bc tests pass

  andrewcoh

  committed
  080f3eb

  Copy full SHA for 080f3eb

• torch reward providers all pass

  andrewcoh

  committed
  f872359

  Copy full SHA for f872359

• fixed bug in discrete

  andrewcoh

  committed
  5886f74

  Copy full SHA for 5886f74

• test_simple_rl/reward providers pass tf/torch

  andrewcoh

  committed
  fe8fdd9

  Copy full SHA for fe8fdd9

Commits on Nov 3, 2020

• ml-agents-envs pass

  andrewcoh

  committed
  9479a65

  Copy full SHA for 9479a65

• Merge branch 'master' into develop-action-buffer

  andrewcoh

  committed
  3a90973

  Copy full SHA for 3a90973

• rename extract to from_dict

  andrewcoh

  committed
  dbf819c

  Copy full SHA for dbf819c

Commits on Nov 4, 2020

• agent processor tests

  andrewcoh

  committed
  d1e2b97

  Copy full SHA for d1e2b97

• fix demo loader tests

  andrewcoh

  committed
  e87effe

  Copy full SHA for e87effe

• test_trajectory fixed

  andrewcoh

  committed
  e0418dc

  Copy full SHA for e0418dc

• fixed recurrent prev_action issue

  andrewcoh

  committed
  5f571a1

  Copy full SHA for 5f571a1

Commits on Nov 5, 2020

• fix test_tf_policy

  andrewcoh

  committed
  9089e63

  Copy full SHA for 9089e63

• fix torch test_ppo

  andrewcoh

  committed
  f8d85fa

  Copy full SHA for f8d85fa

• fix torch utils test

  andrewcoh

  committed
  c21d223

  Copy full SHA for c21d223

• discrete/contionuous unity envs train

  andrewcoh

  committed
  f0f4249

  Copy full SHA for f0f4249

• agent processor tests

  andrewcoh

  committed
  d6eaf8d

  Copy full SHA for d6eaf8d

• fix torch test policy

  andrewcoh

  committed
  e9848b1

  Copy full SHA for e9848b1

• Barracuda inference for hybrid actions (#4611)

  Show description for fbbff02
  dongruoping

  authored
  fbbff02

  Copy full SHA for fbbff02

Commits on Nov 6, 2020

• remove unused import

  andrewcoh

  committed
  b25fc3d

  Copy full SHA for b25fc3d

• add docstrings to AgentAction and ActionLogProbs

  andrewcoh

  committed
  10944f1

  Copy full SHA for 10944f1

• revert demo

  andrewcoh

  committed
  6fcdd3f

  Copy full SHA for 6fcdd3f

• Merge branch 'develop-action-buffer' into develop-hybrid-actions-singleton

  andrewcoh

  committed
  633e3bc

  Copy full SHA for 633e3bc

• 1:1 and continuous/discrete train

  andrewcoh

  committed
  275eef4

  Copy full SHA for 275eef4

• all hybrid simple rl tests pass

  andrewcoh

  committed
  e173f29

  Copy full SHA for e173f29

• move action model to explicit distributions

  andrewcoh

  committed
  8328f16

  Copy full SHA for 8328f16

• remove unused code

  andrewcoh

  committed
  df46ec9

  Copy full SHA for df46ec9

• fix simple test env

  andrewcoh

  committed
  b829957

  Copy full SHA for b829957

• Remove print from ppo tf opti

  Show description for 6d4738b
  

  andrewcoh

  and

  vincentpierre

  authored
  6d4738b

  Copy full SHA for 6d4738b

• bc works

  andrewcoh

  committed
  a0f3dbe

  Copy full SHA for a0f3dbe

Commits on Nov 7, 2020

• torch curiosity tests pass

  andrewcoh

  committed
  8d10587

  Copy full SHA for 8d10587

• remove print bc

  andrewcoh

  committed
  c3f69e6

  Copy full SHA for c3f69e6

Commits on Nov 9, 2020

• rename to ActionTuple

  andrewcoh

  committed
  5c8ec2d

  Copy full SHA for 5c8ec2d

• Merge branch 'develop-action-buffer' of https://github.com/Unity-Technologies/ml-agents into develop-action-buffer

  andrewcoh

  committed
  0441118

  Copy full SHA for 0441118

• Update ml-agents/mlagents/trainers/torch/utils.py

  Show description for 86b6d71
  

  andrewcoh

  and

  vincentpierre

  authored
  86b6d71

  Copy full SHA for 86b6d71

• ActionTuple default is now np.array, not None

  andrewcoh

  committed
  2bf004c

  Copy full SHA for 2bf004c

• fix set_actions_for_agent

  andrewcoh

  committed
  aaf6c59

  Copy full SHA for aaf6c59

• fix action mask in trajectory

  andrewcoh

  committed
  056cf6d

  Copy full SHA for 056cf6d

• Merge branch 'develop-action-buffer' into develop-hybrid-actions-singleton

  andrewcoh

  committed
  74ce2b1

  Copy full SHA for 74ce2b1

• Update ml-agents-envs/mlagents_envs/environment.py

  Show description for 5691f60
  

  andrewcoh

  and

  vincentpierre

  authored
  5691f60

  Copy full SHA for 5691f60

• replace use_discrete with action_sizes in simple_rl

  andrewcoh

  committed
  62fcecb

  Copy full SHA for 62fcecb

• revert demo

  andrewcoh

  committed
  b567fcd

  Copy full SHA for b567fcd

• Merge branch 'develop-action-buffer' of https://github.com/Unity-Technologies/ml-agents into develop-action-buffer

  andrewcoh

  committed
  116580a

  Copy full SHA for 116580a

• Refactor BarracudaModel loader checks (#4629)

  Show description for 3528376
  dongruoping

  authored
  3528376

  Copy full SHA for 3528376

Commits on Nov 10, 2020

• fix default random action

  andrewcoh

  committed
  b152511

  Copy full SHA for b152511

• fix reward provider tests

  andrewcoh

  committed
  bb9988c

  Copy full SHA for bb9988c

• add defaults to ActionTuple constructor

  andrewcoh

  committed
  c488e8e

  Copy full SHA for c488e8e

• remove unused line in traj

  andrewcoh

  committed
  589907a

  Copy full SHA for 589907a

• save only discrete actions as prev

  andrewcoh

  committed
  c8ae8da

  Copy full SHA for c8ae8da

• update make_empty docstring

  andrewcoh

  committed
  c651ebc

  Copy full SHA for c651ebc

• reuse action dict in torch policy for pre_action

  andrewcoh

  committed
  0dc4396

  Copy full SHA for 0dc4396

• add back removed part of test_envs

  andrewcoh

  committed
  434f210

  Copy full SHA for 434f210

• fix mock brain prev action

  andrewcoh

  committed
  714b444

  Copy full SHA for 714b444

Commits on Nov 12, 2020

• default ActionTuple to None

  andrewcoh

  committed
  65d17fe

  Copy full SHA for 65d17fe

• Merge branch 'develop-action-buffer' into develop-hybrid-actions-singleton

  andrewcoh

  committed
  032b7ba

  Copy full SHA for 032b7ba

• default actions are np.array of shape (n_agents, 0)

  andrewcoh

  committed
  4fc60d5

  Copy full SHA for 4fc60d5

• preliminary aciton model tests

  andrewcoh

  committed
  99c2ebc

  Copy full SHA for 99c2ebc

• move AgentAction, ActionLogProbs, and ActionFlattener to separate files

  andrewcoh

  committed
  2bbab9b

  Copy full SHA for 2bbab9b

• add action util files

  andrewcoh

  committed
  6e20e1d

  Copy full SHA for 6e20e1d

• action model and network tests

  andrewcoh

  committed
  7868abf

  Copy full SHA for 7868abf

• Update ml-agents/mlagents/trainers/policy/tf_policy.py

  Show description for a8676fe
  

  andrewcoh

  and

  vincentpierre

  authored
  a8676fe

  Copy full SHA for a8676fe

• fix torch policy tests

  andrewcoh

  committed
  24809e1

  Copy full SHA for 24809e1

• remove actionspec logic simple test env

  andrewcoh

  committed
  45c92cb

  Copy full SHA for 45c92cb

Commits on Nov 13, 2020

• Merge branch 'develop-hybrid-actions-singleton' into develop-hybrid-actions-csharp

  dongruoping

  committed
  63f72df

  Copy full SHA for 63f72df

• check tensor exist before access shape

  dongruoping

  committed
  91afdf8

  Copy full SHA for 91afdf8

Commits on Nov 16, 2020

• remove action spec attribute from policy

  andrewcoh

  committed
  3f1b8d9

  Copy full SHA for 3f1b8d9

• added docstrings to action flattener

  andrewcoh

  committed
  96d0a48

  Copy full SHA for 96d0a48

• added docstrings to action model

  andrewcoh

  committed
  e0ffd3e

  Copy full SHA for e0ffd3e

• Policy output actiontuple (#4651)

  andrewcoh

  authored
  e49f68b

  Copy full SHA for e49f68b

• Merge branch 'develop-action-buffer' into develop-hybrid-actions-singleton

  andrewcoh

  committed
  f25ba54

  Copy full SHA for f25ba54

• remove unused imports test_hybrid

  andrewcoh

  committed
  b712195

  Copy full SHA for b712195

• fix imports in test utils

  andrewcoh

  committed
  41a5a29

  Copy full SHA for 41a5a29

• Merge branch 'develop-hybrid-action-staging' into develop-hybrid-actions-singleton

  andrewcoh

  committed
  793957c

  Copy full SHA for 793957c

• fix subprocess_env_manager check of action info length

  andrewcoh

  committed
  22f9ee7

  Copy full SHA for 22f9ee7

Commits on Nov 17, 2020

• Merge branch 'develop-hybrid-action-staging' into develop-hybrid-actions-singleton

  andrewcoh

  committed
  ca3b492

  Copy full SHA for ca3b492

• resolve all conflicts with staging

  andrewcoh

  committed
  23d499c

  Copy full SHA for 23d499c

• fix test agent processor

  andrewcoh

  committed
  60b6c91

  Copy full SHA for 60b6c91

• fix test trajectory

  andrewcoh

  committed
  a3321c1

  Copy full SHA for a3321c1

• fix test torch distributions

  andrewcoh

  committed
  2601279

  Copy full SHA for 2601279

• fix torch test policy

  andrewcoh

  committed
  05910ef

  Copy full SHA for 05910ef

• fix torch test ppo

  andrewcoh

  committed
  58299fb

  Copy full SHA for 58299fb

• fix test tf policy

  andrewcoh

  committed
  706ddd8

  Copy full SHA for 706ddd8

• Export separate nodes for continuous/discrete actions (#4655)

  dongruoping

  authored
  533f985

  Copy full SHA for 533f985

• fix tensorflow test simple rl

  andrewcoh

  committed
  9a45d4f

  Copy full SHA for 9a45d4f

• Merge branch 'develop-hybrid-actions-singleton' into develop-hybrid-actions-csharp

  dongruoping

  committed
  1eda523

  Copy full SHA for 1eda523

• fix policy action_spec typo

  dongruoping

  committed
  aae5420

  Copy full SHA for aae5420

Commits on Nov 18, 2020

• change method name and formatting

  dongruoping

  committed
  7a10f0b

  Copy full SHA for 7a10f0b

Commits on Dec 3, 2020

• Separate continuous/discrete actions in AgentActionProto (#4698)

  Show description for 6ec6635
  dongruoping

  authored
  6ec6635

  Copy full SHA for 6ec6635

• Merge branch 'develop-hybrid-action-staging' into develop-hybrid-actions-csharp

  dongruoping

  committed
  1afd756

  Copy full SHA for 1afd756

• fix merge

  dongruoping

  committed
  ea6393b

  Copy full SHA for ea6393b

Commits on Dec 4, 2020

• Merge branch 'develop-hybrid-action-staging' into develop-hybrid-actions-csharp

  dongruoping

  committed
  0d3e10d

  Copy full SHA for 0d3e10d

• Force different nodes for new and deprecated action output (#4705)

  Show description for fa3e093
  dongruoping

  authored
  fa3e093

  Copy full SHA for fa3e093

• read deprecated fields in BrainParmeterProto if ActionSpec not set

  dongruoping

  committed
  a3a28ee

  Copy full SHA for a3a28ee

• fix bc tests

  dongruoping

  committed
  f8f4615

  Copy full SHA for f8f4615

• Merge branch 'develop-hybrid-actions-csharp' of https://github.com/Unity-Technologies/ml-agents into develop-hybrid-actions-csharp

  dongruoping

  committed
  4b3ef56

  Copy full SHA for 4b3ef56

• remove yamato trigger

  dongruoping

  committed
  776134c

  Copy full SHA for 776134c

• fix tests

  dongruoping

  committed
  74e6e22

  Copy full SHA for 74e6e22

• fix tests

  dongruoping

  committed
  0f315ea

  Copy full SHA for 0f315ea

• fix torch bc tests

  dongruoping

  committed
  7468dfa

  Copy full SHA for 7468dfa