Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

C# changes for hybrid action spaces #4587

Merged
Merged
Changes from 1 commit
Commits
Show all changes
181 commits
Select commit Hold shift + click to select a range
e692362
hybrid bheavior spec
andrewcoh Aug 25, 2020
f0daeaf
Merge branch 'master' into develop-hybrid-actions
andrewcoh Sep 15, 2020
f976cf0
ignore commit
andrewcoh Sep 21, 2020
92affe6
add action_out to dist
andrewcoh Sep 22, 2020
13d66ce
add action_out abstract method and docstring
andrewcoh Sep 23, 2020
9926300
prepare to merge action_out
andrewcoh Sep 23, 2020
ca281f7
Merge branch 'develop-actions-out' into develop-hybrid-actions
andrewcoh Sep 23, 2020
65ebcec
rename to exported model output
andrewcoh Sep 23, 2020
45d33af
Merge branch 'develop-actions-out' into develop-hybrid-actions
andrewcoh Sep 23, 2020
4e49aad
separate tensors for disc/cont
andrewcoh Sep 23, 2020
a9fb4ed
ppo extended
andrewcoh Sep 29, 2020
a586077
Add test env for hybrid actions, clean up BehaviorSpec (#4522)
Sep 29, 2020
3c85cb3
Better hybrid actions test env (#4523)
Sep 29, 2020
bc2f248
fixing errors
andrewcoh Sep 29, 2020
12f7140
Merge branch 'develop-hybrid-actions' of https://github.com/Unity-Tec…
andrewcoh Sep 29, 2020
702a61b
action models
andrewcoh Oct 1, 2020
1800c87
commenting action pre continuous
andrewcoh Oct 1, 2020
165cdf1
trains successfully
andrewcoh Oct 3, 2020
9f7cdd5
action model as a singleton
andrewcoh Oct 5, 2020
75cef49
removed abstract class
andrewcoh Oct 5, 2020
851e540
use ModuleList
andrewcoh Oct 6, 2020
781bc84
simple rl tests pass
andrewcoh Oct 6, 2020
fb8e704
rename ActionModel
andrewcoh Oct 7, 2020
9268080
remove old behaviorspec
andrewcoh Oct 7, 2020
0375166
remove comments
andrewcoh Oct 8, 2020
f213278
tf tests except gail pass
andrewcoh Oct 9, 2020
74d34aa
add hybrid action capability flag
dongruoping Oct 16, 2020
7bd5497
Add hybrid action capability flag (#4576)
dongruoping Oct 16, 2020
5f0d71a
Merge branch 'master' into develop-hybrid-actions-singleton
andrewcoh Oct 19, 2020
a761c54
update simple rl tests
andrewcoh Oct 19, 2020
cd5d408
ActionSpec and ActionBuffer (#4578)
andrewcoh Oct 19, 2020
f40d142
add separate hybrid test file
andrewcoh Oct 19, 2020
b519e73
Change BrainParametersProto to support ActionSpec (#4579)
dongruoping Oct 19, 2020
7b74e72
Assign new BrainParametersProto fields based on capabilities (#4581)
dongruoping Oct 20, 2020
ba8bdcf
add ActionSpec; test_simple_rl torch passes
andrewcoh Oct 20, 2020
6c93137
remove uneccesary type from set_actions
andrewcoh Oct 20, 2020
03f7e47
ignoring Instance of 'AbstractContextManager' has no 'enter_context' …
andrewcoh Oct 20, 2020
87d2049
fixing tensorflow tests
andrewcoh Oct 20, 2020
a889e8f
use proper spec in environment.py
andrewcoh Oct 20, 2020
24be76f
fix tf bc test
andrewcoh Oct 20, 2020
12fa45a
fix mlagents-envs tests
andrewcoh Oct 20, 2020
ff4f3b8
[bug-fix] Fix Gym and some Policy tests for ActionSpec (#4590)
Oct 21, 2020
3e807c6
remove *_action_* from function names
andrewcoh Oct 22, 2020
fed5c20
make_fake_trajectory/step take ActionSpec arg
andrewcoh Oct 22, 2020
d119c1a
remove ActionType
andrewcoh Oct 22, 2020
5a37dfe
remove self.action_spec from policy/bc
andrewcoh Oct 22, 2020
1e5e440
fix action_spec refs
andrewcoh Oct 22, 2020
194505e
Add __eq__ and __str__ to ActionSpec
andrewcoh Oct 22, 2020
4baaa7a
add static method to create continuous/discrete
andrewcoh Oct 22, 2020
c77724c
ActionBuffer with hybrid actions for RemotePolicy (#4592)
dongruoping Oct 22, 2020
60337b8
fix recurrent sac test
andrewcoh Oct 23, 2020
841110a
fix yamato
andrewcoh Oct 23, 2020
ebd50b2
resolve conflicts
andrewcoh Oct 23, 2020
2e4dcf2
Merge branch 'master' into develop-action-spec
andrewcoh Oct 23, 2020
1337d07
fix entropy_sum after merge
andrewcoh Oct 23, 2020
c05c40e
fix yamato
andrewcoh Oct 23, 2020
1b96170
moved type and shape checking into ActionSpec
andrewcoh Oct 23, 2020
c940d41
removed action_spec.size
andrewcoh Oct 23, 2020
b0d9a48
fix specs in torch util
andrewcoh Oct 23, 2020
d2bb5d0
fixed tests/ -> single validate_action func
andrewcoh Oct 23, 2020
ad144c3
make is_discrete/is_continuous strict
andrewcoh Oct 23, 2020
9090821
add docstrings
andrewcoh Oct 23, 2020
f23e395
rename make_x to creat_x/remove redundant properties
andrewcoh Oct 24, 2020
785848e
make validate action private
andrewcoh Oct 24, 2020
9af9ee9
fix advanced vis encoder simple rl
andrewcoh Oct 24, 2020
600d307
fix recurrent/advanced ppo tests
andrewcoh Oct 24, 2020
42bdfce
fix recurrent sac
andrewcoh Oct 24, 2020
754f5b8
reduce visual advanced steps
andrewcoh Oct 24, 2020
a8813fc
reduce recurrent step/increase batch size
andrewcoh Oct 25, 2020
64091cc
add ActionBuffers and utils
andrewcoh Oct 25, 2020
a8204bd
reduce steps_per_update recurrent sac
andrewcoh Oct 26, 2020
b5ca548
fix AgentExperience typing
andrewcoh Oct 26, 2020
ed11b10
recurrent sac passes locally but fails on CI for inexplicable reasons
andrewcoh Oct 26, 2020
442f29a
increase seq length
andrewcoh Oct 26, 2020
8733ec1
rename create random to random action
andrewcoh Oct 26, 2020
199d15b
rename create empty to empty action
andrewcoh Oct 26, 2020
00a824c
Merge branch 'develop-action-spec' into develop-action-buffer
andrewcoh Oct 26, 2020
b0ed241
Merge branch 'master' into develop-action-buffer
andrewcoh Oct 26, 2020
bfaa249
action buffer passes continuous
andrewcoh Oct 27, 2020
d927497
discrete runs/cont passes
andrewcoh Oct 27, 2020
0d33e1f
debugging discrete
andrewcoh Oct 27, 2020
8f06a67
2d discrete passes
andrewcoh Oct 27, 2020
da1c85a
sac continuous and discrete train
andrewcoh Oct 28, 2020
080f3eb
bc tests pass
andrewcoh Oct 29, 2020
f872359
torch reward providers all pass
andrewcoh Oct 29, 2020
5886f74
fixed bug in discrete
andrewcoh Oct 29, 2020
fe8fdd9
test_simple_rl/reward providers pass tf/torch
andrewcoh Oct 29, 2020
9479a65
ml-agents-envs pass
andrewcoh Nov 3, 2020
3a90973
Merge branch 'master' into develop-action-buffer
andrewcoh Nov 3, 2020
dbf819c
rename extract to from_dict
andrewcoh Nov 3, 2020
d1e2b97
agent processor tests
andrewcoh Nov 4, 2020
e87effe
fix demo loader tests
andrewcoh Nov 4, 2020
e0418dc
test_trajectory fixed
andrewcoh Nov 4, 2020
5f571a1
fixed recurrent prev_action issue
andrewcoh Nov 4, 2020
9089e63
fix test_tf_policy
andrewcoh Nov 5, 2020
f8d85fa
fix torch test_ppo
andrewcoh Nov 5, 2020
c21d223
fix torch utils test
andrewcoh Nov 5, 2020
f0f4249
discrete/contionuous unity envs train
andrewcoh Nov 5, 2020
d6eaf8d
agent processor tests
andrewcoh Nov 5, 2020
e9848b1
fix torch test policy
andrewcoh Nov 5, 2020
fbbff02
Barracuda inference for hybrid actions (#4611)
dongruoping Nov 5, 2020
b25fc3d
remove unused import
andrewcoh Nov 6, 2020
10944f1
add docstrings to AgentAction and ActionLogProbs
andrewcoh Nov 6, 2020
6fcdd3f
revert demo
andrewcoh Nov 6, 2020
633e3bc
Merge branch 'develop-action-buffer' into develop-hybrid-actions-sing…
andrewcoh Nov 6, 2020
275eef4
1:1 and continuous/discrete train
andrewcoh Nov 6, 2020
e173f29
all hybrid simple rl tests pass
andrewcoh Nov 6, 2020
8328f16
move action model to explicit distributions
andrewcoh Nov 6, 2020
df46ec9
remove unused code
andrewcoh Nov 6, 2020
b829957
fix simple test env
andrewcoh Nov 6, 2020
6d4738b
Remove print from ppo tf opti
andrewcoh Nov 6, 2020
a0f3dbe
bc works
andrewcoh Nov 6, 2020
8d10587
torch curiosity tests pass
andrewcoh Nov 7, 2020
c3f69e6
remove print bc
andrewcoh Nov 7, 2020
5c8ec2d
rename to ActionTuple
andrewcoh Nov 9, 2020
0441118
Merge branch 'develop-action-buffer' of https://github.com/Unity-Tech…
andrewcoh Nov 9, 2020
86b6d71
Update ml-agents/mlagents/trainers/torch/utils.py
andrewcoh Nov 9, 2020
2bf004c
ActionTuple default is now np.array, not None
andrewcoh Nov 9, 2020
aaf6c59
fix set_actions_for_agent
andrewcoh Nov 9, 2020
056cf6d
fix action mask in trajectory
andrewcoh Nov 9, 2020
74ce2b1
Merge branch 'develop-action-buffer' into develop-hybrid-actions-sing…
andrewcoh Nov 9, 2020
5691f60
Update ml-agents-envs/mlagents_envs/environment.py
andrewcoh Nov 9, 2020
62fcecb
replace use_discrete with action_sizes in simple_rl
andrewcoh Nov 9, 2020
b567fcd
revert demo
andrewcoh Nov 9, 2020
116580a
Merge branch 'develop-action-buffer' of https://github.com/Unity-Tech…
andrewcoh Nov 9, 2020
3528376
Refactor BarracudaModel loader checks (#4629)
dongruoping Nov 9, 2020
b152511
fix default random action
andrewcoh Nov 10, 2020
bb9988c
fix reward provider tests
andrewcoh Nov 10, 2020
c488e8e
add defaults to ActionTuple constructor
andrewcoh Nov 10, 2020
589907a
remove unused line in traj
andrewcoh Nov 10, 2020
c8ae8da
save only discrete actions as prev
andrewcoh Nov 10, 2020
c651ebc
update make_empty docstring
andrewcoh Nov 10, 2020
0dc4396
reuse action dict in torch policy for pre_action
andrewcoh Nov 10, 2020
434f210
add back removed part of test_envs
andrewcoh Nov 10, 2020
714b444
fix mock brain prev action
andrewcoh Nov 10, 2020
65d17fe
default ActionTuple to None
andrewcoh Nov 12, 2020
032b7ba
Merge branch 'develop-action-buffer' into develop-hybrid-actions-sing…
andrewcoh Nov 12, 2020
4fc60d5
default actions are np.array of shape (n_agents, 0)
andrewcoh Nov 12, 2020
99c2ebc
preliminary aciton model tests
andrewcoh Nov 12, 2020
2bbab9b
move AgentAction, ActionLogProbs, and ActionFlattener to separate files
andrewcoh Nov 12, 2020
6e20e1d
add action util files
andrewcoh Nov 12, 2020
7868abf
action model and network tests
andrewcoh Nov 12, 2020
a8676fe
Update ml-agents/mlagents/trainers/policy/tf_policy.py
andrewcoh Nov 12, 2020
24809e1
fix torch policy tests
andrewcoh Nov 12, 2020
45c92cb
remove actionspec logic simple test env
andrewcoh Nov 12, 2020
63f72df
Merge branch 'develop-hybrid-actions-singleton' into develop-hybrid-a…
dongruoping Nov 13, 2020
91afdf8
check tensor exist before access shape
dongruoping Nov 13, 2020
3f1b8d9
remove action spec attribute from policy
andrewcoh Nov 16, 2020
96d0a48
added docstrings to action flattener
andrewcoh Nov 16, 2020
e0ffd3e
added docstrings to action model
andrewcoh Nov 16, 2020
e49f68b
Policy output actiontuple (#4651)
andrewcoh Nov 16, 2020
f25ba54
Merge branch 'develop-action-buffer' into develop-hybrid-actions-sing…
andrewcoh Nov 16, 2020
b712195
remove unused imports test_hybrid
andrewcoh Nov 16, 2020
41a5a29
fix imports in test utils
andrewcoh Nov 16, 2020
793957c
Merge branch 'develop-hybrid-action-staging' into develop-hybrid-acti…
andrewcoh Nov 16, 2020
22f9ee7
fix subprocess_env_manager check of action info length
andrewcoh Nov 16, 2020
ca3b492
Merge branch 'develop-hybrid-action-staging' into develop-hybrid-acti…
andrewcoh Nov 17, 2020
23d499c
resolve all conflicts with staging
andrewcoh Nov 17, 2020
60b6c91
fix test agent processor
andrewcoh Nov 17, 2020
a3321c1
fix test trajectory
andrewcoh Nov 17, 2020
2601279
fix test torch distributions
andrewcoh Nov 17, 2020
05910ef
fix torch test policy
andrewcoh Nov 17, 2020
58299fb
fix torch test ppo
andrewcoh Nov 17, 2020
706ddd8
fix test tf policy
andrewcoh Nov 17, 2020
533f985
Export separate nodes for continuous/discrete actions (#4655)
dongruoping Nov 17, 2020
9a45d4f
fix tensorflow test simple rl
andrewcoh Nov 17, 2020
1eda523
Merge branch 'develop-hybrid-actions-singleton' into develop-hybrid-a…
dongruoping Nov 17, 2020
aae5420
fix policy action_spec typo
dongruoping Nov 17, 2020
7a10f0b
change method name and formatting
dongruoping Nov 18, 2020
6ec6635
Separate continuous/discrete actions in AgentActionProto (#4698)
dongruoping Dec 3, 2020
1afd756
Merge branch 'develop-hybrid-action-staging' into develop-hybrid-acti…
dongruoping Dec 3, 2020
ea6393b
fix merge
dongruoping Dec 3, 2020
0d3e10d
Merge branch 'develop-hybrid-action-staging' into develop-hybrid-acti…
dongruoping Dec 4, 2020
fa3e093
Force different nodes for new and deprecated action output (#4705)
dongruoping Dec 4, 2020
a3a28ee
read deprecated fields in BrainParmeterProto if ActionSpec not set
dongruoping Dec 4, 2020
f8f4615
fix bc tests
dongruoping Dec 4, 2020
4b3ef56
Merge branch 'develop-hybrid-actions-csharp' of https://github.com/Un…
dongruoping Dec 4, 2020
776134c
remove yamato trigger
dongruoping Dec 4, 2020
74e6e22
fix tests
dongruoping Dec 4, 2020
0f315ea
fix tests
dongruoping Dec 4, 2020
7468dfa
fix torch bc tests
dongruoping Dec 4, 2020
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
check tensor exist before access shape
  • Loading branch information
dongruoping committed Nov 13, 2020
commit 91afdf85bde70367c02b4ac37a7ea458eca8cf5f
Original file line number Diff line number Diff line change
@@ -183,7 +183,8 @@ public static int ContinuousOutputSize(this Model model)
}
else
{
return (int)model.GetTensorByName(TensorNames.ContinuousActionOutputShape)[0];
var continuousOutputShape = model.GetTensorByName(TensorNames.ContinuousActionOutputShape);
return continuousOutputShape == null ? 0 : (int)continuousOutputShape[0];
}
}

@@ -248,7 +249,8 @@ public static int DiscreteOutputSize(this Model model)
}
else
{
return (int)model.GetTensorByName(TensorNames.DiscreteActionOutputShape)[0];
var discreteOutputShape = model.GetTensorByName(TensorNames.DiscreteActionOutputShape);
return discreteOutputShape == null ? 0 : (int)discreteOutputShape[0];
}
}