Extension to continuous action space

[rsuwa]

Is it easy to change the action space from discrete to continuous to implement multi-agent DDPG?

[praveen-palanisamy]

Hey @rsuwa , sorry for the delayed response. I somehow missed this.

Yes. The environments support continuous actions. By default they are configured to use discrete actions as the V-Trace algorithm used in IMPALA supports only discrete actions.

You can start using continuous actions in two ways:

1. Creating a new environment (recommended & it's easy enough; Details are listed below)
2. Modifying existing environment (not-recommended but just needs a one word change)

One way to create a new environment is to simply copy and replicate an existing environment and change what you want and give it a new name. For example, to create a variation of this
UrbanSignalIntersection3Car environment with continuous actions, you can copy this file, rename it (e.g.: urban_signal_intersection_3c_continuous.py) and edit the following line:

macad-gym/src/macad_gym/envs/intersection/urban_signal_intersection_3c.py

Line 20 in d0a2a75

"discrete_actions": True,

and set "discrete_actions": False.

That's it!

You can give this environment a new name (e.g.: UrbanSignalIntersection3CarContinuous) by registering this new environment with it's name like the existing discrete environment shown below:

macad-gym/src/macad_gym/envs/__init__.py

Lines 10 to 11 in d0a2a75

	from macad_gym.envs.intersection.urban_signal_intersection_3c \
	import UrbanSignalIntersection3Car

The last step is to add it to the list of available environments here:

macad-gym/src/macad_gym/__init__.py

Line 21 in d0a2a75

_AVAILABLE_ENVS = {

You can then just use: gym.make("UrbanSignalIntersection3CarContinuous-v0") in your code!

Let me know if you need more help. Also, feel free to submit a pull request with your new environment!