Fixed some bugs and updated Documentation

Assets/ML-Agents/Timers/Crossroad_timers.json

@@ -1 +1 @@
-{"count":1,"self":7.1537695999999995,"total":13.351348999999999,"children":{"InitializeActuators":{"count":64,"self":0.002062,"total":0.002062,"children":null},"InitializeSensors":{"count":64,"self":0.026810999999999998,"total":0.026810999999999998,"children":null},"AgentSendState":{"count":606,"self":0.196108,"total":3.797703,"children":{"GridSensor.Update":{"count":6464,"self":3.55298,"total":3.55298,"children":null},"CollectObservations":{"count":6464,"self":0.014714,"total":0.014714,"children":null},"WriteActionMask":{"count":6464,"self":0.009531,"total":0.009531,"children":null},"RequestDecision":{"count":6464,"self":0.02437,"total":0.02437,"children":null}}},"DecideAction":{"count":606,"self":1.86553,"total":2.023949,"children":{"GridSensor.Write":{"count":6464,"self":0.143345,"total":0.143345,"children":null},"RayPerceptionSensor.Perceive":{"count":6464,"self":0.015073999999999999,"total":0.015073999999999999,"children":null}}},"AgentAct":{"count":606,"self":0.307096,"total":0.307709,"children":{"GridSensor.Update":{"count":1,"self":0.00061299999999999994,"total":0.00061299999999999994,"children":null}}},"GridSensor.Update":{"count":66,"self":0.037447999999999995,"total":0.037447999999999995,"children":null}},"gauges":{"CrossroadLights.CumulativeReward":{"count":67,"max":0.9949333,"min":-1.0026,"runningAverage":0.888096035,"value":0.9905667,"weightedAverage":0.963085353}},"metadata":{"timer_format_version":"0.1.0","start_time_seconds":"1656930979","unity_version":"2020.3.11f1","command_line_arguments":"\/Applications\/Unity\/Hub\/Editor\/2020.3.11f1\/Unity.app\/Contents\/MacOS\/Unity -projectpath \/Users\/stela\/Git\/CYENS\/TrafficFlow -useHub -hubIPC -cloudEnvironment production -licensingIpc LicenseClient-stela -hubSessionId 59157390-fb7b-11ec-8664-852e03a70f20 -accessToken qNiLrSuPDlQ1cBCsjDAIQwT7WacN73fc1t5NX4Acioc011f","communication_protocol_version":"1.5.0","com.unity.ml-agents_version":"2.1.0-exp.1","scene_name":"Crossroad","end_time_seconds":"1656930992"}}
+{"count":1,"self":1.463766,"total":2.453487,"children":{"InitializeActuators":{"count":64,"self":0.001792,"total":0.001792,"children":null},"InitializeSensors":{"count":64,"self":0.041759,"total":0.041759,"children":null},"AgentSendState":{"count":84,"self":0.019884,"total":0.46309,"children":{"GridSensor.Update":{"count":896,"self":0.400651,"total":0.400651,"children":null},"CollectObservations":{"count":896,"self":0.001743,"total":0.001743,"children":null},"WriteActionMask":{"count":896,"self":0.001076,"total":0.001076,"children":null},"RequestDecision":{"count":896,"self":0.039736,"total":0.039736,"children":null}}},"DecideAction":{"count":84,"self":0.39902299999999996,"total":0.440559,"children":{"GridSensor.Write":{"count":896,"self":0.039451,"total":0.039451,"children":null},"RayPerceptionSensor.Perceive":{"count":896,"self":0.002085,"total":0.002085,"children":null}}},"AgentAct":{"count":84,"self":0.041629,"total":0.041629,"children":null}},"gauges":{},"metadata":{"timer_format_version":"0.1.0","start_time_seconds":"1669039146","unity_version":"2020.3.11f1","command_line_arguments":"\/home\/stela\/Unity\/Hub\/Editor\/2020.3.11f1\/Editor\/Unity -projectpath \/home\/stela\/Git\/CYENS\/TrafficFlow -useHub -hubIPC -cloudEnvironment production -licensingIpc LicenseClient-stela -hubSessionId 9be09030-69a1-11ed-9d88-fd227bc9813a -accessToken dy2PgFNNXVOA1nB4yC22lCSDWcqMXBmWtekVvHI2h6g011f","communication_protocol_version":"1.5.0","com.unity.ml-agents_version":"2.2.1-exp.1","scene_name":"Crossroad","end_time_seconds":"1669039149"}}

README.md

@@ -1,11 +1,10 @@
 # TrafficFlow
 
-This Unity package contains work in progress on developing a heterogeneous, multiagent system that simulates traffic flow in the context of Non-Player Characters for game design. A Reinforcement Learning approch was equipped.
+This Unity package contains work in progress on developing a heterogeneous, multi-agent system that simulates traffic flow in the context of Non-Player Characters for game design. A Reinforcement Learning approach was equipped.
 
 To use our work please cite the following as a reference:
 [1] S. Makri and P. Charalambous, "Towards a multi-agent non-player character road network: a Reinforcement Learning approach" _2021 IEEE Conference on Games (CoG)_, 2021, pp. 1-5, doi: [10.1109/CoG52621.2021.9619047](https://doi.org/10.1109/CoG52621.2021.9619047).
-[2] S. Makri and P. Charalambous, "Navigating a road network using Reinforcement
-Learning" (in prep.)
+[2] S. Makri and P. Charalambous, "Navigating a road network using Reinforcement Learning" _2022 International Conference on Interactive Media, Smart Systems and Emerging Technologies (IMET)_, 2022, pp. 1-8, doi: [10.1109/IMET54801.2022.9929765](https://ieeexplore.ieee.org/document/9929765).
 
 
 ### Installation
@@ -31,7 +30,7 @@ git clone --branch v2.1 https://github.com/CYENS/TrafficFlow.git
 
 You then need to edit the `./Packages/manifest.json` and `.Packages/packages-lock.json` files as follows.
 1. In line 8 of `./Packages/manifest.json`, replace `</path/to/ml-agents>` with the path where you installed the `ml-agents` repository.
-1. In line 82 of `./Packages/manifest.json`, replace `</path/to/ml-agents>` with the path where you installed the `ml-agents` repository.
+1. In line 82 of `./Packages/packages-lock.json`, replace `</path/to/ml-agents>` with the path where you installed the `ml-agents` repository.
 
 #### Installing the ML-Agents Unity package
 Open the `TrafficFlow` project in Unity. You should check that the following Unity packages are listed in the Package Manager.
@@ -52,4 +51,4 @@ You can also install these mannually by [adding the local](https://docs.unity3d.
 There are 3 scenes implemented, organised in 3 separate folders under the [`Assets`](./Assets/) folder. Pre-trained model files are provided for each environment. These are located in the respective `Models` folders.
 1. The [`Basic`](./Assets/Basic/Scenes/Basic.unity) scene, where our first experiments were carried out. A single agent crosses the traffic lights to reach its target. The lane is quite wide.
 1. A [`Lane`](./Assets/Lanes/Scenes/Lane.unity) scene, where we extended the `Basic` scene to train two agents crossing the traffic lights from opposite directions in order to reach their goal.
-1. A [`Crossroad`](./Assets/Crossroads/Scenes/Crossroad.unity) scene. Here, we have implemented two different areas. A simple crossroad intersection governed by traffic lights, and a simplified version of the same intersection with motion restricted in a line to help training. For more details you can refer to [Makri and Charalambous (2021)](https://doi.org/10.1109/CoG52621.2021.9619047).
+1. A [`Crossroad`](./Assets/Crossroads/Scenes/Crossroad.unity) scene. Here, we have implement a simple crossroad intersection governed by traffic lights. For more details you can refer to [Makri and Charalambous (2022)](https://ieeexplore.ieee.org/document/9929765).