how to run benchmark in the CoRL20 paper

[zyk516]

High Level Description
[I want to run benchmark in the CoRL20 paper [SMARTS: Scalable Multi-Agent ReinforcementLearning Training School for Autonomous Driving]]

Desired SMARTS version
[0.4.16]

Operating System
[ ubuntu 18.04]

Problems
When I run the commend ' python3.7 run.py /home/idriver/zyk/study/rl/SMARTS/benchmark/scenarios/intersections/4lane -f benchmark/agents/dqn/baseline-lane-control.yaml
', a AssertionError: EndlessGoal() happened,like below.
File "python/ray/_raylet.pyx", line 482, in ray._raylet.execute_task
File "python/ray/_raylet.pyx", line 436, in ray._raylet.execute_task.function_executor
File "/home/idriver/zyk/study/rl/SMARTS/baselines/marl_benchmark/.venv/lib/python3.7/site-packages/ray/util/iter.py", line 1152, in par_iter_next
return next(self.local_it)
File "/home/idriver/zyk/study/rl/SMARTS/baselines/marl_benchmark/.venv/lib/python3.7/site-packages/ray/rllib/evaluation/rollout_worker.py", line 317, in gen_rollouts
yield self.sample()
File "/home/idriver/zyk/study/rl/SMARTS/baselines/marl_benchmark/.venv/lib/python3.7/site-packages/ray/rllib/evaluation/rollout_worker.py", line 621, in sample
batches = [self.input_reader.next()]
File "/home/idriver/zyk/study/rl/SMARTS/baselines/marl_benchmark/.venv/lib/python3.7/site-packages/ray/rllib/evaluation/sampler.py", line 94, in next
batches = [self.get_data()]
File "/home/idriver/zyk/study/rl/SMARTS/baselines/marl_benchmark/.venv/lib/python3.7/site-packages/ray/rllib/evaluation/sampler.py", line 211, in get_data
item = next(self.rollout_provider)
File "/home/idriver/zyk/study/rl/SMARTS/baselines/marl_benchmark/.venv/lib/python3.7/site-packages/ray/rllib/evaluation/sampler.py", line 550, in _env_runner
base_env.poll()
File "/home/idriver/zyk/study/rl/SMARTS/baselines/marl_benchmark/.venv/lib/python3.7/site-packages/ray/rllib/env/base_env.py", line 390, in poll
obs[i], rewards[i], dones[i], infos[i] = env_state.poll()
File "/home/idriver/zyk/study/rl/SMARTS/baselines/marl_benchmark/.venv/lib/python3.7/site-packages/ray/rllib/env/base_env.py", line 442, in poll
self.reset()
File "/home/idriver/zyk/study/rl/SMARTS/baselines/marl_benchmark/.venv/lib/python3.7/site-packages/ray/rllib/env/base_env.py", line 460, in reset
self.last_obs = self.env.reset()
File "/home/idriver/zyk/study/rl/SMARTS/baselines/marl_benchmark/.venv/lib/python3.7/site-packages/benchmark/wrappers/rllib/frame_stack.py", line 204, in reset
return self._get_observations(observations)
File "/home/idriver/zyk/study/rl/SMARTS/baselines/marl_benchmark/.venv/lib/python3.7/site-packages/benchmark/wrappers/rllib/frame_stack.py", line 160, in get_observations
observation = self.observation_adapter(observation)
File "/home/idriver/zyk/study/rl/SMARTS/baselines/marl_benchmark/.venv/lib/python3.7/site-packages/benchmark/wrappers/rllib/frame_stack.py", line 119, in func
observation = cal_obs(env_obs_seq, observation_space, feature_configs)
File "/home/idriver/zyk/study/rl/SMARTS/baselines/marl_benchmark/.venv/lib/python3.7/site-packages/benchmark/common.py", line 889, in cal_obs
obs_np.append(cal_obs(obs, sub_space, feature_configs))
File "/home/idriver/zyk/study/rl/SMARTS/baselines/marl_benchmark/.venv/lib/python3.7/site-packages/benchmark/common.py", line 884, in cal_obs
obs_np[name] = getattr(CalObs, f"cal{name}")(env_obs, *args)
File "/home/idriver/zyk/study/rl/SMARTS/baselines/marl_benchmark/.venv/lib/python3.7/site-packages/benchmark/common.py", line 174, in cal_goal_relative_pos
assert isinstance(goal, PositionalGoal), goal
AssertionError: EndlessGoal()

[zyk516]

I solved this error by using commend 'scl scenario build --clean scenarios/intersections/4lane',but new error happened,like below.
Traceback (most recent call last):
File "/home/idriver/zyk/study/rl/SMARTS/baselines/marl_benchmark/.venv/lib/python3.7/site-packages/ray/tune/trial_runner.py", line 726, in _process_trial
result = self.trial_executor.fetch_result(trial)
File "/home/idriver/zyk/study/rl/SMARTS/baselines/marl_benchmark/.venv/lib/python3.7/site-packages/ray/tune/ray_trial_executor.py", line 489, in fetch_result
result = ray.get(trial_future[0], timeout=DEFAULT_GET_TIMEOUT)
File "/home/idriver/zyk/study/rl/SMARTS/baselines/marl_benchmark/.venv/lib/python3.7/site-packages/ray/worker.py", line 1452, in get
raise value.as_instanceof_cause()
ray.exceptions.RayTaskError(ValueError): ray::PPO.train() (pid=32767, ip=10.236.15.72)
File "python/ray/_raylet.pyx", line 482, in ray._raylet.execute_task
File "python/ray/_raylet.pyx", line 436, in ray._raylet.execute_task.function_executor
File "/home/idriver/zyk/study/rl/SMARTS/baselines/marl_benchmark/.venv/lib/python3.7/site-packages/ray/rllib/agents/trainer.py", line 517, in train
raise e
File "/home/idriver/zyk/study/rl/SMARTS/baselines/marl_benchmark/.venv/lib/python3.7/site-packages/ray/rllib/agents/trainer.py", line 506, in train
result = Trainable.train(self)
File "/home/idriver/zyk/study/rl/SMARTS/baselines/marl_benchmark/.venv/lib/python3.7/site-packages/ray/tune/trainable.py", line 336, in train
result = self.step()
File "/home/idriver/zyk/study/rl/SMARTS/baselines/marl_benchmark/.venv/lib/python3.7/site-packages/ray/rllib/agents/trainer_template.py", line 147, in step
res = next(self.train_exec_impl)
File "/home/idriver/zyk/study/rl/SMARTS/baselines/marl_benchmark/.venv/lib/python3.7/site-packages/ray/util/iter.py", line 756, in next
return next(self.built_iterator)
File "/home/idriver/zyk/study/rl/SMARTS/baselines/marl_benchmark/.venv/lib/python3.7/site-packages/ray/util/iter.py", line 783, in apply_foreach
for item in it:
File "/home/idriver/zyk/study/rl/SMARTS/baselines/marl_benchmark/.venv/lib/python3.7/site-packages/ray/util/iter.py", line 783, in apply_foreach
for item in it:
File "/home/idriver/zyk/study/rl/SMARTS/baselines/marl_benchmark/.venv/lib/python3.7/site-packages/ray/util/iter.py", line 843, in apply_filter
for item in it:
File "/home/idriver/zyk/study/rl/SMARTS/baselines/marl_benchmark/.venv/lib/python3.7/site-packages/ray/util/iter.py", line 843, in apply_filter
for item in it:
File "/home/idriver/zyk/study/rl/SMARTS/baselines/marl_benchmark/.venv/lib/python3.7/site-packages/ray/util/iter.py", line 783, in apply_foreach
for item in it:
File "/home/idriver/zyk/study/rl/SMARTS/baselines/marl_benchmark/.venv/lib/python3.7/site-packages/ray/util/iter.py", line 783, in apply_foreach
for item in it:
File "/home/idriver/zyk/study/rl/SMARTS/baselines/marl_benchmark/.venv/lib/python3.7/site-packages/ray/util/iter.py", line 783, in apply_foreach
for item in it:
[Previous line repeated 1 more time]
File "/home/idriver/zyk/study/rl/SMARTS/baselines/marl_benchmark/.venv/lib/python3.7/site-packages/ray/util/iter.py", line 876, in apply_flatten
for item in it:
File "/home/idriver/zyk/study/rl/SMARTS/baselines/marl_benchmark/.venv/lib/python3.7/site-packages/ray/util/iter.py", line 828, in add_wait_hooks
item = next(it)
File "/home/idriver/zyk/study/rl/SMARTS/baselines/marl_benchmark/.venv/lib/python3.7/site-packages/ray/util/iter.py", line 783, in apply_foreach
for item in it:
File "/home/idriver/zyk/study/rl/SMARTS/baselines/marl_benchmark/.venv/lib/python3.7/site-packages/ray/util/iter.py", line 783, in apply_foreach
for item in it:
File "/home/idriver/zyk/study/rl/SMARTS/baselines/marl_benchmark/.venv/lib/python3.7/site-packages/ray/util/iter.py", line 783, in apply_foreach
for item in it:
[Previous line repeated 1 more time]
File "/home/idriver/zyk/study/rl/SMARTS/baselines/marl_benchmark/.venv/lib/python3.7/site-packages/ray/util/iter.py", line 471, in base_iterator
yield ray.get(futures, timeout=timeout)
ray.exceptions.RayTaskError(ValueError): ray::RolloutWorker.par_iter_next() (pid=32760, ip=10.236.15.72)
File "python/ray/_raylet.pyx", line 482, in ray._raylet.execute_task
File "python/ray/_raylet.pyx", line 436, in ray._raylet.execute_task.function_executor
File "/home/idriver/zyk/study/rl/SMARTS/baselines/marl_benchmark/.venv/lib/python3.7/site-packages/ray/util/iter.py", line 1152, in par_iter_next
return next(self.local_it)
File "/home/idriver/zyk/study/rl/SMARTS/baselines/marl_benchmark/.venv/lib/python3.7/site-packages/ray/rllib/evaluation/rollout_worker.py", line 317, in gen_rollouts
yield self.sample()
File "/home/idriver/zyk/study/rl/SMARTS/baselines/marl_benchmark/.venv/lib/python3.7/site-packages/ray/rllib/evaluation/rollout_worker.py", line 621, in sample
batches = [self.input_reader.next()]
File "/home/idriver/zyk/study/rl/SMARTS/baselines/marl_benchmark/.venv/lib/python3.7/site-packages/ray/rllib/evaluation/sampler.py", line 94, in next
batches = [self.get_data()]
File "/home/idriver/zyk/study/rl/SMARTS/baselines/marl_benchmark/.venv/lib/python3.7/site-packages/ray/rllib/evaluation/sampler.py", line 211, in get_data
item = next(self.rollout_provider)
File "/home/idriver/zyk/study/rl/SMARTS/baselines/marl_benchmark/.venv/lib/python3.7/site-packages/ray/rllib/evaluation/sampler.py", line 602, in _env_runner
observation_fn=observation_fn,
File "/home/idriver/zyk/study/rl/SMARTS/baselines/marl_benchmark/.venv/lib/python3.7/site-packages/ray/rllib/evaluation/sampler.py", line 793, in _process_observations
policy_id).transform(raw_obs)
File "/home/idriver/zyk/study/rl/SMARTS/baselines/marl_benchmark/.venv/lib/python3.7/site-packages/ray/rllib/models/preprocessors.py", line 201, in transform
self.check_shape(observation)
File "/home/idriver/zyk/study/rl/SMARTS/baselines/marl_benchmark/.venv/lib/python3.7/site-packages/ray/rllib/models/preprocessors.py", line 64, in check_shape
observation, self._obs_space)
ValueError: ('Observation ({}) outside given space ({})!', [{'distance_to_center': array([0.]), 'goal_relative_pos': array([96.08002933, -3.17314832]), 'heading_errors': array([-3.10032755e-17, -3.10032755e-17, -3.10032755e-17, -3.10032755e-17,
-3.10032755e-17, -3.10032755e-17, -3.10032755e-17, -3.10032755e-17,
-3.10032755e-17, -3.10032755e-17, -3.10032755e-17, -3.10032755e-17,
-3.10032755e-17, -3.10032755e-17, -3.10032755e-17, -3.10032755e-17,
-1.08741017e-05, -1.08741017e-05, -1.08741017e-05, -1.08741017e-05]), 'neighbor': array([ 0.00000000e+00, 0.00000000e+00, 0.00000000e+00, 0.00000000e+00,
0.00000000e+00, 2.95447156e+01, -5.12161342e-01, 5.00000000e+00,
2.08956771e+01, 2.08868595e+01, 0.00000000e+00, 0.00000000e+00,
0.00000000e+00, 0.00000000e+00, 0.00000000e+00, 0.00000000e+00,
0.00000000e+00, 0.00000000e+00, 0.00000000e+00, 0.00000000e+00,
0.00000000e+00, 0.00000000e+00, 0.00000000e+00, 0.00000000e+00,
0.00000000e+00, 0.00000000e+00, 0.00000000e+00, 0.00000000e+00,
0.00000000e+00, 0.00000000e+00, 0.00000000e+00, 0.00000000e+00,
0.00000000e+00, 0.00000000e+00, 0.00000000e+00, 3.68947212e+01,
-1.03541739e-02, 5.00000000e+00, 2.72770673e+01, -2.48431490e+01]), 'speed': array([50.004]), 'steering': array([-0.])}, {'distance_to_center': array([1.13686838e-13]), 'goal_relative_pos': array([-72.00700147, 31.1937964 ]), 'heading_errors': array([6.20065511e-17, 6.20065511e-17, 6.20065511e-17, 6.20065511e-17,
6.20065511e-17, 6.20065511e-17, 6.20065511e-17, 6.20065511e-17,
6.20065511e-17, 6.20065511e-17, 6.20065511e-17, 6.20065511e-17,
6.20065511e-17, 6.20065511e-17, 6.20065511e-17, 6.20065511e-17,
6.20065511e-17, 3.00841383e-03, 4.28378318e-03, 5.55914556e-03]), 'neighbor': array([ 0. , 0. , 0. , 0. ,
0. , 39.43851168, 2.25096583, 5. ,
24.03296201, 31.27000065, 46.17310702, 14.91588265,
3.09556652, -6.38139025, 45.73000842, 36.8947212 ,
19.86520166, 1.8572538 , -27.27706732, 24.84314897,
0. , 0. , 0. , 0. ,
0. , 0. , 0. , 0. ,
0. , 0. , 0. , 0. ,
0. , 0. , 0. , 0. ,
0. , 0. , 0. , 0. ]), 'speed': array([50.004]), 'steering': array([-0.])}, {'distance_to_center': array([0.]), 'goal_relative_pos': array([-76.03997564, -3.25414312]), 'heading_errors': array([3.87540944e-18, 3.87540944e-18, 3.87540944e-18, 3.87540944e-18,
3.87540944e-18, 3.87540944e-18, 3.87540944e-18, 3.87540944e-18,
3.87540944e-18, 3.87540944e-18, 3.87540944e-18, 3.87540944e-18,
3.87540944e-18, 3.87540944e-18, 3.87540944e-18, 3.87540944e-18,
8.38697317e-06, 8.38697317e-06, 8.38697317e-06, 8.38697317e-06]), 'neighbor': array([ 0. , 0. , 0. , 0. ,
0. , 0. , 0. , 0. ,
0. , 0. , 0. , 0. ,
0. , 0. , 0. , 33.67676719,
24.73229975, 1.36165126, -30.41435226, 14.46000777,
0. , 0. , 0. , 0. ,
0. , 39.43851168, 16.9230466 , 2.33046168,
-24.03296201, -31.27000065, 0. , 0. ,
0. , 0. , 0. , 0. ,
0. , 0. , 0. , 0. ]), 'speed': array([50.004]), 'steering': array([-0.])}], Tuple(Dict(distance_to_center:Box([-inf], [inf], (1,), float32), goal_relative_pos:Box([-inf -inf], [inf inf], (2,), float32), heading_errors:Box([-inf -inf -inf -inf -inf -inf -inf -inf -inf -inf -inf -inf -inf -inf
-inf -inf -inf -inf -inf -inf], [inf inf inf inf inf inf inf inf inf inf inf inf inf inf inf inf inf inf
inf inf], (20,), float32), neighbor:Box([-inf -inf -inf -inf -inf -inf -inf -inf -inf -inf -inf -inf -inf -inf
-inf -inf -inf -inf -inf -inf -inf -inf -inf -inf -inf -inf -inf -inf
-inf -inf -inf -inf -inf -inf -inf -inf -inf -inf -inf -inf], [inf inf inf inf inf inf inf inf inf inf inf inf inf inf inf inf inf inf
inf inf inf inf inf inf inf inf inf inf inf inf inf inf inf inf inf inf
inf inf inf inf], (40,), float32), speed:Box([-inf], [inf], (1,), float32), steering:Box([-inf], [inf], (1,), float32)), Dict(distance_to_center:Box([-inf], [inf], (1,), float32), goal_relative_pos:Box([-inf -inf], [inf inf], (2,), float32), heading_errors:Box([-inf -inf -inf -inf -inf -inf -inf -inf -inf -inf -inf -inf -inf -inf
-inf -inf -inf -inf -inf -inf], [inf inf inf inf inf inf inf inf inf inf inf inf inf inf inf inf inf inf
inf inf], (20,), float32), neighbor:Box([-inf -inf -inf -inf -inf -inf -inf -inf -inf -inf -inf -inf -inf -inf
-inf -inf -inf -inf -inf -inf -inf -inf -inf -inf -inf -inf -inf -inf
-inf -inf -inf -inf -inf -inf -inf -inf -inf -inf -inf -inf], [inf inf inf inf inf inf inf inf inf inf inf inf inf inf inf inf inf inf
inf inf inf inf inf inf inf inf inf inf inf inf inf inf inf inf inf inf
inf inf inf inf], (40,), float32), speed:Box([-inf], [inf], (1,), float32), steering:Box([-inf], [inf], (1,), float32)), Dict(distance_to_center:Box([-inf], [inf], (1,), float32), goal_relative_pos:Box([-inf -inf], [inf inf], (2,), float32), heading_errors:Box([-inf -inf -inf -inf -inf -inf -inf -inf -inf -inf -inf -inf -inf -inf
-inf -inf -inf -inf -inf -inf], [inf inf inf inf inf inf inf inf inf inf inf inf inf inf inf inf inf inf
inf inf], (20,), float32), neighbor:Box([-inf -inf -inf -inf -inf -inf -inf -inf -inf -inf -inf -inf -inf -inf
-inf -inf -inf -inf -inf -inf -inf -inf -inf -inf -inf -inf -inf -inf
-inf -inf -inf -inf -inf -inf -inf -inf -inf -inf -inf -inf], [inf inf inf inf inf inf inf inf inf inf inf inf inf inf inf inf inf inf
inf inf inf inf inf inf inf inf inf inf inf inf inf inf inf inf inf inf
inf inf inf inf], (40,), float32), speed:Box([-inf], [inf], (1,), float32), steering:Box([-inf], [inf], (1,), float32))))
Could anyone help me please?Thanks.

[Gamenot]

Hello @zyk516, thank you for the question. I have a few things I can say about this.

I remember that there were a few features of the scenario files that the CoRL20 code does not support. One of the features it does not support is the mission types that were experimental at the time; EndlessMission, LapMission, and GroupedLapMission. I do not remember why this is the case but the ego_missions given to the smarts.sstudio.types.Scenario given to gen_scenario() should only use smarts.sstudio.types.Mission. The offending line is the mission created at L22 here:

SMARTS/scenarios/intersections/4lane/scenario.py

Lines 18 to 22 in c5a1608

	ego_missions = [
	Mission(
	route=Route(begin=("edge-south-SN", 1, 10), end=("edge-west-EW", 1, "max")),
	),
	EndlessMission(

I think from this we will update the documentation so that this will be less likely to come up again in the future.

The second issue is not quite as clear to me. It looks to be that somehow the observation and observation space does not match. Are you still running the same command as the first time (i.e. python3.7 run.py /home/idriver/zyk/study/rl/SMARTS/benchmark/scenarios/intersections/4lane -f benchmark/agents/dqn/baseline-lane-control.yaml?) A few things could help me understand the issue: the output of pip freeze in your environment, the command you ran, the repository branch if you are using that, the session log files from ~/.smarts, and /tmp/ray/<your session>/logs. I will try to look into it in the meantime.

Next, @RutvikGupta is currently looking at rehabilitating the CoRL20 code for version 0.4.18 of SMARTS. @RutvikGupta if you have the chance could you look into if the Mission restriction is reasonable?

If you have any further questions please ask.

[zyk516]

Hello.@Gamenot ,thank you for your reply.
For the second issue, I run the same command as the first time.I solved this by changing version of gym from 0.20.3 to 0.18.3.
Thanks for your help.

[zyk516]

Hello.@RutvikGupta ,@Gamenot .The same error as #340 happened when I run the command above.The ray version is 1.0.1post, and smarts is 0.4.18, the code branch is SMARTS-marl_benchmark_path_fixes(When I used master branch, the error happened too.)How to solve this?I wish your replay.Thank you.

[Gamenot]

I am surprised that this would happen. Looking through the ray 1.0.1post codebase it looks like the non-default policy ids "should" be applied: https://github.com/ray-project/ray/blob/66c30e3ad01c6e2222effb2dafacb57b79a301f5/rllib/evaluation/rollout_worker.py#L1022

I will investigate more.

[Gamenot]

@zyk516 Is there any chance that an older version of ray got installed after the initial installation?

[zyk516]

Hello,@Gamenot .I tried vay1.18.0 and 0.8.7,the same error would happen.If I delete the line"export_formats: [model, checkpoint]" in baseline-lane-control.yaml, there is no error.And when i run the evaluate.py in benchmark, a new error as blow happened.
Traceback (most recent call last):
File "evaluate.py", line 120, in
log_dir=args.log_dir,
File "evaluate.py", line 103, in main
rollout(trainer, None, metrics_handler, num_steps, num_episodes, log_dir)
File "/home/idriver/zyk/study/rl/SMARTS-marl_benchmark_path_fixes/baselines/marl_benchmark/marl_benchmark/utils/rollout.py", line 53, in rollout
obs = env.reset()
AttributeError: 'NoneType' object has no attribute 'reset'
Could you help me solve this?Thank you very much.

[zyk516]

For the AttributeError: 'NoneType' object has no attribute 'reset', I solved this by doing these things below.
1.add ' trainer_config['evaluation_interval'] = True
trainer.setup(trainer_config)'
after 'trainer = trainer_cls(env=tune_config["env"], config=trainer_config)' in evaluate.py
2.replace 'workers' with 'evaluation_workers' in funtion rollout in utils/rollout.py, for example
env = trainer.evaluation_workers.local_worker().env
After that, there is no error when I run evaluate.py
Could you tell me what I do is right or not.

[dineshresearch]

@RutvikGupta @Gamenot I have used the https://github.com/huawei-noah/SMARTS/tree/marl_benchmark_path_fixes branch and followed the readme instructions to run the MARL_Benchmark

python3.7 run.py scenarios/intersections/4lane -f agents/ppo/baseline-lane-control.yaml --headless

but I have landed up into the below issue

Desired SMARTS version
[0.4.18]

Operating System
[ ubuntu 18.04]

"ray[rllib]==1.0.1.post1"
"gym==0.18.3",
"numpy==1.18.5",
"scipy == 1.4.1"

Problem
(pid=9883) 2021-11-25 22:58:04,598 INFO common.py:779 -- episode 1174638979 started
2021-11-25 22:58:04,825 ERROR trial_runner.py:793 -- Trial PPO_FrameStack_e8a3b_00000: Error processing event.
Traceback (most recent call last):
File "/home/rnd-kdinesh1/anaconda3/envs/marl1/lib/python3.7/site-packages/ray/tune/trial_runner.py", line 726, in _process_trial
result = self.trial_executor.fetch_result(trial)
File "/home/rnd-kdinesh1/anaconda3/envs/marl1/lib/python3.7/site-packages/ray/tune/ray_trial_executor.py", line 489, in fetch_result
result = ray.get(trial_future[0], timeout=DEFAULT_GET_TIMEOUT)
File "/home/rnd-kdinesh1/anaconda3/envs/marl1/lib/python3.7/site-packages/ray/worker.py", line 1452, in get
raise value.as_instanceof_cause()
ray.exceptions.RayTaskError(ValueError): ray::PPO.train() (pid=9874, ip=10.112.115.159)
File "python/ray/_raylet.pyx", line 482, in ray._raylet.execute_task
File "python/ray/_raylet.pyx", line 436, in ray._raylet.execute_task.function_executor
File "/home/rnd-kdinesh1/anaconda3/envs/marl1/lib/python3.7/site-packages/ray/rllib/agents/trainer.py", line 517, in train
raise e
File "/home/rnd-kdinesh1/anaconda3/envs/marl1/lib/python3.7/site-packages/ray/rllib/agents/trainer.py", line 506, in train
result = Trainable.train(self)
File "/home/rnd-kdinesh1/anaconda3/envs/marl1/lib/python3.7/site-packages/ray/tune/trainable.py", line 336, in train
result = self.step()
File "/home/rnd-kdinesh1/anaconda3/envs/marl1/lib/python3.7/site-packages/ray/rllib/agents/trainer_template.py", line 147, in step
res = next(self.train_exec_impl)
File "/home/rnd-kdinesh1/anaconda3/envs/marl1/lib/python3.7/site-packages/ray/util/iter.py", line 756, in next
return next(self.built_iterator)
File "/home/rnd-kdinesh1/anaconda3/envs/marl1/lib/python3.7/site-packages/ray/util/iter.py", line 783, in apply_foreac
File "/home/rnd-kdinesh1/anaconda3/envs/marl1/lib/python3.7/site-packages/ray/util/iter.py", line 783, in apply_foreach
for item in it:
File "/home/rnd-kdinesh1/anaconda3/envs/marl1/lib/python3.7/site-packages/ray/util/iter.py", line 783, in apply_foreach
for item in it:
File "/home/rnd-kdinesh1/anaconda3/envs/marl1/lib/python3.7/site-packages/ray/util/iter.py", line 843, in apply_filter
for item in it:
File "/home/rnd-kdinesh1/anaconda3/envs/marl1/lib/python3.7/site-packages/ray/util/iter.py", line 843, in apply_filter
for item in it:
File "/home/rnd-kdinesh1/anaconda3/envs/marl1/lib/python3.7/site-packages/ray/util/iter.py", line 783, in apply_foreach
for item in it:
File "/home/rnd-kdinesh1/anaconda3/envs/marl1/lib/python3.7/site-packages/ray/util/iter.py", line 783, in apply_foreach
for item in it:
File "/home/rnd-kdinesh1/anaconda3/envs/marl1/lib/python3.7/site-packages/ray/util/iter.py", line 783, in apply_foreach
for item in it:
[Previous line repeated 1 more time]
File "/home/rnd-kdinesh1/anaconda3/envs/marl1/lib/python3.7/site-packages/ray/util/iter.py", line 876, in apply_flatten
for item in it:
File "/home/rnd-kdinesh1/anaconda3/envs/marl1/lib/python3.7/site-packages/ray/util/iter.py", line 828, in add_wait_hooks
item = next(it)
File "/home/rnd-kdinesh1/anaconda3/envs/marl1/lib/python3.7/site-packages/ray/util/iter.py", line 783, in apply_foreach
for item in it:
File "/home/rnd-kdinesh1/anaconda3/envs/marl1/lib/python3.7/site-packages/ray/util/iter.py", line 783, in apply_foreach
for item in it:
File "/home/rnd-kdinesh1/anaconda3/envs/marl1/lib/python3.7/site-packages/ray/util/iter.py", line 783, in apply_foreach
for item in it:
[Previous line repeated 1 more time]
File "/home/rnd-kdinesh1/anaconda3/envs/marl1/lib/python3.7/site-packages/ray/util/iter.py", line 471, in base_iterator
yield ray.get(futures, timeout=timeout)
ray.exceptions.RayTaskError(ValueError): ray::RolloutWorker.par_iter_next() (pid=9883, ip=10.112.115.159)
File "python/ray/_raylet.pyx", line 482, in ray._raylet.execute_task
File "python/ray/_raylet.pyx", line 436, in ray._raylet.execute_task.function_executor
File "/home/rnd-kdinesh1/anaconda3/envs/marl1/lib/python3.7/site-packages/ray/util/iter.py", line 1152, in par_iter_next
return next(self.local_it)
File "/home/rnd-kdinesh1/anaconda3/envs/marl1/lib/python3.7/site-packages/ray/rllib/evaluation/rollout_worker.py", line 317, in gen_rollouts
yield self.sample()
File "/home/rnd-kdinesh1/anaconda3/envs/marl1/lib/python3.7/site-packages/ray/rllib/evaluation/rollout_worker.py", line 621, in sample
batches = [self.input_reader.next()]
File "/home/rnd-kdinesh1/anaconda3/envs/marl1/lib/python3.7/site-packages/ray/rllib/evaluation/sampler.py", line 94, in next
batches = [self.get_data()]
File "/home/rnd-kdinesh1/anaconda3/envs/marl1/lib/python3.7/site-packages/ray/rllib/evaluation/sampler.py", line 211, in get_data

item = next(self.rollout_provider) [41/1879]
File "/home/rnd-kdinesh1/anaconda3/envs/marl1/lib/python3.7/site-packages/ray/rllib/evaluation/sampler.py", line 646, in _env_runner
base_env.send_actions(actions_to_send)
File "/home/rnd-kdinesh1/anaconda3/envs/marl1/lib/python3.7/site-packages/ray/rllib/env/base_env.py", line 399, in send_actions
obs, rewards, dones, infos = env.step(agent_dict)
File "/home/rnd-kdinesh1/Desktop/Dinesh_San/SMARTS_MARL1/SMARTS/baselines/marl_benchmark/marl_benchmark/wrappers/rllib/frame_stack.py", line 195, in step
rewards = self._get_rewards(env_observations, env_rewards)
File "/home/rnd-kdinesh1/Desktop/Dinesh_San/SMARTS_MARL1/SMARTS/baselines/marl_benchmark/marl_benchmark/wrappers/rllib/frame_stack.py", line 172, in _get_rewards
rewards[k] = self.reward_adapter(list(self.frames[k]), env_rewards[k])
File "/home/rnd-kdinesh1/Desktop/Dinesh_San/SMARTS_MARL1/SMARTS/baselines/marl_benchmark/marl_benchmark/wrappers/rllib/frame_stack.py", line 241, in func
goal_dist = distance.euclidean(ego_2d_position, goal_position)
File "/home/rnd-kdinesh1/anaconda3/envs/marl1/lib/python3.7/site-packages/scipy/spatial/distance.py", line 620, in euclidean
return minkowski(u, v, p=2, w=w)
File "/home/rnd-kdinesh1/anaconda3/envs/marl1/lib/python3.7/site-packages/scipy/spatial/distance.py", line 512, in minkowski
u_v = u - v
ValueError: operands could not be broadcast together with shapes (2,) (3,)
2021-11-25 22:58:04,827 INFO trial_runner.py:964 -- Trial PPO_FrameStack_e8a3b_00000: Attempting to restore trial state from last checkpoint.
== Status ==

Can you please help with this soon?

[zyk516]

Hello.@Gamenot,Could you help me please?

[dineshresearch]

Hello.@RutvikGupta ,@Gamenot .The same error as #340 happened when I run the command above.The ray version is 1.0.1post, and smarts is 0.4.18, the code branch is SMARTS-marl_benchmark_path_fixes(When I used master branch, the error happened too.)How to solve this?I wish your replay.Thank you.

Hello @Gamenot @RutvikGupta @zyk516 I am also facing the same issue using the same SMARTS-marl_benchmark_path_fixes branch with ray version is 1.0.1post, and smarts is 0.4.18. This is a blocker for many i guess please kindly resolve soon

[RutvikGupta]

@RutvikGupta @Gamenot I have used the https://github.com/huawei-noah/SMARTS/tree/marl_benchmark_path_fixes branch and followed the readme instructions to run the MARL_Benchmark

python3.7 run.py scenarios/intersections/4lane -f agents/ppo/baseline-lane-control.yaml --headless

but I have landed up into the below issue

Desired SMARTS version [0.4.18]

Operating System [ ubuntu 18.04]

"ray[rllib]==1.0.1.post1" "gym==0.18.3", "numpy==1.18.5", "scipy == 1.4.1"

Problem (pid=9883) 2021-11-25 22:58:04,598 INFO common.py:779 -- episode 1174638979 started 2021-11-25 22:58:04,825 ERROR trial_runner.py:793 -- Trial PPO_FrameStack_e8a3b_00000: Error processing event. Traceback (most recent call last): File "/home/rnd-kdinesh1/anaconda3/envs/marl1/lib/python3.7/site-packages/ray/tune/trial_runner.py", line 726, in _process_trial result = self.trial_executor.fetch_result(trial) File "/home/rnd-kdinesh1/anaconda3/envs/marl1/lib/python3.7/site-packages/ray/tune/ray_trial_executor.py", line 489, in fetch_result result = ray.get(trial_future[0], timeout=DEFAULT_GET_TIMEOUT) File "/home/rnd-kdinesh1/anaconda3/envs/marl1/lib/python3.7/site-packages/ray/worker.py", line 1452, in get raise value.as_instanceof_cause() ray.exceptions.RayTaskError(ValueError): ray::PPO.train() (pid=9874, ip=10.112.115.159) File "python/ray/_raylet.pyx", line 482, in ray._raylet.execute_task File "python/ray/_raylet.pyx", line 436, in ray._raylet.execute_task.function_executor File "/home/rnd-kdinesh1/anaconda3/envs/marl1/lib/python3.7/site-packages/ray/rllib/agents/trainer.py", line 517, in train raise e File "/home/rnd-kdinesh1/anaconda3/envs/marl1/lib/python3.7/site-packages/ray/rllib/agents/trainer.py", line 506, in train result = Trainable.train(self) File "/home/rnd-kdinesh1/anaconda3/envs/marl1/lib/python3.7/site-packages/ray/tune/trainable.py", line 336, in train result = self.step() File "/home/rnd-kdinesh1/anaconda3/envs/marl1/lib/python3.7/site-packages/ray/rllib/agents/trainer_template.py", line 147, in step res = next(self.train_exec_impl) File "/home/rnd-kdinesh1/anaconda3/envs/marl1/lib/python3.7/site-packages/ray/util/iter.py", line 756, in next return next(self.built_iterator) File "/home/rnd-kdinesh1/anaconda3/envs/marl1/lib/python3.7/site-packages/ray/util/iter.py", line 783, in apply_foreac File "/home/rnd-kdinesh1/anaconda3/envs/marl1/lib/python3.7/site-packages/ray/util/iter.py", line 783, in apply_foreach for item in it: File "/home/rnd-kdinesh1/anaconda3/envs/marl1/lib/python3.7/site-packages/ray/util/iter.py", line 783, in apply_foreach for item in it: File "/home/rnd-kdinesh1/anaconda3/envs/marl1/lib/python3.7/site-packages/ray/util/iter.py", line 843, in apply_filter for item in it: File "/home/rnd-kdinesh1/anaconda3/envs/marl1/lib/python3.7/site-packages/ray/util/iter.py", line 843, in apply_filter for item in it: File "/home/rnd-kdinesh1/anaconda3/envs/marl1/lib/python3.7/site-packages/ray/util/iter.py", line 783, in apply_foreach for item in it: File "/home/rnd-kdinesh1/anaconda3/envs/marl1/lib/python3.7/site-packages/ray/util/iter.py", line 783, in apply_foreach for item in it: File "/home/rnd-kdinesh1/anaconda3/envs/marl1/lib/python3.7/site-packages/ray/util/iter.py", line 783, in apply_foreach for item in it: [Previous line repeated 1 more time] File "/home/rnd-kdinesh1/anaconda3/envs/marl1/lib/python3.7/site-packages/ray/util/iter.py", line 876, in apply_flatten for item in it: File "/home/rnd-kdinesh1/anaconda3/envs/marl1/lib/python3.7/site-packages/ray/util/iter.py", line 828, in add_wait_hooks item = next(it) File "/home/rnd-kdinesh1/anaconda3/envs/marl1/lib/python3.7/site-packages/ray/util/iter.py", line 783, in apply_foreach for item in it: File "/home/rnd-kdinesh1/anaconda3/envs/marl1/lib/python3.7/site-packages/ray/util/iter.py", line 783, in apply_foreach for item in it: File "/home/rnd-kdinesh1/anaconda3/envs/marl1/lib/python3.7/site-packages/ray/util/iter.py", line 783, in apply_foreach for item in it: [Previous line repeated 1 more time] File "/home/rnd-kdinesh1/anaconda3/envs/marl1/lib/python3.7/site-packages/ray/util/iter.py", line 471, in base_iterator yield ray.get(futures, timeout=timeout) ray.exceptions.RayTaskError(ValueError): ray::RolloutWorker.par_iter_next() (pid=9883, ip=10.112.115.159) File "python/ray/_raylet.pyx", line 482, in ray._raylet.execute_task File "python/ray/_raylet.pyx", line 436, in ray._raylet.execute_task.function_executor File "/home/rnd-kdinesh1/anaconda3/envs/marl1/lib/python3.7/site-packages/ray/util/iter.py", line 1152, in par_iter_next return next(self.local_it) File "/home/rnd-kdinesh1/anaconda3/envs/marl1/lib/python3.7/site-packages/ray/rllib/evaluation/rollout_worker.py", line 317, in gen_rollouts yield self.sample() File "/home/rnd-kdinesh1/anaconda3/envs/marl1/lib/python3.7/site-packages/ray/rllib/evaluation/rollout_worker.py", line 621, in sample batches = [self.input_reader.next()] File "/home/rnd-kdinesh1/anaconda3/envs/marl1/lib/python3.7/site-packages/ray/rllib/evaluation/sampler.py", line 94, in next batches = [self.get_data()] File "/home/rnd-kdinesh1/anaconda3/envs/marl1/lib/python3.7/site-packages/ray/rllib/evaluation/sampler.py", line 211, in get_data

item = next(self.rollout_provider) [41/1879] File "/home/rnd-kdinesh1/anaconda3/envs/marl1/lib/python3.7/site-packages/ray/rllib/evaluation/sampler.py", line 646, in _env_runner base_env.send_actions(actions_to_send) File "/home/rnd-kdinesh1/anaconda3/envs/marl1/lib/python3.7/site-packages/ray/rllib/env/base_env.py", line 399, in send_actions obs, rewards, dones, infos = env.step(agent_dict) File "/home/rnd-kdinesh1/Desktop/Dinesh_San/SMARTS_MARL1/SMARTS/baselines/marl_benchmark/marl_benchmark/wrappers/rllib/frame_stack.py", line 195, in step rewards = self._get_rewards(env_observations, env_rewards) File "/home/rnd-kdinesh1/Desktop/Dinesh_San/SMARTS_MARL1/SMARTS/baselines/marl_benchmark/marl_benchmark/wrappers/rllib/frame_stack.py", line 172, in _get_rewards rewards[k] = self.reward_adapter(list(self.frames[k]), env_rewards[k]) File "/home/rnd-kdinesh1/Desktop/Dinesh_San/SMARTS_MARL1/SMARTS/baselines/marl_benchmark/marl_benchmark/wrappers/rllib/frame_stack.py", line 241, in func goal_dist = distance.euclidean(ego_2d_position, goal_position) File "/home/rnd-kdinesh1/anaconda3/envs/marl1/lib/python3.7/site-packages/scipy/spatial/distance.py", line 620, in euclidean return minkowski(u, v, p=2, w=w) File "/home/rnd-kdinesh1/anaconda3/envs/marl1/lib/python3.7/site-packages/scipy/spatial/distance.py", line 512, in minkowski u_v = u - v ValueError: operands could not be broadcast together with shapes (2,) (3,) 2021-11-25 22:58:04,827 INFO trial_runner.py:964 -- Trial PPO_FrameStack_e8a3b_00000: Attempting to restore trial state from last checkpoint. == Status ==

Can you please help with this soon?

The above error is fixed in this PR: #1126. Please use this PR branch to setup and run benchmark.

[Adaickalavan]

This issue is being closed as the marl_benchmark has been archived and is not expected to receive any further updates. See #2025.