An error occurs when running Evaluation

[rsuwa]

Reproduction

python evaluate.py scenarios/intersections/4lane -f agents/maddpg/baseline-lane-control.yaml --checkpoint ./log/results/run/4lane-4/MADDPG2_EarlyDone_c3c16_00000_0_2020-11-13_17-32-28/checkpoint_1200/checkpoint-1200

Error message

Traceback (most recent call last):
  File "evaluate.py", line 236, in <module>
    headless=args.headless,
  File "evaluate.py", line 114, in main
    trainer.restore(checkpoint)
  File "/home/ryota/src/github.com/huawei-noah/SMARTS/benchmark/.venv/lib/python3.7/site-packages/ray/tune/trainable.py", line 476, in restore
    self.load_checkpoint(checkpoint_path)
  File "/home/ryota/src/github.com/huawei-noah/SMARTS/benchmark/.venv/lib/python3.7/site-packages/ray/rllib/agents/trainer.py", line 673, in load_checkpoint
    self.__setstate__(extra_data)
  File "/home/ryota/src/github.com/huawei-noah/SMARTS/benchmark/.venv/lib/python3.7/site-packages/ray/rllib/agents/trainer_template.py", line 164, in __setstate__
    Trainer.__setstate__(self, state)
  File "/home/ryota/src/github.com/huawei-noah/SMARTS/benchmark/.venv/lib/python3.7/site-packages/ray/rllib/agents/trainer.py", line 1168, in __setstate__
    self.workers.local_worker().restore(state["worker"])
  File "/home/ryota/src/github.com/huawei-noah/SMARTS/benchmark/.venv/lib/python3.7/site-packages/ray/rllib/evaluation/rollout_worker.py", line 969, in restore
    self.policy_map[pid].set_state(state)
  File "/home/ryota/src/github.com/huawei-noah/SMARTS/benchmark/agents/maddpg/tf_policy.py", line 353, in set_state
    TFPolicy.set_state(self, state)
  File "/home/ryota/src/github.com/huawei-noah/SMARTS/benchmark/.venv/lib/python3.7/site-packages/ray/rllib/policy/tf_policy.py", line 443, in set_state
    optimizer_vars = state.pop("_optimizer_variables", None)
TypeError: pop() takes at most 1 argument (2 given)

Environment

Python 3.7.9
OS: Ubuntu 18.04.5 LTS
CPU: Ryzen 5 3600X
GPU: GeForce RTX 2070
CUDA: 10.0

pip list

Package                  Version      Location
------------------------ ------------ ---------------------------------------------
absl-py                  0.9.0
aiohttp                  3.6.2
aiohttp-cors             0.7.0
aioredis                 1.3.1
apipkg                   1.5
astor                    0.8.1
async-timeout            3.0.1
atari-py                 0.2.6
attrs                    19.3.0
Automat                  20.2.0
beautifulsoup4           4.8.2
blessings                1.7
cachetools               4.1.1
certifi                  2020.6.20
chardet                  3.0.4
click                    7.1.1
cloudpickle              1.3.0
colorama                 0.4.3
colorful                 0.6.0a1
commonmark               0.9.1
constantly               15.1.0
coverage                 5.2.1
cycler                   0.10.0
Cython                   0.29.21
decorator                4.4.2
dm-tree                  0.1.2
execnet                  1.7.1
filelock                 3.0.12
future                   0.18.2
gast                     0.2.2
google                   3.0.0
google-api-core          1.23.0
google-auth              1.23.0
google-pasta             0.2.0
googleapis-common-protos 1.52.0
gpustat                  0.6.0
grpcio                   1.27.2
gym                      0.17.2
h5py                     2.10.0
hiredis                  1.1.0
hyperlink                20.0.1
idna                     2.9
imageio                  2.9.0
importlib-metadata       1.6.0
incremental              17.5.0
joblib                   0.14.1
jsonpatch                1.25
jsonpointer              2.0
jsonschema               3.2.0
Keras-Applications       1.0.8
Keras-Preprocessing      1.1.0
kiwisolver               1.2.0
lz4                      3.0.2
Markdown                 3.2.1
matplotlib               3.3.2
more-itertools           8.2.0
msgpack                  1.0.0
multidict                4.7.5
networkx                 2.4
numpy                    1.18.2
nvidia-ml-py3            7.352.0
opencensus               0.7.11
opencensus-context       0.1.2
opencv-python            4.2.0.32
opencv-python-headless   4.2.0.32
opt-einsum               3.2.0
packaging                20.4
panda3d                  1.10.6.post2
panda3d-gltf             0.6
panda3d-simplepbr        0.4
pandas                   1.0.3
Pillow                   7.0.0
pip                      20.2.2
pluggy                   0.13.1
prometheus-client        0.8.0
protobuf                 3.11.3
psutil                   5.7.0
py                       1.8.1
py-cpuinfo               5.0.0
py-spy                   0.3.3
pyasn1                   0.4.8
pyasn1-modules           0.2.8
pybullet                 2.7.1
pyglet                   1.5.0
Pygments                 2.7.1
PyHamcrest               2.0.2
pynput                   1.6.8
pyparsing                2.4.6
pyrsistent               0.16.0
pytest                   5.4.1
pytest-benchmark         3.2.3
pytest-cov               2.10.0
pytest-forked            1.2.0
pytest-xdist             1.32.0
python-dateutil          2.8.1
python-xlib              0.27
pytz                     2019.3
PyWavelets               1.1.1
PyYAML                   5.3.1
pyzmq                    19.0.0
quadprog                 0.1.7
ray                      1.0.1
redis                    3.4.1
requests                 2.23.0
rich                     7.0.0
rsa                      4.6
Rtree                    0.9.4
scikit-image             0.17.2
scikit-learn             0.22.2.post1
scipy                    1.4.1
setuptools               49.6.0
sh                       1.12.14
Shapely                  1.7.0
six                      1.14.0
sklearn                  0.0
smarts                   0.4.3        /home/ryota/src/github.com/huawei-noah/SMARTS
soupsieve                2.0
supervisor               4.1.0
tableprint               0.9.0
tabulate                 0.8.7
tensorboard              1.15.0
tensorboardX             2.0
tensorflow-estimator     1.15.1
tensorflow-gpu           1.15.0
tensorflow-probability   0.8.0
termcolor                1.1.0
tifffile                 2020.9.3
torch                    1.3.0
torchfile                0.1.0
torchvision              0.4.1
tornado                  6.0.4
trimesh                  3.6.20
Twisted                  20.3.0
typing-extensions        3.7.4.3
urllib3                  1.25.8
visdom                   0.1.8.9
wcwidth                  0.1.9
websocket-client         0.57.0
Werkzeug                 1.0.0
wheel                    0.35.1
wrapt                    1.12.1
yarl                     1.4.2
yattag                   1.13.2
zipp                     3.1.0
zope.interface           5.1.0

[KornbergFresnel]

@rsuwa try ray==0.8.7. We gonna upgrade ray to v1.0 these days

[rsuwa]

@KornbergFresnel I tried ray==0.8.7 and got the same error.

[kaixqu]

Same here. What happened? Dict can have two parameters for pop, but list can only have one parameter. Is this some kind of bug from November 14, 2020?

[ssexuejinwei]

Same here. What happened? Did the checkpoint saved for maddpg have some bugs?

[Adaickalavan]

This issue is being closed as the marl_benchmark has been archived and is not expected to receive any further updates. See #2025.