Update to newest RLlib version?

[ardian-selmonaj]

Is it planned to update MARLlib to the newest versions of Ray RLlib, PyTorch and so on? I mean Ray RLlib 1.8. is really old... and there is already PyTorch 2.2.

[Aequatio-Space]

I thought about it while developing my code, but soon discovered it requires some time since the code structure for the newest RLlib has changed, which means 2 major changes (most of them boilerplate changes) to the codebase:

1. a lot of imports need to be modified, such as the "agents" folder no longer exists, it is now "algorithm".
2. All MARLlib trainers are based on a class construction method build_trainer, which has been discarded since ray 1.9 (see https://docs.ray.io/en/latest/rllib/package_ref/algorithm.html#building-custom-algorithm-classes) , so all trainers need to be rewritten.

Therefore, the update may heavily depend on the availability and time of developers, let us remain hopeful :)

[ardian-selmonaj]

Thank you for the answer. I was just thinking if there might be improvements in reliability and efficiency.

I installed MARLlib through pip install marllib and manually installed ray and torch. BUT:
There is an error when trying to install the packages with the current specifications: pip installl ray==1.8.0 is not available as well as torch==1.9.0. Updating this to the closest new version ray==1.13.0 and torch==1.11.0 does not even run the simple example code provided. Please help!

[Aequatio-Space]

I understand your concern now. I remember installing the environment with torch==2.1.0 and ray==1.8.0 with no problem. (First the torch, then the ray). I downgraded from the latest ray to 1.8.0. Installation should also include ray[rllib] and ray[tune].

I'm not sure why 1.8.0 is not available in your environment, since the versions I fetched from my environment have 1.8.0 presented:
(from versions: 1.4.1, 1.5.0, 1.5.1, 1.5.2, 1.6.0, 1.7.0rc0, 1.7.0, 1.8.0, 1.9.0rc1, 1.9.0rc2, 1.9.0, 1.9.1rc0, 1.9.1, 1.9.2, 1.10.0rc0, 1.10.0, 1.11.0rc0, 1.11.0rc1, 1.11.0, 1.11.1, 1.12.0rc1, 1.12.0, 1.12.1, 1.13.0, 2.0.0rc0, 2.0.0rc1, 2.0.0, 2.0.1, 2.1.0, 2.2.0, 2.3.0rc0, 2.3.0, 2.3.1, 2.4.0, 2.5.0, 2.5.1, 2.6.0, 2.6.1, 2.6.2, 2.6.3, 2.7.0rc0, 2.7.0, 2.7.1, 2.7.2, 2.8.0, 2.8.1, 2.9.0, 2.9.1, 2.9.2)
You can try changing the index source for the pip, which may resolve the problem.

Unfortunately I forgot the exact installation procedure, and further help may require some time to reproduce your situation.

[Yilgrimage]

i meet some bug on old version of RLlib when i extend new env on marllib, and new version of RLlib is also more intuitive，really hope they can update it

[Aequatio-Space]

A makeshift solution is to copy the selected source code from the newest RLlib and put it in the "marllib/patch" directory, manually replacing the hard link from rllib with add_patch.py or command line. It requires pip install -e . though, which may create some inconvenience.