Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adding RLlib compatible Multi-agent environment #895

Open
wants to merge 13 commits into
base: master
Choose a base branch
from

Conversation

elpollouk
Copy link
Contributor

Added a new TurnBasedRllibMultiAgentEnv environment implementation that is compatible with RLlib's multi-agent environment. This environment wraps multiple MalmoEnv environment instances and sends steps to them in turn based sequence to work with multiple Minecraft agents operating in a single gameplay session.

In addition to the TurnBasedRllibMultiAgentEnv environment, I've also added SyncRllibMultiAgentEnv to sync Malmo's actions with their resultant observations. This is managed by sending an idle step request to each Minecraft instance after each real step request to query the resultant state of the environment.

Finally, I've included a new launcher.py script that can be used either directly or by importing into training scripts to launch multiple Minecraft instances. The script clones the Malmo directory into a new temporary directory and launches Minecraft from the new copy. This avoids issues with running multiple instances of Minecraft from the same directory causing conflicts as they try to update the same files.

@martinballa
Copy link

Is it possible to add a new bash file for running the Malmo instances headless?
It should be in 'Minecraft/launchClient_headless.sh' and it should contain the following 2 lines:

#! /bin/bash
xvfb-run -a -e /dev/stdout -s '-screen 0 640x480x16' ./launchClient.sh -port $1 -env > ../out.txt 2>&1

@martinballa
Copy link

I have been working with Adrian's updates for a while and it is much easier to run Malmo, especially on clusters. Unfortunately, when Malmo crashes the launcher keeps hanging and only says "Waiting for N instances..." instead of throwing an exception with an error message. The problem is that each Malmo instance runs on their own process and do not send their output back to the main thread.

@martinballa
Copy link

Another minor issue I had with the launcher is when my run crashes the Malmo instances keep running and I have to manually kill the java processes. Or in some cases it would be great to keep the instances as it takes a few minutes per instance to do a full start-up. If somebody would continue working on this I think these features would be great additions to Malmo. I wanted to make them as issues, but this PR has not been approved yet.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants