Failed to start an environment with obs = env.reset() on macos #659

1931540 · 2022-08-09T08:29:26Z

import gym
import minerl
env = gym.make('MineRLBasaltFindCave-v0')

above code has no error

but got error with env.reset()

obs = env.reset()

I got the following error, and I cannot get minerl to work.

/Users/yubingquan/opt/anaconda3/envs/minerl2/lib/python3.8/runpy.py:127: RuntimeWarning: 'minerl.utils.process_watcher' found in sys.modules after import of package 'minerl.utils', but prior to execution of 'minerl.utils.process_watcher'; this may result in unpredictable behaviour
  warn(RuntimeWarning(msg))
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/yubingquan/opt/anaconda3/envs/minerl2/lib/python3.8/site-packages/gym/wrappers/time_limit.py", line 27, in reset
    return self.env.reset(**kwargs)
  File "/Users/yubingquan/opt/anaconda3/envs/minerl2/lib/python3.8/site-packages/minerl/herobraine/env_specs/basalt_specs.py", line 78, in reset
    return self.env.reset()
  File "/Users/yubingquan/opt/anaconda3/envs/minerl2/lib/python3.8/site-packages/minerl/herobraine/env_specs/basalt_specs.py", line 57, in reset
    return super().reset()
  File "/Users/yubingquan/opt/anaconda3/envs/minerl2/lib/python3.8/site-packages/gym/core.py", line 251, in reset
    return self.env.reset(**kwargs)
  File "/Users/yubingquan/opt/anaconda3/envs/minerl2/lib/python3.8/site-packages/minerl/env/_singleagent.py", line 22, in reset
    multi_obs = super().reset()
  File "/Users/yubingquan/opt/anaconda3/envs/minerl2/lib/python3.8/site-packages/minerl/env/_multiagent.py", line 436, in reset
    self._setup_instances()
  File "/Users/yubingquan/opt/anaconda3/envs/minerl2/lib/python3.8/site-packages/minerl/env/_multiagent.py", line 539, in _setup_instances
    self.instances.extend([f.result() for f in instance_futures])
  File "/Users/yubingquan/opt/anaconda3/envs/minerl2/lib/python3.8/site-packages/minerl/env/_multiagent.py", line 539, in <listcomp>
    self.instances.extend([f.result() for f in instance_futures])
  File "/Users/yubingquan/opt/anaconda3/envs/minerl2/lib/python3.8/concurrent/futures/_base.py", line 437, in result
    return self.__get_result()
  File "/Users/yubingquan/opt/anaconda3/envs/minerl2/lib/python3.8/concurrent/futures/_base.py", line 389, in __get_result
    raise self._exception
  File "/Users/yubingquan/opt/anaconda3/envs/minerl2/lib/python3.8/concurrent/futures/thread.py", line 57, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/Users/yubingquan/opt/anaconda3/envs/minerl2/lib/python3.8/site-packages/minerl/env/_multiagent.py", line 805, in _get_new_instance
    instance.launch(replaceable=self._is_fault_tolerant)
  File "/Users/yubingquan/opt/anaconda3/envs/minerl2/lib/python3.8/site-packages/minerl/env/malmo.py", line 427, in launch
    raise EOFError(
EOFError: /Users/yubingquan/opt/anaconda3/envs/minerl2/lib/python3.8/site-packages/minerl/env/../MCP-Reborn
SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
[15:58:21] [Render thread/INFO]: Environment: authHost='https://authserver.mojang.com', accountsHost='https://api.mojang.com', sessionHost='https://sessionserver.mojang.com', servicesHost='https://api.minecraftservices.com', name='PROD'
[15:58:22] [Render thread/ERROR]: Failed to verify authentication!
[15:58:22] [Render thread/INFO]: Setting user: Player657
[15:58:22] [Render thread/INFO]: Backend library: LWJGL version 3.2.1 SNAPSHOT
---- Minecraft Crash Report ----
// Why did you do that?

Time: 22-8-9 下午3:58
Description: Initializing game

java.lang.ExceptionInInitializerError
	at org.lwjgl.glfw.GLFW.glfwCreateWindow(GLFW.java:1842)
	at net.minecraft.client.MainWindow.<init>(MainWindow.java:91)
	at net.minecraft.client.renderer.VirtualScreen.create(VirtualScreen.java:21)
	at net.minecraft.client.Minecraft.<init>(Minecraft.java:408)
	at net.minecraft.client.main.Main.main(Main.java:154)
Caused by: java.lang.IllegalStateException: GLFW windows may only be created on the main thread and that thread must be the first thread in the process. Please run the JVM with -XstartOnFirstThread. For offscreen rendering, make sure another window toolkit (e.g. AWT or JavaFX) is initialized before GLFW.
	at org.lwjgl.glfw.EventLoop$OffScreen.<clinit>(EventLoop.java:39)
	... 5 more


A detailed walkthrough of the error, its code path and all known details is as follows:
---------------------------------------------------------------------------------------

-- Head --
Thread: Render thread
Stacktrace:
	at org.lwjgl.glfw.GLFW.glfwCreateWindow(GLFW.java:1842)
	at net.minecraft.client.MainWindow.<init>(MainWindow.java:91)
	at net.minecraft.client.renderer.VirtualScreen.create(VirtualScreen.java:21)
	at net.minecraft.client.Minecraft.<init>(Minecraft.java:408)

-- Initialization --
Details:
Stacktrace:
	at net.minecraft.client.main.Main.main(Main.java:154)

-- System Details --
Details:
	Minecraft Version: 1.16.5
	Minecraft Version ID: 1.16.5
	Operating System: Mac OS X (x86_64) version 10.16
	Java Version: 1.8.0_312, Azul Systems, Inc.
	Java VM Version: OpenJDK 64-Bit Server VM (mixed mode), Azul Systems, Inc.
	Memory: 534386904 bytes (509 MB) / 1507852288 bytes (1438 MB) up to 3817865216 bytes (3641 MB)
	CPUs: 8
	JVM Flags: 1 total; -Xmx4G
	Launched Version: ~~NULL~~
	Backend library: LWJGL version 3.2.1 SNAPSHOT
	Backend API: NO CONTEXT
	GL Caps:
	Using VBOs: Yes
	Is Modded: Very likely; Jar signature invalidated
	Type: Client (map_client.txt)
	CPU: <unknown>
#@!@# Game crashed! Crash report saved to: #@!@# /Users/yubingquan/opt/anaconda3/envs/minerl2/lib/python3.8/site-packages/minerl/MCP-Reborn/./crash-reports/crash-2022-08-09_15.58.23-client.txt


Minecraft process finished unexpectedly. There was an error with Malmo.

The text was updated successfully, but these errors were encountered:

1931540 · 2022-08-09T08:32:51Z

here's my conda env setting

name: minerl2
channels:
  - conda-forge
  - defaults
dependencies:
  - python=3.8
  - pip=22.1
  - openjdk=8.0.312
  - pip:
    - git+https://github.com/minerllabs/minerl

Miffyli · 2022-08-09T19:40:37Z

Hey! Are you running on a headless machine without display? MineRL/Minecraft requires a valid display buffer to be able to run. You can use xvfb for this with xvfb-run -a python [your-code] (on ubuntu installation is simple sudo apt install xvfb).

lewisxy · 2022-11-07T21:31:04Z

I got the exact same error on my MacBook. Any update to this? I don't think xvfb-run is available on MacOS.

Miffyli · 2022-11-07T23:54:39Z

Unfortunately MineRL v1.0 does not run on Mac at the moment, and no-one has had time to look into :(

lewisxy · 2022-11-08T04:59:40Z

OK, after hours of debug, I made it work on an MacBook Pro (with Apple Silicon M1 Pro) running MacOS 12.6.1. This solution SHOULD work on any Apple Silicon based Mac. Here is how I did it.

TLDR version

Download or clone the repository. This should create minerl directory.

git clone https://github.com/minerllabs/minerl.git

Modify the patch file by replacing all 3.2.1 to 3.3.1. This will update the version of lwjgl to 3.3.1.
Install the package using pip. Inside minerl directory, run the following command

pip install .

If installation failed, it could be the problem of JDK. The JDK I found working is the Oracle JDK 1.8, you will need to sign-in to download it. This JDK is for x86_64, so you will need Rosette to run it. Other people also has some success with OpenJDK.

If the previous step is successful, it should generate a directory called MCP-Reborn. Modify the launchClient.sh by adding -XstartOnFirstThread in the line that runs java ...... (somewhere close to the bottom of the file), so the new line becomes.

java -Xmx$maxMem -XstartOnFirstThread -jar $fatjar --envPort=$port

Note: you can also change the memory for JVM by changing the 4G defined in this file.

Modify the MCP-Reborn/src/main/java/net/minecraft/client/MainWindow.java. Comment out or remove the line

GLFW.glfwSetWindowIcon(this.handle, buffer);

Also comment out or remove the entire method body of checkGlfwError

public static void checkGlfwError(BiConsumer<Integer, String> glfwErrorConsumer) {
    // remove or comment out anything here
    // DO NOT remove this method declaration
}

Rebuild MCP-Reborn. Inside MCP-Reborn directory, run the following command

./gradlew clean build shadowJar

Overwrite the content inside MCP-Reborn directory in the install location with the content of MCP-Reborn we just modified. The exact install location depends on your python setup. It should be something like somepath/python3.x/site-packages/minerl/MCP-Reborn.
Now you should be able to run the example provided in official documentation. DO NOT run python inside the minerl directory we cloned, as it will mess up the python working directory.

Technical Details

lwjgl starts to support Apple Silicon after version 3.3.0. The latest version at the time of writing is 3.3.1. See their release page for detail.
-XstartOnFirstThread JVM arguments is hinted by the error message. It just took some time for me to figure out where is java called to start the minecraft instance.
This blog explains in detail why the modifications in MainWindow.java are necessary.

It's appreciated if someone can verify this works. As I tried so many things in attempt to fix this, I might forgot to write everything down.

Miffyli · 2022-11-08T18:54:44Z

@lewisxy Oooo this is amazing! @trigaten any chance you could try this? I could also try on my M1 Mac but atm quite busy. It would be awesome to have this stuff merged into code :). It does not look like we are going to sacrifice any functionality. We just need to make sure these changes would still work on Windows and Linux.

richwomanbtc · 2023-01-09T08:13:02Z

I executed the command according to #659 (comment) and confirmed that it works. Thanks @lewisxy!
Environment M1 Mac + python 3.9 + poetry
Commands

git clone https://github.com/minerllabs/minerl.git
sed -i .bak 's/3\.2\.1/3.3.1/' ./minerl/scripts/mcp_patch.diff
cd minerl && poetry run python setup.py
sed -i .bak s/'java -Xmx\$maxMem'/'java -Xmx\$maxMem -XstartOnFirstThread'/ ./minerl/MCP-Reborn/launchClient.sh
sed -i .bak /'GLFW.glfwSetWindowIcon(this.handle, buffer);'/d ./minerl/MCP-Reborn/src/main/java/net/minecraft/client/MainWindow.java
sed -i .bak '125,136s/^/\/\//' ./minerl/MCP-Reborn/src/main/java/net/minecraft/client/MainWindow.java
cd minerl/MCP-Reborn && ./gradlew clean build shadowJar
cd ../../../ && poetry add "git+https://github.com/minerllabs/minerl"
cp -rf minerrl/minerl/MCP-Reborn/* .venv/lib/python3.9/site-packages/minerl/MCP-Reborn/

jhsul · 2023-02-23T14:35:13Z

This worked and I was able to install minerl on my m2 mac (woot). However, python is having trouble finding the minerl.data module:

Traceback (most recent call last):
  File "...", line 2, in <module>
    import minerl.data
ModuleNotFoundError: No module named 'minerl.data'

Did you encounter this error at all?

lewisxy · 2023-02-24T02:03:46Z

I believe minerl.data only works for older version (<=0.4.x), and it is removed in newer version (>=1.0.x).

jhsul · 2023-02-24T19:43:31Z

Oh, I see. That'll do it. Thanks!

drozdj · 2023-05-23T10:06:37Z

@Miffyli can confirm that @lewisxy solution above has also worked for me after the mixin 05.22.23 update. M1 Pro chip

carolinezhang1 · 2024-08-18T05:27:07Z

@lewisxy Thank you so much for your helpful tutorial! I followed your steps but ended up getting this error: GLFW error 65548: Cocoa: Regular windows do not have icons on macOS. I was wondering if you have gotten an error like this before? I commented out the lines that you mentioned in MainWindow.java but I'm still getting the error. Do you have any ideas on how I could troubleshoot? Thank you so much!

Add -XstartOnFirstThread to launch command. See minerllabs#659 (comment)

This is the error: GLFW error 65548: Cocoa: Regular windows do not have icons on macOS. Please make sure you have up-to-date drivers (see aka.ms/mcdriver for instructions). Fix from minerllabs#659 (comment) See more info LWJGL/lwjgl3#695

1931540 changed the title ~~Failed to start an environment with obs = env.reset()~~ Failed to start an environment with obs = env.reset() on macos Aug 9, 2022

rk1a mentioned this issue Oct 5, 2022

Error on reset() when using wrapped MineRL env BASALT-2022-Karlsruhe/ka-basalt-2022#71

Closed

Miffyli mentioned this issue Feb 22, 2023

Installing minerl via pip fails during gradle build #693

Closed

jhsul mentioned this issue Feb 25, 2023

TypeError: string indices must be integers openai/Video-Pre-Training#25

Closed

Miffyli mentioned this issue Mar 22, 2023

Issues with creating my first agent #698

Closed

Miffyli mentioned this issue May 24, 2023

error obs=env.reset() // #705

Closed

Miffyli mentioned this issue Jun 19, 2023

Issue installing MineRL package on M2 Macbook #715

Closed

AsWali mentioned this issue Oct 7, 2023

Mac M1 conda env danijar/dreamerv3#30

Open

Miffyli mentioned this issue Dec 10, 2023

Installation error #741

Open

Miffyli mentioned this issue Apr 18, 2024

GLFW Issue openai/Video-Pre-Training#41

Closed

Miffyli mentioned this issue Apr 30, 2024

Unable to install MineRL openai/Video-Pre-Training#43

Open

Miffyli mentioned this issue Aug 20, 2024

Unable to run mineRL environment; zombie processes #781

Open

twoturtles added a commit to twoturtles/minerl that referenced this issue Oct 13, 2024

Update launch command

efa0bcd

Add -XstartOnFirstThread to launch command. See minerllabs#659 (comment)

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Failed to start an environment with obs = env.reset() on macos #659

Failed to start an environment with obs = env.reset() on macos #659

1931540 commented Aug 9, 2022 •

edited

Loading

1931540 commented Aug 9, 2022

Miffyli commented Aug 9, 2022

lewisxy commented Nov 7, 2022

Miffyli commented Nov 7, 2022

lewisxy commented Nov 8, 2022 •

edited

Loading

Miffyli commented Nov 8, 2022

richwomanbtc commented Jan 9, 2023

jhsul commented Feb 23, 2023

lewisxy commented Feb 24, 2023

jhsul commented Feb 24, 2023

drozdj commented May 23, 2023

carolinezhang1 commented Aug 18, 2024

Failed to start an environment with obs = env.reset() on macos #659

Failed to start an environment with obs = env.reset() on macos #659

Comments

1931540 commented Aug 9, 2022 • edited Loading

1931540 commented Aug 9, 2022

Miffyli commented Aug 9, 2022

lewisxy commented Nov 7, 2022

Miffyli commented Nov 7, 2022

lewisxy commented Nov 8, 2022 • edited Loading

TLDR version

Technical Details

Miffyli commented Nov 8, 2022

richwomanbtc commented Jan 9, 2023

jhsul commented Feb 23, 2023

lewisxy commented Feb 24, 2023

jhsul commented Feb 24, 2023

drozdj commented May 23, 2023

carolinezhang1 commented Aug 18, 2024

1931540 commented Aug 9, 2022 •

edited

Loading

lewisxy commented Nov 8, 2022 •

edited

Loading