Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Windows] using ray.init(), unknown error occurred when packaging the program to an exe using cx_Freeze #38405

Closed
Amier2 opened this issue Aug 14, 2023 · 4 comments
Assignees
Labels
bug Something that is supposed to be working; but isn't @external-author-action-required Alternate tag for PRs where the author doesn't have labeling permission. QS Quantsight triage label windows

Comments

@Amier2
Copy link

Amier2 commented Aug 14, 2023

What happened + What you expected to happen

This is the simple code I use for packaging. Running the exe after packaging results in an error
use_main.py

import ray
print('更新算法')
ray.init()
print('ray.init完成')

This code runs fine directly, but when it's packaged and executed as 'use_main.exe', an error occurs
Moreover, it causes my computer to become very sluggish as it keeps generating processes incessantly

D:\agent_ray\build\exe.win-amd64-3.8>use_main.exe
更新算法
2023-08-14 22:26:44,535 ERROR services.py:1207 -- Failed to start the dashboard
2023-08-14 22:26:44,535 ERROR services.py:1232 -- Error should be written to 'dashboard.log' or 'dashboard.err'. We are printing the last 20 lines for you. See 'https://docs.ray.io/en/master/ray-observability/ray-logging.html#logging-directory-structure' to find where the log file is.
2023-08-14 22:26:44,536 ERROR services.py:1242 -- Couldn't read dashboard.log file. Error: [Errno 2] No such file or directory: 'C:\\Users\\r7318\\AppData\\Local\\Temp\\ray\\session_2023-08-14_22-26-14_649513_28428\\logs\\dashboard.log'. It means the dashboard is broken even before it initializes the logger (mostly dependency issues). Reading the dashboard.err file which contains stdout/stderr.
2023-08-14 22:26:44,537 ERROR services.py:1276 -- Failed to read dashboard.err file: cannot mmap an empty file. It is unexpected. Please report an issue to Ray github. https://github.com/ray-project/ray/issues
2023-08-14 22:26:55,485 INFO worker.py:1621 -- Started a local Ray instance.
[2023-08-14 22:28:33,456 E 28428 28256] core_worker.cc:201: Failed to register worker 01000000ffffffffffffffffffffffffffffffffffffffffffffffff to Raylet. IOError: [RayletClient] Unable to register worker with raylet. Unknown error

What could be the cause of this issue?

event_GCS.log

{"custom_fields":{"ip":"127.0.0.1","node_id":"55df57e00934f09296afdb5540dac8130a46d25f8cb5766dbd33ab4c"},"event_id":"5c79a3594a4a54e8db955ed91922cc87997f","host_name":"LAPTOP-0EMEUIMI","label":"RAY_NODE_REMOVED","message":"The node with node id: 55df57e00934f09296afdb5540dac8130a46d25f8cb5766dbd33ab4c and address: 127.0.0.1 and node name: 127.0.0.1 has been marked dead because the detector has missed too many heartbeats from it. This can happen when a \t(1) raylet crashes unexpectedly (OOM, preempted node, etc.) \\n\t(2) raylet has lagging heartbeats due to slow network or busy workload.","pid":"22804","severity":"ERROR","source_type":"GCS","timestamp":1692023329}

Versions / Dependencies

Python 3.8.17 | packaged by conda-forge | (default, Jun 16 2023, 07:01:59) [MSC v.1929 64 bit (AMD64)] on win32
cx_Freeze==6.15.5
ray==2.6.2

Reproduction script

use_main.py

import ray
print('更新算法')
ray.init()
print('ray.init完成')

setup.py

#coding=utf-8
from cx_Freeze import setup,Executable
setup(
    name="test",
    version="1.0",
    description="Test application",
    author="none",
    executables=[Executable("use_main.py")]
)

run python setup.py build

replace the build/exe.win-amd64-3.8/lib/ray folder with the site-packages/ray folder from the python environment you're using. If you encounter errors indicating that certain packages are missing, similarly copy them from the environment to 'build/exe.win-amd64-3.8/lib

then run use_main.exe, an unexpected error occur

Issue Severity

Medium: It is a significant difficulty but I can work around it.

@Amier2 Amier2 added bug Something that is supposed to be working; but isn't triage Needs triage (eg: priority, bug/not-bug, and owning component) labels Aug 14, 2023
@HamidShojanazeri
Copy link

Same here, with ray 2.6.2, tried to make sure about pydantic ( 1.10.12) version as suggested here too, didn't help.

@ArturNiederfahrenhorst ArturNiederfahrenhorst changed the title [<Ray component: Core|RLlib|etc...>] using ray.init(), unknown error occurred when packaging the program to an exe using cx_Freeze [Core] using ray.init(), unknown error occurred when packaging the program to an exe using cx_Freeze Aug 14, 2023
@jjyao jjyao added core Issues that should be addressed in Ray Core QS Quantsight triage label labels Aug 14, 2023
@jjyao
Copy link
Collaborator

jjyao commented Aug 14, 2023

@mattip could you see if you can reproduce it?

@jjyao jjyao removed the core Issues that should be addressed in Ray Core label Aug 14, 2023
@mattip
Copy link
Contributor

mattip commented Aug 15, 2023

I see these lines in the error report:

Couldn't read dashboard.log file. Error: [Errno 2] No such file or directory: 'C:\\Users\\r7318\\AppData\\Local\\Temp\\ray\\session_2023-08-14_22-26-14_649513_28428\\logs\\dashboard.log'. It means the dashboard is broken even before it initializes the logger (mostly dependency issues). Reading the dashboard.err file which contains stdout/stderr.

Failed to read dashboard.err file: cannot mmap an empty file. It is unexpected.

So the dashboard is not starting up, probably missing some dependencies. Are there any errors in the log files? You can search for Traceback.

replace the build/exe.win-amd64-3.8/lib/ray folder with the site-packages/ray folder from the python environment you're using. If you encounter errors indicating that certain packages are missing, similarly copy them from the environment to 'build/exe.win-amd64-3.8/lib

I am not a cx_freeze expert, but this does not sound reproducible. What exactly did you copy? There should be a way to add all the dependencies to the script used to run cx_feeze.

Maybe you will have more luck running without the dashboard: ray.init(include_dashboard=False)

@rkooo567 rkooo567 changed the title [Core] using ray.init(), unknown error occurred when packaging the program to an exe using cx_Freeze [Windows] using ray.init(), unknown error occurred when packaging the program to an exe using cx_Freeze Aug 21, 2023
@anyscalesam
Copy link
Contributor

anyscalesam commented Sep 26, 2023

@Amier2 can you please advise here?

UPDATE: 11/8 as no response; please re-open if this issue is still occuring on latest ray (28 at time of this message).

@anyscalesam anyscalesam added the @external-author-action-required Alternate tag for PRs where the author doesn't have labeling permission. label Oct 2, 2023
@anyscalesam anyscalesam removed the triage Needs triage (eg: priority, bug/not-bug, and owning component) label Dec 4, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something that is supposed to be working; but isn't @external-author-action-required Alternate tag for PRs where the author doesn't have labeling permission. QS Quantsight triage label windows
Projects
None yet
Development

No branches or pull requests

6 participants