Getting IOError: could not get source code when defining an @ray.remote function

[julienforgeat]

Hi,

I am running Ray in a Docker container, I can run python test/runtest.py but when I try to actually write code, I am getting IOError as follow:

>>> import ray
>>> ray.init(num_workers=1)
1932:M 24 Mar 22:20:50.149 # WARNING: The TCP backlog setting of 511 cannot be enforced because /proc/sys/net/core/somaxconn is set to the lower value of 128.
1932:M 24 Mar 22:20:50.149 # Server started, Redis version 3.9.102
1932:M 24 Mar 22:20:50.149 # WARNING overcommit_memory is set to 0! Background save may fail under low memory condition. To fix this issue add 'vm.overcommit_memory = 1' to /etc/sysctl.conf and then reboot or run the command 'sysctl vm.overcommit_memory=1' for this to take effect.
1932:M 24 Mar 22:20:50.149 # WARNING you have Transparent Huge Pages (THP) support enabled in your kernel. This will create latency and memory usage issues with Redis. To fix this issue run the command 'echo never > /sys/kernel/mm/transparent_hugepage/enabled' as root, and add it to your /etc/rc.local in order to retain the setting after a reboot. Redis must be restarted after THP is disabled.
Waiting for redis server at 127.0.0.1:54264 to respond...
Warning: Reducing object store memory because /dev/shm has only 536870912 bytes available. You may be able to free up space by deleting files in /dev/shm. If you are inside a Docker container, you may need to pass an argument with the flag '--shm-size' to 'docker run'.
[INFO] (/ray/src/plasma/plasma_store.cc:908) Allowing the Plasma store to use up to 0.43GB of memory.
[INFO] (/ray/src/local_scheduler/local_scheduler.cc:242) Started worker with pid 1946
Not starting the web UI because the web UI requires Python 3.
{'object_store_addresses': [ObjectStoreAddress(name='/tmp/plasma_store11252270', manager_name='/tmp/plasma_manager19950947', manager_port=14602)], 'redis_address': '127.0.0.1:54264', 'local_scheduler_socket_names': ['/tmp/scheduler11107798'], 'node_ip_address': '127.0.0.1'}
>>> 
>>> @ray.remote
... def f(x):
...   return x
... 
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/opt/conda/lib/python2.7/site-packages/ray-0.0.1-py2.7.egg/ray/worker.py", line 2104, in remote
    return make_remote_decorator(num_return_vals, num_cpus, num_gpus)(args[0])
  File "/opt/conda/lib/python2.7/site-packages/ray-0.0.1-py2.7.egg/ray/worker.py", line 2014, in remote_decorator
    function_id_hash.update(inspect.getsource(func).encode("ascii"))
  File "/opt/conda/lib/python2.7/inspect.py", line 701, in getsource
    lines, lnum = getsourcelines(object)
  File "/opt/conda/lib/python2.7/inspect.py", line 690, in getsourcelines
    lines, lnum = findsource(object)
  File "/opt/conda/lib/python2.7/inspect.py", line 538, in findsource
    raise IOError('could not get source code')
IOError: could not get source code

Maybe there is something wrong with my env or I did something stupid during setup and I will retry from scratch but if by chance the error is familiar and you can provide a pointer, that is very much appreciated.

[robertnishihara]

Ah this is a bug #349. I'll fix this (I'm a bit surprised we haven't run into this more).

As a quick work around, it should work inside of ipython. E.g., do pip install ipython and then start up ipython at the command line, and then try the same sequence of commands.

[julienforgeat]

Thanks, this is working now 👍

I did a search before reporting but for some reason, my environment reports IOError while the issue you linked is giving OSError.

[robertnishihara]

Oh that's good to know! It looks like this is a Python 2 versus 3 difference.