-
-
Notifications
You must be signed in to change notification settings - Fork 24
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Downloading data with parfive>=2 inside a streamlit app fails with a RuntimeError #132
Comments
Without looking into it too much, I would assume streamlit is running the whole app in a thread pool of somekind, which is causing the error. I would be happy to accept a fix which turns a The only snag is, there's no automated testing for the behaviour in a notebook (which relies on threads itself) so we would need to make sure everything there is working as expected. You would get a lot of bonus points if you could figure out a way to reproduce this without streamlit (i.e. run a script in a thread pool executor or something), because then we could add it to our test suite. |
This sounds great and hank you for your comments, they helped me a lot. I think I have progressed in this. So what I did is the following: First, I tried to reproduce this withought streamlit. Well actually this was unexpectedly easier than I thought ;p This can be done by the following logic:
I use Now to resolve the issue the best I have so far is the following: I changed the static method of the parfive Downloader class into the following:
Doing this there is an I think I can use the first part to make a working test and the second to pass around the issue when the fucntion is called out of the main thread. What do you think of this? |
This sounds great! Please go ahead and open a PR and we can make any tweaks we need to there. |
I have opened a PR for this here #133 |
Closed by #133 |
Data download with parfive>=2.0.0 breaks inside streamlit app with the following error
RuntimeError: set_wakeup_fd only works in main thread of the main interpreter
(see full error in AthKouloumvakos/PyThea#13 (comment))So how to reproduce this in a minimal example:
I make a fresh environment:
Make a test.py with the:
and run this from the command line with
streamlit run test.py
What the above script does is to load the streamlit app in the browser and when the 'Download with Fido' button is pressed, downloads some fits files using sunpy's FIDO.
This is when the error
RuntimeError: set_wakeup_fd only works in main thread of the main interpreter
comes in.The full error is:
It seems that
loop.add_signal_handler(sig, task.cancel)
somewhere fails. This happens only when I use this inside streamlit.When trying to pass by this problem like this:
the fits files download normally.
My sunpy.system_info() are:
General
#######
OS: Mac OS 13.3.1
Arch: 64bit, (i386)
sunpy: 4.1.5
Installation path: /Users/opt/anaconda3/envs/test/lib/python3.9/site-packages/sunpy-4.1.5.dist-info
Required Dependencies
#####################
astropy: 5.2.2
numpy: 1.24.3
packaging: 23.1
parfive: 2.0.2
Optional Dependencies
#####################
asdf: 2.15.0
asdf-astropy: 0.4.0
beautifulsoup4: 4.12.2
cdflib: 0.4.9
dask: 2023.5.0
drms: 0.6.3
glymur: 0.12.5
h5netcdf: 1.1.0
h5py: 3.8.0
lxml: 4.9.2
matplotlib: 3.7.1
mpl-animators: 1.1.0
pandas: 2.0.1
python-dateutil: 2.8.2
reproject: 0.10.0
scikit-image: 0.20.0
scipy: 1.9.1
sqlalchemy: 2.0.13
tqdm: 4.65.0
zeep: 4.2.1
The text was updated successfully, but these errors were encountered: