Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Slow sync func #85

Open
lk2322 opened this issue Jan 4, 2022 · 6 comments
Open

Slow sync func #85

lk2322 opened this issue Jan 4, 2022 · 6 comments
Assignees

Comments

@lk2322
Copy link
Contributor

lk2322 commented Jan 4, 2022

This func block the entire thread for a couple of seconds. Maybe run it in loop.run_in_executor?

def search_youtube(self, title):

@lk2322
Copy link
Contributor Author

lk2322 commented Jan 4, 2022

We can use decorator like this

from functools import wraps, partial
import asyncio

def run_async(func):
    @wraps(func)
    async def run(*args, loop=None, executor=None, **kwargs):
        if loop is None:
            loop = asyncio.get_event_loop()
        pfunc = partial(func, *args, **kwargs)
        return await loop.run_in_executor(executor, pfunc)

    return run

@Raptor123471
Copy link
Owner

Can you provide an example? I have tried using a similar method but it does not work.

@lk2322
Copy link
Contributor Author

lk2322 commented Feb 2, 2022

If I didn't forget something, I just used this decorator and added await before search_youtube

@malikmajai
Copy link

Did we found a solution to this issue that we can apply?

@malikmajai
Copy link

If I didn't forget something, I just used this decorator and added await before search_youtube

?

@RafaelSolVargas
Copy link
Contributor

If I didn't forget something, I just used this decorator and added await before search_youtube

Well, i tried to do that, the main problem in that it's not that simple. To be more specific, the preload function that uses asyncio to run multiple tasks at once doesn't support running coroutine or awaitable functions, and as you can see, the search_youtube funct is within the func down, forcing the def down to be a coroutine (so that he can use await search_youtube).

async def preload(self, song):
if song.info.title != None:
return
def down(song):
if song.host == linkutils.Sites.Spotify:
song.info.webpage_url = self.search_youtube(song.info.title)

This information was collected in this stackoverflow: https://stackoverflow.com/questions/46074841/why-coroutines-cannot-be-used-with-run-in-executor
I tried creating two search_youtube functions, one sync and one async, but other bugs were showing up and crashing the bot, so I gave up.

The solution to execute multiple coroutine functions at once is also in the stackoverflow, it's the second answer, but it's not simple and will require some major changes in the logic that interact with YoutubeDL.

https://github.com/RafaelSolVargas/Vulkan/blob/2dbc6c3984b2024ee14502b8b8cd8fd06c967f75/Music/Downloader.py#L108-L124
This is how another discord music bot deal with that exactly problem, the key here is to have an organized way to work with YoutubeDL, possibly using a class

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

4 participants