-
Notifications
You must be signed in to change notification settings - Fork 1.5k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Spawning 16 (or more) [Isolate]s with [sleep] freeze the application #51254
Comments
/cc @aam @mkustermann |
The issue is well known: We only allow a certain number of threads to be active in an isolate group to avoid issues with the memory subsystem (see #51261 for more context) There's a workaround to this, which is basically telling the Dart VM that the isolate isn't actively needed, which can currently be done via custom C code: auto isolate = Dart_CurrentIsolate();
Dart_ExitIsolate();
{
<... run blocking C Code, e.g. sleep() or mpv_wait_event() ...>
}
Dart_EnterIsolate(isolate); Filed #51261 for possibly letting the VM do that automatically based on boolean. |
@alexmercerind Do you have the option to write custom C code for the few places where you know C code can block for longer time? If so, is your use case standalone dart or flutter (or both)? (Flutter is hiding all Dart C API symbols - such as |
Hi @mkustermann! Thanks a lot for the detailed explanation & information. I'm really glad. Sure! Flutter recently added a new project template for FFI plugins, where one can build custom C/C++ shared library targets before build. I think I can include these Dart API headers there & export the required C methods for access inside Dart SDK. Alternatively, I think I can (I believe this will save you people a lot of work as-well since my case seems very specific): I'm thinking to call Infact, libmpv also offers a non-wait API for handling events, but since those event callbacks are invoked from another thread, it caused Dart VM to crash due to #37022,. So, I went with Thanks again! |
@mkustermann Is there a possibility to actually use |
For now I just created a new FFI plugin package which spawns a https://github.com/alexmercerind/media_kit/blob/b5b980e29b2f88f226a58c1c5eda416dcb19b598/media_kit_native_event_loop/src/media_kit_native_event_loop.cc#L43-L46 |
This is a huge implementation overhead to a seemingly simple feature. @mkustermann Will you be willing to accept a PR exposing these functions? |
The 3 needed API functions are quite stable and I don't see a big reason not to expose them. Would be happy to accept a PR - though our ffi testing setup has some particularities, so maybe it's easier if we do it: made cl/292722. Will add relevant reviews to see if they have objections. Will think a bit more about whether we can do it in an automatic way (see #51261) that works nicely. |
@SynSzakala We could easily expose these functions, but before that I want to check with you regarding the timelines. If we land this change today it is going to reach master or dev Flutter channel relatively quickly, but it is going to take more than 6 months to reach a stable Flutter release. So we have a choice - to land something quick and dirty which you can only use on a master channel for a while - or spend this time to design a more complete and future-proof solution. Which Flutter channel do you use? |
@mraleph I'm aware of that, it'll be completely fine for us to migrate to the master channel (or back-port the fix to a fork somehow, if that's possible). It's still less work than adding an additional asynchronous native layer (especially given large API surfaces). |
I agree with @SynSzakala comment, that it is just way easier to write that kind of logic in Dart and TBH I'm into letting more advanced usecases for DL APIs. I currently don't have any issue that this change would resolve, but this change gives FFI users flexibility and it wouldn't already burden people that are currently using stuff like NativePorts. EDIT: I think something like this dart+cpp code wrapping bonjour bonjour_ffi if the change is applied, could be written using mostly isolates and a small native code library, if the native code can yield the isolate when sync waiting for the FDs/sockets, eliminating the need for me to ship libuv (but then there's the overhead of using too many isolates, but then it would be way less painful to bundle that library into, say, a Flutter FFI plugin) (of course, it could be done entirely in Dart if I could wrap the FDs into RawSockets, but this is issue #46196). |
For applications that want to have arbitrary number of isolates call into native code that may be blocking, we expose the API functions that allows those native threads to exit an isolate before running long/blocking code. Without the ability to exit/re-enter isolate, one may experience deadlocks as we have a fixed limit on the number of concurrently executing isolates atm. In the longer term we may find a way to do this automatically with low overhead, see [0]. But since those API functions are quite stable and we already expose e.g. `Dart_{Enter,Exit}Scope`, I don't see a reason not to expose `Dart_{Enter,Exit}Isolate`. [0] Issue #51261 Issue #51254 TEST=ffi{,_2}/dl_api_exit_enter_isolate_test Change-Id: I91c772ca962fddb87919663fea07939a498fa205 Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/292722 Commit-Queue: Martin Kustermann <kustermann@google.com> Reviewed-by: Daco Harkes <dacoharkes@google.com> Reviewed-by: Ryan Macnak <rmacnak@google.com>
This reverts commit a251281. Reason for revert: FFI tests fail to link on Windows, fail to load on product-mode Android Original change's description: > [vm] Expose Dart_{CurrentIsolate,ExitIsolate,EnterIsolate} > > For applications that want to have arbitrary number of isolates call > into native code that may be blocking, we expose the API functions that > allows those native threads to exit an isolate before running > long/blocking code. > > Without the ability to exit/re-enter isolate, one may experience > deadlocks as we have a fixed limit on the number of concurrently > executing isolates atm. > > In the longer term we may find a way to do this automatically > with low overhead, see [0]. But since those API functions are quite > stable and we already expose e.g. `Dart_{Enter,Exit}Scope`, I don't > see a reason not to expose `Dart_{Enter,Exit}Isolate`. > > [0] Issue #51261 > > Issue #51254 > > TEST=ffi{,_2}/dl_api_exit_enter_isolate_test > > Change-Id: I91c772ca962fddb87919663fea07939a498fa205 > Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/292722 > Commit-Queue: Martin Kustermann <kustermann@google.com> > Reviewed-by: Daco Harkes <dacoharkes@google.com> > Reviewed-by: Ryan Macnak <rmacnak@google.com> Change-Id: I05ad5b9ce24754a68693160e470f8eb71a812c75 No-Presubmit: true No-Tree-Checks: true No-Try: true Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/294860 Auto-Submit: Ryan Macnak <rmacnak@google.com> Commit-Queue: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com> Bot-Commit: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
For applications that want to have arbitrary number of isolates call into native code that may be blocking, we expose the API functions that allows those native threads to exit an isolate before running long/blocking code. Without the ability to exit/re-enter isolate, one may experience deadlocks as we have a fixed limit on the number of concurrently executing isolates atm. In the longer term we may find a way to do this automatically with low overhead, see [0]. But since those API functions are quite stable and we already expose e.g. `Dart_{Enter,Exit}Scope`, I don't see a reason not to expose `Dart_{Enter,Exit}Isolate`. Difference to original CL: Do use STL synchronization primitives (as the ones in runtime/bin are not always available in shared libraries) [0] Issue #51261 Issue #51254 TEST=ffi{,_2}/dl_api_exit_enter_isolate_test Change-Id: Id817e8d4edb3db35f029248d62388cbd0682001d Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/294980 Reviewed-by: Daco Harkes <dacoharkes@google.com>
This has landed now on We'll continue the discussion on how to do this better / automatically in #51261 (to avoid the custom C code that exits + enters isolate). |
I also experienced this, is there anything the same? |
@azkadev Generally if the isolate yield back to the event loop there should be no freezing of the application. This is mainly a problem if one e.g. calls out to C code in many isolates and that C code blocks. For this scenario one can use a new C API to exit the current isolate and re-enter when the blocking C call has finished. See f9eb97f (You can use |
Hi! I'm in a situation where I'm using
Isolate
s to "poll" for events. But as soon as there are more than 16 isolates in Dart SDK or 8 in Flutter, whole application freezes. Please refer to code below.Expected Behavior
"Alive!" is printed out to the console every second.
Actual Behavior
No output is ever printed out to the console as if application is frozen.
When using the following code in Dart SDK & Flutter:
Once kIsolateSpawnCount is set to 15 in Dart SDK or 7 in Flutter, "Alive!" is printed every second as expected.
Dart Example
Flutter Example
dart --version
I assume Flutter spawns it's own isolates. I don't know however, I have limited knowledge of this.
I'm working on
package:media_kit
& one of the users recently reported that as soon as they create 8Player
instances, everything freezes up. I could reduce down problem to the code I shared above. Still, you can see these lines which run on anotherIsolate
for every instance ofPlayer
, where I'm callingmpv_wait_event
, passing negative duration means waiting until next event comes up.Now I'm considering to create a separate shared library which will do the "polling" using C++'s
std::thread
, but it will be a pain & hacky approach to ship another shared library separately for each platform (while rest of the code is in 100% Dart).On the other hand, using
Isolate
already present in Dart SDK was handy & worked correctly.Any guidance or explanation will be really helpful.
Thanks!
The text was updated successfully, but these errors were encountered: