Spawning 16 (or more) [Isolate]s with [sleep] freeze the application

[alexmercerind]

Hi! I'm in a situation where I'm using Isolates to "poll" for events. But as soon as there are more than 16 isolates in Dart SDK or 8 in Flutter, whole application freezes. Please refer to code below.

Expected Behavior

"Alive!" is printed out to the console every second.

Actual Behavior

No output is ever printed out to the console as if application is frozen.
When using the following code in Dart SDK & Flutter:

1. Dart SDK: "Alive!" is printed as long as [kIsolateSpawnCount] is less than 16.
2. Flutter: "Alive!" is printed as long as [kIsolateSpawnCount] is less than 8.

Once kIsolateSpawnCount is set to 15 in Dart SDK or 7 in Flutter, "Alive!" is printed every second as expected.

Dart Example

import 'dart:io';
import 'dart:async';
import 'dart:isolate';

/// Expected Behavior
/// -----------------
/// "Alive!" is printed out to the console every second.
///
/// Actual Behavior
/// ---------------
/// No output is ever printed out to the console as if application is frozen.
/// When using the following code in Dart SDK & Flutter:
/// 1. Dart SDK: "Alive!" is printed as long as [kIsolateSpawnCount] is less than 16.
/// 2. Flutter: "Alive!" is printed as long as [kIsolateSpawnCount] is less than 8.
///
/// Once [kIsolateSpawnCount] is set to 15 in Dart SDK or 7 in Flutter, "Alive!" is printed every second as expected.
///

const kIsolateSpawnCount = 16;

void isolate(message) {
  // [sleep]ing one [Isolate] should not affect rest of the [Isolate]s, main [Isolate] or the event loop.
  // Note: [Future.delayed] works as intended.
  sleep(const Duration(days: 1));
}

Future<void> main() async {
  for (int i = 0; i < kIsolateSpawnCount; i++) {
    await Isolate.spawn(isolate, null);
  }
  Timer.periodic(
    const Duration(seconds: 1),
    (e) {
      print('Alive!');
    },
  );
}

Flutter Example

import 'dart:io';
import 'dart:async';
import 'dart:isolate';

import 'package:flutter/material.dart';

/// Expected Behavior
/// -----------------
/// "Alive!" is printed out to the console every second.
///
/// Actual Behavior
/// ---------------
/// No output is ever printed out to the console as if application is frozen.
/// When using the following code in Dart SDK & Flutter:
/// 1. Dart SDK: "Alive!" is printed as long as [kIsolateSpawnCount] is less than 16.
/// 2. Flutter: "Alive!" is printed as long as [kIsolateSpawnCount] is less than 8.
///
/// Once [kIsolateSpawnCount] is set to 15 in Dart SDK or 7 in Flutter, "Alive!" is printed every second as expected.
///

const kIsolateSpawnCount = 8;

void isolate(message) {
  // [sleep]ing one [Isolate] should not affect rest of the [Isolate]s, main [Isolate] or the event loop.
  // Note: [Future.delayed] works as intended.
  sleep(const Duration(days: 1));
}

void main() {
  runApp(const MyApp());
}

class MyApp extends StatefulWidget {
  const MyApp({Key? key}) : super(key: key);

  @override
  State<MyApp> createState() => _MyAppState();
}

class _MyAppState extends State<MyApp> {
  @override
  void initState() {
    super.initState();
    WidgetsBinding.instance.addPostFrameCallback((_) async {
      for (int i = 0; i < kIsolateSpawnCount; i++) {
        await Isolate.spawn(isolate, null);
      }
      Timer.periodic(
        const Duration(seconds: 1),
        (e) {
          debugPrint('Alive!');
        },
      );
    });
  }

  @override
  Widget build(BuildContext context) {
    return const MaterialApp(
      home: HomeScreen(),
    );
  }
}

class HomeScreen extends StatelessWidget {
  const HomeScreen({Key? key}) : super(key: key);

  @override
  Widget build(BuildContext context) {
    return Scaffold(
      appBar: AppBar(
        title: const Text('test'),
      ),
      body: Center(
        child: ElevatedButton(
          onPressed: () {
            Navigator.of(context).push(
              MaterialPageRoute(
                builder: (context) => const HomeScreen(),
              ),
            );
          },
          child: const Text('Go!'),
        ),
      ),
    );
  }
}

dart --version

Dart SDK version: 2.18.6 (stable) (Tue Dec 13 21:15:14 2022 +0000) on "windows_x64"

I assume Flutter spawns it's own isolates. I don't know however, I have limited knowledge of this.

---

I'm working on package:media_kit & one of the users recently reported that as soon as they create 8 Player instances, everything freezes up. I could reduce down problem to the code I shared above. Still, you can see these lines which run on another Isolate for every instance of Player, where I'm calling mpv_wait_event, passing negative duration means waiting until next event comes up.

Now I'm considering to create a separate shared library which will do the "polling" using C++'s std::thread, but it will be a pain & hacky approach to ship another shared library separately for each platform (while rest of the code is in 100% Dart).
On the other hand, using Isolate already present in Dart SDK was handy & worked correctly.

Any guidance or explanation will be really helpful.
Thanks!

[mraleph]

/cc @aam @mkustermann

[mkustermann]

The issue is well known: We only allow a certain number of threads to be active in an isolate group to avoid issues with the memory subsystem (see #51261 for more context)

There's a workaround to this, which is basically telling the Dart VM that the isolate isn't actively needed, which can currently be done via custom C code:

auto isolate = Dart_CurrentIsolate();
Dart_ExitIsolate();
{
  <... run blocking C Code, e.g. sleep() or mpv_wait_event() ...>
}
Dart_EnterIsolate(isolate);

Filed #51261 for possibly letting the VM do that automatically based on boolean.

[mkustermann]

@alexmercerind Do you have the option to write custom C code for the few places where you know C code can block for longer time? If so, is your use case standalone dart or flutter (or both)?

(Flutter is hiding all Dart C API symbols - such as Dart_ExitIsolate() - so we may need to add those two functions to our NativeApi.initializeApiDLData & include/dart_api_dl.h)

[alexmercerind]

Hi @mkustermann!

Thanks a lot for the detailed explanation & information. I'm really glad.

Sure! Flutter recently added a new project template for FFI plugins, where one can build custom C/C++ shared library targets before build. I think I can include these Dart API headers there & export the required C methods for access inside Dart SDK.

Alternatively, I think I can (I believe this will save you people a lot of work as-well since my case seems very specific):

I'm thinking to call mpv_wait_event within C/C++ using std::thread (avoiding to use Isolate) & send the raw address (int) of the mpv_event* using SendPort/ReceivePort mechanism to Dart (where I can interpret it using dart:ffi). This will avoid any block/sleep within Dart. I have used this once before when dealing with #37022.

Infact, libmpv also offers a non-wait API for handling events, but since those event callbacks are invoked from another thread, it caused Dart VM to crash due to #37022,. So, I went with mpv_wait_event + Isolate, but it seems this has it's own limitations too.

Thanks again!

[SynSzakala]

@mkustermann Is there a possibility to actually use Dart_CurrentIsolate etc. in the native code of an Flutter application? Since, as you mentioned, these functions are not exported it seems that the workaround cannot be applied.

[alexmercerind]

For now I just created a new FFI plugin package which spawns a std::thread & forwards data to Dart VM through SendPort. Inside original root package, I just load the native library if found otherwise fallback to Isolate based solution.

https://github.com/alexmercerind/media_kit/blob/b5b980e29b2f88f226a58c1c5eda416dcb19b598/media_kit_native_event_loop/src/media_kit_native_event_loop.cc#L43-L46
https://github.com/alexmercerind/media_kit/blob/main/media_kit/lib/src/libmpv/core/initializer.dart

[SynSzakala]

This is a huge implementation overhead to a seemingly simple feature.

@mkustermann Will you be willing to accept a PR exposing these functions?

[mkustermann]

@mkustermann Is there a possibility to actually use Dart_CurrentIsolate etc. in the native code of an Flutter application? Since, as you mentioned, these functions are not exported it seems that the workaround cannot be applied.
@mkustermann Will you be willing to accept a PR exposing these functions?

The 3 needed API functions are quite stable and I don't see a big reason not to expose them. Would be happy to accept a PR - though our ffi testing setup has some particularities, so maybe it's easier if we do it: made cl/292722. Will add relevant reviews to see if they have objections.

Will think a bit more about whether we can do it in an automatic way (see #51261) that works nicely.

[mraleph]

@SynSzakala We could easily expose these functions, but before that I want to check with you regarding the timelines. If we land this change today it is going to reach master or dev Flutter channel relatively quickly, but it is going to take more than 6 months to reach a stable Flutter release. So we have a choice - to land something quick and dirty which you can only use on a master channel for a while - or spend this time to design a more complete and future-proof solution.

Which Flutter channel do you use?

[SynSzakala]

@mraleph I'm aware of that, it'll be completely fine for us to migrate to the master channel (or back-port the fix to a fork somehow, if that's possible). It's still less work than adding an additional asynchronous native layer (especially given large API surfaces).
Of course, designing and implementing solution as discussed in #51261 would be very desirable, since that completely removes the need for adding native code. But, from what I understand there's still some discussion on how to do that right.
So, from my perspective it would be reasonable to expose the Dart_*Isolate functions anyway, since the solution discussed in #51261 might not cover every use-case (e.g. some kind of conditional blocking in the native code).

[Piero512]

I agree with @SynSzakala comment, that it is just way easier to write that kind of logic in Dart and TBH I'm into letting more advanced usecases for DL APIs. I currently don't have any issue that this change would resolve, but this change gives FFI users flexibility and it wouldn't already burden people that are currently using stuff like NativePorts.

EDIT: I think something like this dart+cpp code wrapping bonjour bonjour_ffi if the change is applied, could be written using mostly isolates and a small native code library, if the native code can yield the isolate when sync waiting for the FDs/sockets, eliminating the need for me to ship libuv (but then there's the overhead of using too many isolates, but then it would be way less painful to bundle that library into, say, a Flutter FFI plugin) (of course, it could be done entirely in Dart if I could wrap the FDs into RawSockets, but this is issue #46196).

[mkustermann]

This has landed now on main branch in the Dart SDK. Once it rolls up to flutter engine and flutter framework main branches, it will be available to use.

We'll continue the discussion on how to do this better / automatically in #51261 (to avoid the custom C code that exits + enters isolate).

[azkadev]

I also experienced this, is there anything the same?
has it been fixed?

[mkustermann]

I also experienced this, is there anything the same?
has it been fixed?

@azkadev Generally if the isolate yield back to the event loop there should be no freezing of the application. This is mainly a problem if one e.g. calls out to C code in many isolates and that C code blocks. For this scenario one can use a new C API to exit the current isolate and re-enter when the blocking C call has finished. See f9eb97f (You can use dart:ffis NativeApi.initializeApiDLData pass it to C code, which can include dart_api_dl.h from the Dart SDK (can make a copy of it) initialize it and then call the 2 new API functions)