Neon + Tokio Thread Pool

[crazyfrozenpenguin]

Hi!

I'm getting the following error:

error[E0277]: the trait bound `*mut std::os::raw::c_void: std::marker::Send` is not satisfied in `neon::scope::RootScope<'_>`
   --> src/lib.rs:113:23
    |
113 |     let future = POOL.spawn_fn(move || {
    |                       ^^^^^^^^ within `neon::scope::RootScope<'_>`, the trait `std::marker::Send` is not implemented for `*mut std::os::raw::c_void`
    |
    = note: `*mut std::os::raw::c_void` cannot be sent between threads safely
    = note: required because it appears within the type `neon::internal::vm::Isolate`
    = note: required because it appears within the type `neon::scope::RootScope<'_>`
    = note: required because of the requirements on the impl of `std::marker::Send` for `&mut neon::scope::RootScope<'_>`
    = note: required because it appears within the type `neon::vm::FunctionCall<'_, neon::js::JsObject>`
    = note: required because it appears within the type `[closure@src/lib.rs:113:32: 138:6 c:neon::vm::FunctionCall<'_, neon::js::JsObject>]`

And here's the code:

lazy_static! {
    static ref POOL: CpuPool = CpuPool::new(if num_cpus::get() > 1 {num_cpus::get() - 1} else {1});
}

fn encrypt_and_notify(call: Call) -> JsResult<JsUndefined> {

    let c = call;

    let future = POOL.spawn_fn(move || {
        // Retrieve arg0 - String to encrypt
        let decrypted: String = c.arguments.require(c.scope, 0).unwrap().check::<JsString>().unwrap().value();

        // Retrieve arg1 - RSA Key name
        let key_name: String = c.arguments.require(c.scope, 1).unwrap().check::<JsString>().unwrap().value();

        // Get RSA Key from map
        let map = KEYS.lock().unwrap();
        let rsa_key = map.get(&key_name).unwrap().get_key();

        // Encrypt
        let mut result = vec![0; rsa_key.size()];
        rsa_key.public_encrypt(decrypted.as_bytes(), &mut result, PKCS1_PADDING).unwrap();
        let result_b64 = result.as_slice().to_base64(STANDARD).to_string();

        // Retrieve arg2 - Callback function
        let func: Handle<JsFunction> = c.arguments.require(c.scope, 2).unwrap().check::<JsFunction>().unwrap();

        // Callback function with base 64 string result as argument
        let args: Vec<Handle<JsString>> = vec![JsString::new(c.scope, result_b64.as_ref()).unwrap()];
        func.call(c.scope, JsNull::new(), args).unwrap().check::<JsUndefined>();

        let res: Result<String, ()> = Ok(result_b64);
        res
    });

    Ok(JsUndefined::new())
}

I'm really new to rust and I sure that I'm missing something. No idea what though :(

[jedireza]

Yep, the error is telling the truth. Deep down the Neon representation of the V8 isolate is a c_void (docs here: std::os::raw::c_void). Thus we can't move variables rooted in that isolate off the thread safely. Someone please correct my terminology/understanding if I'm wrong.

Do checkout the wonderful work in #214 though. Once that lands, we'll get proper background work off the main thread.

If you don't need async callbacks, but just want to parallelize the work you're doing that is possible today. The home page of neon-bindings.com shows that you can use Rayon this way.

lines.into_par_iter()             // iterate in parallel
     .map(|line| wc(line, word))  // count words for each
     .sum()                       // calculate the total

The full word counting example is a bit old and I'm not sure if it's compatible with the latest Neon, nonetheless you can find it here: https://github.com/dherman/wc-demo

I hope that helps.

[crazyfrozenpenguin]

Thanks for the answer. I did notice #214 before but since it was not yet available, I decided to try something using Tokio crate.

The async callbacks are required as a conduit to resolve a JavaScript side promise:

function encryptAsync(decrypted, rsaKey) {
  return new Promise((resolve, reject) => {
    addon.encryptAndNotify(decrypted, rsaKey, (result) => resolve(cloneString(result)) );
  })
}

[dherman]

Yes, I'm afraid this is not something you can do; JS values and APIs can only be accessed from the main thread. We are investigating the possibility of more general threading APIs than the task API, so hopefully you'll be able to do more and more things with threads. But it will always be the case that JS operations are only on the main thread.