wasm-pack libc error

[DevinBayly]

Hi there,

I understand that it's not exactly the aim of this package, but I'm trying to do linear algebra in the browser with a ndarray web assembly module. When I use wasm-pack build workflow to run the factorize examples I get a number of libc related errors that aren't present in the standard system build. What's strange (or not at all) is that the rest of the ndarray crate lends itself nicely to wasm conversion.

If you have a second, could you briefly glance over these errors, and help me determine whether there's any workaround? I'd love to be able to use the developing ndarray-linalg crate to handle the transformation, and factorization needs that folks may have.

If this is way out of the scope of your tool, then feel free to pass on this issue. Thanks!

//lib.rs
extern crate cfg_if;
extern crate wasm_bindgen;

mod utils;

use cfg_if::cfg_if;
use wasm_bindgen::prelude::*;

cfg_if! {
    // When the `wee_alloc` feature is enabled, use `wee_alloc` as the global
    // allocator.
    if #[cfg(feature = "wee_alloc")] {
        extern crate wee_alloc;
        #[global_allocator]
        static ALLOC: wee_alloc::WeeAlloc = wee_alloc::WeeAlloc::INIT;
    }
}

extern crate ndarray;
extern crate ndarray_linalg;

use ndarray::*;
use ndarray_linalg::*;

// Solve `Ax=b`
fn solve() -> Result<(), error::LinalgError> {
    let a: Array2<f64> = random((3, 3));
    let b: Array1<f64> = random(3);
    let _x = a.solve(&b)?;
    println!("in solve {:?}",_x);
    Ok(())
}

// Solve `Ax=b` for many b with fixed A
fn factorize() -> Result<String, error::LinalgError> {
    let a: Array2<f64> = random((3, 3));
    let f = a.factorize_into()?; // LU factorize A (A is consumed)
    let mut ret_string = String::new();
    for _ in 0..10 {
        let b: Array1<f64> = random(3);
        let _x = f.solve_into(b)?; // solve Ax=b using factorized L, U
        ret_string += format!("in factorize {:?}",_x);
    }
    Ok(ret_string)
}

#[wasm_bindgen]
extern {
    fn alert(s: &str);
}

#[wasm_bindgen]
pub fn greet() -> String {
    match factorize() {
        Ok(st) => st,
        Err(e) => format!("broke something")
    }
}

//cargo.toml
[package]
name = "test-linalgwasm"
version = "0.1.0"
authors = [""]

[lib]
crate-type = ["cdylib", "rlib"]

[features]
default = ["console_error_panic_hook"]

[dependencies]
cfg-if = "0.1.2"
ndarray= "0.12"
ndarray-linalg = { version = "0.10", features = ["openblas"] }
wasm-bindgen = "0.2"

# The `console_error_panic_hook` crate provides better debugging of panics by
# logging them with `console.error`. This is great for development, but requires
# all the `std::fmt` and `std::panicking` infrastructure, so isn't great for
# code size when deploying.
console_error_panic_hook = { version = "0.1.1", optional = true }

# `wee_alloc` is a tiny allocator for wasm that is only ~1K in code size
# compared to the default allocator's ~10K. It is slower than the default
# allocator, however.
#
# Unfortunately, `wee_alloc` requires nightly Rust when targeting wasm for now.
wee_alloc = { version = "0.4.2", optional = true }

[dev-dependencies]
wasm-bindgen-test = "0.2"

[profile.release]
# Tell `rustc` to optimize for small code size.
opt-level = "s

build error

  [1/9] Checking `rustc` version...
  [2/9] Checking crate configuration...
  [3/9] Adding WASM target...
  info: component 'rust-std' for target 'wasm32-unknown-unknown' is up to date
  [4/9] Compiling to WASM...
      Updating crates.io index
     Compiling proc-macro2 v0.4.23
     Compiling unicode-xid v0.1.0
     Compiling wasm-bindgen-shared v0.2.28
     Compiling cfg-if v0.1.6
     Compiling num-traits v0.2.6
     Compiling lazy_static v1.2.0
     Compiling openblas-src v0.6.1
     Compiling matrixmultiply v0.1.15
     Compiling num-complex v0.2.1
     Compiling rawpointer v0.1.0
     Compiling ndarray v0.12.0
     Compiling libc v0.2.43
     Compiling rand_core v0.3.0
     Compiling either v1.5.0
     Compiling wasm-bindgen v0.2.28
     Compiling log v0.4.6
     Compiling cblas-sys v0.1.4
     Compiling lapacke-sys v0.1.4
  error[E0432]: unresolved imports `libc::c_char`, `libc::c_double`, `libc::c_float`, `libc::c_int`
    --> ~/.cargo/registry/src/git.luolix.top-1ecc6299db9ec823/cblas-sys-0.1.4/src/lib.rs:13:12
     |
  13 | use libc::{c_char, c_double, c_float, c_int};
     |            ^^^^^^  ^^^^^^^^  ^^^^^^^  ^^^^^ no `c_int` in the root
     |            |       |         |
     |            |       |         no `c_float` in the root
     |            |       no `c_double` in the root
     |            no `c_char` in the root
  
  error[E0412]: cannot find type `c_double` in module `libc`
    --> ~/.cargo/registry/src/git.luolix.top-1ecc6299db9ec823/cblas-sys-0.1.4/src/lib.rs:17:36
     |
  17 | pub type c_double_complex = [libc::c_double; 2];
     |                                    ^^^^^^^^ not found in `libc`
  
  error[E0412]: cannot find type `c_float` in module `libc`
    --> ~/.cargo/registry/src/git.luolix.top-1ecc6299db9ec823/cblas-sys-0.1.4/src/lib.rs:21:35
     |
  21 | pub type c_float_complex = [libc::c_float; 2];
     |                                   ^^^^^^^ not found in `libc`
  
  error: aborting due to 3 previous errors
  
  Some errors occurred: E0412, E0432.
  For more information about an error, try `rustc --explain E0412`.
  error: Could not compile `cblas-sys`.
  warning: build failed, waiting for other jobs to finish...
  error[E0432]: unresolved imports `libc::c_char`, `libc::c_double`, `libc::c_float`, `libc::c_int`
    --> ~/.cargo/registry/src/git.luolix.top-1ecc6299db9ec823/lapacke-sys-0.1.4/src/lib.rs:13:12
     |
  13 | use libc::{c_char, c_double, c_float, c_int};
     |            ^^^^^^  ^^^^^^^^  ^^^^^^^  ^^^^^ no `c_int` in the root
     |            |       |         |
     |            |       |         no `c_float` in the root
     |            |       no `c_double` in the root
     |            no `c_char` in the root
  
  error[E0412]: cannot find type `c_double` in module `libc`
    --> ~/.cargo/registry/src/git.luolix.top-1ecc6299db9ec823/lapacke-sys-0.1.4/src/lib.rs:17:36
     |
  17 | pub type c_double_complex = [libc::c_double; 2];
     |                                    ^^^^^^^^ not found in `libc`
  
  error[E0412]: cannot find type `c_float` in module `libc`
    --> ~/.cargo/registry/src/git.luolix.top-1ecc6299db9ec823/lapacke-sys-0.1.4/src/lib.rs:21:35
     |
  21 | pub type c_float_complex = [libc::c_float; 2];
     |                                   ^^^^^^^ not found in `libc`
  
  error: aborting due to 3 previous errors
  
  Some errors occurred: E0412, E0432.
  For more information about an error, try `rustc --explain E0412`.
  error: Could not compile `lapacke-sys`.
\ warning: build failed, waiting for other jobs to finish...

[jturner314]

sfackler/rust-openssl#1016 has discussion of an an issue very similar to this one. It looks like the libc crate doesn't define c_char/c_double/c_float/c_int for WASM, and the discussion indicates that interoperability between Rust/C/Fortran in WASM isn't mature yet.

The rulinalg crates provides pure-Rust implementations of linear algebra operations, which would probably work nicely with WASM today. It looks like nalgebra also provides pure-Rust linear algebra.

@termoshtt What are your thoughts about adding rulinalg/nalgebra as an alternative backend that can be selected in place of LAPACK/LAPACKE?

[termoshtt]

What are your thoughts about adding rulinalg/nalgebra as an alternative backend that can be selected in place of LAPACK/LAPACKE?

Pure Rust implementation will be useful even if there are some lack of subroutines. AFAIK, rulinalg/nalgebra does not have LAPACK interface. Do you think it is easy to create compatible layer for them?

[jturner314]

I'm not sure. I haven't looked at the rulinalg types, but the nalgebra types seem almost as general as ndarray's for 1-D and 2-D arrays; the only exception appears to be negative strides. Instead of trying to provide a LAPACK interface over nalgebra, I think the simplest approach would be to implement conversions between ndarray <-> nalgebra types and call the relevant methods on the nalgebra types, since the linear algebra methods are implemented directly on nalgebra matrices (e.g. LU decomposition). So, with an nalgebra backend, there wouldn't be any lapack_traits; everything would be handled in the higher level traits. Unfortunately, this approach doesn't fit cleanly into the existing implementation.

[termoshtt]

ndarray <-> nalgebra conversion seems useful, but I think this conversion itself should be in ndarray crate (or separate ndarray-nalgebra crate) rather than this crate. ndarray -> nalgebra -(LU)-> nalgebra -> ndarray like wrapper can be a possible backend with the conversion functionality.

[jturner314]

I think this conversion itself should be in ndarray crate (or separate ndarray-nalgebra crate) rather than this crate.

I agree. I've started investigating how to perform the conversions (dimforge/nalgebra#473).

[DevinBayly]

It's not probably the most efficient conversion code, but I wrote this yesterday to work out an svd

pub fn svd(&self) -> Nd {
        let count = self.array.shape();
        let inside = self.array.view().to_owned().into_raw_vec();
        let nalg_arr = DMatrix::from_iterator(count[0],count[1],inside);
        let nalgebra::SVD {u,v_t:vt,singular_values} = nalg_arr.svd(true,true);
        let svd_vt_data = vt.unwrap().data;
        Nd{
            array:Array::from_shape_vec([1,svd_vt_data.len()],svd_vt_data.to_vec()).unwrap().into_dyn()
        }
    }

[jturner314]

@DevinBayly Be careful with that, because .to_owned() doesn't guarantee that you'll get a particular memory layout (see its docs). I think this will work (but I haven't tested it): DMatrix::from_iterator(self.rows(), self.cols(), self.t().iter().cloned()). (The .t() is necessary to iterate in column-major order as DMatrix::from_iterator expects.)

[DevinBayly]

@jturner314 oh thanks for pointing that out, and supplying a work around.

[DevinBayly]

Just for completeness ill leave this post here

#[macro_use(array)]
extern crate ndarray;
use ndarray::Array;
use nalgebra::{DMatrix,DVector};

fn main() {
    let a = array![[1,2,3],[4,5,6]];
    let b = a.reversed_axes();
    let nalg = DMatrix::from_iterator(b.cols(),b.rows(),b.iter().cloned());
    println!("{} works",nalg);
}

note that cols() should be the first argument, not rows() upon testing to get out the same matrix as in the ndarray case

[DevinBayly]

should I close this thread?