Vector in Eigen Library is indeed much faster

[Kelvin-Ng]

We thought that it is slow before. I think it was because we did not use it correctly. See Lazy Evaluation and Writing Functions Taking Eigen Types as Parameters to understand how to use it correctly.

I have tried with my own program of dense vector version of SVRG (single machine only). After changing from our own Vector to VectorXd in Eigen, the speed has improved from about 20 seconds to 8 seconds.

However, there are some problems about the interface.

First, sparse and dense vector in Eigen do not have the same iterator interface, but I have managed to solve it by some tricks.

Second, I face some problems in arithmetics among sparse and dense vectors. I have not found a solution yet.

For

VectorXd dvec(5);
SparseVector<double> svec(5);

dvec + svec and svec + dvec works fine. However, something like 10 * dvec + 20 * svec does not compile. To make it compile, we must write something ugly like 10 * dvec + (20 * svec).eval(). It means that we must compute 20 * svec into a sparse vector instead of a lazy update expression. So, the benefit of lazy update is lost. Also, (10 * dvec).eval() + 20 * svec does not work.

Despite the difficulty, due to the much higher performance, I think it worth spending some time to see how to make it work.

I don't want to write our own Vector with similar functionality as Eigen as this is a really tedious job.

[Kelvin-Ng]

I have just tried boost::ublas. For dense vector, it is slightly faster than our own Vector. For sparse vector, it is much slower than our own Vector...

[ylchan87]

I think
(10 * dvec).sparseView() + 20 * svec
should compile. Not sure about speed though

[Kelvin-Ng]

Yes, it compiles, but then the whole program is slower than using our own sparse vector. I am not sure about whether it is because of sparseView() or sparse vector itself.

[Kelvin-Ng]

I have modified the source code of Eigen. Now I can do arithematics among dense and sparse vector without using sparseView(). It runs faster than our own Vector.

However, I am not sure if my modification is correct, although my program gives reasonable results with my modified Eigen.

[Kelvin-Ng]

Oh, forgot to update the issue there.

I submitted a pull request to Eigen to solve the problem about arithmetic among dense and sparse vector here, but the original author said that it is not a good fix and made his own fix here. So, the problem will be gone in next stable release I guess. Before that, using this, which is the stable version with the fix, may also work.

[kygx-legend]

I suggest to file a bug in their issues tracker first: http://eigen.tuxfamily.org/bz/. 👍

[Kelvin-Ng]

@kygx-legend It is already fixed at https://bitbucket.org/eigen/eigen/commits/043094f17aae

[kygx-legend]

I see. I mean the next time.

[kygx-legend]

@Kelvin-Ng how's the status?