Decomposition

FastICA

The design in FastICA's callable nonlinear functions G and G' was suboptimal by being separated. Very often the derivative can be computed faster if it's computed at the same time as the function, and this fits FastICA, because they are always computed at the same time.

For small datasets, the calls to g and gprime sometimes overthrow the costs to np.dot and eigh.

For a random X of shape (500, 500), here is the before and after. This is a small but consistent speedup, and for certain functions it could make more of a difference.

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
     1599   30.089    0.019   30.089    0.019 {numpy.core._dotblas.dot}
      200   25.427    0.127   25.788    0.129 decomp.py:196(eigh)
        1    1.678    1.678   59.546   59.546 fastica_.py:94(_ica_par)
      199    1.490    0.007    1.490    0.007 fastica_.py:229(g)
      200    0.284    0.001   41.156    0.206 fastica_.py:40(_sym_decorrelation)

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
     1599   28.578    0.018   28.578    0.018 {numpy.core._dotblas.dot}
      200   27.604    0.138   27.944    0.140 decomp.py:196(eigh)
        1    1.615    1.615   61.189   61.189 fastica_.py:88(_ica_par)
      199    1.569    0.008    1.570    0.008 fastica_.py:219(gprime)
      199    0.995    0.005    0.995    0.005 fastica_.py:215(g)
      200    0.272    0.001   42.587    0.213 fastica_.py:40(_sym_decorrelation)

And the output of %timeit:

import numpy as np
from sklearn.decomposition import fastica
from fastica_ import fastica as new_ica

np.random.seed(0)
X = np.random.randn(200, 200)
for algorithm in ('deflation', 'parallel'):
        for fun in ('logcosh', 'exp', 'cube'):
                print '%s %s:' % (algorithm, fun)
		print 'old: ',
                %timeit -r10 fastica(X, algorithm=algorithm, fun=fun)
		print 'new: ',		
		%timeit -r10 new_ica(X, algorithm=algorithm, fun=fun)


deflation logcosh:
old:  1 loops, best of 10: 1.38 s per loop
new:  1 loops, best of 10: 1.25 s per loop
deflation exp:
old:  1 loops, best of 10: 2.26 s per loop
new:  1 loops, best of 10: 2.13 s per loop
deflation cube:
old:  1 loops, best of 10: 463 ms per loop
new:  1 loops, best of 10: 465 ms per loop
parallel logcosh:
old:  1 loops, best of 10: 23.9 s per loop
new:  1 loops, best of 10: 23.3 s per loop
parallel exp:
old:  1 loops, best of 10: 23.1 s per loop
new:  1 loops, best of 10: 22.8 s per loop
parallel cube:
old:  1 loops, best of 10: 1.22 s per loop
new:  1 loops, best of 10: 1.11 s per loop