Support of half version of isaac blas function

[listenlink]

Hi @ptillet ,

fp16 of isaac blas functionality are added and under review at intel#5
please feel free to come up with your opinion about the patch:) thank you!

[ptillet]

Very good, thanks a lot.

I've left a couple of comments to improve this patch, but it looks already pretty good! :)

[ptillet]

So it seems like the half version of value_scalar is never used in practice. Trying to use it would probably lead to some undefined behavior because of all the casts and the lack of an emulation layer for printing and arithmetic operations (the operators just return the arguments).

I think it would be better to only add support for half on arrays. It would considerably reduce the size of the pull request and also avoid all this undefined behavior due to the lack of an emulation layer -- that wouldn't be used in Blas anyway.

There is also no tests for clBlasHgemm. This header-only implementation of half-precision float could be used http://half.sourceforge.net/.

I will work on this over the week-end!
Philippe

PS: I have very little time to maintain ISAAC-1.0 as I'm working on a ISAAC-2.0 internally; sorry for being so picky about the maintainability of the added code. There are already a lot of unused feature in ISAAC (I initially wanted it to have a full expression-tree engine but slowly shifted the focus towards BLAS only) so I'd like to keep pull requests as small as possible. Once all your pull requests get through, I will officially release ISAAC-1.1 ; it's time for a new release :)

[listenlink]

Sorry for late response because of the leave for Chinese new year. I am working on refining the code to remove half version on value_scalar and only half on arrays remained, and will push the new PR soon.

We have half-blas-test-suite internally for testing, it also use http://half.sourceforge.net/ for cpu reference.

[ptillet]

I hope you had a great Chinese New Year! I also celebrated with 火锅 and 麻辣香锅, my favorite!

Thanks for the changes :)

[CNugteren]

Just a small remark: CLBlast does support half-precision already and has half-precision tests, perhaps it can be helpful for you.

[ptillet]

@listenlink What intel iGPUs (if any) support half-precision? On Broadwell the performance of hGEMM and hGEMV is quite bad (15% slower than sGEMM and sGEMV). It is particularly surprising for io-bound GEMV. I don't really see what more we could do on the code-generation size to speed things up; I'll just assume it's HW or compiler limitations (i.e., FP16-32 conversions) until I have more details.

@CNugteren That's good to know :) I'll re-tune ISAAC for fp16 on AMD and NVidia hardware to see if the poor performance persists

[listenlink]

@ptillet there may be something strange on the tune python files, I just hack the BLAS2 size in tune.py to be only [(2560,2560)], then run python main.py --float16 --float32 --reduce_2d_cols
the first time shows:

----------------
REDUCE-2D-COLS
FLOAT16
----------------
  ...
  ...
2560, 2560       : [##############################] 100% [36 GB/S] (4,16,4,1,512
2560, 2560       : [##############################] 100% [36 GB/S] (4,16,4,1,512)
----------------
REDUCE-2D-COLS
FLOAT32
----------------
  ...
  ...
2560, 2560       : [#                             ]   4% [23 GB/S] (1,16,16,4,64
2560, 2560       : [##############################] 100% [29 GB/S] (2,8,32,4,32)

The second time when have save folder, re-run the comand shows,

----------------
Devices available:
----------------
[x] - GPU - Intel(R) HD Graphics on Intel(R) OpenCL
----------------
REDUCE-2D-COLS
FLOAT16
----------------
2560, 2560       : [##############################] 100% [15 GB/S] (4,16,4,1,512)
----------------
REDUCE-2D-COLS
FLOAT32
----------------
2560, 2560       : [##############################] 100% [25 GB/S] (2,8,32,4,32)

[gongzg]

@ptillet From BDW, Intel Gen graphics fully support FP16 and ideally, HFLOPS should be double of FLOPS. The IO bandwidth should remain the same. In practice, we see about 1.5x performance with FP16 data format compare to FP32 format for some computation bound kernel such as convolution kernel. For many GEMM case, we should see similar performance gain. We are working on the optimized FP16 GEMM kernel now, I will discuss more about the FP16 GEMM performance after we finish that work.