Apple Clang support

[StrikerRUS]

Since we've started to support Apple Clang in #1501, we should clarify installation instructions with the notes about the possibility of compiling certain versions of LightGBM with Apple Clang.

For ordinary version we have the requirement CMake >=3.12 because of FindOpenMP. And what about

• MPI version;
• GPU version;
• HDFS version

?

ping @guolinke @chivee @levyfan @huanzhang12 @ebernhardson @henry0312

[chivee]

I'm not aware of the exact minimum version, but here is some empirical choice:
* openMPI 1.10 for fully Infiniband support
* GPU: nvidia-36* maybe . I think it only depends on users' devices

[StrikerRUS]

@chivee Sorry, probably I was unclear, but I wanted to know about the general possibility of compiling these LightGBM versions with AppleClang and its required version for doing this.

[StrikerRUS]

@guolinke

How about the binary(python wheel) ?
can we replace it with clang version ? and then users don't need to install gcc. However, it may still require to upgrade clang.

Maybe it's overkill, but what about providing several Python wheels with different versions? For instance, lightgbm, lightgbm-gpu (like tensorflow does), lightgbm-gcc (compiled with gcc on macOS and with MinGW on Windows).

[StrikerRUS]

To speak about MPI version, compilation at Travis with Apple Clang 8.1 is successful:
https://travis-ci.org/Microsoft/LightGBM/jobs/407115085#L3681

[StrikerRUS]

Hi @huanzhang12 !

Don't you know what Invalid Work Group Size error means?
https://travis-ci.org/Microsoft/LightGBM/builds/409547152

https://github.com/Microsoft/LightGBM/tree/gpu-apple (base branch is macos)

The error appears even after adding OpenCL headers from Khronos too. https://stackoverflow.com/a/23079478

This file cannot be placed into system directory due to SIP on new macOS, so I made the following:

https://github.com/Microsoft/LightGBM/blob/9bdc0f45966024f136d4f4d881d7993041d82296/.travis/test.sh#L84-L87
https://github.com/Microsoft/LightGBM/blob/9bdc0f45966024f136d4f4d881d7993041d82296/.travis/test.sh#L107

[StrikerRUS]

ping @huanzhang12

[huanzhang12]

Don't you know what Invalid Work Group Size error means?

Apple's OpenCL implementation on CPU only supports a (1,1,1) workgroup size (in other words, 1 serial thread). Most practical OpenCL program uses a few hundreds work items in each work group (in LightGBM, this is 256). This is Apple's implementation issue and cannot be easily fixed.

This is a known issue:
#667 (comment)
clMathLibraries/clFFT#39
clMathLibraries/clFFT#84

In other words, we cannot test the GPU code on macOS in travis (building should be fine).

[StrikerRUS]

Thank you very much, @huanzhang12 ! As usual, detailed answer :-)

Shame on me that I've missed #667.

To speak about the issue title, are there any special requirements for Apple Clang/CMake/Boost version?

In general, I think it's good to write several words in the installation guide about macOS building process of the GPU version. Did I get it right that built-in Apple's OpenCL works fine in case of real GPU device? And does Khronos headers are needed in this case?

[StrikerRUS]

@huanzhang12

[huanzhang12]

If it compiles on macOS, it should be able to run on real GPU devices. I don't think Khronos headers help if the compilation already works without it. However, I have no macOS system to test Apple OpenCL build. More tests should be done to draw a final conclusion.

[StrikerRUS]

@huanzhang12 Thanks! Got it! I agree that no instruction is better than misleading one.

[StrikerRUS]

Default compiler for macOS is now Apple Clang (#1727 (comment)).

All other unresolved issues mentioned here have their own GitHub issues. So, closing this.