Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[ci] Run Linux OpenCL tests against POCL instead of the AMD App SDK #5282

Merged
merged 4 commits into from
Jun 13, 2022
Merged

[ci] Run Linux OpenCL tests against POCL instead of the AMD App SDK #5282

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
9f9ab6f
Run OpenCL tests against POCL instead of the AMD App SDK
jgiannuzzi Jun 8, 2022
a767164
Update .ci/setup.sh
jgiannuzzi Jun 12, 2022
33efd81
Run Linux gpu source on default Python version
jgiannuzzi Jun 13, 2022
e146c71
[docs] Update GPU Targets Table
jgiannuzzi Jun 13, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
23 changes: 15 additions & 8 deletions .ci/setup.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -71,14 +71,21 @@ else # Linux
sudo apt-get install --no-install-recommends -y \
libboost1.74-dev \
ocl-icd-opencl-dev
cd $BUILD_DIRECTORY # to avoid permission errors
curl -sL -o AMD-APP-SDKInstaller.tar.bz2 https://github.com/microsoft/LightGBM/releases/download/v2.0.12/AMD-APP-SDKInstaller-v3.0.130.136-GA-linux64.tar.bz2
tar -xjf AMD-APP-SDKInstaller.tar.bz2
mkdir -p $OPENCL_VENDOR_PATH
mkdir -p $AMDAPPSDK_PATH
sh AMD-APP-SDK*.sh --tar -xf -C $AMDAPPSDK_PATH
mv $AMDAPPSDK_PATH/lib/x86_64/sdk/* $AMDAPPSDK_PATH/lib/x86_64/
echo libamdocl64.so > $OPENCL_VENDOR_PATH/amdocl64.icd
if [[ $IN_UBUNTU_LATEST_CONTAINER == "true" ]]; then
sudo apt-get install --no-install-recommends -y \
pocl-opencl-icd
else
sudo apt-get install --no-install-recommends -y \
libhwloc-dev \
libtinfo-dev \
ocl-icd-dev \
pkg-config \
zlib1g-dev
git clone --depth 1 --branch v1.8 https://github.com/pocl/pocl.git
cmake -B pocl/build -S pocl -DCMAKE_BUILD_TYPE=release -DCMAKE_C_COMPILER=clang -DCMAKE_CXX_COMPILER=clang++ -DCMAKE_CXX_FLAGS=-stdlib=libc++ -DPOCL_INSTALL_ICD_VENDORDIR=/etc/OpenCL/vendors -DPOCL_DEBUG_MESSAGES=OFF -DSTATIC_LLVM=ON -DINSTALL_OPENCL_HEADERS=OFF -DENABLE_SPIR=OFF -DENABLE_POCLCC=OFF -DENABLE_TESTS=OFF -DENABLE_EXAMPLES=OFF
StrikerRUS marked this conversation as resolved.
Show resolved Hide resolved
cmake --build pocl/build -j 4
jgiannuzzi marked this conversation as resolved.
Show resolved Hide resolved
sudo cmake --install pocl/build
fi
fi
if [[ $TASK == "cuda" || $TASK == "cuda_exp" ]]; then
echo 'debconf debconf/frontend select Noninteractive' | debconf-set-selections
Expand Down
6 changes: 3 additions & 3 deletions .ci/test.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -179,16 +179,16 @@ if [[ $TASK == "gpu" ]]; then
grep -q 'std::string device_type = "gpu"' $BUILD_DIRECTORY/include/LightGBM/config.h || exit -1 # make sure that changes were really done
if [[ $METHOD == "pip" ]]; then
cd $BUILD_DIRECTORY/python-package && python setup.py sdist || exit -1
pip install --user $BUILD_DIRECTORY/python-package/dist/lightgbm-$LGB_VER.tar.gz -v --install-option=--gpu --install-option="--opencl-include-dir=$AMDAPPSDK_PATH/include/" || exit -1
pip install --user $BUILD_DIRECTORY/python-package/dist/lightgbm-$LGB_VER.tar.gz -v --install-option=--gpu || exit -1
pytest $BUILD_DIRECTORY/tests/python_package_test || exit -1
exit 0
elif [[ $METHOD == "wheel" ]]; then
cd $BUILD_DIRECTORY/python-package && python setup.py bdist_wheel --gpu --opencl-include-dir="$AMDAPPSDK_PATH/include/" || exit -1
cd $BUILD_DIRECTORY/python-package && python setup.py bdist_wheel --gpu || exit -1
pip install --user $BUILD_DIRECTORY/python-package/dist/lightgbm-$LGB_VER*.whl -v || exit -1
pytest $BUILD_DIRECTORY/tests || exit -1
exit 0
elif [[ $METHOD == "source" ]]; then
cmake -DUSE_GPU=ON -DOpenCL_INCLUDE_DIR=$AMDAPPSDK_PATH/include/ ..
cmake -DUSE_GPU=ON ..
fi
elif [[ $TASK == "cuda" || $TASK == "cuda_exp" ]]; then
if [[ $TASK == "cuda" ]]; then
Expand Down
14 changes: 1 addition & 13 deletions .vsts-ci.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -55,21 +55,14 @@ jobs:
gpu_source:
TASK: gpu
METHOD: source
# on Ubuntu 14.04, gpu_source build segfaults with newer version of Python
# (and newer version of scipy as a consequence)
PYTHON_VERSION: '3.7'
PYTHON_VERSION: '3.9'
jgiannuzzi marked this conversation as resolved.
Show resolved Hide resolved
swig:
TASK: swig
steps:
- script: |
echo "##vso[task.setvariable variable=BUILD_DIRECTORY]$BUILD_SOURCESDIRECTORY"
echo "##vso[task.setvariable variable=LGB_VER]$(head -n 1 VERSION.txt)"
echo "##vso[task.prependpath]$CONDA/bin"
AMDAPPSDK_PATH=$BUILD_SOURCESDIRECTORY/AMDAPPSDK
echo "##vso[task.setvariable variable=AMDAPPSDK_PATH]$AMDAPPSDK_PATH"
LD_LIBRARY_PATH=$AMDAPPSDK_PATH/lib/x86_64:$LD_LIBRARY_PATH
echo "##vso[task.setvariable variable=LD_LIBRARY_PATH]$LD_LIBRARY_PATH"
echo "##vso[task.setvariable variable=OPENCL_VENDOR_PATH]$AMDAPPSDK_PATH/etc/OpenCL/vendors"
displayName: 'Set variables'
- script: |
echo '$(Build.SourceVersion)' > '$(Build.ArtifactStagingDirectory)/commit.txt'
Expand Down Expand Up @@ -139,11 +132,6 @@ jobs:
CONDA=$HOME/miniforge
echo "##vso[task.setvariable variable=CONDA]$CONDA"
echo "##vso[task.prependpath]$CONDA/bin"
AMDAPPSDK_PATH=$BUILD_SOURCESDIRECTORY/AMDAPPSDK
echo "##vso[task.setvariable variable=AMDAPPSDK_PATH]$AMDAPPSDK_PATH"
LD_LIBRARY_PATH=$AMDAPPSDK_PATH/lib/x86_64:$LD_LIBRARY_PATH
echo "##vso[task.setvariable variable=LD_LIBRARY_PATH]$LD_LIBRARY_PATH"
echo "##vso[task.setvariable variable=OPENCL_VENDOR_PATH]$AMDAPPSDK_PATH/etc/OpenCL/vendors"
displayName: 'Set variables'
# https://github.com/microsoft/azure-pipelines-agent/issues/2043#issuecomment-687983301
- script: |
Expand Down