Add support inference on SYCL devices #9800

razdoburdin · 2023-11-21T12:01:57Z

In continuation of #9691
This PR adds experimental support of prediction with SYCL devices.

The key changes are:

Added the new predictor: sycl_predictor, which is calling if SYCL device parameter is specified.
Currently only the PredictBatch method has a native SYCL implementation.
For building with SYCL support, user should specify the key -DPLUGIN_SYCL=ON for cmake.
C++ unit tests for SYCL were rewrited for better unification with another tests in the repo.
Python tests for SYCL prediction were added.
CI configuration files were renewed, SYCL tests were added.

plugin/sycl/predictor/predictor.cc

napetrov · 2023-11-21T16:34:23Z

tests/ci_build/conda_env/linux_sycl_test.yml

+- conda-forge
+- intel
+dependencies:
+- python=3.8


should be it 3.8 or higher?

Have used the same config, as is already used for cpu

Co-authored-by: Nikolay Petrov <nikolay.a.petrov@intel.com>

plugin/sycl/README.md

Co-authored-by: Nikolay Petrov <nikolay.a.petrov@intel.com>

.github/workflows/main.yml

.github/workflows/python_tests.yml

include/xgboost/context.h

trivialfis · 2023-11-22T09:15:28Z

include/xgboost/context.h

@@ -262,7 +269,6 @@ struct Context : public XGBoostParameter<Context> {
   */
  template <typename CPUFn, typename CUDAFn, typename SYCLFn>
  decltype(auto) DispatchDevice(CPUFn&& cpu_fn, CUDAFn&& cuda_fn, SYCLFn&& sycl_fn) const {
-    static_assert(std::is_same_v<std::invoke_result_t<CPUFn>, std::invoke_result_t<CUDAFn>>);


In case of non-sycl device, the DispatchDevice with this assert will be called. So, it was just a duplicated assert.

tests/python-sycl/test_sycl_prediction.py

plugin/sycl/device_manager.h

trivialfis · 2023-11-22T09:29:04Z

plugin/sycl/device_manager.h

+#include "xgboost/context.h"
+
+namespace xgboost {
+namespace sycl {


Are you sure you want to use sycl namespace in the first PR? In the future, you will have to keep doing ::sycl for the global sycl namespace.

Yes. This option is not perfect, but looks like the best one.

plugin/sycl/README.md

plugin/sycl/device_manager.cc

plugin/sycl/README.md

plugin/sycl/device_manager.h

tests/cpp/plugin/test_sycl_predictor.cc

Co-authored-by: Alexandra <alexandra.epanchinzeva@intel.com>

plugin/sycl/README.md

napetrov · 2023-11-23T11:30:40Z

plugin/sycl/README.md

+```
+
+## Dependencies
+To build the plugin, install [Intel® oneAPI DPC++/C++ Compiler](https://www.intel.com/content/www/us/en/developer/tools/oneapi/dpc-compiler.html).


I think we also should mention that once build - for running plugin you have to resolve DPC compiler runtime + drivers

plugin/sycl/README.md

trivialfis · 2023-11-30T09:05:03Z

Hi, could you please help take a look into the CI errors?

razdoburdin · 2023-11-30T09:06:30Z

Hi, could you please help take a look into the CI errors?

working on it right now. Will fix it asap.

trivialfis · 2023-11-30T09:34:17Z

no worries, we trigger AWS workers for the CI manually. It was just a friendly reminder that the CI has been triggered.

razdoburdin · 2023-11-30T14:03:49Z

no worries, we trigger AWS workers for the CI manually. It was just a friendly reminder that the CI has been triggered.
Fixed it for my local machine. Please try to launch CI again.

Dmitry Razdoburdin added 9 commits November 21, 2023 01:11

adding support of sycl predictor

f6059b0

fix compilation error

7cf7e90

fix compilation errors; linting

923ce39

fix

49ddce9

fix dispatching

0196219

lint CMakeFiles

bf28f57

fix tests

16d30d0

linting

792043c

remove some dead code

46a26d8

napetrov reviewed Nov 21, 2023

View reviewed changes

plugin/sycl/predictor/predictor.cc Outdated Show resolved Hide resolved

napetrov reviewed Nov 21, 2023

View reviewed changes

Update plugin/sycl/predictor/predictor.cc

2b951f1

Co-authored-by: Nikolay Petrov <nikolay.a.petrov@intel.com>

napetrov reviewed Nov 21, 2023

View reviewed changes

plugin/sycl/README.md Outdated Show resolved Hide resolved

Update plugin/sycl/README.md

f0fe696

Co-authored-by: Nikolay Petrov <nikolay.a.petrov@intel.com>

trivialfis reviewed Nov 22, 2023

View reviewed changes

aepanchi reviewed Nov 22, 2023

View reviewed changes

razdoburdin and others added 8 commits November 22, 2023 12:09

Update plugin/sycl/README.md

87dbf2e

Co-authored-by: Alexandra <alexandra.epanchinzeva@intel.com>

Update plugin/sycl/README.md

44ec01e

Co-authored-by: Alexandra <alexandra.epanchinzeva@intel.com>

Update plugin/sycl/README.md

9d5f054

Co-authored-by: Alexandra <alexandra.epanchinzeva@intel.com>

Update plugin/sycl/device_manager.cc

605e220

Co-authored-by: Alexandra <alexandra.epanchinzeva@intel.com>

Update plugin/sycl/device_manager.h

2ee8727

Co-authored-by: Alexandra <alexandra.epanchinzeva@intel.com>

Update tests/cpp/plugin/test_sycl_predictor.cc

7f6e29c

Co-authored-by: Alexandra <alexandra.epanchinzeva@intel.com>

apply suggestions; linting

0e6f09d

Merge branch 'master' into dev/sycl/predictor_support

3f0f0ea

napetrov reviewed Nov 23, 2023

View reviewed changes

plugin/sycl/README.md Show resolved Hide resolved

napetrov reviewed Nov 23, 2023

View reviewed changes

plugin/sycl/README.md Show resolved Hide resolved

Dmitry Razdoburdin added 2 commits November 23, 2023 23:57

update Readme.md

6a8310c

fix misprint

ac5a432

razdoburdin and others added 2 commits November 30, 2023 11:18

fix compilation error for test_gpu_predictor.cu

ae078a1

mark InitOutPredictions virtual

58d5986

trivialfis approved these changes Dec 4, 2023

View reviewed changes

trivialfis merged commit 381f1d3 into dmlc:master Dec 4, 2023
28 checks passed

razdoburdin deleted the dev/sycl/predictor_support branch December 4, 2023 08:57

razdoburdin mentioned this pull request Dec 4, 2023

Sycl implementation for objective functions #9846

Merged

razdoburdin mentioned this pull request Jan 26, 2024

SYCL. Add some basic functional for sycl implementation of partition builder. #10011

Merged

This was referenced Feb 14, 2024

SYCL. Add some basic functional for sycl implementation of GHistIndexMatrix. #10045

Merged

SYCL. Add functional for sycl implementation of RowSetCollection. #10057

Merged

razdoburdin mentioned this pull request Jun 3, 2024

2.1.0 release news. [skip ci] #10378

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add support inference on SYCL devices #9800

Add support inference on SYCL devices #9800

razdoburdin commented Nov 21, 2023 •

edited

Loading

napetrov Nov 21, 2023

razdoburdin Nov 21, 2023

trivialfis Nov 22, 2023

razdoburdin Nov 22, 2023

trivialfis Nov 22, 2023

razdoburdin Nov 22, 2023

napetrov Nov 23, 2023

razdoburdin Nov 24, 2023

trivialfis commented Nov 30, 2023

razdoburdin commented Nov 30, 2023

trivialfis commented Nov 30, 2023

razdoburdin commented Nov 30, 2023

Add support inference on SYCL devices #9800

Add support inference on SYCL devices #9800

Conversation

razdoburdin commented Nov 21, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

trivialfis commented Nov 30, 2023

razdoburdin commented Nov 30, 2023

trivialfis commented Nov 30, 2023

razdoburdin commented Nov 30, 2023

razdoburdin commented Nov 21, 2023 •

edited

Loading