[kernel] Add ICX compiler.

[changqi1]

How to build xFT w/ Intel compiler(ICX/ICPX)

Step 1） Install dpc++ compiler:

# Download dpcpp offline package
sh l_dpcpp-cpp-compiler_p_2024.0.0.49524_offline.sh

Step 2） Install conda-forge env:

mkdir -p ~/miniforge3
wget "https://github.com/conda-forge/miniforge/releases/latest/download/Miniforge3-$(uname)-$(uname -m).sh" -O ~/miniforge3/miniforge.sh
bash ~/miniforge3/miniforge.sh -b -u -p ~/miniforge3
rm -rf ~/miniforge3/miniforge.sh
~/miniforge3/bin/conda init bash
~/miniforge3/bin/conda init zsh

# !!! Creat new terminal
conda --version
conda create --name xft python=3.8
conda activate xft

Step 3) Install oneccl and dependencies:

git clone https://github.com/intel/xFasterTransformer.git
cd xFasterTransformer/3rdparty/
sh prepare_oneccl.sh
pip install -r ../requirements.txt

Step 4) Initialize env:

cd xFasterTransformer/
mkdir build && cd build

conda activate xft
source /opt/intel/oneapi/setvars.sh # Or source ~/intel/oneapi/setvars.sh
source ../3rdparty/oneccl/build/_install/env/setvars.sh

export CC=icx
export CXX=icpx
cmake .. -DWITH_PIPELINE_PARALLEL=ON -DWITH_GPU=ON # -DBUILD_WITH_SHARED_LIBS=ON
make -j

[changqi1]

@a3213105 Please check chatglm2&3 for deleting NormT param.
@pujiang2018 Add WITH_GPU to build w/ ICX

[pujiang2018]

previous code contains 2 template parameters:
"template <typename WeiT, typename NormT>"
So, here the normalization operator will always be RmsNorm?

[changqi1]

Yes, synced w/ Gui Sheng. He answered me it is rms_norm currently in chatglm2/3.
By the way, ICX could compile template template default param.