Support regression task using Angel on Spark GBDT

[Sherry-Xiaoyan-Zhang]

GBDT currently supports only classification. Would you add regression task support later, please? Thanks.

[bluesjjw]

We have regression now.
Please try the FP-GBDT branch.
the document

[Sherry-Xiaoyan-Zhang]

I tried on my regression task by following the instruction provided the document. However, when I load my gbdt model on test dataset and even train dataset, I only got 2 unique prediction value. Could you check the regression code, please? If the code is correct or further info about my task is needed, please let me know. Thanks.

[bluesjjw]

Could you show your command and the results here?
I will check it.

[Sherry-Xiaoyan-Zhang]

Sure.
Train Stage:
./spark-submit
--name “xxx-gdbt"
--master local[$NUM_EXECUTORS]
--conf spark.executor.instances=$NUM_EXECUTORS
--conf spark.executor.memory=3g
--conf spark.executor.cores=1
--conf spark.driver.memory=2g
--conf spark.shuffle.registration.maxAttempts=1
--conf spark.ps.jars=$SONA_ANGEL_JARS
--conf spark.ps.instances=$NUM_EXECUTORS
--conf spark.ps.cores=1
--conf spark.ps.memory=3g
--jars $SONA_ANGEL_JARS
--class com.tencent.angel.spark.ml.tree.gbdt.trainer.GBDTTrainer
$JAR_LIBS/spark-on-angel-mllib-${ANGEL_VERSION}.jar
angel.train.data.path:$INPUT_TRAIN angel.validate.data.path:$INPUT_TEST angel.save.model.path:$MODEL_OUTPUT_PATH
ml.gbdt.task.type:regression ml.gbdt.loss.func:rmse ml.gbdt.eval.metric:rmse ml.learn.rate:0.1 ml.gbdt.split.num:10 ml.gbdt.tree.num:20 ml.gbdt.tree.depth:7 ml.feature.index.range:4000000 ml.gbdt.feature.sample.ratio:1.0

Predict stage:
./spark-submit
--name “xxx-gbdt”
--master local[$NUM_EXECUTORS]
--conf spark.executor.instances=$NUM_EXECUTORS
--conf spark.executor.memory=3g
--conf spark.executor.cores=1
--conf spark.driver.memory=2g
--conf spark.shuffle.registration.maxAttempts=1
--conf spark.ps.jars=$SONA_ANGEL_JARS
--conf spark.ps.instances=$NUM_EXECUTORS
--conf spark.ps.cores=1
--conf spark.ps.memory=3g
--jars $SONA_ANGEL_JARS
--class com.tencent.angel.spark.ml.tree.gbdt.predictor.GBDTPredictor
$JAR_LIBS/spark-on-angel-mllib-${ANGEL_VERSION}.jar
angel.load.model.path:$MODEL_OUTPUT_PATH
angel.predict.data.path:$INPUT_TRAIN angel.predict.out.path:$PRED_OUTPUT_PATH \

Then I check the $PRED_OUTPUT_PATH folder for prediction value.

[bluesjjw]

Sorry for the delay, it's international workers' day in China.

If the dimension of the training dataset is 4 million, using 3G for the executor is probably below the requirement. So I suspect the training task is not successful.
Could you check the log of driver after the training task? I want to make sure the model is successfully saved.

[Sherry-Xiaoyan-Zhang]

hi, sorry for the delay. I used housing data downloaded from https://www.csie.ntu.edu.tw/~cjlin/libsvmtools/datasets/regression.html to test again, which contains 506 records, 13 features.
I got both model and predictions, however, only one unique value in predictions.
Since I won't use this lib at least for now, if you have passed your own test cases and are sure that it works properly. I can close issue for now.
Thanks.

Below are my bash scripts.
Train bash script:
NUM_EXECUTORS=4
ANGEL_HOME=$(pwd)
JAR_LIBS=$ANGEL_HOME/target/universal/talogetljsonparser-1.0/libs
echo $JAR_LIBS
SONA_ANGEL_JARS=""
for entry in "$JAR_LIBS"/*
do
SONA_ANGEL_JARS=$SONA_ANGEL_JARS,$entry
done
SONA_ANGEL_JARS=${SONA_ANGEL_JARS:1}
echo $SONA_ANGEL_JARS

INPUT_TRAIN="$ANGEL_HOME/resources/dataset/housing"
INPUT_TEST="$ANGEL_HOME/resources/dataset/housing"
MODEL_OUTPUT_PATH="$ANGEL_HOME/resources/dataset/housing_gbdt_model"
ANGEL_VERSION="2.2.1"

xxx/bin/spark-submit
--name "gdbt-train"
--master local[$NUM_EXECUTORS]
--conf spark.executor.instances=$NUM_EXECUTORS
--conf spark.executor.memory=5g
--conf spark.executor.cores=1
--conf spark.driver.memory=2g
--conf spark.shuffle.registration.maxAttempts=1
--conf spark.ps.jars=$SONA_ANGEL_JARS
--conf spark.ps.instances=$NUM_EXECUTORS
--conf spark.ps.cores=1
--conf spark.ps.memory=3g
--jars $SONA_ANGEL_JARS
--class com.tencent.angel.spark.ml.tree.gbdt.trainer.GBDTTrainer
$JAR_LIBS/spark-on-angel-mllib-${ANGEL_VERSION}.jar
angel.train.data.path:$INPUT_TRAIN angel.validate.data.path:$INPUT_TEST angel.save.model.path:$MODEL_OUTPUT_PATH
ml.gbdt.task.type:regression ml.gbdt.loss.func:rmse ml.gbdt.eval.metric:rmse ml.learn.rate:0.1 ml.gbdt.split.num:10 ml.gbdt.tree.num:20 ml.gbdt.tree.depth:7 ml.feature.index.range:4000 ml.gbdt.feature.sample.ratio:1.0

Test bash script:
NUM_EXECUTORS=4
ANGEL_HOME=$(pwd)
JAR_LIBS=$ANGEL_HOME/target/universal/talogetljsonparser-1.0/libs
echo $JAR_LIBS
SONA_ANGEL_JARS=""
for entry in "$JAR_LIBS"/*
do
SONA_ANGEL_JARS=$SONA_ANGEL_JARS,$entry
done
SONA_ANGEL_JARS=${SONA_ANGEL_JARS:1}
echo $SONA_ANGEL_JARS

INPUT_TRAIN="$ANGEL_HOME/resources/dataset/housing"
INPUT_TEST="$ANGEL_HOME/resources/dataset/housing"
MODEL_OUTPUT_PATH="$ANGEL_HOME/resources/dataset/housing_gbdt_model"
PRED_OUTPUT_PATH="$ANGEL_HOME/resources/dataset/housing_pred"
ANGEL_VERSION="2.2.1"

xxx/bin/spark-submit
--name "gdbt-test"
--master local[$NUM_EXECUTORS]
--conf spark.executor.instances=$NUM_EXECUTORS
--conf spark.executor.memory=6g
--conf spark.executor.cores=2
--conf spark.driver.memory=2g
--conf spark.shuffle.registration.maxAttempts=1
--conf spark.ps.jars=$SONA_ANGEL_JARS
--conf spark.ps.instances=$NUM_EXECUTORS
--conf spark.ps.cores=1
--conf spark.ps.memory=3g
--jars $SONA_ANGEL_JARS
--class com.tencent.angel.spark.ml.tree.gbdt.predictor.GBDTPredictor
$JAR_LIBS/spark-on-angel-mllib-${ANGEL_VERSION}.jar
angel.load.model.path:$MODEL_OUTPUT_PATH
angel.predict.data.path:$INPUT_TEST angel.predict.out.path:$PRED_OUTPUT_PATH \