Add Precision Recall AUC as an metric for binary classification

[kdlin]

For binary classification problems, Precision Recall AUC (as opposed to ROC AUC) is a good metric for unbalanced data. Currently, I have to use a custom eval function even though I use logloss as the ojective function. As a result, both logloss and PR-AUC are used to stop training. (I use early_stopping_rounds= 25 to stop the iterations.)

I would like to stop the iterations with just PR-AUC as the metric. Using custom eval function slows down the speed of LightGBM too. Additionally, XGBoost has PR-AUC as a metric. (They called it aucpr.)

I propose that PR-AUC to be added as a built-in metric.

My workaround is as follows for the time being:

model=lgb.train(params, lgb_train,
num_boost_round=2000,
valid_sets=[lgb_valid],
feval=f_pr_auc,
early_stopping_rounds=25,
verbose_eval=50)

def pr_auc(y_true, probas_pred):
p, r, _ = precision_recall_curve(y_true, probas_pred)
return auc(r, p)

def f_pr_auc(probas_pred, y_true):
probas_pred=sigmoid(probas_pred)
labels=y_true.get_label()
p, r, _ = precision_recall_curve(labels, probas_pred)
score=auc(r,p)
return "pr_auc", score, True

[btrotta]

Closed in favor of being in #2302. PR is welcome for this feature!

[btrotta]

Reopened since there is an active PR.

[StrikerRUS]

Closed via #3347.