init_score not working in LGBMClassifier

[8549]

Description

Setting init_score param in LGBMClassifier doesn't work because the internal call to lightgbm.basic.list_to_1d_numpy() fails. But if called manually the function works and returns correctly (see example). Or is there something more to it and i didn't understand how this param works?

Reproducible example

https://colab.research.google.com/drive/1LooDVMlITs-mlSi8hjRyjwOZuMCl73ZR?usp=sharing

Environment info

LightGBM version or commit hash:
2.2.3
Command(s) you used to install LightGBM

pip install lightgbm

[StrikerRUS]

@8549 Thanks for rising this issue!

First of all, please let me note that you are using very old version of LightGBM (2.2.3). The actual one is 3.2.1. Use pip to update it:

pip install lightgbm -U

As for the error you've encountered, please check fit() arguments' description carefully.

...

eval_set : list or None, optional (default=None)
    A list of (X, y) tuple pairs to use as validation sets.

...

eval_init_score : list of arrays or None, optional (default=None)
    Init score of eval data.

...

You should pass lists for eval_* arguments, one item per validation pair of X and y.
Just fix these lines in your code:

...
    eval_set=[(X_val, y_val)],
    eval_init_score=[np.full_like(y_val, init_val, dtype=float)],
...

[StrikerRUS]

Also, please read the following comment and next time include a code in your issue on GitHub.
#4511 (comment)

from sklearn import datasets, model_selection
import lightgbm
import numpy as np

X, y = datasets.make_classification(
    n_samples=30000,
    n_features=20,
    n_informative=6,
    n_redundant=4,
    n_classes=2,
    random_state=42)

X_train, X_test, y_train, y_test = model_selection.train_test_split(X, y, random_state=42)
X_fit, X_val, y_fit, y_val = model_selection.train_test_split(X_train, y_train, random_state=42)

init_val = 0.013392793462664946

model = lightgbm.LGBMClassifier(learning_rate=0.01, silent=False)
model.fit(
    X_fit,
    y_fit,
    init_score=np.full_like(y_fit, init_val, dtype=float),
    eval_set=(X_val, y_val),
    eval_init_score=np.full_like(y_val, init_val, dtype=float).tolist(),
    early_stopping_rounds=20,
    verbose=100
    )

y_pred = model.predict(X_test)

print()
print(f"Test's ROC AUC: {metrics.roc_auc_score(y_test, y_pred):.5f}")
print(f"Test's logloss: {metrics.log_loss(y_test, y_pred):.5f}")

lightgbm.basic.list_to_1d_numpy(np.full_like(y_fit, init_val, dtype=float))
# array([0.01339279, 0.01339279, 0.01339279, ..., 0.01339279, 0.01339279, 0.01339279], dtype=float32)

lightgbm.__version__
# '2.2.3'

Logs:

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-12-505280180a1a> in <module>()
     20     eval_init_score=np.full_like(y_val, init_val, dtype=float).tolist(),
     21     early_stopping_rounds=20,
---> 22     verbose=100
     23     )
     24 

7 frames
/usr/local/lib/python3.7/dist-packages/lightgbm/basic.py in list_to_1d_numpy(data, dtype, name)
     81     else:
     82         raise TypeError("Wrong type({0}) for {1}.\n"
---> 83                         "It should be list, numpy 1-D array or pandas Series".format(type(data).__name__, name))
     84 
     85 

TypeError: Wrong type(float) for init_score.
It should be list, numpy 1-D array or pandas Series

[8549]

Thank you very much for the detailed help. I was able to run my code.

[github-actions]

This issue has been automatically locked since there has not been any recent activity since it was closed. To start a new related discussion, open a new issue at https://github.com/microsoft/LightGBM/issues including a reference to this.