refit in Python does not support weights

[jtilly]

Summary

The refit method in Python does not support weights (or in fact anything other than data and labels). This is because here, the training data set gets created using:

train_set = Dataset(data, label, silent=True)

It would be great if refit accepted additional arguments (or kwargs) specifically for the Dataset call.

Passing in the train_set directly would also be an option. But since we need to predict first, we would also need to pass the training data in as data frame, which is not so nice.

Motivation

I'm using re-fit in an application where weights are very important.

I'm happy to open a PR if this sounds useful.

[StrikerRUS]

Please refer to #1629 (comment).

Actually, only the X and label are used in refit task, other fields like weight, group are not used.

[jtilly]

Thanks for pointing me to the comment, but weight does get used in refit (also init_score, etc).

Example: tree with three observations that makes one split. Using the weights, I can determine the value of the leaf for the bigger group.

import numpy as np
import lightgbm

X = np.array([1, 2, 2]).reshape((3, 1))
label = np.array([1, 2, 3])

data = lightgbm.basic.Dataset(X, label)

booster = lightgbm.engine.train(
    {
        "min_data_in_bin": 1,
        "min_data_in_leaf": 1,
        "learning_rate": 1,
        "boost_from_average": False,
    },
    data,
    num_boost_round=2,
)

booster.predict(X)
# array([1. , 2.5, 2.5])

# let's refit (to make sure it works)
booster_refit = booster.refit(X, label, decay_rate=0.0)
booster_refit.predict(X)
# array([1. , 2.5, 2.5])

# use weights (I added data_set_kwargs)
booster_refit = booster.refit(
    X, label, decay_rate=0.0, data_set_kwargs={"weight": np.array([1.0, 0.0, 1.0])}
)
booster_refit.predict(X)
# array([1., 3., 3.])

booster_refit = booster.refit(
    X, label, decay_rate=0.0, data_set_kwargs={"weight": np.array([1.0, 1.0, 0.0])}
)
booster_refit.predict(X)
# array([1., 2., 2.])

[guolinke]

Closed in favor of being in #2302. We decided to keep all feature requests in one place.

Welcome to contribute to this feature! Please re-open this issue (or post a comment if you are not a topic starter) if you are actively working on implementing this feature.

[TremaMiguel]

I'll take this to work, here is the plan I'll follow, any changes to it are welcome.

• Context:
  refit method accepts kwargs argument passed to predict method. While Dataset accepts weight parameter, among others, refit method can be changed to support additional arguments to pass to Dataset.

• Solution:
  refit method accept a kwargs_for_dataset parameter to pass weight parameter to Dataset initialization here.
  refit method accepts a kwargs_for_predict parameter to pass original params to predict method here.

• Tests:
  add new test refit case to test the new implementation as this one test_refit

• Documentation:
  document parameter changes here

[github-actions]

This issue has been automatically locked since there has not been any recent activity since it was closed.
To start a new related discussion, open a new issue at https://github.com/microsoft/LightGBM/issues
including a reference to this.