Could we add a booster similar to gblinear used by xgboost?

[flippercy]

Hi:

Is it possible to add a linear booster similar to gblinear used by xgboost, please? Combined with monotone_constraint, it will be a very valuable alternative for building linear models.

Let me know if you need any specific user case to justify this request.

Appreciate your help!

@jameslamb

[Fish-Soup]

Hi
LightGBM already supports a parameter, linear_tree=True which fits a linear relationships inside the leaves of the tree. However the minimum value for n_trees is 2. If this could be lowered to 1 then i would imagine this would solve your feature request? with I imagine far less work.

[flippercy]

Hi @Fish-Soup:

Thank you for the reply but I am afraid not. On the one hand, a single boosting tree with linear function as leaves is different from a full boosting model with a linear booster; on the other hand, currently monotone_constraint does not work at the level of leaves when linear_tree=True.

[Denisevi4]

what if you set num_leafs to 1?

[flippercy]

@Denisevi4 as far as I know num_leaves has to be greater than 1.

[guolinke]

hi @shiyu1994 @btrotta , I think num_leave=1 could be achieved, can you take a look?

[flippercy]

Thank you very much for revisiting this request! @guolinke, @shiyu1994, @btrotta and @jameslamb

[shiyu1994]

I think currently, a tree with num_leave=1 will simply terminate the training process of GBDT. See

LightGBM/src/boosting/gbdt.cpp

Lines 406 to 407 in 01568cf

	if (new_tree->num_leaves() > 1) {
	should_continue = true;

In addition, current linear_tree cannot achieve this, even we can train trees with single leaves. Because the linear trees will only fit linear models on the split features in the path from the root node. With a single leaf node, there's no such split feature. So the root will still contain only a constant value.

But I believe a small modification of GBDT and LinearTreeLearner can achieve the goal of gblinear.
Perhaps we should put this to the feature request and voting hub for now.

[StrikerRUS]

Closed in favor of being in #2302. We decided to keep all feature requests in one place.

Welcome to contribute this feature! Please re-open this issue (or post a comment if you are not a topic starter) if you are actively working on implementing this feature.

[flippercy]

In addition, if such a feature can be added in the future, it would be great to make it compatible with monotone_constraints so the sign of the coefficient of each variable in the linear model can be specified.

Thank you.