How to pass multiple hyperparameters to LightGBM after optimization?

Question

I have used another optimization algorithm that returns me best params for Light GBM.

hyper_optimized_clf_classifier = Util.hp_opt(lgb.LGBMClassifier(silent=True, random_state=1), X, y, score, verbose=True,
                   n_estimators =(hp.quniform,50,500,50),
                   learning_rate =(hp.qloguniform, np.log(0.05), np.log(0.4),0.001),
                    min_child_weight =(hp.qloguniform,np.log(3),np.log(200),1),
                    reg_lambda = (hp.qloguniform, np.log(2), np.log(100),1),
                    num_leaves = (hp.qloguniform, np.log(5),np.log(64),1),
                    subsample = (hp.quniform, 0.5, 1, 0.05),
                   colsample_bytree = (hp.quniform, 0.4, 1, 0.05),
                    max_depth = (hp.quniform, 2, 15, 1),
                    subsample_freq = (hp.quniform, 1, 10, 1),
                    max_bin = (hp.qloguniform, np.log(15), np.log(1023),1),
                    max_evals=100)

If I try to get_params, I get a dictionary of best optimized params I want to pass to train again:

hyper_optimized_clf_classifier.get_params()


{'boosting_type': 'gbdt',
 'class_weight': None,
 'colsample_bytree': 0.45,
 'importance_type': 'split',
 'learning_rate': 0.057,
 'max_depth': 14,
 'min_child_samples': 20,
 'min_child_weight': 20.0,
 'min_split_gain': 0.0,
 'n_estimators': 450,
 'n_jobs': -1,
 'num_leaves': 5,
 'objective': None,
 'random_state': 1,
 'reg_alpha': 0.0,
 'reg_lambda': 2.0,
 'silent': True,
 'subsample': 1.0,
 'subsample_for_bin': 200000,
 'subsample_freq': 6}

I tried to pass these params as a list of values to the light gbm for training again:

['gbdt', None, 0.45, 'split', 0.057, 14, 20, 20.0, 0.0, 450, -1, 5, None, 1, 0.0, 2.0, True, 1.0, 200000,6]

    clf = lgb.LGBMClassifier(list(hyper_optimized_clf_classifier.get_params().values()))

but it does not recognize it.

"LightGBMError: Unknown boosting type gbdt,none,0.45,split,0.057,14,20,20.0,0.0,450,-1,5,none,1,0.0,2.0,true,1.0,200000,6"

Asked By: ERJAN

||

Source

Answer 1

Here’s a reproducible example based on the question’s data and @Misha’s comment for future reference.

from lightgbm import LGBMClassifier

lgbm_clf = LGBMClassifier()

lgbm_params = {
 'colsample_bytree': 0.45,
 'learning_rate': 0.057,
 'max_depth': 14,
 'min_child_weight': 20.0,
 'n_estimators': 450,
 'num_leaves': 5,
 'random_state': 1,
 'reg_lambda': 2.0,
 'subsample': 0.99,
 'subsample_freq': 6}

lgbm_clf.set_params(**lgbm_params) 

lgbm_clf

Output (note: params set at default levels were removed, as they would not be displayed here anyway):

LGBMClassifier(colsample_bytree=0.45, learning_rate=0.057, max_depth=14,
               min_child_weight=20.0, n_estimators=450, num_leaves=5,
               random_state=1, reg_lambda=2.0, subsample=0.99,
               subsample_freq=6)

Answered By: mirekphd

How to pass multiple hyperparameters to LightGBM after optimization?

Question:

Answers: