Building a XGBoost model

1. pre-process :

           1.1 Boruta to clean data column-wise
           1.2 TomekLinks to clean data row-wise

2. training (use a random sample to do this step if the data size is too big):

          2.1 create a basic with fixed learning rate and n_estimators:
                                       XGBRFClassifier(objective='binary:logistic',
                                                                     n_estimator=X,
                                                                     learning_rate=0.1,
                                                                     n_jobs=-1)
         
          2.2 grid search for optimal 'max_depth' and 'min_child_weight'.
                          -- use scoring='roc_auc'
                          -- use cv = RepeatedStratifiedKFold(n_splits=3, n_repeats=3)

          2.3 get 'current_best' by using gridsearch.best_estimator, then fit current_best again.

         2.4 then grid search for 'gamma' with current_best, when it's done, get 'current_best' by using gridsearch.best_estimator, then fit current_best again.

         2.5 then grid search for 'subsample' and 'colsample_bytree', when it's done, get 'current_best' by using gridsearch.best_estimator, then fit current_best again.

         2.6 then grid search for 'learning_rate' , when it's done, get 'current_best' by using gridsearch.best_estimator, then fit current_best again.


3. final training: use all data to fit the mdl with all the optimized params.

4. Evaluatiion.
                 
               

Generate an input data array to train the neural network

A simple loop-in-loop program.


%matlab code 
clc; clear all; close all;
A=[0.1, 0.2, 0.3, 0.1, 0.1, 0.2]; % assumed initial value
B=[0.5, 0.1, 0.1, 0, 0.2, 0.1];
C=[0, 0.2, 0.2, 0.4, 0.1, 0.1]; 
final=[];
r=1;
for i= 0:10
    for j=0:10
         for k=1:10
sum_up=(i*A+j*B+k*C);
                total=sum(sum_up);
                result=sum_up/total;
                row=[i, j, k, result];
final(r,:)=row;
                r=r+1;
    end
    end
end