Ai+ Training

Lesson 1: How Gradient Boosting works

We will review the concepts starting from the definition of the Decision Tree, through Random Forests to how Gradient Boosting can be seen as gradient descent where each step is a tree.  Along the way, we will highlight the small details that make a big difference when it comes to configuring Gradient Boosting models.

Lesson 2: Review of Gradient Boosting Packages

We will present and use all the major Gradient Boosting implementations, including scikit-learn, XGBoost, CatBoost, LightGBM, and StructureBoost, demonstrating the relative strengths and weaknesses of each one.

Lesson 3: Training in Practice, Setting Parameters, Cross-Validation

We will work hands-on with several data sets and demonstrate best practices for exploring data and experimenting with model parameters.  We will show the value of early stopping, and demonstrate how best to use it in a cross-validated setting.  Along the way, we will write a grid search function from scratch that avoids many of the pitfalls in existing functions and demonstrate how to use it in a "targeted" fashion.

Lesson 4: Evaluating and Understanding the Model

After building a model, we will work through various ways to analyze and understand the model.  We will review all the major metrics used, discuss when they are relevant or irrelevant, and learn how to put them in their proper context.  This includes:

- Looking at ICE-plots to understand the practical impact of individual variables and if/how they interact with others

- Using the SHAP package and how to use those values to give meaningful "reasons" to a specific prediction.

- Examining the calibration of the model, understanding the consequences of poor calibration, and how to fix it.

By the end, you will be able to describe how the model works, which variables have the most impact, and in which scenarios one should be cautious applying the model.