Logistic Regression and Decision Tree Predictive Modeling: A Fit Function Comparison

Abstract

Decision Tree is a predictive modeling technique that is used in classification problems. There are two main types of Decision Tree predictive models: classification or regression. The classification model has a categorical outcome variable. The regression model has a continuous outcome variable. We focused on the classification model approach in our comparison because it parallels the logistic regression binary outcome. The Decision Tree and Logistic Regression predictive methods did not have the same classification accuracy because of the “cut value” used for the first node in the Decision Tree. They are both classification methods which differ in their selection of the first “most” important variable in the classification procedure. Decision Tree predictive modeling does not permit the selection of the “first” node because it is based on the highest variable Information Gain while Logistic Regression has the advantage of determining and selecting variable entry from predictor significance. Logistic Regression slightly outperformed the Decision Tree classification method.