Threshold For Predictor Importance
4 views (last 30 days)
Suppose you have a set of predictors X1, X2, X3, X4,.....Xn, and a Response variable Y. If after computing predictor importance for X1--Xn against Y, how does one determine a threshold that is statistical; not just an arbitrary cut off?
Gaurav Garg on 26 Feb 2021
To determine threshold in case of decision trees, Information Gain is calculated for each of the nodes. The node with the highest Information Gain is selected as the decision node, and the same process is repeated for the next level nodes as well.
The way Information Gain is calculated can differ from categorical data to numerical data.
For more information, you can look at the following links -
Please note that Information gain is computed statistically. You can also easily compute Information Gain on your dataset and then compare it with the decision tree returned by the model.