Increasing Accuracy of Random Forest Algorithm by Decreasing Variance

This study aims to add a level of randomization to the process of building a tree within a random forest. This extra randomization step is achieved by adopting a sectioning technique of a feature’s set of values to search for the optimal threshold at each tree node split. According to the proposed section-based random forest algorithm (SBRF), on each node split of the decision tree, the following steps are performed: first, sorting the chosen feature’s values, then dividing them into equal sections; next, randomly pick a candidate threshold from each section, evaluate each candidate threshold against a predetermined criterion, and finally, choose the best candidate threshold among them. As a result, SBRF produces models of less variance and not higher bias than the models created by random forest, consequently decreasing the generalization error.