0

Read some documentation (for example) I know that there are many types of decision tree (Cart, ID3 and so on). I also know that Random Forest is a particolar algorithm that use a set of decision tree.

My question is: in random forest, what kind of decision tree are used? (cart, id3,..)

Inuraghe
  • 481
  • 4
  • 17

2 Answers2

3

In short : It can be any type of tree inside forest :)

Random forest is an ensemble of many decision trees.The success of a random forest highly depends on using uncorrelated decision trees. If we use same or very similar trees, overall result will not be much different than the result of a single decision tree. Random forests achieve to have uncorrelated decision trees by bootstrapping and feature randomness.

So what type of tree are inside ? This depends on implementation. Generally, any bootstrap-aggregated attribute-bagged learner based on trees (any of them) is called Random Forest. You get different flavors using different trees.

For example : Function randomForest() in R uses CART algorithm

CART, C4.5 or C5.0 , any of these can be used to grow a forest

Vineesh Vijayan
  • 196
  • 1
  • 4
  • Ok so, in R are tree CART by default? I immagine that I can select the model that I want (CART, ID3, ...). Is possible to have a forest with different types of tree? – Inuraghe Mar 21 '22 at 14:40
  • 1
    R is using CART by default, and scikit-learn uses an optimised version of the CART algorithm. Ideally a forest implementation is tied with one type of tree . means scikit decision tree and R randomForest are implemented in CART , it is not possible to select another – Vineesh Vijayan Mar 21 '22 at 14:49
3

I agree with Vineesh's answer, +1, but have some more details that won't fit well as comments.

Historically, random forests were introduced by Breiman (taking inspiration from Dietterich, Ho, and Amit and Geman); Breiman also introduced the CART algorithm (with Friedman, Olshen, and Stone), so it's probable he meant for random forest to be based on CART trees.

However, the method works for Quinlan-family trees (ID3, C4.5, C5.0) or others. As an example, the software package weka has its own implementation of C4.5 called J48, and that is the basis for their own random forest method.

Ben Reiniger
  • 11,770
  • 3
  • 16
  • 56
  • Thank you. Can you confirm that R use only CART tree and I can't use another method? – Inuraghe Mar 21 '22 at 15:00
  • 1
    I haven't used it (I just did an internet search following your comment), but wsrf is apparently an R package implementing a forest of C4.5 trees: https://rdrr.io/cran/wsrf/man/wsrf.html. It takes a more adaptive approach to subsampling the features, so whether it still counts as a "random forest" is perhaps a matter of debate. – Ben Reiniger Mar 21 '22 at 15:35