A computer programmer enchanted by symbiose between big data and statistics · 9y ·
In the field of Statistical Learning, when you want to fit a model to an observed set of data points (in order to be able to predict values for future data) you have a choice between either:
- simple methods (or inflexible methods): Those methods are opinionated (high bias) you choose your model with no regard to the inherent characteristics of your observed data, for example you say I'll use a linear model to fit the observations.
- complex methods (or flexible methods): Those methods tries to take into account the inherent regularities of the observed data, the term flexible means that they are like chameleon you can push them to the extreme and they will fit your data perfectly (for example by using a high order polynomial model) to the point that they become useless because they will not be able to predict future data (high variance).
The bias-variance tradeoff tells us that no method is better than the other and that you need to choose according to your needs and the data you have. I like to think of this tradeoff as another instance of "No Free Lunch Theorem".
17K views ·
View upvotes
· 1 of 3 answers
Something went wrong. Wait a moment and try again.