Statistical learning refers to a vast set of tools for understanding data. Types of statistical learning:
- Supervised statistical learning involves building a statistical model for predicting, or estimating, an output based on one or more inputs
- Unsupervised statistical learning, there are inputs but no supervising output; nevertheless we can learn relationships and structure from such data
Linear regression is used for predicting quantitative values, such as an individual’s salary
Statistical Learning History:
- Around 1900, Legendre and Gauss published method of least squares papers (a form of linear regression)
- In 1936, Fisher proposed linear discriminant analysis for predicting qualitative regression problems.
- In 1940, various authors proposed logistic regression for solving the regression problem
- In the early 1970s, Nelder and Wedderburn coined the term generalized linear models for an entire class of statistical learning methods that include both linear and logistic regression as special cases.
- In 1970, many linear models were available however, fitting a nonlinear relationship was computationally infeasible at that time.
- In 1980, the computing technology had improved so it can solve nonlinear problems.
- In mid 1980, Breiman, Friedman, Olshen and Stone introduced classification and regression trees, and were among the first to demonstrate the power of a detailed practical implementation of a method, including cross-validation for model selection
- In 1986, Hastie and Tibshirani coined the term generalized additive models for a class of nonlinear extensions to generalized linear models
- In the past years, progress on statistical learning techniques and tools has been marked by the increasing availability of powerful and relatively user-friendly software