3 Linear Methods for Regression

• 3.1 Introduction
• 3.2 Linear Regression Models and Least Squares
  • 3.2.1 Example: Prostate Cancer
  • 3.2.2 The Gauss-Markov Theorem
  • 3.2.3 Multiple Regression from Simple Univariate Regression
  • 3.2.4 Multiple Outputs
• 3.3 Subset Selection
  • 3.3.1 Best-Subset Selection
  • 3.3.2 Forward- and Backward-Stepwise Selection
  • 3.3.3 Forward-Stagewise Regression
  • 3.3.4 Prostate Cancer Data Example (Continued)
• 3.4 Shrinkage Methods
  • 3.4.1 Ridge Regression
  • 3.4.2 The Lasso
  • 3.4.3 Discussion: Subset Selection, Ridge Regression and the Lasso
  • 3.4.4 Least Angle Regression
• 3.5 Methods Using Derived Input Directions
  • 3.5.1 Principal Components Regression
  • 3.5.2 Partial Least Squares