# The Elements of Statistical Learning

# 1 Introduction

# 2 Overview of Supervised Learning

## 2.1 Introduction

## 2.2 Variable Types and Terminology

## 2.3 Two Simple Approaches to Prediction

### 2.3.1 Linear Models and Least Squares

### 2.3.2 Nearest-Neighbor Methods

### 2.3.3 From Least Squares to Nearest Neighbors

## 2.4 Statistical Decision Theory

## 2.5 Local Methods in High Dimensions

## 2.6 Statistical Models, Supervised Learning and Function Approximation

### 2.6.1 A Statistical Models for the Joint Distribution Pr(X,Y)

### 2.6.2 Supervised Learning

### 2.6.3 Function Approximation

## 2.7 Structured Regression Models

### 2.7.1 Difficulty of the Problem

## 2.8 Classes of Restricted Estimators

### 2.8.1 Roughness Penalty and Beyesian Methods

### 2.8.2 Kernel Methods and Local Regression

### 2.8.3 Basis Functions and Dictionary Methods

## 2.9 Model Selection and Bias-Variance Tradeoff

# 3 Linear Methods of Regression

## 3.1 Introduction

## 3.2 Linear Regression Models and Least Squares

## 3.3 Subset Selection

## 3.4 Shrinkage Methods

## 3.6 Discussion: A Comparison of the Selection

## 3.7 Multiple Outcome Shrinkage and Selection

## 3.9 Computational Considerations

# 4 Linear Methods for Classiflcation

## 4.1 Introduction

## 4.2 Linear Regression of an Indicator Matrix

## 4.3 Linear Discriminant Analysis

## 4.4 Logistic Regression

## 4.5 Separating Hyperplanes