- Level Foundation
- Duration 22 hours
- Course by University of Washington
-
Offered by
About
Case Study - Predicting Housing Prices In our first case study, predicting house prices, you will create models that predict a continuous value (price) from input features (square footage, number of bedrooms and bathrooms,...). This is just one of the many places where regression can be applied. Other applications range from predicting health outcomes in medicine, stock prices in finance, and power usage in high-performance computing, to analyzing which regulators are important for gene expression. In this course, you will explore regularized linear regression models for the task of prediction and feature selection. You will be able to handle very large sets of features and select between models of various complexity. You will also analyze the impact of aspects of your data -- such as outliers -- on your selected models and predictions. To fit these models, you will implement optimization algorithms that scale to large datasets. Learning Outcomes: By the end of this course, you will be able to: -Describe the input and output of a regression model. -Compare and contrast bias and variance when modeling data. -Estimate model parameters using optimization algorithms. -Tune parameters with cross validation. -Analyze the performance of the model. -Describe the notion of sparsity and how LASSO leads to sparse solutions. -Deploy methods to select between models. -Exploit the model to form predictions. -Build a regression model to predict prices using a housing dataset. -Implement these techniques in Python.Modules
What is this course about?
5
Videos
- Welcome!
- What is the course about?
- Outlining the first half of the course
- Outlining the second half of the course
- Assumed background
4
Readings
- Important Update regarding the Machine Learning Specialization
- Slides presented in this module
- Reading: Software tools you'll need
- Get help and meet other learners. Join your Community!
Regression fundamentals
4
Videos
- A case study in predicting house prices
- Regression fundamentals: data & model
- Regression fundamentals: the task
- Regression ML block diagram
1
Readings
- Slides presented in this module
The simple linear regression model, its use, and interpretation
4
Videos
- The simple linear regression model
- The cost of using a given line
- Using the fitted line
- Interpreting the fitted line
An aside on optimization: one dimensional objectives
6
Videos
- Defining our least squares optimization objective
- Finding maxima or minima analytically
- Maximizing a 1d function: a worked example
- Finding the max via hill climbing
- Finding the min via hill descent
- Choosing stepsize and convergence criteria
An aside on optimization: multidimensional objectives
2
Videos
- Gradients: derivatives in multiple dimensions
- Gradient descent: multidimensional hill descent
Finding the least squares line
4
Videos
- Computing the gradient of RSS
- Approach 1: closed-form solution
- Approach 2: gradient descent
- Comparing the approaches
2
Readings
- Optional reading: worked-out example for closed-form solution
- Optional reading: worked-out example for gradient descent
Discussion and summary of simple linear regression
1
Assignment
- Simple Linear Regression
5
Videos
- Influence of high leverage points: exploring the data
- Influence of high leverage points: removing Center City
- Influence of high leverage points: removing high-end towns
- Asymmetric cost functions
- A brief recap
1
Readings
- Download notebooks to follow along
Programming assignment
1
Assignment
- Fitting a simple linear regression model on housing data
1
Readings
- Fitting a simple linear regression model on housing data
Multiple features of one input
5
Videos
- Multiple regression intro
- Polynomial regression
- Modeling seasonality
- Where we see seasonality
- Regression with general features of 1 input
1
Readings
- Slides presented in this module
Incorporating multiple inputs
4
Videos
- Motivating the use of multiple inputs
- Defining notation
- Regression with features of multiple inputs
- Interpreting the multiple regression fit
Setting the stage for computing the least squares fit
3
Videos
- Rewriting the single observation model in vector notation
- Rewriting the model for all observations in matrix notation
- Computing the cost of a D-dimensional curve
1
Readings
- Optional reading: review of matrix algebra
Computing the least squares D-dimensional curve
6
Videos
- Computing the gradient of RSS
- Approach 1: closed-form solution
- Discussing the closed-form solution
- Approach 2: gradient descent
- Feature-by-feature update
- Algorithmic summary of gradient descent approach
Summarizing multiple regression
1
Assignment
- Multiple Regression
1
Videos
- A brief recap
Programming assignment 1
1
Assignment
- Exploring different multiple regression models for house price prediction
1
Readings
- Exploring different multiple regression models for house price prediction
Programming assignment 2
1
Assignment
- Implementing gradient descent for multiple regression
2
Readings
- Numpy tutorial
- Implementing gradient descent for multiple regression
Defining how we assess performance
2
Videos
- Assessing performance intro
- What do we mean by "loss"?
1
Readings
- Slides presented in this module
3 measures of loss and their trends with model complexity
5
Videos
- Training error: assessing loss on the training set
- Generalization error: what we really want
- Test error: what we can actually compute
- Defining overfitting
- Training/test split
3 sources of error and the bias-variance tradeoff
3
Videos
- Irreducible error and bias
- Variance and the bias-variance tradeoff
- Error vs. amount of data
OPTIONAL ADVANCED MATERIAL: Formally defining and deriving the 3 sources of error
2
Videos
- Formally defining the 3 sources of error
- Formally deriving why 3 sources of error
Putting the pieces together
1
Assignment
- Assessing Performance
2
Videos
- Training/validation/test split for model selection, fitting, and assessment
- A brief recap
Programming assignment
1
Assignment
- Exploring the bias-variance tradeoff
1
Readings
- Polynomial Regression
Characteristics of overfit models
3
Videos
- Symptoms of overfitting in polynomial regression
- Overfitting demo
- Overfitting for more general multiple regression models
2
Readings
- Slides presented in this module
- Download the notebook and follow along
The ridge objective
5
Videos
- Balancing fit and magnitude of coefficients
- The resulting ridge objective and its extreme solutions
- How ridge regression balances bias and variance
- Ridge regression demo
- The ridge coefficient path
1
Readings
- Download the notebook and follow along
Optimizing the ridge objective
4
Videos
- Computing the gradient of the ridge objective
- Approach 1: closed-form solution
- Discussing the closed-form solution
- Approach 2: gradient descent
Tying up the loose ends
1
Assignment
- Ridge Regression
4
Videos
- Selecting tuning parameters via cross validation
- K-fold cross validation
- How to handle the intercept
- A brief recap
Programming Assignment 1
1
Assignment
- Observing effects of L2 penalty in polynomial regression
1
Readings
- Observing effects of L2 penalty in polynomial regression
Programming Assignment 2
1
Assignment
- Implementing ridge regression via gradient descent
1
Readings
- Implementing ridge regression via gradient descent
Feature selection via explicit model enumeration
5
Videos
- The feature selection task
- All subsets
- Complexity of all subsets
- Greedy algorithms
- Complexity of the greedy forward stepwise algorithm
1
Readings
- Slides presented in this module
Feature selection implicitly via regularized regression
3
Videos
- Can we use regularization for feature selection?
- Thresholding ridge coefficients?
- The lasso objective and its coefficient path
Geometric intuition for sparsity of lasso solutions
4
Videos
- Visualizing the ridge cost
- Visualizing the ridge solution
- Visualizing the lasso cost and solution
- Lasso demo
1
Readings
- Download the notebook and follow along
Setting the stage for solving the lasso
4
Videos
- What makes the lasso objective different
- Coordinate descent
- Normalizing features
- Coordinate descent for least squares regression (normalized features)
Optimizing the lasso objective
3
Videos
- Coordinate descent for lasso (normalized features)
- Assessing convergence and other lasso solvers
- Coordinate descent for lasso (unnormalized features)
OPTIONAL ADVANCED MATERIAL: Deriving the lasso coordinate descent update
1
Videos
- Deriving the lasso coordinate descent update
Tying up loose ends
1
Assignment
- Feature Selection and Lasso
2
Videos
- Choosing the penalty strength and other practical issues with lasso
- A brief recap
Programming Assignment 1
1
Assignment
- Using LASSO to select features
1
Readings
- Using LASSO to select features
Programming Assignment 2
1
Assignment
- Implementing LASSO using coordinate descent
1
Readings
- Implementing LASSO using coordinate descent
Motivating local fits
1
Videos
- Limitations of parametric regression
1
Readings
- Slides presented in this module
Nearest neighbor regression
3
Videos
- 1-Nearest neighbor regression approach
- Distance metrics
- 1-Nearest neighbor algorithm
k-Nearest neighbors and weighted k-nearest neighbors
3
Videos
- k-Nearest neighbors regression
- k-Nearest neighbors in practice
- Weighted k-nearest neighbors
Kernel regression
2
Videos
- From weighted k-NN to kernel regression
- Global fits of parametric models vs. local fits of kernel regression
k-NN and kernel regression wrapup
1
Assignment
- Nearest Neighbors & Kernel Regression
4
Videos
- Performance of NN as amount of data grows
- Issues with high-dimensions, data scarcity, and computational complexity
- k-NN for classification
- A brief recap
Programming Assignment
1
Assignment
- Predicting house prices using k-nearest neighbors regression
1
Readings
- Predicting house prices using k-nearest neighbors regression
What we've learned
3
Videos
- Simple and multiple regression
- Assessing performance and ridge regression
- Feature selection, lasso, and nearest neighbor regression
1
Readings
- Slides presented in this module
Summary and what's ahead in the specialization
2
Videos
- What we covered and what we didn't cover
- Thank you!
Auto Summary
Dive into the world of Data Science & AI with "Machine Learning: Regression," led by Coursera. Focused on predicting housing prices, learners will master regularized linear regression, feature selection, and model optimization, using Python. This foundational course spans 1320 minutes and offers practical skills in handling large datasets, tuning parameters, and deploying predictive models. Ideal for those seeking to understand regression's applications across various industries, with a subscription option available.

Emily Fox

Carlos Guestrin