Mathematics for Machine Learning: PCA
This intermediate-level course introduces the mathematical foundations to derive Principal Component Analysis (PCA), a fundamental dimensionality reduction technique. We'll cover some basic statistics of data sets, such as mean values and variances, we'll compute distances and angles between vectors using inner products and derive orthogonal projections of data onto lower-dimensional subspaces.
Monthly Subscription Starting at AED 99 + VAT
- Level Professional
- Duration 21 hours
- Course by Imperial College London
-
Offered by
About
This intermediate-level course introduces the mathematical foundations to derive Principal Component Analysis (PCA), a fundamental dimensionality reduction technique. We'll cover some basic statistics of data sets, such as mean values and variances, we'll compute distances and angles between vectors using inner products and derive orthogonal projections of data onto lower-dimensional subspaces. Using all these tools, we'll then derive PCA as a method that minimizes the average squared reconstruction error between data points and their reconstruction. At the end of this course, you'll be familiar with important mathematical concepts and you can implement PCA all by yourself. If you’re struggling, you'll find a set of jupyter notebooks that will allow you to explore properties of the techniques and walk you through what you need to do to get on track. If you are already an expert, this course may refresh some of your knowledge. The lectures, examples and exercises require: 1. Some ability of abstract thinking 2. Good background in linear algebra (e.g., matrix and vector algebra, linear independence, basis) 3. Basic background in multivariate calculus (e.g., partial derivatives, basic optimization) 4. Basic knowledge in python programming and numpy Disclaimer: This course is substantially more abstract and requires more programming than the other two courses of the specialization. However, this type of abstract thinking, algebraic manipulation and programming is necessary if you want to understand and develop machine learning algorithms.Modules
Welcome to this course
-
1
Videos
-
5
Readings
-
1
Discussion
-
1
Plugin
1
Discussions
- Nice to meet you!
1
Videos
- Introduction to the course
5
Readings
- About Imperial College & the team
- How to be successful in this course
- Grading policy
- Additional readings & helpful references
- Set up Jupyter notebook environment offline
Mean values
-
2
Videos
-
1
Assignment
1
Assignment
- Mean of datasets
2
Videos
- Welcome to module 1
- Mean of a dataset
Variances and covariances
-
2
Videos
-
1
Readings
-
2
Assignment
2
Assignment
- Variance of 1D datasets
- Covariance matrix of a two-dimensional dataset
2
Videos
- Variance of one-dimensional datasets
- Variance of higher-dimensional datasets
1
Readings
- Symmetric, positive definite matrices
Linear transformation of datasets
-
3
Videos
-
2
Lab
-
1
Programming
- Mean/covariance of a dataset + effect of a linear transformation
2
Labs
- NumPy Tutorial
- Mean/covariance of a dataset + effect of a linear transformation
3
Videos
- Effect on the mean
- Effect on the (co)variance
- See you next module!
Dot product
-
2
Videos
-
1
Assignment
1
Assignment
- Dot product
2
Videos
- Welcome to module 2
- Dot product
Inner products
-
6
Videos
-
1
Readings
-
2
Lab
-
1
Programming
-
3
Assignment
- Inner products and angles
3
Assignment
- Properties of inner products
- General inner products: lengths and distances
- Angles between vectors using a non-standard inner product
2
Labs
- Inner products and angles
- Optional: K-nearest Neighbors Algorithm
6
Videos
- Inner product: definition
- Inner product: length of vectors
- Inner product: distances between vectors
- Inner product: angles and orthogonality
- Inner products of functions and random variables (optional)
- Heading for the next module!
1
Readings
- Basis vectors
Projections
-
6
Videos
-
1
Readings
-
1
Lab
-
1
Programming
-
2
Assignment
- Orthogonal projections
2
Assignment
- Projection onto a 1-dimensional subspace
- Project 3D data onto a 2D subspace
1
Labs
- Orthogonal projections
6
Videos
- Welcome to module 3
- Projection onto 1D subspaces
- Example: projection onto 1D subspaces
- Projections onto higher-dimensional subspaces
- Example: projection onto a 2D subspace
- This was module 3!
1
Readings
- Full derivation of the projection
PCA derivation
-
5
Videos
-
4
Readings
-
1
Assignment
1
Assignment
- Chain rule practice
5
Videos
- Welcome to module 4
- Problem setting and PCA objective
- Finding the coordinates of the projected data
- Reformulation of the objective
- Finding the basis vectors that span the principal subspace
4
Readings
- Vector spaces
- Orthogonal complements
- Multivariate chain rule
- Lagrange multipliers
PCA algorithm
-
5
Videos
-
1
Readings
-
1
Plugin
-
2
Lab
-
1
Programming
-
1
Assignment
- PCA
1
Assignment
- Steps of PCA
2
Labs
- Principal Components Analysis (PCA)
- Optional: Demonstrations of PCA
5
Videos
- Steps of PCA
- PCA in high dimensions
- Other interpretations of PCA (optional)
- Summary of this module
- This was the course on PCA
1
Readings
- Did you like the course? Let us know!
Auto Summary
Dive into "Mathematics for Machine Learning: PCA," an intermediate course in Data Science & AI. Led by Coursera, this course focuses on the mathematical foundations of Principal Component Analysis (PCA), covering essential statistics, vector calculations, and orthogonal projections. Spanning 1260 minutes, it requires a solid grasp of linear algebra, multivariate calculus, and Python programming. Ideal for professionals seeking to master PCA, it offers practical Jupyter notebooks for hands-on learning. Available through Starter and Professional subscriptions.
Marc Peter Deisenroth