Course Abstract

Duration: 6 hours

1. Intro to Linear Algebra

This topic, Intro to Linear Algebra, is the first in the Machine Learning Foundations series.

It is essential because linear algebra lies at the heart of most machine learning approaches and is especially predominant in deep learning, the branch of ML at the forefront of today’s artificial intelligence advances. Through the measured exposition of theory paired with interactive examples, you’ll develop an understanding of how linear algebra is used to solve for unknown values in high-dimensional spaces, thereby enabling machines to recognize patterns and make predictions.

The content covered in Intro to Linear Algebra is itself foundational for all the other topics in the Machine Learning Foundations series and it is especially relevant to Linear Algebra II.

Over the course of studying this topic, you'll:

  • Understand the fundamentals of linear algebra, a ubiquitous approach for solving for unknowns within high-dimensional spaces.

  • Develop a geometric intuition of what’s going on beneath the hood of machine learning algorithms, including those used for deep learning.

  • Be able to more intimately grasp the details of machine learning papers as well as all of the other subjects that underlie ML, including calculus, statistics, and optimization algorithms.

2. Linear Algebra II: Matrix Operations

This topic, Linear Algebra II: Matrix Operations, builds on the basics of linear algebra. It is essential because these intermediate-level manipulations of tensors lie at the heart of most machine learning approaches and are especially predominant in deep learning.

Through the measured exposition of theory paired with interactive examples, you’ll develop an understanding of how linear algebra is used to solve for unknown values in high-dimensional spaces as well as to reduce the dimensionality of complex spaces. The content covered in this topic is itself foundational for several other topics in the Machine Learning Foundations series, especially Probability & Information Theory and Optimization.

Over the course of studying this topic, you'll:

  • Develop a geometric intuition of what’s going on beneath the hood of machine learning algorithms, including those used for deep learning.
  • Be able to more intimately grasp the details of machine learning papers as well as all of the other subjects that underlie ML, including calculus, statistics, and optimization algorithms.
  • Reduce the dimensionality of complex spaces down to their most informative elements with techniques such as eigendecomposition, singular value decomposition, and principal components analysis.

DIFFICULTY LEVEL: BEGINNER

Instructor Bio:

Dr Jon Krohn

Chief Data Scientist, Author of Deep Learning Illustrated | untapt

Dr. Jon Krohn

Jon Krohn is Chief Data Scientist at the machine learning company untapt. He authored the 2019 book Deep Learning Illustrated, an instant #1 bestseller that was translated into six languages. Jon is renowned for his compelling lectures, which he offers in-person at Columbia University, New York University, and the NYC Data Science Academy. Jon holds a Ph.D. in neuroscience from Oxford and has been publishing on machine learning in leading academic journals since 2010; his papers have been cited over a thousand times.

Course Outline

1. Data Structures for Algebra

  • What Linear Algebra Is
  • A Brief History of Algebra
  • Tensors
  • Scalars
  • Vectors and Vector Transposition
  • Norms and Unit Vectors
  • Basis, Orthogonal, and Orthonormal Vectors
  • Arrays in NumPy
  • Matrices
  • Tensors in TensorFlow and PyTorch


2. Common Tensor Operations

  • Tensor Transposition
  • Basic Tensor Arithmetic
  • Reduction
  • The Dot Product
  • Solving Linear Systems


3. Matrix Properties

  • The Frobenius Norm
  • Matrix Multiplication
  • Symmetric and Identity Matrices
  • Matrix Inversion
  • Diagonal Matrices
  • Orthogonal Matrices


4. Eigendecomposition

  • Eigenvectors
  • Eigenvalues
  • Matrix Determinants
  • Matrix Decomposition
  • Application of Eigendecomposition


5. Matrix Operations for Machine Learning

  • Singular Value Decomposition (SVD)
  • The Moore-Penrose Pseudoinverse
  • The Trace Operator
  • Principal Component Analysis (PCA): A Simple Machine Learning Algorithm
  • Resources for Further Study of Linear Algebra