Section 2.3: Linear Algebra, Computed

A linear model's prediction is a dot product plus a bias, a table of listings times a vector of weights is a matrix product, a nearest neighbor is a distance, and fitting a line is a projection. This section covers the linear algebra the next modules run on, computed in NumPy rather than proved: vectors and the dot product, matrices and matrix multiplication, distance and norms, projection and least squares, and the rank that tells you when two columns carry the same information.

Section 2.2's arithmetic paired one value with its partner, element by element. The operations here combine whole rows and columns at once, and they fail in their own quiet ways: * where @ was meant returns a table where a vector of predictions was wanted, the widest-spread features dominate an unscaled distance, and two columns that say the same thing leave a fitted model's weights meaningless while its predictions still look right. Module 3's first models predict with exactly these operations.

Lessons

  1. Vectors and the Dot Product - a listing as a vector, and the dot product as a prediction and as an angle.
  2. Matrices and Matrix Multiplication - every prediction at once with X @ w, and the element-wise product that only looks like it.
  3. Distance and Norms - how far apart two listings are, and why the features whose values spread the widest dominate it.
  4. Projection, Orthogonality and Least Squares - the closest point on a line, and fitting a model's weights with np.linalg.lstsq.
  5. Rank, Linear Independence and Collinear Columns - columns that carry no new information, and what they do to a fit.

By the end you will be able to:

  • Compute a linear model's predictions for one listing and for a whole table, and catch * where @ was meant by the result's shape.
  • Measure distances between listings, find a nearest neighbor, and show how scaling the features changes it.
  • Fit weights by least squares with an intercept column, and check that the residual is orthogonal to every column.
  • Detect collinear columns with np.linalg.matrix_rank and the condition number, and explain why their individual weights cannot be trusted.